Computationalism and consciousness with Nic Rouleau, Jaan Aru, and Borjan Milinkovic.

This transcript is automatically generated; we strive for accuracy, but errors in wording or speaker identification may occur. Please verify key details when needed.

Main Episode

[00:00] Jaan Aru: Say a very brief intro, Poky, and then you can take a moment to think what you want to add. So the key question that has bothered us and that brought me and Poky together is the following: on the one hand, we have people who are sure that AI is conscious, and I personally don't think there's too much scientific evidence for that. On the other hand, we have people who say, "No, only biology can be conscious." And we think, and perhaps you guys also think, that the true answer is somewhere in the middle. Biology as it is today is way more complex than anything that large language models can instantiate, but it's still some types of computation that we can and should scientifically understand. So our paper and the presentation was kind of inspired by this idea that we should go forward, try to pinpoint the specific computations that underlie consciousness, and unfortunately not too many people are actually interested in that. People are screaming here, "AI is conscious," or screaming, "They are never conscious." But there is this important scientific question that we want to tackle, and I guess you guys are also interested in.

[01:24] Borjan Milinkovic: I think generally there isn't much more to add there. I do agree with Jan that this was our main call where there was not any focus on particularly some middle ground between the two and a broader definition of why particular artificial systems right now can or cannot have some level of sentience or consciousness. And on the other hand, with biology and neurobiology, there was not enough consideration or argument as to why biological systems might house or be the unique vessels of some conscious or sentient experience. Another thing that I also believe was I'm dissatisfied with our own paper about is distinguishing between intelligence and sentience or consciousness, which I would've liked to have been able to go into slightly deeper, but, of course, word limits are a problem.

[02:41] Jaan Aru: It will be always the next paper, Poky.

[02:44] Borjan Milinkovic: Yeah. There'll be a next paper.

[02:45] Jaan Aru: Next paper. You're dealing with it next paper.

[02:48] Borjan Milinkovic: But also one other thing to add is for me, I have a slightly optimistic perspective on computation coming from a neuroscience perspective. A lot of neuroscientists who are maybe in the field of trying to understand artificial systems want to just potentially claim that what the brain is doing is not computation. While for me, I want to claim that maybe we need to democratize the term computation away from just Turing computation. Turing holds this monolithic ownership over computation or the Church-Turing thesis, and for me, I'd rather democratize the term computation because I like it, and I think it can be broadened to include biophysical systems. So that's kind of the underlying drive for me, is to develop a different formalism to computation that is constrained by what physics allows. And I'm not the first maybe person to think of this, and there's a lot of things, but from our perspective, we kind of have this notion of multiscalarity and truly parallel computations happening across scales, the way they might be integrated in a way and things like that. So yeah. And really the focus was on sentience or subjective felt experience rather than intelligence. And yeah, I still think that there is more to flesh out there even for us, and yeah.

[04:38] Jaan Aru: Interesting.

[04:39] Michael Levin: Super. Thanks. That's a great introduction. I have a bunch of things to ask about. Nick, do you want to go first or shall I?

[04:46] Nicolas Rouleau: Sure. It's fascinating. I'd love to know more about this. So my first question would be, I guess, foundational, which is do you think, first of all, do you think that consciousness is substrate dependent? And whether or not you do, do you think that other cognitive functions like learning, problem-solving, decision-making, these easier things, do you think those are also substrate dependent or not?

[05:20] Borjan Milinkovic: I can try and answer from my perspective. I think consciousness might depend on being able to-- So I'm trying to see how this fits into the same terminology. But yes, to an extent it needs to be substrate dependent, but not in the essence that it completely is. And that was what I or what we really attempted to do in the paper, is try and pick out features of substrate dependence that might need to be included, and this is kind of the phrase I mentioned at the end. It's like the computation that physics allows or that is constrained by the physics. So I do think that there is some part to play, some causal part to play, in the morphology of the system and the kind of features of activity that it might allow. And I do think that these computations need to be related to the physical material system in some sense. Absolutely. But whether it is one hundred percent substrate dependent on biology, I would off the cuff claim no as a-

[06:50] Jaan Aru: Yeah. I mean, that's the same thing for us, that there are these people who think that consciousness and other things are completely abstract, completely kind of substrate free, and there are others who say, "Oh, no, it has to be biology." But again, I think that there's the middle ground, which is something that Pokki also tried to articulate, that some computations are more likely to happen in certain substrates because the substrate kind of constrains which kinds of computations can happen and, for example, perhaps in large language models, certain computations can't happen that are more likely to happen in biology. So we are not saying that it only can happen in brains, and in the paper, and I guess in the presentation also, we say that probably there is a way to neuromorphic computation. But we are at the same time also saying that substrate matters in the sense that perhaps you cannot get the right types of computations in any substrate so that there is also a continuum there, right? I don't know what you think about it.

[08:07] Borjan Milinkovic: Yeah.

[08:07] Nicolas Rouleau: Well, I was in recent months, there's been, there have been some papers that have come out, sort of echoing things that people have been saying for almost a century now, which is that brains have many properties that are like analog computers. And during your presentation and indeed during your summary, I can't stop thinking about whether your interpretation of this continuum hinges on the architecture of the computational system. If LLMs were implemented not by digital computers, but rather by analog computers, or if we had substrates that were arranged in different circuits that were more akin to analog computing, would that change the calculus for you? Is an analog computer more likely to be conscious than a digital computer? And how does that map to various materials? What do you think about this sort of thing?

[09:12] Borjan Milinkovic: That's great. So I wanted to answer your—whether it's related to whether substrate dependence is necessary for decision-making and all of that. That I'm not sure of. And then, going now to what you just mentioned, there's a lot of—so I need to distinguish analog computations and analog computing to other stuff. There is a lot of work in theoretical computer science that is quite scattered, and what analog computation means can be very, very different. Some of the first kind of analog systems that Shannon worked on are quite different to the way we think about analog computing later on. And whether we compute on real numbers necessarily does not mean this needs to be—this can be a simulation as well. It doesn't need to be a kind of physically instantiated computation. So that is vastly different notions of analog computation that I have not yet consolidated in my mind, or there is no consilience. And then there's also, like, you're right about some of the new work from Earl Miller that is coming out with analog computations being one of the, I guess, premises of or the underlying features of consciousness. And I do like that work. I think it's really good. And for us, it's kind of this mix again. Like, so yes, I do think that given the different substrate, I do believe different computations are possible because it is what I term as a kind of structural ontological primitive that is necessary for the computation. So analog computing or some kind of field computations might be one of those, but it's not necessarily sufficient for consciousness. For us, we outline three features, but since then we've also developed much more work into embodied systems and other features that could add to this biological computationalism we speak of. Yeah, I think that's-

[11:29] Jaan Aru: Nick, you're right. I mean, it depends on the substrate, and if LLMs were differently instantiated, it would also change the calculus for us. The key that we're after is really finding these properties X, Y, Z that really are crucial for consciousness, say. And we do think that these properties X, Y, Z are way more complex than simply saying, "Oh, it's a global workspace," for example. There is something more interesting. But yeah, it is probably the-- So we don't have anything against LLMs per se. Simply it seems that this substrate is a very peculiar one where they are currently implemented.

[12:17] Borjan Milinkovic: I should also like to add something else to that. It's more historical in a sense. And I've always had this assumption that Turing, for example, or Church were mathematicians, not physicists. And in a way, the abstract notion of a Turing machine is built on recursion theory and whether the function, where all recursive functions are computable in some sense. It was only later that there was this notion that they found the physical instantiation in Boolean logic gates where the von Neumann machine can be used. And there is an essence already there that the kind of physical primitives are the ones that instantiate the Turing machine. There is this kind of implicit circularity there. While for us, it's kind of coming from the bottom up, the other way around, trying to not just think about this notion as mathematicians, but rather think about it from the physical basis as well. So given that physics has some constraints to the way computations might be able to run and the additional constraints of biology, maybe we can begin to understand the particular operations that happen in brains or in systems before we then abstract away from them slowly and slowly.

[13:44] Michael Levin: So could I just confirm a couple of things? First, if you've got two systems that have exactly the same behavior, so the input/output map is exactly the same, but the inner architecture is different. So one is built on the multi-scale, continuous principles that you were talking about. One is the classic kind of thing. But the output is exactly the same, let's say. And I guess if you don't wanna stipulate to that, that would be interesting, too. But it seems to me that we can say that, you know, it seems likely that the output can be the same. And in that case, would you say that their status is actually different, even if the output is exactly the same?

[14:27] Jaan Aru: Yes.

[14:28] Borjan Milinkovic: Yes.

[14:30] Michael Levin: Okay.

[14:30] Borjan Milinkovic: But I think it depends on the output we're talking about. This is also-

[14:35] Jaan Aru: Yeah, exactly

[14:35] Borjan Milinkovic: ... a, a kind of-

[14:36] Jaan Aru: How precise, et cetera. But I think Mike already knows this, that we could talk about the precision of the output and which kinds of tasks, et cetera, et cetera. But I think in principle, for the first approximation, yes.

[14:52] Borjan Milinkovic: Yeah.

[14:53] Michael Levin: Well, that's an important contradiction to the fact that, basically you're saying that it is in fact possible to have a proper zombie that overtly has all the right behaviors, but because of the special causal architecture or whatever kind of architecture differences, it doesn't have the right status. So that's, of course, different than the standard computationalist view, I think, and it's important. So, okay, cool. All right. And then a couple of other quick questions. In talking about analog versus digital computers, I want to dig in a little bit about the role of the observer and the perspective. Like, is there really such a thing as a digital computer? I mean, everything is fundamentally made of analog parts. The observer can impose some models on it and say that, "I'm just gonna... I'm picking a threshold. Everything below that is this, above that is that." And maybe we tune the materials to make that easier. But what exactly is a digital computer in your view? Are there any such things? And where does that status distinction actually come from?

[16:11] Borjan Milinkovic: I had a similar conversation just recently with a colleague, and I do think I tend to align with your thinking: what is a digital computer? And really, you're right, like logic gates kind of flip one and zeros given a particular voltage, like for, I think, Mac laptops, like 3.7 millivolts or something. So it really is a threshold that's drawn by the kind of architecture that you've built. But the computation itself happens on those singular flip values in a way. So I'm guessing, like, the digital computer in essence exists because the computation itself is performed on the single threshold value, whether it's 3.7 or 1.4. 3.7, 1 point, it's like a zero if it's 1.4 millivolts. 3.7, one if it's 3.7, for example. So I think the digital computations exist because they're the only states that the system, the physical system, can take.

[17:06] Michael Levin: So I'm very skeptical of that only because Josh Bongard and I have been developing this polycomputing view. I'm skeptical of the idea that there is the computation, you know? I think what we have are observers, and I... And I mean, I understand we've all agreed to sort of primarily observe the system in a particular way and map what we think of as bits, you know, and think this is what the algorithm is doing and so on. But, but I think a lot of that is by convention of observers, and I'm not sure actually that that is the computation the system is really doing from the inside. And I think this ends up actually being important for all these questions of language models and AIs and all of that because we have to remember that while we as observers... You know, it's a little bit like with a human. You can talk to somebody, or you can have somebody else who analyzes body posture, voice pitch, a psychoanalyst, somebody who's analyzing physiology. And these are orthogonal channels of things going on that may have little to nothing to do with what the verbal output is. And I actually have a suspicion that we may have something similar going on with a lot of these systems, where we hyper-focus on one channel because we've made it talk, and so we're now sort of focusing on the things it says, and people ask it, "Do... Are you conscious? Do you have, you know, do you, do you, uh, do you, are you concerned when we're gonna turn you off?" Like, all this stuff. But primarily, all of this is going through one particular interface, which I think may or may not have much to do with what's actually going on inside. And I think it bottoms out in this question of is there, is there a sense in which there is the computation that something is doing, or is it really very observer-dependent? And I think it's the latter.

[18:56] Borjan Milinkovic: I like that view, but I'm not familiar completely with the polycomputing idea. Could you just give me a brief? Sounds interesting.

[19:05] Michael Levin: Sure, yeah. The very simple philosophical claim is that there is no one objective true answer to what computation a process is doing, that it's observer-relative. Basically, there's a set of physical events, something is going on, two observers can look at that event and have equally valid models. I mean, in fact, multiple observers could have equally valid models of what computation, if any, the thing is doing. So, for example, Josh and his postdoc, Atusa Parsa, had these nice papers where they looked at this kind of vibrational medium. You look at the exact same set of events one way and you see an AND gate. You look at it a different way and you see an OR gate. Something like that, right? And it's the exact same set of events, and so you can't ask, "Well, what is this thing really doing," right? And I think just philosophically that makes sense because computation is a formal model that we impose on events, and you can perhaps map it differently. In biology, it becomes really important because I think what's actually going on in multiscale living systems is that every level, no level has any kind of ground truth about what everybody else is doing. And all of the components of a living system are simultaneously trying to interpret and also hack, but for the purposes of this, interpret what everybody else is doing. So you see something going on and you think, "To me, the most useful thing that looks like is a clock. I'm gonna treat that as a clock." And something else looks at that. So I'm gonna key off of these regularities in your timing. Somebody, some other subsystem looks at exactly the same thing and says, "Oh, you're making this awesome molecule that I can use, and I'm gonna do whatever. And by the way, by looking at how fast it degrades, I can also tell something about you and what, you know, whatever." So, there's lots of examples. We have a couple papers with him on this, and there's lots of examples of how this plays out. For example, in biology, one of the coolest things about it is that what it means is that evolution has a choice. So when you have a complex system, and evolution is trying to make changes to it, you have a choice. You can change the actual substrate in order to change the functionality and the computations, or you can change how the observers interpret it. And the benefit of doing the latter is that if you have strong dependencies, right? So there are lots of things like around the cytoskeleton and energy production machinery where evolution finds it really hard to make changes because so many things depend on it. You change one thing and a ton of stuff is gonna break, so it's kind of locked in. And so you have those limitations. But instead, if what you do is simply add observers with different perspectives, then you can squeeze lots of new functionality out of exactly the same set of events. You don't need to break any old dependencies, leave them in place, but you're now adding new perspectives where the same thing is doing multiple jobs, right? And we actually... Atusa and I actually did some work on this. It isn't published yet, but we did some work on this, looking at how basically giving evolution the choice of putting in a new, of tweaking the computational actual, you know, the actual computational functionality versus adding observers. And it's really cool. Evolution is very good at adding new observers and just saying, "I'm not touching this at all. I'm gonna leave it exactly in place, but I can interpret it by looking at it in different ways. I can get multiple utility out of it." So that's polycomputing. So, I mean, so it's just, I think for all these questions, it's important to understand when we put labels on these systems, whose labels they are and-

[22:59] Borjan Milinkovic: Yeah.

[22:59] Michael Levin: And then especially if we're gonna talk about consciousness, and I guess this is my next question, is to, is to... And I'm not too hung up on sharp definitions, but just to get your idea of when you say consciousness, like what exactly you mean. Because if it's really important to have a first-person perspective, since we're talking about consciousness, then I think we have to ask about observers and what does the system itself see. Does the system see itself as continuous versus discrete, in addition to how we see it?

[23:30] Borjan Milinkovic: So I like this polycomputing idea very much, and I think it aligns with what we were trying to define in the paper, is that there is no privileged scale of computation in the first place. Also, I really like this notion of observers, though I would not have called them that and generalized, but I guess it aligns exactly. Something that me and Jan are also trying to do further on from this is formalize that idea, in a way where there are events in the universe, physical events in some case, and there are states of the universe or some kind of constraints that can occur, and these could potentially be considered observers from the perspective that you're speaking about, and it is through the interaction of a particular state or an observer and the physical system in which the computation occurs. So the computation occurs as a relational quality rather than the state, like a state transition, matrix. So could-

[24:38] Jaan Aru: Yeah. I, I, I really-

[24:40] Borjan Milinkovic: I, I-

[24:40] Jaan Aru: I'll send this along. Chris Fields has some really good papers on this, on widening this or this very fundamental concept of observers from quantum theory.

[24:49] Borjan Milinkovic: Yeah

[24:50] Jaan Aru: ... but not talking about microscopic events, talking about everything scale-free.

[24:54] Borjan Milinkovic: I would say that that sounds very much up our alley and what we're looking to do next.

[25:01] Jaan Aru: Mm.

[25:01] Borjan Milinkovic: So that would be great. But also on the consciousness thing, I think for me, basically it's felt experience. And I think the key focus there is the felt. There is some kind of notion of being able to at least a partial global signal that can propagate across a system to be able to, within some time interval, be able to integrate the quality of what it is like. So the feeling of felt experience rather than subjective experience, which I differentiate from felt experience, which might be a more complex notion that we're usually used to defining from philosophy and neuroscience and psychology. I'm more thinking about felt experience from simpler biological systems, and that to me would be a very clean definition of consciousness. I don't know about Jan. I think he's along the same lines.

[26:08] Jaan Aru: Well, I mean, I agree. That's why we're working together. But I think going to Mike's question, we still both look or think that there is some perspective, right? Also to have this experience, there is some perspective. So it's very interesting that you bring it up. And Mike, I don't know if you have thought further about it. If you think about consciousness from that perspective, then what is the observer then or what are the observers in this framework of yours, Mike?

[26:44] Michael Levin: Well, so, so I... And this, and so I guess the only, the interesting thing I could say here, I think, is the following, and I already, I was already bugging Nick about a couple of weeks ago. I was thinking about the following. When people talk about consciousness, they almost uniformly focus on what Bokie just said, which is felt experience. And what I think is happening there is that we're focusing on the input or the read side of consciousness. So here I am, I... What am I getting from the world? How, what is it, what does it feel like to be in whatever perspective I have? But there's a flip side, which I think is the write side, W-R-I-T-E, the write side, which is often, I think almost uniformly neglected. And I think the way to get at it is I draw a little square, like a square, a two-by-two square of the positions around consciousness. So one position, right, if you're an eliminative materialist, you say there is no, there is no actual, what's it like. There is no doing of anything. It's just both of these things are an illusion, and it's just sort of, you know, chemistry go and physics go on, and that's it, right? So that's one position. The other position-

[27:59] Borjan Milinkovic: Mm

[27:59] Michael Levin: The opposite position would be some sort of strong interactionist dualist position, which says, "No, I, I absolut- there, there really are felt experiences, and also there are the other side, so I can act in the world," right? I have actual will that changes what happens in the physical world, right? So then there's the epiphenomenalist position which says the felt experience exists, but not the action. In other words, you can't actually change. You know, it's epi... You do in fact feel things, but it's epiphenomenal. You can't do-- You can read, but you can't write. It's the world evolves according to physical rules, and yes, uh, you do, you really do have some kind of conscious states, but this idea that you're gonna change what happens is fictitious, right? So that that's kind of epiphenomenalism. But there isn't a fourth square yet, which... and this is why I've been playing with this idea of flipping the epiphenomenalism. Now, I'm not saying I believe this. I'm just saying you-

[28:57] Borjan Milinkovic: Mm

[28:57] Michael Levin: ... it's important to think about all the four possibilities. The fourth square inverts the epiphenomenalism and focuses on the other side. Not what is it like to be or what is it like to feel, but what is it to act. So the idea is... So the opposite of what's it like to be a bat, what is it to act bat-like, that kind of thing. And it's the doing. And so, so I really, in my thinking about consciousness and minds, I think a lot about the responsibility of not just what does it feel like to receive whatever is going on, but the responsibility of taking the next step, of choosing among multiple options, right? Of having to integrate everything that has happened to you before into some coherent, like what you, you know, you... I mean, not to get into free will, but what you don't wanna be free from is your own commitments, thoughts of, you know, um-

[29:51] Jaan Aru: Past. Mm-hmm

[29:52] Michael Levin: ... right, from your own past. Being free from your own past is just like what's... That's not a good mind. So anyway. So that's all. So I think in all of these things, we have to look at the action part as much as the receiving part, I think.

[30:08] Borjan Milinkovic: I would agree with that. So maybe our associations with felt are different, or then I'm not using felt correctly because I... Even during the presentation, I think I mentioned this, something to me that differs about—maybe both mine and Jan's opinion with the predictive processing or active inference is that actions are usually some kind of subsumed or subservient to the perception, and for me it's flipped. It's the other way around. I do hold a very similar opinion in regards to that my felt experience doesn't need to be a felt experience of some incoming signals in a way. It's more the felt experience of self-generated patterns of activity that allows you to distinguish self from other in some sense. That might be one criterion, but that would be the primary focus there. So in my respect as well, that kind of self-generated pattern in order to distinguish self from other emerges from this need to move an entire body, body. So needing to, in some sense, shift a material lump to express it in the easiest way. So yeah, I would say I'm in the camp of action first, which is... I'm quite inspired by Peter Godfrey-Smith and then your work as well. And I think Fred Kaiser, and there's a few of these people working on it, and I think my thoughts align on it quite a bit. Yeah.

[31:42] Michael Levin: I mean, one thing to build off of that, the thing about biology is that long before muscle and nerve showed up, living systems were taking actions in all kinds of spaces that are hard for us to visualize. So, metabolic space, transcriptional space, physiological state space. So you don't... This business of embodiment, it doesn't have to be... You don't have to be moving mass through the three-dimensional world. As a simple chemical system, you still have to navigate the space of all kinds of possibilities. And as we've been studying, even small, really small molecular networks can already do associative conditioning and habituation and things like that. So this stuff starts very early on, and I think we have to ask then what is the... If we think that movement in 3D space and the integrated control of your body and all of that is fundamental to consciousness, and I've said this to both Peter and Fred, I think that we then can port that pretty much all the way down, and we can ask, you know... Yes, okay, because of our own commitment to vision and whatever, we're obsessed with three-dimensional space. But living things navigate all kinds of spaces, and they've been doing it long before we had muscles. So, you know, what is it to be a small molecular network doing probabilistic inference and causal, you know, learning and doing all of these things?

[33:21] Borjan Milinkovic: That's a great point. I think I'm actually in the process of learning from your work, Mike. For me, I previously bottomed out at conductive tissue-

[33:34] Michael Levin: Mm

[33:35] Borjan Milinkovic: ... and the need for some kind of whole body or partially whole body contractile motion. And now I'm thinking, trying to run and see how much I agree or disagree with it, but that's been a new phase of learning for me to see how far down we can take it. Yeah. I think it's been a pleasure to extend what I thought was the bottoming out level of where I would call felt experience to see how far that extends.

[34:08] Michael Levin: I mean, there's also a funky thing there. This is the recent work of Federico Pagosi in my group, where we were looking at causal emergence metrics like the Tononi kind of type of, right, like phi and those kinds of things. So we've been looking at those in small gene regulatory networks as we train them, and there's something very interesting that happens. As you train them, the causal emergence goes up, and as the causal emergence goes up, they get better at learning, and so there's this positive feedback loop, right? But there are different kinds of networks, and some of them, as you train them, the causal emergence just goes up and up and up, right? Others, while you're interacting with them, it shoots up, but as soon as you stop stimulating them, it drops back down. And to me, it reminds me—I don't know, you guys can co-comment on this. It just qualitatively looks like certain networks aren't sophisticated enough to keep themselves awake. It's like they-

[35:14] Borjan Milinkovic: Mm-hmm.

[35:14] Michael Levin: When you're not pushing on them, they just dissolve into the void, and they can't hold their stuff together.

[35:20] Borjan Milinkovic: Yeah, totally. Mm-hmm.

[35:21] Michael Levin: But others can. Others can do it in between, and it's like this notion of what's, what's Claw doing in between you prompting it, right? Is it, is there anything? Well, some of these things do nothing, and they disappear, and others don't. And you can-- We now have the math to actually distinguish which ones are which. So, I think these are extremely minimal models for some of the kind of stuff we're talking about.

[35:45] Nicolas Rouleau: In the language of IIT, you might have something there added on to the sort of canonical theory, which would be something like durability or some lasting property. Yes, the system is integrated, but can it be crystallized in form or function for some period of time so that it exists for longer, right? The intrinsic existence that IIT talks about. There's really... It's not clear that that existence is being recreated or if it comes in and out of existence or if it exists for long periods of time. I mean, they do have ways to deal with these sorts of things. But I do wonder, when you talk about all these things that cells were doing before muscles and nervous tissue existed, there's all sorts of interactions happening, and what you're seeing in your simulations or in the lab with these gene regulatory networks, that's been happening for a very long time. All these circuits may have they may exist on a continuum of integration over time. And I wonder if we have to revive something like the concept of a homunculus, but extend that out to different systems, not just brains. And by homunculus, I mean in the sort of Hewlings-Jackson sense of a representational space where you have parts of the world that are mapped onto other parts. And I think of the cell as existing in such a way where there are all sorts of events occurring, and they... You know, we can talk about these couplings. If you have an event that involves two parts and you have a-- I think in brain concepts, so it can think about pre and post-synaptic, but it can just be a pre and a post across some interface. So you can think of the post as reading the pre and as kind of reading a truth claim about the world. And in the abstract sense, we sort of assume that the coupling is perfect or that the coupling is reliable. But in the real world, couplings are in flux. And so you may have a reading of, well, the system is off, but in fact, the system is on, it's just not coupled. And so there's all sorts of things happening here in terms of interactions that, you know, just get-- we're not including alternate options into our models of how things work. But something like a homunculus that could integrate all these events that are occurring inside the cell, for example, in a very complex gene regulatory network, something that could piece all that together and map it in such a way where you get a kind of global readout of what's happening in that system, something like that might be useful to try to find in the world in different systems, in different unconventional, putative minds.

[38:52] Borjan Milinkovic: Mm-hmm.

[38:53] Michael Levin: We've been looking at-- So actually, S-Santhosh and I have looked a little bit at, in these networks of regions of the network that represent the whole thing and that kind of, like the homunculus kind of idea. Some of the thing is some of these networks are not large and complex at all. Like, the smallest network that you can make associative conditioning out of is four. Four, four subunits. Like, that's it. They don't have to be very large. And there is, you know, it's-- what you were saying about durability, well, one of the funny things is that when you look at the graph of the causal emergence every time we stimulate it, even the ones that fall asleep in between, as you stimulate them, the next bump is gonna be higher and higher. So they do return to zero, but the bump isn't the same. So there's-- they save state somewhere, right?

[39:40] Borjan Milinkovic: Yeah

[39:40] Michael Levin: ... they're stateful in that way. And so even when the thing falls asleep, it also reminds me of the old story of the people with brain damage, for they can't form new memories. And so I think it was, you know, who told the story where you walk in, shake the guy's hand, go around, come back in, he shakes it as if brand new. But one time you stick him with a pin, you know, and then you come back and he's like, "Yeah, I don't really, I don't really shake hands." So he don't remember the original event, but something stuck, you know?

[40:15] Borjan Milinkovic: Yeah.

[40:15] Michael Levin: And it reminds me of this, is like, the thing you're measuring comes down, but there's a persistent state that's saved dynamically somewhere. And if you can do that in a network of under 10 subunits, these crazy things that we make with trillions of parameters and whatnot, I mean, who the hell knows what, what-

[40:37] Borjan Milinkovic: Mm

[40:37] Michael Levin: ... what's going on in there.

[40:38] Nicolas Rouleau: So I wonder if any of us disagree with— Because this durability is a kind of working memory, but not in the brain sense, in the general sense. Does anyone disagree that memory would be a kind of contingency for a conscious system?

[40:55] Borjan Milinkovic: I would actually agree. Previously, I've also done some work on another level on that, but I would otherwise agree that this is a core component. I would also like to use this chance to thank both of you because I am hearing that it's dinner time for me, for my family. So I really love this discussion. It's really thoughtful, but I have to stick with my family. I will watch the recording, but I'm truly thankful here for your time, Mike and Nick. Thank you.

[41:29] Michael Levin: Cool. Thanks. Yeah. Good to see you.

[41:32] Borjan Milinkovic: See ya.

[41:32] Nicolas Rouleau: Take care.

[41:33] Borjan Milinkovic: I would agree with the continuity you speak of. I would call it self-determination in a way. And I think that if I'm thinking about the causal emergence measure you're using, I'm not sure if it's the same one or if it's still based on effective information. But if it is, I think one great extension to that would be actually within the measure itself to have a notion of self-determination or this self-actuation. And that's something that I've been currently working on with a different measure from effective information and from causal emergence, but it's on the same idea, picking out macroscopic observables that in some sense have a self-determining capacity. And I definitely would agree that I guess memory or some kind of information storage is required or is a contingency to conscious experience.

[42:36] Nicolas Rouleau: Is there any objective claim that's being made when we talk about self versus non-self, or is that entirely a subjective claim? If a supervising non-object could view everything that's happening, including things that we consider to be subjects and things that we consider to be non-subjects, do you think that distinction could ever be made from the outside, so to speak? Or can that kind of distinction only be made from the perspective of a subject?

[43:13] Borjan Milinkovic: Wow. That is honestly a great question, and I'm not witty enough to answer it on the spot. I don't know.

[43:23] Nicolas Rouleau: The reason I bring this up is because I tend to view self and non-self from a cybernetic perspective. It's really difficult to dissociate them when you're including them in a feedback loop.

[43:40] Borjan Milinkovic: Mm-hmm.

[43:40] Nicolas Rouleau: If in fact you think environment is driving organism, but organism is also directly driving environment and vice versa, it's very difficult to separate them. They look like, at least in that frame, they kinda look like one thing. And yet, cells distinguish self and non-self. When you ask a person to distinguish themselves from their environment, I mean, there's certain brain injuries that will actually make that distinction go away perceptually. But generally, people will say, "This object is not part of me," and so on.

[44:17] Borjan Milinkovic: Yeah.

[44:18] Nicolas Rouleau: But I've always wondered, from an outside perspective, the miasma just floating around, could... Is there any distinguishing factor that can actually separate a subject from its environment, or is that just illusory?

[44:35] Borjan Milinkovic: I might have a better answer now. I'm thinking about it in the context of autonomy, which is something that I'm primarily focused on in my theoretical thinking. And I align with some other work done on autonomy where there is a notion of constitutive autonomy, which is the idea that your border between self and the environment is driven by the kind of recurrent processing within the self, within the system itself. And there is a notion of interactive autonomy, which is autonomy from the environment. So this is like two slightly different measures. And in this sense, I think that there is a clear and kind of operational definition of how you can consider a dynamical subsystem within a larger dynamical universe as a self from the outside as well as from the inside. Inside, it would be high constitutive autonomy. Outside, it would be low interaction, so higher autonomy from the interactive environment. One other thing that I should note, and I wonder what you think about this, is something that I've been inspired by. I think Humberto Maturana said it in one of his papers where he says, "A being comes into existence when a world is severed in two." And I love this idea because it kind of defines an environment relative to every agent. So there is a kind of... There is a degree of autonomy or there is a degree of agency given a particular scale from an environment, as well as the fact that the environment is induced by the agent which has the autonomous border itself. So what might be an environment or a kind of working environment for a cell is very different from a working environment for a heart or for a brain, for example. And there is some kind of integrated quality across these scales. But yeah, I think that was an extension on your point, but I'm also interested to hear what you guys think of that, the induction of an environment.

[46:47] Michael Levin: I mean, so I have a framework where the boundary between self and world is defined by the size of the goals. So I have this notion of the cognitive light cone, which is not how far your senses reach and how far your effectors reach. It's the size in some space. It's the size of the biggest goal you can pursue. So what are the states in the world that you actively care about managing? And the size of the states that you are concerned with tells me whether, you know... Like, for example, in space-time, it tells me whether you're a bacterium that only cares about the local sugar concentration versus a dog that doesn't care about what happens three weeks from now but does have a bigger area that... Right? And experimentally, where this comes up is two places. One is our story on what happens with cancer and what happens when individual cells disconnect from the electrical network of the rest of the body, and their cognitive light cone shrinks to the point where they're just amoebas again. And as far as they're concerned, the rest of the body is just external environment. Whereas before... And what has happened there is that prior to that, they were part of an electrical network that shared memories. It had this collective memory thing going on where their goal state was this enormous abstract thing. We're building a limb, you know, with five fingers, and no individual cell knows what a finger is, but the collective absolutely does. And the collective is able to support a standing bioelectrical pattern that tells them what the heck they should be working on. But once they disconnect from that, it's gone. So from that sense, the cancer cells are not more selfish. They are just have smaller selves. So their selves have contracted and, you know, the rest of the body is just external environment. But the other, the other side of that, that's really instructive is the embryonic side. So you look at a blastoderm, and there's, you know, 100,000 cells or something, and you look at that and you say, "Oh, there's an embryo." Well, you have to ask, what is there one of? What are we counting when we say there's one embryo? I mean, it's, you know, the hundreds of thousands of cells, molecular networks inside of that. Like, what is there one of? And I think what you could say is that what there's one of is a kind of shared delusion almost. It's a shared vision of, we are all supposed to be taking this journey in anatomical space. We're far away. We're a single cell, right? Or, you know, a flat disc right now, but we're supposed to be this complex thing that's gonna be a gastrula and then the neurula and whatever. And so everybody's in agreement that that's where they're going. And one of the cool things you can do is, and, you know, I used to do this in duck embryos. You can, they're nice and flat, and you can take a little needle and you put some scratches in that blastoderm and separate them into islands. And when you do that, every island doesn't feel the rest of it and says, "Well, I'm an embryo, and I'm gonna self-organize into an embryo." And you find out a couple things from that. First, that the number of individuals in an embryo is not fixed. It's not genetically determined. It's anywhere from zero to, you know, half a dozen or more, right? And it's a dynamic process of self, of, you know, pulling yourself together and figuring out, "Where do I end and the outside world begins?" And then the other cool part is that if you make the scratches but you let them heal, then you end up with conjoined multiples, so twins, triplets, and so on in the same blastoderm. And then you get some really cool stuff because the cells in the middle of two have to figure out, "Am I the right side of this guy, or am I the left side of this guy?"

[50:23] Nicolas Rouleau: Mm-hmm.

[50:23] Michael Levin: And you actually get... And so this explains for the first time why I did this in, '96 or something. It explains for the first time why conjoined twins, a lot of times in human conjoined twins, one of the twins has laterality defects. Their heart might be on the wrong side and whatever. And it's precisely because of this. When you're sitting next to another twin, there's a lot of uncertainty about what am I part of? And that-

[50:46] Nicolas Rouleau: Mm-hmm

[50:46] Michael Levin: ... that am I at the left side or the right side or what? And figuring out where that boundary is is not trivial. And so to get back to your original point, Nick, I think that there are, again, multiple perspectives here. I think we as external observers can try to put some labels on things, but I think for significant... I think these agents are interesting to the degree that they have their own opinion on the matter. And so, right? So when the system itself is trying to define inside from outside in some way, whether that's a single cell boundary or an embryo boundary or whatever, I think its opinion carries a lot of weight, too. And to the extent that it has strong self-models and self-opinions about that, it becomes a much more interesting agent, where it's not just external observers that can do this kind of estimation of what we see. How many individuals do we see?

[51:44] Nicolas Rouleau: Yeah. Yeah, it's really interesting. And like-

[51:47] Michael Levin: Yeah.

[51:47] Nicolas Rouleau: I mean, you can even do this stuff statistically, like if you have large data sets and you're looking for patterns in the data set that might exist, depending upon how you apply something like a factor analysis, you can end up with a very large set of distinct factors. Or you could force cluster things into two or three or four, depending upon how you set up the statistical model. And I wonder, based upon both your comments, like it seems like boundaries are really important, and the word divide is doing a lot of work. Like, what, what does divide mean in, like, to divide a world? And I wonder if what this just means is, like there's a kind of... You can think of every object in the universe as having a relationship with every other object, and some objects are hidden from other objects, either by distance or by charge mismatches or all sorts of contingencies that make them less likely to interact. And so you could actually seriate every object in the universe with every other. You could say, like, this particle has more interaction with this one, less with this one, less with this one, less with this one. You could just do that, and you could create a kind of gradient of separation, let's say. And then, that's just the most extreme form. But you could do that with biological systems, chemical systems, physical systems, where you're just looking at relationships between parts, let's say. I mean, maybe I'm putting the cart before the horse here and calling things parts and then looking for a way to separate things. But you could do this kind of operation where you're looking at the subset of the whole and then kind of defining relationships between stuff and seeing how it breaks apart. And I think you're right. Like, in the end, depending upon what lens you apply or what algorithm you apply or what kind of sorting mechanism is brought to bear on this large set of things, you're going to get different divisions. Like, I don't know that there is an answer in terms of like, well, there are actually two parts here or there are three parts.

[54:06] Michael Levin: I-

[54:07] Nicolas Rouleau: There, there are potentially many answers to that question.

[54:10] Michael Levin: I would tend to agree with that, because I generalize the notion of observers, as I mentioned before. And in my case, if we're looking at a system itself and how it develops agency or autonomy in some sense, I would consider the environment to be the observer that constrains the particular system. So the coarse graining of the system itself is relevant and relational to the environment, which as something that induces constraints on it-

[54:44] Borjan Milinkovic: ... is to me an observer-like quality. And I've been thinking about how to define this observer for a long time because the way you think about it from classical physics is the fact that this observer is some kind of human with some perception in essence. But that's not how we think about it when we look at a quantum physical level or just in general, standard model of physics, that an observer is a relational quality from the outside and... Yeah, so for me, the kind of macroscopic grain at which some agency's maximized has to do with the environment that induces it. And this kind of goes back to Mike's comment actually that it depends on the size of the state space in which it is interacting in. And to me, that just speaks to maybe the emergent complexity which is induced given a particular agent and how it interacts with an environment. So, yeah.

[55:42] Nicolas Rouleau: That's really interesting. That connects with something Mike and I have talked about before in conversations. It's a kind of paradox, which is in order to get one, you actually need two. In order to have an object that's distinguishable from the world, to say there's one, you actually need to start with two. It's a very odd thing because if it's relational, if you need the environment, then you actually can't have one until you have two. It's just very backward.

[56:13] Borjan Milinkovic: But that's that division event I spoke of or that poetic comment by Maturana. It's exactly what I completely agree with that too.

[56:24] Michael Levin: Chris Fields and I have this thing coming soon on the symmetry between agent and environment and how systems offload computation to the environment and then basically it ends up being extremely symmetrical to say, like, who's doing the work here? It becomes really interesting. So yeah.

[56:46] Borjan Milinkovic: Yeah.

[56:47] Michael Levin: Cool.

[56:48] Borjan Milinkovic: Very cool.

[56:50] Michael Levin: Thanks very much, guys. Really, really interesting as usual. So, Boki, you said you were working on some new stuff. Send me whatever you guys have. Please send it along. I'll send you some stuff.

[57:03] Borjan Milinkovic: I would love to. That's great.

[57:05] Michael Levin: Yeah.

[57:05] Borjan Milinkovic: I think we align on many things and I like this-