Watch Episode Here
Listen to Episode Here
Show Notes
This is a 1-hour working meeting discussing Alexander Ororbia's and Karl Friston's paper on Mortal Computations and their relevance to our work in reprogramming biology and cognition more generally. A few papers:
Alexander and Karl's paper: https://arxiv.org/abs/2311.09589
Michael Levin's paper to which they referred a couple of times: https://osf.io/preprints/osf/4b2wj
CHAPTERS:
(00:00) Mortal computation and morphology
(09:29) Uniqueness, replicability, energetics
(19:26) Gene networks as learners
(27:58) Dynamics, memory, Markov blankets
(38:58) Designing and probing systems
(48:56) Opacity across physical scales
PRODUCED BY:
SOCIAL LINKS:
Podcast Website: https://thoughtforms-life.aipodcast.ing
YouTube: https://www.youtube.com/channel/UC3pVafx6EZqXVI2V_Efu2uw
Apple Podcasts: https://podcasts.apple.com/us/podcast/thoughtforms-life/id1805908099
Spotify: https://open.spotify.com/show/7JCmtoeH53neYyZeOZ6ym5
Twitter: https://x.com/drmichaellevin
Blog: https://thoughtforms.life
The Levin Lab: https://drmichaellevin.org
Transcript
This transcript is automatically generated; we strive for accuracy, but errors in wording or speaker identification may occur. Please verify key details when needed.
[00:00] Michael Levin: Alex, if you could start and just give a few more words on this, the whole notion of mortal computation and specifically the issue of programmability and morphology and how we can distinguish when we're looking at a system, how can we distinguish what aspects of mortal computation we're looking at. Anything along those lines would be very useful for us. Maybe we could start there?
[00:35] Alexander Ororbia: I did read last night the paper you shared with me and Karl and Chris and some of your notions in there. A lot of it sounds like parts of mortal computation. In terms of the morphology part, the argument that Karl and I make in that paper is that the structure is critically important to the actual system. There's a lot of this idea of the amorphic formulation of computational models. Karl and Chris had a paper that predated the mortal computation paper, but we developed that metaphor further. You can draw a dot-and-arrow diagram of a neural network and that endows it with this pedagogical morphology. But it doesn't matter. It's just a pile of linear algebra that will do its calculations. We have to go through the von Neumann computing architecture to transmit the weight memory. That's where this great thermodynamic cost comes in. Living systems, Michael, are inherently morphic. This is the part we didn't have in the paper, but your paper used the word polycomputing, which is the idea that a substance can compute many different things simultaneously. The idea is that it's all about the substrate. Mortal computation says that if we're going to think about artificial intelligence, general intelligence, or machine intelligence, we're going about it in the wrong direction by having this divorce of the computational architecture from the morphology or the substrate that this is going to be enacted on. Because living systems, if you change the morphology, you change the properties of the system, you change what it can compute, what it can do. You talk about the liquid brain or the idea that we have this self over time, which isn't at odds with mortal computation. The idea is that you're constantly changing, you're persisting, you want to persist, but your system is going through autopoiesis. Mortal computation absorbs that and says we should be designing systems from that perspective. You were talking about programmability. I think that was an interesting part that we didn't really get to chat a whole lot about.
[03:24] Alexander Ororbia: It wasn't 100% clear what was meant by programming the morphology or the system. After reading your paper, I think that gave the answer as to how, because I said originally, I'm thinking that the morphology or the substrate dictates strictly what you can and can't do. Yes, it will change and it will repair and go through damage or do things like self-replication. But I wasn't thinking about the human designer. Let's say if we are designing a chimeric system or something new, manipulating that morphology so easily, it's more like we're going to be looking at computational simulations of that, and that would be programmable. I was talking about we could do software simulation of anthrobots or xenobots of that form. Then you could set the properties of the environment in the system roughly according to something you want to look at and then simulate it and see what it does. Your paper and your work is an example of you can directly program the genetic aspects of a system, or you can manipulate the bioelectrical chemicals. You had an example of a tadpole or a froglet where if we apply the right electrical stimulation, you can get it to grow a tail or get it to grow a leg. So I think that is where the programmability is, and then that would be more — you would have actual experience programming the morphologies, whereas I was thinking more from the perspective of neural networks, where we have the structure, and we want that to now be at the very top of what Carl and I call mills. You remember in the paper, there's the mortal inference, learning and selection, and at the very, very top is structure. That would be something that I think is under-explored in the area: maybe we have neurogenesis, synaptogenesis, and then the system has to use that as another aspect of how it evolves over time. That was thought of, or at least the way I was writing it originally, it's doing that in its own way. You're not really involved in saying I'm going to help you do model selection. But maybe if you encode priors, that was the other aspect I had. If you were encoding certain constraints, you could skip ahead from what evolution would naturally randomly walk you to and say these types of structures are invalid. That would be programming structure from that perspective. I think it's a tricky topic because I wasn't entirely sure what exactly "2" was meant by programmability, because you have experience doing that. So the answer is yes, it can be done from your perspective, and we would just be translating it to chimeric systems or artificial systems rather than it only being biological material.
[06:14] Michael Levin: The kind of programmability I had in mind — just as an example, we have these flatworms, these planaria, and you can chop them into pieces and every piece regrows a complete, perfectly patterned little worm. You could ask the question, how does it know how many heads to make? It turns out that there's a body-wide electrical pattern that dictates the number and location of the head. The amazing thing about that substrate is that if you change that pattern, the tissue will hold it. We can change the pattern to say, no, two heads instead of one, and it holds. Those worms, in perpetuity, despite their completely normal genetics, will continue to regenerate as two-headed worms. It's a very minimal example of reprogrammability because we don't have complete control yet. In the future we will, but at the moment we don't. It is an example where the hardware is hard in an important sense. The genetics are normal; there are no weird nanomaterials, no genomic editing, no synthetic biology circuits — it's stock hardware. But because of this physiological experience that it's had, it now has a different pattern that it uses as the target morphology of what it's going to do if it gets cut. That's the kind of plasticity it has: the material is basically the same, but it has a memory of a past event, and that memory guides how it behaves in anatomical space in the future. That's the kind of thing I wanted to explore with respect to this framework.
[07:51] Alexander Ororbia: And I don't think what you described is at odds with what you would do with a mortal computer. The idea is that that's where the programmability comes into play. You're encoding that and then seeing how the morphology in the system evolves over time. If you want the two-headed worm example, I don't see any reason why that wouldn't translate to artificial or non-biological systems if we are able to formulate what that morphology looks like. I think the key is setting that up. And that's what Karl and I have at the very end of the paper. We talk about, for example, someone like me, a computational neuroscientist, computer scientist. I don't have access to xenobots or the biological material that I'd love to, or organoids. I really find those fascinating. But we could simulate them. And that's what we have in the appendix, actually, in the already long paper, digital morphologies. That could be a way to bridge the gap between the computational researchers and researchers like yourself. Can we come up with benchmarks or system setups that I could play with the properties, mathematical models of these biological morphologies, and then do investigation without the costs and the barriers to entry to working with biological material or some of the things I don't have. So I don't see anything at odds. I don't know if Karl would want to add anything that I might be missing.
[09:22] Karl Friston: That was very fluent. I think you've covered everything there.
[09:29] Chris Fields: This dimension of programmability could also be expressed as the dimension from uniqueness, which mortal computers, as I understand it, have more of than my laptop, to replicability or exact replicability, copyability. To the extent that a system is really unique, you can't program it, in part because you don't have two copies of it, so you can't test what you're doing in any way. You can't test reproducibility if you're working with a system that's completely unique. Biological systems are somewhere in the middle. Laptops are intended to be way out on the extreme replicability end. Because a laptop is completely replicable, it's a completely generic entity; it's almost a Turing machine, almost just an abstraction. When we're programming, we can treat it as an abstraction, and we don't have to worry about where the power comes from, why the thing maintains the same shape over time, etc. If it doesn't maintain the same shape over time, you take it to the repair shop or recycle it and buy another one. In biological systems, you have to worry about all of that. As you point out in the paper, and as the 4E people have been pointing out for decades now, that's part of the algorithm, or that's part of the operating system: that shape. It's not involved at all in my laptop or in the operating system. Even the operating system can treat the hardware as an abstraction. We can identify those two axes, the dimension of programmability and the dimension of uniqueness. One of the things that you emphasize in your paper was the energetics of using the body as part of the operating system. It meant that you didn't have to pay for a lot of memory, for example, or quite so much processing power. You do have to pay the cost of keeping the body intact. But, as you pointed out, at least in organisms, that's cheaper than the cost of stamping out more laptops and then equipping them with enough voltage to keep them in the classical domain so that they don't start acting like quantum computers, which they actually are. It would be useful to try to relate this issue of resource cost to the issue of uniqueness as well as the issue of programmability. I suspect those are distinct dimensions, but the usable area of that state space involves a lot of correlation between those dimensions.
[13:44] Alexander Ororbia: Yeah, I agree. I think that would be very interesting to explore. I wonder, Michael, if that resonated with you, because I feel like that touches on pairing the mortal computation paper and the paper you shared with all of us. I think the two start to get into that idea of uniqueness, programmability, and resource cost. There are some different dimensions we could explore. I also wanted to comment, Chris, that when I first was writing the paper and then I shared it with Karl, I wasn't thinking of biological systems as an ideal target. I was emulating some aspects of those systems, so I'd be giving up some uniqueness because you said it lies in between the immortal laptop and the perfectly unique system itself. I was concerned with the artificial intelligence community liking the idea of immortal computation and that complete divorcing. You do lose that reproducibility the moment you leave even a few steps away from immortal computation, because the substrate is important. Karl and I argue even more strongly that it's the morphogenesis too, and the changing process, which complements your paper, Michael, as well. The idea is that change is also very important, and evolution over time is not something you'll have on a typical deep neural network that lives on top of the von Neumann architecture. My last comment, Chris, is yes, that's exactly the key: in-memory processing is what we want. That's why bringing ourselves as close as possible to the hardware — until we reach the Landauer limit — optimizes thermodynamic cost. As you, Karl, and everyone here has shown over time, that's the flip side to the information-theoretic variational free energy: the thermodynamic free energy. At the end of the day, we want to be there because that's what biological systems are. They are much closer to the Landauer limit than anything in machine intelligence today. That suggests it's getting worse because large transformers are very costly. Karl and I also note the carbon footprint, so that's a good motivator.
[16:15] Chris Fields: I find that energetic analysis very compelling. I'm very interested in why biological systems can be quite so efficient. I think in many cases they're efficient because they're able to use quantum resources when they're doing molecular computing and maybe even when they're doing macromolecular computing. One other comment about a dimension you mentioned in the paper was this dimension of explainability, which AI is very obsessed with. As one gets away from reproducibility, the explanation problem gets harder and harder. In the limit of a unique system, the explanation problem is infinitely hard because you can't do experiments, because you can't replicate anything. So we have that other access to work with also.
[17:42] Alexander Ororbia: Maybe it would be interesting, since you were bringing up these three axes that I caught — uniqueness, programmability, explainability. We also did talk about resource cost, putting out this grid. And then, you saw Michael, and I presented, you guys would have seen in the paper, but I presented it a couple of times, the different types of things that Karl and I consider variants of mortal computers. And so obviously Xenobot is a mortal computer. It had a lot more qualities after I reread papers. But even the silicon model that we had for the non-biological model from Ashby, we can never forget the great Homeostat. Maybe we could plot these a little bit on those axes too, what degrees they're trading off. Obviously, we need to figure out which one of these starts to get real close to, as you said, Chris, the really unique. And then that would be the extreme one where explainability would be really difficult. We could plot where those are. That could be an interesting figure to show. I'm sure, Michael, you probably have other examples that Karl and I might have missed. There might be some other biological chimeric systems, things that are even less biological but have a little bit of it. You did touch on nanotechnology as well in your polycomputing paper you shared with us. Maybe there might be some in soft robotics. There might be something there too that could count as variations to mortal computers that trade off on these axes. That's something else I thought of as Chris is explaining.
[19:26] Michael Levin: Another model system to think about. We do have simulators of some of this stuff. We should be in touch and give you access to some of that because maybe you can do some analyses. We have bioelectric simulators and things like that. Another kind of model system to think about — this is something Patrick is doing at the bench, and I have somebody doing the computational analysis — are these gene regulatory networks. The abstraction is quite simple. In the continuous case, it's a few ODEs. They're nodes that turn each other on and off, and that's it. If you study these things, you find some really interesting features. For us, one of the most interesting things is that if you do temporary stimulation of the different nodes — you just grab one of the node values and crank it up or down for a little bit, and you keep the structure of the network completely fixed. You're not changing the weights, you're not changing the topology, the hardware is completely fixed. All you get to do is temporarily raise or lower the activation of any node, then you wait and see what happens. If you do that, and treat it in the behavioral science context, you can show things like habituation, sensitivity, basically six different kinds of memory, including Pavlovian conditioning. These things learn. We've been very interested in this question. We have a couple of papers showing how they learn. One of the really interesting things is where is the learning stored? This is something that all of the reviewers of the original two papers got hung up on because we say the hardware does not change. They said, great, then you can't have it because where could the memory possibly be? It's a dynamical-systems thing where they get chased into a regime where future stimuli will cause very different outcomes because of their history than past outcomes. I wonder. This is what I was going to ask you to think about from your frameworks' perspective. I wonder if the business of uniqueness is related to this issue, called privacy: the idea that there is an inner perspective to a system that's had a certain set of experiences. It has a history in the world that is not available to outside observers. We spent a lot of time with my postdoc, Federico, thinking about: you look at a network, can you tell whether it's been trained? And if so, can you read its mind? You won't get it from the hardware. The nodes are no different. We have a visualizer that tries to show various aspects. On the left and right of the screen, you start with a hardware view that never changes. As it learns over multiple experiences and stimuli, something absolutely changes. We have some ways of thinking about it. This question — can you as an outsider, is there anything about mortal computation that speaks to what you can tell about a system as an outside observer versus what you know as the system yourself from the inner perspective? Is that something you think about?
[23:07] Alexander Ororbia: I'm going to give a piece of it, and then I'm going to hope Carl can tag in a little bit, because I think he can flesh this out a little bit better. This might be confusion over what you might have explained, Michael, about the reviewers. You said, I fixed the hardware, and on top of that, I fixed the plasticity, because you said we can't change the values of the synapses or the connection strengths. I do think in mortal computation we did address this. Carl and I decomposed it, and it goes back to Mills. But I might be misunderstanding. We're going to decouple the privacy and the observer perspective because I want to hear what Carl might have to say to that. For why learning would still happen, even when you fix those things, it's just the inference. The way that we looked at it in Mills was there are these different time scales of learning. If you were to pin the structure of the S in Mills and then pin the L and say you can't modify those, we still had one more piece, which was the very fast time scale. You talk in your poly computing paper; I've done a lot of work in that. Carl obviously has done a lot as well in predictive coding, predictive processing. We always have the inference dynamics. The idea is that, and I'm sure you thought of this, is why I was surprised the reviewers were not understanding. There's short-term plasticity. The idea is that when you're doing expectation maximization in a predictive coding network, I can still change the neuronal activities, the firing rates or the spiking rates, depending on what model you're constructing. The synapses never change. Forget about the morphology, because that's a whole nother ball game. I would get adaptation. There was a very interesting paper that came out two weeks ago. Wolfgang Maz and spiking neural nets talked about, "I don't need to modify the synapses. I'm going to do everything in my spiking neural architecture with just homeostatic variables," which he didn't call them, but that's just the adaptive thresholds. If these change, we have short-term non-synaptic adaptation; you get all these effects, and he actually showed it. It's a machine intelligence task, showing in all these tasks without learning in the sense of modifying synapses. That was very interesting: you can go very far. I'll try to dig up that paper. It was something I wanted to go into more detail on later myself. In Mills, we're saying we're under mortal, but the inference dynamics and the fact that these still follow the gradient flow of the variational free energy that defines your system or your functionals would explain why that adaptation would happen. I'm sure you thought of that. I don't know why the reviewer specifically would not have said, "This doesn't make sense: how could you learn if you had pinned all three? It's a static system; you are freezing it in time," which would baffle me. That's my comment: I do think the framework speaks to that because Carla and I were very adamant about the separation of time scales, at least these big time scales. There's always intermediate ones you could bring up. You need them all because there's a causal circularity if you want to build the most powerful type of mortal computer. There was a sentence I can't remember because Carla and I have done many revisions of that paper. It might have been in one of the earlier ones where I mentioned something like: even though I'm saying morphology is important, technically if I was only allowed one, I still have Mills. It's just a very simple search space. We know that you're here; you can't change the architecture. So we didn't break our framework. That would allow us to subsume machine learning and say machine learning is a very narrow case. It is doing something that Mills could explain. It's not mortal, but at least it has a fixed topology and synaptic plasticity is there. We are just speeding up the inference dynamics by making it one step because we don't use EM most times in deep neural nets. That was my comment about addressing learning: if you fix so much, why would it still happen? I definitely think Mills, that piece of the backbone of mortal computation, would speak to that. Now, in terms of the observer effect, what does that tell us about what's going on inside our tag team? Carl, what do you have to say to that?
[27:58] Karl Friston: Before I address that, which in my world was a very simple answer, you can't. That was an interesting exchange and interesting examples. I was thinking from the point of view of classical flows and physics that would provide a simple picture of how you can remember stuff without changing your connection weights. Alex, you identified the key thing here, which is the temporal scale. It was interesting you introduced Wolfgang Maass because he, for many years, has been the king of liquid computation and echo-state machines, which has the same kind of semantics as liquid brain. It's a very powerful black-boxy kind of dynamical system approximator that has been proposed as one architecture for doing predictive processing and model computation. The key point that has just been made is that the dynamics matter and the dynamics are shaped by the landscape, Lagrangian, variation, free energy, whatever you want. That is a function of the implicit gradients that depend upon the sensitivity of the nodes in any given network. That sensitivity can either be read as a connection strength or as sensitivity in terms of the extent to which I change my internal dynamics given a particular external perturbation. That becomes time- and context-sensitive with any nonlinearities. If you're talking about a nonlinear system, the bright line between the connection strengths and the current effective connectivity in this context, in this part of phase space or state space, becomes very blurred. If you're writing down the differential equations, you could go one of two ways. You could write down a differential equation with many variables representing interactions between different types of states and the rate of change of any particular state that would entail the nonlinearity in question. Or you could say one subset of these variables changes very slowly. Call them connection strengths. Lift those out of the equations and you're left with a simpler set of autonomous differential equations parameterized by other states that change very slowly. Mathematically, you haven't done anything but introduce a separation of temporal time scales. In so doing, you have a different kind of rhetoric where initially you were talking about voltage-sensitive receptors and sensitivity and contextualization conductances and the like, which sets the synaptic efficacy, fluctuating moment to moment. Now you're talking about these being the connection strengths, the parameters of your structure in a mills like context or the strengths of your connections or weights in a machine learning context. The only difference is the time scale. Referring to Mike's example, how can you have memory without changing your connectivity? You're appealing to initial conditions in the context of a nonlinear, random dynamical system.
[31:59] Karl Friston: At what point would you start calling this the kind of memory that could be encoded in terms of connection strengths. In those kinds of systems where the key, not second-order, nonlinear interactions rest upon a subset of variables that change very, very slowly. Under that adiabatic approximation, we'll call this a different kind of memory. It's just because it's slightly slower. I liked the emphasis on the separation of time scales because I think that would have dissolved the reviewers' concerns. If you were just talking about really fast learning in the moment, that is all in the nonlinearities and the dynamics. I keep emphasizing the nonlinearities, Mike, because of the paradox of change. So as soon as you have nonlinear dynamics in any system that has, at one particular time scale, an attracting set, or a random or pullback attractor, you have that itinerancy, which means that there will be some form of changing sensitivity to all the things that I am coupled to. That is definitional of things that have that biotic characteristic set. The nonlinearities, from a classical perspective, are absolutely key here and resolve a lot of the distinctions and give you a relatively simple picture: if there was some way to tell the next version of me my initial conditions in the past version of me, you can, I would imagine, quite simply write down systems that have this kind of memory, which does not involve a change in the connection weights. If you wanted to simulate that remarkable fact that the worms remember that they are on a two-headed trajectory even when they start again, the deep question here is how on earth do they inherit the initial conditions that characterized the termination of their parent or what they inherited from. I think, again, that speaks to this coupling between different temporal scales. Is this a messenger RNA, and how does that propagate through to the electric fields, and how does top-down causation get back in again? It's a fascinating example. I've never heard that before. I'm sure you've told me, but I probably ignored it because it's so remarkable. It's not easy to explain. To answer the question, can you ever know what's going on inside a system? No. I say no polemically from the point of view of the Fianjia principle: you can never know what's beneath a Markov blanket. You can never know what's on the other side of a holographic screen. That's the whole point of a holographic screen or a Markov blanket. All you can do is bring a best guess, an "as-if" explanation to the polycomputing bulk on the other side, which is a simple observation. The whole point of that screen or Markov boundary is that there is a conditional independence given what you can measure. So you can never know other than infer, via what you measure from the behaviour, the inputs and the outputs of a particular system.
[36:03] Michael Levin: Two questions. One is: is it just a flat no, or is there a degree that is easier to know for certain kinds of systems, and for advanced living cognitive systems it's really no? Or is it always the same, or is it a matter of degree?
[36:22] Karl Friston: If you're directly at me, the answer, I'm afraid, is always no. But I don't mean that. The question here, how do you infer what kind of Bayesian mechanics or polycomputation is going on underneath a Markov blanket or inside a cell or inside a brain? This question is my day job and the day job of nearly every neuroscientist. It's peeking underneath the Markov blanket in a non-invasive way that doesn't destroy it to understand the mechanics and to test hypotheses about what is going on. But you're always testing hypotheses. You will never know. There will be situations where the functional anatomy or the architecture reveals itself through non-invasive imaging, for example, or even invasive techniques of the kind that you use every day. All you're doing is testing hypotheses about what you think the generative model is under the hood. Once you know the generative model, then you know the Lagrangian. If you know the Lagrangian, you know the intrinsic or internal dynamics, and you can tell a story about polycomputing, Bayesian mechanics, perception, memory, basal cognition. But these are just stories predicated on the Lagrangian that governs the intrinsic dynamics. The free energy principle brings to the table that you can express that Lagrangian as a function of a probabilistic generative model. So your job is now to identify the functional form and structure of that model and all the processes that it entails. But every time you do that, you're just testing a hypothesis: theory of mind for me, theory of mind for your xenobots, and theory of mind for yourselves. You can break down at different scales. It would be possible to ask about the sense-making and sentient behaviour of a single cell in my brain if you're able to isolate it and get inside there and do molecular or cellular biology, but you would no longer be looking at my brain at that scale. But you could cut across scales in the good old-fashioned way and start to tell an internally coherent story about how it all fits together across scales.
[38:58] Alexander Ororbia: Maybe back off what Karl said quickly, not to interrupt you, Michael, because I had a question for Karl; he triggered an interesting thought. And to say to you, Michael, that's also why I was hesitant: by definition, and since mortal computation rests on the Markov blanket and is underwritten by the free energy principle, I commit to that as well — the answer is no, you can't see what's under the Markov blanket, and that's why I was curious to know if Karl would give me anything I didn't know about. So that was one comment to you. To Karl and to everyone: mortal computation also subsumes artificial systems. I was curious to know if, Karl, if we were to design the internal states, all the dynamics — we have the environment, but let's say you simulate it, you're designing the environment, and you design the Markov blanket — because we talk about, even in our paper, potential sketches of things you could use to build the boundary, the transduction pumps, and all the sub-pumps to actually build a viable artificial organism. Now we have the internal dynamics. We are the designer, and we have specified internal, external, and the boundary. Is there something I'm missing, because we've created the Markov blanket, so now we know what's on the other side? The answer to Michael is not for natural systems that we obviously did not create, but for systems we made. Is there some other concept I'm missing? You would be able to say, I know everything because I built the internal states and specified every bit of the dynamics. And let's say we constructed the environment and the Markov blanket boundary. What about that case? Are we now inspecting it because we made it? We obviously don't need to infer it; we know it. I was curious to know your thought on designed internal states, designed external states, and designed Markov blankets, if that makes any sense.
[41:09] Karl Friston: Mike's gone to the door, so I'll respond to that. I'm not going to give you any deep philosophical insight you don't already have, but just a very practical one. What you've just described is an application of the free energy principle as a method to simulate various mortal computations in the service of building hypotheses about how this thing might work mortally. Practically, that's what we use the free energy principle for. The design is, at least mathematically, very straightforward. All you need to know is the generative model. All you need to be able to write down is a probability distribution of all the causes and consequences that constitute your system. That's it. If you can write that down and instantiate that in a von Neumann architecture, you then just solve the equations of motion that are the gradient flows that will have a solenoidal component on that Lagrangian and you can simulate sentient behaviour and sense making, perception of action, self-organisation, everything that you want to do. Why would you ever want to do that? In order to test hypotheses that this reproduces the kind of behavioural thing of interest, which for somebody like me would be a psychiatric patient, for somebody else a multicellular organism. You are now using a simulation as a way of generating predictions that then you can match against the observable parts of the system of interest, which are just the surface. Just the action of that system. To the extent that the sensory inputs of that system are also known, that's all you have access to. That's how we practically use active inference. For example, we create simulations of Bayesian mechanics in a given paradigm, and then we adjust the generative model, or specifically the priors of that generative model, until it renders empirically observed choice behaviour the most likely under the probability distribution of the actions of my simulated agent. In that sense, you're specifying the structure. But one could argue that even treating the laptop computer that is so non-unique because it affords the opportunity to abstract and do these simulations, coming back to Chris's point, even then you don't know what's going on underneath the hood. In a conversation with people designing some risk architectures and looking at the most efficient buses, they have to guess what's being passed here and there and measure it and get proxies like temperature. You can specify the initial conditions and the structure and you can reboot and reset it. You can, to a certain precision, specify the initial conditions and the structure on which the ensuing dynamics will occur. But to know the message passing of a computer, even in simulation, you would return to your hard no. Certainly on the quantum level that Chris was referring to, it would be unknowable. It's an interesting point. Just foreground the role of simulations in this, and it comes back to why do we want to know all this? It's to formalise hypotheses in terms of simulations that now embody our hypothesis and then look at the empirical system to see whether that hypothesis was correct.
[45:32] Chris Fields: I add another point of view on this. If we think about what we do in practice with ordinary computers, where we have built the thing, part of building the thing, it's not just assembling the hardware. We also put a lot of work into building these interfaces that we call programs. If I'm using a debugging tool, where I can run a program in one window and see what's happening at some level of the execution trace in some other window, what I've done is constructed a Markov blanket, effectively, to use that language, that has different IO channels that access different parts of what's going on in the device. We could think about, from a biological perspective, we have these cells that come equipped with their own native IO channels. But there's nothing that says that we couldn't build more channels into the things so that we could see more about what was going on inside, not by penetrating the Markov blanket, but by adding some IO capacity to the Markov blanket. What does that mean physically? It means you're using a different interaction because it's the interaction that defines the blanket as a set of information-transmitting states. We always have the hard no of the Markov blanket, but we also, from an engineering perspective, because we can interact with these systems in ways that other parts of their environments can't interact with them or don't interact with them at least, we're a part of the environment that can open up new communication channels through the blanket by changing the interaction that effectively changes the state space in which the blanket is defined.
[48:29] Karl Friston: Augmenting with reporters or this ultragenetics is a good example of that.
[48:38] Chris Fields: fMRI is a good example of that.
[48:44] Karl Friston: Yes, yeah.
[48:45] Chris Fields: We were just adding an IO channel to the brain that wasn't there before.
[48:56] Michael Levin: No, please, Carl, keep going.
[48:58] Karl Friston: The catch word in my world is non-invasive brain imaging, and that has a meaning that you are not invasive, but there is a whole century's worth of invasive studies and lesion deficit models and depth recordings and the like, which I think speak to this: how far into the Markov blanket can you peer without destroying the thing that you're trying to, in the Heisenberg sense. I'll shut up and have a quiet cigarette while I listen to you now.
[49:39] Chris Fields: It's not invasive by convention in that you're invading the brain or the magnetic field that wasn't there before. You're not damaging it much.
[49:54] Alexander Ororbia: Like Carl said, it's invasive, but I wonder what that tells us about augmenting the Markov blanket. Carl usually will say, if we want to non-invasively understand it, you can't peer under the Markov blanket. So mortal computation is connecting towards the idea of what you're saying, Chris. We can augment, we can add IO channels. You are designing, engineering these things. Now this was something that did not exist. Michael, how does that shed light on the question that you originally wanted to get at, which is peering at these internal states, because I think this is an indirect way. Carl gave me a great answer about if I just design—because my brain goes to the ultimate engineering—I'll just design it all myself. I even was thinking about if I build the hardware, but Carl's right: even when you get to hardware, you're guessing a lot of times, even with the best educated guesses. So there's still the Markov blanket that you're not breaching. But if you perturb or change, augmenting the cell membrane with something that your lab's group is an example of modifying these things. What does that do to your question? How does that shed light on what you're thinking?
[51:13] Michael Levin: I even wanted to talk about a much more annoying aspect of this, which is to take it way down. So nevermind brains, nevermind even cells. My question also extends to that transition from you got a pendulum, you got a thermostat. And you can build these things up and then eventually at some point you get to a cell. I'm still curious about whether this impenetrability goes all the way down so we can't read the mind of a pendulum either, or there is some sense of progressive opacity as you climb this continuum of cognition from extremely simple systems where I think the conventional story is it's all third person accessible. We know exactly what's going on, but at some point you don't. I'm curious, when does that happen? And if we think there's a phase transition here or if we think this is smooth, I tend to think everything is more or less smooth in these cases, but maybe I'm wrong. That's one thing I wanted to probe a little bit: how does this play out when you don't start at the brain, where we can all agree it's very opaque, but what about from the most simple physical systems? How do you get to that opacity? On the flip side, something that's very interesting, I think an implication of this is that we can't really know, and we're all inferring. If you take the approach that I took in this memories paper, where your future self has to make a lot of guesses as to what your memories mean, because they were written down by your past self, and you don't have all the metadata, you have to now interpret these compressed n-grams. That leads to a more disturbing question: we can't even tell what we used to think. We can guess, but we don't know if this is the case. If that impenetrability holds, then it's there with respect to our past self and our past memories too. So that's wild. I don't know what we all have to say about those.
[53:31] Chris Fields: I'll refer to this wonderful thing called the Conway–Kochen theorem. Conway of the Game of Life and Kochen of quantum contextuality. It was published three decades ago. They proved, using mainly relativity theory, that if they considered a generic observational scenario and the observer has free will in the following sense: that what the observer does is not completely determined by that observer's past light cone. If the observer is not subject to local determinism, then the thing being observed is not subject to local determinism either. At the end of the paper they ask, "Do we really mean that electrons have free will in the same sense that observers do?" The answer is yes; they say so emphatically in their paper.
[54:43] Alexander Ororbia: If you did on the free will theorem.
[54:47] Chris Fields: And if you drag quantum theory into the picture, then you get an equally strong result that any system has to be able to effectively choose its own semantics for how it interprets whatever incoming information is. And if you take that choice away, then you get entanglement. The system ceases to have a separate identity. I think the answer, the in principle answer about opacity is that it goes all the way down.
[55:28] Karl Friston: Can I follow up on that and refer to Chris's inner screen hypothesis and the notion of an irreducible Markov blanket? If you've got a system that has no internal Markov blankets, has no deep structure or hierarchical structure or heterarchical structure, then that is where the hard no would apply or the hard yes of unknowability. But if it has internal Markov blankets, you can peel away and invasively or non-invasively get within that. What you're talking about is that vague gradation of things that are noble and unknowable. It's just the hierarchical, the depth of the Markov blankets of the Markov blankets. There is a kernel of either an irreducible Markov blanket, which you can never get into because you change the thing itself. There's also a limiting case from the point of view of classical physics, which is when the inner, intrinsic internal states are the empty set. So things like inner particles or stones don't have internal states, they just have Markov boundary states. You're absolutely right to think of this as a gradation, that there are inert things that are defined operationally in the sense that their internal states are the empty set. You could also say that there are sessile things that don't have active states. They are just complete, all of their states of this kind of particle are just sensory states, they're just inputs. Inputs that can also influence the outside. There's no restriction that sensory states have to not influence the outside. They still can be observable. Then you get to things that now have a non-empty active sector of the holographic screen or the Markov blanket. These are things that move. These are natural kinds that have mobility or motility. Then you get to things whose internal states now have a Markov blanket within them. These would be the kinds of things that can plan. These would be usually multicellular things, certainly compartmentalized things. At each stage, the knowability depends upon whether either the internal sets are empty or not, or they are irreducible in the sense there are no Markov blankets within those. It's a simple mathematical picture of that gradation that speaks exactly to the electron through to the pendulum, to the thermostat through to a smart thermostat that starts to worry about whether you want it warmer or colder or not and starts to plan ahead and moves from homeostasis to allostasis. All of this would speak to different scales, equipping Markov blankets and inducing a deep structure.
