Watch Episode Here
Listen to Episode Here
Show Notes
This is a 30 minute talk on Hopfield networks solving propositional satisfiability problems, by N. Weber, W. Koch, O. Erdem, and T. Froese (from the Okinawa Institute of Science and Technology in Japan), followed by a 30 minute discussion of potential applicability of this technique to linking the beliefs and preferences of an agent to a controller that can be used to implement them behaviorally.
CHAPTERS:
(00:02) Embodied Cognition Overview
(02:35) Hopfield Networks And SAT
(24:05) Agents Navigating Need Space
(29:16) Internal Maps And Decoding
(33:30) Logical Sentences As Dynamics
(37:35) Training Minimal Virtual Minds
(41:04) Linking Logic To Embodiment
(45:25) Weighing Drives And Stability
PRODUCED BY:
SOCIAL LINKS:
Podcast Website: https://thoughtforms-life.aipodcast.ing
YouTube: https://www.youtube.com/channel/UC3pVafx6EZqXVI2V_Efu2uw
Apple Podcasts: https://podcasts.apple.com/us/podcast/thoughtforms-life/id1805908099
Spotify: https://open.spotify.com/show/7JCmtoeH53neYyZeOZ6ym5
Twitter: https://x.com/drmichaellevin
Blog: https://thoughtforms.life
The Levin Lab: https://drmichaellevin.org
Transcript
This transcript is automatically generated; we strive for accuracy, but errors in wording or speaker identification may occur. Please verify key details when needed.
[00:02] Mark James: I'm Mark James. I'm a philosopher in cognitive science. Tesh and I are both part of the Embodied Cognitive Science Unit. We're situated in the Okinawa Institute of Science and Technology in Japan. Our PI in the unit is Tom Froese. The unit was set up in the tradition of someone like Francisco Varela, intended to be a scientific inquiry into a generally non-representational approach to cognition. The guiding model of our unit is the idea that interaction matters. That's interaction between brain, body, and environment, but also the individual person and the various scales that comprise them. We have projects on the individual sense of agency and sensory substitution. We have projects looking at dyadic interaction, real-time reciprocal interaction face-to-face or in digitally mediated environments. We have people looking at collectives, criticality in swarms of bees or in teams. We also have archaeologists looking at the emergence of social complexity and even astrobiologists looking at resilience at a planetary scale. At least part of the ambition is to acknowledge the synthesis across these scales and how all these things interact, which is ongoing work. Our unit PI himself tends to be focused on questions related to the interaction of mind and matter, so questions around consciousness. Personally, I'm interested in how all these interactions can be leveraged in a practical sense to serve the redirection of our behavior towards health, broadly speaking. Tesh is one of the people working on the models that we find helpful in all of this to think about how these systems learn and evolve together. That's the notion of self-optimization. I'm happy to jump back in for any questions, but I'll let Tesh take it over from here.
[02:35] Natalia Weber: Thank you, Mark. Tasha or Natalia officially, just a bit about my background, I'm originally trained as a chemist and I did theoretical chemistry for my masters. Then during my PhD in OIST, I found an opportunity to switch to the field of cognitive science and I got very lucky in that I could combine the skills I had with a topic I find very interesting. The work that I will present you today is based on this paper we published in December. In it, we show that we can leverage the power of associative memory of Hopfield neural networks to solve complex logical problems, specifically propositional satisfiability problems. This paper is fairly technical, but it ties neatly with what I want to do in my own research project, which is related to creativity. Because with this computational model, we get to explore how dynamical systems can break constraints of a problem that it is supposed to solve. Constraints are integral to creativity. Michael, you're familiar with this model, but in the interest of being self-contained, I will give an introduction to Hopfield networks and then satisfiability problems. I'll show how we can color maps with it. Hopfield neural networks, named after John Hopfield, are a form of recurrent network, meaning that there are bidirectional connections between all the nodes of the network. To describe the dynamics of the system, we can use a vector of states S, where each node can be described either as a discrete or continuous value. In this work, we use a bipolar value of -1 and 1. We update the system asynchronously, meaning that at each step of the simulation, only one node is randomly picked and updated. The output of that node is a function of a weighted sum of all the inputs from all the nodes in the network. In this specific work, we use a simple step threshold function. With this state update, the network will start to rearrange itself depending on the sign and the value of the weight W_ij between any two nodes. S_i might switch sign or S_j might switch sign. These rearrangements of signs will result in a different state of the system. We can then evaluate the difference between these two states using the so-called energy function described by this mathematical formula, which reflects the misalignment of states within the network. The minus sign indicates that the energy decreases as the network moves towards a stable state. At some point when the energy does not change anymore, we know that the system reached a stable state or an attractor. Hopfield neural networks are recurrent neural networks that converge to a stable state. This convergence to a stable state can be used in several ways. First, we can find the solution to a complex combinatorial problem. For example, the traveling salesman problem. This is a problem where you are given a list of cities and need to find the shortest route visiting all the cities exactly once and returning to the starting point. Hopfield and Tank show that if the connections of the Hopfield neural network, the weight matrix W, correspond to the constraints of this problem, then the natural dynamics of the system is to converge to a stable state that will correspond to a locally optimal solution to that problem, meaning one of the possible routes. Another way we can use this convergence to a stable state of Hopfield networks is to recall patterns, for example. This is what Hopfield did in his original work. He showed that we can take some patterns, for example images, and use Hebbian learning to store them in the network's connections. This is a learning rule often summarized as "cells that fire together, wire together." If the system is then introduced to a partial pattern, a corrupted version of the stored memory, it will act as an associative memory and converge to a state representing that stored memory. What I mentioned briefly before is that the usual dynamics of the Hopfield network is to converge to a local minimum. This often results in a sub-optimal solution. In fact, in that traveling salesman problem paper, Hopfield and Tank mentioned that the system will only find the shortest path 50% of the time. This problem, however, can be resolved if we use the associative memory capability of the Hopfield network.
[06:52] Natalia Weber: They showed that if we take some abstract combinatorial optimization problem and the constraints of that problem are represented in the weights, and we also update the weights using Hebbian learning with a certain learning rate alpha, meaning that we imprint on the weight matrix every state it visits, and we reset the system to some random initial state every time it converges to a local minimum, then the system can reach a global minimum, which potentially means the best solution. This model, which we address as the self-optimization model in our group, was shown to be general enough to model various complex adaptive systems, from gene regulation networks to self-age agents and socio-political networks, to name a few. If I'm not mistaken, in the recent paper with you, Michael, you are using a slight modification of this model to show transition in agency. However, all of this research focuses mainly on abstract problems. Abstract problems are very important for providing understanding of the general behavior of the process. But when the weight matrix doesn't represent a concrete set of constraints, it is hard to analyze what exactly learning does to the system. This is what we wanted to investigate in this work. We show that the model can also solve concrete problems, specifically the satisfiability problems or SAT problems. By having concrete problems, we can then provide an insight into the relationship between learning and breaking of the problem constraints. Just before I jump to the results, I want to give a quick overview of satisfiability problems. Satisfiability problems, or SAT in short, are questions in logic where you need to determine if there is a way to assign truth values to variables such that a certain propositional formula is true. This has immense applications since many important real-world problems in different scientific fields can be expressed as MAX-KSAT, which means maximum satisfiability problems with a maximum of K literals per clause. I'll explain what that means in a bit. This includes anything from planning problems in engineering and verifying software in computer science to solving bioinformatics problems. To give an example of a SAT problem, let's look at the Liar's problem. Imagine we have a room with four people, Alice, Bob, Cal, and Dan. Some or all of them make statements. Alice says, Dan is a liar. Dan says, Bob is a truth-teller. Cal says, Bob is a liar. Our goal, given those statements, is to find out who is a truth-teller and who is a liar. These facts can be represented by a state vector S, with the fact that the i-th person is a liar or truth-teller represented by the state si, where if si equals 1, it denotes that person i is a truth-teller and si equals 0 denotes that person being a liar. From these statements, using these representations, we can build logical relations, and from these relations we can get our propositional formula. If somebody doesn't have a background in satisfiability, I recommend this link below as an initial tutorial. I initially didn't have a background in satisfiability, so this was very useful to me. In this propositional formula, each term in the brackets is essentially a clause. There are two literals per clause and they're separated by an operator. The operator inside the first term is an or, or disjunction, and the operators connecting each clause are and, or conjunctions. This is a formula of 6 clauses, a conjunction of disjunctions. Each clause has two literals. So this is a MAX-2-SAT problem, with only two literals per clause. This will be relevant in a bit. Now that we have this propositional formula, the next thing we would do is put it into a SAT solver.
[11:09] Natalia Weber: There is a plethora of SAT solvers out there. There's a competition on SAT solvers every year. What the SAT solver does is output two possible outputs. Either the proposition is satisfiable, and then it will provide us with an assignment of the values that we're interested in — in this case, Alice and Kyle are liars, Bob and Dan are truth-tellers — or it will output unsatisfiable, meaning given those relations we cannot satisfy this propositional formula. In this work, we're interested in whether we can take this propositional formula and use this SOL model such that if there is a solution, the system will converge to a stable state with energy equal to 0, and each node will represent our assignment. If there is no solution, the state will be at higher energy, where each unit of energy represents how many clauses were not satisfied. What we needed is a way of mapping this SAT problem to the Hopfield neural network. Fortunately, this mapping was already performed in the 90s, first by Gadi Pinkins and then followed by Juan Abdul, and his method is the one we implement here. In short, given the description of the problem by the propositional formula, this method allows us to describe the formula as an energy expression. We can compare this expression term by term with the expression of the energy of the Hopfield network. This gives us a weight matrix that describes the SAT problem. For this specific problem, a Max-SAT problem, the liars problem. Similarly to when the weights describe the constraints of the traveling salesman problem and the dynamics give us the shortest path, by using this method we can use any SAT problem and the dynamics of the system will give us the solution. Using this model, we can look again at the liars problem. We took a bigger example: 50 people with 34 statements. Person one says person 44 is a liar, and so on. In plot A is how this weight matrix initially looks that represents those 34 statements. Then we see the solutions: there are essentially three distributions of energies. From zero to 1,000, the dark blue is regular Hopfield dynamics, no learning. This shows what happens if we just run the simulation for this weight matrix on the Hopfield dynamics, let it converge, and then reset it again. The system explores all possible solutions. Sometimes it finds a correct solution. Sometimes it converges to a stable state of energy 4, meaning there are four clauses that were not satisfied. The red part after 1,000 is when we turn on the learning; this is the SOL model. After a certain amount of time, the entire system converges to a global solution, the correct one with energy equal to 0, meaning it managed to find a state that satisfies all the constraints. Once we turn off the learning (light blue line), even though we continue resetting the system to random initial states, it stays on the global solution. This is how the weight matrix looks at the end of the simulation. This is a system that augments its behavior with associative memory of its own attractors and finds a global solution, hence optimization. This simple example shows that the SOL model can solve SAT problems and that learning improves the chance of arriving at a global solution. However, a more interesting scenario to examine is what happens when there is no solution to the problem.
[15:27] Natalia Weber: For this, we use the map coloring problem. Here we have the map of South America and the goal is to cover all regions by distinct colors such that no two bordering regions have the same color. We only want to cover this with two colors. This is not a solvable problem. You need four colors for this specific map. We find this example interesting and I'll explain why. First of all, we represent the state of the map by a matrix. Here S is a matrix where SIJ equals 1 denotes that region I has color J. This is a bit more complex notation, but it allows us to generalize the problem to cases when we want to use more than two colors. This is why we chose this notation. This problem can be represented by these three requirements. Each region must be colored. A region cannot be two distinct colors at the same time, because with this notation we can have a state where the same region has different colors, which doesn't make sense. This is a requirement that we need to have. The third requirement is that regions that share a border should have a different color. This is how we represent it logic-wise. It's a bit more complex than the liar's problem. These are three different sets of constraints. Our propositional formula would be basically a conjunction of those three sets. What we did is we also added weights to the sets of clauses, to these constraints. These are our preferences, and I'll explain what it means. If we don't add any weights, the three sets of constraints all have the same weight. What you see here is how the weight matrix of the map covering problem of South America would look. We might have a situation where a certain region doesn't have a proper color. In order to avoid it, we tried to put more weight on the first two sets of constraints, meaning that each region, first and foremost, must be a proper color. Another way is, instead of putting more weight on the first set of constraints, we can adjust the weight on the third set of constraints by the border length between two countries. Adjacent regions are less forced to be covered differently if they have a shorter border. For example, Peru and Chile share a very short border, so if they're covered the same color, it's not that bad. However, if we cover Chile and Argentina in the same color, since they share such a long border, that would be more problematic. This is what this third set of weights means. Here are the results. Plot A and plot B show how the weight metrics representing the problem look before and after the simulation. D shows the final product of the simulation, the converged state.
[19:44] Natalia Weber: And this, blue, dark blue and light blue are proper colors, and orange is non-color. It's a bit hard to visualize what is non-color. This is how we visualize it. We see that there are three regions that are not a proper color, and the energy for that is E3. That's correlated. There are three clauses that were not satisfied. However, in this situation, there's no point talking about borders between countries, because when you have non-covers, this is why we did these additional weights. In the second scenario, where we put weights on making sure that all the regions have proper cover, everything covered properly as anticipated. What we can see is that, if you squeeze your eyes, there are 8 borders that were violated between the different regions, and the corresponding energy is 8. And the third one with the borders one that we tried out is even worse energy-wise, but we wanted to try it out and see what it gets us. Just to have a short summary right now: so far I was discussing the results from a satisfiability angle. We saw that the SO model can solve SAT problems and we examined how we can manipulate our preferences through different solutions when we have this scenario where there is no correct solution. I would like to switch to a particularly interesting result, and that is what we see here in the last plot in the corner. If we zoom on it, we can see the region of Costa Rica in this specific run has the same color as Panama. Since nothing is attached to Costa Rica in this map, in principle, it should not cost anything to the system to flip the color of Costa Rica to be distinct from the one Panama has. However, this is not the case. This example shows how heavy learning can dramatically modify the weight matrix that encodes the problem of the constraints, such that some of these constraints are no longer contained within it. In the unmodified weight matrix, the cover of Costa Rica can be trivially changed to satisfy the constraint on having opposite colors across borders. However, a state with that cover flipped is no longer an energetic minimum of the learned weight matrix. We find it interesting because this would never happen in the traditional SAT solver. Of course, this is not a good solution, but this is obvious since this problem has no solution. The point is to investigate how breaking of constraints happens through learning. That is hard to do in an abstract problem because you can't understand what a particular state means in relation to the original weight matrix. With concrete problems like this, we can explore domains that were previously inaccessible. The next step would be to investigate different problems and see how we can use this knowledge to explore different types of domains. To summarize again, some limitations and future work: the SO model is much slower than the state-of-the-art SAT solver. We were not trying to beat the SAT solvers. The SO model has some potential for parallelization and it can be tweaked to work faster, but we find that it might be interesting for engineers interested in biological possibility, since it is much more biologically realistic than a SAT solver. Another thing is that the SL model—we did Max 2-SAT problems, and the SL model cannot, in principle, handle SAT instances with more than two literals per clause. That's what you would think. However, we know that any K-SAT problem can be reduced to three-SAT problem and any three-SAT problem can be reduced to Max 2-SAT problem in both cases at the expense of a linear number of new variables. This means that, in principle, any K-SAT problem can be reduced to Max 2-SAT problem and then the SL model can be used to solve it. We will have many more hidden variables. About future work on this specific topic: now that we know we have these kinds of connections between initial constraints and what happens to them after learning, we can ask what the system learned. For example, if we look at the learned weight matrix, can we convert back to the logic and see what clauses were eliminated? That's one idea for future work. Finally, I would like to acknowledge my co-authors, Werner Koch and Ozan Erdem, of course Tom. And this is our lovely unit. Thank you for listening.
[24:05] Michael Levin: Thanks so much. That's super interesting. I have a couple of thoughts. The first thing is, what occurs to me is that the initial set of requirements, the logical statements that you're modeling, you could think of as a set of beliefs or preferences by an active agent. And what then you're doing is you're producing a network that can act as a controller for that agent that would navigate some space consistent with its beliefs or preferences. I think it's a very interesting map because you're starting, and I can tell you some related things that we're doing, but you're moving from a statement of what the system either believes to be true or would like to be true. And now you've got an actual bio-realistic controller that seeks to implement it. By satisfying it, what you're really doing is navigating a space to a region where the things that the agent is looking for are in fact true. Does that make sense to you or have I misunderstood? It seems like this is an agent navigation task.
[25:34] Natalia Weber: I love the way you think about it as an agent navigation task. So far my angle on this was always more technical. I always look for what this agent represents, but I would love to try it out. How would you describe the problem?
[25:58] Michael Levin: When I say agent, this could be a cell or an autonomous vehicle or whatever it's going to be. I am in a world and I have multiple constraints or multiple drives or needs, and your goal is to have all of them met. Sometimes it's impossible to have them all met. My drive is that I want, in your initial case, this one is a liar, that one's not a liar. I want to be in a region of my space where this thing is true, and this thing is also true, and this other thing is not true. I want to not be hungry, I want to reproduce, and I want to be out of the danger zone in my environment. Those are three things you want. There are many areas where you can have one of the three, but not the other two, or two of the three, and maybe if you're lucky, there's a region of the space where all of those three things can be met; that's where you would like to go. This sounds to me like a typical problem of drive satisfaction for an agent that has a multi-dimensional set of things that it wants to maximize or optimize. That's the thing you want. Now the question is, how do you get there? You've got this landscape. How do you find the place that's going to meet all your requirements? It sounds like you've got a mechanism here, which is actually a biorealistic controller of how you navigate that landscape. What I love about this is that one way to think about the spectrum of agency is to really think about what the world looks like from the perspective of that agent. If you're dealing with a bowling ball on a landscape, all you need to know is your vision, as a third-party observer, of the landscape. That tells you the whole story of what's going to happen. If you've got a mouse on the landscape, your view of the landscape doesn't matter that much. What you want to know is the mouse's internal representation of that landscape: where has he been rewarded, punished, what are the valences. Because you're not going to be able to predict his motion from your view of the landscape. You need to know what his view is. What I see here is a really interesting formalism for specifying what are the drives and the goals and the preferences of an organism. You're basically producing a vision of the internal map of that organism of how it feels about that space and where it's going to be happiest. Where it's going to be happiest is where all the constraints are met, but at least having some of them met is better than none. That's super cool, and now you've shown a map of the landscape and the internal, first-person perspective—the agent's view of the landscape—and an actual controller, an actionable biorealistic controller that it can use to get where it needs to go. I think that's amazing, and there are so many applications of this; it's super cool.
[29:16] Natalia Weber: Awesome. Sounds cool. Thank you. I like the way the vision of internal map. Speculating a little bit about what would be the internal map of mice is a bit too much. My goal initially was to understand what you can manipulate in the environment such that you can satisfy different constraints. Then you can see how that can help. The actual map maybe doesn't even matter. What matters is we have this controller that we can modify.
[29:55] Michael Levin: Forget the mouse, but I think easily you could produce a minimal agent in a virtual world that has some needs that it would like, some variables that it would like to be in a certain level. You can either go with a true-false thing, or you could do fuzzy logic and say, I want my hunger level maximized, but this other thing, I want my temperature in between this and this. You have some statements of things you prefer, and use that Hopfield network as a controller and watch the agent navigate. You can watch it navigate its world. I think this is incredibly powerful for linking, because what you started with is an explicit description of what the agent might prefer. And from that, you got to a dynamical actual controller that might implement it. And that's really useful. I could think of all sorts of model systems where that would be useful. I'm also thinking in reverse, the thing you said at the end about trying to read the map. That sounds to me like a neural decoding task, where the idea is that neuroscientists want to scan the brain and from that recover the propositional information that's there. They want to read a bunch of electrophysiological signals and say, you remember having breakfast like this, you believe in this, you've never seen this, and this is your preference. They want to recover those things. This also, and we do this all the time, but we do it outside the brain. We look at an early embryonic tissue and we say, OK, what shape do you believe you need to be? Because those are the kinds of things when we recover these bioelectrical pre-patterns, we can recover goal states, anatomical goal states. To us, this is just a navigation of anatomical state space where we say I see where you're trying to get to: this particular region where you've got five fingers and this size. We try to do that decoding too: look at the bioelectrics and say, OK, so what do you think you are? You a zebrafish? Are you a frog tail? What do you know, what are you? I really like this example too, your model, because that's a simplified model where you can practice that and you can say, can we look at the controller and the time profile of the controller and extract from that the logical statements, basically read the mind of the creature. What do you want to be true? What are your goals?
[32:45] Natalia Weber: We cannot do that yet.
[32:50] Michael Levin: No, I understand. It's certainly in the neural context; it's also extremely hard. But this is a simplified version where we can test some techniques because there are AI tools that are coming online for inverting some of these things. It's an inverse problem, right? This is what you're trying to do. I think that's incredibly interesting. Do you know the work of Patrick Grimm on the dynamical systems view of logical statements?
[33:30] Natalia Weber: No, I do not, and I would be happy if you can forward a paper to him.
[33:34] Michael Levin: I'll send you; he has a number of papers. Briefly, it goes like this: Grimm is a philosopher in New York, and in the 90s he developed the following thing: Consider the simple liar paradox. You've got a sentence that says "this sentence is false." The reason it's a paradox is that if you insist on a single unchanging truth value, then you're stuck, because that doesn't work. However, Grimm did two things. One is he allowed it to have time, which means that you can view it as an oscillator, because as you go through the sentence it's true, but then it's false. You get the simple oscillator that goes up and down. He gave it time, which immediately removes the paradoxical nature of it, because now there's no problem: it becomes a dynamical system. The next thing he did was go to fuzzy logic and say that sentences can have any value from zero to 1. Now you can have groups of sentences that go like this. Sentence A might say, "I am 80% as true as sentence B is false." And sentence B might say, "I am 30% true." There are two sentences and they refer to each other. Now that you view them as dynamical systems, you can plot them. You can either plot them as a function of time and get time-dependent behavior, or use the extra dimension and make a static shape out of it. Now, depending on your sentences, some are fractal, some have weird attractors, some stabilize. Some groups of sentences stabilize, some don't; some have all kinds of attractors. It shows a connection between logical statements and propositional content and dynamical behavior. Getting behavior out of logic, getting spatial — the Grimm thing is a kind of spatial navigation in a truth space. I think that is extremely interesting. We're doing a bunch of work on that now, looking at different types of sentences. The liar paradox is a basic oscillator. A simple statement that has one true value just sits there. This thing oscillates. Other systems have all kinds of complex behaviors. Once it becomes a dynamical system, you can try to do something really crazy. I don't have any results on this yet, but as we've shown with models, dynamical systems models of gene regulatory networks are often trainable. They have six different kinds of memory that they can form from inputs, including Pavlovian conditioning; you can literally train these things. Biological networks are much more trainable than random networks. I think evolution actually likes that property. Now you can think about training groups of logical sentences. If that's true, you're not really training the sentences; you're training an extremely simple virtual mind that contains those sentences as its cognitive commitments. That's what you're training, even though you're neglecting the rest of it — we're not modeling the rest of the organism, just that. I need to think about this more, but I think that's one of the things you have here: an amazing connection between the propositional content of an agent's mind and a behavior, a dynamical system controller that guides its behavior in space. I think that's pretty wild.
[37:35] Natalia Weber: I like the way you phrase it. Maybe that will be the next paper.
[37:38] Michael Levin: Pretty wild. Yeah.
[37:45] Natalia Weber: I don't have a lot to add to what you said. That's a very nice way of thinking about it on a more meta level. I would love to think about it. If you're interested and we can do it together—the next problem that the agent will be solving and then try to implement in this model—I would be happy to do it.
[38:05] Michael Levin: I'd love to collaborate. We have a number of simple virtual systems that we use for making hybrids where we have neurons. One of the things we have is living cells that are instrumentized to inhabit a virtual world. By the signals they put out, that moves them around in a virtual environment, and then they get rewards and punishments. It's a brain and a vat living in this virtual environment. One thing we can do is if we work on that inverse problem that we talked about, maybe we could apply it and pull out the cognitive commitments of that organism. Imagine you have a set of cells, they learn to navigate an environment where certain regions are good or bad. Then we use the reverse of the technique you just showed to extract a set of sentences where the thing literally tells you, "I hate this region over here, I like this region here, and this is the region where all my needs are met. This is the region I like you." We might be able to extract those statements, very simplified, with just a couple dimensions, three dimensions or something.
[39:25] Natalia Weber: I think I understand what you would like to do. If you look at the weight matrices at the end, they're very simple. They're like rank one. So lots of information is gone. It's great. I wonder how would that need to be modified if you want to remember more than one statement, maybe it needs to get a bit more complex.
[39:50] Michael Levin: There's also the issue of, of course, there's going to be more than one. Is that true? I think that's true. There's going to be more than one set of statements that are consistent with any given matrix. So we're not recovering one unique mental snapshot that belongs to each of these things, where this guy may be the simplest one, which is OK. That's when we do theory of mind and agent-agent interaction. Again, we never know exactly what anybody's thinking, but you say, what's the simplest model of my conversation partner that explains what's going on here? So I think that's fine. I think that's perfectly reasonable. That's interesting. Let me think about this some more. This is great. I'll come up with a couple of simple scenarios for things we can actually model in this way.
[40:54] Natalia Weber: Sounds fantastic. Thank you. Yeah.
[40:57] Michael Levin: Awesome.
Natalia Weber: Thank you so much.
[40:58] Michael Levin: Any other thoughts from Remedy? Werner, Mark, anything? Anything you guys want to add?
[41:04] Mark James: Just reflect on what you're saying. It's something we're looking at in a different domain at the moment from a behavioral design perspective and trying to say propositional content is maybe one dimension, but you also have the motivations and the sets of capacities and skills from a more embodied perspective, and then the affordances of the environment. So ultimately, if we can get to the point where we're modeling the interaction between all of those elements, that would be fantastic. But simple is probably a good idea.
[41:38] Michael Levin: One way to add that: I've been thinking about this for our case, because if you have a group of sentences that refer to each other in the kind of grim diagrams that I was talking about, that's all well and good, and you get a shape, but if they only refer to each other, they're not tethered to any kind of real world. They're self-contained; they only refer to each other. But you can imagine having atoms in there that do refer to a real world, or a virtual world, for their embodiment. You can say sentence A could be, I am 50% as true as sentence B times however true sensor one is. Sensor one is some aspect of the—it's reading some aspect of the outside world. Now that system is not just the self-contained logical system that cycles in and of itself, but it's actually linked to the outside world. You can imagine multiple agent interactions where here's a group of sentences, A through C, and here's one C through F, and they have C in common, which means there's a kind of chemistry when they come together. They have something in common and they might end up being incompatible or they might be compatible, which knocks them into a particular region of the space. You can imagine multi-agent interactions and you can absolutely tether them to the real world by including variables in your sentences that correspond to some kind of affordance in the real world. I think you could...
[43:10] Mark James: Is there something else that seems to be interesting there? What you're describing really does map to the kind of phenomenology of the development of behavior. We have a bunch of commitments and then we're trying to work out where the congruences between them are. We often fail to do that. As human beings, we get stuck at times and we don't know where the congruence is. That leads to ambivalence and ultimately at times to disorder. So maybe there's the possibility here to start: I have all these sets of things and there is some tension and maybe it seems incompatible. Can we have a model that helps us understand how you resolve those tensions?
[43:58] Werner Koch: I have one comment on what you said earlier about an agent and trying to satisfy the desires or needs. If you model that, a critical problem is how you weigh the different desires. They all have the same weight, and it's set up such that you cannot satisfy all constraints simultaneously. What sort of weights do you put in? Or ideally, you would want the system to have a way to determine that itself, because in the map coloring problem it's very difficult to have the different sets of constraints. You cannot use the same weights, because then, in a problem where no solution exists, it will most likely find a solution that doesn't even have proper colors, which is why you should have weights attached to that in one form or another. But it's not obvious how you would let the system decide that itself. If you have an agent with their desires trying to be met, you run into the same problem. How do you decide that? If you had some physical model that would lead to weights automatically, then that's an option. But if it's just the system in an abstract space, then it's not immediately obvious how you would do that. Ideally, you would have the system self-adjust.
[45:25] Michael Levin: No, it's a great question. I think I'm going to show this to Mark Solms because he's been thinking about these things a lot. He builds minimal agents with drives, with orthogonal drives. I think one way to do it would be to introduce a time aspect. So you're an agent and you have a bunch of drives. The first thing that's going to kick in is thirst. That kicks in before hunger. You're not really going to be worried about hunger if you're really thirsty. At some point, they both become critical. Some things have a gradient sharpness, like a slope where certain desires ramp up very quickly and others ramp up slowly. These weights might change. In a real organism, that's probably set to some extent anyway by evolution or by prior experience. Here, you could let it have control over those weights. For example, some of the values of your nodes in your network could feed back to tweak some of the weights. I can't even imagine what would happen if you did that, but that's easy enough to check. It's also...
[46:47] Werner Koch: That sounds like something that would easily become unstable. The Hopfield network is stable; it converges to some state. But if you have feedback from the state into the weight matrix directly, the Hebbian learning does that to a certain extent. That is feedback, but it's stable if the learning rate is low enough. If you start introducing direct feedback from the state into the weights, then I can see this spiraling out of control very quickly. Then you have the same problem on a meta level that you as the experimenter need to twist knobs on the algorithm to keep stability.
[47:30] Michael Levin: Maybe you as the algorithm, or maybe we have a second network, a metacognitive component. I could imagine ways to play with it. It may even be a psychological model where you have some incompatible drives that can't all be true at once. And then eventually you decide that this one thing is not worth it. It's killing all the other stuff. I really like that chocolate cake in the middle of the night, but it's messing up my other stuff. And so I'm going to decide that that's it. That one's out now. And so some ability to add that metacognitive loop where there's some ability to re-weight them based on how things are going. If your experience is that you could make 9 out of 10 happen. It's that 10th one that always — I'm not paying attention to that anymore. You could imagine in biology some of that works. You're never going to be able to cross things like thirst and so on off your list. But there are other things you can give up, and I bet there are experimental data, probably with rats, where after a while they decide that within this environment there's one thing they need to give up and then life gets better. It seems we could try, on top of the system you showed today, various kinds of changes that'll make that fly.
[48:58] Natalia Weber: That's a very simple model. There are many ways you can make it more complex and break our heads trying to make it stable.
[49:07] Michael Levin: Thank you so much. That was really interesting. Let me ponder it some more and we'll be in touch. I would be really interested in working together on this. It's great.
[49:21] Natalia Weber: Thank you so much. Nice talking to you.
