BEHAVE
When the Behave routine is called, the first act is to set a series of variables that I call A/C variables that are tied to a series of internal state concepts. The concepts are these:
1) Power (battery level)
2) Duration (how long have I been running since I left the power dock).
3) Distance to Power (how far away is the power dock)
4) Distance to Nearest (just because I can)
5) Known (how big is the world map)
6) Sensor Doubt (I don’t believe my eyes).

For each of these items, there are two variables, one for anxiety and one for confidence (hence the A/C). The reason for this is that there needs to be competing pressures. If decreasing power ALWAYS increased the propensity of a certain result, then that reduces the flexibility of the platform as changes will always be in a certain direction. There needs to be a force pushing back, as well. This actually shows up in biological systems at even the cellular level. Consider that DNA is transcribed into mRNA which codes for the amino acids in proteins. The DNA is just pumping out the mRNA, which goes out to the ribosomes to be translated into proteins. However, there are also factors in the cell that are actively degrading the mRNA. Therefore, if the amplification of the signal is not sufficiently high, then the attenuation of the signal keeps it from getting through, and no proteins are created. I have added the same kind of amplification and attenuation by having the competing anxiety and confidence variables for each of those internal state items, though I ought to also add that I have generally moved away from dictating the direction of the impact of either anxiety or confidence. The result is that there are two independent vectors that can impact a decision based on each state item, but they do not necessarily have to oppose each other. If evolution directs it, there are cases where they might act in concert.

The setting of the A/C variables is entirely driven by the genome. What factors go into setting either the A or the C variable for any item is determined by genes, as is the impact on any particular item on the level of the variable. I’m trying to remove my impact on what factors the bot evaluates, and to what extent, though I have probably not gone far enough, and will need to revisit this code before I show it anywhere.

Once those A/C variables have been set, the Behave routine goes through a tree of behaviors. The first check is to see whether the bot is afraid. This is based on the state of several of the A/C variables (not all of them, though), as well as what happened last time the Behave routine was called, what caused this call, and a few other things. If the bot is afraid, then it will either call the Panic behavior, or the Nervous behavior (because panicking is not always a good idea). If there is no fear, then the next check is to see whether or not the bot is hungry (fear trumps hunger). If not hungry, the next check is to see whether the bot is concerned about how far it is from the power dock. This behavior can be quite complex, as the bot might do as little as continuing to map, but re-prioritizing the list of targets that the OL gives it to favor targets in the direction of the power dock, or it could simply stop mapping and travel towards the dock. Lastly, if none of the other behaviors is chosen, then the bot will go about mapping, which could be quite a complicated task, and I haven’t even begun to write it. If anybody has any other behaviors that ought to fit into that ladder, I’d be happy to contemplate them.

One thing to note is that a variety of A/C variables are used to determine a level when checking for each of the behaviors. The level is then compared against a threshold (which is also in the genome). I only included the A/C variables that seemed even plausibly connected to the behavior. Had I included all of the A/C variables in each decision, and let the GA determine which were important, then this would closely resemble a NN implementation. However, I can’t quite bring myself to do this. I suspect that the GA will learn a bit faster if clearly irrelevant inputs are removed. On the other hand, the function to determine each behavior becomes the same (I’ll post one later) if all A/C variables are included, which quickly becomes a more universal solution.

So that’s the overview of how the high brain works. First is setting a bunch of competing or complementing variables covering how the bot feels about its place in the world. These variables primarily drive the decision as to the gross behavior category that the bot takes on, and within that gross behavior category there can be multiple actual behaviors. Every step of the way, the genome is driving the decision making. Genes drive how the A/C variables are set. Genes drive how (and even which) A/C variables go into choosing a behavior, and what impact they have. And finally, genes drive how the actual behavior is carried out. For that reason, with none of the actual behaviors written yet, the genome is already above 2100 genes, and will probably end up around 4000.

So how can I train this thing? I certainly can’t create a random population of genomes, feed each one to the bot and let it run. That would take forever, and wouldn’t work well anyways. Is the training needed to teach the bot to seek objects and gaps the same training as that which is needed to teach it not to let its batteries run too low? Or the same as that which is needed to teach it to avoid (or seek out) large heat sources (people)? It is very unlikely that the training for one objective would be the same as the training for most (if any) others. The solution that I came up with is to have the robot dream.

ROBOT DREAMS
There will be one module called the Dream Master. Each computer will have a module called the Dream Meister (DM). When the DM is ready to run, it calls out to the Dream Master, which responds with a dream. A dream consists of a virtual map and a scenario. It can be as simple as “My left track is slipping a little”, or “I’m in a big unknown space and there is a scary monster chasing me”, or as complex as “I’m in a complex world with no known power dock, fairly high power, and there appears to be a problem with my left track.” So there are dreams large and small, the first one would train the response to track slippage, the next would train the fear response, while the last would train all sorts of different things. I can think up any number of different dreams, and each will be stylized and written to the database, such that any DM can revisit a past dream for further training. In theory, it might be possible to randomize the dreams and have them auto generate, but I certainly don’t want to start out there, as I suspect that training to any dream will be reasonably slow, so I want to be specific about which dreams exist to maximize efficiency.

When the DM gets its dream, it checks for the number of processor cores it has available, then creates a number of Dream Controller (DC) objects based on the number of cores (each DC will operate on its own thread). Each DC object hosts a Self objects kind of like the Matrix. The DC provides the virtual world that the Self is immersed in. When the Self requests a sensor read, the DC determines what the read should be (unless the dream scenario requires that some sensors don’t work). If the Self moves in a certain direction, the DC determines what sensor reads will come in, if any, and where the Self ends up after the move (therefore, the DC acts as a virtual Dead Reckoning module). Originally, I intended the DC to operate in real time, which meant that it might be reasonable to have several DC objects per processor core, but I have since realized that I can greatly speed up time for the dreams, which changes that calculation. Since the Self is alerted either by a sensor read, or the sweep timer, the DC can take the current motion of the Self and calculate which will be the next item to alert the Self, and trigger it immediately without waiting. All it will have to do is alter the duration that the Self is seeing so that the Self thinks a second or two has passed when it really hasn’t. By dilating time for the dreams in this fashion, the learning will go MUCH faster, but this also means that the DC objects will not be lingering much, if any, so I think the calculation of the number of DC objects created by the DM will start out at (2 * cores) – 1. This means that the quad core systems can run 7 simultaneous DC threads, each of which will be training against the same dream. However, if you looked at the thread on GA systems posted earlier, you can see that I like to use a population of 100, and run for 100 generations, and STRONGLY feel that two levels of evolution are needed, which means 10,100 total generations are needed for full training. In this case, I think I will drop the population size and generation count, but dreams, especially complex ones, will probably be fairly costly.