Robot Dreams

[Shaggy Hiker]

This thread is not so much a question as a means to start a conversation about an AI that I am working on. I’m looking for comments and suggestions on several different parts (any part of this, really), and hopefully it will also give others ideas for their own projects. The posting is lengthy, and more parts, including code, and outlines will be added at various points. Much of the initial part is the background for how I got to this point, it can be skipped, if you choose.

HISTORY
I started working on a project about 15 years back that was to be a simulation of a lake seen from the point of view of a predator swimming in the lake (predators are prey until they reach a certain size). The behavior of all the other organisms in the lake would be based on a complex genetic algorithm (GA) that would determine just three things: Direction, speed, and duration. I wasn’t coding in any higher level behavior. The organism wouldn’t really know why it was acting, nor would the observer. The organism wasn’t told to feed, nor was it told that it SHOULD feed. Instead, the genome determined how it felt about the state of its stomach, which might or might not push the organism into feeding behavior. However, there wasn’t any specific feeding behavior, as the organism would start off not knowing what to eat, how to eat, or even that eating was a solution to the problem of an empty stomach.

When writing the code to decide the behavior (which was just the direction to swim, speed, and duration), I went to great lengths to put every decision and the magnitude of any response into the genome, with the result that the genome was well over 1000 genes, with each gene being a bit. The thresholds at which responses happened, and the magnitude of the response, were both in the genome, though which factors were considered at any one time was not in the genome…generally. Along the way I wrote a couple GA based programs that were used to build parts of the main program, as well as being spinoffs of the main program, and in doing so, I learned a few things about how many generations they should run and ways to improve the performance. Most of those notes can be seen in this thread:

http://www.vbforums.com/showthread.php?t=553187

I was in the process of revamping the behavior in a way that would push the genome to over 2000 genes when somebody stole the computer that had the current code on it, and when I went back to look at my notes, I couldn’t figure out what I had been working on. That was only a minor disappointment, though, because I was also becoming aware that I would never be able to finish the project. For the project to have worked, it required a 3D interface, and I totally suck at art. There are dozens of books out there telling people how to do transformations, rotations, and all like that. It’s just math, so it isn’t all that hard. The problem I had is that I am so bad at art that though I could understand HOW to rotate the scene, I couldn’t tell whether I SHOULD rotate the scene. I couldn’t even do a horrid job without extensive trial and error, and a good job was out of the question. The other problem was that any estimate I had about the rate of evolution suggested that the top predators might not learn how to hunt in less than months of continual runtime. Would anybody be interested in an evolving simulation? Perhaps. How about if you had to leave it running for months to see any notable changes? Not so much. On the other hand, I had enough experience with GA based programs that I felt fairly certain that the concept would work, in which case the predators might learn their job so effectively that the AI would be better than human intelligence. The human controlled fish would be no match for the AI fish, despite the two having the same physical system, seeing the same things, and living in the same world. That would be a worthy goal in itself. However, with the theft of the code, I faced the fact that I could never complete the UI, and might as well turn my attention elsewhere. I had a neat AI concept, but needed a program that was complex enough to make use of it, yet had a simple enough interface that I could create it with my seriously deficient art skills.

CURRENT SITUATION
One thing I considered was a chess playing program, but I quickly discovered that only bad and novice players use intelligence to play the game, good players do not. What would be the point of creating an artificial form of a bad chess player? After that, I hit upon the more practical idea of building a robot. As a field, Robotics is roughly at the same point that computing was at when Steve and Steve created the first Apple…or maybe a bit earlier than that. There are a handful of hobby platforms, not much standardization, and only a handful of sensors that are both within the price range of hobbyists, and are effective (and many of those are only marginally so). One of the results of this is that the bulk of robots being built by hobbyists are intended to run only for a few minutes, and do only one, often trivial, task. It’s kind of like the early hobby computing where even a half-assed calculator was a significant achievement for a hobbyist.

I wanted something that would run indefinitely, which meant that I needed a relatively robust platform (to hold the heavier batteries needed for lengthier runtimes), and low power components. I also wanted a complex brain capability. If the complex brain was on the robot, that would increase power consumption, which would require bigger batteries (or seriously shorter runtimes), and a larger footprint, none of which was desirable. Therefore, I decided to move the processing off of the robot itself. The robot in my design is nothing more than a platform to move around a handful of sensors, with just enough processing power to be able to communicate the sensor reads out to a separate computer via Bluetooth. As a side note, one of the results of this decision is that there is no reason to only have a single robot. As long as the brain is moved to my home network, and off of the hardware, then there can be any number of bots wandering around doing either the same, or different things, all controlled by the same brain, up to the limit of the bandwidth clutter on the Bluetooth frequencies. In effect, the robot is the house, not the mobile hardware. However, for now, there is just a single bot, which is an exploration and mapping bot.

[Shaggy Hiker]

LEARNING SYSTEMS
There are two major categories of learning systems, though they would appear to overlap considerably. The two categories are Neural Nets (NN) and GAs. The difference is parallel learning versus serial learning. A GA requires a population of competing individual cases, where each one is tested in turn, and the next generation is derived from selective combination of the best performers. On the face of it, that’s not a very practical application when it comes to robotics. Even if I could afford to have dozens of bots running around, I probably would have problems communicating with all of them, and the learning would take forever (during which time it would be like herding cats, as all the different bots were moving nearly randomly as they learned). A neural net, on the other hand, is generally a serial system. Once the network has been set up, it can learn over time using feedback. A good site for learning about NN systems is here:

http://www.ai-junkie.com/ann/evolved/nnt1.html

However, one thing to note about NN systems is that they tend to result in a fairly static, deterministic algorithm. In fact, a fully trained NN might be written as a single mathematical equation, albeit a gawdawful one. Any plasticity left in the process would arise only from there being such a large variety of inputs that a situation may not be identical to any previous example. Another item about this rigidity is that an NN appears not to take into account the previous state of the system, which is not the case in behavior, where the way we felt a moment ago impacts the way we feel now. Therefore, a net as simple and direct as the ones in the previous link were not of any value for robotic learning, where I wanted a more plastic learning system.

DESIGN
So now that I have ruled out the two major techniques, I will describe what I did, and show how I used both of them. At the low level of the brain, the hardware of the bot communicates via Bluetooth with the Brainstem module, which sends sensor read requests to the hardware, and listens for messages from the hardware. It also packages those messages and sends them out using the UDP class found in post #7 of this thread:

http://www.vbforums.com/showthread.php?t=509619

There are several other modules spread around the computers, all of which are deterministic, have no current AI component, and serve as the mid-brain. Some examples are the Object Librarian (OL) that classifies detections into points, and points into objects to build up the world map (which is a SQL Server Express DB running on a different computer). The OL might also try to infer information about objects, such as whether they are likely to move, or can be moved, which could mean that there is the potential for an AI application concerning pattern recognition of the data in the world map, but I’m not going there at this time. Another example would be the Dead Reckoning module (DR), which will try to keep track of where the bot is. Since it can only see by sonar, and can otherwise only track revolutions of its wheels and the compass heading, the DR is acting something like the navigation of a submarine, where only occasional locations can be positively identified, and some means of discriminating errant readings needs to be worked out. That portion isn’t written yet, either. There are other modules in theory, or partially constructed, that do other deterministic things, as well.

Above all this sits the High Brain, which is a single Self class. There are two events that will trigger the Self to call the Behave routine. The first event would be a sensor reading, and the second event would be the tick of a timer. I added the timer because there will be times when no sensors are being read, and the bot will need to occasionally reconsider its course of action (or inaction). The timer has an interval of about 2s, at this time, but that may prove to be too fast. Ultimately, the High Brain needs to determine a direction, and send commands to the brain stem to tell the bot how fast to move each track, or which sensor to read. Those are the only possible outputs. There is no duration, currently, so unlike the fish model, there is only direction and speed, which manifests in the speed of each track. This decision is the result of the Behave routine.

[Shaggy Hiker]

BEHAVE
When the Behave routine is called, the first act is to set a series of variables that I call A/C variables that are tied to a series of internal state concepts. The concepts are these:
1) Power (battery level)
2) Duration (how long have I been running since I left the power dock).
3) Distance to Power (how far away is the power dock)
4) Distance to Nearest (just because I can)
5) Known (how big is the world map)
6) Sensor Doubt (I don’t believe my eyes).

For each of these items, there are two variables, one for anxiety and one for confidence (hence the A/C). The reason for this is that there needs to be competing pressures. If decreasing power ALWAYS increased the propensity of a certain result, then that reduces the flexibility of the platform as changes will always be in a certain direction. There needs to be a force pushing back, as well. This actually shows up in biological systems at even the cellular level. Consider that DNA is transcribed into mRNA which codes for the amino acids in proteins. The DNA is just pumping out the mRNA, which goes out to the ribosomes to be translated into proteins. However, there are also factors in the cell that are actively degrading the mRNA. Therefore, if the amplification of the signal is not sufficiently high, then the attenuation of the signal keeps it from getting through, and no proteins are created. I have added the same kind of amplification and attenuation by having the competing anxiety and confidence variables for each of those internal state items, though I ought to also add that I have generally moved away from dictating the direction of the impact of either anxiety or confidence. The result is that there are two independent vectors that can impact a decision based on each state item, but they do not necessarily have to oppose each other. If evolution directs it, there are cases where they might act in concert.

The setting of the A/C variables is entirely driven by the genome. What factors go into setting either the A or the C variable for any item is determined by genes, as is the impact on any particular item on the level of the variable. I’m trying to remove my impact on what factors the bot evaluates, and to what extent, though I have probably not gone far enough, and will need to revisit this code before I show it anywhere.

Once those A/C variables have been set, the Behave routine goes through a tree of behaviors. The first check is to see whether the bot is afraid. This is based on the state of several of the A/C variables (not all of them, though), as well as what happened last time the Behave routine was called, what caused this call, and a few other things. If the bot is afraid, then it will either call the Panic behavior, or the Nervous behavior (because panicking is not always a good idea). If there is no fear, then the next check is to see whether or not the bot is hungry (fear trumps hunger). If not hungry, the next check is to see whether the bot is concerned about how far it is from the power dock. This behavior can be quite complex, as the bot might do as little as continuing to map, but re-prioritizing the list of targets that the OL gives it to favor targets in the direction of the power dock, or it could simply stop mapping and travel towards the dock. Lastly, if none of the other behaviors is chosen, then the bot will go about mapping, which could be quite a complicated task, and I haven’t even begun to write it. If anybody has any other behaviors that ought to fit into that ladder, I’d be happy to contemplate them.

One thing to note is that a variety of A/C variables are used to determine a level when checking for each of the behaviors. The level is then compared against a threshold (which is also in the genome). I only included the A/C variables that seemed even plausibly connected to the behavior. Had I included all of the A/C variables in each decision, and let the GA determine which were important, then this would closely resemble a NN implementation. However, I can’t quite bring myself to do this. I suspect that the GA will learn a bit faster if clearly irrelevant inputs are removed. On the other hand, the function to determine each behavior becomes the same (I’ll post one later) if all A/C variables are included, which quickly becomes a more universal solution.

So that’s the overview of how the high brain works. First is setting a bunch of competing or complementing variables covering how the bot feels about its place in the world. These variables primarily drive the decision as to the gross behavior category that the bot takes on, and within that gross behavior category there can be multiple actual behaviors. Every step of the way, the genome is driving the decision making. Genes drive how the A/C variables are set. Genes drive how (and even which) A/C variables go into choosing a behavior, and what impact they have. And finally, genes drive how the actual behavior is carried out. For that reason, with none of the actual behaviors written yet, the genome is already above 2100 genes, and will probably end up around 4000.

So how can I train this thing? I certainly can’t create a random population of genomes, feed each one to the bot and let it run. That would take forever, and wouldn’t work well anyways. Is the training needed to teach the bot to seek objects and gaps the same training as that which is needed to teach it not to let its batteries run too low? Or the same as that which is needed to teach it to avoid (or seek out) large heat sources (people)? It is very unlikely that the training for one objective would be the same as the training for most (if any) others. The solution that I came up with is to have the robot dream.

ROBOT DREAMS
There will be one module called the Dream Master. Each computer will have a module called the Dream Meister (DM). When the DM is ready to run, it calls out to the Dream Master, which responds with a dream. A dream consists of a virtual map and a scenario. It can be as simple as “My left track is slipping a little”, or “I’m in a big unknown space and there is a scary monster chasing me”, or as complex as “I’m in a complex world with no known power dock, fairly high power, and there appears to be a problem with my left track.” So there are dreams large and small, the first one would train the response to track slippage, the next would train the fear response, while the last would train all sorts of different things. I can think up any number of different dreams, and each will be stylized and written to the database, such that any DM can revisit a past dream for further training. In theory, it might be possible to randomize the dreams and have them auto generate, but I certainly don’t want to start out there, as I suspect that training to any dream will be reasonably slow, so I want to be specific about which dreams exist to maximize efficiency.

When the DM gets its dream, it checks for the number of processor cores it has available, then creates a number of Dream Controller (DC) objects based on the number of cores (each DC will operate on its own thread). Each DC object hosts a Self objects kind of like the Matrix. The DC provides the virtual world that the Self is immersed in. When the Self requests a sensor read, the DC determines what the read should be (unless the dream scenario requires that some sensors don’t work). If the Self moves in a certain direction, the DC determines what sensor reads will come in, if any, and where the Self ends up after the move (therefore, the DC acts as a virtual Dead Reckoning module). Originally, I intended the DC to operate in real time, which meant that it might be reasonable to have several DC objects per processor core, but I have since realized that I can greatly speed up time for the dreams, which changes that calculation. Since the Self is alerted either by a sensor read, or the sweep timer, the DC can take the current motion of the Self and calculate which will be the next item to alert the Self, and trigger it immediately without waiting. All it will have to do is alter the duration that the Self is seeing so that the Self thinks a second or two has passed when it really hasn’t. By dilating time for the dreams in this fashion, the learning will go MUCH faster, but this also means that the DC objects will not be lingering much, if any, so I think the calculation of the number of DC objects created by the DM will start out at (2 * cores) – 1. This means that the quad core systems can run 7 simultaneous DC threads, each of which will be training against the same dream. However, if you looked at the thread on GA systems posted earlier, you can see that I like to use a population of 100, and run for 100 generations, and STRONGLY feel that two levels of evolution are needed, which means 10,100 total generations are needed for full training. In this case, I think I will drop the population size and generation count, but dreams, especially complex ones, will probably be fairly costly.

[Shaggy Hiker]

However, there is an interesting result of all these dreams. I was intending that dreams would only evolve certain subsets of the genome, and that will still be possible. Each dream will include a mask that will hold some genes inviolate, while allowing the others to evolve, but after further thought, this is not a common scenario. Therefore, what genomes should the DM be using for any given dream? Should it be starting with random genomes each time? Initially, they will have to. However, once I have a bunch of winning genomes, I can form a pool of these winners in the database, along with rankings of how well they have done against various different dreams. Therefore, a DM can be instructed by the Dream Master to take the winners pool and run it against a certain dream, with the winner being added to the winners pool, possibly replacing one of the weaker performers. This means that, over time, the winners pool will get better and better at solving a variety of problems.

This also leaves open the possibility that the genome being used by the real Self instance (the one that is driving the actual bot), could switch dynamically. If a genome in the winners pool is really good at certain situations, but weak at others, it could be swapped in under some conditions, and out for other situations. This would either lead to a very plastic behavioral model, or robotic schizophrenia. It also would require some kind of expert system to decide which genome to be using. Kind of a meta-AI.

[Shaggy Hiker]

Here's an example of one of the functions to determine a certain behavior (Hunger in this case).

Private Function CheckForHunger() As Boolean
        Dim thresh As Integer = CInt(gList(183)) << 8 Or gList(182)
        Dim zero2OneCurve = Function(arg As Integer, deflator As Integer) (1 / (1 + Math.E ^ (-arg / deflator)))

        If (gList(176) And 128) > 0 Then
            thresh = -thresh
        End If

        thresh += EvaluateImpact(aPower, 184)
        thresh += EvaluateImpact(cPower, 188)
        thresh += EvaluateImpact(cDistanceToPower, 192)
        thresh += EvaluateImpact(aDistanceToPower, 196)
        'Some interactions
        thresh += EvaluateImpact(CInt(zero2OneCurve(aDistanceToPower - gList(200), 201) * aPower), 202)
        thresh += EvaluateImpact(CInt(zero2OneCurve(gList(206) - cDistanceToPower, 207) * cPower), 208)
        thresh += EvaluateImpact(CInt(zero2OneCurve(gList(212) - cDistanceToPower, 213) * aPower), 214)
        thresh += EvaluateImpact(CInt(zero2OneCurve(aDistanceToPower - gList(218), 219) * cPower), 220)

        If thresh >= (CInt(gList(225)) << 8 Or gList(224)) Then
            Return True
        Else
            Return False
        End If

    End Function

The code starts by building the starting value of the threshold out of the genome, which is just a byte array, so a bit of manipulation is needed to put together integers from it. The starting threshold can be above or below zero, so whether the value is negative or not is part of the genome. Note that this threshold (which is missnamed) is compared against a level in the genome at the end of the method. Since that level is always positive, a starting threshold that starts out seriously negative would take lots of input to bring it up above that level.

Each of the A/C variables that I thought applied is added to the threshold using the following method that requires the genes used to be four sequential bytes starting at the gOffset point in the genome:

 Private Function EvaluateImpact(ByVal input As Integer, ByVal gOffset As Integer) As Integer
        Dim temp As Integer = 0
        Dim one2oneCurve = Function(arg As Integer, deflator As Integer) (1 / (1 + Math.E ^ (-arg / deflator)) - 0.5) * 2

        'Is it even used.
        If (gList(gOffset) And 1) > 0 Then
            'Is it inverted?
            If (gList(gOffset) And 2) > 0 Then
                temp = CInt(one2oneCurve(gList(gOffset + 1) - input, gList(gOffset + 2)) * gList(gOffset + 3))
            Else
                temp = -CInt(one2oneCurve(gList(gOffset + 1) - input, gList(gOffset + 2)) * gList(gOffset + 3))
            End If

            'Now for the smoothing.
            If temp > 0 Then
                If (gList(gOffset) And 4) > 0 Then
                    temp = 0
                End If
            Else
                If (gList(gOffset) And 8) > 0 Then
                    temp = 0
                End If
            End If
        End If

        Return temp
    End Function

The OneToOneCurve is a lambda that is totally unnecessary, since it is only used once, and could have been just written out, but when I originally wrote this, the lambda was used in over a dozen places, so it was justified. The Lambda takes arg, and converts it into a value from -1 to 1. If arg is negative, then the output will be negative. The deflator value will impact how quickly the curve approaches -1 or 1. The larger the deflator, the less quickly the curve approaches the endpoints. This means that the smaller the deflator, the smaller the impact of a change in arg, while the smaller the deflator, the larger the impact in a change in arg.

The way the lambda function is used is that the A/C variable (which is always a byte, but that is irrelevant) is subtracted from a threshold gene in the genome. This means that there is a level at which the A/C variable has no impact, and the further it gets from that level, the stronger the impact (whether negative or positive), as the A/C variable is converted to a range from -1 to 1 around the level. That converted range is then multiplied by a different gene (byte) that holds the impact of the A/C.

A couple other genes are used to modify the result. One gene will cause the A/C variable not to be used at all. A different gene will invert the result. A third gene will adjust the shape of the result to remove all negative numbers, and a fourth gene will adjust the shape of the result to remove all positive numbers. By adding these genes, the range of possible impacts of any one A/C variable can be wildly flexible. Giving the GA a very large search space tends to improve the quality of the outcome, in my experience.

[Shaggy Hiker]

A class to handle a population of genomes (which will come later) that are effectively byte arrays where each bit is a gene (as opposed to GAs based on classes):

<Serializable()> Class Population
    'Genetic members.
    Private popList As List(Of Genome)
    Private sortPopList As List(Of Genome)
    Private mPopSize As Integer
    Private mMutationRate As Integer 'Mutation chance per thousand genes.

    'Other
    Private mRand As New Random
    Private mBest As Integer
    Private mNextGenome As Integer
    Private mReadyToEvolve As Boolean

#Region "Constructors"

    'This constructor creates a population of new genomes.
    Public Sub New(ByVal pSize As Integer, ByVal geneCount As Integer)
        popList = New List(Of Genome)
        sortPopList = New List(Of Genome)
        mPopSize = pSize
        For x As Integer = 1 To mPopSize
            popList.Add(New Genome(geneCount, mRand))
            sortPopList.Add(popList.Item(popList.Count - 1))
        Next
        mNextGenome = 0
        mReadyToEvolve = False
    End Sub

    'A constructor to build a Population from a save.
    Public Sub New(ByVal populationList As List(Of Genome))
        popList = populationList
        sortPopList = New List(Of Genome)
        mPopSize = popList.Count

        mNextGenome = -1
        For x = 0 To popList.Count - 1
            If popList(x).Quality = -1 Then
                mNextGenome = x
            End If
            sortPopList.Add(popList(x))
        Next

        If mNextGenome = -1 Then
            mReadyToEvolve = True
        End If
    End Sub

#End Region


#Region "Properties"

    'This property should only be used for saving the state.
    Public ReadOnly Property WholePop() As List(Of Genome)
        Get
            Return popList
        End Get
    End Property

    Public ReadOnly Property GetGenome(ByVal index As Integer) As Genome
        Get
            If index >= 0 AndAlso index < popList.Count Then
                Return popList.Item(index)
            Else
                Return Nothing
            End If
        End Get
    End Property

    Public ReadOnly Property BestOrdinal() As Integer
        Get
            SetBest()
            Return mBest
        End Get
    End Property

    Public ReadOnly Property GetNextGenome() As Genome
        Get
            If mNextGenome = -1 Then
                Return Nothing
            Else
                Return popList(mNextGenome)
            End If
        End Get
    End Property

    Public ReadOnly Property GetNextOrdinal() As Integer
        Get
            Return mNextGenome
        End Get
    End Property

    Public ReadOnly Property ReadyToEvolve() As Boolean
        Get
            Return mReadyToEvolve
        End Get
    End Property

    Public ReadOnly Property PopSize() As Integer
        Get
            Return popList.Count
        End Get
    End Property

    Public ReadOnly Property RankNOrdinal(ByVal rank As Integer) As Genome
        Get
            If rank > 0 AndAlso rank <= sortPopList.Count Then
                sortPopList.Sort()
                Return sortPopList(rank)
            Else
                Return Nothing
            End If
        End Get
    End Property

#End Region

#Region "Methods"

    Public Sub AdvanceCount()
        mNextGenome += 1
        If mNextGenome = popList.Count Then
            mNextGenome = -1
            mReadyToEvolve = True
        End If
    End Sub

    Public Sub DoEvolve()
        Dim x As Integer
        Dim p1 As Integer = -1
        Dim p2 As Integer = -1
        Dim y As Integer
        Dim newPop As New List(Of Genome)

        popList.Sort()
        sortPopList.Clear()
        For x = 0 To popList.Count - 1

            Do
                Dim t As Integer = mRand.Next(0, 2 * (mPopSize + 1))
                Dim accum As Integer

                'Need to do this 1 based for the math to work out.
                For y = popList.Count To 1 Step -1
                    accum += y
                    If t < y Then
                        If p1 = -1 Then
                            'If P1 is not set, then this is it.
                            p1 = y
                            Exit For
                        ElseIf y <> p1 Then
                            'If this parent is not the same as P1, then this is it (if p2 were set, the loop would have exited).
                            p2 = y
                            Exit For
                        Else
                            'p1=p2, draw again. Odds of this are 10% at best for this use.
                            Exit For
                        End If
                    End If
                Next
            Loop While p1 = -1 OrElse p2 = -1

            'Now there are two parents. Create the next genome.
            newPop.Add(New Genome(popList(p1), popList(p2), mMutationRate, mRand))
            sortPopList.Add(newPop.Item(newPop.Count - 1))
        Next

        popList = newPop
        mNextGenome = 0
        mReadyToEvolve = False
    End Sub

    Private Sub SetBest()
        Dim tQ As Integer = -1

        If popList Is Nothing Then
            Return
        End If

        For x As Integer = 0 To popList.Count - 1
            If popList(x).Quality > tQ Then
                mBest = x
                tQ = popList(x).Quality
            End If
        Next
    End Sub

#End Region

End Class

[Shaggy Hiker]

The genome itself. Normally, sex is handled at the population level, but in this case, with a byte array, it was a bit more elegant to handle it at the genome level.

<Serializable()> Public Class Genome
    Implements IComparable

    Private gList As List(Of Byte)
    Private mQuality As Integer

    'This constructor creates a new, random, genome of the size supplied as an argument.
    Public Sub New(ByVal gSize As Integer, ByVal rand As Random)
        Dim buff(gSize - 1) As Byte

        gList = New List(Of Byte)
        rand.NextBytes(buff)

        gList.AddRange(buff)
        mQuality = -1
    End Sub

    'This constructor builds a new genome out of two parents with a mutation rate of mutationRate/1000. This will handle tenths of a percent
    'mutation.
    Public Sub New(ByVal parent1 As Genome, ByVal parent2 As Genome, ByVal mutationRate As Integer, ByVal rand As Random)
        Dim x, y As Integer
        Dim xOver As Integer
        Dim parentPointer As Integer = 0 'Start with parent 1
        Dim bArray(1) As Byte
        Dim gBuild As Byte
        Dim mask As Byte = 1
        Dim temp As Integer

        'Walk through the genome bytes.
        For x = 0 To parent1.GeneCount - 1
            'Put the bytes somewhere convenient.
            bArray(0) = parent1.GeneByte(x)
            bArray(1) = parent2.GeneByte(x)
            'walk through the bits in the byte.
            For y = 0 To 7
                'Get a random check for crossover.
                temp = rand.Next(1, 101)
                If temp < xOver Then
                    'Crossover, swap the parent byte pointed to.
                    If parentPointer = 1 Then
                        parentPointer = 0
                    Else
                        parentPointer = 1
                    End If
                    xOver = 0 'A crossover happened, set the potential back to 0.
                Else
                    xOver += 5 '5% increase in crossover potential per gene.
                End If

                'Now check for mutation
                If rand.Next(0, 1000) < mutationRate Then
                    'Mutation was signaled.
                    'XOR negates the entire parent byte. Only the one bit will be needed, but by XORing the whole byte, it doesn't matter which bit is needed.
                    gBuild = (gBuild Or ((bArray(parentPointer) Xor CByte(&HFF)) And (mask << y)))
                Else
                    'At this time, we have the correct parent gene being pointed to by the parentPointer. Just get the gene.
                    gBuild = (gBuild Or (bArray(parentPointer) And (mask << y)))
                End If

            Next

            'The new byte has been built, add it to the list.
            gList.Add(gBuild)
            'And clear it for the next round.
            gBuild = 0

        Next

        mQuality = -1
    End Sub

#Region "Properties"

    Public Property GeneByte(ByVal index As Integer) As Byte
        Get
            If index < 0 OrElse index >= gList.Count Then
                Return 0
            Else
                Return gList(index)
            End If
        End Get
        Set(ByVal value As Byte)
            If index >= 0 AndAlso index < gList.Count Then
                gList(index) = value
            End If
        End Set
    End Property

    Public ReadOnly Property GeneCount() As Integer
        Get
            Return gList.Count
        End Get
    End Property

    Public Property Quality() As Integer
        Get
            Return mQuality
        End Get
        Set(ByVal value As Integer)
            mQuality = value
        End Set
    End Property

    Public ReadOnly Property GeneList() As List(Of Byte)
        Get
            Return gList
        End Get
    End Property

#End Region

    Public Function CompareTo(ByVal obj As Object) As Integer Implements System.IComparable.CompareTo
        If TypeOf obj Is Genome Then
            Return mQuality.CompareTo(CType(obj, Genome).Quality)
        End If
    End Function

End Class

[Shaggy Hiker]

This thread is kind of an archive of various ideas I've tinkered with on this project, which is fine with me. Hopefully they will be of interest to somebody.

I'm currently working on a recursive semi-neural net designer engine. I looked into neural nets, since I was creating something that was clearly similar, but I wasn't all that thrilled with what I found. For one thing, I don't like the neuron mechanism that seems most common. It's fine for brains, but is it fine for computers? I'm not convinced that it is. Therefore, I created my own variation on a neuron.

The other thing that I didn't like about neural net design was the idea of an input layer, hidden layer, output layer. Unless I am misunderstanding what people are doing there, I feel that the layers are much more vague than that. Therefore, while out skiing one day, I thought up a different design that appeals to me more, but is way too intricate, so I am writing a visual designer.

The idea is that a series of inputs, which can be either binary or analog (a continuous range of values), feed into a structure called a Neural Path (NP). Each input into an NP will be treated in a different way. An analog input (such as the power level of a robot) would be run through a neuron that would transform the input. In effect, this transformation is equivalent to answering the question, "what is the impact of this input as it pertains to this NP?" A binary input would be either ignored or accepted (I can't think what else could be done with it), and if accepted, it would add in a value of X, where X is part of the genome, and is therefore a heritable trait. This would mean that the impact of a binary input on the current NP would be X or 0. Interestingly, another input into an NP could be the output of a different NP, and that child NP could use the same binary and analog inputs as the parent NP, but would have the freedom to evaluate them differently...or not, depending on how the brain is designed.

Therefore, the NP are recursive classes. Each NP holds a List (of NP), and each NP also has a list of execution steps it must perform to evaluate input, which includes having all child NPs evaluate, as needed. At least, that's the way the design will work. Doing this would be inefficient, though, so the visual brain designer will allow the user to design NPs by dragging inputs into the design screen, but once the design is completed, all the child NPs will go away, and the execution plan will just be a series of calls to a function, many of which could technically be done in parallel (though that would gain me nothing, since the function would be nothing but CPU intensive math).

So, that's the basic idea I'm working on. However, I just went to a talk about hybridization, and got thinking about genetic algorithms, in general. In every GA I have ever written, and every GA I have ever seen, the genomes are evaluated, ranked, then some routine creates the next generation by some kind of selective mating such that higher ranked genomes have better chances of mating. Exactly how that routine works varies, but it is also completely unnatural. No species is monotonic in choosing a mate. It is to the great benefit of the vast majority of us that everybody has different selection criteria when choosing a mate. If one guy is selecting primarily based on brains, while the next guy is selecting primarily based on....another b word, the two would evaluate any person differently. I know of no GA that does this, largely by design. Every choice is ranked on just one axis, though that axis could be made up of the sum of the contribution of a variety of factors.

Therefore, I am considering changing the GA around to incorporate multi-axis selection, and am pondering how to do this. In my case, the robot will certainly have multiple axis that it could evaluate on. For example, it could favor a cautious approach to power usage, or a greater curiosity (measured by objects found or new area explored), or a greater activity (measured by ground covered), and so on. All of these would be axis of evaluation with each candidate falling somewhere along each axis. Of course, this would require an active agent performing the mating. Parent A would be the active parent, and would have a selection matrix. It would evaluate all members of the population other than itself, and choose based on how they fit the selection matrix it had. Naturally, the selection matrix would also be part of the genome such that the selection matrix of the offspring would be a blend of the matrixes used by the parents.

This design has some problems, though. For one thing, if the population is 100, does this mean that everybody gets to mate? Do they all have to mate with an unmated partner? Doing that would halve the population and probably put an end to selection. A better option would be that every member of the population would be able to mate once, and produce one offspring, and the only person they could not mate with would be themselves. This won't work if the selection matrix is simplistic. For example, if each individual favored exactly one axis, then the best individual on that axis would mate with everybody who favored that axis, and nobody else. This would mean that one individual would become the parent of a large number of the next generation. That's a sure recipe for inbreeding.

Two variations, both of which should probably be adopted, would be these: The selection matrix should be rich, and each individual should have a decreasing chance of mating. The former would mean that the selection matrix would not choose an axis to favor, but would weight each axis differently. The choice individual might be the one that was high on one axis, low on another, and middle on a third, because the selection matrix favored the first somewhat, ignored the second, and favored the third a LOT. The second factor would be that the favorability of mating with any individual would decline based on the number of offspring that individual had already fathered.

Anyways, those are my current thoughts. If anybody has any suggestions, I'd like to hear them.

[Peter Porter]

Hey, Shaggy. I know your thread is old, but I was wondering how your AI project is coming along. A few years back I thought about writiing a simple AI, starting out as a chatbot, but got distracted by alot of life changing events.

Anyway, I was gonna start out programming a way to identify sentence types. Once I had that down, I was gonna write an algorithm to disect these sentences, and sort their information in it's memory nodes for future reference.

It was a simple idea to get started, but over the past 8 years I haven't had a chance to test it.

[Peter Porter]

I just discovered some helpful articles by Hannes DuPreez on CodeGuru, who also provides code as well as zipped projects.

The first article is setting up a neural network in VB:
https://www.codeguru.com/columns/vb/...ic-and-ai.html

His next article is to train a neural network to distinguish between right and wrong:
https://www.codeguru.com/columns/vb/...algorithm.html

I haven't tested his code, and I'm sure you're way past all of the above, but it might be a starting point for me.

Also gonna reread everything you've written and linked to.

[Shaggy Hiker]

I abandoned this project in favor of one that is more likely to have value. AI has kind of left the hobbyist behind, and I had something equally interesting, and more likely to be beneficial, to work on.

[sapator]

And what's stopping me from believing that Moti beat you to it since his excellent book was released in full in 2011?!

(Sorry for spamming, it was itching me for hours now and I succumbed )

[Shaggy Hiker]

Yeah, well, now I'm making glass, based on his excellent tutorial. If you haven't read that chapter, it is pretty easy reading.