Developing Widelands's (strategic game) AI with genetic / evolutionary algorithm - Part 2

Developing Widelands's (strategic game) AI with genetic / evolutionary algorithm - Part 2

This part II. will cover following topics:

* New inheritance logic
* New type of neurons
* Automated testing
* Some statistics

New inheritance logic

After some thinking current generation of new DNA of AI players look like this:

There are 4 distinct DNA sets in the code, and new AI player takes genes randomly from all of them and performs mutation with probability about 1/15 afterwards.

Moreover each of 4 input DNAs are linked to test map, I picked the maps so that they are different and covers various conditions (limited space, snow, green land). These four maps are used for testing and DNA of best AI player from particular map is copied to the source code and used for generation of new players in next iteration of test. For more info see Automated Testing section

Consideration:
Is random inheritance from 4 parents evenly the best approach?

New types of neurons:

I felt a need for more complicated neurons, that have two inputs and output is based on its combination, so I came up with "bi_neurons", that receives two bools and with probability of 1/4 returns "weight", otherwise 0. Idea is f.e. scenario:
- if my strength is growing 
- and enemy's strength is decreasing
-> return some value
(otherwise return 0)

Keep it mind, the AI is to be blackbox and I am not to say which of possible combinations:

• me up, enemy down
• me up, enemy up
• me down, enemy up
• me down, enemy down

should return a value and what value it should be. This is the purpose of "evolution" to figure it out. The value is than used as a part of score when deciding about attacking the enemy.

Automated testing

After some struggling with widelands ability to launch "unattended" games, I come up with this setup:

• I have 4 scripts, and each one runs games in loop - using the same map (4 maps are used for testing), writes logs to files (file names are unique, of  course)
• I was not able to redirect it to Xvfb, so the games pops up on screen, so I decided to run test while I am not on PC (at night f.e.)
• Lua scripts are terminating a every game after some gametime (2 - 4 hours, depending on map size), that in real time means 10 - 30 minutes of game
• duration of "test session" is about 4 hours, so I am getting 10 - 30 games per map during "session"
• All 4 scripts are running concurently (that means 4 games concurently)
• I have python script that goes over acquired logs and pick player with higher score and prints its DNA, so I can easily copy it to cc file and recompile

Some statistics

As I probably mentioned, I am using score to evaluate "fitness" of AI player and it is also used to find out if best AI player is better that best AI player from previous session. Usually it is so and with each iteration best score goes up by 10% (evaluated per map).
I have to say that spread of scores per map in a session is very big, f.e. on map Elven Forests I right now have these results (from 27 games from last test session):

Winner score:   1814
Average score:  910
Lowest score:   223

For this stage of development the variance is no issue, but for final version AI will have to provide more consistent performance.


Part I.