Thanks for this module, Roguedjack.
I've played only one custom battle, but can offer some feedback.
I played Ancient Britons against Romans, AI level 3, northern european wooded, medium sized battle.
The Romans, it turned out, had a flanking force, which could have hampered the AI in the early stages of the battle. I had a good advantage on my right flank, with many slingers, chariots and light horse, to swing around to the back lines and get some free rout percentage by taking out two artillery, and a cavalry unit (see below).
The AI initially used its skirmishers well - but then when they retreated, the majority of the AI skirmishers spent the rest of their turns milling about in a wood in the back lines, frequently "retreating", and so were hors de combat - which was an advantage for me, as my skirmishers were left unmolested to roam around the back-lines creating mischief. I don't recall the base game AI having so many skirmishers idle, so this could be something to look at.
The AI used one unit of Theuropherei, on my left flank, very well. They were held back and then committed late in the battle, launching a series of flanking attacks upon pre-existing melees to break and (almost) roll up my left flank, although this was helped by my my left flank anchor, my best warband, having pursued a routing roman unit off into the distance.
The AI early on allowed its sole unit of cavalry to be detained by my light horse. They did not charge me, but sat immobile and were decimated by missile fire, (my slingers joined the party) and eventually they auto-broke. Later on the AI made effective use of its flanking force cavalry by catching and routing one of my chariots and some light horse.
I took a narrow victory, Romans 67% vs me 56%. The game was very close, at first I thought I had it easily won, then it looked like I would lose, until a few romans finally auto-broke. I felt that the AI was close to reversing the result, had its skirmishers been better handled, there seems to be some hesitancy or test that keeps being triggered, causing them to be stuck away from the fight. On the whole it handled other infantry well.
I have different thoughts about the potential for wargame AI. Having been involved with chess engine development, I understand the limitations of search trees and exponentially expanding permutations of future moves and results. But just as in chess, positional evaluation in wargame AI is important. If, in an ideal world, the same effort that Chess AI has enjoyed were to be spent on wargame AI, we would see real improvement. Obviously this won't happen, and I agree that the most improvement can be had by picking the low-hanging fruit - that is, optimising the current move AI as best we can. Thanks for your work