My journey of improving the AI

4X strategy game from Proxy Studios

Moderators: Pandora Moderators, Slitherine Core

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 523
Joined: Fri May 23, 2014 3:21 pm

My journey of improving the AI

Post by Xilmi »

I was curious if some people might want to know what exactly this voluntary-fan-developer-guy is doing with his privilege of having access to the source-code and what you can probably expect to come out of it.

So I decided to make this thread, where I talk about the things I'm experimenting with and will try to keep the intrigued reader informed.

Since talking about everything I've done so far would be a little lengthy, I'll, as of now, limit it to what I've done yesterday.

I had been experimenting with a different way of how tile-improvements are selected to be built and continued with that yesterday.
I redid a lot of what I tried before and the result was comparatively satisfying.
The AI now knows to build suburbs and pollution-processors and it values forrests higher, especially as Terra Salvum. If someone builds a farm now depends on their amount of farmers compared to total population.
There's still an issue with higher-yield-fields not having a higher priority for their top-yield being improved. This sometimes results in mines on food-resource-hills and habitats on the top of mountains.
But I believe that should be easy to fix by giving improvements a slightly higher value if they are on a field that already has a high amount of this improvement.

Since the hardest part of the game for players and AI alike, probably is managing an economical crisis, I put some effort into trying out crisis-management-techniques for the AI.
These include: Lowering Taxes, reducing pollution by using fewer factory-workers, an emergency-program for unhappy cities where it's either: build buildings or focus on research but don't just spam more troops. First get out of the crisis. For even more imergent cases, where a city is unhappy and has to put most of their population to farmers just to stay alive, I've allowed the AI to make use of razing.

Since some of the things for these emergency-plans affect population-management I also had another look at that. The hardest part about it, clearly is balancing out mines+workers vs. scientists. I changed how miners are assigned. Should be much more efficient now. However, it is hard to come up with a good algorithm to find a good balance between production and research. Ideally it should consider all sorts of stuff:
Need for military, unbuilt buildings/advancements, if there are techs left to research(ok, that's a no-brainer), how efficient the mining-sources are, what the pollution looks like.

So basically imagine: You have Population-Farmers, and now you determine how many of those you want to use as scientists. The rest will be used as miners and workers. (Luckily there's no need to find an algorithm to balance miners vs. workers, there is one in place that already is really, really, good)
This rest can be anywhere between "0" and "all".
You have access to all the data above. How would you go about it? Or just imagine you are the player and play the game (duh!) how do you determine it in this case?

My tests were quite pleasing. I let "very easy" AIs play against "very hard" aliens. They all survived the initial rush and the eclipse and built up their economy afterwards. However, they are technological behind to what you are supposed to have to face "very hard"-Messari-Invasion. This probably is mostly because my current production vs. science-algorithms are too much in favour of production... but then again this seemed necessary to stop the early aliens.
"Very easy" AI is best to test the crisis management since morale-boni of higher levels usually mean: There won't be any crisis for them to cope with.
Once very-easy-AI can deal with the Messari on "very-hard" alien-aggression, I'll be very confident.

Btw. the biggest mistake of the unaltered AI in crisis-management with aliens around is that they always try to replace their formers without making sure they can keep them alive. Just changing it to: "Require at least cities+1 military-units before making formers" already meant a dramatical improvement.
The second biggest was to apparently ignore Aliens in their counter-designs. Having them consider the aliens right away and thus using Flamethrowers meant another big leap.

My goals for this evening are:
Fix the ignorance about high-yield tiles when selecting improvements.
Finding a competent algorithm to evaluate production vs. science.
Fix a bug I must have made somewhere with building-selection. For an unknown reason there is some circumstance, that can make the evaluation go negative despite me thinking that all factors are positive. oO.

So I hope this was an interesting read for everyone interested. Please share your thoughts and let me know if you want to see more updates like that.
Hints about science vs. production management are much appreciated!
ForesterSOF
Private First Class - Opel Blitz
Private First Class - Opel Blitz
Posts: 1
Joined: Mon Jan 05, 2015 12:22 pm

Re: My journey of improving the AI

Post by ForesterSOF »

I all sound good. Keep up the good work.
Zak0r
Sergeant - Panzer IIC
Sergeant - Panzer IIC
Posts: 189
Joined: Sat Jan 25, 2014 1:21 pm

Re: My journey of improving the AI

Post by Zak0r »

Was it intentional to test the very easy AI? Very easy gets a malus and therefore probably isn't the ideal setup for such tests. Easy actually has the same handicap as the player.
Flash Jack
Lance Corporal - Panzer IA
Lance Corporal - Panzer IA
Posts: 19
Joined: Sat Oct 04, 2014 11:34 pm

Re: My journey of improving the AI

Post by Flash Jack »

Hi,

Full marks for having a go at this project.

I've enjoyed reading your update and if you feel like doing any more in a similar vein it'd be greatly appreciated.

Getting a 'behind the scenes' peek and an insight into how you're approaching the problems is very interesting.

Cheers,
Flash Jack
Sapiento
Lance Corporal - SdKfz 222
Lance Corporal - SdKfz 222
Posts: 20
Joined: Sat Nov 23, 2013 12:51 pm

Re: My journey of improving the AI

Post by Sapiento »

Good to hear that someone is still interested to improve this game. I still don't understand why the developers haven't put more effort into this. They could have turned a very good game into a great game.
Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 523
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi »

Thank you all for you interest in this little report.

Yesterday wasn't very productive. I basically fixed the two self-caused bugs I mentioned at the end of the last post and then went on to trying out approaches to the population-management.
None of them were overly satisfying. It either was too few science or too few production. Also Production vs. Mining was wacky. The way I did it, they eventually tended to build up qute a lot of "Overmins". Never felt about right.

But now I made a new concept and postet it in german in another forum. Will be home late today (about 22 cet) but I plan to realize and test the new approach anyways. I'm pretty confident about it. It also is partially based on research that Zak0r has done for me. (Playing himself, noting down values and then reporting those to me)
Zak0r wrote:Was it intentional to test the very easy AI? Very easy gets a malus and therefore probably isn't the ideal setup for such tests. Easy actually has the same handicap as the player.
I know that and most tests will probably be ran on easy for that reason. Especially those where I will actually play myself and not just observe the AI. However, since I specifically wanted to test crisis-management, I had to make sure that a crisis occurs. And due to the Morale-Malus on Very Easy this was clearly the best way to ensure it. Picking Togra as main observation-target enhanced this effect even further.
They were in a state that I consider crisis for a pretty long time. They also had to use "Raze City" several times.
They eventually got out of it but as I said: They couldn't cope with the "Very Hard" Messari-Invasion.
Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 523
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi »

Mixed feelings about yesterday.

The new population-management algorithm was a success!

It works like I think it should and results in a fluent tech-pace while neglecting neither military nor economical buildup.
I also could cut back some of the less well thought-out crisis-management-handling techniques.
The ones remaining are: Lowering taxes, prioritizing buildings/formers/colonizers over military and in emergencies still: razing down population

I've seen some minor issues with improvement-building. Like forest now seem to be slightly overvalued, especially by Terra Salvum. But I'm not even sure about that.

However, my otherwise cheerful mood got a big hit when the game crashed on me in the late-game of an interesting "Easy AI's" vs. "Very Hard" Aliens-Observer-Game.
I couldn't get the debugging-tools to work and thus am clueless about the reason of the bug. It's not one of these "usual" access-violations I caused and fixed before.

In the game itself:
Togra and Solar Dynasty have been wiped out by Imperium and Divine Ascension. Other than that everyone else kept peaceful throughout the game. Relations were "Generous" of everyone with everyone else between the remaining 5 factions.
Messari had been beaten since quite a while. I watched some impressive combined efforts of DA and Imperium to kill the dreaded motherships before they even reached any city.
The first factions were already getting their Black-Hole-Generators and surely were getting close to Science-Victory. Was hard to tell who was gonna make it. Everyone but Imperium was still competative. Terra Salvum had skipped the Aqua-Formers which made them fall behind a little aswell.
Most of the older cities were awesome powerhouses with highly positive morale and full of buildings.
So all looking quite well... except for the crash. :\

So todays agenda is isolating and fixing the damned crash-issue.
If this can be done with time left, I'll start tackling the last big issue: Tech-Picking order. I see some huge potential there. Combining good mechanisms with good decisions should eventually lead to good results.
Sawelios
Lance Corporal - Panzer IA
Lance Corporal - Panzer IA
Posts: 12
Joined: Mon Jan 05, 2015 12:02 pm

Re: My journey of improving the AI

Post by Sawelios »

Cool. By the way, if you're interested in bug-hunting, I can offer you a fat exemplar :). In my present game I have no possibility to build Colonizers in my city, but instead I have two identical options for building Formers... :)
Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 523
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi »

I've never seen anything like that. What OS are you using? Do you have a screenshot of that?
As of yet I mostly hunt "selfmade" bugs, since I know I have caused them, I also know where to look for them.

So far I've fixed 1 bug that was there before and like 20 that I first included myself. :o
SephiRok
Proxy Studios
Proxy Studios
Posts: 1024
Joined: Thu Jul 19, 2012 10:19 am
Contact:

Re: My journey of improving the AI

Post by SephiRok »

Sawelios wrote:Cool. By the way, if you're interested in bug-hunting, I can offer you a fat exemplar :). In my present game I have no possibility to build Colonizers in my city, but instead I have two identical options for building Formers... :)
Hey Sawelios,

You probably just deleted the Colonizer unit design and added another Former design (you could have 100 identical ones if you wanted).
Rok Breulj
Designer and Programmer
Proxy Studios
Sawelios
Lance Corporal - Panzer IA
Lance Corporal - Panzer IA
Posts: 12
Joined: Mon Jan 05, 2015 12:02 pm

Re: My journey of improving the AI

Post by Sawelios »

SephiRok wrote:
Sawelios wrote:Cool. By the way, if you're interested in bug-hunting, I can offer you a fat exemplar :). In my present game I have no possibility to build Colonizers in my city, but instead I have two identical options for building Formers... :)
Hey Sawelios,

You probably just deleted the Colonizer unit design and added another Former design (you could have 100 identical ones if you wanted).
:lol: :lol: Yeah, thanks, I figured it out already...
Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 523
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi »

Over the past couple of days I have mostly been focussed on Technology-selection.
It deemed especially necessary to get important techs that have useless prerequisites. So right now for the evaluation half of the value of a future tech is added to it's prerequisites.

I also found a really good way to slow down AI-expansion and thus having them make their core-cities more powerful. They now consider how well their existing cities are developed before deciding wether to build new cities or not.

But the most important thing for you to know is: I've now commited all the changes I made back to Proxy-Studios and asked them to make a first beta available out of what I've done so far.
This hopefully means that you can see everythin for yourself pretty soon.

In my opinion the differences should already be very noticeable.

The games single-player-experience should be much more challenging and interesting.
thrudda
Private First Class - Opel Blitz
Private First Class - Opel Blitz
Posts: 1
Joined: Sun Dec 13, 2009 7:54 pm

Re: My journey of improving the AI

Post by thrudda »

Xilmi wrote:But the most important thing for you to know is: I've now commited all the changes I made back to Proxy-Studios and asked them to make a first beta available out of what I've done so far.
This hopefully means that you can see everythin for yourself pretty soon.

In my opinion the differences should already be very noticeable.
Yay! Thanks for all the hard work.
JetJaguar
Lance Corporal - SdKfz 222
Lance Corporal - SdKfz 222
Posts: 23
Joined: Mon May 13, 2013 4:21 pm

Re: My journey of improving the AI

Post by JetJaguar »

Xilmi wrote:Over the past couple of days I have mostly been focussed on Technology-selection.
It deemed especially necessary to get important techs that have useless prerequisites. So right now for the evaluation half of the value of a future tech is added to it's prerequisites.

I also found a really good way to slow down AI-expansion and thus having them make their core-cities more powerful. They now consider how well their existing cities are developed before deciding wether to build new cities or not.

But the most important thing for you to know is: I've now commited all the changes I made back to Proxy-Studios and asked them to make a first beta available out of what I've done so far.
This hopefully means that you can see everythin for yourself pretty soon.

In my opinion the differences should already be very noticeable.

The games single-player-experience should be much more challenging and interesting.
Thanks, Ail. I'm really looking forward to this! Please continue to keep us posted.
Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 523
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi »

There had been a little bit of back and forth conversation between Sephirok and me.

I had to adjust the code to align with Proxy-Studios code-conventions.
While doing so I noticed some small things that I wanted to change and unfortunately caused some bugs.
That was on monday.

Yesterday, while testing, I ran into another bug, I wasn't aware of.
I contacted Sephirok again and he told me that he wants me to be happy with my changes first before building the beta.

Frankly that was right after I realized that my technology-picking-mechanism did not really work like I expected it to do.
It was not terribly bad but it meant I would lie, if I said I was happy with it.
But since I really want to get the Beta-Patch out, to see everyones reaction, I rewrote most of it and made sure it really works now.
And since I was at it, I gave another look at why the AI never built Engeneering Bays or Field Labs, figured it out, and fixed it too.

At about 1:30 AM I then felt quite happy and told Sephirok so.
So I'm hoping that they are now going to build the beta-patch. :)

My plans for the nearer future are to look at unit-management and diplomacy.

Here's some points I'm worried about in the regard of unit-management:

The AI loves to make its units jump into the water, where they are easy pray for the aliens and, in case of a war, other players.
Watching a 19-strength mech embarking for a little swim to then be killed by a 2-strength-water-alien is just heartbraking!
This has to stop. I want them to avoid water unless it is really necessary. That's what ships and planes are there for.
Then there's all those cases where the AI does not heal up properly.
Badly injured soldier goes back into city, heals up to 1.8/2.5 and then just walks right into the face of the aliens, that hurt him again.
Of course you cannot wait to heal up when you are shelled by spitter-artillery. But in any other case, waiting this extra round to be full-hp instead of just almost full-HP can and will often make the difference between survival and death.
Every unnecessarily lost unit in the early-game will slow down your progress significantly so you have to be extra-carefull with those.

And here's what I think about diplomacy:

Bribing AIs to go to war with each other seems a big issue. Due to this I've turned certain doom into victory many times.
Also bribing someone to go to war with someone you aren't at war with yourself is even worse. Preemtively getting them off your back and have them devote their economy to military while you can savely tech up and build infrastructure is a cheap trick to dramatically lower difficulty.
Two AI's that are both stronger than you? Let them kill their units on each other and then join in with the winning side to snatch some cities!
AI should be a lot more wary about this stuff. It should still be possible but way harder.
Then everything that involves paying credits in diplomacy is just a joke right now. Why? Because the amount of credits asked for is relative to what you have. For the AI it doesn't make a difference if you pay 500 credits or 22, if that's what you have. So just by spending your credits or running low taxes to not generate a lot of them in the first place, you will basically make tributes and payments to have them sign contracts a non-issue.
The amount of credits asked for should scale with what you could have! You want that contract signed? Then save up for it! Not spend your cash on a buyout and get the contract signed for your remaining 16 credits.
Sapiento
Lance Corporal - SdKfz 222
Lance Corporal - SdKfz 222
Posts: 20
Joined: Sat Nov 23, 2013 12:51 pm

Re: My journey of improving the AI

Post by Sapiento »

Sounds like a bunch of good ideas.
Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 523
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi »

First Beta of my Improvements is now available! :)
CB60
Private First Class - Wehrmacht Inf
Private First Class - Wehrmacht Inf
Posts: 6
Joined: Sat Jun 14, 2014 12:15 am

Re: My journey of improving the AI

Post by CB60 »

Thanks, but where?
Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 523
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi »

CB60 wrote:Thanks, but where?
It is available in both Steam and the Standalone client. in steam right click The game and select properties. There you shall be able to find a beta-tab where you can enable the beta from a dropdown menu.
CB60
Private First Class - Wehrmacht Inf
Private First Class - Wehrmacht Inf
Posts: 6
Joined: Sat Jun 14, 2014 12:15 am

Re: My journey of improving the AI

Post by CB60 »

Thanks man,it's been so long since I fired this up.
Post Reply

Return to “Pandora - First Contact”