My journey of improving the AI

4X strategy game from Proxy Studios

Moderators: Pandora Moderators, Slitherine Core

Zak0r
Sergeant - Panzer IIC
Sergeant - Panzer IIC
Posts: 188
Joined: Sat Jan 25, 2014 1:21 pm

Re: My journey of improving the AI

Post by Zak0r » Sun Feb 01, 2015 9:18 am

Often when I ready Xilmi's analyses how the AI should think (what city to take, how much population, how well is it defended, do I have the needed troops, what do I still need (maybe also how good are my trade pacts with the target faction?)) it is exactly how I think about that ingame. :D

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 513
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Mon Feb 02, 2015 1:24 pm

Zak0r wrote:Often when I ready Xilmi's analyses how the AI should think (what city to take, how much population, how well is it defended, do I have the needed troops, what do I still need (maybe also how good are my trade pacts with the target faction?)) it is exactly how I think about that ingame. :D
And yet you always surprise me in our multiplayer-games.

You know that your war against me in the last game kinda encouraged me...

First I was like: "Aww, the poor AI still can't do anything about Zak0rs timing-attacks."

But when you made me look equally helpless, the incompetence of the AI dealing with you shone in another light. ^^

Zak0r
Sergeant - Panzer IIC
Sergeant - Panzer IIC
Posts: 188
Joined: Sat Jan 25, 2014 1:21 pm

Re: My journey of improving the AI

Post by Zak0r » Mon Feb 02, 2015 4:28 pm

You gotta love those ships. :) But it wouldn't have worked if I didn't encounter first the aspidoch who killed my ship and then the leviathan who I could kill by sacrificing one ship and gaining 8 exp and the tech with the other. The Aspidoch was the next to be killed by the stronger ship then and suddenly I had that 40-strength ship in early era 2 already. That was a very long stretched but still somehow spontaneous plan. :D

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 513
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Thu Feb 05, 2015 7:19 pm

I prepared everything for the next build of the beta.

Here's the changes compared to the current version:

Added:
* Added two sets of new icons for farmers, miners, workers and scientists that gain higher yields than normal.
* Added option in Pandora.xml to switch between the two sets of new icons.
* Added a mechanism for the AI to check how usefull a war will be and make its decision to go to war dependand on that.

Changed:
* AI now will, in average, commit more scientists to research.
* AI now considers pollution when assigning population to jobs.
* AI will no longer grant you gifts all the time just because they like you.
* AI players are now more difficult to convince to declare war on others.
* AI now will prioritize devices that best fit the unit-type.
* AI now can and will use fungus once it has the two advancements boosting its yield.
* AI will now allocated more minerals into boosting its newly found cities.
* AI will now lower their standing to someone capturing a city depending on that cities importance for its previous owner.
* AI now takes device and unit-type into consideration, when deciding which units to use as defenders.
* AI no longer considers distance when picking defenders for its cities.
* AI no longer voluntarily gifts away its credits.
* AI will now use bombardment and nukes primarily to kill transports and disrupt healing in besieged cities.
* AI will now be much more carefull about where to use blackholes.
* AI will now rather save a WMD for better use than to risk overnuking a target.
* Depending on their aggression-level AIs now may consider making more units as a means of expansion rather than building colonizers.
* AI will now rather pillage tile-improvements instead of retreating if it considers the defenses of a city too strong.
* Improved AI behavior when confronted with long-range-units.
* AI will now be more careful about where to colonize.
* AI will now keep units inside their cities if they are not needed otherwise.
* AI will now consider WMD-buildings more valuable.
* AI now is much more likely to build naval units in coastal cities.
* AI will no longer value research once it has enough science for a research-victory.

Fixed:
* Fixed AI ignoring how you react to a tribute-offer. Standing will now properly change.

JetJaguar
Lance Corporal - SdKfz 222
Lance Corporal - SdKfz 222
Posts: 23
Joined: Mon May 13, 2013 4:21 pm

Re: My journey of improving the AI

Post by JetJaguar » Thu Feb 05, 2015 9:20 pm

Xilmi wrote:I prepared everything for the next build of the beta.

Here's the changes compared to the current version:

Added:
* Added two sets of new icons for farmers, miners, workers and scientists that gain higher yields than normal.
* Added option in Pandora.xml to switch between the two sets of new icons.
* Added a mechanism for the AI to check how usefull a war will be and make its decision to go to war dependand on that.

Changed:
* AI now will, in average, commit more scientists to research.
* AI now considers pollution when assigning population to jobs.
* AI will no longer grant you gifts all the time just because they like you.
* AI players are now more difficult to convince to declare war on others.
* AI now will prioritize devices that best fit the unit-type.
* AI now can and will use fungus once it has the two advancements boosting its yield.
* AI will now allocated more minerals into boosting its newly found cities.
* AI will now lower their standing to someone capturing a city depending on that cities importance for its previous owner.
* AI now takes device and unit-type into consideration, when deciding which units to use as defenders.
* AI no longer considers distance when picking defenders for its cities.
* AI no longer voluntarily gifts away its credits.
* AI will now use bombardment and nukes primarily to kill transports and disrupt healing in besieged cities.
* AI will now be much more carefull about where to use blackholes.
* AI will now rather save a WMD for better use than to risk overnuking a target.
* Depending on their aggression-level AIs now may consider making more units as a means of expansion rather than building colonizers.
* AI will now rather pillage tile-improvements instead of retreating if it considers the defenses of a city too strong.
* Improved AI behavior when confronted with long-range-units.
* AI will now be more careful about where to colonize.
* AI will now keep units inside their cities if they are not needed otherwise.
* AI will now consider WMD-buildings more valuable.
* AI now is much more likely to build naval units in coastal cities.
* AI will no longer value research once it has enough science for a research-victory.

Fixed:
* Fixed AI ignoring how you react to a tribute-offer. Standing will now properly change.
Looks excellent!
* AI will now lower their standing to someone capturing a city depending on that cities importance for its previous owner
How exactly does this work?

also: is tomorrow a possible release date? Thanks in advance.

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 513
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Thu Feb 05, 2015 9:30 pm

JetJaguar wrote:How exactly does this work?
new standing = current standing - population of captured city / total population of old city-owner * ( 1 - aggressiveness of the faction who's standing is changed)

So warmongers will care less about when you capture cities than peaceful factions do.
JetJaguar wrote:also: is tomorrow a possible release date? Thanks in advance.
I hope it will be tomorrow. Unfortunately I'm not at home over the weekend so if Sephirok/void find something that I need to fix I won't be here to do it.

Zak0r
Sergeant - Panzer IIC
Sergeant - Panzer IIC
Posts: 188
Joined: Sat Jan 25, 2014 1:21 pm

Re: My journey of improving the AI

Post by Zak0r » Fri Feb 06, 2015 9:46 am

* AI will no longer value research once it has enough science for a research-victory.

Does this mean the AI will also stop researching if research victory is turned off? If yes, then I think it should be changed to "when it has enough science to research all techs in the game" for this special case.

Other than that: Good work! :D

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 513
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Fri Feb 06, 2015 2:35 pm

When research-victory is turned off, it'll stop researching once it has all techs. This might be a little late but I doubt many players play this way so impact might be rather small.
Did some bug-fixes today, and am having a blast with how it plays now in my let's play. I wasn't too confident about diplomacy but it feels really good now.
Hope you'll also be getting this patch while I'm gone over the weekend. ^^

Zak0r
Sergeant - Panzer IIC
Sergeant - Panzer IIC
Posts: 188
Joined: Sat Jan 25, 2014 1:21 pm

Re: My journey of improving the AI

Post by Zak0r » Sat Feb 07, 2015 9:10 am

But what if the AI has a lot of lowlvl tech left at the end? Then it would waste valuable turns gathering more science it would not need to get those techs.

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 513
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Tue Feb 10, 2015 10:19 am

I agree. I need a mechanism to count the research needed for the remaining techs to be able to stop teching early.

After I finished my last test-game I realized that the AI was building way too few troops.

This was mainly caused by the new population-allocation-mechanism which was suited primarily to boost the economy of new cities and advance technologically quickly.

There now are two population-allocation-mechanisms:

The one I had before, where the majority of minerals are allocated to new cities and established cities mainly do research.

And a new one, which I'd call war-mode.
In this the minerals are primarily spent where they are most efficient and the amount of desired scientists is halved.

Currently the new one is only used in war. But I'm thinking that there should be an algorithm where technological superiority and aggressivity of a faction are taken into account and it would also be used to prepare wars.

An aggressive faction that is technologically ahead then instead of trying to get more ahead would rather churn out units and try to conquer its neigbours.

Zak0r
Sergeant - Panzer IIC
Sergeant - Panzer IIC
Posts: 188
Joined: Sat Jan 25, 2014 1:21 pm

Re: My journey of improving the AI

Post by Zak0r » Wed Feb 11, 2015 12:17 pm

This could be also a good point to sharpen the personalities of the factions. Togra, Noxium, Terra Salvum and the Ambassadors could prefer the first while the rest could prefer conquering. One could treat the aggression scale as a scale between those two poles.

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 513
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Thu Feb 12, 2015 12:32 am

I just finished a game with this mechanism in place.

The interesting thing is that Morgan Industries (with an Aggression of 0.3) conquered by far the most.

They probably slipped into this "mode" when they entered their first war but then they never got out of it as they just continued conquering one after the other.

At the end the Free Drones had insane research:
Image

But it was too late to stop me from getting science-victory first.

I feel the war-target-selection is still tricky.

In this build (which according to Sephirok will become the next beta-build) targets are mostly selected by how well it fits from an opportunistic point of view.
Seeing how much Morgan conquered this way, the approach works pretty well.
However, attacking the top-dog rarely is particularly appealing in that regard. So a faction which everyone secretly hates still mostly can deal with the opposition one by one.

This is really hard to balance. Attack the biggest threat but miss good opportunities to conquer cities or go for the easy targets and risk your rival growing too strong?

Actually it feels pretty human-alike as it is now. I think most human players would also prefer the easy target instead of risking everything by declaring war on the strongest player.
But I'm not sure if this behaviour is best for the playing-experience. It might be a little immersion-breaking.

Zak0r
Sergeant - Panzer IIC
Sergeant - Panzer IIC
Posts: 188
Joined: Sat Jan 25, 2014 1:21 pm

Re: My journey of improving the AI

Post by Zak0r » Thu Feb 12, 2015 8:13 am

Maybe this particular problem could be resolved if you introduce alliances that are formed for such wars and actually work together against a stronger target.

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 513
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Fri Feb 13, 2015 11:48 am

Zak0r wrote:Maybe this particular problem could be resolved if you introduce alliances that are formed for such wars and actually work together against a stronger target.
Well, not really.
You still have the same question.
Imagine you are quite powerfull but not the most powerful. There's a weak neighbour to your right.
He probably is willing to ally with you anyways. But would this really be the best for you?

You still have to make the best decision:

Do I ally with him and then we both attack the strongest faction together or do I annex his cities to my own empire hoping to give me the edge I need to perhaps make me the strongest?

Even for a human player like myself this question isn't easily answered.

In our multiplayer-games there always is this Zak0r and from the approaches of how to deal with him the "trying to unite with others and go for him"-approach did not necessarily reap the best results.
You see: Sometimes it feels much better to just finish second rather than getting your capital blackholed into oblivion.

I also think a mechanism that does too well in "top-dog-stopping" (e.G.: everyone declares war on him and allies with everyoneone else) is much easier to exploit ("I stay in close second and shortly before the game is over I pull ahead before the rest can really react.") than if there simply are very strong empires that you have to actually beat yourself in order to win.

So I think that selfish and opportunistic behaviour, while not necessarily the hardest to win against, will at least not offer weird ways of exploiting it. Also: "An AI that tries to win." sound better than "An AI that tries to prevent you from winning.", doesn't it?

But to sum it up:

If I knew how to beat you, I would probably have done so. And since I don't know how, I also cannot program the AI to do it. ^^

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 513
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Wed Mar 04, 2015 7:28 pm

Gotta love when something works exactly as intended:

Image

Image

Image

Ain't it heart-warming how perfect Lady Vermillion executed this siege?

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 513
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Thu Mar 12, 2015 6:14 pm

New record for earliest tech-victory for an Easy-AI: Turn 256

Image

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 513
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Tue Mar 17, 2015 10:48 am

I think I can announce a new milestone:

My main-beta-tester Zak0r no longer seems to be able to beat the game on "Very-Hard".

The combination of the current boni and play that actually looks pretty smart simply seems too much.

We will have to conduct tests on "Hard" and then, depending on how these go, actually readjust (lower!) the boni the AI gets on Hard and Very-Hard.

The aim for very-hard is to be barely beatable but not everytime by the best players.


The change that seemed to play the biggest role in this breakthrough was a new dynamic threat-estimation system, that, instead of deciding whether to build units or economy on a score-based evaluation-system, adjusts a percentage value of how much of the economies value should be invested in units.

This ensures the AI never completely over- or undercommits on unit-production while still allowing for distinct situational and faction-specific playstyle-differences.

For example a peacefull AI with noone nearby might settle for 20% of his economy-value in units, while an aggressive-AI with a nearby neighbour might consider 80% the better value.

In wartime the rules change again. The stronger one will stick with the percentage-based-system to not fall behind those who are not engaged in wars, while the faction who thinks is losing, will completely commit to military in order to ensure its survival.

This goes hand in hand with improvements in unit-management like surrounding cities and bringing artillery for siege, improved tile-management, more sophisticated diplomatic decisions and adjustments to the dynamic population-allocation-system.

So all in all, I think I did a decent job in improving the AI so far.

But that of course doesn't mean I'm going to stop here. My ultimate goal still is to make the AI as challenging as possible even without any boni! Pandora shall always be the first mention when someone asks for a 4x-game that actually has good AI!

AdvancedMilitaryComm
Lance Corporal - Panzer IA
Lance Corporal - Panzer IA
Posts: 15
Joined: Sun Aug 21, 2011 11:17 am

Re: My journey of improving the AI

Post by AdvancedMilitaryComm » Tue Mar 17, 2015 11:04 pm

Awesome! Thanks for the hard work!

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 513
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Wed Mar 18, 2015 7:33 pm

Improved the previous record by another 45 turns down to turn 211.
Non-ambassador-factions took 17 turns longer to accumulate a similar amount of Science, though.
Image

Apheirox
Senior Corporal - Ju 87G
Senior Corporal - Ju 87G
Posts: 95
Joined: Tue Dec 03, 2013 10:54 pm

Re: My journey of improving the AI

Post by Apheirox » Sat Mar 28, 2015 5:15 pm

Xilmi wrote:But that of course doesn't mean I'm going to stop here. My ultimate goal still is to make the AI as challenging as possible even without any boni! Pandora shall always be the first mention when someone asks for a 4x-game that actually has good AI!
Relating to a thread in which someone complained about the new AI difficulty, though: As you improve the AI, the descriptions of the difficulty levels should change to match, as well. On 'easy' difficulty, the game should be fairly easy and accessible to somebody not a hardcore 4X player - not a bruising challenge. There must remain an entry-level difficulty where players not experienced with these games aren't simply slaughtered (Master of Orion used the aptly named 'Tutor' level). You should talk to the developers about having more difficulty levels added so that these are descriptive. It's perfectly OK for Pandora to be a challenging game, of course, so the scale might simply go from 'very easy' and 'easy' over 'medium' but then onto a vocabulary that in fact describes the situation: 'hard' and 'very hard', 'brutal', 'extreme', 'impossible'. If 'very easy' simply crushes new players with no option for getting to grips with the game that is rather off-putting.

TLDR: I'm simply saying the difficulty level descriptions should indeed be descriptive, and that there must remain difficulty levels that aren't too demanding such that the game remains accessible to anyone.

Post Reply

Return to “Pandora - First Contact”