My journey of improving the AI

4X strategy game from Proxy Studios

Moderators: Pandora Moderators, Slitherine Core

Rosseau
Staff Sergeant - Kavallerie
Staff Sergeant - Kavallerie
Posts: 334
Joined: Sun Nov 15, 2009 5:27 am

Re: My journey of improving the AI

Post by Rosseau » Fri May 29, 2015 10:51 pm

I get a run time error when I opt-in Steam beta and when I install 1.6 manually on a separate drive where your 1.55 was working before.

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 485
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Sat May 30, 2015 12:31 pm

Can you screenshot the error-message and attach the logs in c:\Users\<USER>\AppData\Roaming\Proxy Studios\Pandora Beta\Logs\ ?

Zak0r
Sergeant - Panzer IIC
Sergeant - Panzer IIC
Posts: 180
Joined: Sat Jan 25, 2014 1:21 pm

Re: My journey of improving the AI

Post by Zak0r » Thu Aug 06, 2015 11:17 am

Xilmi wrote:I'll do the specialization-thing first.
It's the easiest and at the same time the one that I guess has the greatest overall benefit.
The most important about it is that food and minerals are preferably harvested in cities without any terrain-modifier-bonus on either science or production so that no potentially more efficient scientists have to be working on fields and in mines while other, less suitable scientists do the science.
Basically something like that: If (cityhasterritorysciencebonus) then farmerpriority*=0.9 and minerpriority*=0.9.
It's a little late but I would like to point out one specific aspect:
I think it might be useful to have some fungus even in science cities because of the possibility to "hide" the pollution there. If you look at a highend city you will have potential -4, -8 and -16 pollution from the three buildings (and -2 from the project) which most likely won't be used completely by the scientists there while other cities will probably reach that limit easily and require additional purifiers. I didn't do the math but it could be better to have some fungus in a science city to better distribute overall pollution and then be able to build more factories or other stuff instead of purifiers. (especially when looking at the recent experimental change in science cost and the resulting boost for production focused strategies which I like very much)

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 485
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Thu Aug 06, 2015 2:13 pm

This already is taken into account.

There is an amount of workers, miners and scientists that the AI will strife for.
They are taken from a pool of available workers, miners and scientists where each of them has a value for how good he is at that and they are sorted in a list from which the most efficient ones for each job are taken in a round-robin-style.

For miners the ordering value clearly depends on how high the yield would be. But it is now reduced by an adequate amount if the city has a lab or xenite-flowers.
So a miner who mines 8 minerals in a non-lab-city will have a score of 8.
A miner in a lab-city will have a score of 5.33.

You seem to be talking about the fungi-planting. Research-Labs have a maximum value of 2 when the city has 50% or more of its population in scientists. Fungus value varies between 1 and 3, depending on the average minerals mined per miner and whether the city has or hasn't pollution. When you get the tech and most miners are still mining 2 and 3 mineral-nodes, the value to plant a fungus will definately be above 2.

I think the AIs tile-improvemnt managament is one of the parts that I've spent most of the time on. The good thing is: It's easy to see if it does the wrong thing.

Diplomatic behaviour still is much harder because there's stuff you simply cannot really say is right or wrong unless you actually play. For example: I tested the AI being more aware of backstabbing and what happened... It was overdone and noone dared to declare war anymore.
Thing is: I can't really say: Was it really overdone? It looks like it but on the other hand: What would happen if someone declared war now. Would he get away with it or would he actually be backstabbed, as he fears.

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 485
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Wed Aug 12, 2015 3:18 pm

Just wanted to inform that I'm on something that feels really important and might be one of the biggest remaining AI-weaknesses.

It is about how the AI executes attacks on cities.

Currently when a war starts the AI will send all units that are not assigned to defensive tasks, to attack a target. Prefarably closer ones. If no closer targets are available and at least one enemy city has been scouted the units will make their way there.

This straightforward approach will work as long as the amount of forces is overwhelming enough.

However, the AI would do it no matter if it has 10 leftover units or only 2.

If 2 leftover units reach the target they would realize: Well, we are too few to conquer the city, so lets siege it and wait for reinforcements. However, with defensive forces at the city, they would not neceassarily survive to the arrival of eventual reinforcements.
The reinforcements would come anyways. And maybe just to realize that there's nothing to reinforce and then lay siege on the city aswell and likely die while doing so aswell.

As you can guess and probably have seen in your games against the AI:
This approach is not exactly the art of war and more often than not allows the defender to go into an effective counter-attack once the enemy-forces stop tripling in.

So what is the new approach?

Very simple: Don't send the units out to siege the city if they do not appear numberous enough to perform a successfull siege!

Before the units that are supposed to attack the city are actually sent out to do so, it might be a good idea to count their strenght, guess (or know) the strenght of their target and if they don't appear to be enough simply cancel their order. In that case they will be taken over by the Roam-Agent who tells them to return to the closest city.

This should ensure that whenever an attack is not cancelled it should contain "enough" units. Otherwise the military actions simply are avoided and more units can be produced before actually launching the attack. The same mechanism also allows the AI to retreat their troops to safety if they realize defenses are stronger than they assumed.

I expect that change to greatly increase the AIs capabilities in:
a) avoiding losses when a city is too well defended by retreating
b) avoiding losses when a city is well but not too well defended by sieging it with enough units
If the AI should get into a pointless war, it then will not commit to it in order to not achieve anything anyways. The troop-count should not suffer unless they are well inferior to their enemy in which case they will die defending and not after losing their troops in foreign territory and then not have enough enough defensive capabilities.

ErissN6
Sergeant First Class - Panzer IIIL
Sergeant First Class - Panzer IIIL
Posts: 390
Joined: Wed Nov 20, 2013 11:34 pm
Location: France

Re: My journey of improving the AI

Post by ErissN6 » Thu Aug 13, 2015 3:17 am

Xilmi wrote:If 2 leftover units reach the target they would realize: Well, we are too few to conquer the city, so lets siege it and wait for reinforcements.
At least they can be probe, then the AI knows what to really send.

dmantione
Corporal - Strongpoint
Corporal - Strongpoint
Posts: 62
Joined: Sun Apr 27, 2014 1:48 pm

Re: My journey of improving the AI

Post by dmantione » Thu Aug 13, 2015 8:41 am

Orbital scans are a better tool to observe what needs to be sent.

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 485
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Thu Aug 13, 2015 11:45 am

ERISS wrote:At least they can be probe, then the AI knows what to really send.
That's what the AI did before and it simply didn't turn out to be a good idea most of the time. Yes they can get lucky and find a city that is so badly defended that the probing units can take it right away but most of the time it was a bad idea.

Of course the AI still does scan to see the real amount of troops and if they have done so use this number instead of the estimate. But that's not the main point here.

The point is to limit losses by making sure an attack-force is big enough.

By the way, if you have installed "in-between-patches" in the past you can give the current state a try:

http://www.civforum.de/showthread.php?9 ... ost6831886

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 485
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Tue Aug 25, 2015 8:16 am

I've been tinkering with tactical improvements of the AI. But a mean bug had prevented me from progressing for quite some time. Now it is finally fixed.

I'll do some more testing but I want to announce what you can expect in the future:

The AI now will be way more aggressive in defensive-Scenarios. This might cause them lose their cities faster but it is much harder to conquer a city without suffering own casualties.
The AI now will always spread their units, not only when they reach a target. This means AoE will be much harder to use. This especially will make transport-invasions more dangerous as there won't be such big stacks to bomb all at once.
Also the units move more aware of the territory they are moving through. Catching infantry on plains shall become rarer. They'll prefer to move through rough terrain.

In the moment it seems like the aggression has been lowered too much. Mostly due to the fear of being backstabbed. So that's what I'm on now.

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 485
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Tue Sep 29, 2015 9:51 am

I'm trying to find out the best algorithm to determine it is a good time to expand so I can teach this to the AI.

I think a highly adaptive algoithm taking a lot of parameters into account would be much better than something like "at X population".

The current approach weights lack of habitat against unbuilt buildings, wheras habitat provided by suburbs is ignored and the unbuilt buildings are weighted according to how big the city they are missing in is in comparison to the whole empire. For this, if a city is less than size 8, it is considered as it would be size 8 though.

However, I feel that this leads to somewhat too fast expansion by the AI.
The old approach was basically the same except that habitat provided by subrubrs was not ignored.
Most of the time it worked just fine but in some cases, especially when there was a lack of minerals allowing to city to get it's buildings done, which in turn meant more research and thus more unbuilt buildings, it could lead to massive cities surrounded by nothing but suburbs.
The new mechanism prevents those ugly, unproductive suburb-cities but tends to found so many cities that the AI is forced to pump resources in before they get productive which in turn opens a good window of opportunity to amass military and then simply take these cities from them.

I've had situations where the AI was getting their 7th and 8th city while I had only 3.

I'm thinking that instead of ignoring the suburb-provided habitat I should consider at least a fraction of it. How big it is may depend on the mineral-richness of the city in question.
Is the city mineral-deprived then I should want to expand in order to get minerals from elsewhere. If I have enough minerals I should expand only if really necessary and get more units from the production I otherwise would have to invest in buildings.

Also I'm not so sure about this "unbuilt buildings" thing at all. Mineral-deprivations means more research means more unbuilt buildings means lower drive to expand means mineral deprivation is not going to be resolved.

I'm considering to rewrite the whole algorithm. Maybe a simpler algorithm could be better.
I'd ideally like to somehow consider the base-mineral-income of the city in comparison to its population.

If anyone has ideas about this they are very welcome.

ErissN6
Sergeant First Class - Panzer IIIL
Sergeant First Class - Panzer IIIL
Posts: 390
Joined: Wed Nov 20, 2013 11:34 pm
Location: France

Re: My journey of improving the AI

Post by ErissN6 » Wed Sep 30, 2015 6:22 am

Maybe there must not be one best algorythm?
1) The best, with the power of computer, will each time beat a human (see chess): no game.
2) So, you must include defects. The better way is to make several algorythms, having human personalities with their preferences and so their predictability.
3) The difficulty level should make the AI have several strategies/personnalities, to be able to adapt better (so you must have an analysis sytem) or being less predictable once we guessed its pattern.

Xilmi
Pandora Community Developer
Pandora Community Developer
Posts: 485
Joined: Fri May 23, 2014 3:21 pm

Re: My journey of improving the AI

Post by Xilmi » Wed Sep 30, 2015 3:00 pm

ERISS wrote:Maybe there must not be one best algorythm?
1) The best, with the power of computer, will each time beat a human (see chess): no game.
2) So, you must include defects. The better way is to make several algorythms, having human personalities with their preferences and so their predictability.
3) The difficulty level should make the AI have several strategies/personnalities, to be able to adapt better (so you must have an analysis sytem) or being less predictable once we guessed its pattern.
I've tried one algorithm yesterday. It looked mostly okay but it has a flaw, which I'm not happy about.
For me it was important to get rid of the ties to unbuild builings/advancements.
My new algorithm was to include potential yet unbuilt morale-improvements into the habitat-situation as I'd presume that resolving current morale-issues in existing cities is better than running away from the problems by founding new ones.
However, this is inconsistent since pollution- and morale-improvements once build no longer lower the expansion-drive. So it's just a little delay but does not take care about the real situation.
My next approach will be to use a certain multiplier for suburbs.
On average a suburb worsens a tiles potential by 1/3 - 1/2. So simply counting suburb-provided habitat as half as effective sounds reasonable.
I will keep the potential unbuilt habitat-improvements as anti-expansion-factor and then try again.

Now to address your points:
1) Well, I don't have a problem with that until it actually happened. But so far I'm still able to confortably beat Medium AI and have to resort to higher difficulties for a challenge. So not an issue yet and no reason to not strive for the best.
2) I don't want different algorithms I want an algorithm flexible enough to adjust to the situation.
3) The difficulty-level already does affect the AIs playstyle. This is mainly due to the AIs reaction on morale-problems. When the AI has morale issues it will focus on resolving these with high priority. Building buildings instead of units, lowering taxes and even halting pop-growth producing no food are counter-measures used by the AI when they get into morale-issues.
A Very-Easy-AI will constantly be pressured to run these counter-measures while a very-hard-AI won't have to deal with problems like these most of the time.

Doing something against predictability is hard. The thing is: When I let the AI act to my best knowledge I will know how you they do it. They could only become less predictable if I let them play less than what I think is ideal. And that would not really help my cause.
As the autor of most of the AI-algorithsms in charge, I usually can predict them very well. The point is:
If they really do everything as intended, how am I going to exploit that?
I shouldn't be able to.

Here's an example of how my current game has been going:
I started surrounded by Ambassadors, Noxium, Terra-Salvum and Imperium.
I kept my troop count really high to not look like an easy target. That was well doable since my starting location had plenty minerals and my faction was Divine Ascension.
However, I fell behind in science, of course.

The AIs are, in my opinion expanding too quickly. So it happened that the Imperium put a city really close to my capital.
I checked their strenght and how much they are liked. They were solid military but hated by everyone while I was loved.
So I thought I could risk it.
It almost worked out.
One turn after I declared war on them the Solar Dynasty also declared war on them but Noxium-Corp declared war on me. I had not expected it but knew it could happen.
The turn after that everyone else who was not at war yet also declared war on Imperium.
So the thing I knew was: The imperium has to survive for 11 turns otherwise I could be in huge trouble.
I conquered 1 city of them and 1 of Noxium. Luckily the Imperium still had one city left when Noxium offered peace again.
What would have happened if the Imperium would have died quicker than Noxium offering peace to me?
Well, most likely Everyone would have declared war on me.

Thing is: The others benefitted much more from the the war than I did. I got a crappy size 2 city from Imperium and a mediocre city from Noxium that was almost surrounded by my cities anyways.
They got equal shares of the core-cities of the Imperium.

Now I can't do anything but peacefully or cold-warfully build up my economy and military. The only faction that has a worse position than I have is Terra Salvum but they dominate in every aspect. Noxium and Ambassadors are weak enough so that a 1v1-war could be viable but they have an isolated position on the map which means it is way more likely that others will attack me instead of them if a war breaks out.
So I can just watch how I fall further and more behind of the Solar Dynasty and Terra Salvum who have benefitted most from the demolition of Heid.
The question is not so much if I lose but how. I might survive till someone gets tech-victory but I also might fall so far behind in military that someone will declare war on me. I think that if Solar Dynasty wasn't right over Terra Salvum and scares them from declaring war, they would already have done so.

Um... why did I write all this. Yes... It was to say that even though I can very well predict what each AI will do in which situation, I still don't see how I could have benefited from that if they do everything correctly. Much like in chess.

ErissN6
Sergeant First Class - Panzer IIIL
Sergeant First Class - Panzer IIIL
Posts: 390
Joined: Wed Nov 20, 2013 11:34 pm
Location: France

Re: My journey of improving the AI

Post by ErissN6 » Thu Jul 21, 2016 11:41 am

Xilmi wrote:even though I can very well predict what each AI will do in which situation, I still don't see how I could have benefited from that if they do everything correctly. Much like in chess.
So, it's what I say: When an AI is well done, there is no longer a game as we can only lose, like in chess.
That's why a better game AI has too programmed flaws, not only qualities, they must be humanized.
for the easy difficulty. Getting ahead of the AI in terms of research and development on this level is quite easy, which means you are setting yourself up to be disliked.
It might actually be easier to survive on a harder difficulty since as long as you don't pose a meaningful threat to someone else' victory-ambitions,
So, that's still not an easy difficulty, it's a nasty trap for beginners, which should make them quit the game.
It's like the AI are hidden, and then SURPRISE! all on the player! But maybe you have made the AI military far weaker for the beginner to overcome the lynch?

Post Reply

Return to “Pandora - First Contact”