How has the A.I been improved ?

A new story begins...
The sequel to a real classic: Panzer Corps is back!

Moderator: Panzer Corps 2 Moderators

KarlXII
Senior Corporal - Ju 87G
Senior Corporal - Ju 87G
Posts: 80
Joined: Mon Nov 19, 2007 9:42 am

How has the A.I been improved ?

Post by KarlXII »

I would like an Developers Diary or an article in this forum about how the A.I in Panzer Corps 2 will be improved.

The A.I in Panzer Corps was not very good, relying mostly on quantity rather than manouvering and tactical brilliance. Has any efforts got into improving that core formula ?
And why have a battlefield map with echelons of units totally separated from each other and only activated if the player comes nearby ? Why not cooperate between the echelons on a larger scale or allow a weak echelon to retreat back to a larger one for stronger defense ?
Rudankort
FlashBack Games
FlashBack Games
Posts: 3836
Joined: Sat Aug 21, 2010 2:23 pm
Contact:

Re: How has the A.I been improved ?

Post by Rudankort »

Yes, I know that many people are interested in the new AI, and we plan to provide more details about it in the future. Just one comment I have is, this is always a decision of campaign designer how much freedom to give to the AI. Some designers prefer to give it more freedom than others. In Panzer Corps you can certainly notice this difference in approach between different campaigns. No matter how smart the new AI is going to be, this difference in designer approach will likely remain.
Stephen1024
Sergeant First Class - Panzer IIIL
Sergeant First Class - Panzer IIIL
Posts: 362
Joined: Wed Sep 16, 2015 12:34 pm

Re: How has the A.I been improved ?

Post by Stephen1024 »

Rudankort wrote: Thu Oct 18, 2018 11:08 am Yes, I know that many people are interested in the new AI, and we plan to provide more details about it in the future. Just one comment I have is, this is always a decision of campaign designer how much freedom to give to the AI. Some designers prefer to give it more freedom than others. In Panzer Corps you can certainly notice this difference in approach between different campaigns. No matter how smart the new AI is going to be, this difference in designer approach will likely remain.
Hi Rudankort
Having different designer approaches is a positive thing as makes things different and enables new approaches. Having one system or designer often makes things to much same. Personally speaking I hope you have lots choice on difficulty. AI important but getting good balance on difficulty is more important. I am carer at times easy to have time play challenging game play other times don't want be pushed challenge wise. To difficult will push people away and to easy will, so option to make it harder or easier are very important I think.
verstaubtgesicht
Senior Corporal - Destroyer
Senior Corporal - Destroyer
Posts: 117
Joined: Mon Feb 18, 2013 9:39 pm

Re: How has the A.I been improved ?

Post by verstaubtgesicht »

Take a clue from chess. Even a half-decent computer program can beat a very good human player and the top machines can eat the world champ for breakfast. Chess is a much more complicated game than PzC... Can't you teach the PzC AI to avoid the stupidest mistakes like blindly attacking a unit backed up by artillery (a human player would simply move in front of the unit that he wants to attack, then check if there is arty behind, and only attack if there is no arty)? Same for the air force - fighter moves next to an enemy bomber and attacks it even though there is fighter protection? Why can't the AI learn how to use arty to soften up a target before attacking? Why is the AI programmed to enter mandatorily a victory location city AND then attack any adjacent enemy unit no matter what the odds are? The list goes on and on...
Rudankort
FlashBack Games
FlashBack Games
Posts: 3836
Joined: Sat Aug 21, 2010 2:23 pm
Contact:

Re: How has the A.I been improved ?

Post by Rudankort »

Stephen1024 wrote: Fri Nov 02, 2018 10:22 am Hi Rudankort
Having different designer approaches is a positive thing as makes things different and enables new approaches. Having one system or designer often makes things to much same. Personally speaking I hope you have lots choice on difficulty. AI important but getting good balance on difficulty is more important. I am carer at times easy to have time play challenging game play other times don't want be pushed challenge wise. To difficult will push people away and to easy will, so option to make it harder or easier are very important I think.
I think, no one is going to complain about the lack of difficulty options in Panzer Corps 2. In fact, some people think that we are adding too many. :)
verstaubtgesicht wrote: Mon Nov 12, 2018 8:11 pm Take a clue from chess. Even a half-decent computer program can beat a very good human player and the top machines can eat the world champ for breakfast. Chess is a much more complicated game than PzC...
Chess is not more complicated than Panzer Corps, on the contrary it is much easier. There are many reasons for this: very small game board and number of pieces, very simple rules, lack of uncertainty in combat calculations and fog of war, all this makes it pretty trivial to calculate variations on modern hardware. In Panzer Corps it is not practical to calculate all variations even one turn into the future, so different approaches are needed.

However, mistakes like the ones you describe are definitely possible to avoid, and we are working on it.
verstaubtgesicht
Senior Corporal - Destroyer
Senior Corporal - Destroyer
Posts: 117
Joined: Mon Feb 18, 2013 9:39 pm

Re: How has the A.I been improved ?

Post by verstaubtgesicht »

[/quote]

Chess is not more complicated than Panzer Corps, on the contrary it is much easier. There are many reasons for this: very small game board and number of pieces, very simple rules, lack of uncertainty in combat calculations and fog of war, all this makes it pretty trivial to calculate variations on modern hardware. In Panzer Corps it is not practical to calculate all variations even one turn into the future, so different approaches are needed.

However, mistakes like the ones you describe are definitely possible to avoid, and we are working on it.
[/quote]

If you program like Stockfish for chess, with all the rules, then yes. But if you program like AlphaZero, then no. Just let a massive AI self-thought program (playing against itself a few million games) going, and you'll see what AI can do.

Glad that you are at least fixing the obvious ones....
Rudankort
FlashBack Games
FlashBack Games
Posts: 3836
Joined: Sat Aug 21, 2010 2:23 pm
Contact:

Re: How has the A.I been improved ?

Post by Rudankort »

verstaubtgesicht wrote: Fri Nov 16, 2018 1:32 am If you program like Stockfish for chess, with all the rules, then yes. But if you program like AlphaZero, then no. Just let a massive AI self-thought program (playing against itself a few million games) going, and you'll see what AI can do.

Glad that you are at least fixing the obvious ones....
This does not change the fact that a game like Panzer Corps is much more complex than Chess. Although Chess has a very big number of possible positions, number of states in Panzer Corps is many orders of magnitude greater, and transition from one state to another is much more computationally intensive (for example, one calculation of combat outcome takes a lot of cycles, and there is not much room to optimize it). There are other problems as well, like the lack of a standard starting state. This means that any learning process will take much longer on the same hardware. Although I'm watching recent successes of (deep) reinforced learning with great interest, and I'm pretty sure that I will try to apply these methods to Panzer Corps at some point, right now I'm not sure that it is even feasible on modern hardware to reach reasonable play level in reasonable time. I'm curious what makes you so optimistic about applying these methods in more complex games.
verstaubtgesicht
Senior Corporal - Destroyer
Senior Corporal - Destroyer
Posts: 117
Joined: Mon Feb 18, 2013 9:39 pm

Re: How has the A.I been improved ?

Post by verstaubtgesicht »

Rudankort wrote: Fri Nov 16, 2018 8:33 am
verstaubtgesicht wrote: Fri Nov 16, 2018 1:32 am If you program like Stockfish for chess, with all the rules, then yes. But if you program like AlphaZero, then no. Just let a massive AI self-thought program (playing against itself a few million games) going, and you'll see what AI can do.

Glad that you are at least fixing the obvious ones....
This does not change the fact that a game like Panzer Corps is much more complex than Chess. Although Chess has a very big number of possible positions, number of states in Panzer Corps is many orders of magnitude greater, and transition from one state to another is much more computationally intensive (for example, one calculation of combat outcome takes a lot of cycles, and there is not much room to optimize it). There are other problems as well, like the lack of a standard starting state. This means that any learning process will take much longer on the same hardware. Although I'm watching recent successes of (deep) reinforced learning with great interest, and I'm pretty sure that I will try to apply these methods to Panzer Corps at some point, right now I'm not sure that it is even feasible on modern hardware to reach reasonable play level in reasonable time. I'm curious what makes you so optimistic about applying these methods in more complex games.
Rudankort, we are at different levels here. I know close to zero about computer programming, and you know a lot. All I know is what I read in the papers. https://www.nytimes.com/2016/12/14/maga ... ening.html .
People have tried for decades to come up with a good language translation program, and despite herculean efforts by many smart people, even an 8th grader could tell a human translation from a computer one. Then Google introduced this self-learning deal, and bingo - literary scholars from both Japan and the US could not tell the difference in translations of Hemingway's work into Japanese! Pls don't tell me that PanzerCorps is more complicated than that... Then the AlphaZero happened - beats every computer (I don't think that human chess players even dared to try) after learning chess from scratch by playing against itself for 4hrs! Now, I have no clue how this thing works, and I gather that you are not Sergey Brin's buddy and don't have access to $100M worth farm of processors, but just try your best, ok?
Rudankort
FlashBack Games
FlashBack Games
Posts: 3836
Joined: Sat Aug 21, 2010 2:23 pm
Contact:

Re: How has the A.I been improved ?

Post by Rudankort »

verstaubtgesicht wrote: Mon Nov 19, 2018 8:55 pm Rudankort, we are at different levels here. I know close to zero about computer programming, and you know a lot. All I know is what I read in the papers. https://www.nytimes.com/2016/12/14/maga ... ening.html .
People have tried for decades to come up with a good language translation program, and despite herculean efforts by many smart people, even an 8th grader could tell a human translation from a computer one. Then Google introduced this self-learning deal, and bingo - literary scholars from both Japan and the US could not tell the difference in translations of Hemingway's work into Japanese! Pls don't tell me that PanzerCorps is more complicated than that... Then the AlphaZero happened - beats every computer (I don't think that human chess players even dared to try) after learning chess from scratch by playing against itself for 4hrs! Now, I have no clue how this thing works, and I gather that you are not Sergey Brin's buddy and don't have access to $100M worth farm of processors, but just try your best, ok?
Yes, you are right, recent successes in machine learning are very impressive. But there are also problems. I'm sure you've seen how people are disliking the AI in games like Civ. Firaxis has a lot of resources, why haven't they already implemented a great AI based on machine learning? Yes, the AI generates texts which cannot be distinguished from human, and at the same time, I've noticed lately that Internet is getting increasingly full of machine-generated texts of terrible quality. They often come up in search, and it's really frustrating. Maybe you have seen them too. Yes, in some ways, human language is certainly more complex than Panzer Corps. But as a product of human brain, it seems to be very well suited to neural networks, plus there is a huge body of material (e. g. translations made by humans) which can be used to train the system. Also, all people can learn their language, but not all people can play Panzer Corps (or Chess, for that matter) on a good level. ;) The bottom line is, it's a very complex topic, but usually successes appear where there is some very serious science backed by good funding. :)

But anyway. I won't pretend that I'm a huge expert in this topic, and my views might be just as wrong as yours. I agree that this direction is very perspective, and I'm very much willing to try it. No promises at this point, but yeah, I'll try my best. :)
Kiane
Corporal - 5 cm Pak 38
Corporal - 5 cm Pak 38
Posts: 35
Joined: Tue Dec 13, 2011 10:34 pm

Re: How has the A.I been improved ?

Post by Kiane »

AI may be the hardest thing to do in programming (as evidenced by the fact that hardly anybody has ever done it particularly well!). I was very pleased with myself when I got Dijkstra's working for a Pacman ripoff game I made... :)

I hope you are able to improve the AI, but at the same time it needs to be stupid sometimes for the player to have an historical experience.
Rudankort
FlashBack Games
FlashBack Games
Posts: 3836
Joined: Sat Aug 21, 2010 2:23 pm
Contact:

Re: How has the A.I been improved ?

Post by Rudankort »

Kiane wrote: Fri Nov 30, 2018 10:05 am I hope you are able to improve the AI, but at the same time it needs to be stupid sometimes for the player to have an historical experience.
It's a good point. Looking at the ongoing Panzer Corps tournament, it is clear that competitive play in this kind of a game requires a very specific playing style, which many people would not enjoy at all from the AI. On one hand, it is much more "exploitative" of game rules, with things like disbanding half your army to get useful units perfectly normal. On the other hand, playing like this requires extensive knowledge of each scenario, to an extent where you can often predict or guess where the opponent's units might get on each turn etc. Finally, this playing style requires you to make sure you finish off every unit you tackle, to avoid it being rebuilt cheaply. And once again, this is often achieved by tactics which look quite "exploitative" of game rules (e. g. cut all routes of retreat by trucks, and then force the unit to surrender). THIS is how the AI would play if it learnt to play in AlphaGo style. And many people would not like it at all.

However, to have a strong AI opponent as an option is of course desirable.
dalfrede
Colonel - Fallschirmjäger
Colonel - Fallschirmjäger
Posts: 1461
Joined: Wed Mar 15, 2017 7:48 pm

Re: How has the A.I been improved ?

Post by dalfrede »

In PzC there are 3 AI levels. I assume that PzC2 will multiple AI levels as well.

Will there be a description the various levels?
How will these levels be beta tested. Will the users decide which levels will be tested?
Or will there be 'formal' testing of the various levels?
There comes a time on every project when it is time to shoot the engineer and ship the damn thing.
Rudankort
FlashBack Games
FlashBack Games
Posts: 3836
Joined: Sat Aug 21, 2010 2:23 pm
Contact:

Re: How has the A.I been improved ?

Post by Rudankort »

dalfrede wrote: Fri Nov 30, 2018 5:40 pm In PzC there are 3 AI levels. I assume that PzC2 will multiple AI levels as well.

Will there be a description the various levels?
How will these levels be beta tested. Will the users decide which levels will be tested?
Or will there be 'formal' testing of the various levels?
Panzer Corps 2 will have different levels of AI as well, but how different we will manage to make them is still an open question. My approach is to make the AI as good as I can, and then "dumb it down" for different levels. The AI will have a number of other configuration parameters too (e. g. does it purchase new units or not and which ones, does it replace and how much etc.)

On random maps, you can select AI level for each computer player. In a historical campaign, scenario balance might depend on AI settings, but as an option it will still be possible to change the level (as in Panzer Corps). I cannot instruct beta testers which levels to test - they will play what they enjoy the most. :) But I hope that with enough play testers all levels will get a sufficient coverage. We will of course also do testing in house.
Gwaylare
Panzer Corps Tournament 3rd Place
Panzer Corps Tournament 3rd Place
Posts: 188
Joined: Sun Sep 04, 2011 7:17 am

Re: How has the A.I been improved ?

Post by Gwaylare »

verstaubtgesicht wrote: Fri Nov 16, 2018 1:32 am

Chess is not more complicated than Panzer Corps, on the contrary it is much easier. There are many reasons for this: very small game board and number of pieces, very simple rules, lack of uncertainty in combat calculations and fog of war, all this makes it pretty trivial to calculate variations on modern hardware. In Panzer Corps it is not practical to calculate all variations even one turn into the future, so different approaches are needed.

However, mistakes like the ones you describe are definitely possible to avoid, and we are working on it.
If you program like Stockfish for chess, with all the rules, then yes. But if you program like AlphaZero, then no. Just let a massive AI self-thought program (playing against itself a few million games) going, and you'll see what AI can do.

Glad that you are at least fixing the obvious ones....
AlphaZero is a great approach of machine learning, but up to now they are just able to play boardgames like chess or go. Rules of those games are not very complex. There is a small number of units available together with quite simple movement descriptions. Combat is trivial, just attacker wins and you are moving one unit per turn only. Modern computer games are much more complex than those boardgames and are getting more complex with every year.
The geatest difficulty for an AI is the high number of options it has in a single turn. A game with one option per turn is trivial to solve. A game like tic-tac-toe has 9 options in the beginning with one option less with every turn. This is still simple. Checkers has some more options per turn and this was solved in 2007. Using chess again as an example, in your first turn you have 20 different options. In later turns you have some more, may be 100 - 200 options. Calculation for more n turns this are about 100^n options, I guess.
In Panzer Corps a simple fighter has more than 200 options to move and you have plenty of different units to move and a lot of different opponent units to take care of. Up to that the sequence of moves does matter sometimes and on top of that there possibilities like buying new units somewhere. Not to mention something like fog of war. So yes I think modern computer games are much more complex then chess is.

Another major difference is, that games like chess are analysed at a very professional level for about hundreds of years. So it is much easier to evaluate if you are doing right or wrong. So we are playing those game at very low level, may be like hobbiests play chess. There are no professionals or analysts for Panzer Corps, we are all playing just for fun.

I think it will need some more time till AIs are able to solve really complex tasks. The problem is not to teach them the rules. Exceptions are what life makes difficult ;-)

Best regards
Gwaylare
huckc
Master Sergeant - U-boat
Master Sergeant - U-boat
Posts: 531
Joined: Mon Apr 24, 2017 1:38 pm
Location: USA

Re: How has the A.I been improved ?

Post by huckc »

Libratus is probably the AI most applicable to Panzer corps, and is the most impressive imo, since it's built to handle games with incomplete information. Last year it absolutely destroyed some of the best no-limit holdem players in the world heads up, which is a game with incomplete information and lots of move types: call, fold, raise any amount. It also needs to throw in bluffs a certain percentage of time.

It has a memory of you and while it's not playing it'll go over previous games in order to fix any holes where the human player got the upper hand. So for example in Panzer Corps if the player punished it with lots of Stuka attacks it'll re-balance itself by buying a few more AA and fighters. The player will then use a different tactic causing the AI to adjust again and so on. It will eventually win every game since it's perfectly balanced to play against you and isn't emotional or blunders.

https://en.wikipedia.org/wiki/Libratus
Rudankort
FlashBack Games
FlashBack Games
Posts: 3836
Joined: Sat Aug 21, 2010 2:23 pm
Contact:

Re: How has the A.I been improved ?

Post by Rudankort »

Gwaylare wrote: Fri Dec 07, 2018 5:18 pm There are no professionals or analysts for Panzer Corps...
You know, when reading AARs on this forum, including ones from you and huckc, I begin to doubt this statement. :) Back in beta times we were pretty sure that Ultimate difficulty was unplayable, that's the reason why it was disabled in the first place...
Sourdust
Senior Corporal - Destroyer
Senior Corporal - Destroyer
Posts: 102
Joined: Mon Feb 06, 2012 1:34 am

Re: How has the A.I been improved ?

Post by Sourdust »

Best AI I have seen in a strategic/operational wargame is probably DC3 - it is one of the few that gives a plausible impression of having a "strategic sense", or an awareness of the overall shape of a frontline, for instance, which informs decisions that give the appearance of being at a higher level than just individual unit movements. For instance, DC3 AI will conduct withdrawals from an area of the front if it looks like it is at risk of being surrounded. It still falls far short of a human player in this respect, but I was impressed.

Runners-up include Unity of Command, with it's uncanny ability to cycle units through an attack hex and breakthrough to cut off your supply. And SSG's old AIs in Battlefront series were impressive - see "The Ardennes Offensive" and "Korsun Pocket" etc.

I don't know much about AI design myself. I would like to see an AI (and a game as a whole) that de-emphasises the importance of killing core units. In PC currently, the AI seems to prioritise killing core units of the player, even if it makes no sense from a scenario perspective. There's a certain logic here, but it feels frustrating and immersion-breaking that the AI will unhinge an entire defensive position just to kill off a unit.

Of course, the AI should still try to kill units, but its overriding goals should be to hang on to objectives - so it should not sacrifice lots of its own units and good positioning just to kill off a single one of the player's units. No idea how to translate all of that into AI design!
tenshin111
Sergeant - 7.5 cm FK 16 nA
Sergeant - 7.5 cm FK 16 nA
Posts: 243
Joined: Sun Aug 23, 2015 2:23 pm

Re: How has the A.I been improved ?

Post by tenshin111 »

Sourdust wrote: Fri Dec 14, 2018 3:37 am Best AI I have seen in a strategic/operational wargame is probably DC3 - it is one of the few that gives a plausible impression of having a "strategic sense", or an awareness of the overall shape of a frontline, for instance, which informs decisions that give the appearance of being at a higher level than just individual unit movements. For instance, DC3 AI will conduct withdrawals from an area of the front if it looks like it is at risk of being surrounded. It still falls far short of a human player in this respect, but I was impressed.
What's DC3? I'm curious now :)
PeteMitchell
Major-General - Tiger I
Major-General - Tiger I
Posts: 2328
Joined: Tue Jul 10, 2018 1:18 pm

Re: How has the A.I been improved ?

Post by PeteMitchell »

Has there been a Dev Diary on the AI improvement yet?
Comprehensive Battlefield Europe AAR:
http://www.slitherine.com/forum/viewtopic.php?f=145&t=86481
Rudankort
FlashBack Games
FlashBack Games
Posts: 3836
Joined: Sat Aug 21, 2010 2:23 pm
Contact:

Re: How has the A.I been improved ?

Post by Rudankort »

PeteMitchell_2 wrote: Fri Jan 11, 2019 7:29 pm Has there been a Dev Diary on the AI improvement yet?
No, and it is not coming in the next dev diary. But if all goes well, I might be able to do it in the one coming after the next one.
Post Reply

Return to “Panzer Corps 2”