List of good AI & AI testing.
Posted: 15 Mar 2023 02:08
Update / tldr; here is the page of all AI tested and results, for anyone just wanting to find a known good one - click me!
Well hello there!
OpenTTD AI's, some of them are great, some of them (purposefully) suck, and some of them just suck and crash. Ultimately though I've spent the past few days testing a load of AI's on stock OpenTTD 13.0, with no GRFs and generic map settings.
The game was then played on full speed, whilst I only had some basic passenger trains to keep my company from dying. The only stops was when the AI crashed, i then carried on the game without resetting the AI, as for me a single crash means the AI failed, it sucks, 'literally unplayable' as gamers say.
Most of the AIs i saw and tested were from developers who are no longer around, even the better ones (the AI) have authors who (Based on last Github commit, or last forum usage) have moved on to other things. This leaves us with a lot of AIs that are either broken, or will work for a while and then crash with errors, usually the crashes are because it used too much CPU time (...i mean i have 24 cores an on i9-12900k, I'm sure you'd have found time somewhere, i'm not going anywhere), or it had an error in an array (most of these I've not debugged specifically to date), some of them also are unstable if you do anything a bit too interesting, such as build a town within the game, which is a perfectly not-cheats only thing you can do in single player, but at least one AI crashed at this, whilst some require very precise settings, or something basic (like airplanes) to not'be disabled. The other big issue is, how long to leave an AI before marking it as abandoned and a free for all (again where licences allow) for updating by anyone who wants to commit time to an otherwise good AI with a repairable bug?
Some AIs seem to be basically indestructable, even if they haven't got an update in quite some time. So I'm not sure what causes AI instability, is it OpenTTD updates that alter behaviour that need adapting to in AI code, or does it suggest the AIs in question were always a bit prone to crashing? For me the most crushing part (of this game i wasn't playing, only fast-forwarding anyway), some AI's survived a loooong time, but then randomly errored, which of course forces a reset and it undoes its beautiful massive network, which may be crushing for any long term players of games who use AI.
So, why did I make a thread? Good question! As weeks go by, the AI situation seems to be... slowly getting worse. Because of long-term illness I now find myself at home, a lot. So I'm considering taking on an AI as a project, taking a working but perhaps slightly flawed AI that has been long abandoned (as far as we can tell) by its original author, but is still actively used by people. What is the protocol for taking on an AI like that and just patching it, keeping it ticking over, fixing bugs. Is it a case of just cloning things (where licences allow, of course) and taking it on, or do I need to fork it as a specifically separate thing, with a new name in BaNaNaS which also means people who want to keep updates need to know that has happened, and abandon any current games with a ticking AI timebomb (potentially), and start a new one, with other stuff.
I can also fairly easily make a list, on a web page (if orudge still does it, maybe a page on TT-F domain?) or forum listing all the AI's (...a lot of spare time as I said), and their known stability with the current version, plus maybe one or two columns noting known issues (like creating new towns as i mentioned above),and known things that will cause it to crash, and how compatible it is with GRFs and other such, if at all. I feel it's also needed that there is a more simpler chart in-game when you click on AI.
Other thoughts, but definitely not doable by me. Is it possible to make AIs checkpoint themselves every so often. Then if a script errors, it can be, either automatically or otherwise, reset back to the state it was in at that checkpoint, tiles, units and all. Although I suspect in this state, it wouldn't work because the script will, unless they've got a bit of random to it, just make the same decisions on the same situations and crash again? I dunno, just a thought to try making a script error not as fatal.
Lastly of course, is it worth me doing any of this, keeping some abandoned AI's updated to some degree, do people actually use the AI? I feel the way it is now, AIs are a bit of a scary prospect for new players as you don't know precisely what you're getting into.
On a different note, is it possible to make links on the script output clickable? Some AI's have links like this to report bugs, but making them non-clickable doesn't encourage reports and therefore repairs.
Well hello there!
OpenTTD AI's, some of them are great, some of them (purposefully) suck, and some of them just suck and crash. Ultimately though I've spent the past few days testing a load of AI's on stock OpenTTD 13.0, with no GRFs and generic map settings.
The game was then played on full speed, whilst I only had some basic passenger trains to keep my company from dying. The only stops was when the AI crashed, i then carried on the game without resetting the AI, as for me a single crash means the AI failed, it sucks, 'literally unplayable' as gamers say.
Most of the AIs i saw and tested were from developers who are no longer around, even the better ones (the AI) have authors who (Based on last Github commit, or last forum usage) have moved on to other things. This leaves us with a lot of AIs that are either broken, or will work for a while and then crash with errors, usually the crashes are because it used too much CPU time (...i mean i have 24 cores an on i9-12900k, I'm sure you'd have found time somewhere, i'm not going anywhere), or it had an error in an array (most of these I've not debugged specifically to date), some of them also are unstable if you do anything a bit too interesting, such as build a town within the game, which is a perfectly not-cheats only thing you can do in single player, but at least one AI crashed at this, whilst some require very precise settings, or something basic (like airplanes) to not'be disabled. The other big issue is, how long to leave an AI before marking it as abandoned and a free for all (again where licences allow) for updating by anyone who wants to commit time to an otherwise good AI with a repairable bug?
Some AIs seem to be basically indestructable, even if they haven't got an update in quite some time. So I'm not sure what causes AI instability, is it OpenTTD updates that alter behaviour that need adapting to in AI code, or does it suggest the AIs in question were always a bit prone to crashing? For me the most crushing part (of this game i wasn't playing, only fast-forwarding anyway), some AI's survived a loooong time, but then randomly errored, which of course forces a reset and it undoes its beautiful massive network, which may be crushing for any long term players of games who use AI.
So, why did I make a thread? Good question! As weeks go by, the AI situation seems to be... slowly getting worse. Because of long-term illness I now find myself at home, a lot. So I'm considering taking on an AI as a project, taking a working but perhaps slightly flawed AI that has been long abandoned (as far as we can tell) by its original author, but is still actively used by people. What is the protocol for taking on an AI like that and just patching it, keeping it ticking over, fixing bugs. Is it a case of just cloning things (where licences allow, of course) and taking it on, or do I need to fork it as a specifically separate thing, with a new name in BaNaNaS which also means people who want to keep updates need to know that has happened, and abandon any current games with a ticking AI timebomb (potentially), and start a new one, with other stuff.
I can also fairly easily make a list, on a web page (if orudge still does it, maybe a page on TT-F domain?) or forum listing all the AI's (...a lot of spare time as I said), and their known stability with the current version, plus maybe one or two columns noting known issues (like creating new towns as i mentioned above),and known things that will cause it to crash, and how compatible it is with GRFs and other such, if at all. I feel it's also needed that there is a more simpler chart in-game when you click on AI.
Other thoughts, but definitely not doable by me. Is it possible to make AIs checkpoint themselves every so often. Then if a script errors, it can be, either automatically or otherwise, reset back to the state it was in at that checkpoint, tiles, units and all. Although I suspect in this state, it wouldn't work because the script will, unless they've got a bit of random to it, just make the same decisions on the same situations and crash again? I dunno, just a thought to try making a script error not as fatal.
Lastly of course, is it worth me doing any of this, keeping some abandoned AI's updated to some degree, do people actually use the AI? I feel the way it is now, AIs are a bit of a scary prospect for new players as you don't know precisely what you're getting into.
On a different note, is it possible to make links on the script output clickable? Some AI's have links like this to report bugs, but making them non-clickable doesn't encourage reports and therefore repairs.