Bananas - Python client?

[michalc]

I'm writing something I'm calling OpenTTDLab https://github.com/michalc/OpenTTDLab (really early version: proof of concept really) to run sets of experiments using OpenTTD from Python, e.g. from JupyterLab.

I would like to make it really easy to use it with AIs published at https://bananas.openttd.org/, so without having to even fire up OpenTTD "manually" to download them.

Would it be acceptable for me to write a Python client to do this? I note at https://bananas.openttd.org/ it does say "For downloading content, please use OpenTTD's built-in content downloader", so I'm not sure if I should really.

[TrueBrain]

What a cool project! Do keep us updated on how it is going, and how you are using it.

For your questions, there are a few possible answer. We have been willing to add in-game the ability to automatically download all missing content from BaNaNaS, if you load a game that is missing any. So you can just do ./openttd -g <savegame> -DOWNLOADSTUFF, and that it makes it happen. Via the UI it is already possible, but not in-game yet.
But, as that goes, nobody actually implemented that yet, so we don't have that yet.

For downloading yourself via a Python client or what-ever, there isn't any issue to do so. Our ToS allows you to download the latest version of content; and although the idea of course is that it is done via the OpenTTD client, that can't be guarded.

That said, if you go that path, I do have three requests:

1. cache as much as possible. Please do not download on every run all the files you need. Downloads costs us money, we live on donations, so please keep it reasonable.
2. please use the HTTPS to download the actual files. Although the initial handshake is via a custom TCP protocol, once you got the IDs of the files you are looking for, there is a dance via HTTPS to get the actual download URLs. Please invest time in using this, over downloading via TCP (although that is also possible). The reason here again is money. Downloads via TCP are handled via AWS, for which we pay 0,09 dollar per GB transferred. It is REALLY expensive.
3. please never download ALL content from BaNaNaS. Although there are a few hurdles to prevent you to do that accidentally, please also don't try to look for ways to do this. Again the reason here is cost.

In other words: be a good neighbour, and go nuts. Be a bad neighbour .. and I might get mad :P

If you need any help deciphering how the TCP protocol and HTTPS dance works, let me know. Easiest is via Discord, but via this Discussion is also fine.

Happy hacking!

[michalc]

This is incredibly helpful.

The requests are very reasonable: will of course abide (and have implemented some caching already in OpenTTDLab, but more to be done).

Do keep us updated on how it is going, and how you are using it.

Will do!

If you need any help deciphering how the TCP protocol and HTTPS dance works, let me know

I may take you up on that :-)

Thank you!!

[michalc]

Have now implemented a basic/rough version: https://github.com/michalc/OpenTTDLab/blob/60f848a30073b92e012837177d642f5030f4ff86/openttdlab.py#L272

[TrueBrain]

Looking good :)

[TrueBrain]

To maybe inspire you a bit what we have been wanting for years now, but never really got the time to build, or didn't take the time to build:

When we release a new OpenTTD version, it is very hard for us to see if any regression creeped in. Yes, we have unit tests. Yes, we have regression tests. But still .. regressions can happen, and over 3 axis:

• CPU / memory degradation (lower game fps.
• Rendering degradation (lower video fps)
• Crashing savegames that used to work.

For the first, I wrote some scripts last year, where I run a whole set of savegames from time to time to benchmark their performance. It is just a collection of scripts linked together, outputting a csv with CPU / memory usage for each game. This I load into Grafana to create graphs. Source can be found here: https://github.com/TrueBrain/OpenTTD-performance

For the second, we have nothing. Well, yes, our own experience when starting the game. But nothing systematic, or which tells us what commit made it better / worse.

The last one, once upon a time I started https://github.com/TrueBrain/OpenTTD-savegames, and LordAro also gave it a few gos, but never anything came from it. From all three, this for us is the most important one: to know if a new commit / version breaks savegames that used to work.

For all this, we kinda miss a framework to help us do this. We can manage the compute unit (VM or whatever) to run it on, but we do not have any tooling assisting us running this every commit, or every week, or whatever.

Then I do not know your actual goal of OpenTTDLab, but just in case you are looking for some inspiration what us, OpenTTD Developers, would really help out .. an idea / insight in what is troubling us. Anything you can do in that regard to help us out, would be greatly appreciated. Even if it is: this tool isn't actually designed for it, but if you do A B C, you can use it for that goal, would be perfect.

Not saying you need to do anything with this comment; but if it so happens to align with anything you are doing, do let me know please :)

[michalc]

So my envisioned use is a bit different... it's more on the science-y/research side of things, especially AIs. I'm wanting to make it as easy as possible to replicate/reproduce research results that use OpenTTD. There are a few published papers that use OpenTTD, but it's hard to reproduce their results (or even in some cases I suspect, for the authors to replicate their own results because it's clear there were manual steps involved)

Or at least, do that for a few simple cases/take steps towards it being easier.

But: I think there is (or rather, could be) overlap between our goals: both of these things are all about reproducibility/determinism... I will have a ponder. I did want to make OpenTTDLab have lots of defaults but then also be flexible, so maybe that flexibility could lend itself to regression tests...