Order/timetable data export: request for comments

[merni-ns]

Order/timetable data export: request for comments

Exporting order lists/timetables to some kind of file is a feature that has been discussed several times. This would allow players to analyse and design timetables in more detail and ease using external tools such as a spreadsheet or more advanced programs.

Now that a JSON library is included in OpenTTD (since #11235 and #11290), JSON is the obvious format for such an export. I intend to work on a PR to enable exporting of order lists and timetables to JSON. There will be a button to export the timetable of a particular vehicle, and perhaps a console command or so to export the timetables of all vehicles in the game as well.

The JSON file can be converted to any other useful format or otherwise used by player-developed tools; that is outside the scope here. But this requires a standard format for the JSON, and before that, agreement on what should be included. Here is a preliminary proposal for that.

Data to be included

• Name of the vehicle
• Whether it shares orders or not
• Timetable start date
• A list of the orders and travels between them, each with its index, as follows.
• For each station order:
  • The station name.
  • The action on load and on unload (if possible, full load, etc)
  • Whether the order is go, go via, go non-stop, go non-stop via.
  • Stop location (near, far, middle)
  • Wait time for timetable.
• For each waypoint order: the waypoint name
• For each depot order:
  • The depot name.
  • The action at the depot (service only, service and stop, stop)
• For each conditional order:
  • The variable checked
  • The comparison made
  • The value compared to
  • The jump target
  • The travel time (in case the condition is true)
  • The speed limit (in case the condition is true)
• For each travel between manual orders (as shown in the timetable GUI):
  • The travel time
  • The speed limit

General points/questions

Comments are welcome on anything here, and in particular on the following points:

• I plan to exclude implicit orders. Is this a good thing, or is there some use for including them in an export?
  • Note that as far as timetabling is concerned implicit orders are ignored. There are no wait times for them, and travel time can only be set for the last travel before a manual order, which applies to the whole section between manual orders.
• Should the timetable start date be part of the export? The current lateness? The total time taken to complete it?
• Are there any other data which would be useful in an export?
• Should times always be in ticks or in the units displayed in the timetable GUI (which vary according to settings)?
• Is there a more useful way of indicating stations, waypoints and depots than by their name?
• For shared orders, should the number of vehicles sharing that order be indicated? Or even a list of them?
• If there is an "export orders of all vehicles", should shared orders be exported once for each vehicle or only once for the group?
• Please note that importing timetable data into the game is not something I plan to immediately work on, but perhaps it may happen in the future. In that case the format should allow for that (e,g., with regard to shared orders).

Sample JSON

Here is a very rough sample of how the JSON could look for a single vehicle. The orders may not really make sense, this is just to show the structure. The comments are for my own understanding, they are not part of the actual proposal.

JSON sample

{
	"vehicle": "Train 123",
	"shared": true,
	"orders": [
		{
			"index": 1,
			"type": "goto_station", // goto_station, goto_depot, goto_waypoint, conditional
			"station": "Trondford West",
			"unload": "unload_if_possible", // OrderUnloadFlags
			"load": "full_load_any", // OrderLoadFlags
			"stop": "no_stop_at_intermediate_stations", // OrderNonStopFlags
			"stop_location": "far_end", // OrderStopLocation
			"wait_time": 211, // in ticks
		},
		{
			"index": 2,
			"type": "travel",
			"travel_time": 225, // in ticks
			"speed_limit": 100, // in internal unit = 1.00584 km/h = 0.625 mph
		},
		{
			"index": 3,
			"type": "goto_waypoint",
			"waypoint": "Trondford Waypoint",
		},
		{
			"index": 4,
			"type": "conditional", 
			"variable": "reliability", // OrderConditionVariable
			"comparison": "more_than", // OrderConditionComparator
			"value": 25, // (all variables have either boolean or numeric values)
			"target": 7, // index
			"travel_time": 255, // considered if the condition is executed
			"speed_limit": 20, // do.
		},
		{
			"index": 5,
			"type": "travel",
		},
		{
			"index": 6,
			"type": "goto_depot", // only in case of a ODTFB_PART_OF_ORDERS
			"depot": "Trondford Train Depot",
			"action": "service_only", // OrderDepotActionFlags
		},
		{
			"index": 7,
			"type": "travel",
			"travel_time": 300,
		}
		
	]
}

[tabytac]

This looks good, just would like to mention features like Schedule Dispatch, not in the vanilla game but in JGRPP, and as so, could the implementation be flexible enough so that extra features like this could be added by patches (patch packs) and future updates. On the top of my head, the new unbunching that was discussed on the discord today and other JGRPP features like slots and counters to name a few.

[merni-ns]

The unbunching feature would just be another tag on depot orders with "unbunch": true or whatever.

With regard to patch features, this is really a point for patchpack authors to decide, and perhaps @JGRennison would like to comment on it. I do not have enough knowledge about JGRPP's slots, counters, etc to comment meaningfully. In scheduled dispatch, a single dispatch schedule is always assigned to a single order, so this should not be too difficult to handle within this structure -- again, it would be a (nested) tag added to station orders.

[ldpl]

Why invent new format when timetables can just be read out of the save file?

[TrueBrain]

Honestly, it is a bit weird argument. You can say the same about screenshots: no need to export it to PNG (or worse: BMP), as all the information is in the savegame.

It is about ease-of-access. And telling people to make a savegame, analyse that to digest JSON, analyse that again to put it back into perspective, having to know how internally timetables are constructed, is just not user-friendly. Instead, having an export which directly puts the timetable information in context, is.

[ldpl]

There is a lot of data you need to make a screenshot that isn't stored in the save. Sprites are in grf files, their offsets are hardcoded into the game and so on. But in the case of timetables it's all there.

From the ease of use POV savegames are the same if not better. You just open the save with you timetable tool or converter tool if timetable tool is external and can't be extended. Same as with json format except player will first need to find how to export that json from the game. Only difference is that it's a bit harder for timetable/converter tool developer to understand savegame format than json that's tailored for their tool.

[ldpl]

And json export is a pandora box. You can take a look at how much data cmclient export in json. It was all useful for one tool or another at some point. https://github.com/citymania-org/cmclient/blob/master/src/citymania/cm_export.cpp

[TrueBrain]

I am paraphrasing what you are trying to say here, see if I understand it better:

Basically, it seems you are saying: be careful with adding a JSON output for timetables, you might end up with all kind of different exports for one idea / tool or the other, which might become an unmaintainable mess?

[ldpl]

Yes, json format is effectively an external API with all the consequences that entails. Also having two ways of doing the same thing makes it confusing for tool developers. Should they still use savegame format or is it now deprecated and they need to use/request json exports?

[TrueBrain]

Really cool idea; if you need help implementing, you know where to find us :D

Before you know OpenTTD becomes a spreadsheet game :D