openttd-client/docs/PROTOCOL.md

# Protocol Internals

This client supports the modern OpenTTD Game Port protocol (TCP 3979), specifically as implemented in JGRPP.

## X25519 PAKE Authentication
OpenTTD 14+ and JGRPP use a Password-Authenticated Key Exchange to prevent plaintext password leakage.

### Key Derivation (KDF)
We use **Blake2b** (64-byte digest) to derive two 32-byte session keys.
- **Input:** `SharedSecret (32)` + `ServerPublicKey (32)` + `ClientPublicKey (32)` + `Password (string)`
- **Output:**
  - `0..31`: Client-to-Server Key
  - `32..63`: Server-to-Client Key

### Handshake Nonces
The server provides a 24-byte nonce in the `ServerAuthenticationRequest`. This nonce is used for the AEAD challenge during the auth response and for the initial stream encryption setup.

## Admin Network (TCP 3977)
The Admin Network allows external applications to monitor and control the server. It supports both unsecured and secure (X25519 PAKE) authentication.

### Secure Authentication
Similar to the Game Port, the Admin Network uses X25519 PAKE for secure authentication. 
- **Packet:** `AdminJoinSecure` starts the handshake.
- **Encryption:** Once enabled via `ServerEnableEncryption`, all subsequent traffic is encrypted using XChaCha20-Poly1305.

### Update Frequencies
Admins can subscribe to various updates (Date, Client Info, Company Info, etc.) at different frequencies (Poll, Daily, Weekly, Monthly, Quarterly, Annually, Automatic).

### Vehicle Listing
The Admin Network has no native packet or `AdminUpdateType` for listing individual vehicles — `ServerCompanyStats` only reports aggregate per-company vehicle counts (trains/lorries/buses/planes/ships). To retrieve an actual vehicle list, this client sends a `list_vehicles` command over the GameScript JSON channel (`AdminGamescript`/`ServerGamescript`) via `list_vehicles()`. This requires a companion GameScript running server-side that understands the `list_vehicles` command and replies with vehicle data through `ServerGamescript`.

**Important:** the server only forwards `ServerGamescript` packets to admins that have subscribed with `update_frequency(AdminUpdateType.Gamescript, AdminUpdateFrequency.Automatic)` (enforced server-side in `NetworkAdminGameScript`, which checks `update_frequency[ADMIN_UPDATE_GAMESCRIPT]`). Call `update_frequency()` for `Gamescript` before `list_vehicles()`, or the response is silently dropped.

When a `list_vehicles` request carries a `request_id` field, the AdminBridge GameScript echoes it back in the reply (backward compatible: absent otherwise).

### Timetable Query
The stock GameScript API has no timetable getters, so this project patches the server (see `docker/patches/`) to add read-only getters to `GSOrder` (`GetTimetableWaitTime`, `GetTimetableTravelTime`, `IsWaitTimetabled`, `IsTravelTimetabled`, `IsWaitFixed`, `IsTravelFixed`, `GetLeaveType`, `GetTimetableMaxSpeed`, `GetTimetableLateness`, `GetTimetableStartTick`, `GetCurrentOrderTime`, `GetTimetableTotalDuration`). On top of that, `get_timetable()` sends a request over the same GameScript JSON channel as vehicle listing and awaits the correlated reply — an **authoritative snapshot** of the live game state, unlike the passive observer on the game port (see below).

- **Request:** `{"command": "get_timetable", "vehicle_id": N, "request_id": X}` — `request_id` is a client-side monotonic counter used to match the reply to the awaiting caller.
- **Reply (success):** `{"command": "get_timetable", "vehicle_id": N, "request_id": X, "lateness": ..., "start_tick": ..., "current_order_time": ..., "total_duration": ..., "orders": [{"position", "wait_time", "travel_time", "wait_timetabled", "travel_timetabled", "wait_fixed", "travel_fixed", "leave_type", "max_speed"}, ...]}` (booleans encoded as 0/1).
- **Reply (error):** same envelope with an `"error"` field instead of the data: `"invalid_vehicle"` (no such vehicle) or `"response_too_large"` (the reply exceeded the admin packet size limit, possible with very many orders). `get_timetable()` raises `ValueError` for these.

Replies carrying a `request_id` that matches a pending request resolve that request and are **not** delivered to the `on_gamescript` callback; all other `ServerGamescript` traffic reaches the callback unchanged. The same `update_frequency` subscription requirement applies (`get_timetable()` subscribes automatically on first use). Since GameScripts do not tick while the game is paused, a query against a paused server times out (`asyncio.TimeoutError`).

### Station Listing
Like vehicles, the Admin Network has no native packet for enumerating individual stations (`ServerCompanyStats` only reports an aggregate per-company station count). `list_stations()` sends a `list_stations` command over the same GameScript JSON channel and the companion AdminBridge GameScript replies with station data through `ServerGamescript` (`{"command": "list_stations", "stations": [{"id", "name", ...}, ...]}`). It is fire-and-forget, so the reply is delivered to the `on_gamescript` callback — subscribe to `Gamescript` updates first, exactly as for `list_vehicles()`. An optional `company_id` field scopes the list to one company.

### Station Query
`get_station()` fetches an authoritative snapshot of one station's live cargo state over the same GameScript JSON channel, awaiting the correlated reply — the station analogue of `get_timetable()`. Unlike timetables, the getters it relies on (`GSStation.GetCargoWaiting`, `GetCargoPlanned`, `GetCargoRating`) are part of the **stock** GameScript API, so this needs no server patch. The per-cargo reply exposes both the **real-time** amount currently waiting and the **planned** amount routed through the station by the cargodist link graph.

- **Request:** `{"command": "get_station", "station_id": N, "request_id": X}` — `request_id` is the same client-side monotonic counter used by `get_timetable()`, matching the reply to the awaiting caller.
- **Reply (success):** `{"command": "get_station", "station_id": N, "request_id": X, "name": ..., "location": <tile>, "owner": <company_id>, "cargo": [{"cargo_id", "waiting", "planned", "rating"}, ...]}` — `waiting` is the real-time units at the station (`GetCargoWaiting`), `planned` is the link-graph planned flow (`GetCargoPlanned`, 0 when cargo distribution is off for that cargo), and `rating` is the acceptance rating as a percentage (0-100, `GetCargoRating`) or `null` when the station has no rating for that cargo yet. Only cargo the station has handled appears.
- **Reply (error):** same envelope with an `"error"` field instead of the data: `"invalid_station"` (no such station) or `"response_too_large"`. `get_station()` raises `ValueError` for these.

Correlation, the `update_frequency` subscription requirement (auto-subscribed on first use), and the paused-game timeout behave exactly as described for the Timetable Query above.

### Station Cargo Flow Breakdown
`get_station_cargo()` drills into a single cargo type at one station and returns how its **waiting** (real-time) and **planned** amounts split across the cargo distribution (cargodist) link graph. Cargodist tags every unit with a **source** station (`from`, where it was first loaded) and a **next hop** (`via`, the next station it travels to toward its final destination). There is no per-station store of the *final* destination — the routing destination is the next hop — so the breakdown is offered along those two axes. The GS reads them with the stock `GSStation.GetCargoWaiting{From,Via,FromVia}` / `GetCargoPlanned{From,Via,FromVia}` scalars and the `GSStationList_Cargo{Waiting,Planned}By{From,Via}` (and `…ViaByFrom` / `…FromByVia`) list classes — again no server patch.

- **Request:** `{"command": "get_station_cargo", "station_id": N, "cargo_id": C, "request_id": X}`, optionally with `"from_station"` and/or `"via_station"` filters.
- **Reply (success):** `{"command": "get_station_cargo", "station_id": N, "cargo_id": C, "request_id": X, "waiting": ..., "planned": ..., "waiting_by_from": [{"station", "amount"}, ...], "planned_by_from": [...], "waiting_by_via": [...], "planned_by_via": [...]}`. `waiting`/`planned` are the (filtered) totals; each `*_by_from` list groups by source station and each `*_by_via` list groups by next hop (zero-amount entries omitted). A `station` of `65535` (`STATION_INVALID`) means the source was deleted or — as a next hop — the cargo has no onward routing / is consumed here (also the only next hop for cargo using manual, non-cargodist distribution). Any supplied `from_station`/`via_station` filter is echoed back.
- **Filters:** `via_station` restricts the query (and the `*_by_from` breakdowns) to cargo whose next hop is that station; `from_station` restricts it (and the `*_by_via` breakdowns) to cargo from that source; supplying both makes `waiting`/`planned` the exact source-and-next-hop amount. Pass `65535` to target `STATION_INVALID`.
- **Reply (error):** same envelope with an `"error"` field: `"invalid_station"`, `"invalid_cargo"`, `"invalid_from_station"`/`"invalid_via_station"` (a filter that is neither a valid station nor `STATION_INVALID`), or `"response_too_large"`. `get_station_cargo()` raises `ValueError` for these.

### Cargo Listing
Cargo appears in the replies above (and in the `cargo_waiting` / vehicle events) as a bare numeric `cargo_id`. Those ids index the cargo table the loaded NewGRFs build for the running game, so they are **not stable across games** — the same id can be coal in one save and grain in another. `list_cargo()` resolves them, sending a `list_cargo` command over the same GameScript JSON channel and awaiting the correlated reply. The GS reads the table with the stock `GSCargoList`, `GSCargo.GetCargoLabel` and `GSCargo.IsFreight`, so this needs no server patch.

- **Request:** `{"command": "list_cargo", "request_id": X}`.
- **Reply (success):** `{"command": "list_cargo", "request_id": X, "cargo": [{"cargo_id": N, "label": "COAL", "freight": 0|1}, ...]}` — one entry per cargo type in the game, in `GSCargoList` order rather than by id, so index the list by `cargo_id`. `label` is the four-character NewGRF cargo label (`GetCargoLabel`, underscore-padded: `OIL_`), or `""` if the GS could not read it; `freight` is 1 for freight cargo and 0 for the rest (passengers, mail, …).
- **Reply (error):** the GS defines no error for this command, so `list_cargo()` never raises `ValueError` — only `asyncio.TimeoutError` (paused game, GS not loaded) and `ConnectionError`.

Correlation and the `update_frequency` subscription requirement (auto-subscribed on first use) are as described for the Timetable Query above. Unlike `list_vehicles()`/`list_stations()` — which are fire-and-forget despite the similar name — this one is awaitable and its reply does **not** reach the `on_gamescript` callback.

### Dispatch Query
`get_dispatch()` fetches an authoritative snapshot of a vehicle's **scheduled dispatch** state over the GameScript JSON channel, the vehicle analogue of `get_timetable()` for JGRPP's scheduled dispatch feature. Like the timetable getters, the dispatch getters it relies on are added by a **server patch** (`docker/patches/0002-*`, adding `GSOrder.GetScheduledDispatch*` / `IsScheduledDispatchEnabled`), so it needs the patched JGRPP build. Correlation, the auto-subscribe, and the paused-game timeout behave exactly as for the Timetable Query.

- **Request:** `{"command": "get_dispatch", "vehicle_id": N, "request_id": X}`.
- **Reply (success):** `{"command": "get_dispatch", "vehicle_id": N, "request_id": X, "enabled": 0|1, "schedules": [{"index", "duration", "start_tick", "delay", "reuse_slots", "slots": [{"offset", "flags"}, ...]}, ...]}`. `enabled` is whether scheduled dispatch is turned on for the vehicle; each schedule reports its `duration` (ticks), `start_tick`, `delay` (max allowed delay), `reuse_slots` (0/1) and its `slots` (each a departure `offset` within the duration plus a 16-bit `flags` word). These are the same schedules and slots edited by the game-port dispatch methods.
- **Reply (error):** same envelope with an `"error"` field: `"invalid_vehicle"` or `"response_too_large"`. `get_dispatch()` raises `ValueError` for these.

### Game Events
Every other GameScript command above is request/reply. `subscribe_events()` instead opens a **push** stream: the AdminBridge GameScript sends event batches over `ServerGamescript` as things happen, unsolicited and without a `request_id`. For the calling API and the semantics of each kind, see the [Game Events Usage Guide](EVENTS.md).

- **Subscribe request:** `{"command": "subscribe_events", "request_id": X}` plus any of the optional narrowing fields `"events"` (array of kind strings), `"interval"` (ticks between state samples, default 10), `"company_id"`, `"vehicles"`, `"stations"`, `"cargo"` (arrays of ids), `"min_cargo_delta"` (default 1) and `"include_cargo"` (bool, default true).
- **Subscribe reply:** `{"command": "subscribe_events", "request_id": X, "events": [accepted kinds in catalogue order], "interval": N}`, or the same envelope with an `"error"` field: `"unknown_event"` (plus the offending `"event"`), `"invalid_interval"`, `"invalid_min_cargo_delta"`, or `"invalid_cargo"` (plus the offending `"cargo_id"`). `subscribe_events()` raises `ValueError` for these. Subscribing replaces any previous subscription and resets the bridge's sampling baseline.
- **Unsubscribe:** `{"command": "unsubscribe_events", "request_id": X}` → `{"command": "unsubscribe_events", "request_id": X, "events": []}`. This also drops the sampling state.
- **Event batch (unsolicited):** `{"command": "events", "events": [{"event": <kind>, "tick": T, ...}, ...]}`. Batches carry at most 24 events per packet, so one poll can produce several; a poll that generated more than 200 events is truncated and followed by a single `{"event": "events_dropped", "count": N}` entry. Because these batches carry no `request_id`, `receive_ServerGamescript` routes them to the event consumers (`on_event` / `wait_for_event()`) instead of the generic `on_gamescript` callback; every other GameScript payload reaches `on_gamescript` unchanged. The usual `update_frequency(Gamescript, Automatic)` subscription applies and is set up automatically by `subscribe_events()`.

Event kinds come from two sources. `vehicle_arrive`, `vehicle_depart` and `cargo_waiting` are **synthesised** by the bridge, because the engine raises no GameScript event for them: every `interval` ticks it samples `GSVehicle.GetState`/`GetLocation` for the watched vehicles and `GSStation.GetCargoWaiting` for the watched station-cargo pairs, and emits an event per change against the previous sample (so a stop shorter than the interval is never reported, and the first sample after subscribing only sets a baseline). All the remaining kinds — `vehicle_crashed`, `station_first_vehicle`, `industry_open`/`industry_close`, `town_founded`, `company_new`/`company_in_trouble`/`company_bankrupt` and the four `subsidy_*` kinds — are engine events forwarded verbatim from `GSEventController`. Only those the engine actually raises for a **deity** script are available: `ET_VEHICLE_LOST`, `ET_VEHICLE_WAITING_IN_DEPOT` and `ET_VEHICLE_UNPROFITABLE` are `@api ai` only and can never reach a GameScript, so the bridge does not offer them. Unlike the timetable and dispatch queries, none of this needs a server patch — every getter used is stock GameScript API.

## Vehicle Orders & Timetables (Game Port DoCommands)
Unlike vehicle listing, a vehicle's order list, timetables and scheduled dispatch have no writable GameScript API surface (this project adds read-only timetable and dispatch getters via server patches — see "Timetable Query" and "Dispatch Query" above). Reading and modifying them requires real engine commands (`DoCommand`s) sent over the **game port** (TCP 3979) via `ClientCommand`/`ServerCommand` packets, not the Admin Network. This section covers the wire format; for how to call the methods and what each parameter means, see the [Vehicle Timetables Usage Guide](TIMETABLES.md).

### Command envelope
Both `ClientCommand` and `ServerCommand` share this body: `company (uint8)`, `cmd (uint16 LE, index into the `Commands` enum)`, `error_msg (uint16 LE, StringID, use 0)`, `tile (uint32 LE, always 0 for these commands)`, `payload_len (uint16 LE)`, `payload (payload_len bytes)`, `callback (uint8, use 0)`, `callback_param (uint32 LE, only present if callback != 0)`. `ServerCommand` additionally appends `frame (uint32 LE)` and `my_cmd (uint8 bool)`, and is a **broadcast echo of the request** (no success/failure code) sent to every joined client, not just the sender.

### Payload integer encoding
Command payload fields follow JGRPP's generic serialiser, which picks the wire width from the C++ type's **size**: types of ≤1 byte are sent as a fixed `uint8`, exactly 2 bytes as a fixed `uint16` (LE), and 4/8-byte types as a variable-length **varuint** (`write_varuint`/`read_varuint` in `protocol.py` — a UTF-8-like prefix encoding, not LEB128; signed fields use zigzag via `write_varuint_signed`/`read_varuint_signed`). This is why `VehicleID` (a 4-byte pool id) is a varuint while `VehicleOrderID` (a `uint16`) is a fixed `uint16`.

### Command IDs and payload tuples
| Method | `cmd` | payload |
|---|---|---|
| `add_order()` | 52 (`InsertOrder`) | `VehicleID (varuint), sel_ord (uint16), order_type (uint8), order_flags (uint16), DestinationID (uint16)` |
| `remove_order()` | 51 (`DeleteOrder`) | `VehicleID (varuint), VehicleOrderID (uint16)` |
| `change_timetable()` | 174 (`ChangeTimetable`) | `VehicleID (varuint), VehicleOrderID (uint16), ModifyTimetableFlags (uint8), value (varuint), ModifyTimetableCtrlFlags (uint8)` |
| `set_vehicle_on_time()` | 176 (`SetVehicleOnTime`) | `VehicleID (varuint), apply_to_group (uint8 bool)` |
| `autofill_timetable()` | 177 (`AutofillTimetable`) | `VehicleID (varuint), bool (uint8), bool (uint8)` |
| `set_timetable_start()` | 180 (`SetTimetableStart`) | `VehicleID (varuint), bool (uint8), StateTicks (signed varuint)` |
| `set_scheduled_dispatch()` | 205 (`SchDispatch`) | `VehicleID (varuint), enabled (uint8 bool)` |
| `add_dispatch_slot()` | 206 (`SchDispatchAdd`) | `VehicleID (varuint), schedule_index (varuint), offset (varuint), interval (varuint), extra_slots (varuint), slot_flags (uint16), route_id (uint8)` |
| `remove_dispatch_slot()` | 207 (`SchDispatchRemove`) | `VehicleID (varuint), schedule_index (varuint), offset (varuint)` |
| `set_dispatch_duration()` | 208 (`SchDispatchSetDuration`) | `VehicleID (varuint), schedule_index (varuint), duration (varuint)` |
| `set_dispatch_start_date()` | 209 (`SchDispatchSetStartDate`) | `VehicleID (varuint), schedule_index (varuint), StateTicks (signed varuint)` |
| `clear_dispatch_schedule()` | 213 (`SchDispatchClear`) | `VehicleID (varuint), schedule_index (varuint)` |
| `add_dispatch_schedule()` | 214 (`SchDispatchAddNewSchedule`) | `VehicleID (varuint), StateTicks (signed varuint), duration (varuint)` |
| `remove_dispatch_schedule()` | 215 (`SchDispatchRemoveSchedule`) | `VehicleID (varuint), schedule_index (varuint)` |

### Scheduled dispatch (JGRPP)
A vehicle's order list can carry several **dispatch schedules**, each with a duration, a start tick and a set of departure **slots** (offsets within the duration). The methods above edit them over the game port (`add_dispatch_schedule()`/`remove_dispatch_schedule()` create and delete schedules; `add_dispatch_slot()`/`remove_dispatch_slot()`/`clear_dispatch_schedule()` manage a schedule's slots; `set_dispatch_duration()`/`set_dispatch_start_date()` adjust a schedule; `set_scheduled_dispatch()` toggles the feature for the vehicle). `add_dispatch_slot()` can add several evenly spaced slots at once via its `interval`/`extra_slots` parameters. The stock JGRPP command set covers ~22 dispatch commands (routes, departure tags, per-slot flags, adjust/swap/duplicate, …); the client implements this common core. There is no game-port read; for an authoritative view of the resulting schedules use the Admin Network's `get_dispatch()` (see "Dispatch Query" below).

### Adding & removing orders
`add_order()` issues `CMD_INSERT_ORDER`, which appends a new order before `sel_ord` (pass `0xFFFF`/`INVALID_VEH_ORDER_ID` to append to the end). The client currently builds "go to station" orders only. The `order_type` byte is bit-packed: **bits 0-3** hold the `OrderType` (`1` = `OT_GOTO_STATION`), **bits 4-5** the `OrderStopLocation`, and **bits 6-7** the `OrderNonStopFlags`. The stop location defaults to `PlatformFarEnd` (`2`) because near-end/middle/through are **train-only** and the server rejects (`CMD_ERROR`, no state change) any other value for road vehicles, ships, or aircraft. `order_flags` is the 16-bit load/unload word (`0` = load-if-possible + unload-if-possible). `DestinationID` is the target `StationID`. `remove_order()` issues `CMD_DELETE_ORDER` for the order at a given position. Both are **broadcast** back as `ServerCommand` like any DoCommand; the client does not currently decode those echoes into observed order state, so verify results via the admin `get_timetable()` order count.

### Ownership requirement
A command is rejected unless it's issued by the company that owns the target vehicle — join that company via `join_company()` with a real company id (not 255/spectator) before calling any order or timetable method. A malformed or wrong-company packet is treated as illegal and the client is kicked; a well-formed command that merely fails validation (e.g. an order the vehicle can't serve) is silently dropped with no state change and no kick.

### Reading timetables — no query command exists on the game port
There is no getter `DoCommand` for orders/timetables anywhere in the protocol. `get_vehicle_timetable()` works by passively decoding `ServerCommand` broadcasts (including the sender's own) as they arrive — it only reflects **changes made after the client joined**. A vehicle's pre-existing timetable (set before this client connected) is invisible until something changes it again; seeing it upfront would require parsing the `ORDR`/`VEHS` chunks of the initial savegame transfer (`ServerMapData`), which this client does not implement. For an authoritative read, use the Admin Network's `get_timetable()` instead (see "Timetable Query" above).

## Stream Encryption (AEAD)
Once `ServerEnableEncryption` is received, all subsequent packets use **XChaCha20-Poly1305** (Authenticated Encryption with Associated Data).

### Encrypted Packet Format
On the wire, encrypted packets have the following structure:
1.  **Length (2 bytes):** Big-endian uint16 of the *entire* remaining packet.
2.  **MAC (16 bytes):** The Poly1305 authentication tag.
3.  **Ciphertext (variable):** The encrypted payload.

### Decryption Logic
The `OpenTTDProtocol` layer uses an `IncrementalAuthenticatedEncryption` state from the Monocypher library. It maintains the nonce state internally. If a MAC check fails (indicating corruption or a wrong key), the client immediately closes the connection (`SocketClosed`).

## Keep-Alive (Simulation Synchronization)
OpenTTD is a lockstep simulation. The server sends `ServerFrame` packets periodically.
- **Client Requirement:** You must respond with a `ClientAck` containing the frame number and a one-time `token` provided in the frame packet.
- **Timeout:** If the server does not receive an ACK for several in-game days, it will disconnect the client with error code 17 (`TimeoutComputer`).