asyncapi: 3.0.0 id: 'urn:levimc:bedd:websocket' info: title: Be.D.D. WebSocket API version: 0.4.0 description: | The control API of a Minecraft Bedrock game (or a Bedrock Dedicated Server) that runs the BeDD LeviLamina mod. Test scripts, agents, bedboo and the BeDD manager all talk to a game this way. Every request is a JSON text message `{id, op, ...}`; the game answers each one with a JSON text message that echoes `id`: `{id, ok:true, ...}` or `{id, ok:false, error, code?, key?}`. One connection is served in order, one request at a time; open several connections to run requests side by side. The tap (`{op:"tap", action:"subscribe"}`) turns a connection into a binary stream of game packets. The operations are written from the client's point of view: a client `send`s requests and receives replies (and tap frames). ## Three operation modes All three are on at once (a config key set to `false` turns one feature off). `ping` lists them with `defaultMode` and `modes`. 1. **bedboo** (the default and the first choice): semantic game ops on the client's own input pipeline and network session: `world`, `look`, `input`, `slot`, `action`, `button`, `settings`, `send`, `stack`, `timescale`, `join`, `tap`. Small JSON in, small JSON out; this is what [bedboo](https://github.com/LiteLDev/bedboo)'s transport `bedd` uses. 2. **ui** (for menus and screens the bedboo ops do not cover): `ui` answers the screen in front as one line per control (JSON UI and Ore UI alike) and acts on a control by its `ref` (click, set text, scroll). 3. **input** (the last resort): what a user does with a keyboard and mouse, plus screenshots: `key`, `text`, `mouse`, `raw`, `release`, `flush`, `focus`, `window`, `screenshot`. Screenshots and pixel coordinates cost an agent far more tokens than a `ui` view and break when the layout changes: use them only when the other two modes cannot do the job. `ping`, `state`, `manager` and `close` belong to no mode (tag `system`). ## Finding a game A game writes `%LOCALAPPDATA%\BeDD\run\.json` (the root is `BEDD_HOME` when set) while its WebSocket server listens and deletes it at exit; ignore a file whose process is gone: `{protocol: 2, kind: "client"|"server", pid, url: "ws://127.0.0.1:/v1", token, instance, instanceDir, label, modVersion, startedAt}`. `BeDD.exe ps --json` lists the same endpoints. license: name: LGPL-3.0-or-later url: https://www.gnu.org/licenses/lgpl-3.0.html externalDocs: description: WebSocket API reference url: https://levimc.org/BeDD/reference/websocket tags: - name: bedboo description: Semantic game ops (the default mode). - name: ui description: The UI tree of the game's screens. - name: input description: Simulated keyboard and mouse, and screenshots (the last resort). - name: system description: Health, state, the manager link and closing the game. - name: server description: Ops of the server build (BeDD inside a Bedrock Dedicated Server). defaultContentType: application/json servers: local: host: '127.0.0.1:{port}' protocol: ws pathname: /v1 title: A game on this PC summary: The WebSocket server of one game (or server) running BeDD. description: | Bound to 127.0.0.1 only. The port is config `wsPort` (default 0: a free port the system picks at each start); read the `url` and `token` of the game from its discovery file `%LOCALAPPDATA%\BeDD\run\.json` or from `BeDD.exe ps --json`. Authenticate with `Authorization: Bearer ` or `?token=` (else HTTP 401). A handshake that carries an `Origin` header is refused with HTTP 403 unless that origin is listed in config `allowOrigins` (default empty), so a web page in a browser cannot reach a game. At most `wsMaxClients` (default 8) connections at once. Text messages are UTF-8 JSON; a message is at most 16 MiB; no extensions (no compression). variables: port: description: The port of the game's WebSocket server (from its discovery file). examples: - '53021' security: - $ref: '#/components/securitySchemes/bearer' - $ref: '#/components/securitySchemes/queryToken' channels: v1: title: /v1 summary: The one WebSocket of a game. description: | Every request, reply and tap frame goes over this connection (the server's pathname `/v1`). Requests are text messages; replies are text messages that echo the request's `id`; tap frames are binary messages, only on a connection that subscribed to the tap. messages: pingRequest: { $ref: '#/components/messages/pingRequest' } pingReply: { $ref: '#/components/messages/pingReply' } stateRequest: { $ref: '#/components/messages/stateRequest' } stateReply: { $ref: '#/components/messages/stateReply' } managerRequest: { $ref: '#/components/messages/managerRequest' } managerReply: { $ref: '#/components/messages/managerReply' } closeRequest: { $ref: '#/components/messages/closeRequest' } closeReply: { $ref: '#/components/messages/closeReply' } worldRequest: { $ref: '#/components/messages/worldRequest' } worldReply: { $ref: '#/components/messages/worldReply' } lookRequest: { $ref: '#/components/messages/lookRequest' } lookReply: { $ref: '#/components/messages/lookReply' } inputRequest: { $ref: '#/components/messages/inputRequest' } inputReply: { $ref: '#/components/messages/inputReply' } slotRequest: { $ref: '#/components/messages/slotRequest' } slotReply: { $ref: '#/components/messages/slotReply' } actionRequest: { $ref: '#/components/messages/actionRequest' } actionReply: { $ref: '#/components/messages/actionReply' } buttonRequest: { $ref: '#/components/messages/buttonRequest' } buttonReply: { $ref: '#/components/messages/buttonReply' } settingsRequest: { $ref: '#/components/messages/settingsRequest' } settingsReply: { $ref: '#/components/messages/settingsReply' } sendRequest: { $ref: '#/components/messages/sendRequest' } sendReply: { $ref: '#/components/messages/sendReply' } stackRequest: { $ref: '#/components/messages/stackRequest' } stackReply: { $ref: '#/components/messages/stackReply' } timescaleRequest: { $ref: '#/components/messages/timescaleRequest' } timescaleReply: { $ref: '#/components/messages/timescaleReply' } joinRequest: { $ref: '#/components/messages/joinRequest' } joinReply: { $ref: '#/components/messages/joinReply' } tapRequest: { $ref: '#/components/messages/tapRequest' } tapReply: { $ref: '#/components/messages/tapReply' } tapFrame: { $ref: '#/components/messages/tapFrame' } uiRequest: { $ref: '#/components/messages/uiRequest' } uiReply: { $ref: '#/components/messages/uiReply' } oreuiRequest: { $ref: '#/components/messages/oreuiRequest' } oreuiReply: { $ref: '#/components/messages/oreuiReply' } keyRequest: { $ref: '#/components/messages/keyRequest' } textRequest: { $ref: '#/components/messages/textRequest' } mouseRequest: { $ref: '#/components/messages/mouseRequest' } rawRequest: { $ref: '#/components/messages/rawRequest' } releaseRequest: { $ref: '#/components/messages/releaseRequest' } flushRequest: { $ref: '#/components/messages/flushRequest' } focusRequest: { $ref: '#/components/messages/focusRequest' } windowRequest: { $ref: '#/components/messages/windowRequest' } windowReply: { $ref: '#/components/messages/windowReply' } screenshotRequest: { $ref: '#/components/messages/screenshotRequest' } screenshotReply: { $ref: '#/components/messages/screenshotReply' } inputDoneReply: { $ref: '#/components/messages/inputDoneReply' } serverPingReply: { $ref: '#/components/messages/serverPingReply' } serverStateReply: { $ref: '#/components/messages/serverStateReply' } serverRequest: { $ref: '#/components/messages/serverRequest' } serverReply: { $ref: '#/components/messages/serverReply' } serverTimescaleReply: { $ref: '#/components/messages/serverTimescaleReply' } commandRequest: { $ref: '#/components/messages/commandRequest' } commandReply: { $ref: '#/components/messages/commandReply' } errorReply: { $ref: '#/components/messages/errorReply' } operations: # ---- system -------------------------------------------------------------------------------- ping: action: send channel: { $ref: '#/channels/v1' } title: ping summary: Health check; protocol version, modes and window. tags: [{ $ref: '#/components/tags/system' }] messages: [{ $ref: '#/channels/v1/messages/pingRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/pingReply' } - { $ref: '#/channels/v1/messages/serverPingReply' } - { $ref: '#/channels/v1/messages/errorReply' } state: action: send channel: { $ref: '#/channels/v1' } title: state summary: ping's fields plus counters, hooks, mode and connection. tags: [{ $ref: '#/components/tags/system' }] messages: [{ $ref: '#/channels/v1/messages/stateRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/stateReply' } - { $ref: '#/channels/v1/messages/serverStateReply' } - { $ref: '#/channels/v1/messages/errorReply' } manager: action: send channel: { $ref: '#/channels/v1' } title: manager summary: The BeDD manager attaches to the game (keeps it alive and off screen). tags: [{ $ref: '#/components/tags/system' }] messages: [{ $ref: '#/channels/v1/messages/managerRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/managerReply' } - { $ref: '#/channels/v1/messages/errorReply' } close: action: send channel: { $ref: '#/channels/v1' } title: close summary: Closes the game. tags: [{ $ref: '#/components/tags/system' }] messages: [{ $ref: '#/channels/v1/messages/closeRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/closeReply' } - { $ref: '#/channels/v1/messages/errorReply' } # ---- bedboo -------------------------------------------------------------------------------- world: action: send channel: { $ref: '#/channels/v1' } title: world summary: Snapshot of the screen and the local player. tags: [{ $ref: '#/components/tags/bedboo' }] messages: [{ $ref: '#/channels/v1/messages/worldRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/worldReply' } - { $ref: '#/channels/v1/messages/errorReply' } look: action: send channel: { $ref: '#/channels/v1' } title: look summary: Reads or turns the player's yaw and pitch. tags: [{ $ref: '#/components/tags/bedboo' }] messages: [{ $ref: '#/channels/v1/messages/lookRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/lookReply' } - { $ref: '#/channels/v1/messages/errorReply' } input: action: send channel: { $ref: '#/channels/v1' } title: input summary: Moves the player inside the game's input pipeline (a dead-man switch). tags: [{ $ref: '#/components/tags/bedboo' }] messages: [{ $ref: '#/channels/v1/messages/inputRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/inputReply' } - { $ref: '#/channels/v1/messages/errorReply' } slot: action: send channel: { $ref: '#/channels/v1' } title: slot summary: Reads or selects the hotbar slot. tags: [{ $ref: '#/components/tags/bedboo' }] messages: [{ $ref: '#/channels/v1/messages/slotRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/slotReply' } - { $ref: '#/channels/v1/messages/errorReply' } action: action: send channel: { $ref: '#/channels/v1' } title: action summary: Drop, drop all, inventory, perspective, pause, an arm swing. tags: [{ $ref: '#/components/tags/bedboo' }] messages: [{ $ref: '#/channels/v1/messages/actionRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/actionReply' } - { $ref: '#/channels/v1/messages/errorReply' } button: action: send channel: { $ref: '#/channels/v1' } title: button summary: Attack, use or pick, held for ticks or until a block changes. tags: [{ $ref: '#/components/tags/bedboo' }] messages: [{ $ref: '#/channels/v1/messages/buttonRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/buttonReply' } - { $ref: '#/channels/v1/messages/errorReply' } settings: action: send channel: { $ref: '#/channels/v1' } title: settings summary: Reads or sets the client's render distance; the client then requests that chunk radius itself. tags: [{ $ref: '#/components/tags/bedboo' }] messages: [{ $ref: '#/channels/v1/messages/settingsRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/settingsReply' } - { $ref: '#/channels/v1/messages/errorReply' } send: action: send channel: { $ref: '#/channels/v1' } title: send summary: Sends one serverbound packet through the local player (allow-listed ids only). tags: [{ $ref: '#/components/tags/bedboo' }] messages: [{ $ref: '#/channels/v1/messages/sendRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/sendReply' } - { $ref: '#/channels/v1/messages/errorReply' } stack: action: send channel: { $ref: '#/channels/v1' } title: stack summary: Item stack requests through the client, or what the client shows. tags: [{ $ref: '#/components/tags/bedboo' }] messages: [{ $ref: '#/channels/v1/messages/stackRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/stackReply' } - { $ref: '#/channels/v1/messages/errorReply' } timescale: action: send channel: { $ref: '#/channels/v1' } title: timescale summary: Reads or sets the game speed (and pause) of a world this game hosts, or of the server. tags: [{ $ref: '#/components/tags/bedboo' }, { $ref: '#/components/tags/server' }] messages: [{ $ref: '#/channels/v1/messages/timescaleRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/timescaleReply' } - { $ref: '#/channels/v1/messages/serverTimescaleReply' } - { $ref: '#/channels/v1/messages/errorReply' } join: action: send channel: { $ref: '#/channels/v1' } title: join summary: Joins a server by address, leaves the world (host null), or reports where the client plays. tags: [{ $ref: '#/components/tags/bedboo' }] messages: [{ $ref: '#/channels/v1/messages/joinRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/joinReply' } - { $ref: '#/channels/v1/messages/errorReply' } tap: action: send channel: { $ref: '#/channels/v1' } title: tap summary: Tap status, or subscribe this connection to the packet stream. tags: [{ $ref: '#/components/tags/bedboo' }] messages: [{ $ref: '#/channels/v1/messages/tapRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/tapReply' } - { $ref: '#/channels/v1/messages/errorReply' } tapFrames: action: receive channel: { $ref: '#/channels/v1' } title: tap frames summary: Binary messages on a connection subscribed to the tap, one frame each. tags: [{ $ref: '#/components/tags/bedboo' }] messages: [{ $ref: '#/channels/v1/messages/tapFrame' }] # ---- ui ------------------------------------------------------------------------------------ ui: action: send channel: { $ref: '#/channels/v1' } title: ui summary: Answers the screen in front as one line per control, or acts on a control by ref. tags: [{ $ref: '#/components/tags/ui' }] messages: [{ $ref: '#/channels/v1/messages/uiRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/uiReply' } - { $ref: '#/channels/v1/messages/errorReply' } oreui: action: send channel: { $ref: '#/channels/v1' } title: oreui summary: The raw DOM of the Ore UI view (`ui` already lists its controls). tags: [{ $ref: '#/components/tags/ui' }] messages: [{ $ref: '#/channels/v1/messages/oreuiRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/oreuiReply' } - { $ref: '#/channels/v1/messages/errorReply' } # ---- input --------------------------------------------------------------------------------- key: action: send channel: { $ref: '#/channels/v1' } title: key summary: Presses, holds or releases a key. tags: [{ $ref: '#/components/tags/input' }] messages: [{ $ref: '#/channels/v1/messages/keyRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/inputDoneReply' } - { $ref: '#/channels/v1/messages/errorReply' } text: action: send channel: { $ref: '#/channels/v1' } title: text summary: Types text (one WM_CHAR per UTF-16 unit). tags: [{ $ref: '#/components/tags/input' }] messages: [{ $ref: '#/channels/v1/messages/textRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/inputDoneReply' } - { $ref: '#/channels/v1/messages/errorReply' } mouse: action: send channel: { $ref: '#/channels/v1' } title: mouse summary: Moves the virtual cursor (to a point or a UI node) and clicks, presses or scrolls. tags: [{ $ref: '#/components/tags/input' }] messages: [{ $ref: '#/channels/v1/messages/mouseRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/inputDoneReply' } - { $ref: '#/channels/v1/messages/errorReply' } raw: action: send channel: { $ref: '#/channels/v1' } title: raw summary: Relative mouse movement (the camera while the mouse is grabbed) and buttons. tags: [{ $ref: '#/components/tags/input' }] messages: [{ $ref: '#/channels/v1/messages/rawRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/inputDoneReply' } - { $ref: '#/channels/v1/messages/errorReply' } release: action: send channel: { $ref: '#/channels/v1' } title: release summary: Releases every held key and button and ends the input override. tags: [{ $ref: '#/components/tags/input' }] messages: [{ $ref: '#/channels/v1/messages/releaseRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/inputDoneReply' } - { $ref: '#/channels/v1/messages/errorReply' } flush: action: send channel: { $ref: '#/channels/v1' } title: flush summary: Waits until everything posted was dispatched. tags: [{ $ref: '#/components/tags/input' }] messages: [{ $ref: '#/channels/v1/messages/flushRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/inputDoneReply' } - { $ref: '#/channels/v1/messages/errorReply' } focus: action: send channel: { $ref: '#/channels/v1' } title: focus summary: Re-sends the "focus gained" messages. tags: [{ $ref: '#/components/tags/input' }] messages: [{ $ref: '#/channels/v1/messages/focusRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/inputDoneReply' } - { $ref: '#/channels/v1/messages/errorReply' } window: action: send channel: { $ref: '#/channels/v1' } title: window summary: Sets the client area size; never activates the window. tags: [{ $ref: '#/components/tags/input' }] messages: [{ $ref: '#/channels/v1/messages/windowRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/windowReply' } - { $ref: '#/channels/v1/messages/errorReply' } screenshot: action: send channel: { $ref: '#/channels/v1' } title: screenshot summary: An image of the game window, even while it is covered or off screen. tags: [{ $ref: '#/components/tags/input' }] messages: [{ $ref: '#/channels/v1/messages/screenshotRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/screenshotReply' } - { $ref: '#/channels/v1/messages/errorReply' } # ---- server build -------------------------------------------------------------------------- server: action: send channel: { $ref: '#/channels/v1' } title: server summary: Server build only - the level's tick, tps, pause, port and players. tags: [{ $ref: '#/components/tags/server' }] messages: [{ $ref: '#/channels/v1/messages/serverRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/serverReply' } - { $ref: '#/channels/v1/messages/errorReply' } command: action: send channel: { $ref: '#/channels/v1' } title: command summary: Server build only - runs a command as the server console would. tags: [{ $ref: '#/components/tags/server' }] messages: [{ $ref: '#/channels/v1/messages/commandRequest' }] reply: channel: { $ref: '#/channels/v1' } messages: - { $ref: '#/channels/v1/messages/commandReply' } - { $ref: '#/channels/v1/messages/errorReply' } components: tags: bedboo: name: bedboo description: Semantic game ops (the default mode). ui: name: ui description: The UI tree of the game's screens. input: name: input description: Simulated keyboard and mouse, and screenshots (the last resort). system: name: system description: Health, state, the manager link and closing the game. server: name: server description: Ops of the server build. securitySchemes: bearer: type: http scheme: bearer description: | `Authorization: Bearer ` on the WebSocket handshake. The token (64 hex characters, 32 random bytes, new at every start) is in the game's discovery file. A wrong or missing token is HTTP 401. Any handshake with an `Origin` header not listed in config `allowOrigins` is HTTP 403. queryToken: type: httpApiKey name: token in: query description: | `ws://127.0.0.1:/v1?token=`, for clients that cannot set handshake headers (the browser-style `WebSocket` of Node and Deno). The same `Origin` rule applies. schemas: Id: description: Any JSON value; echoed in the reply as given. type: [string, number, 'null'] Ok: type: boolean const: true TimeoutMs: type: integer minimum: 0 description: How long to wait for the game before answering `timeout`. Point: type: object properties: x: { type: integer } y: { type: integer } Size: type: object properties: w: { type: integer } h: { type: integer } Vec3: type: object properties: x: { type: number } y: { type: number } z: { type: number } Rect: type: object description: Client-area pixels (or fractions of the client area for `boundsNorm`). required: [x, y, w, h] properties: x: { type: number } y: { type: number } w: { type: number } h: { type: number } Button: type: string enum: [left, right, middle] default: left ErrorCode: type: string description: | Why an op failed. Errors without a `code` (an unknown op, a malformed request, a game thread that did not process input in time) carry only `error`. - `disabled`: the op's config key is `false` (the reply names it in `key`), or BeDD is unloading. - `not_in_level`: the client is not in a world (no client tick within 20 tick intervals, at least 1 s); on the server build, the server has not started. - `refused`: the game refused the request (a packet id outside `sendAllow`, no block targeted for `releaseOn:"blockChange"`, ...). - `timeout`: the game did not get there within `timeoutMs` (it may still). - `bad_args`: missing or invalid fields, or bytes the game cannot read. - `busy`: something else holds it (a screen is open, a newer `look` replaced this one, a join is in progress, the tap already has a subscriber). - `stopped`: BeDD is unloading (the game closes). - `join_failed`: the client gave up joining and shows why. - `not_allowed`: the host is off this PC and config `loopbackOnly` keeps the game from it (not in `allowHosts`). - `unavailable`: a game hook the op needs did not install (see the game log). - `no_client`: the client instance does not exist yet. - `remote`: the world is on a server this game does not host (its speed cannot change from here). - `server_led`: that server runs BeDD and leads the game speed: change it there. - `bad_request`: the op cannot act on the request (an unknown `action`, a `ui` action without `ref`, `setText` without `text`, a binary message, `tap subscribe` / `manager attach` off a connection, `manager audio` / `manager input` from a connection that is not the attached manager). - `internal`: a `ui` action's input step answered unreadably (a bug: report it). - `stale_ref`: no such ref (its screen closed), or its control is gone or is another control now; `candidates` lists look-alike controls with their refs. - `occluded`: the control is hidden, outside the window, or covered at the point an action would hit (`occludedBy`). - `not_ready`: no UI to read yet (the game is loading, no scene), or the Ore UI page did not answer in time. - `unsupported`: the target cannot do that (a whole Ore UI page), or this game version's UI could not be read. enum: - disabled - not_in_level - refused - timeout - bad_args - busy - stopped - join_failed - not_allowed - unavailable - no_client - remote - server_led - bad_request - internal - stale_ref - occluded - not_ready - unsupported FeatureKey: type: string description: The config key that turns the op off when `false` (all default to `true`). enum: [gameHooks, tap, allowSend, allowStack, ui] Mode: type: string enum: [bedboo, input, ui] Request: type: object description: The envelope of every request. required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { type: string } Reply: type: object description: The envelope of every successful reply. required: [ok] properties: id: { $ref: '#/components/schemas/Id' } ok: { $ref: '#/components/schemas/Ok' } ErrorReply: type: object required: [ok, error] properties: id: { $ref: '#/components/schemas/Id' } ok: { type: boolean, const: false } error: { type: string, description: 'Lower-case English for humans; match on `code`, not on this.' } code: { $ref: '#/components/schemas/ErrorCode' } key: { $ref: '#/components/schemas/FeatureKey' } additionalProperties: true WindowInfo: type: object properties: hwnd: { type: integer, description: 'The game window handle (0 before it exists).' } managed: { type: boolean, description: 'A manager is attached.' } client: { $ref: '#/components/schemas/Size' } cursor: { $ref: '#/components/schemas/Point' } realForeground: { type: boolean, description: 'The game window really is in the foreground.' } audio: { type: boolean, description: 'The game''s sound plays on this PC (turned on by the manager, `manager` action `audio`).' } grabbed: { type: boolean, description: 'The game holds the mouse (in a world: it confines and re-centres the cursor and turns the camera with relative movement).' } Modes: type: object description: The ops of each mode. properties: bedboo: { type: array, items: { type: string } } input: { type: array, items: { type: string } } ui: { type: array, items: { type: string } } StackSlot: type: object properties: slot: { type: integer } item: { type: string } count: { type: integer } stackId: { type: integer } Effective: type: [object, 'null'] description: As that side's Minecraft reported after it last applied a change (null before the first). properties: scale: { type: number } paused: { type: boolean } ticksPerSecond: { type: number } UiNode: type: object properties: ref: { type: string, description: 'The control''s ref: the same while its screen lives, across reads and connections.' } role: { type: string, enum: [button, toggle, text_input, slider, scroll, text, item, group] } name: { type: string, description: 'JSON UI: the control name. Ore UI: aria-label, title, alt, placeholder, name or data-testid.' } text: { type: string, description: 'The text it shows.' } value: { description: 'A toggle state, slider value or text field content.' } state: type: object description: 'Flags such as visible, enabled, focused, checked, selected.' additionalProperties: { type: boolean } bounds: { $ref: '#/components/schemas/Rect' } boundsNorm: { $ref: '#/components/schemas/Rect' } actions: { type: array, items: { type: string, enum: [click, setText, scroll] } } children: { type: array, items: { type: string }, description: 'Refs of the child nodes.' } source: type: object description: 'Where the node comes from: `jsonui` (path, type), `oreui` (a whole Ore UI screen: url, route) or `dom` (an element of its page: path, type = tag, url, route).' properties: kind: { type: string, enum: [jsonui, oreui, dom] } path: { type: string } type: { type: string } url: { type: string } route: { type: string } additionalProperties: true hash: { type: string, description: 'Changes when the node changes.' } messages: # ---- system ------------------------------------------------------------------------------ pingRequest: name: ping title: ping request tags: [{ $ref: '#/components/tags/system' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: ping } examples: - name: ping payload: { id: 1, op: ping } pingReply: name: ping title: ping reply (game) tags: [{ $ref: '#/components/tags/system' }] payload: allOf: - $ref: '#/components/schemas/Reply' - $ref: '#/components/schemas/WindowInfo' - type: object required: [protocol, pid, defaultMode, modes] properties: protocol: { type: integer, const: 2 } kind: { type: string, enum: [client, server] } pid: { type: integer } defaultMode: { const: bedboo } modes: { $ref: '#/components/schemas/Modes' } examples: - name: ping payload: id: 1 ok: true protocol: 2 pid: 23116 defaultMode: bedboo modes: bedboo: [world, look, input, slot, action, button, send, stack, timescale, join, tap] input: [key, text, mouse, raw, release, flush, focus, window, screenshot] ui: [ui, oreui] hwnd: 4589122 managed: true client: { w: 1280, h: 720 } cursor: { x: 640, y: 360 } realForeground: false serverPingReply: name: ping title: ping reply (server build) tags: [{ $ref: '#/components/tags/server' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object required: [protocol, pid, target] properties: protocol: { type: integer, const: 2 } pid: { type: integer } target: { const: server } examples: - name: ping payload: { id: 1, ok: true, protocol: 2, pid: 8812, target: server } stateRequest: name: state title: state request tags: [{ $ref: '#/components/tags/system' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: state } examples: - name: state payload: { id: 2, op: state } stateReply: name: state title: state reply (game) tags: [{ $ref: '#/components/tags/system' }] payload: allOf: - $ref: '#/components/schemas/Reply' - $ref: '#/components/schemas/WindowInfo' - type: object properties: stats: type: object description: 'Counters: synthetic, swallowedInput, swallowedFocus, passed, realForegroundSeen, api, gameInput, loopbackRebinds, serverQueriesRefused, audio, tapLoopback, send, journal, sendsRefused.' additionalProperties: true clients: { type: integer, description: 'WebSocket clients connected now.' } held: type: array description: 'Keys and mouse buttons down in the virtual state (the left keys for Shift, Ctrl, Alt), each with the connection that pressed it.' items: type: object properties: vk: { type: integer } client: { type: integer, description: 'The connection id (0: none).' } hooks: type: object description: 'Each game hook a feature installed (false while the feature is off).' additionalProperties: { type: boolean } mode: { type: string, enum: [ClientOnly, ClientServer, Remote], description: 'Config `mode`.' } connection: type: object description: Where the client plays. properties: connection: { type: string, enum: [none, local, remote] } inLevel: { type: boolean } host: { type: string } port: { type: integer } examples: - name: state payload: id: 2 ok: true hwnd: 4589122 managed: true client: { w: 1280, h: 720 } clients: 2 mode: ClientOnly connection: { connection: local, inLevel: true, host: '', port: 0 } hooks: { netTap: true } stats: { synthetic: 412, swallowedInput: 0 } serverStateReply: name: state title: state reply (server build) tags: [{ $ref: '#/components/tags/server' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: target: { const: server } clients: { type: integer } loopbackOnly: { type: boolean } stats: type: object properties: loopbackRebinds: { type: integer } serverQueriesRefused: { type: integer } managerRequest: name: manager title: manager request summary: The BeDD manager GUI attaches to the game. description: | While a connection is attached the game window stays off screen, off the taskbar and out of Alt+Tab. With config `manager` (default `true`) a game that no manager attaches to within 30 s of its window existing, or whose attached connection drops, closes itself. With `manager:false` the window stays hidden and the game never closes on its own. Only the attached manager connection (otherwise `bad_request`): - `audio`: plays the game's sound on this PC (`on: true`) or silences it (`on: false`, the default; also when the manager's connection drops). - `input`: the user's keyboard and mouse while they control the game in the manager window, as `events` in order. Nothing waits (no flush, no hold); keys and buttons stay down until their up event, a `release` event, or the connection dropping. Replies `{ok, grabbed, audio}`. tags: [{ $ref: '#/components/tags/system' }] payload: type: object required: [op, action] properties: id: { $ref: '#/components/schemas/Id' } op: { const: manager } action: { type: string, enum: [attach, audio, input] } on: { type: boolean, description: 'audio: the sound on (true) or off (false).' } events: type: array description: 'input: the events, in order.' items: type: object required: [t] properties: t: type: string enum: [key, char, move, raw, button, wheel, release] description: | key: `vk` (left/right-specific for Shift, Ctrl, Alt), `down`. char: `c`, a UTF-16 code unit (WM_CHAR). move: `x`, `y` in client pixels. raw: `dx`, `dy` relative movement. button: `button` left|right|middle|x1|x2, `down`. wheel: `notches` (positive = away from the user). release: let go of everything this connection holds. vk: { type: integer, minimum: 1, maximum: 254 } down: { type: boolean, default: true } c: { type: integer, minimum: 1, maximum: 65535 } x: { type: integer } y: { type: integer } dx: { type: integer } dy: { type: integer } button: { type: string, enum: [left, right, middle, x1, x2] } notches: { type: integer } examples: - name: attach payload: { id: 1, op: manager, action: attach } - name: audio payload: { id: 2, op: manager, action: audio, on: true } - name: input payload: { id: 3, op: manager, action: input, events: [{ t: move, x: 640, y: 360 }, { t: key, vk: 87, down: true }, { t: raw, dx: 12, dy: -3 }] } managerReply: name: manager title: manager reply tags: [{ $ref: '#/components/tags/system' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: attached: { type: boolean } label: { type: string, description: 'From the env `BEDD_LABEL` of the game.' } audio: { type: boolean, description: 'The game''s sound plays on this PC.' } grabbed: { type: boolean, description: '`input` only: the game holds the mouse.' } additionalProperties: true closeRequest: name: close title: close request tags: [{ $ref: '#/components/tags/system' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: close } examples: - name: close payload: { id: 9, op: close } closeReply: name: close title: close reply description: Sent before the game starts closing; the connection then closes with it. tags: [{ $ref: '#/components/tags/system' }] payload: { $ref: '#/components/schemas/Reply' } # ---- bedboo ------------------------------------------------------------------------------ worldRequest: name: world title: world request tags: [{ $ref: '#/components/tags/bedboo' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: world } examples: - name: world payload: { id: 3, op: world } worldReply: name: world title: world reply description: Config key `gameHooks`. A snapshot the client thread takes (screen every frame, player every tick). tags: [{ $ref: '#/components/tags/bedboo' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: inLevel: { type: boolean } levelEpoch: { type: integer, description: '+1 per world joined.' } topScreen: { type: string, description: 'For example hud_screen, pause_screen.' } mouseGrabbed: { type: boolean } showingMenu: { type: boolean } inputMode: { type: string, enum: [mouse, touch, gamepad, undefined] } player: type: [object, 'null'] description: Null outside a world. properties: runtimeId: { type: integer } eye: { $ref: '#/components/schemas/Vec3' } rot: type: object properties: yaw: { type: number } pitch: { type: number } slot: { type: integer } tick: { type: integer, description: 'Client ticks seen.' } examples: - name: in a world payload: id: 3 ok: true inLevel: true levelEpoch: 1 topScreen: hud_screen mouseGrabbed: true showingMenu: false inputMode: mouse player: { runtimeId: 1, eye: { x: 0.5, y: 65.62, z: 0.5 }, rot: { yaw: 0, pitch: 0 }, slot: 0 } tick: 5120 lookRequest: name: look title: look request description: | Config key `gameHooks`. Without `yaw`/`pitch`: a probe. Degrees as in player_auth_input: yaw 0 = +Z, 90 = -X; pitch -90 = up, clamped to +-89.9. Replies once both are within 0.5 degrees. tags: [{ $ref: '#/components/tags/bedboo' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: look } yaw: { type: number } pitch: { type: number, minimum: -90, maximum: 90 } timeoutMs: allOf: [{ $ref: '#/components/schemas/TimeoutMs' }] default: 1000 maximum: 60000 examples: - name: look down a little payload: { id: 4, op: look, yaw: 90, pitch: 30 } lookReply: name: look title: look reply tags: [{ $ref: '#/components/tags/bedboo' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: inLevel: { type: boolean, description: 'Probe only.' } yaw: { type: number } pitch: { type: number } tick: { type: integer } turns: { type: integer, description: 'Ticks it took (action form).' } inputRequest: name: input title: input request description: | Config key `gameHooks`. Without fields: a probe. A dead-man switch: the override ends `ttlMs` after the request (or after `ttlTicks` client ticks), on `ttlMs:0` and on `release`. It outlives the connection that set it (a one-shot caller may send it and hang up). tags: [{ $ref: '#/components/tags/bedboo' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: input } move: type: object description: 'player_auth_input move vector: x = left - right, z = forward - back, each in [-1, 1].' properties: x: { type: number, minimum: -1, maximum: 1 } z: { type: number, minimum: -1, maximum: 1 } jump: { type: boolean } sneak: { type: boolean } sprint: { type: boolean } ttlMs: { type: integer, minimum: 0, maximum: 60000, default: 300 } ttlTicks: { type: integer, minimum: 1, maximum: 1200, description: 'Not with ttlMs (bad_args).' } examples: - name: walk forward for a second payload: { id: 5, op: input, move: { x: 0, z: 1 }, sprint: true, ttlMs: 1000 } inputReply: name: input title: input reply tags: [{ $ref: '#/components/tags/bedboo' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: active: { type: boolean } via: { type: string, enum: [hooks, keys] } hooks: type: object properties: raw: { type: boolean } state: { type: boolean } tick: { type: integer } applied: { type: integer } ticksLeft: { type: integer } slotRequest: name: slot title: slot request description: Config key `gameHooks`. Without `slot`, a probe. Replied once the player holds the slot. tags: [{ $ref: '#/components/tags/bedboo' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: slot } slot: { type: integer, minimum: 0, maximum: 8 } timeoutMs: allOf: [{ $ref: '#/components/schemas/TimeoutMs' }] default: 1000 examples: - name: hold slot 3 payload: { id: 6, op: slot, slot: 2 } slotReply: name: slot title: slot reply tags: [{ $ref: '#/components/tags/bedboo' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: slot: { type: [integer, 'null'] } via: { type: string, enum: [keys, none] } tick: { type: integer } actionRequest: name: action title: action request description: | Config key `gameHooks`. Without `name`, a probe listing the names. `swing` swings the arm with no attack (LocalPlayer::swing; the client sends the animate packet itself) and works with a screen open; the others need the world in front (`busy` otherwise). tags: [{ $ref: '#/components/tags/bedboo' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: action } name: { type: string, enum: [drop, dropAll, inventory, perspective, pause, swing] } hand: type: string enum: [main, off] default: main description: 'swing only: the arm that swings (animate carries no hand: the off hand shows on this client only).' timeoutMs: allOf: [{ $ref: '#/components/schemas/TimeoutMs' }] default: 1000 examples: - name: open the inventory payload: { id: 7, op: action, name: inventory } actionReply: name: action title: action reply tags: [{ $ref: '#/components/tags/bedboo' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: names: { type: array, items: { type: string } } name: { type: string } hand: { type: string, enum: [main, off], description: 'swing: the arm that swung.' } tick: { type: integer } buttonRequest: name: button title: button request description: | Config key `gameHooks`. Without `button`, a probe. Never blocks the connection. When the connection that pressed a button closes, a hold without an end (`down` without `holdTicks` or `releaseOn`) is released; a `click`, a `holdTicks` hold and a `releaseOn` hold run to their end (a `releaseOn` hold at most 1200 ticks more). tags: [{ $ref: '#/components/tags/bedboo' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: button } button: { type: string, enum: [attack, use, pick] } action: { type: string, enum: [down, up, click], default: click } holdTicks: { type: integer, minimum: 1, description: 'Click default 2.' } releaseOn: { type: string, enum: [blockChange] } timeoutMs: allOf: [{ $ref: '#/components/schemas/TimeoutMs' }] default: 1000 examples: - name: mine the targeted block payload: { id: 8, op: button, button: attack, action: down, releaseOn: blockChange } buttonReply: name: button title: button reply tags: [{ $ref: '#/components/tags/bedboo' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: held: { type: array, items: { type: string } } block: { $ref: '#/components/schemas/Vec3' } tick: { type: integer } settingsRequest: name: settings title: settings request description: | Config key `gameHooks`. Without `renderDistance`, a probe. `renderDistance` (chunks) sets the video option "Render Distance" of every renderer (normal, Vibrant Visuals, ray tracing), each kept to its own values by the game and not saved to options.txt, then the client sends request_chunk_radius itself; the server answers with the radius it grants (chunk_radius_update, on the tap). Works in a world only (`not_in_level`). tags: [{ $ref: '#/components/tags/bedboo' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: settings } renderDistance: { type: integer, minimum: 1, maximum: 255, description: 'Chunks.' } timeoutMs: allOf: [{ $ref: '#/components/schemas/TimeoutMs' }] default: 1000 examples: - name: render 8 chunks payload: { id: 9, op: settings, renderDistance: 8 } settingsReply: name: settings title: settings reply tags: [{ $ref: '#/components/tags/bedboo' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: renderDistance: { type: [integer, 'null'], description: 'Chunks the client renders now (probe: null before the first world).' } requested: { type: integer } inLevel: { type: boolean, description: 'Probe only.' } tick: { type: integer } sendRequest: name: send title: send request description: | Config key `allowSend`. Without `buf`, a probe. `buf` is the base64 of one serverbound packet (varint header + body, as bedrock-protocol serializes it), sent through the local player. Ids outside `sendAllow` (default 9 text, 77 command_request, 78 command_block_update, 97 book_edit) are `refused`; 1, 30, 144 and 147 are always refused. `sendAllow: "*"` allows every id except the packets the client sends itself (login, handshakes, resource pack response, tick_sync, animate, respawn, show_credits, player_skin, subclient_login, set_local_player_as_initialized, network_stack_latency, client cache, emote_list, subchunk_request, request_network_settings, serverbound_loading_screen, and the four above). tags: [{ $ref: '#/components/tags/bedboo' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: send } buf: { type: string, contentEncoding: base64 } timeoutMs: allOf: [{ $ref: '#/components/schemas/TimeoutMs' }] default: 2000 maximum: 10000 sendReply: name: send title: send reply tags: [{ $ref: '#/components/tags/bedboo' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: allow: { type: array, items: { type: integer }, description: 'Probe: the ids send takes (sendAllow; with "*" every one spelled out).' } allowAll: { type: boolean, description: 'Probe: sendAllow is "*".' } packetId: { type: integer } name: { type: string } examples: - name: sent payload: { id: 10, ok: true, packetId: 77, name: CommandRequestPacket } stackRequest: name: stack title: stack request description: | Config key `allowStack`. Without fields, a probe. `buf` is the base64 of one whole item_stack_request packet (id 147); `view:true` returns what the client shows. tags: [{ $ref: '#/components/tags/bedboo' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: stack } buf: { type: string, contentEncoding: base64 } view: { type: boolean } timeoutMs: allOf: [{ $ref: '#/components/schemas/TimeoutMs' }] default: 2000 stackReply: name: stack title: stack reply tags: [{ $ref: '#/components/tags/bedboo' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: enabled: { type: boolean } via: { const: raw } stats: type: object properties: sent: { type: integer } answered: { type: integer } rejected: { type: integer } failed: { type: integer } resyncs: { type: integer } requestIds: { type: array, items: { type: integer } } map: type: object description: "Caller's request ids (as strings) to the client ids sent." additionalProperties: { type: integer } inventory: { type: array, items: { $ref: '#/components/schemas/StackSlot' } } ui: { type: array, items: { $ref: '#/components/schemas/StackSlot' } } open: type: object properties: containerId: { type: integer } type: { type: string } items: { type: array, items: { $ref: '#/components/schemas/StackSlot' } } timescaleRequest: name: timescale title: timescale request description: | Not behind a config key. Without `scale`/`paused`, a probe. On a game it changes a world the game hosts (client and local server together); on the server build, the server. tags: [{ $ref: '#/components/tags/bedboo' }, { $ref: '#/components/tags/server' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: timescale } scale: { type: number, minimum: 0.05, maximum: 20, description: '1 = normal (20 ticks a second); clamped.' } paused: { type: boolean } timeoutMs: allOf: [{ $ref: '#/components/schemas/TimeoutMs' }] description: 'Default 2000 on a game, 5000 on the server build.' examples: - name: double speed payload: { id: 11, op: timescale, scale: 2 } timescaleReply: name: timescale title: timescale reply (game) tags: [{ $ref: '#/components/tags/bedboo' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: scale: { type: number } paused: { type: boolean } source: { type: string, enum: [default, config, local, server] } tps: type: object properties: client: { type: number } server: { type: [number, 'null'] } ticksPerSecond: { type: number } effective: type: object properties: client: { $ref: '#/components/schemas/Effective' } server: { $ref: '#/components/schemas/Effective' } local: { type: boolean } inLevel: { type: boolean } serverTimescaleReply: name: timescale title: timescale reply (server build) tags: [{ $ref: '#/components/tags/server' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: scale: { type: number } paused: { type: boolean } source: { type: string, enum: [default, config, local] } tps: type: object properties: client: { type: 'null' } server: { type: number } ticksPerSecond: { type: number, description: '20 x scale, 0 while paused.' } joinRequest: name: join title: join request description: | Without `host`: where the client plays. With `host`, the client joins that server through the game's own external-server join, from the title screen or from a world (which it leaves first), answering the "not rated" and "Trust this server?" prompts. With `host: null`, the client only leaves the world it is in (Save & Quit / Disconnect) and goes back to its menus: `left:true` (`started:true` without `wait`; with `wait`, once the world is saved and no progress screen shows), or `left:false` when it was in no world. Codes: `busy` (a join or leave is in progress), `timeout`, `stopped`, `no_client`, `unavailable`, `not_allowed` (a host off the PC that `loopbackOnly` keeps the game from: not in `allowHosts`; with `allowHosts` "*" the host is resolved and added, `not_allowed` only when it does not resolve). tags: [{ $ref: '#/components/tags/bedboo' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: join } host: { type: [string, 'null'], description: 'The server; null: leave the world.' } port: { type: integer, default: 19132 } wait: { type: boolean, description: 'Reply once in the level (joined:true) or, with host null, back in the menus.' } timeoutMs: allOf: [{ $ref: '#/components/schemas/TimeoutMs' }] default: 60000 examples: - name: join a local BDS payload: { id: 12, op: join, host: 127.0.0.1, port: 19132, wait: true } - name: leave the world payload: { id: 13, op: join, host: null, wait: true } joinReply: name: join title: join reply tags: [{ $ref: '#/components/tags/bedboo' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: started: { type: boolean } joined: { type: boolean } left: { type: boolean, description: 'A world was left first.' } mode: { type: string } connection: { type: string, enum: [none, local, remote] } host: { type: [string, 'null'] } port: { type: integer } screen: { type: string, description: 'The top screen when the attempt ended.' } inLevel: { type: boolean } joining: { description: 'The attempt in progress.' } last: { description: 'How the last attempt ended.' } allowHosts: description: 'Probe: what the game may reach off the PC: allowHosts, their addresses and the hosts joined with "*"; "*" when loopbackOnly is off.' oneOf: [{ type: array, items: { type: string } }, { const: '*' }] anyHost: { type: boolean, description: 'Probe: join may reach any host (allowHosts "*", or loopbackOnly off).' } tapRequest: name: tap title: tap request description: | Config key `tap`. Without `action`: the status. `action:"subscribe"` answers `{ok:true}` and from then on this connection receives binary messages, one tap frame each (HELLO, the join journal's replay, then the live stream). One subscriber at a time (another gets `busy`); a subscriber more than 32 MiB behind gets DROPPED and the connection is closed. Send nothing else on a subscribed connection. tags: [{ $ref: '#/components/tags/bedboo' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: tap } action: { type: string, enum: [subscribe] } examples: - name: subscribe payload: { id: 1, op: tap, action: subscribe } tapReply: name: tap title: tap reply tags: [{ $ref: '#/components/tags/bedboo' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: enabled: { type: boolean } attached: { type: boolean, description: 'A subscriber is connected.' } link: type: object properties: state: { type: string, enum: [up, down] } type: { type: string, enum: [remote, local] } address: { type: string } journal: type: object properties: frames: { type: integer } bytes: { type: integer } complete: { type: boolean } lastReplay: { type: object, additionalProperties: true } replay: { type: boolean } counters: type: object properties: in: { type: integer } out: { type: integer } dropped: { type: integer } sent: { type: integer } readers: { type: integer } hooks: { type: object, additionalProperties: true } tapFrame: name: tapFrame title: tap frame summary: One binary WebSocket message per frame. description: | Little-endian: `u32 len (bytes after this field) | u8 kind | u8 flags | u16 0 | u64 seq | u64 tUs (steady clock microseconds) | payload`. kind 1 CLIENTBOUND (a game packet: varint header + body), 2 SERVERBOUND_ECHO, 4 MARKER (UTF-8 JSON: HELLO, LINK_UP, LINK_DOWN, REPLAY_BEGIN, REPLAY_END, DROPPED); readers ignore other kinds. Flags: 1 replay, 2 reserialized, 4 local link, 8 suppressed. See the tap reference. contentType: application/octet-stream tags: [{ $ref: '#/components/tags/bedboo' }] payload: type: string format: binary externalDocs: url: https://levimc.org/BeDD/reference/tap # ---- ui ---------------------------------------------------------------------------------- uiRequest: name: ui title: ui request description: | Config key `ui`. Without `action`, a read; with `action` and `ref`, an action through the same virtual mouse and keyboard as the input ops. Read forms: - `{op:"ui"}`: `view`, the screens of `scope` as text. A header line `screen ` (an Ore UI screen: `screen (Ore UI)`), then one line per control or text, `- ""` or `- ` (no text), then ` = `, `checked`/`unchecked`, `disabled`, `focused`, `selected` as they apply, then ` []` when it takes an action. Indentation is nesting. Six or more alike lines in a row are one line ending ` x [in ]`: `{op:"ui", ref}` lists them. Past `maxNodes` lines the first 3/4 and the last 1/4 stay, with a line saying how many were cut. A last line `under: ...` names the screens below. - `{op:"ui", find}`: only the lines whose text, name or value contains `find` (case-insensitive). - `{op:"ui", ref}`: that control and everything below it, nothing folded. - `{op:"ui", ref, detail:true}`: `node`, every field of that control (UiNode, plus `ancestors`). - `{op:"ui", format:"json"}`: the tree as objects, for programs (`nodes` by ref, `roots`, `screens`, `active`). A ref names one control (scene + path) for as long as its screen is on the stack, across reads and connections; it is never moved to another control. tags: [{ $ref: '#/components/tags/ui' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: ui } scope: { type: string, default: active, description: '`active` (the screen in front), `all`, a screen name, or a screen id (`scene-...`).' } find: { type: string } ref: { type: string } detail: { type: boolean, default: false } format: { type: string, enum: [text, json], default: text } maxNodes: { type: integer, minimum: 1, description: 'text: lines (default 80); json: nodes (default config `uiMaxNodes`).' } depth: { type: integer, minimum: 0, description: 'Levels below each screen root.' } includeInvisible: { type: boolean, default: false } compact: { type: boolean, default: true, description: '`false`: every control, layout panels included.' } timeoutMs: { type: integer, minimum: 100, default: 3000 } debug: { type: boolean, default: false, description: 'json/detail: per node the components and their vtables.' } action: { type: string, enum: [click, setText, scroll] } button: { $ref: '#/components/schemas/Button' } holdMs: { type: integer, minimum: 50 } anchor: { type: string, enum: [center, top_left, top, top_right, left, right, bottom_left, bottom, bottom_right], default: center } offsetX: { type: integer, default: 0 } offsetY: { type: integer, default: 0 } force: { type: boolean, default: false, description: 'Act although another control covers the point.' } text: { type: string, description: 'setText: the new content.' } clear: { type: boolean, default: true, description: 'setText: CTRL+A, BACK first.' } submit: { type: boolean, default: false, description: 'setText: ENTER after.' } delayMs: { type: integer, minimum: 0, description: 'setText: between characters.' } notches: { type: integer, default: 1, description: 'scroll: wheel notches (positive scrolls down).' } observeMs: { type: integer, minimum: 0, maximum: 5000, default: 150, description: 'How long to wait before reading the result.' } examples: - name: read payload: { id: 20, op: ui } - name: find payload: { id: 21, op: ui, find: quit } - name: click payload: { id: 22, op: ui, action: click, ref: e1 } - name: type into a text box payload: { id: 23, op: ui, action: setText, ref: e53, text: My World } uiReply: name: ui title: ui reply tags: [{ $ref: '#/components/tags/ui' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: view: { type: string, description: 'Read (text) and `ref`: the lines. Action: present only when another screen (or Ore UI route) came to the front: its view.' } screen: { type: string, description: 'Read and action: the screen in front.' } truncated: { type: boolean, description: 'maxNodes cut lines (text) or nodes (json).' } node: { $ref: '#/components/schemas/UiNode' } action: { type: string, description: 'Action: the action done.' } ref: { type: string, description: 'Action: the target.' } point: { $ref: '#/components/schemas/Point' } changed: { type: boolean, description: 'Action: whether any text, state, place or the screen in front changed.' } target: { type: [string, 'null'], description: 'Action: the target''s line now; null when it is gone.' } snapshot: { type: string, description: 'json: a name for this read (informational).' } tick: { type: integer, description: 'json: client ticks seen.' } client: { $ref: '#/components/schemas/Size' } screens: { type: array, items: { type: object }, description: 'json: the scene stack, bottom first.' } active: { type: string, description: 'json: the id of the screen in front.' } roots: { type: array, items: { type: string }, description: 'json: refs of the root nodes.' } nodes: type: object description: 'json: every node by ref.' additionalProperties: { $ref: '#/components/schemas/UiNode' } examples: - name: pause screen payload: id: 20 ok: true screen: pause_screen truncated: false view: | screen pause_screen - button "Resume Game" [e30] - button "Settings" [e31] - button "Save & Quit" [e33] - text "Game is paused" under: hud_screen, hud_crosshair_screen, in_game_play_screen - name: click payload: id: 22 ok: true action: click ref: e1 point: { x: 800, y: 428 } changed: true target: '- button "Play" [e1]' screen: /play/all view: | screen /play/all (Ore UI) - button "Create new world" [e25] - button "My World 11/14/26" [e27] oreuiRequest: name: oreui title: oreui request description: | Config key `ui`. The raw DOM of the Ore UI view (world list, settings and the other HTML screens), read by a read-only script run in the view's V8 context; it never changes the page. `ui` already lists the page's controls with refs: use this to see what `ui` leaves out. Nodes have no refs; `bounds` are game client pixels (`mouse` `x`/`y`). Written against the V8 9.x in the game's v8.dll; another major version answers `unsupported` unless `force` is set. Errors are `unsupported` (with `findings`), `not_ready` (no Ore UI screen ran script within `timeoutMs`) and `disabled`. tags: [{ $ref: '#/components/tags/ui' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: oreui } url: { type: string, description: 'Wait for a view whose page URL (location.href, e.g. `ui://./hbui/index.html`) contains this.' } maxNodes: { type: integer, minimum: 1, maximum: 50000, default: 2000 } maxText: { type: integer, minimum: 8, maximum: 4096, default: 200, description: 'Characters kept of each text.' } timeoutMs: { type: integer, minimum: 100, default: 3000, description: 'How long to wait for the view to run script.' } force: { type: boolean, default: false, description: 'Run on a V8 major version the op was not written for.' } examples: - name: read payload: { id: 23, op: oreui } oreuiReply: name: oreui title: oreui reply tags: [{ $ref: '#/components/tags/ui' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: url: { type: string, description: 'The page URL (location.href, e.g. `ui://./hbui/index.html`); the route is in `ui`''s screen line.' } title: { type: string } width: { type: integer } height: { type: integer } nodes: type: array description: Document order; `children` are indexes into this array. items: type: object properties: tag: { type: string } id: { type: string } class: { type: string } testId: { type: string, description: 'data-testid' } role: { type: string } type: { type: string, description: 'The type attribute.' } act: { type: integer, enum: [1, 2], description: 'Takes a click: 1 by tag, ARIA role, tabindex, test id or event handler; 2 only by Ore UI hover styling.' } name: { type: string, description: 'aria-label, title, alt, placeholder or name.' } text: { type: string, description: 'Own text of the element.' } value: { type: string } state: { type: object, additionalProperties: { type: boolean } } bounds: { $ref: '#/components/schemas/Rect' } children: { type: array, items: { type: integer } } truncated: { type: boolean } findings: { type: object, description: 'Gameface and V8 versions and how the DOM was read.' } # ---- input ------------------------------------------------------------------------------- keyRequest: name: key title: key request description: 'Names: letters, digits, ESC, ENTER, BACK, TAB, SPACE, SHIFT, CTRL, ALT, LEFT/UP/RIGHT/DOWN, HOME, END, PGUP, PGDN, INSERT, DELETE, F1..F24, SLASH, GRAVE, MINUS, EQUALS.' tags: [{ $ref: '#/components/tags/input' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: key } key: { type: string } vk: { type: integer, minimum: 1, maximum: 254 } action: { type: string, enum: [press, down, up], default: press } holdMs: { type: integer, minimum: 0, default: 60 } timeoutMs: allOf: [{ $ref: '#/components/schemas/TimeoutMs' }] default: 5000 examples: - name: escape payload: { id: 30, op: key, key: ESC } textRequest: name: text title: text request tags: [{ $ref: '#/components/tags/input' }] payload: type: object required: [op, text] properties: id: { $ref: '#/components/schemas/Id' } op: { const: text } text: { type: string } delayMs: { type: integer, minimum: 0, default: 0 } timeoutMs: allOf: [{ $ref: '#/components/schemas/TimeoutMs' }] default: 5000 examples: - name: chat payload: { id: 31, op: text, text: hello } mouseRequest: name: mouse title: mouse request description: | Moves the virtual cursor (if a point or a node is given), then presses buttons or scrolls on the virtual GameInput mouse. A point is `x`,`y` (client pixels), `rx`,`ry` (fractions of the client area) or `ref` (a point of a `ui` control: `anchor`, `offsetX`/`offsetY`; `stale_ref`, `occluded`; the reply adds `changed`, `target`, `screen` and `view` as a `ui` action does). tags: [{ $ref: '#/components/tags/input' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: mouse } action: { type: string, enum: [move, down, up, click, wheel], default: move } x: { type: integer } y: { type: integer } rx: { type: number, minimum: 0, maximum: 1 } ry: { type: number, minimum: 0, maximum: 1 } ref: { type: string } anchor: { type: string, default: center } offsetX: { type: integer, default: 0 } offsetY: { type: integer, default: 0 } force: { type: boolean, default: false } button: { $ref: '#/components/schemas/Button' } notches: { type: integer, default: 1 } holdMs: { type: integer, minimum: 50, default: 80 } settleMs: { type: integer, minimum: 0, default: 50 } afterMs: { type: integer, minimum: 0, default: 50 } timeoutMs: allOf: [{ $ref: '#/components/schemas/TimeoutMs' }] default: 5000 examples: - name: click at a fraction payload: { id: 32, op: mouse, action: click, rx: 0.498, ry: 0.474 } - name: click a ui node payload: { id: 33, op: mouse, action: click, ref: e42 } rawRequest: name: raw title: raw request tags: [{ $ref: '#/components/tags/input' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: raw } action: { type: string, enum: [move, down, up, click, wheel], default: move } dx: { type: integer, description: Mouse counts. } dy: { type: integer } steps: { type: integer, minimum: 1, default: 1 } button: { $ref: '#/components/schemas/Button' } notches: { type: integer, default: 1 } holdMs: { type: integer, minimum: 50, default: 80 } examples: - name: turn right payload: { id: 34, op: raw, dx: 300, steps: 5 } releaseRequest: name: release title: release request description: Releases every held key and button, whoever pressed it, and ends the `input` override. tags: [{ $ref: '#/components/tags/input' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: release } flushRequest: name: flush title: flush request tags: [{ $ref: '#/components/tags/input' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: flush } timeoutMs: allOf: [{ $ref: '#/components/schemas/TimeoutMs' }] default: 5000 focusRequest: name: focus title: focus request tags: [{ $ref: '#/components/tags/input' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: focus } windowRequest: name: window title: window request tags: [{ $ref: '#/components/tags/input' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: window } clientWidth: { type: integer, minimum: 0 } clientHeight: { type: integer, minimum: 0 } examples: - name: 1280x720 payload: { id: 35, op: window, clientWidth: 1280, clientHeight: 720 } windowReply: name: window title: window reply tags: [{ $ref: '#/components/tags/input' }] payload: allOf: - $ref: '#/components/schemas/Reply' - $ref: '#/components/schemas/WindowInfo' - type: object properties: result: { description: 'What the layout change did.' } screenshotRequest: name: screenshot title: screenshot request description: | PrintWindow (PW_RENDERFULLCONTENT) of the game window, encoded with WIC; works while the window is covered or off screen. Costly for an agent: prefer `world` or `ui`. tags: [{ $ref: '#/components/tags/input' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: screenshot } format: { type: string, enum: [png, jpeg], default: png } quality: { type: integer, minimum: 1, maximum: 100, description: 'JPEG quality.' } maxWidth: { type: integer, minimum: 1, default: 960, description: 'Scaled down to at most this width.' } region: { $ref: '#/components/schemas/Rect' } examples: - name: small jpeg payload: { id: 36, op: screenshot, format: jpeg, quality: 70, maxWidth: 640 } screenshotReply: name: screenshot title: screenshot reply tags: [{ $ref: '#/components/tags/input' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object required: [width, height, format, data] properties: width: { type: integer } height: { type: integer } format: { type: string, enum: [png, jpeg] } data: { type: string, contentEncoding: base64 } inputDoneReply: name: inputDone title: input op reply description: The reply of key, text, mouse, raw, release, flush and focus, after the game thread dispatched the input. tags: [{ $ref: '#/components/tags/input' }] payload: { $ref: '#/components/schemas/Reply' } # ---- server build ------------------------------------------------------------------------ serverRequest: name: server title: server request tags: [{ $ref: '#/components/tags/server' }] payload: type: object required: [op] properties: id: { $ref: '#/components/schemas/Id' } op: { const: server } serverReply: name: server title: server reply tags: [{ $ref: '#/components/tags/server' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: tick: { type: integer } tps: { type: number } paused: { type: boolean } port: { type: integer } players: type: array items: type: object properties: name: { type: string } xuid: { type: string } commandRequest: name: command title: command request tags: [{ $ref: '#/components/tags/server' }] payload: type: object required: [op, command] properties: id: { $ref: '#/components/schemas/Id' } op: { const: command } command: { type: string, description: 'A leading / is optional.' } timeoutMs: allOf: [{ $ref: '#/components/schemas/TimeoutMs' }] default: 10000 examples: - name: time payload: { id: 40, op: command, command: time set day } commandReply: name: command title: command reply tags: [{ $ref: '#/components/tags/server' }] payload: allOf: - $ref: '#/components/schemas/Reply' - type: object properties: success: { type: boolean } successCount: { type: integer } output: { type: array, items: { type: string } } errorReply: name: error title: error reply tags: - { $ref: '#/components/tags/system' } payload: { $ref: '#/components/schemas/ErrorReply' } examples: - name: disabled payload: { id: 10, ok: false, code: disabled, key: allowSend, error: 'send is off (config.json "allowSend")' } - name: not in a world payload: { id: 6, ok: false, code: not_in_level, error: the client is not in a world } - name: stale snapshot payload: { id: 21, ok: false, code: stale_snapshot, error: the screens changed since snapshot s17 } - name: unknown op payload: { id: 99, ok: false, error: "unknown op 'fly'" }