; TerminalPane pins the live Terminal to the bottom when NOT in copy mode.
```
When a pane is in **copy mode**, the keyboard actor derives it fresh on every keydown from
`copyModeStates[activePaneId]` (a focused float always takes priority) and routes keys to the client
engine instead of tmux — so switching or closing the pane instantly stops copy-mode routing with no
extra plumbing. A pane in the **scroll view** routes nothing: the keystroke closes the view and
carries on to tmux.
Copy mode is **per-pane**: `TerminalPane` reads `copyModeStates[paneId]` and renders `ScrollbackTerminal`
when present, otherwise the live `Terminal`. State for a pane is pruned when the pane leaves copy mode or
is closed — the `TMUX_STATE_UPDATE` reconciliation in `appMachine.ts` owns that pruning.
## Entry and Exit
**Scroll view — entry and exit:**
- Wheel-up / touch-scroll-up on a pane with history that is **not** on the alternate screen and is
**not** tracking the mouse — `usePaneMouse` / `usePaneTouch` send `ENTER_SCROLL_MODE`. That gate is
what keeps the view out of nvim, htop, less and Claude Code: a full-screen application's scroll is
its own, forwarded as SGR or arrow keys (`scrollUtils.sendScrollLines`).
- The first real keystroke closes it (`keyboardActor` sends `EXIT_SCROLL_MODE`) and the key goes on to
the pane, so typing while scrolled up lands at the prompt. Modifier-only presses do not, or holding
Shift to extend a selection would close what you are selecting from. `Escape` closes it and is spent.
- Scrolling back to the bottom with nothing selected closes it (`copyMode_scroll`).
- No `send-keys -X cancel` and no re-entry cooldown on the way out — both belong to a tmux mode this
view never enters.
**Copy mode — entry and exit:**
- `prefix + [` — the `copy-mode` command is intercepted in `appMachine` and raised as `ENTER_COPY_MODE`
(the original command still forwards to tmux, which flips `in_mode`).
- A pane reported `in_mode` by tmux (e.g. a CLI `copy-mode`, a custom binding) — the `TMUX_STATE_UPDATE`
reconciliation initializes a client copy-mode state and fetches its full history.
- `q` / `Escape` / `y` — handled by tmux's `copy-mode-vi` bindings; the pane leaves `in_mode` and the
reconciliation drops the client state.
- Scrolling back to the bottom of history with no active selection — `copyMode_scroll` auto-exits.
A selection holds the view open — copy mode's own, or the browser's in the scroll view, which
the scroll event reports (`nativeSelection`) since only the component can see it.
- A 2-second re-entry cooldown (`COPY_MODE_REENTRY_COOLDOWN`) prevents a stale `in_mode` flag from
immediately re-opening copy mode after an exit.
The mouse never opens copy mode. A wheel gesture opening a mode with a cursor and vi keys is what
this split exists to undo.
## Scrollback loading
Row indices are absolute: a row's index is `history_size + `. A view can open before tmux
has reported the pane's real `history_size` (it starts near zero), so every response carries the size
it was answered with and rows already merged are re-keyed by the difference
(`shiftScrollbackRows`) before the new chunk lands. Without that the screen rows sit on top of the
oldest history, and a copy of the whole scrollback comes back with its tail at the front.
History is loaded lazily as structured cells, never as a client-maintained scrollback buffer of live
output. On entry the client fetches the full backlog; as the user scrolls, `getNeededChunk` requests
200-line chunks around the viewport and `mergeScrollbackChunk` merges them into the `lines` map
(tracked by `loadedRanges`). Rows that exist but aren't loaded yet render as dim placeholders.
The fetch is a single adapter call — `adapter.invoke('get_scrollback_cells', { paneId, start, end })` —
implemented per transport:
- **HTTP/SSE server** — `ClientCommand::GetScrollbackCells` runs `capture-pane -p -e -S start -E end`
and parses it with `parse_scrollback_to_cells` (`tmuxy-server/src/sse.rs`, `tmuxy-core`).
- **Tauri desktop** — the `get_scrollback_cells` command mirrors the same capture + parse
(`tmuxy-tauri-app/src/commands.rs`).
- **Fully client-side (v86, CheerpX)** — there is no server, so `GuestEngine.captureScrollback` runs the same
`capture-pane` over the in-browser control connection (bracketed by unique markers so its lines can
be picked out of the stream) and hands the raw text to the core's `parse_scrollback` (a wasm export
of `parse_scrollback_to_cells`), reusing the identical ANSI parser instead of a JS reimplementation.
## Keyboard
Copy-mode vi keybindings are reimplemented client-side in `copyModeKeys.ts` (`handleCopyModeKey`). While
a pane is in copy mode, `keyboardActor.ts` calls `preventDefault()` on every keydown and dispatches
`COPY_MODE_KEY`; the `copyMode_key` action resolves it to cursor motion, selection, page/word/line
motions, yank, or exit against the pane's `CopyModeState`. Supported keys include `h`/`j`/`k`/`l` (+
arrows), `0`/`$`, `w`/`b`/`e`, `H`/`M`/`L`, `gg`/`G`, `Ctrl-u`/`Ctrl-d`/`Ctrl-b`/`Ctrl-f`, `Space`/`v`
(char select), `V` (line select), `y` (yank), and `q`/`Escape` (exit). Yank and `Ctrl/Cmd-C` copy the
extracted selection to the system clipboard via the keyboard actor's native `copy` event.
## Mouse and selection
**In copy mode**, `usePaneMouse` drives the client selection:
- **Drag** — starts a char selection at the drag origin and extends it as the cursor moves
(`COPY_MODE_SELECTION_START` / `COPY_MODE_CURSOR_MOVE`). Auto-scrolls when dragging past an edge.
- **Double-click** — selects the word under the cursor (`COPY_MODE_WORD_SELECT`).
- **Triple-click** — selects the whole logical line (`COPY_MODE_LINE_SELECT`), expanded across wrapped rows.
- **Release** of a drag copies the selection to the clipboard and leaves copy mode
(`COPY_MODE_MOUSE_COPY`), as tmux's `MouseDragEnd1Pane → copy-pipe-and-cancel` does — the default
config binds exactly that, so a native tmux client behaves the same. tmux leaves its mode at once;
the client's view stays for the copy blink and then closes (`COPY_MODE_COPIED_EXIT`).
**Everywhere else** the hook deliberately does nothing on those gestures and prevents no default, so
the browser selects — which also means double-click word boundaries come from the engine rather than
a cell heuristic. Right-click reads that selection (`utils/nativeSelection.ts`) and opens the
selection's menu over it; with nothing selected it opens the pane's own menu instead, the one its
header opens (`PaneContextMenu`). A right-click does not move browser focus to the hidden keyboard
input, which would collapse the selection it is about to read. While the menu is up
the selection is pinned (`SelectionContextMenu`): the menu takes focus on open and each item on
hover, and WebKit collapses the document selection whenever focus moves — so whatever collapses it,
it is put back until the menu closes.
A browser selection lives in DOM nodes, so the scroll view keeps them: `ScrollbackTerminal` mounts
each row as its own absolutely positioned node, repaints a row only when its content changes (never
because the window moved), and keeps every row a selection spans mounted while it lasts — so the
selection survives scrolling, in either direction and all the way to the bottom, and reads back
whole, since selection text is document order.
**Wheel / touch** scroll the native container while a view is open, a whole row at a time
(`takeWholeRows` / `scrollByRows` in `hooks/scrollUtils.ts`) — the sub-row part of a trackpad delta is
carried into the next event rather than painted, so the container always rests on a row boundary and
the top row is never half drawn. That matches the rest of the pane: copy mode moves a cursor that
lives on a row, and a full-screen application is sent rows. `onScroll` reports the new top row
via `COPY_MODE_SCROLL`. Mouse-tracking applications (`mouse_any_flag`, e.g. nvim/htop) receive
forwarded SGR mouse sequences instead — that path is unchanged.
A **swipe that opens a view** has two hazards, both about the gesture outliving the DOM it started
on. The browser cancels a touch sequence when its target leaves the document, so the live screen
stays mounted (hidden) behind an open view instead of being replaced, and the move/end listeners sit
on the window rather than on the pane — a pane-level listener stopped hearing the swipe at the very
row that opened the view. And until React has re-rendered with the view open, each further move
still asks to open one, so `ENTER_SCROLL_MODE` on an already-open view scrolls it further instead of
rebuilding it at the bottom. Without those three, a long swipe landed one row above the live screen.
## Clipboard
**Copy on select**, as the Claude Code TUI does it: the mouse release that ends a browser selection
over terminal text (a drag, a double or triple click, on the live screen or in the scroll view) puts
the text on the clipboard at once, blinks it, and raises an info notification saying how many
characters went. The selection stays on screen, and the screen holds still under it. The next key —
`Ctrl+C`, `Cmd+C`, a letter — clears the selection and is processed as typed: `Ctrl+C` interrupts
the shell, a letter goes to the pane, `Cmd+C` copies the same text once more before the highlight
goes. A modifier on its own leaves it. The same text still selected on a later release is not copied
twice; once the selection has gone, selecting it again copies again. The keyboard actor's clipboard
module owns this (`machines/actors/keyboard/imeAndClipboard.ts`); the touch hold on a phone keeps
its own Copy/Cancel bar.
Selected text is extracted client-side (`extractSelectedText`, which joins wrapped rows into logical
lines). Every copy blinks the copied text where it is (`utils/copyFlash.ts`). Copy mode paints its own
selection, so its copy sets `copiedAt` on the view, the scrollback marks itself `data-copied`, and the
view closes when the blink is over — keyboard yank, `Ctrl/Cmd-C` and a drag release alike. A browser
selection is covered by fixed boxes laid over its rectangles instead, on `Cmd+C` and on the
right-click **Copy**. With animations off the highlight holds steady for the same time. Keyboard yank (`y`/`Enter`) and `Ctrl/Cmd-C` set the extracted text on the keyboard actor's
native `copy` event (`document.execCommand('copy')` → `clipboardData`). The right-click **Copy** action
writes it via `navigator.clipboard.writeText`; the selection context menu's other item, **Send keys**,
types the selection into the pane. Either closes the view.
## Key files
| File | Responsibility |
| -------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| `packages/tmuxy-ui/src/components/ScrollbackTerminal.tsx` | Virtual-scrolling renderer for loaded scrollback + client cursor/selection |
| `packages/tmuxy-ui/src/components/TerminalPane.tsx` | Chooses `ScrollbackTerminal` vs live `Terminal`; owns the native scroll container and `onScroll` |
| `packages/tmuxy-ui/src/machines/app/actions/copyMode.ts` | XState actions for both views: enter/exit, cursor/selection, scroll, chunk merge, prefetch |
| `packages/tmuxy-ui/src/utils/nativeSelection.ts` | Reading the browser's selection and selecting the word under a point |
| `packages/tmuxy-ui/src/machines/app/states/copyMode.ts` | Wires `COPY_MODE_*` events to their actions (idle-only) |
| `packages/tmuxy-ui/src/machines/actors/keyboardActor.ts` | Intercepts keydowns in copy mode → `COPY_MODE_KEY` / `COPY_SELECTION`; native clipboard `copy` handler |
| `packages/tmuxy-ui/src/utils/copyModeKeys.ts` | Pure vi-key handler (`handleCopyModeKey`): motions, selection, page/word/line, yank, exit |
| `packages/tmuxy-ui/src/utils/copyMode.ts` | Pure helpers: scrollback merge, row re-keying, gap detection, needed-chunk detection, selected-text extraction |
| `packages/tmuxy-ui/src/hooks/usePaneMouse.ts` / `usePaneTouch.ts` | Mouse/touch → copy-mode enter, selection, and native scroll |
| `packages/tmuxy-ui/src/machines/actors/tmuxActor.ts` | `FETCH_SCROLLBACK_CELLS` → `adapter.invoke('get_scrollback_cells')` → `COPY_MODE_CHUNK_LOADED` |
| `packages/tmuxy-core/src/lib.rs` | `parse_scrollback_to_cells` (shared by server, Tauri, and the wasm core) |
| `packages/tmuxy-ui/src/infra/transport/drivers/guest/GuestEngine.ts` | `captureScrollback` (marker-bracketed capture) + `parseScrollback` for the in-browser deployment |
## Related
- [STATE-MANAGEMENT.md](https://github.com/flplima/tmuxy/blob/main/docs/STATE-MANAGEMENT.md) — the `copyMode` slice and `copyModeStates` context
- [NON-GOALS.md](https://github.com/flplima/tmuxy/blob/main/docs/NON-GOALS.md) — why client scrollback rendering is the one scrollback-like feature we implement
- [DATA-FLOW.md](https://github.com/flplima/tmuxy/blob/main/docs/DATA-FLOW.md) — the SSE/HTTP/Tauri/v86 transports the scrollback fetch rides on
========================================================================
docs/DATA-FLOW-DEPLOYMENTS.md
https://github.com/flplima/tmuxy/blob/main/docs/DATA-FLOW-DEPLOYMENTS.md
========================================================================
# Data Flow: Desktop IPC, Deployments and Endpoints
The Tauri transport, real-world deployment scenarios and the extra HTTP endpoints. Part of [DATA-FLOW.md](https://github.com/flplima/tmuxy/blob/main/docs/DATA-FLOW.md).
## Transport: Tauri IPC (Desktop Version)
The Tauri desktop app bypasses the network stack entirely:
**`invoke(cmd, args)`** — Client-to-server commands (equivalent to HTTP POST). Calls Rust functions directly through Tauri's IPC bridge. The `TauriAdapter` dynamically imports `@tauri-apps/api/core` to call `invoke()`.
**Tauri events** — Server-to-client state updates (equivalent to SSE). The `TauriEmitter` pushes state changes with `emit_to(, "tmux-state-update", …)`, addressed to one window rather than broadcast. The frontend listens via `listen('tmux-state-update', handler)`.
Tauri IPC has lower latency than HTTP since communication is in-process.
**One monitor per GUI window.** The desktop app can have several OS windows open on one tmux session (Window ▸ New Window). Each is its own client: its own webview, its own control-mode monitor, and its own tmux session, with the sessions members of one tmux _session group_ — they share every window and pane while each keeps its own current window, which is what puts two tabs of one session on screen at once. The first window holds the group's base session (the one `TMUXY_SESSION` names) and every later one attaches to `` (`tmux -CC new-session -A -s `
4. All subsequent commands use HTTP POST to `/commands?session=`
5. State updates stream via SSE in real-time
**Security considerations (critical):**
The tmuxy server has **no TLS**, and on a routable address its only gate is an HTTP Basic password (`--password` / `TMUXY_PASSWORD`) — or none, with `--no-auth`. See SECURITY.md. Exposing it directly on a public IP means:
- Anyone who discovers the IP and port can try the password, with no rate limit — and with `--no-auth` simply controls your tmux session
- All traffic is in cleartext (eavesdropping reveals terminal content and lets attackers inject commands; Basic-auth credentials are only base64, not encrypted)
- `run-shell` commands allow arbitrary code execution on the server
- File reading endpoints have no path restrictions
**Required mitigations for this scenario:**
1. **Never expose tmuxy directly to the internet.** Use one of:
- SSH tunnel: `ssh -L 9000:localhost:9000 user@vm` (recommended for single user)
- VPN: WireGuard, Tailscale, or similar (recommended for mobile access)
- Reverse proxy with authentication: nginx/Caddy + basic auth or OAuth + TLS (start the server with `--allowed-host `)
2. **Keep the default address for a tunnel:** plain `tmuxy server` listens on 127.0.0.1, which is all an SSH tunnel needs
3. **Set a password on any other address:** `TMUXY_PASSWORD=… tmuxy server --host …` gates all routes with HTTP Basic auth — a barrier against opportunistic scans, but pair it with TLS since credentials cross the wire in cleartext
4. **Use a reverse proxy for TLS:** The server does not support HTTPS natively
See [SECURITY.md](https://github.com/flplima/tmuxy/blob/main/docs/SECURITY.md) for the full threat model and recommendations.
**What works today:**
- SSE streaming with delta protocol works well over high-latency connections
- `EventSource` auto-reconnects on brief network drops
- Multi-client support — multiple browser tabs/devices can connect simultaneously
- Viewport auto-sizing across clients by `@tmuxy-size` (the smallest by default; the largest, the latest used, or a fixed grid every client scales to fit)
- Mobile browsers work (touch events are translated to mouse events)
**Limitations:**
- No offline capability — requires constant network connection
- No compression — JSON payloads can be large during rapid output (mitigated by delta protocol)
- Only HTTP Basic auth (`--password`) built in; for anything stronger rely on external layers (SSH, VPN, reverse proxy)
- Latency affects typing feel — no local echo or input prediction (see [NON-GOALS.md](https://github.com/flplima/tmuxy/blob/main/docs/NON-GOALS.md))
### Scenario 4: Fully Client-Side — Real tmux in the Browser (v86 or CheerpX + WASM)
No server at all. Real tmux runs inside an in-browser guest machine, and its `tmux -CC` control-mode stream is parsed by **tmuxy-core compiled to WebAssembly** — the same Rust parser and state aggregator the native server runs. Two guests share everything but the machine:
- **v86** — an x86 emulator running a small buildroot Linux restored from a pre-booted state snapshot; the control client is on its UART.
- **CheerpX** — an x86-to-WebAssembly JIT running 32-bit Alpine binaries from an ext2 image streamed over HTTP; the control client is on its console, through the guest's `tmuxy-console` relay. Faster and with no kernel to boot, but it needs a cross-origin isolated page (COOP + COEP headers), and its kernel lacks enough that the guest's tmux is patched and every pane runs under `tmuxy-pty` (see `packages/tmuxy-ui/cheerpx/README.md`).
```
TmuxyApp ──invoke(run_tmux_command)──> GuestTmuxAdapter ──> GuestEngine (V86Engine | CheerpXEngine)
▲ │ send
│ ▼
onStateChange <── tmuxy-wasm (parse + aggregate) <──stream── tmux -CC in the guest
```
Key pieces (under `tmuxy-ui/src/infra/transport/drivers/`):
| Piece | Responsibility |
| --- | --- |
| `guest/GuestEngine` | The machine-independent control-mode client: the wasm core, tick/sync timers, `%exit`→fatal detection, the attach-and-sync loop with a per-attach guest setup, and a serialized boot/reset lifecycle. Marker-bracketed commands (tracked mutations, scrollback captures, in `guest/controlMode.ts`) send their markers as separate lines, as the native monitor does — invariant A6 in INVARIANTS.md. |
| `guest/GuestTmuxAdapter` | The driver behind `GuestTransportLive`: translates frontend commands for raw control-mode stdin (separator + format-expansion rewrites per TMUX.md), serves themes/keybindings/images locally. The first frame is the finished start: the session is sized to the client's viewport before the first state is answered, as the server does, and an init whose work outlives its commands (a shell line it typed) names a tmux option it sets when done (`initDoneOption`), which `connect()` waits for. Files the page hands it (`files`: text, or a URL it fetches) are in the guest's `/mnt` before the init runs: v86's 9p share, a CheerpX in-memory device mounted at the same path. |
| `v86/V86Engine` | The v86 machine: a UART writer paced under the rate the guest drains its input (v86's UART queue is unbounded; the guest reads ~45 KB/s, and a slower pace turns the queue into the bottleneck behind every keystroke) and serial coalescing. Snapshot restores rewind the filesystem, so the guest setup is re-applied on every attach. |
| `v86` shared engine | Opt-in: many adapters reuse one booted machine; each consumer restores the pinned snapshot with a fresh WASM core (~1s) instead of cold-booting (~5s). |
| `cheerpx/CheerpXEngine` | The CheerpX machine: CheerpX from its CDN, the image with a fresh IndexedDB overlay per page load, the relay on the console. |
The public demo site (`packages/tmuxy-demo`) runs CheerpX where the page is cross-origin isolated and v86 elsewhere; `?guest=v86` asks for v86. Its dev server sends the isolation headers; GitHub Pages, which hosts the static export, cannot, so a service worker adds them to the site's own responses (`public/coi-sw.js`), reloading a page it does not serve yet (a first visit, a hard reload) once to put it under the worker. A browser without service workers gets v86. The Storybook `v86` stories run v86. Assets (kernel, BIOS, state snapshot, ext2 image, wasm bindings) are served statically; nothing leaves the browser.
The v86 guest is CI-tested: the `storybook-v86-probe` job builds the wasm bindings and guest assets (cached on the build-script hash), starts a Storybook dev server, and drives the gate stories on one shared page (`tmuxy-ui/scripts/probe-spikes.mjs`); every `v86`-tagged story runs nightly in `nightly-v86.yml`. The deterministic stories run in the parallel `storybook-probe` job. See `docs/TESTS.md` § Storybook Tests. The CheerpX guest has no automated test yet.
## Additional API Endpoints
Beyond the core SSE/HTTP protocol, the web server exposes:
| Endpoint | Method | Purpose |
| --- | --- | --- |
| `/events` | GET | SSE stream (state updates, connection info) |
| `/commands` | POST | tmux commands (no authentication unless `--password` is set — see SECURITY.md) |
| `/api/images/{pane_id}/{image_id}` | GET | Serve a decoded inline-image blob |
| `/themes/{name}.css`, `/custom.css` | GET | The user's stylesheets from the config dir (a built-in theme name falls back to the bundled copy) |
The stylesheet routes are answered by `tmuxy_core::session::resolve_user_css` in both builds: the web server serves them at the app's origin (ahead of the embedded bundle, and ahead of the Vite proxy in dev mode), the desktop app over its `tmuxycss:` scheme, and `tmuxy-ui/src/utils/themeManager.ts` picks between the two. Both hosts poll the config dir once a second (`transport::watch_config`): a saved stylesheet becomes a `css-changed` push, a saved `tmuxy.conf` is sourced on the control-mode connection and followed by the `keybindings` and `theme-settings` pushes a client's own `source-file` gets. `tmuxy config reload` is that second path — it marks the file modified and nothing more. See [ARCHITECTURE.md](https://github.com/flplima/tmuxy/blob/main/docs/ARCHITECTURE.md) for what the config dir holds.
Every route is gated by the optional `--password` Basic auth. See [SECURITY.md](https://github.com/flplima/tmuxy/blob/main/docs/SECURITY.md).
========================================================================
docs/DATA-FLOW.md
https://github.com/flplima/tmuxy/blob/main/docs/DATA-FLOW.md
========================================================================
# Data Flow
This document describes how data moves through tmuxy in different deployment scenarios, covering the SSE/HTTP protocol, Tauri IPC, and real-world use cases.
## Transport: SSE/HTTP (Web Version)
The web version uses two HTTP endpoints on the Axum server:
**`GET /events?session=`** — Server-Sent Events stream (server-to-client):
- `connection-info` — Connection ID, default shell, and whether the server is tracing (sent on connect)
- `keybindings` — Prefix key and all key bindings from tmux config
- `theme-settings` — Theme name/mode, appearance (surface opacities, blur flag) and the `@tmuxy-css-*` variables from tmux config; re-sent after a `source-file`, and after the server itself sources a saved `tmuxy.conf`
- `css-changed` — A stylesheet under the user's config dir (`themes/*.css`, `custom.css`) was saved; the client re-fetches its stylesheets and the themes list. No payload
- `state-update` — Full state snapshots and incremental deltas (serialized JSON)
- `clipboard` — OSC 52 clipboard payloads forwarded from terminal applications
- `tmux-message` — `{ text }`, a `display-message` tmux showed this client (its `%message` notification), shown in the snackbar. Named `tmux-message`, never `message`: that is `EventSource`'s default event type, which `onmessage` receives too. The desktop app emits the same name.
- `ping` — keepalive on an idle stream, once a second; the client times a silent link out by it (see Reconnecting below)
- `log`, `tmux-error`, `fatal` — Diagnostic and error notifications. The error event is named `tmux-error`, never `error`: the browser hands a server event called `error` to `EventSource.onerror` too, and the adapter would treat every reported error as a dropped connection.
**`POST /commands?session=`** — HTTP POST (client-to-server):
- Request body: `{ "cmd": "command_name", "args": {...} }`
- Response: `{ "result": ... }` or `{ "error": "message" }`
- **Loopback by default** — a routable `--host` needs an HTTP Basic password (or an explicit `--no-auth`), and every API route refuses requests from other origins — see [SECURITY.md](https://github.com/flplima/tmuxy/blob/main/docs/SECURITY.md).
SSE was chosen over WebSocket because: server-to-client is the dominant direction, `EventSource` has built-in browser reconnection, SSE works through all proxies/CDNs, and the standard `Last-Event-Id` mechanism gives us a clean reconnect path (see below).
### Reconnecting
The adapter reopens a dropped stream on an exponential backoff (1s, 2s, 4s … capped at 30s), and three rules keep that from turning into a long wait on a link that drops often and briefly — a laptop waking, a Wi-Fi roam:
- **The backoff belongs to one outage.** A connection that was established and then dropped starts the next one on a fresh schedule, after a short beat. One schedule for the page's whole life only climbs, and a few drops in every reconnect waited out the cap.
- **A reason to retry now beats the timer.** The browser's `online` event, the tab becoming visible, and the overlay's _Retry now_ button (`RECONNECT_NOW` → `adapter.reconnectNow()`) all abandon the current wait and try immediately. None of them does anything while connected.
- **Silence is a drop.** The server's keepalive is a named `ping` event rather than an SSE comment (a page never sees comments), sent whenever the stream has been idle for a second. Once a client has seen one, five seconds without any event ends the connection — a link that dies without an error is otherwise noticed minutes later, if at all. A server that sends no pings is never timed out.
### A refused stream is not a dropped one
`EventSource` reports every failure as the same bare error, with no status: a server that is restarting and a server that refuses the request look identical, and retrying the second never ends. So when a stream fails before it ever opened, the adapter asks the same URL with a plain `fetch` and reads the status. A 403 is the request guard's verdict on this page's `Host` or origin (see [SECURITY.md](https://github.com/flplima/tmuxy/blob/main/docs/SECURITY.md)) — the retry loop stops and the reason in the response body is shown as the fatal error. No answer, a 5xx, or a stream that does open are outages, and the supervised retry carries on.
### SSE resync via `Last-Event-Id`
Every event the server broadcasts is tagged with a monotonic per-session sequence id (set as the SSE `id:` field). `EventSource` persists the last received id across reconnects and sends it back as the `Last-Event-Id` request header on retry. The server keeps a small ring buffer of recent events per session — the last 128, within 16 MB (`EVENT_BUFFER_SIZE`, `EVENT_BUFFER_MAX_BYTES` in `tmuxy-server/src/state.rs`) — and replays everything strictly newer than the supplied id before resuming the live stream. If the client's id is older than the buffer head (a long disconnect, or a burst of large deltas that outran the byte budget), nothing is replayed: the first live delta's `seq` does not follow the client's last one, and the sequencer refetches a full state — no client-side panic, no data corruption.
This is independent from the delta protocol's own `seq` field: the SSE id keeps the _transport_ in sync after a reconnect; the delta `seq` keeps the _application state_ in sync after each individual update.
See [DATA-FLOW-DEPLOYMENTS.md](https://github.com/flplima/tmuxy/blob/main/docs/DATA-FLOW-DEPLOYMENTS.md) for these sections: Transport: Tauri IPC (Desktop Version); Connection Lifecycle (Tauri); Real-World Deployment Scenarios; Additional API Endpoints.
## Adapter Pattern
The frontend reaches tmux through one Effect service, `TmuxTransport` (`tmuxy-ui/src/infra/transport/TmuxTransport.ts`): `connect`/`disconnect`, `invoke` for commands (fire-and-forget `run_tmux_command` among them), `query` for read-only tmux commands answered in-band on the control-mode connection — each failing with a typed `AdapterError` — and `subscribe`, one ordered `Stream` of tagged transport events (state, connection info, reconnection, keybindings, theme settings, logs, errors, fatal, detach, clipboard), so the first state never overtakes the connection info it depends on. Four Layers provide it (`layers.ts`): HTTP/SSE and Tauri IPC, which share one stream stage (`stateFeed.ts`: decode, sequence, and on a gap or an undecodable payload refetch the full state once at the last viewport the client reported), and the in-browser demo and v86 engines. Each Layer wraps a driver under `drivers/` whose events go through one generic hub (`infra/eventHub.ts`). The app builds one `ManagedRuntime` from the Layer for its environment (`infra/runtime.ts`), and the XState actors run every effect on it; tests swap in a scripted driver behind the real Layer (`src/test/transport.ts`).
## Connection Lifecycle (Web)
1. Client opens SSE connection: `GET /events?session=`
2. Server assigns a unique `connection_id`
3. Server checks if a `TmuxMonitor` exists for the session:
- **No monitor:** Spawns a new monitor (connects `tmux -CC`), stores handle in `SessionConnections`
- **Has monitor:** Subscribes to the existing broadcast channel and replays from the ring buffer if the client supplied a `Last-Event-Id`
4. Client receives `connection-info` event with connection ID and default shell
5. Client sends `get_initial_state` (via HTTP POST) with its viewport size (cols, rows)
6. Server stores the client size, computes the minimum viewport across all clients, and sends a resize command through the monitor's control mode connection
7. Client receives full state snapshot, then incremental deltas as tmux state changes. The stream (one full state, then deltas against the server's previous emission) is the client's state from its first full state on; the `get_initial_state` answer is a separate snapshot, so once the stream is synced it only fills panes the stream has not sent content for yet. It is the starting state only before the stream's first full state, after a delta sequence gap, or on a new connection (`adoptInitialState` in `tmuxy-ui/src/domain/deltaProtocol.ts`).
8. On disconnect: server removes the client, recomputes minimum viewport, and shuts down the monitor if no clients remain
## State Update Flow
```
tmux server
│ control mode stdout (%output, %layout-change, etc.)
▼
TmuxMonitor ← runtime (drives the aggregator,
│ executes the SideEffects it returns)
│ aggregator.step(event) → SideEffect[]
▼
StateAggregator (sans-IO state machine)
│ no I/O of its own — only describes what should happen
▼
SideEffect dispatch (refresh-panes, emit-state, store-image, ...)
│
│ emit_state path:
▼ ┌─────────────────────┐
│ StateEmitter trait │
└──────────┬──────────┘
┌──────────────────────┴──────────────────────┐
▼ ▼
SseEmitter TauriEmitter
(per-session ring buffer + broadcast) (app.emit())
│ │
┌─────┼─────┐ │
▼ ▼ ▼ ▼
Client1 Client2 Client3 Tauri frontend
```
The monitor multiplexes control-mode events with timer-driven flushes (throttle, settling, layout debounce, periodic sync) and external commands (resize, run-command, shutdown). The exact set of `tokio::select!` arms drifts as we tune timings; the durable contract is:
- The aggregator decides **what** state effects exist.
- The monitor decides **when** to flush them (throttle / debounce / settle).
- The emitter decides **where** they go (SSE broadcast vs Tauri event).
## Command Execution Flow
Frontend `adapter.invoke(cmd, args)` is decoded into a typed `ClientCommand` variant on the server (or routed straight through Tauri IPC). Mutating commands route through the monitor's control-mode connection — never through external subprocesses, because external `tmux` calls crash tmux 3.5a when a control-mode client is attached (see [TMUX.md](https://github.com/flplima/tmuxy/blob/main/docs/TMUX.md)).
Reads go the same way: scrollback, key bindings, theme settings and snapshot checks are command lists answered in-band by the monitor (`tmuxy-core/src/transport.rs`, shared by both transports). Sync helpers in `executor::*` remain only for CLI paths with no client attached.
**Two operations, the same on every transport.** `run_tmux_command` is a mutation: it is handed to the monitor's control-mode channel fire-and-forget and resolves to `null` — the result of a control-mode command arrives later as a state event, not as the response. `query_tmux` is a read: the monitor brackets the command with marker lines, collects the `%begin…%end` blocks between them, and answers with what the command printed (`MonitorCommand::RunCommandWithReply`); a tmux `%error` rejects the call with tmux's message. A mutation is bracketed the same way, with nobody waiting for the output: when tmux answers it with `%error`, the monitor emits the message as an error event (`SseEvent::Error` / `tmux-error`), which the frontend shows in the snackbar — the only way the user learns why a split or a kill did nothing. Keystrokes (`send-keys`, pinned or not) are the exception and go out bare: one command per key, and the only way they fail is a pane that is already gone from the layout. Web (`ClientCommand::QueryTmux`) and desktop (the `query_tmux` Tauri command) implement both identically, and the frontend reaches them as `adapter.invoke('run_tmux_command', …)` and `adapter.query(command)`. There is no subprocess path and no allowlist: nothing a client sends reaches a shell.
**Failures carry a kind.** A failed command answers `{ "error": "", "kind": … }` on both transports (the HTTP body of `POST /commands`, the object a Tauri command rejects with): `tmux` when tmux itself refused it (a query answered with `%error`), `unavailable` (no monitor, session or channel; a timeout), `invalid` (a malformed payload or id) and `forbidden` (a blocked command). The client maps `tmux` to `TmuxError` and the rest to `TransportError` (`tmuxy-ui/src/infra/transport/AdapterError.ts`). See `tmuxy_core::CommandError`.
```
Frontend
│ adapter.invoke(cmd, args)
▼
┌──────────────────────────────────────────┐
│ Backend │
│ ┌──────────────┐ ┌────────────────┐ │
│ │ Web: POST │ │ Tauri: │ │
│ │ /commands │ │ invoke() │ │
│ └──────┬───────┘ └─────────┬──────┘ │
│ ▼ ▼ │
│ ClientCommand variant Tauri command │
│ │ │ │
│ └──────────┬───────────┘ │
│ ▼ │
│ command_router::route_command │
│ │ │
│ monitor channel (mutations and │
│ in-band reads, transport.rs) │
└────────────────────┬─────────────────────┘
▼
Monitor stdin → tmux
```
## Delta Protocol
After the initial full state snapshot, the server sends incremental deltas to minimize bandwidth:
- Each delta has a `seq` number for ordering; a full state and the `get_initial_state` answer carry the `seq` they were taken at, so the delta right after either one is checked against it
- Deltas contain only changed fields: modified panes (content, cursor, metadata), added/removed panes, added/removed windows, active pane/window changes, status line changes
- The frontend merges deltas into its cached state via `handleStateUpdate()` in `tmuxy-ui/src/domain/deltaProtocol.ts`
- If a delta arrives with a sequence gap, the client requests a full state resync
## Keyboard Input Flow
1. User presses a key in the browser
2. `keyboardActor` captures the DOM `keydown` event. A hidden text input (`tmuxy-ui/src/utils/mobileKeyboard.ts`) owns browser focus on every device and follows the pane holding the keyboard (pane, float, or dock); an IME can only begin a composition inside an editable element. Text typed into it reaches the actor as `input` events and is sent once; its keydowns still run through the prefix and binding checks so a bound printable key wins
3. If in copy mode: key routed to `COPY_MODE_KEY` handler (handled client-side, see [COPY-MODE.md](https://github.com/flplima/tmuxy/blob/main/docs/COPY-MODE.md))
4. If prefix key pressed: enters prefix mode, waits for next key to match a binding
5. Otherwise the keydown is classified as text or chord (see below) and `keyboardActor` sends `SEND_TMUX_COMMAND` — literal text with `send-keys -l`, a chord as a tmux key name
6. The `KeyBatcher` in the adapter batches rapid keystrokes (e.g., typing "hello") into single `send-keys` commands
7. Command reaches tmux via control mode stdin
8. tmux processes the keystroke and sends `%output` event back through control mode stdout
9. Monitor emits a state delta with the updated pane content
10. Frontend re-renders the affected pane
### Text vs. chord
Step 5 cannot be decided by "no modifiers held". Keyboards compose characters through three paths that all set flags reading like a chord:
| Path | What the keydown looks like | Example |
| --- | --- | --- |
| Dead key | the finished character on one keydown, stamped `keyCode` 229 because the OS composed it through the IME | `´` then `a` → `á` |
| macOS Option as compose key | `altKey` set, but the letter already replaced by a non-ASCII character | Option+c → `ç` |
| AltGr third level | legacy ctrl+alt flags; only the `AltGraph` modifier state separates it from a real Ctrl+Alt chord | AltGr+Q on ABNT2 → `@` |
Forwarding one of these as a tmux key name is not rejected — tmux accepts `send-keys M-ç` and delivers ESC + `ç`, an unbound meta sequence the application discards. The classifier reads a non-ASCII key under bare Alt (and any key with `AltGraph`) as text, an ASCII key under Alt as a chord, and anything under Ctrl or Cmd as a chord. IME composition bypasses keydown and sends the committed string whole on `compositionend`.
## Servers, Sessions and the 4-Step Connect Flow
A server is one tmux socket — local (`-L` / `-S` under `/tmp/tmux-$UID/`) or remote over SSH — and holds zero or more sessions. Saved entries live in `~/.config/tmuxy/servers.json` (`packages/tmuxy-core/src/servers.rs`), enriched on list with live local sockets from `packages/tmuxy-core/src/servers/discovery.rs`. Switching among them is the session switcher (`components/SessionMenu.tsx`, and `components/SessionList.tsx` on the detached overlay), both drawing `components/SessionPicker.tsx`: a filter-as-you-type box over `sessionSwitcherSpec`'s list, which scrolls inside a capped box. Reaching a new one is one flow, `tmuxy session connect` (`packages/tmuxy-cli/bin/tmuxy/session-connect`), which `tmuxy connect` with no arguments and the switcher's "Connect over SSH…" float both run. It asks for the destination, then takes 4 stages:
```
[1. ssh / localhost] ──► [2. tmux -V] ──► [3. listing sessions] ──► [4. attach]
```
Each stage prints `✓` on success or `✗` with verbatim `ssh`/`tmux` stderr on failure. The server is saved to `servers.json`, and the attach is published as `TMUXY_CONNECT_*` for the desktop app to pick up (see [TMUX-CONFIG.md](https://github.com/flplima/tmuxy/blob/main/docs/TMUX-CONFIG.md)).
---
## Related
- [ARCHITECTURE.md](https://github.com/flplima/tmuxy/blob/main/docs/ARCHITECTURE.md) — the components each transport connects
- [STATE-MANAGEMENT.md](https://github.com/flplima/tmuxy/blob/main/docs/STATE-MANAGEMENT.md) — what the state updates flowing over these transports contain
- [TMUX.md](https://github.com/flplima/tmuxy/blob/main/docs/TMUX.md) — why mutations must ride control mode rather than a subprocess
- [SECURITY.md](https://github.com/flplima/tmuxy/blob/main/docs/SECURITY.md) — what the unauthenticated `POST /commands` endpoint exposes
- [INVARIANTS.md](https://github.com/flplima/tmuxy/blob/main/docs/INVARIANTS.md) — the sequencing promises the stream makes (`S` and `C` rows) and what tests them
========================================================================
docs/DESKTOP.md
https://github.com/flplima/tmuxy/blob/main/docs/DESKTOP.md
========================================================================
# Desktop-only features
tmuxy is one React UI served two ways: the **web app** (`tmuxy-server`, a
browser tab) and the **desktop app** (`tmuxy-tauri-app`, the same UI in a
WKWebView on macOS or WebKitGTK on Linux, over native IPC). Everything about
tmux — panes, tabs, floats, groups, copy mode, widgets, the CLI — is the same
on both. What follows is the list of things only the desktop app has, and
why: each one needs something a browser tab is not given.
The desktop app runs on **macOS and Linux**. There is no Windows build.
| Feature | Why the web cannot have it | Where it lives |
| --- | --- | --- |
| Trackpad gestures: two-finger slide between tabs, pinch to zoom a pane, pinch in to the "all tabs" view | A page never learns when the fingers leave the trackpad. A slide reaches it as `wheel` events with no phase, and the only sign of a lift is silence — the same silence as fingers resting on the pad, so every web implementation either snaps back under a pause or guesses from a timer. The shell sees the real phases (`NSEvent.phase` / `momentumPhase` on macOS; GTK's smooth-scroll stream and its stop event on Linux) and hands them to the page. | `packages/tmuxy-tauri-app/src/gestures.rs` → `window.tmuxyGesture` → `machines/actors/gestureActor.ts`; drawn by `GestureStage.tsx` |
| Several GUI windows on one tmux session, with the Window menu and its shortcuts | A browser tab is one window. | `packages/tmuxy-tauri-app/src/windows.rs`, `components/menus/WindowMenu.tsx`, `utils/guiWindows.ts` |
| Window styles (iTerm2's set), the native title bar, dragging the window by its status bar | Window chrome belongs to the shell. On macOS the title bar is transparent and full-size — the status bar is the title bar — so "No Title Bar" hides the three window buttons (and the page drops the room it keeps for them) rather than rewriting the window's decorations, which would rebuild the style mask without the full-size-content bit and leave a bare title strip above a pushed-down page. A style change that leaves full screen or a maximized frame waits for that exit to land before it applies. Each applied style is announced to the page as `window-style`. | `packages/tmuxy-tauri-app/src/window_style.rs`, `titlebar.rs`, `components/StatusBar.tsx`, `utils/guiWindows.ts` |
| Window transparency: the blurred desktop behind the chrome (macOS), and View ▸ Enable Window Transparency to turn it off | A page cannot see through its own browser window; the desktop's window is transparent and puts a native blur behind the page. The switch is `@tmuxy-transparency`, remembered in `tmuxy.state.json`: off, the page paints its surfaces opaque and every window's blur is cleared at once; the opacities and `@tmuxy-blur` keep their values for when it goes back on. | `packages/tmuxy-tauri-app/src/gui/window_setup.rs`, `monitor/emitter.rs` (`emit_theme_settings`)
`packages/tmuxy-ui/src/utils/appearanceManager.ts` | | Native menus: the menu bar (macOS) and native popups for every context menu (macOS + Linux) — not the session switcher, whose filter box a native popup cannot hold — built from the frontend's menu spec; the in-app menus are the fallback, and what `@tmuxy-native-menus off` keeps | A page has no menu bar and no native popups; the web app draws the same specs with its own floating menu. The desktop sets the bar from the page (`NativeMenuBar`), debounced on the spec's shape and on window focus so the focused window owns it; a popup that is refused falls back to the in-app menu in the same gesture. An item's icon is the in-app menu's own strokes, rasterised in the page in the system's menu text colour — the platform takes a plain bitmap, not a template it recolours — so the bar is rebuilt when the system appearance changes. | `packages/tmuxy-ui/src/domain/menus/` (the specs, and `icons.ts`)
`packages/tmuxy-ui/src/infra/menus/nativeMenu.ts`, `nativeIcons.ts`
`packages/tmuxy-ui/src/components/menus/NativeMenuBar.tsx`, `useNativeMenu.ts`, `menuHost.ts` | | Other servers in the session menu, and `tmuxy session connect` to add one | A web client is already talking to the one server that serves it. | `components/SessionMenu.tsx` | | Links opened by the system (browser, viewer), images served over an app scheme | A page may only open a tab; the shell asks the OS. The web app serves the same images over an HTTP route instead. | `utils/openUrl.ts`, `components/Terminal.tsx` (`tmuxyimg:`) | | Restart the app | A page can only reload itself. | `utils/restartApp.ts` | The switch for all of it is `isTauri()` (`utils/platform.ts`), set from the Tauri IPC bridge being present; `main.tsx` also marks the document with `data-desktop` for styling. Nothing else probes the platform. ## Trackpad gestures The pane grid follows the fingers (`GestureStage` writes each step as an inline `transform`, nothing re-renders), the tab being pulled in is drawn beside it from its real panes, and the release decides: past the mark the tab switches at once and the grid runs home; short of it the grid slides back. The decision and the drawing are `machines/app/actions/gestures.ts` and `utils/gestures.ts`; the only thing the shell supplies is _where the fingers are and when they lift_. **The contract** (one call per native scroll event the shell claims): ``` window.tmuxyGesture('slide', phase, dx, dy, t) ``` - `phase`: `began` · `changed` · `ended` · `cancelled` · `momentum` · `momentum-ended`. The fingers are on the pad from `began` to `ended` or `cancelled`, for as long as that takes — a pause is simply no event. The momentum phases are the tail macOS sends after a lift; Linux has none. - `dx`, `dy`: points, positive when the fingers move right or down (the content follows the fingers). - `t`: the event's own timestamp in seconds, so the speed a release carries on at is measured on the device's clock. **What the shell claims.** A scroll stream whose first steps are horizontal (the same axis rule the actor used to apply, `|dx| > 1.5·|dy|`) from a touchpad (`hasPreciseScrollingDeltas` on macOS; a touchpad device on Linux), while `@tmuxy-gesture-swipe-tabs` is on — the page is told the flag through a command whenever it changes. The claimed events, and the momentum tail that follows them, never reach the webview: the page does not scroll on them, and nothing has to `preventDefault`. Vertical scrolls and mouse wheels are left alone. A slide over a pane whose program tracks the mouse is still claimed: Ghostty gates its tab switch on that, but Claude Code and nvim track the mouse, so the gate left most tabs with no slide at all, and what such a program gives up is only a horizontal two-finger scroll. **macOS** installs an `NSEvent` local monitor for `.scrollWheel` from Rust (`objc2-app-kit`): it runs before `-[NSApplication sendEvent:]`, so before the WKWebView, and returning `nil` is what swallows an event. It does not consult the Trackpad preference "Swipe between pages": that switch is about page history, the two-finger scroll arrives whatever it says, and `@tmuxy-gesture-swipe-tabs` is the one setting for the tab slide. **Linux** connects `scroll-event` on the WebKitGTK widget: two-finger motion is `GDK_SCROLL_SMOOTH` ending in a stop event at lift-off, before any kinetic scrolling, and `Propagation::Stop` is what swallows it. A pinch needs none of this: WebKit reports it to the page itself as `gesturestart` / `gesturechange` / `gestureend` with the cumulative scale, and those have phases. The actor keeps a ctrl+wheel pinch too, which is how libinput's pinch arrives on WebKitGTK. The web build runs none of the gesture code: `AppProvider` leaves the machine's no-op gesture actor in place off the desktop, so the browser keeps its own wheel, pinch-zoom and history swipe. The Storybook stories under _Mocked App / Trackpad Gestures_ opt the actor in (`config.desktopGestures`) and drive it with the shell's own calls, which is how the JS half is tested on the web build; the native half is covered by the desktop suite. ======================================================================== docs/INVARIANTS-UI-LIFECYCLE.md https://github.com/flplima/tmuxy/blob/main/docs/INVARIANTS-UI-LIFECYCLE.md ======================================================================== # Invariants — UI-mode records, session lifecycle The `U` and `L` rows of the ledger in [INVARIANTS.md](https://github.com/flplima/tmuxy/blob/main/docs/INVARIANTS.md), which holds the column legend, the seam prefixes and the other rows. Same rules: cite a row in a commit on its seam, and a row's enforcer is a test title that exists. ## U — UI-mode records | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | U1 | At most one copy-mode record per pane; exit closes only if seen in mode, and cooldown ignores stale `in_mode`. | ordering | `copyMode.ts`, `docs/COPY-MODE.md` | `reconcilePaneMode.test.ts › a snapshot…`, `› is ignored…`; `copyMode.test.ts › the end of a blink…` (examples) | | U2 | Resize preview holds for two matching quiet updates or a 2 s fallback that never clobbers newer resizes. | liveness | `appMachine.ts`, `layout.ts` | `layout.test.ts › the 2s RESIZE_COMPLETED…`, `› …does not clobber…` (two-quiet-updates hold is —) | | U3 | Only the latest scrollback fetch per pane applies; different panes fetch concurrently without cancelling. | ordering | `tmuxActor.ts` | `tmuxActor.test.ts › interrupts the previous fetch…`, `› different paneId fetches…` | | U4 | Group switch holds swap over stale snapshots, drops if a pane dies, and never selects parked/stash panes. | safety | `appMachine.ts`, `windowOps.ts` | `tests/7-regression-bugs.test.js › Clicking three pane-group tabs…` [nightly] (E2E only) | | U5 | Every menu renders a `domain/menus/` spec and dispatches via `executeMenuAction` or machine events. | safety | `domain/menus/`, `SpecMenu.tsx`, `nativeMenu.ts` | `specs.test.ts`; `menuActions.test.ts › sends the op of every op-backed item…` | ## L — session lifecycle | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | L1 | Snapshot restore runs after attach/sync and holds autosave from before spawn so placeholders aren't saved and live sessions aren't overwritten. | ordering | `transport.rs`, `session_snapshot.rs` | `session_snapshot.rs › a_restore_in_flight…`; `transport.rs › a_running_session_is_not_restored_over` | | L2 | Restore plans avoid `new-window`, split panes in index order, apply layouts after splits, and restore focus last. | ordering | `session_snapshot.rs` | `session_snapshot.rs › a_restore_never_uses…`, `› focus_is_restored_last`, `› a_reordered_group…`; `tests/15-session-snapshot.test.js › the shape survives…` [nightly] | | L3 | New sessions are never created on a closing connection. | safety | `appMachine.ts`, `HttpAdapter.ts` | `tests/4-session-connectivity.test.js` (regression example) | | L4 | Disconnect during `connecting` transitions to `disconnected`, never a stuck spinner. | liveness | `appMachine.ts` (`connecting`) | `disconnected.test.ts › ends a connection…`, `› ends a reconnection…` | ======================================================================== docs/INVARIANTS.md https://github.com/flplima/tmuxy/blob/main/docs/INVARIANTS.md ======================================================================== # Invariants Cross-layer promises, in one place, each with the test that holds it. Doc comments explain _how_ an invariant is kept; this file indexes _what_ is promised across a seam and _whether anything checks it_. Module-local assertions stay in code. | Column | Meaning | | --- | --- | | `Id` | Stable name prefixed by seam. Cite in commits and test titles; never renumber. | | `Kind` | `ordering` (A before B), `safety` (X never happens), `liveness` (Y happens within a bound). Ordering/liveness want a property or interleaving test, not one scripted run. | | `Stated in` | Code defining the mechanism. | | `Enforced by` | `file › test title`. `(example)` marks a single scripted run standing in for a property; `—` means uncovered. | | Prefix | Seam | | --- | --- | | `A` | Control-mode events → `StateAggregator` → `SideEffect`s (`tmuxy-core`, sans-IO) | | `M` | `SideEffect`s → `TmuxMonitor` dispatch → tmux commands (`tmuxy-core` runtime) | | `X` | Tmux connection and command router (`tmuxy-core`) | | `S` | Server → client stream: SSE seq, ring buffer, first state (`tmuxy-server`) | | `C` | Client stream stage: sequencer, resync, per-frame coalescing (`tmuxy-ui`) | | `Q` | Client command queue and keyboard batching (`tmuxy-ui`) | | `P` | Client model: predictions and reconcile (`TmuxClientModel`) | | `U` | UI-mode records: copy mode, resize preview, scrollback, menus | | `L` | Session lifecycle: attach, restore, autosave | See `AGENTS.md`: a bug-fix commit on a seam cites or adds a row, and a row's enforcer is a test title that exists. ## A — aggregator effects | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | A1 | Step effect order: `AdoptUntaggedWindows` → `StoreImages`/`WriteClipboard` → `SendTmuxCommand`/`CommandReply` → `RefreshAfterWindowAdd` → `RefreshPanes` → `ResumePane` → `EmitState`. | ordering | `state.rs` (`step_at`) | `state_aggregator_props.rs › effects_within_a_step…`, `› the_seam_events_reach…` (property) | | A2 | Window add/close relists before emit: `RefreshAfterWindowAdd` precedes `EmitState`. | ordering | `state.rs` (`step_at`) | `state_aggregator_props.rs › window_add_refresh_precedes_any_emit` (property); A1 | | A3 | Tag-only steps promote to `Window` change so untagged windows emit immediately. | liveness | `state.rs` (`step_at`) | — | | A4 | Captures route by marker, never arrival order or shape; dead-pane captures are discarded. | safety | `state.rs` (`queue_captures`) | `state.rs › capture_routing…`, `› capture_marker…`, `› capture_for_dead_pane…`, `› a_failed_capture…`; `state_aggregator_props.rs › queue_captures…` (property) | | A5 | Output between reply markers skips pane/window parsers; mismatched END fails the older reply cleanly. | safety | `state.rs` (replies) | `state.rs › a_query_between_markers…`, `› a_mismatched_end_marker…`, `› non_list_panes_output…` | | A6 | Reply-wrapped commands send `BEGIN`, cmd, `END` in order, each its own line; `END` runs even after `%error`. | ordering | `state.rs` (`reply_wrapped_lines`); `guest/controlMode.ts` (`TrackedCommandBook`, `CaptureBook`) | `state.rs › reply_markers_collect…`, `› a_failed_block_fails…`; `reply_channel.rs › replies_come_back…` (real tmux); `trackedCommandBook.test.ts › sends its markers as their own lines…`, `› a command tmux refuses resolves…`; `AppOptimistic › RefusedSplitReportsTmuxsReason` (v86) | | A7 | Settling emits at most one `EmitState` and clears by safety ceiling (silently if idle). | liveness | `state.rs` (`arm_settling`, `tick`) | `state_aggregator_props.rs › settling_window…`, `› settling_round_trips`, `› suppressed_window…` (properties); `state_aggregator_sideeffects.rs › tick_after_deadline…`, `› tick_safety_timeout…` | | A8 | `change_type` surfaces during suppressed emissions so settling timers extend on window changes. | liveness | `state.rs` (`step_at`) | `state_aggregator_sideeffects.rs › change_type_is_surfaced…`, `› step_at_extends…`; `state_aggregator_props.rs › emit_state_change…` (property) | | A9 | Moved panes hold `%output` until capture lands and drop old content; others keep content during capture. | safety | `state.rs` (refill) | — | | A10 | Stray `%end`/`%error` without `%begin` never produces a command response. | safety | `parser.rs` | `parser.rs › stray_end_without_begin_is_ignored`; `pane_bytes_props.rs › …arbitrary_lines`, `› …hostile_notifications` (properties) | | A11 | A pane's output bytes reach the emulator exactly, wherever tmux cut its `%output` lines — a character split across lines included; pane output is never read as text. | safety | `parser.rs` (`parse_line`), `connection/spawn.rs` | `pane_bytes_props.rs › output_cut_anywhere_carries_the_bytes` (property); `control_mode_push_api.rs › a_character_split_across_output_lines_reaches_the_screen_whole` | | A12 | An in-browser guest's query reply is what came between its markers less tmux's notifications: another pane's `%output` arriving meanwhile stays out, a `run-shell` job's lines (printed after its block closed) stay in, and a block closes only on the `%end`/`%error` that repeats its `%begin`. | safety | `guest/controlMode.ts` (`CaptureBook`) | `trackedCommandBook.test.ts › keeps tmux's notifications out of the reply…`, `› keeps a line inside a block…` | ## M — monitor dispatch | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | M1 | Monitor dispatches step effects one-to-one in exact aggregator order. | ordering | `monitor.rs` | `monitor.rs › a_step_sends_its_effects_commands_in_effect_order` (example; order is A1) | | M2 | Window add/close and initial sync send `list-panes` before `list-windows` (stash last) in monitor and wasm. | ordering | `state.rs`, `monitor.rs` | `monitor_command_order.rs › a_window_relist_sends_panes_before_windows…` | | M3 | `RefreshPanes` sends `list-panes` before per-pane `capture-pane`s for fresh cursor positions. | ordering | `state.rs`, `monitor.rs` | `monitor_command_order.rs › a_pane_refresh_lists_panes_before_every_capture` (property) | | M4 | Windows set `window-size manual` and `aggressive-resize off` before first `resizew` so SIGWINCH never races captures. | ordering | `monitor.rs` | `monitor.rs › a_window_is_told_to_stop_sizing_itself_before_it_is_sized` (example) | | M5 | Client-size pass runs after the step's own effects, never ahead of `EmitState`. | ordering | `monitor.rs` | — | | M6 | `GetState` waits for sync, captures, and queued refreshes (15 s cap); first state awaits resize up to 750 ms. | liveness | `monitor.rs`, `transport.rs`, `sse.rs` | `sse.rs › a_clients_first_state…`, `› a_first_state_waits…`, `› a_client_that_sets_no_size…`; `initial_state.rs`, `initial_state_history.rs` (real tmux) | | M7 | Every reply resolves within `REPLY_TIMEOUT`, or immediately on send failure. | liveness | `monitor.rs` | `reply_channel.rs` (success/error; timeout is —) | | M8 | Reply ids are monotonic so expired reply blocks never match newer ones. | safety | `monitor.rs` | — | | M9 | `%session-changed` for another session is filtered before reaching the aggregator. | safety | `monitor.rs` | — | | M10 | Resizes retry <=3 times per size after new tmux reports; new sizes reset count; unmonitored sizes re-request. | liveness | `monitor.rs`, `sse.rs` | `monitor.rs › a_window_tmux_did_not_resize…`, `› …not_asked_again…`, `› a_size_tmux_keeps_refusing…`, `› a_new_size_starts…`, `› a_window_with_no_panes…`; `sse.rs › a_size_that_reached_no_monitor…` | | M11 | UI choices re-apply after `source-file` (UI wins), and enforced settings after both. | ordering | `monitor.rs` | `enforced_settings.rs › connecting_enforces_the_settings…` (final values; order is —) | ## X — the tmux connection and the router | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | X1 | Client commands keep pins intact, refuse `resize-window`, and rewrite `new-window` to `splitw ; breakp`. | safety | `command_router.rs` | `command_router.rs › a_pinned_list…`, `› resize_window_is_refused…`, `› new_window_is_rewritten…`; `pinned_split_race.rs › a_pinned_split_survives…` (real tmux) | | X2 | No tmux subprocess runs while control mode is attached; all reads/writes use control mode (`docs/TMUX.md`). | safety | `command_router.rs`, `docs/TMUX.md` | `tests/cli/` (socket flag); ESLint ban on `tmuxExec` in E2E (`docs/TESTS.md`) | | X3 | Control-mode client is never killed with SIGKILL; it detaches with a 3 s grace period. | safety | `connection.rs` | — | | X4 | Attach and create serialize under one lock via `new-session -A`; attach never creates a session. | safety | `connection.rs` | `connection.rs › creating_a_session_attaches…`, `› attaching_never_creates_a_session` (lock is —) | ## S — server stream | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | S1 | Per-session broadcast seqs count from 1 (0 = empty) and the ring preserves concurrent broadcast order. | ordering | `tmuxy-server/src/state.rs` | `state.rs › seqs_are_monotonic…`, `› concurrent_broadcasters_keep…`, `› replay_since_returns…`, `› ring_buffer_drops_oldest…` | | S2 | No duplicates or fillable gaps: connect replays from `Last-Event-Id + 1`; lag replays ring tail (dropping `<= last_replayed`) or leaves one edge gap for C1. | safety | `sse.rs` | `sse.rs › the_first_broadcast…`, `› a_lagged_stream…`, `› a_client_that_stops_reading…`, `› a_client_beyond_the_buffer…` | | S3 | Reconnecting beyond the ring serves no replay so the first live delta triggers a gap refetch (C1). | safety | `sse.rs` | `sse.rs › a_client_beyond_the_buffer…` (`can_replay_from`; connect path is —) | | S4 | Streams end on shutdown; the last client leaving keeps the monitor for a 2 s grace period. | liveness | `sse.rs` | `sse.rs › open_stream_ends…`, `› the_last_client_leaving_keeps…` | | S5 | Monitor reconnect gives up after 5 short runs and broadcasts fatal so clients stop retrying. | liveness | `sse.rs` | — | ## C — client stream stage | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | C1 | Deltas apply only to their base state; missing, misordered, repeated or undecodable deltas drop and refetch full state. | safety | `stateSequencer.ts`, `stateFeed.ts`, `deltaProtocol.ts` | `stateSequencer.test.ts › asks for a resync…`, `› treats a payload…`; `stateFeed.test.ts › a payload…`; `deltaProtocol.test.ts`; `HttpAdapter.test.ts › a state update…` | | C2 | Pre-full-state deltas are ignored; full states/answers carry `seq` so a delta lost after sync triggers a gap. | safety | `stateSequencer.ts`, `deltaProtocol.ts` | `stateSequencer.test.ts › ignores a delta…`, `› a delta lost…`; `stateFeed.props.test.ts › never paints backwards…` (property); `deltaProtocol.test.ts › no gap…` | | C3 | At most one refetch runs in flight; arriving deltas are boundedly held, and failed refetches retry on next delta. | safety | `stateFeed.ts`, `stateSequencer.ts` | `stateFeed.props.test.ts › never paints backwards…` (property); `stateFeed.test.ts › a gap refetches…`; `stateSequencer.test.ts › the deltas…` | | C4 | Post-refetch publish merges the answer, held deltas `> seq`, and queued deltas monotonically; held gaps refetch. | ordering | `stateFeed.ts`, `stateSequencer.ts` | `stateFeed.props.test.ts › never paints backwards…` (property) | | C5 | Once synced, initial-state answers only fill empty panes; before sync or after a gap, they set starting state. | ordering | `deltaProtocol.ts`, `stateSequencer.ts` | `HttpAdapter.test.ts › an initial-state answer…`, `› …does not take back…`, `› after a gap…`; `stateSequencer.test.ts › after a gap…`; `stateFeed.props.test.ts › never paints backwards…` (property) | | C6 | New connections or session switches reset sequencing so old delta seqs never gate new streams. | safety | `HttpAdapter.ts`, `TauriAdapter.ts` | — | | C7 | Transport publishes events in arrival order (including pre-`connect` subscribers); web frame-hold never reorders state ahead of earlier events. | ordering | `TmuxTransport.ts`, `events.ts`, `stateFeed.ts` | `TmuxTransport.test.ts › publishes what the driver pushes…`; `stateFeed.test.ts › publishes a full state…` (examples; per-frame hold is —) | | C8 | States coalesce to one per frame (latest wins); older states never paint after newer ones. | ordering | `stateFeed.ts` | — | | C9 | Each pane id appears at most once in client state, and a delta removes exactly the panes it nulls. | safety | `deltaProtocol.ts` | `deltaProtocol.props.test.ts` (property) | | C10 | Dropped/silent streams retry with backoff, online/visibility/Retry reconnect immediately, refused streams stop, and `EventSource` is singleton. | liveness | `HttpAdapter.ts` | `HttpAdapter.test.ts › keeps retrying…`, `› ten drops…`, `› a pinged stream…`, `› a stream the server refuses…`, `› dedupes concurrent…`; `tests/4-session-connectivity.test.js › six drops…` [nightly] | ## Q — command queue and input | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | Q1 | Mutations serialize in issue order so splits never land in a tab a queued `select-window` is leaving. | ordering | `HttpAdapter.ts`, `TauriAdapter.ts` | `HttpAdapter.test.ts › runs run_tmux_command…` (example; Tauri is —); `tests/4-session-connectivity.test.js › 2 panes → reload…` | | Q2 | Keystroke batches share the mutation queue and non-`send-keys` flush pending batches first. | ordering | `TauriAdapter.ts`, `HttpAdapter.ts`, `keyBatching.ts` | `keyBatching.test.ts › flushAll drains…` (batcher only; mutation interleaving is —) | | Q3 | Lone keystrokes send on leading edge; literal/special order holds; hex-byte sends (`-H`) never batch. | ordering | `keyBatching.ts` | `keyBatching.test.ts › sends an isolated…`, `› preserves literal→special…`, `› preserves special→literal…`, `› never intercepts…`; `tests/1-input-interaction.test.js › Type → backspace…` | | Q4 | Client reads (`query_tmux`) run unserialized; order vs. mutations holds in-band on control mode (A6). | ordering | `HttpAdapter.ts`, `docs/DATA-FLOW.md` | — | ## P — client model | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | P1 | Placeholder (predicted) pane or window ids are never sent to tmux as targets — not by a dispatched op, a binding's pin, or a tab switch. | safety | `commands.ts`, `dispatch.ts`, `keyboardActor.ts` (`bindingPin`), `layout.ts` (`layout_selectTab`) | `dispatch.test.ts › leaves the target to tmux…`, `› once tmux has confirmed them…`; `keyboardActor.bindings.test.ts › leaves a placeholder window and pane unpinned…`; `selectTab.test.ts › switches the view to a placeholder tab…` | | P2 | Ops mark in-flight before send and always settle: unacked sweeps at `OP_STALE_TIMEOUT_MS`, acked at `OP_ACKED_STALE_TIMEOUT_MS`, errors roll back. | liveness | `TmuxStore.ts`, `model.ts`, `types.ts` | `model.test.ts › stale ops…`, `› in-flight ops…`; `TmuxStore.test.ts › dispatch rolls back…`, `› TransportError…` | | P3 | Pending ops re-reconcile on a timer so time-based verdicts fire on an idle stream. | liveness | `TmuxStore.ts` | `tests/7-regression-bugs-b.test.js › Creating tabs then switching keeps the rendered tab in sync with tmux` [nightly] (its `waitForSettled` holds while a switch is pending; on an idle terminal only this timer can release it) | | P4 | Confirmed focus ops linger (`FOCUS_CONFIRM_LINGER_MS`), yield to a 3rd pane after `FOCUS_SUPERSEDE_GRACE_MS`, drop on newer focus, and skip dead panes. | ordering | `ops.ts`, `model.ts` | `ops.test.ts › holds a confirmed…`, `› holds through stale…`, `› holds briefly…` (examples; `dropSupersededFocusOps` is —); `tests/2-layout-navigation.test.js › Clicking an inactive tab…` [nightly] | | P5 | In-flight splits never claim the same pane id within a reconcile pass. | safety | `ops.ts` (`reconcile`) | — | | P6 | `NewWindow` placeholders match only tab windows with a pane; the patch hides unmatched newborns. | safety | `windowOps.ts` | `ops.test.ts › stays pending until…`, `› patch hides a tab-typed-but-paneless…` | | P7 | Session switch clears pending ops and committed state before the new session's first state. | ordering | `tmuxStoreActor.ts`, `appMachine.ts` | `workflows.test.ts › TmuxStore — clear…` (store only; order vs. first state is —) | | P8 | `MoveWindow` predicts dropped strip order immediately and holds until server places the window beside its anchor. | ordering | `windowOps.ts`, `tabOverview.ts` | `ops.test.ts › MoveWindow predict…`; `tests/2-layout-navigation.test.js › a tab dragged along the strip…` | | P9 | A focus the client shows is one it asked tmux for: every optimistic active-pane change goes out with an op that makes tmux select that pane, so tmux's next state confirms it rather than taking it back. | safety | `focusAndDrag.ts` (`focusAndDrag_selectPaneGroupTab`, `focusAndDrag_focusPane`) | `AppPaneGroups.stories.tsx › ClickingTheShownMembersTabTakesTheFocus` [v86] (the group tab of the member on screen; other focus paths are —) | ## U and L — UI-mode records, session lifecycle In [INVARIANTS-UI-LIFECYCLE.md](https://github.com/flplima/tmuxy/blob/main/docs/INVARIANTS-UI-LIFECYCLE.md), the ledger's companion. ## What the ledger says today - **Uncovered (`—`)**: A3, A9, M5, M8, M9, X3, S5, C6, C8, Q4, P3, P5. - **Partial**: M7 (timeout), M11 (order), X4 (lock), S3 (connect), C7 (frame hold), Q1 (Tauri), Q2 (interleaving), P4 (`dropSupersededFocusOps`), P7 (first-state order), U2 (2-update hold). - **Example only**: M1, M4, Q1, U1. - **Property**: C3–C5 (`stateFeed.props.test.ts`) and S2 (`sse.rs`). ## Related - `docs/STATE-MANAGEMENT.md`, `docs/DATA-FLOW.md`, `docs/TESTS.md`, `docs/TMUX.md` ======================================================================== docs/NON-GOALS.md https://github.com/flplima/tmuxy/blob/main/docs/NON-GOALS.md ======================================================================== # Non-Goals This document describes what tmuxy intentionally does NOT do. These are conscious decisions, not missing features. ## Target Audience Tmuxy is for people whose work **runs in tmux** — long-lived sessions, agents left working, a laptop picked up from a phone — whether they know tmux's keys yet or not. The README's "beginner-friendly" means a newcomer can drive it with the mouse, the menus and the key hints and learn the keys as they go; it does not mean tmuxy courts people who do not want tmux at all, and it is not a replacement for a standalone terminal emulator. ## Core Philosophy **Tmuxy is a tmux UI, not a terminal emulator.** We delegate terminal emulation to tmux. We render what tmux gives us. We don't re-implement what tmux already handles well. --- ## Non-Goals ### 1. Full Terminal Emulation We do NOT maintain a local terminal state machine (cursor position, SGR attributes, scroll regions). Tmux already does this. ### 2. Live Local Scrollback Buffer We do NOT continuously buffer a pane's output on the client. tmux is the only owner of scrollback history. What we _do_ render client-side is a **view** of it: the mouse wheel in a normal shell opens the client-only scroll view, which renders scrollback **fetched on demand from tmux** (`get_scrollback_cells` → `capture-pane`), lazily in chunks as the user scrolls, and discards it when the view closes; tmux itself is never put into copy mode by the wheel — see [COPY-MODE.md](https://github.com/flplima/tmuxy/blob/main/docs/COPY-MODE.md). In alternate screen (vim, less), the wheel sends arrow keys. So scrolling works and scrollback renders client-side, but the history always comes from tmux at scroll time, never from a buffer we keep in sync with live output. **Why no live buffer?** - Duplicates tmux's work - Unbounded memory growth on the client - State divergence risk against the pane tmux actually holds - Lost on page refresh anyway ### 3. Local Search (Cmd+F / Ctrl+F) We do NOT implement browser-style find-in-page for terminal content. Users search via tmux copy mode (`Prefix + [` then `/` or `?`). **Why?** A find-in-page would have to search history we don't hold — the copy-mode `lines` map covers only the chunks fetched around the current viewport, so a client-side search would silently miss matches outside it. Searching in tmux searches the real buffer. ### 4. Local Echo / Input Prediction We do NOT predict keystrokes locally to reduce perceived latency (like mosh does). Every keystroke round-trips through tmux. **Why?** - Only benefits high-latency connections - Tmuxy is primarily for local/LAN use - Adds significant complexity - Risk of prediction errors ### 5. Binary Protocol / Compression We use JSON over SSE/HTTP, not binary encoding (MessagePack, Protobuf) or compression. **Why?** - For local tmux, bandwidth is not the bottleneck - JSON is debuggable and simple - Premature optimization ### 6. Unicode Width Calculation We do NOT maintain our own `wcwidth` tables for character width calculation. Tmux renders the grid; we display it. Text selection does not need one either: copy mode selects cells of the grid tmux laid out, one cell per column, where a wide glyph's second column is a cell of its own (blank). A selection's columns are therefore tmux's columns, with no width worked out on our side. See [COPY-MODE.md](https://github.com/flplima/tmuxy/blob/main/docs/COPY-MODE.md). ### 7. Canvas/WebGL Rendering We use DOM rendering (spans), not canvas or WebGL. **Why?** - DOM is simpler and works - Native text selection works with DOM - Accessibility (screen readers) works with DOM - Canvas/WebGL is premature optimization ### 8a. Resuming Processes Across a Restart A session snapshot (`tmuxy_core::session_snapshot`) records a session's SHAPE — tabs, splits, floats, sidebars, groups, each pane's directory and the program it ran — and rebuilds that. It does not and cannot resume the processes: no portable mechanism checkpoints a live process with its ptys and sockets, and the one that tries (CRIU) is Linux-only and privileged. A restored pane offers its program at the prompt; a program that wants to come back to where it was says how through `tmuxy pane restore-cmd` (`claude --resume`, `nvim -S`).
Shell variables, half-typed input and remote connections are gone with the
process, as they are in tmux-resurrect and Zellij. A pane group comes back whole —
its members parked out of view return parked, in pane-id order — but the
position of the visible member among them is not kept: it comes back first.
### 8. Simultaneous Multi-Session Views
We do NOT render more than one tmux session at a time. One tmuxy instance is attached to one session; there is no split view across sessions and no cross-session pane layout.
Switching between them _is_ supported — the session switcher (a dropdown that filters as you type, `SessionMenu.tsx`) and `tmuxy session switch` reattach without a page reload. The switcher also attaches to a different tmux socket, locally or over SSH, by retargeting the running monitor rather than relaunching.
### 9. SSH via Web Server
SSH connections (remote server attachment) are only available in the Tauri desktop app. The web server accesses the host's local tmux; there is no browser-to-SSH tunnel.
---
## What We DO
- Render tmux panes accurately
- Handle keyboard input and forward to tmux
- Support tmux window/pane operations (split, resize, close, navigate)
- Provide a clean, modern UI for tmux
- Parse OSC sequences for hyperlinks and clipboard (targeted, not full emulation)
- Decode terminal image protocols (iTerm2 OSC 1337, Kitty Graphics, Sixel) and render them as inline `![]()
` placements — see [RICH-RENDERING.md](https://github.com/flplima/tmuxy/blob/main/docs/RICH-RENDERING.md)
- Auto-reconnect on connection drop
- Handle flow control for stability
---
## Revisiting Non-Goals
These decisions can be revisited if:
1. Target audience changes (e.g., targeting non-tmux users)
2. A feature becomes trivial due to other work (e.g., local scrollback after OSC parsing)
3. Strong user demand with clear use cases
Until then, we stay focused on being the best tmux UI, not another terminal emulator.
## Related
- [COPY-MODE.md](https://github.com/flplima/tmuxy/blob/main/docs/COPY-MODE.md) — Client-side scrollback rendering (the one scrollback-like feature we implement; history fetched from tmux on demand)
- [RICH-RENDERING.md](https://github.com/flplima/tmuxy/blob/main/docs/RICH-RENDERING.md) — Image protocol support (iTerm2, Kitty, Sixel) and OSC 8 hyperlinks
========================================================================
docs/PERFORMANCE-BASELINE.md
https://github.com/flplima/tmuxy/blob/main/docs/PERFORMANCE-BASELINE.md
========================================================================
# Performance: Measured Baseline and Retention
The numbers measured so far, long-session retention, and snapshot costs. Part of [PERFORMANCE.md](https://github.com/flplima/tmuxy/blob/main/docs/PERFORMANCE.md).
## Measured baseline
First full run of both harnesses. Treat these as the current baseline to
regress against, not as fixed constants — re-run the harnesses after any change
to the parse/aggregate/delta pipeline or the transport.
### Axis A — core + client processing (`cargo bench -p tmuxy-core`)
The bench fills panes with a real screenful (empty grids made content cost
look free).
Devcontainer (aarch64), same machine for both columns. "Before" is the
per-cell deep-copy pipeline; "after" is the `Arc`-shared-content pipeline
(`TmuxPane.content: Arc`, `ptr_eq` skip in the grid diff):
| Bench | Before | After | Change | What it is |
| --- | --- | --- | --- | --- |
| `full_sync` | 206 µs | 136 µs | −34% | first full snapshot + screenful, 2-pane 80×24 |
| `delta_rename` | 160 µs | **23.5 µs** | −85% | single-field change on an already-synced grid |
| `output_burst/200` | 413 µs | 321 µs | −22% | 200-line flood parse+aggregate |
| `output_burst/2000` | 2.52 ms | 2.37 ms | −6% | 2000-line flood parse+aggregate |
**The construction pathology is fixed.** Before, a one-field delta cost ~78%
of a full sync because every update deep-copied each pane's cell grid three
times (content-cache clone, `prev_state` clone, full-grid diff walk). Grids
are now `Arc`-shared between snapshots: an unchanged pane costs a refcount
bump, and the diff skips it by pointer identity. A metadata-only delta is
**6.8× cheaper** and no longer scales with grid size. Locked in by the
`metadata_delta_shares_content_and_omits_grids` test in
`tmuxy-core/src/control_mode/state.rs`.
Remaining honest cost: when content _does_ change, extraction + line diff
still walk the grid (the µs-scale `full_sync`/burst numbers above) — that is
real work the pipeline must do, and byte parsing itself remains cheap
(~40 MiB/s).
### Axis B — transport (input → paint)
Release `tmuxy-server` on loopback, 26 keystrokes per condition spaced 400 ms
apart (clean per-key round trips, no batching), driven headless through the
real `POST /commands` + `GET /events` path. RTT injected with a one-off
latency proxy, since removed. All latencies in ms.
| Condition | Injected 1-way / RTT | p50 | p95 | p99 | max | pending | added vs C0 |
| --- | --- | --- | --- | --- | --- | --- | --- |
| C0 direct | 0 / 0 | 22.3 | 42.3 | 44.4 | 67.4 | 0 | — |
| C1 LAN | 30 / ~60 | 86.7 | 99.0 | 100.0 | 243.2 | 0 | +64 |
| C2 typical remote VM | 75 / ~150 | 173.4 | 194.5 | 194.6 | 215.7 | 0 | +151 |
| C3 far / bad mobile | 150 / ~300 | 325.8 | 355.8 | 356.4 | 359.4 | 0 | +304 |
| C4 150 ms RTT + 5% loss | 75 / ~150 + loss | 589.2 | 710.2 | 978.1 | 1034.5 | 1 | tail blows up |
**Transport is a clean additive term.** The added latency over the C0 floor
tracks the injected RTT almost exactly (+64, +151, +304) — SSE+POST introduces
no head-of-line amplification on a clean link, and there is no local echo, so
every millisecond of RTT lands directly on input→paint. The **~22 ms tracker
floor** (C0) is send→apply: POST round trip + server step + SSE + client
apply. (Measured against a debug server the floor was ~24.5 ms — only ~2 ms
higher — which confirms it is transport-bound, not compute-bound.)
**The tracker starts at send, not at keydown.** `markInput()` fires when the
command leaves the adapter — so any client-side delay _before_ the send is
invisible to the table above. That mattered: the `KeyBatcher` used to hold
every keystroke for its full 16 ms window before sending, an extra ~16 ms of
real, user-felt latency the tracker never saw. Measured keydown→paint
(MutationObserver on the pressed letter's echo, debug server, same machine):
| keydown→paint | always-batch (old) | leading-edge flush (new) |
| --- | --- | --- |
| p50 | 42.5 ms | **25.4 ms** |
| p95 | 62.5 ms | 38.8 ms |
| max | 66.0 ms | 40.9 ms |
The `KeyBatcher` now sends an isolated keystroke immediately (leading edge)
and opens its 16 ms window for what follows; a non-empty trailing flush
re-opens the window, so sustained fast input (paste, key-repeat) still
coalesces to ~one send per frame. Keydown→POST for an isolated key dropped
from ~17 ms to ~1 ms. The `key-echo` interaction in the Axis-C suite measures
this dimension now — on every commit, on both targets, and gated — which is
why the throwaway harness that produced the table above no longer exists.
The tracker and HUD remain the transport (send→apply) instrumentation.
**Loss is where the transport model actually hurts (C4).** At the same 150 ms
base RTT as C2, 5% loss-as-retransmit-stall pushes p99 from 195 ms to 978 ms and
leaves an input still un-applied (`pending` 1) — TCP head-of-line blocking
turning one lost segment into a multi-hundred-ms stall of everything behind it.
This, not steady-state RTT, is the signal a QUIC/WebTransport move would flatten.
**Throughput is decoupled from output volume.** A `seq 1 50000` flood (50k
lines) produced only **~6–8 client-side state updates** total, peaking at ~35
updates/sec, with `pending` never above 2 — the client never backed up. tmuxy
renders the _current visible grid_, not the scrollback, so the server's
aggregator coalesces an arbitrarily large burst into a handful of snapshots.
The real throughput ceiling is the Axis-A cost of extracting + diffing a
changed grid (µs-scale per snapshot, see the bench table), not client render
count. (Scrollback replay is copy-mode's job, a separate client-side path —
see [COPY-MODE.md](https://github.com/flplima/tmuxy/blob/main/docs/COPY-MODE.md).)
### Where tmuxy sits vs terminal emulators (honest framing)
tmuxy is **not** a local GPU terminal emulator; it is a tmux UI that renders
server-parsed cell-grid state to the DOM over a transport. The category
difference matters when comparing:
| Class | Input latency (reference) | Output throughput | Network |
| --- | --- | --- | --- |
| Native GPU (alacritty/kitty/wezterm) | ~5–45 ms (Typometer/Dan Luu) | multi-GB/s `cat` | none (local) |
| Browser/xterm.js (VS Code terminal) | higher; DOM/canvas render cost | DOM/canvas-bound | none (local) |
| mosh | ~0 perceived (local echo) | predicted locally | RTT hidden |
| **tmuxy** | ~25 ms keydown→paint + RTT | volume-decoupled (see above) | the whole point |
The honest read: tmuxy's ~25 ms local keydown→paint floor is competitive with
the upper end of a local GPU terminal, and its snapshot model makes it
structurally immune to output-volume blowups. But it has **no input
prediction / local echo** (an explicit Non-Goal, see
[NON-GOALS.md](https://github.com/flplima/tmuxy/blob/main/docs/NON-GOALS.md), "Local Echo / Input Prediction"), so unlike mosh it pays the full RTT on every
keystroke — which is fine on LAN (C1) and acceptable on a typical remote VM
(C2, ~180 ms p50) but degrades on high-RTT links (C3+).
### Open improvement areas (against measured bottlenecks)
1. **Transport — targeted, not blanket.** The curve shows SSE+POST is a clean
additive-RTT transport with no HoL cost until loss. The measurable QUIC/
WebTransport win is specifically the C4 loss tail (p99 195 → 978 ms), not
steady-state RTT. Input prediction (a Non-Goal) is the only thing that hides
RTT itself; the data says revisit it only for genuinely high-RTT (C3+) remote
use, not for LAN/typical-remote.
2. **Navigation shells out at all.** `Ctrl+hjkl` runs the `nav` helper through
`run-shell` + bash, ~30 ms before the script does anything, while the Rust
monitor already holds the window/pane/group state the script rediscovers.
The client predicts the move, so the user does not wait for it, but moving
navigation into the server is the honest fix.
## Axis D — retention over a long session
Every axis above measures one operation against a budget. None of them can see
a leak: a retained reference keeps each individual interaction just as fast
while the heap, the DOM and the server's memory climb underneath it, and no
suite runs long enough for the climb to matter. Axis D is the question nothing
else asks — **does a long session degrade?**
| --- | --- |
| --- | --- |
| Harness | `packages/tmuxy-ui/scripts/measure-soak.mjs` (`npm run perf:soak`) |
| Load | Tens of MB of output through one pane, then hundreds of split/close cycles |
| Measures | JS heap, DOM nodes, JS event listeners (CDP `Performance.getMetrics`), server RSS |
| Verdict | A **plateau**, not a ceiling: each measure's second-half mean against its first-half mean, counted as growth only while it is still climbing at the end |
| Where | `soak` job in `.github/workflows/nightly-perf.yml` — nightly, table in the run summary |
| Gating | Not yet. `--gate` turns it on once the nightly numbers have a known shape |
Three things about it are deliberate and worth not undoing:
**It asserts a plateau rather than a number.** Absolute heap and RSS depend on
the machine, the Chrome build and what else the runner is doing, so a ceiling in
megabytes is either meaningless or permanently red. Growth between the halves of
one run divides all of that out, and it is also the only form that distinguishes
a leak from warm-up — which a single before/after reading cannot. Warm-up can
still lift the second half over the first (the output phase fills the pane with
rows for its first few windows), so a measure past its tolerance also has to be
climbing between the start and the end of the second half: a leak is, a filled
pane is flat by then.
**Load comes from the CLI, not the keyboard.** Axis C types, because the
keystroke path is its subject. Axis D's subject is what the app retains while
rendering someone else's output, so it drives load through `tmuxy pane send`.
That is also a practical necessity: typing a long command line into a pane drops
characters often enough that the load generator sometimes runs a different
command than the one it was given.
**It needs a tmux socket of its own.** It types into the active pane and splits
panes in whatever session the server it is pointed at is serving. A server
started with no `TMUX_SOCKET` serves the default `tmuxy` socket — on a dev
machine, the session someone is working in.
It is the one perf job that is scheduled, because its product is a trend rather
than a number: a soak run answers a yes/no question about that run — did
everything plateau — and the answer is only interesting as a sequence. One
night proves nothing; thirty nights show a slope. For the same reason it prints
its verdict table into the run's step summary, where it is readable from the
run page: an artifact is only read by someone who already suspects something.
CI runs it with a smaller load than the harness defaults — a shared runner is
slower at both halves, and the plateau comparison needs its windows to be clean
more than it needs them to be large.
## Session snapshots
A snapshot (`tmuxy_core::session_snapshot`) is written on every change to a
session's SHAPE, debounced one second, and once more on shutdown. Measured on an
M-series mac, a session of three windows (a two-pane split, a float, a tab):
| --- | --- |
| --- | --- |
| Snapshot file | ~1.2 KB (JSON, pretty-printed) |
| Autosave cost | two `list-*` queries through control mode plus one `ps` per pane, off the async runtime; nothing when the serialized form is unchanged |
| `tmuxy session save` (subprocess path) | 0.63 s wall, 0.05 s CPU — the wall is tmux round-trips, one per query and per `ps` |
| Writes over a build-up of three windows | 5 files (one per settled change), never one per keystroke: output is not a shape change (`is_structural`) |
Screen contents are not captured unless `--scrollback N` asks; a capture is one
more control-mode round-trip per pane and competes with the stream Axis C is
measured on, which is why it is opt-in.
========================================================================
docs/PERFORMANCE.md
https://github.com/flplima/tmuxy/blob/main/docs/PERFORMANCE.md
========================================================================
# Performance Measurement
How tmuxy tracks speed, and how to run each measurement. There are **two
independent axes**, they cost in different places, and they need different
harnesses. Conflating them is the most common way to measure the wrong thing.
| Axis | What it costs | Harness | Network |
| --- | --- | --- | --- |
| **A. Core + client processing** | parse → aggregate → delta → apply → render CPU, render churn, frames-to-paint | v86/wasm probes + native criterion bench | removed |
| **B. Transport** | wire RTT, head-of-line stalls, reconnect/roaming | `latencyTracker` + the `?perf` HUD | the whole point |
| **C. Whole interaction** | everything one keypress sets off, end to end, per user action | `measure-interactions.mjs` in CI | included |
Axes A and B decompose a single round trip; axis C asks a different question —
**how much does one thing a user does actually cost**, with every layer, every
extra round trip and every subprocess the action drags along still in the
number. An interaction can regress badly (a binding grows a `run-shell`, a
window-level refresh lands on the keystroke path) without A or B moving at all,
which is exactly the gap it exists to close.
The v86/wasm path measures Axis A precisely _because_ it removes the network as
a variable — so it is a clean "how fast is our code with zero transport cost"
baseline. It structurally cannot see Axis B (there is no socket in that path).
A transport change (SSE+POST → QUIC/WebTransport, or local → remote-VM) moves
Axis B only. Measure both, and the difference is the transport's contribution.
## Axis A — core + client processing
### Native benchmark (deterministic, CI-friendly)
`packages/tmuxy-core/benches/core_pipeline.rs` (criterion). Runs the **exact**
`Parser` + `StateAggregator` + `to_state_update` code that the browser runs via
`tmuxy-wasm` — one source of truth, no VT reimplementation — so native numbers
track the wasm hot path. Three groups: `full_sync` (first snapshot),
`delta_rename` (single-field delta), `output_burst` (`seq`-style flood, reported
as bytes/sec).
Run: `cargo bench -p tmuxy-core`. These are absolute, reproducible numbers —
the right place to catch a regression in the parse/aggregate/delta pipeline.
`tests/perf/compare-core-bench.mjs` reads criterion's output and judges it the same
two ways Axis C does, for the same reason:
| Signal | Compared against | On regression |
| --- | --- | --- |
| ratio between two benchmarks of the same run | a budget in `compare-core-bench.mjs` | **fails the job** |
| absolute mean | `tests/perf/core-pipeline-baseline.json`, keyed by platform | warns in the step summary |
The ratio is what makes this a gate at all. Absolute nanoseconds from a shared
runner cannot block a merge, and a platform with no baseline recorded has
nothing to compare against — but `delta_rename / full_sync` is a statement
about the _shape_ of the pipeline that holds on any machine. It is budgeted at
0.4 and measures 0.11–0.17 depending on the machine; before grids were
`Arc`-shared it was ~0.78, because a one-field delta deep-copied every pane's
cell grid and therefore scaled with grid size. If that pathology returns, this
ratio catches it without anyone having recorded a number first.
A benchmark named in a budget but missing from a run also fails, so renaming a
bench cannot silently drop its gate.
**Refreshing `tests/perf/core-pipeline-baseline.json`.** A baseline must be measured
on the platform it describes, so the file holds an entry per platform and CI
never writes it — a baseline change is always a reviewed commit, the same rule
as `tests/perf/interaction-baseline.json`. For your own machine, run `cargo bench -p
tmuxy-core --bench core_pipeline` and then `node tests/perf/compare-core-bench.mjs
--update-baseline`, which merges the run under its platform key. For the CI
platform, take `core-pipeline-report.json` from the `core-pipeline-bench-`
artifact of a nightly `perf measurements` run you trust and paste
it under `platforms["linux-x64"]`. Until someone does, a CI run records its
numbers and warns about nothing; the ratio budget gates regardless.
### v86/wasm story probes (integration, relative)
`packages/tmuxy-ui/scripts/probe-spikes.mjs` drives every `v86`-tagged
Storybook story (real tmux in an x86 emulator, real Rust core in wasm) in one
shared page. It enforces the in-story budgets (glitch/size-jump, paint-frame
immediacy, render-commit, asset-weight, throughput/ordering) and, with
`PROBE_TIMINGS_JSON=` set, also emits a per-story wall-clock report.
Those durations are **relative** regression signals (shared engine,
runner-load-sensitive, emulator byte-pacing artifacts), not production
latencies — useful for trend, not for absolute claims.
See [TESTS-STORYBOOK.md](https://github.com/flplima/tmuxy/blob/main/docs/TESTS-STORYBOOK.md) for the budgets and the CI wiring.
## Axis B — transport latency
Instrumentation the running product previously lacked entirely.
`packages/tmuxy-ui/src/infra/latencyTracker.ts` records the **input → paint**
round trip: when a keystroke/command leaves the client (`markInput`) and when
the resulting state update is applied (`recordUpdate`). It reports a latency
distribution (p50/p95/p99/max), the count of outstanding un-applied inputs
(`pending`), the applied-update rate, and the worst recent inter-update gap
(the stall signal). Inputs are matched to applies FIFO, so a burst of inputs
with few applies — a head-of-line stall — shows up as rising `pending` and
inflating latency, which is exactly what a roaming/QUIC transport would flatten.
It is **dev-gated**: disabled by default, so the two hooks in `HttpAdapter` and
`TauriAdapter` are a boolean check and an early return in production. Enable via
`?perf` in the URL, `localStorage.tmuxyPerf = '1'`, `window.__tmuxyPerf = true`
before connect, or `latencyTracker.setEnabled(true)`. When enabled at load, the
`PerfHud` overlay (`packages/tmuxy-ui/src/components/PerfHud.tsx`) shows the live
numbers; its store updates are coalesced to one animation frame so it can't
distort what it measures.
### Measuring a link
Drive the app over the link in question with the HUD open (or read
`window.__tmuxyLatency.getSnapshot()`) to get the input→paint distribution for
that transport — the question the v86/wasm harness structurally cannot answer.
The C1–C4 rows in the results below came from a one-off latency-injection
proxy that no longer exists; a controlled re-run needs delay injected the same
way, between the browser and a real server, on `POST /commands` and the
`GET /events` stream.
Because the transports run over TCP, real packet loss reaches the app as delay
(head-of-line retransmit), not dropped events; the loss condition in that table
modelled it as a random extra stall rather than truly dropping bytes.
## Axis C — whole-interaction latency (the CI regression gate)
`packages/tmuxy-ui/scripts/measure-interactions.mjs` drives the handful of
things a tmuxy user does constantly and times each one from the real browser
`keydown` to the first DOM change that shows the result — arming on the
keydown, resolving on the first probe change after it. Everything goes through
the real user path: the key reaches the page, the keyboard actor resolves the
tmux binding, the command crosses the transport, tmux acts, state comes back.
Nothing is short-circuited with an adapter call, so a regression anywhere in
that chain lands in the number.
Measured today: `key-echo`, `pane-nav-keyboard`, `pane-zoom-toggle`,
`pane-split`, `tab-switch`.
**Two targets, one suite.** The same interactions are measured against the web
app and against the desktop app, because they reach the same Rust core over
different transports — `POST /commands` + SSE versus Tauri IPC — and a
transport regression on one is invisible in the other. What is measured lives
once in `packages/tmuxy-ui/scripts/lib/perf-harness.mjs`; only the driver
differs:
| Target | Harness | Driver | Report |
| --- | --- | --- | --- |
| web | `measure-interactions.mjs` | Playwright (or an existing Chrome over CDP) | `target: "web"` |
| desktop | `measure-interactions-tauri.mjs` | WebdriverIO → `tauri-driver` → WebKitWebDriver | `target: "tauri"` |
Baselines are keyed by `/`, so desktop milliseconds are never
compared against web ones measured on the same machine. The ratio budgets are
shared: a keystroke round trip is the cheapest complete operation on either
transport, so "how many keystrokes does a split cost" is a fair question to ask
of both.
The desktop harness drives the **release** binary (`target/release/tmuxy`), on
its own tmux socket (`tmuxy-perf`), and adopts an already-running `tauri-driver`
rather than fighting one — so it slots into a job that has already built the app
and started a driver.
### The runner-noise problem, and the ratio that solves it
Absolute milliseconds on a GitHub runner are unusable as a merge gate — the
same commit can vary two-fold between runs. So each interaction is also
reported as a **ratio to the keystroke echo measured in the same run on the
same machine**. A keystroke is the cheapest complete round trip tmuxy has; a
loaded runner inflates it and every other interaction together, and the ratio
divides that out. Only a real change in how much _work_ an interaction does
moves a ratio.
That gives two signals with very different standing, and
`compare-interactions.mjs` treats them differently:
| Signal | Compared against | On regression |
| --- | --- | --- |
| ratio to `key-echo` | a per-interaction budget in `compare-interactions.mjs` | **fails the job** |
| absolute p50 | `tests/perf/interaction-baseline.json`, keyed by platform | warns in the step summary |
Budgets are ceilings with headroom, not targets. Raising one is a deliberate
edit that shows up in review; tightening one belongs in the commit that earns
it. The baseline is keyed by platform because milliseconds from a macOS laptop
say nothing about `ubuntu-latest` — a platform with no baseline yet simply
leaves the absolute column blank and rides on the ratio gate.
### Running it
```
npm run perf:interactions -- --url http://localhost:9000 --samples 12 \
--label local --out tests/perf/interaction-report.json
npm run perf:compare -- --report tests/perf/interaction-report.json
```
Add `--cdp http://localhost:9222` to attach to the Chrome the dev environment
already runs instead of launching one (the devcontainer deliberately does not
install Playwright's browsers; CI is the reverse and launches). Refresh the
committed baseline for a platform with `npm run perf:compare -- --report … --update-baseline`
and commit the result. For the Linux platforms, run the "perf measurements"
workflow (`.github/workflows/nightly-perf.yml`, `interaction-baseline` job)
instead: it measures on a runner, with more samples than the gate job takes,
and opens a PR with the refreshed file. Either way a baseline change lands
through review — CI never writes one straight to `main`.
That job is on demand, not scheduled, and a PR rather than a commit, for one
reason: a baseline that refreshes itself ratchets to whatever the runner did
last, which is the opposite of a baseline. A nightly run of it wrote artifacts
nobody opened, and the file it exists to feed was never once refreshed from
one — because the review used to mean downloading an artifact and copying a
file over by hand. The PR keeps the review and removes the chore; read the
milliseconds before merging, since a runner having a bad night produces a
baseline that hides real regressions for as long as it stands.
The desktop equivalent needs the app built first, and takes the binary rather
than a URL:
```
(cd packages/tmuxy-tauri-app && npx tauri build --no-bundle)
npm run perf:interactions:tauri -- --samples 12 --label local \
--out tests/perf/interaction-report-tauri.json
npm run perf:compare -- --report tests/perf/interaction-report-tauri.json
```
CI runs both nightly, in `nightly-perf.yml` ("perf measurements"), uploading
the report as an artifact; the ratio budgets fail the run there, so a
regression shows as a red nightly the next morning:
| Job | Measures | Builds |
| --- | --- | --- |
| `interaction-latency` | Axis C, web | the frontend and the release server |
| `interaction-latency-desktop` | Axis C, desktop | the release Tauri binary, under Xvfb |
| `core-bench` | Axis A bench | tmuxy-core under the bench (release) profile |
They ran per commit until the per-commit `desktop` job switched to a debug
build: the desktop measurement needs the release binary, and that build was the
longest step on every push's critical path. With no branch protection the
per-commit gates were informational anyway. `workflow_dispatch` with `gates`
checked runs all three on demand.
### Measured
Local server on an isolated socket, macOS arm64, 10 samples per interaction,
tmux 3.7. Same machine and same session shape for both columns. Treat the
ratios as the durable part and the milliseconds as machine-specific.
| Interaction | before | after | × keystroke (before → after) |
| --- | --- | --- | --- |
| `key-echo` | 29.2 ms | 38.3 ms | 1× → 1× |
| `pane-nav-keyboard` | 356 ms | **10.1 ms** | 12.2× → **0.3×** |
| `pane-zoom-toggle` | 235.1 ms | **92.3 ms** | 8.1× → 2.4× |
| `pane-split` | 82.1 ms | 78.1 ms | 2.8× → 2.0× |
| `tab-switch` | 3.5 ms | 4.7 ms | 0.1× → 0.1× |
"Before" is the state this harness was written to explain: keyboard pane
navigation cost twelve keystroke round trips. What the three fixes were, and
how the number moved, is improvement area 4 below.
The sub-keystroke numbers are not magic — `tab-switch` and (now)
`pane-nav-keyboard` paint from the optimistic prediction before tmux answers.
The harness measures what the user sees, and for a predicted op that is the
prediction; the server's answer arrives later and reconciles.
See [PERFORMANCE-BASELINE.md](https://github.com/flplima/tmuxy/blob/main/docs/PERFORMANCE-BASELINE.md) for these sections: Measured baseline; Axis D — retention over a long session; Session snapshots.
## What's still absent (by choice, for now)
- No live production telemetry / metrics endpoint — the adaptive throttle in the
Rust monitor keeps its counters internal; latency tracking is dev-gated.
- No client-side input prediction / local echo — an explicit Non-Goal (see
[NON-GOALS.md](https://github.com/flplima/tmuxy/blob/main/docs/NON-GOALS.md), "Local Echo / Input Prediction"). Axis B exists in part to decide, with data,
whether a high-latency use case ever justifies revisiting that.
- **No committed baseline for the platforms CI actually runs on.**
`tests/perf/interaction-baseline.json` holds `darwin-arm64/*` only, and
`tests/perf/core-pipeline-baseline.json` holds `darwin-arm64` only. Both CI perf
jobs therefore ride on their ratio gates alone and leave the absolute column
blank. That is the designed fallback, not a failure — but it means a uniform
slowdown that inflates every number together, keystroke echo included, is
currently invisible on CI. Seeding a `linux-x64` interaction baseline is
still a human act, but no longer a manual one: run the "perf measurements"
workflow (`.github/workflows/nightly-perf.yml`) and it opens a PR with the
measured file for you to read and merge. The review is the point — CI must
never ratchet a baseline to whatever the runner did last — and it had never
happened because the review used to mean downloading an artifact and copying
a file over by hand. The core bench baseline is seeded from the report
`rust-tests` uploads on every commit (see Axis A above).
- **No Axis B measurement in CI.** The RTT curve and the latency-injection
proxy are a controlled experiment run by hand, not a gate — injected delay is
the independent variable, so there is nothing for a runner to regress.
- **Axis D does not gate yet.** Its plateau tolerances are guesses until a
corpus of nightly runs exists to calibrate them against, and a guessed
threshold on a long job produces a nightly nobody trusts. The numbers are
published every night in the run summary; turning on `--gate` in the workflow
is the follow-up.
## Related
- [TRACING.md](https://github.com/flplima/tmuxy/blob/main/docs/TRACING.md) — the trace file that generalizes `latencyTracker` into a cross-layer timeline
- [DATA-FLOW.md](https://github.com/flplima/tmuxy/blob/main/docs/DATA-FLOW.md) — the transports Axis B measures
- [NON-GOALS.md](https://github.com/flplima/tmuxy/blob/main/docs/NON-GOALS.md) — input prediction and the other latency techniques we deliberately skip
- [TESTS.md](https://github.com/flplima/tmuxy/blob/main/docs/TESTS.md) — the Storybook probes the Axis-A story measurements ride on, and where the Axis-C job sits among the suites
========================================================================
docs/RICH-RENDERING-BROWSER.md
https://github.com/flplima/tmuxy/blob/main/docs/RICH-RENDERING-BROWSER.md
========================================================================
# A live browser in a pane
Companion to [RICH-RENDERING.md](https://github.com/flplima/tmuxy/blob/main/docs/RICH-RENDERING.md): `tmuxy widget browser`, the moving-picture
case of the image pipeline, built in.
`tmuxy widget browser ` (also `tmuxy browser`) is the moving-picture case
above, built in: a headless Chrome drawn into a pane and driven from it. The
shell entry is `packages/tmuxy-cli/bin/tmuxy/tmuxy-widget-browser`, which only
finds a node 22 or newer (the script opens its own WebSocket, so it needs no
package) and runs `browser.js` beside it; the pure mappings live in
`browser-lib.js` and are what `tests/cli/cli-browser.test.js` reads.
The script starts a Chrome of its own (`--chrome`, `TMUXY_CHROME`, or the
first of google-chrome, chromium and Playwright's chromium it finds) or
attaches to one by its DevTools endpoint (`--cdp`: a `ws://` URL, an
`http://host:port`, or a port — what `agent-browser get cdp-url` prints
works). It opens the page in a tab whose viewport is the pane's cells, every
row but the last, at a fixed 8 × 16 pixels a cell (`--cell`), and takes a
DevTools screenshot of it `--fps` times a second (the DevTools screencast was
tried first and stalls in a headless window after its first frames; a
screenshot never does). A frame that differs from the last is one Kitty
transfer in `f=100` form — the JPEG (or `--png`) bytes as Chrome encoded them,
in 4096-byte chunks, wrapped in tmux's passthrough — sized to the cells with
`c=` and `r=` and anchored at the home cell, so it replaces the previous frame
in place through the same-anchor rule above, one write at a time. A frame
asked for while a navigation swaps the page's renderer fails and the next
tick asks again. The last row is a status line in plain text: the
page's title and address, so `tmuxy pane capture` still says what the pane
shows. On a resize the tab follows the pane and the next frame is the new
size.
The pane's input is the page's: the script puts the terminal on the alternate
screen with the cursor hidden and the mouse tracked (`?1003h`, `?1006h`), and
reads its stdin raw. Printable text is inserted, named keys and Ctrl chords
are key presses, and the SGR mouse reports the UI forwards while the program
tracks the mouse become clicks, drags and wheels on the pixel at the middle
of their cell — which is why the viewport is cells times a known cell size: a
cell is a block of pixels the page can be addressed by. A script drives it
the same way a person does: `tmuxy pane send -t ` types into the page,
and an SGR report sent as literal keys is a click on that cell. Ctrl+Q ends
the session and leaves the alternate screen, which takes the pictures with
it; Ctrl+C reaches the page.
Limits worth knowing: a frame is a screenshot, so a page that changes between
two ticks shows only what the second one caught, and `--fps` is a ceiling the
pane's output path may not reach; the picture is a cell grid's
worth of pixels (640 × 368 for an 80 × 24 pane), so small text is small;
audio, file dialogs, popups and new windows stay in the headless browser; and
the keyboard layer forwards what the pane delivers, so a key the UI or tmux
keeps for itself (a prefix chord) never reaches the page. The E2E
`tests/17-widget-browser.test.js` drives the whole path on CI's runner, whose
image ships a Chrome.
========================================================================
docs/RICH-RENDERING-CELL-GRID.md
https://github.com/flplima/tmuxy/blob/main/docs/RICH-RENDERING-CELL-GRID.md
========================================================================
# The Cell Grid Invariant
Why every glyph must land on tmux's cell grid, and how widths are kept in agreement. Part of [RICH-RENDERING.md](https://github.com/flplima/tmuxy/blob/main/docs/RICH-RENDERING.md).
## The cell grid invariant
A terminal is a grid of cells, but a browser lays out text by glyph advance. Everything tmuxy paints into a pane must therefore be addressed in **cell coordinates**, never allowed to inherit a position from the surrounding text flow. Three rules follow, and each has already been violated once:
**0. One measurement, one cell width.** The font's natural advance is a fraction of a pixel (FiraCode at 15px measures 9.23px), and boxes laid out at multiples of a fractional advance land on fractional pixel edges that the compositor snaps — so a wide pane's text ends a pixel away from where its box does, and adjacent pane outlines stop coinciding. The size actor therefore measures the advance once (inside the pane container, with the real `.terminal-content` styles), snaps it to a whole device pixel, and publishes two custom properties on the app root: `--cell-w`, the snapped cell width every cell-addressed box is a multiple of, and `--cell-gap`, the `letter-spacing` that pads the natural advance up to `--cell-w` so text really advances one cell per character. That same snapped width is the `charWidth` the pane rectangles and the cols→tmux calculation use, so the grid, the text, the cursor and the images all derive from a single number. `ch` is deliberately **not** the grid unit: it is the advance of "0" _without_ letter-spacing, so once the advance is snapped `1ch` is no longer a cell. `cellsToCss` in `terminalShared.ts` is the only place that spells the unit out; it falls back to `1ch` outside the app (component stories) — see `utils/cellMetrics.ts`, `stories/cellGrid.tsx`.
**1. Position by cell, not by flow.** Style-group spans are pinned to an exact cell count (`width: calc( * var(--cell-w))`), so a glyph whose advance differs from a cell paints within or over its own box instead of displacing the rest of the line. The cursor obeys the same rule: it is an absolutely positioned overlay at the cursor's cell in grid units (`--cell-w` and `--line-height-terminal`), in both renderers. It was previously spliced inline into the line's text, which made its position depend on the advance of every glyph before it; a webfont that had not finished loading, an RTL script, or a dingbat then pushed it off the grid.
**1b. Row edges reach the pane border.** Each pane reaches half a separator column past its cells (`--pane-h-padding-left/right` on `.pane-content`), and a row painted edge to edge — a status line, an editor gutter — used to stop half a cell short of the border on both sides. The rows cannot paint that padding themselves: the touch-scroll container they live in clips at its own box. `RowEdges` (`components/RowEdges.tsx`) sits beside that container inside the padded box and paints one strip per row in the padding columns — the first cell's background on the left, the last cell's on the right, bottom-anchored like the terminal. Colours come from `rowEdgeBackground` in `terminalRendering.ts`; inverse video counts the way the cell renders.
The right strip is painted only for a row that reaches the **last column**, and nothing is painted past a row's cells at all. A row's last cell is not its right edge: tmux reports a row as long as its content, so the last cell of a line of block art is a pixel of the picture, and taking it for an edge ran that colour to the border — an ASCII-art logo drew a bar across the pane, which no other terminal does (the cells beyond the content hold the default background). A status line does reach the last column, which is the case the padding was painted for.
**1c. Paint a cell's glyph into a cell.** Block Elements (`█ ▀ ▌ ▘ ░`) are drawn as CSS gradients rather than font outlines, because a font inks less than the cell box and stacked block art comes out striped (`components/blockGlyphs.ts`). The box they are painted into is a style-group span, which covers however many consecutive cells share a style — so the tile has to be sized in cells and repeated, not in percent. Sized in percent, a run of `▌` was one left half across the whole run and a quadrant covered half of it: every logo and bar built from half blocks came out stretched at exactly the width of the run that drew it.
**2. Index by cell, not by UTF-16 offset.** One cell may hold several code units — a variation selector (`U+FE0F`), a combining mark, a ZWJ or skin-tone sequence. Any code that joins a line's cells into a string and then indexes that string with a column number is wrong: the two diverge at the first multi-unit cell and stay diverged for the rest of the line. This is why the cursor character comes from the cell array and why auto-detected URL ranges are translated through a per-cell offset map before being compared against cell indices.
`isWideChar` (in `terminalShared.ts`, shared by both renderers) decides which cells get their own box. It inspects the whole cell string, not just the first code point, because emoji presentation is requested by a trailing `U+FE0F` that the base code point knows nothing about. It deliberately over-detects — a one-cell glyph in its own one-cell box renders identically, so the only cost of a false positive is an extra span. Regional indicators are in its ranges because a flag cell starts with one (see below).
The opposite mismatch also exists: a glyph that is **one** column to tmux but wider than one cell in the font. `⎿` (U+23BF, the elbow Claude Code draws before every tool result) is ~1.6 cells in FiraCode Nerd Font, and Nerd Font icons in the private-use area can be nearly 2. Grouped into a run, such a glyph laid the rest of the run out ~0.6 cell too long — the box stayed pinned, but the text inside painted past it and over the run that followed. `utils/glyphFit.ts` measures a symbol's advance against the font's cell once (a hidden probe with the real `.terminal-content` styles; the ratio is size-independent and cached per cell string, cleared when the webfont finishes loading), and a glyph that measures more than 1.15 cells gets its own 1-cell box with the glyph `scale()`d into it from the left edge (`.terminal-fit-glyph`). Only code points from U+2000 up are measured, so ordinary text costs one compare per cell. Story: `TerminalLine › FatNarrowGlyph`.
### Cell widths must agree with tmux
tmux reports cursor positions and lays out everything on **its** grid; tmuxy re-emulates the raw output with the vendored `vt100` crate. Any character whose width the two disagree on shifts every later cell on that line — the line looks fine on its own but the cursor lands two cells off, and the next absolute cursor move writes over the wrong columns. Per-code-point East Asian Width (what `unicode-width` gives) is not enough: tmux 3.4+ combines sequences into one cell in `screen_write_combine`, and the vendored `vt100` (`packages/vt100/src/screen.rs`) mirrors those rules exactly:
| Input | tmux / tmuxy cell | Naive per-code-point width |
| --- | --- | --- |
| ZWJ sequence `👩💻` | one 2-column cell (the character after a ZWJ joins the previous cell) | 4 columns |
| emoji + skin tone `👍🏽` | one 2-column cell (only for tmux's list of 66 base emoji; the modifier alone is its own 2-column character) | 4 columns |
| two regional indicators `🇺🇸` | one 2-column cell (a third indicator starts a new cell) | 2 × 1 column |
| text symbol + VS16 `❤️` | one 2-column cell (tmux default: `variation-selector-always-wide` on) | 2 columns |
| combining mark `é` | joins the previous cell, through a wide character's continuation half | 1 column |
| `U+3164` Hangul filler | dropped | 1 column |
| ZWJ / VS16 at column 0 | dropped | — |
| sequence longer than the cell buffer (22 bytes; tmux 21) | the overflowing character starts a new cell | — |
These are the rules of tmux 3.4 and later. tmux 3.2 sizes ZWJ sequences, skin-tone sequences and VS16 symbols differently, and tmuxy does not follow it there — a known limitation on the oldest supported tmux, described in [TMUX.md](https://github.com/flplima/tmuxy/blob/main/docs/TMUX-VERSION-BUGS.md#tmux-version).
The E2E test in `tests/3-rendering-protocols.test.js` ("Wide glyphs stay on the tmux cell grid") is the contract: it prints each case, parks the shell in `read`, and compares the _rendered_ cursor and marker positions against tmux's own `#{cursor_x}`. Add a row to that test whenever a width rule changes.
========================================================================
docs/RICH-RENDERING.md
https://github.com/flplima/tmuxy/blob/main/docs/RICH-RENDERING.md
========================================================================
# Rich Terminal Rendering
Tmuxy intercepts terminal escape sequences that would otherwise be passed through to (or dropped by) tmux and renders them as real DOM elements. The browser shows actual `` for hyperlinks, `![]()
` for inline images, and clipboard requests round-trip through `navigator.clipboard`.
This document covers what's supported, how the pipeline works, and how to test each protocol.
## Supported protocols
| Protocol | DCS / OSC / APC | Backend | Frontend | Notes |
| ------------------------------------ | ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **OSC 8 — Hyperlinks** | `ESC ] 8 ; … ; ST` | `control_mode/osc.rs` (style on cell) | `TerminalLine.tsx` → `` | Parsed from the control-mode stream; no `terminal-features` setting needed. Clicks open through `utils/openUrl.ts`, not the anchor's own navigation: `window.open` on the web, the `open_url` Tauri command (system browser, http(s)/mailto only) on the desktop, whose webview denies new windows. Auto-detected URLs (`utils/urlDetect.ts`) use the same path and are live only while the platform link modifier is held |
| **OSC 1337 — iTerm2 Inline Images** | `ESC ] 1337 ; File=… : BEL` | `control_mode/images.rs::try_parse_iterm2` | `Terminal.tsx` → `
` | Base64 of any browser-renderable format |
| **APC \_G — Kitty Graphics** | `ESC _ G ; ESC \` | `control_mode/images.rs::try_parse_kitty` | same | Supports chunked transfers whose final packet sets `m=0` or omits `m`, formats `f=24`/`f=32`/`f=100` (the bytes of an `f=100` image are kept as they came and typed by their signature, so a GIF sent that way animates — what the demo's nyan cat does, `packages/tmuxy-demo/public/guest/icat`), ZLIB compression (`o=z`), and Unicode placeholders (`U=1`) used by Kitty's tmux passthrough mode |
| **DCS Pq — Sixel** | `ESC P q … ESC \` | `control_mode/images.rs::try_parse_sixel` | same | Decoded by `icy_sixel`, re-encoded as PNG before serving. Only a DCS whose bytes before `q` are digits and `;` is Sixel: other DCS strings holding a `q`, such as nvim's XTGETTCAP query (`ESC P + q`), are not pictures |
| **OSC 52 — Clipboard** | `ESC ] 52 ; c ; ST` | `control_mode/osc.rs` parser → `StateEmitter::write_clipboard` → SSE `clipboard` event (web) / `tmux-clipboard` (Tauri) | the transport's `Clipboard` event → `TMUX_CLIPBOARD` event → `navigator.clipboard.writeText` in appMachine | Outbound only — pasting back is not implemented. Covered end to end by `tests/3-rendering-protocols.test.js` Scenario 14, which reads the clipboard write. |
| **DEC Line Sizes — DECDHL / DECDWL** | `ESC # 3`, `ESC # 4`, `ESC # 5`, `ESC # 6` | `packages/vt100` (`LineSize` per row) + `extract_cells_with_urls` (`CellStyle.size`) | `TerminalLine.tsx` & `terminalRendering.ts` → `.terminal-scaled` (`dwl`, `dhl-top`, `dhl-bottom`) | `ESC # 3`/`# 4` render top/bottom halves of 2×2 double-height lines; `ESC # 6` renders 2×1 double-width; `ESC # 5` resets to single width. Preserved across `capture-pane` refreshes while the row's text matches |
| **OSC 66 — Kitty Text Sizing** | `ESC ] 66 ; s=:w= ; ST` | `control_mode/osc.rs` (`emit_osc66`, `sized_cells`) | same | `s=2` emits `dhl-top` on row `r` and mirrors `dhl-bottom` onto row `r+1`; `w=2` (`s=1:w=2`) emits `dwl` |
**Where the cell -> URL mapping comes from.** The link is recorded against the
cells vt100 reports as it writes them: `OscParser::process` strips the escapes
and hands back the byte ranges a link was open over, and `PaneState` feeds those
ranges to vt100 one character at a time, marking whatever cell the cursor
actually landed on. The marks then travel with their line, driven by the vt100
screen's own scroll counter. (The parser used to derive coordinates from a
private cursor it advanced on newlines and printable ASCII only — blind to every
CSI cursor move, so the URL drifted onto unrelated text.) Because tmux strips
OSC 8 from `capture-pane` history, the map is cleared on a capture refresh and
repopulates from live output — a link is not restored by reloading the page.
**A mark is only good while its text is.** Each mark also records the character
that was under it, and the map is read back a row at a time
(`OscParser::row_urls`), never by coordinate alone: a run of adjacent cells
sharing one URL was written as one label, so it is validated as one — if any
character under it has changed, the whole run goes. vt100 carries no per-cell
hyperlink attribute, so a cell being overwritten is invisible to a coordinate
map; without the check, an application that repaints in place (Claude Code,
vim, htop) scrolled nothing and cleared nothing, and every frame's text
inherited the last frame's link — a long underline over unrelated text. The
run, not the cell, is the unit, or a redraw that happens to refill a cell with
the same character leaves a stray underline under it.
**Auto-detected URLs stop where a URL must.** `utils/urlDetect.ts` matches the
characters RFC 3986 allows in a URI, plus Unicode letters, digits and marks for
internationalised addresses. It is an allowlist on purpose: terminal UIs butt
their text against separators with no space in between, and a class defined by
what it excluded swallowed all of them — `…/bar│Files:12│Status:ok` came back
as one link covering most of the line. Detection runs over a _logical_ line:
rows that only soft-wrapped (the same full-width rule the copy path uses,
`isWrappedRow`) are joined first, so a URL longer than the pane is one link on
every row it covers, each opening the whole address. OSC 8 links already arrive
whole on every row from the server. Either kind is one `` per row, so
hovering any row marks every anchor with the same target (`utils/linkHover.ts`).
OSC 8 has been supported for a long time. The image protocols landed together with the OSC 52 parser — all parsing lives in `tmuxy-core/src/control_mode/images.rs` and `tmuxy-core/src/control_mode/osc.rs` — but only the SSE `clipboard` event + `TMUX_CLIPBOARD` plumbing finished the round-trip into `navigator.clipboard.writeText`. On the frontend, `Terminal.tsx` renders image placements and `TerminalLine.tsx` renders hyperlink cells.
## How tmux preserves the sequences
Tmuxy speaks **tmux control mode** (`-CC`), and tmux forwards every escape sequence the running application emits inside `%output` events on stdout. That includes passthrough payloads. Earlier versions of this document claimed image escapes were stripped — that was wrong. The Rust parser sits on top of the control-mode byte stream and never needs to touch the pty directly.
Two tmux options matter:
- `set -g allow-passthrough on` — required so tmux doesn't refuse to relay the sequences. The monitor sets this automatically on every session at connect time (see `sync_initial_state` in `tmuxy-core/src/control_mode/monitor.rs`).
- `set -g default-terminal "tmux-256color"` (or another terminfo entry that does **not** include image capabilities) — keeps tmux from trying to act on the sequences itself.
The parser is permissive about line wrapping and stray printable bytes between chunked Kitty packets, since tmux re-flows output around its own column wrapping.
An application inside tmux wraps these escapes in **DCS passthrough** (`ESC P tmux; ESC \`), which is how they reach a terminal sitting behind tmux at all. tmuxy is not behind tmux — it reads the pane stream over control mode — so the wrapper arrives here intact and the parser undoes it before anything else (`unwrap_passthrough`). Sending the escapes bare works too, but wrapping is better even here: tmux reads a lone APC string as a **pane title** (`input_exit_apc`), so an unwrapped Kitty frame turns the pane's title into base64 while it draws.
A program can draw a moving picture this way: move the cursor to the picture's corner and emit one `OSC 1337` image per frame, with `preserveAspectRatio=0` when the picture should fill its cells exactly (so a clicked cell maps straight to one of its pixels). The same-anchor replacement below gives flicker-free frames, and the per-pane store cap (`MAX_PANE_IMAGE_BYTES` in `tmuxy-core/src/transport/bindings.rs`) is what keeps a pane that publishes a picture several times a second from growing without limit.
A picture is retired in one of four ways. A screen clear takes it away: `ESC [ 2 J`, or `ESC [ J` straight after the cursor goes home (how `clear` and curses applications empty the screen), and a full reset (`ESC c`). A screen switch sets it aside: entering the alternate screen (`?1049` / `?1047` / `?47`) hides the main screen's pictures, anything drawn there is dropped on leaving, and the main screen's come back — a placement is anchored to cells rather than drawn into them, so vt100 clearing its grid never touched it, and an image outlived the command or the editor that showed it. Kitty's `a=d` deletes placements (the default `d=a` and `d=A` selectors, which mean "all of them"; the narrower ones address images by a kitty id tmuxy does not keep, so they are left alone). And a new placement anchored at the same cell **replaces** the one already there, so a preview that homes the cursor and repaints keeps exactly one frame on screen without stacking — and without the gap that deleting first would leave while the next frame crosses control mode. The frontend keys each `![]()
` by that anchor rather than by the placement id, so a repaint reuses the element: the browser holds the previous picture until the new bytes have decoded, where building a fresh element painted nothing until they arrived and the preview blinked once a second.
## A live browser in a pane
`tmuxy widget browser ` (also `tmuxy browser`) is the moving-picture case above, built in: a
headless Chrome drawn into a pane as Kitty frames and driven from it. See
[RICH-RENDERING-BROWSER.md](https://github.com/flplima/tmuxy/blob/main/docs/RICH-RENDERING-BROWSER.md).
## Where the bytes come from
The decoder hands the host two things: a placement (where on the grid, how big) and the picture's bytes. Placements travel with the pane state; the bytes do not, so each host serves them itself and the frontend asks for whichever URL that host understands.
| Host | Serves the bytes as | Registered in |
| ---------------- | ----------------------------------------- | ---------------------------- |
| Web server | `/api/images//` | `tmuxy-server/src/state.rs` |
| Desktop app | `tmuxyimg://localhost//` | `tmuxy-tauri-app/src/gui.rs` |
| Storybook, tests | whatever `window.__tmuxyImageSrc` returns | the story or test |
A host that keeps no bytes renders no pictures, whatever the decoder did with them: `StateEmitter::store_images` defaults to discarding, and the desktop app took that default for a long time while its frontend asked for the web server's route, so every image protocol silently drew nothing there. Neither URL is cached — placement ids restart with the process, so a cached response is the previous run's picture.
## End-to-end pipeline
```
running app
│ writes escape bytes
▼
tmux (-CC, passthrough)
│ wraps as %output %
▼
tmuxy-core / control_mode/parser.rs
│ feeds raw payload to
▼
control_mode/images.rs::ImageParser::process
│ ─► extracts payload, decodes, stores image bytes in pane state
│ ─► returns ImageProcessResult { stripped: bytes_to_render,
│ new_placements: [(id, StoredImage)…] }
▼
ServerPane.images (snake_case, serialized over SSE)
│
▼
appMachine.helpers::transformServerState (snake → camel via camelize())
│
▼
TmuxPane.images : ImagePlacement[]
│
▼
TerminalPane.tsx → Terminal.tsx
│ renders 
▼
GET /api/images/{pane_id}/{image_id}
│ served by tmuxy-server (see api_routes in state.rs)
▼
stored bytes (already in the format the browser expects: PNG/JPEG/WebP)
```
`ImageParser` keeps the raw bytes per `(pane_id, image_id)` so that re-renders, viewport changes, and reconnects all read the same content. Sixel input is converted to PNG once at parse time so the server doesn't repeat the decode on every fetch.
## Placement geometry
Each `ImagePlacement` carries:
- `id` — monotonic per-pane image counter, matches the URL path
- `row`, `col` — top-left cell coordinates (`row` is signed `i32`: `0..height-1` on the live viewport, negative when scrolled into scrollback history)
- `width_cells`, `height_cells` — bounding box in terminal cells
- `protocol` — one of `iterm2`, `kitty`, `sixel`
The frontend positions the `![]()
` absolutely inside `.terminal-images` in cell units — `--cell-w` horizontally, `--line-height-terminal` vertically — so the image stays aligned to the same cell grid as the surrounding text. This is the same grid the rows and the cursor use; see [The cell grid invariant](https://github.com/flplima/tmuxy/blob/main/docs/RICH-RENDERING-CELL-GRID.md). When `width=auto` / `height=auto` is requested, the parser converts pixels to cells using the pane's current cell size estimate.
**Content-tracked scrolling and scroll/copy mode.** `PaneState::sync_url_scroll` reads `vt100::Screen::scroll_delta()` as terminal output is processed and calls `ImageParser::shift_rows_up` / `shift_rows_down` to shift each placement's `row` in lockstep with the surrounding text. Placements that scroll above the live viewport (`row < 0`) are retained until their bottom edge (`row + height_cells`) passes the scrollback limit (`IMAGE_HISTORY_ROWS`), and `ScrollbackTerminal.tsx` renders `.terminal-images` at `top: calc((historySize + row) * var(--line-height-terminal))` so images remain visible when scrolling back in scroll or copy mode. Primary-screen placements also survive pane zoom and resize (`ImageParser::reset_for_resize` only clears placements on the alternate screen, where full-screen apps redraw on `SIGWINCH`).
See [RICH-RENDERING-CELL-GRID.md](https://github.com/flplima/tmuxy/blob/main/docs/RICH-RENDERING-CELL-GRID.md) for this section: The cell grid invariant.
## Testing
### Manual
```bash
# OSC 8 hyperlink
printf '\e]8;;https://example.com\e\\Click me\e]8;;\e\\\n'
ls --hyperlink=auto
# iTerm2 inline image (requires `imgcat` from iTerm2 utilities, or any
# tool that emits OSC 1337 File=…)
imgcat path/to/image.png
# Kitty graphics protocol
kitten icat path/to/image.png # the official client
# or any application using zellij / ranger's kitty image preview
# Sixel
img2sixel path/to/image.png
chafa --format sixel path/to/image.png
```
Inside a real tmuxy session, each of these renders the image inline in the pane.
### Automated
- **Rust unit tests** (`tmuxy-core/src/control_mode/images.rs`) cover the parsers and placement geometry — including chunked Kitty transfers, RGBA-to-PNG conversion, and Sixel decode.
- **E2E** runs a real tmux session, emits each protocol with a shell command, and asserts the `![]()
` shows up in the page (see `tests/`).
## Other rich-rendering opportunities
These are documented because they reuse the same passthrough pipeline; whether to implement them is a product decision, not a technical one.
- **OSC 9 / OSC 777 — Desktop notifications**
- **OSC 9;4 — ConEmu progress bars**
- **OSC 133 — FinalTerm semantic zones** (prompt / command / output markers)
- **OSC 7 — Working-directory hints** (already partially wired for tab titles)
## Related
- [TMUX.md](https://github.com/flplima/tmuxy/blob/main/docs/TMUX.md) — control-mode protocol and version quirks
- [DATA-FLOW.md](https://github.com/flplima/tmuxy/blob/main/docs/DATA-FLOW.md) — SSE/HTTP delivery of placements and image bytes
- [NON-GOALS.md](https://github.com/flplima/tmuxy/blob/main/docs/NON-GOALS.md) — what we still deliberately don't render
========================================================================
docs/SECURITY-CONTENT.md
https://github.com/flplima/tmuxy/blob/main/docs/SECURITY-CONTENT.md
========================================================================
# Security: Untrusted Content and Recorded State
Previewed files, session snapshots and hostile pane output. Part of [SECURITY.md](https://github.com/flplima/tmuxy/blob/main/docs/SECURITY.md).
## A Preview Reads One File
`tmuxy open ` shows a local markdown document or HTML page in a pane (the `preview` widget, see [ARCHITECTURE-WIDGETS.md](https://github.com/flplima/tmuxy/blob/main/docs/ARCHITECTURE-WIDGETS.md)). A file is the least trusted thing tmuxy draws: a page downloaded from anywhere, a README from a cloned repository, a report a tool wrote from data it did not check. The design assumes the document is hostile and keeps two questions apart — what it may **read**, and what it may **do**.
**What may be read is decided by tmux, not by the client or the document.** The UI reads the file through a client command, `read_preview`, naming only the pane; the path is the pane's `@tmuxy-pane-preview`, honoured only while `@tmuxy-pane-widget` is `preview`. `tmuxy-widget-preview` writes both out of band through `run-shell` and the widget clears its tag on exit, so neither pane output nor a client can point a preview at another file, and a pane that has gone back to being a shell reads nothing. Both transports call the same check (`tmuxy_core::preview`), on every read. This replaced a file route that read any path the server process could; nothing serves an arbitrary path any more.
**A document's own references are held to its folder.** A resource the document links — a stylesheet, a script, an image, a font — is read only when its path, relative to the document, stays inside the document's folder once symlinks are resolved, passes through no hidden name (`.ssh`, `.env`, `.git`, `..`), and is a type a page loads (`RESOURCE_EXTENSIONS`). A hostile page in `~/Downloads` cannot name `../.aws/credentials` as a stylesheet, nor `notes.txt` beside it. Every read is a regular file under a size cap (8 MiB a document, 16 MiB a resource), done off the async workers. The client applies the same rules before it asks (`domain/preview.ts`) so that what the server would refuse costs no round trip, but the server's check is the one that holds.
**What a document may do depends on what it is.**
- **Markdown is rendered by the app**, in the app's own origin, so nothing in it may execute: raw HTML in the document is not rendered at all, a link opens in the system browser only when it is `http(s)` or `mailto` and never navigates the app, and a remote image is not loaded unless the preview was opened with `--net`, since loading it tells its host the document was read. A ```mermaid fence becomes an SVG on this side of the boundary, where anything that executes can drive the API, so it is sanitized twice (SEC-20): by mermaid (`securityLevel: 'strict'`), and again at the sink (`widgets/preview/sanitizeSvg.ts`) — parsed in an inert document, every `on*` attribute and every element that fetches or runs removed (SVG's animation elements too, which could write a `javascript:` link after the walk), the HTML in a label held to text formatting, `
`packages/tmuxy-ui/src/utils/appearanceManager.ts` | | Native menus: the menu bar (macOS) and native popups for every context menu (macOS + Linux) — not the session switcher, whose filter box a native popup cannot hold — built from the frontend's menu spec; the in-app menus are the fallback, and what `@tmuxy-native-menus off` keeps | A page has no menu bar and no native popups; the web app draws the same specs with its own floating menu. The desktop sets the bar from the page (`NativeMenuBar`), debounced on the spec's shape and on window focus so the focused window owns it; a popup that is refused falls back to the in-app menu in the same gesture. An item's icon is the in-app menu's own strokes, rasterised in the page in the system's menu text colour — the platform takes a plain bitmap, not a template it recolours — so the bar is rebuilt when the system appearance changes. | `packages/tmuxy-ui/src/domain/menus/` (the specs, and `icons.ts`)
`packages/tmuxy-ui/src/infra/menus/nativeMenu.ts`, `nativeIcons.ts`
`packages/tmuxy-ui/src/components/menus/NativeMenuBar.tsx`, `useNativeMenu.ts`, `menuHost.ts` | | Other servers in the session menu, and `tmuxy session connect` to add one | A web client is already talking to the one server that serves it. | `components/SessionMenu.tsx` | | Links opened by the system (browser, viewer), images served over an app scheme | A page may only open a tab; the shell asks the OS. The web app serves the same images over an HTTP route instead. | `utils/openUrl.ts`, `components/Terminal.tsx` (`tmuxyimg:`) | | Restart the app | A page can only reload itself. | `utils/restartApp.ts` | The switch for all of it is `isTauri()` (`utils/platform.ts`), set from the Tauri IPC bridge being present; `main.tsx` also marks the document with `data-desktop` for styling. Nothing else probes the platform. ## Trackpad gestures The pane grid follows the fingers (`GestureStage` writes each step as an inline `transform`, nothing re-renders), the tab being pulled in is drawn beside it from its real panes, and the release decides: past the mark the tab switches at once and the grid runs home; short of it the grid slides back. The decision and the drawing are `machines/app/actions/gestures.ts` and `utils/gestures.ts`; the only thing the shell supplies is _where the fingers are and when they lift_. **The contract** (one call per native scroll event the shell claims): ``` window.tmuxyGesture('slide', phase, dx, dy, t) ``` - `phase`: `began` · `changed` · `ended` · `cancelled` · `momentum` · `momentum-ended`. The fingers are on the pad from `began` to `ended` or `cancelled`, for as long as that takes — a pause is simply no event. The momentum phases are the tail macOS sends after a lift; Linux has none. - `dx`, `dy`: points, positive when the fingers move right or down (the content follows the fingers). - `t`: the event's own timestamp in seconds, so the speed a release carries on at is measured on the device's clock. **What the shell claims.** A scroll stream whose first steps are horizontal (the same axis rule the actor used to apply, `|dx| > 1.5·|dy|`) from a touchpad (`hasPreciseScrollingDeltas` on macOS; a touchpad device on Linux), while `@tmuxy-gesture-swipe-tabs` is on — the page is told the flag through a command whenever it changes. The claimed events, and the momentum tail that follows them, never reach the webview: the page does not scroll on them, and nothing has to `preventDefault`. Vertical scrolls and mouse wheels are left alone. A slide over a pane whose program tracks the mouse is still claimed: Ghostty gates its tab switch on that, but Claude Code and nvim track the mouse, so the gate left most tabs with no slide at all, and what such a program gives up is only a horizontal two-finger scroll. **macOS** installs an `NSEvent` local monitor for `.scrollWheel` from Rust (`objc2-app-kit`): it runs before `-[NSApplication sendEvent:]`, so before the WKWebView, and returning `nil` is what swallows an event. It does not consult the Trackpad preference "Swipe between pages": that switch is about page history, the two-finger scroll arrives whatever it says, and `@tmuxy-gesture-swipe-tabs` is the one setting for the tab slide. **Linux** connects `scroll-event` on the WebKitGTK widget: two-finger motion is `GDK_SCROLL_SMOOTH` ending in a stop event at lift-off, before any kinetic scrolling, and `Propagation::Stop` is what swallows it. A pinch needs none of this: WebKit reports it to the page itself as `gesturestart` / `gesturechange` / `gestureend` with the cumulative scale, and those have phases. The actor keeps a ctrl+wheel pinch too, which is how libinput's pinch arrives on WebKitGTK. The web build runs none of the gesture code: `AppProvider` leaves the machine's no-op gesture actor in place off the desktop, so the browser keeps its own wheel, pinch-zoom and history swipe. The Storybook stories under _Mocked App / Trackpad Gestures_ opt the actor in (`config.desktopGestures`) and drive it with the shell's own calls, which is how the JS half is tested on the web build; the native half is covered by the desktop suite. ======================================================================== docs/INVARIANTS-UI-LIFECYCLE.md https://github.com/flplima/tmuxy/blob/main/docs/INVARIANTS-UI-LIFECYCLE.md ======================================================================== # Invariants — UI-mode records, session lifecycle The `U` and `L` rows of the ledger in [INVARIANTS.md](https://github.com/flplima/tmuxy/blob/main/docs/INVARIANTS.md), which holds the column legend, the seam prefixes and the other rows. Same rules: cite a row in a commit on its seam, and a row's enforcer is a test title that exists. ## U — UI-mode records | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | U1 | At most one copy-mode record per pane; exit closes only if seen in mode, and cooldown ignores stale `in_mode`. | ordering | `copyMode.ts`, `docs/COPY-MODE.md` | `reconcilePaneMode.test.ts › a snapshot…`, `› is ignored…`; `copyMode.test.ts › the end of a blink…` (examples) | | U2 | Resize preview holds for two matching quiet updates or a 2 s fallback that never clobbers newer resizes. | liveness | `appMachine.ts`, `layout.ts` | `layout.test.ts › the 2s RESIZE_COMPLETED…`, `› …does not clobber…` (two-quiet-updates hold is —) | | U3 | Only the latest scrollback fetch per pane applies; different panes fetch concurrently without cancelling. | ordering | `tmuxActor.ts` | `tmuxActor.test.ts › interrupts the previous fetch…`, `› different paneId fetches…` | | U4 | Group switch holds swap over stale snapshots, drops if a pane dies, and never selects parked/stash panes. | safety | `appMachine.ts`, `windowOps.ts` | `tests/7-regression-bugs.test.js › Clicking three pane-group tabs…` [nightly] (E2E only) | | U5 | Every menu renders a `domain/menus/` spec and dispatches via `executeMenuAction` or machine events. | safety | `domain/menus/`, `SpecMenu.tsx`, `nativeMenu.ts` | `specs.test.ts`; `menuActions.test.ts › sends the op of every op-backed item…` | ## L — session lifecycle | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | L1 | Snapshot restore runs after attach/sync and holds autosave from before spawn so placeholders aren't saved and live sessions aren't overwritten. | ordering | `transport.rs`, `session_snapshot.rs` | `session_snapshot.rs › a_restore_in_flight…`; `transport.rs › a_running_session_is_not_restored_over` | | L2 | Restore plans avoid `new-window`, split panes in index order, apply layouts after splits, and restore focus last. | ordering | `session_snapshot.rs` | `session_snapshot.rs › a_restore_never_uses…`, `› focus_is_restored_last`, `› a_reordered_group…`; `tests/15-session-snapshot.test.js › the shape survives…` [nightly] | | L3 | New sessions are never created on a closing connection. | safety | `appMachine.ts`, `HttpAdapter.ts` | `tests/4-session-connectivity.test.js` (regression example) | | L4 | Disconnect during `connecting` transitions to `disconnected`, never a stuck spinner. | liveness | `appMachine.ts` (`connecting`) | `disconnected.test.ts › ends a connection…`, `› ends a reconnection…` | ======================================================================== docs/INVARIANTS.md https://github.com/flplima/tmuxy/blob/main/docs/INVARIANTS.md ======================================================================== # Invariants Cross-layer promises, in one place, each with the test that holds it. Doc comments explain _how_ an invariant is kept; this file indexes _what_ is promised across a seam and _whether anything checks it_. Module-local assertions stay in code. | Column | Meaning | | --- | --- | | `Id` | Stable name prefixed by seam. Cite in commits and test titles; never renumber. | | `Kind` | `ordering` (A before B), `safety` (X never happens), `liveness` (Y happens within a bound). Ordering/liveness want a property or interleaving test, not one scripted run. | | `Stated in` | Code defining the mechanism. | | `Enforced by` | `file › test title`. `(example)` marks a single scripted run standing in for a property; `—` means uncovered. | | Prefix | Seam | | --- | --- | | `A` | Control-mode events → `StateAggregator` → `SideEffect`s (`tmuxy-core`, sans-IO) | | `M` | `SideEffect`s → `TmuxMonitor` dispatch → tmux commands (`tmuxy-core` runtime) | | `X` | Tmux connection and command router (`tmuxy-core`) | | `S` | Server → client stream: SSE seq, ring buffer, first state (`tmuxy-server`) | | `C` | Client stream stage: sequencer, resync, per-frame coalescing (`tmuxy-ui`) | | `Q` | Client command queue and keyboard batching (`tmuxy-ui`) | | `P` | Client model: predictions and reconcile (`TmuxClientModel`) | | `U` | UI-mode records: copy mode, resize preview, scrollback, menus | | `L` | Session lifecycle: attach, restore, autosave | See `AGENTS.md`: a bug-fix commit on a seam cites or adds a row, and a row's enforcer is a test title that exists. ## A — aggregator effects | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | A1 | Step effect order: `AdoptUntaggedWindows` → `StoreImages`/`WriteClipboard` → `SendTmuxCommand`/`CommandReply` → `RefreshAfterWindowAdd` → `RefreshPanes` → `ResumePane` → `EmitState`. | ordering | `state.rs` (`step_at`) | `state_aggregator_props.rs › effects_within_a_step…`, `› the_seam_events_reach…` (property) | | A2 | Window add/close relists before emit: `RefreshAfterWindowAdd` precedes `EmitState`. | ordering | `state.rs` (`step_at`) | `state_aggregator_props.rs › window_add_refresh_precedes_any_emit` (property); A1 | | A3 | Tag-only steps promote to `Window` change so untagged windows emit immediately. | liveness | `state.rs` (`step_at`) | — | | A4 | Captures route by marker, never arrival order or shape; dead-pane captures are discarded. | safety | `state.rs` (`queue_captures`) | `state.rs › capture_routing…`, `› capture_marker…`, `› capture_for_dead_pane…`, `› a_failed_capture…`; `state_aggregator_props.rs › queue_captures…` (property) | | A5 | Output between reply markers skips pane/window parsers; mismatched END fails the older reply cleanly. | safety | `state.rs` (replies) | `state.rs › a_query_between_markers…`, `› a_mismatched_end_marker…`, `› non_list_panes_output…` | | A6 | Reply-wrapped commands send `BEGIN`, cmd, `END` in order, each its own line; `END` runs even after `%error`. | ordering | `state.rs` (`reply_wrapped_lines`); `guest/controlMode.ts` (`TrackedCommandBook`, `CaptureBook`) | `state.rs › reply_markers_collect…`, `› a_failed_block_fails…`; `reply_channel.rs › replies_come_back…` (real tmux); `trackedCommandBook.test.ts › sends its markers as their own lines…`, `› a command tmux refuses resolves…`; `AppOptimistic › RefusedSplitReportsTmuxsReason` (v86) | | A7 | Settling emits at most one `EmitState` and clears by safety ceiling (silently if idle). | liveness | `state.rs` (`arm_settling`, `tick`) | `state_aggregator_props.rs › settling_window…`, `› settling_round_trips`, `› suppressed_window…` (properties); `state_aggregator_sideeffects.rs › tick_after_deadline…`, `› tick_safety_timeout…` | | A8 | `change_type` surfaces during suppressed emissions so settling timers extend on window changes. | liveness | `state.rs` (`step_at`) | `state_aggregator_sideeffects.rs › change_type_is_surfaced…`, `› step_at_extends…`; `state_aggregator_props.rs › emit_state_change…` (property) | | A9 | Moved panes hold `%output` until capture lands and drop old content; others keep content during capture. | safety | `state.rs` (refill) | — | | A10 | Stray `%end`/`%error` without `%begin` never produces a command response. | safety | `parser.rs` | `parser.rs › stray_end_without_begin_is_ignored`; `pane_bytes_props.rs › …arbitrary_lines`, `› …hostile_notifications` (properties) | | A11 | A pane's output bytes reach the emulator exactly, wherever tmux cut its `%output` lines — a character split across lines included; pane output is never read as text. | safety | `parser.rs` (`parse_line`), `connection/spawn.rs` | `pane_bytes_props.rs › output_cut_anywhere_carries_the_bytes` (property); `control_mode_push_api.rs › a_character_split_across_output_lines_reaches_the_screen_whole` | | A12 | An in-browser guest's query reply is what came between its markers less tmux's notifications: another pane's `%output` arriving meanwhile stays out, a `run-shell` job's lines (printed after its block closed) stay in, and a block closes only on the `%end`/`%error` that repeats its `%begin`. | safety | `guest/controlMode.ts` (`CaptureBook`) | `trackedCommandBook.test.ts › keeps tmux's notifications out of the reply…`, `› keeps a line inside a block…` | ## M — monitor dispatch | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | M1 | Monitor dispatches step effects one-to-one in exact aggregator order. | ordering | `monitor.rs` | `monitor.rs › a_step_sends_its_effects_commands_in_effect_order` (example; order is A1) | | M2 | Window add/close and initial sync send `list-panes` before `list-windows` (stash last) in monitor and wasm. | ordering | `state.rs`, `monitor.rs` | `monitor_command_order.rs › a_window_relist_sends_panes_before_windows…` | | M3 | `RefreshPanes` sends `list-panes` before per-pane `capture-pane`s for fresh cursor positions. | ordering | `state.rs`, `monitor.rs` | `monitor_command_order.rs › a_pane_refresh_lists_panes_before_every_capture` (property) | | M4 | Windows set `window-size manual` and `aggressive-resize off` before first `resizew` so SIGWINCH never races captures. | ordering | `monitor.rs` | `monitor.rs › a_window_is_told_to_stop_sizing_itself_before_it_is_sized` (example) | | M5 | Client-size pass runs after the step's own effects, never ahead of `EmitState`. | ordering | `monitor.rs` | — | | M6 | `GetState` waits for sync, captures, and queued refreshes (15 s cap); first state awaits resize up to 750 ms. | liveness | `monitor.rs`, `transport.rs`, `sse.rs` | `sse.rs › a_clients_first_state…`, `› a_first_state_waits…`, `› a_client_that_sets_no_size…`; `initial_state.rs`, `initial_state_history.rs` (real tmux) | | M7 | Every reply resolves within `REPLY_TIMEOUT`, or immediately on send failure. | liveness | `monitor.rs` | `reply_channel.rs` (success/error; timeout is —) | | M8 | Reply ids are monotonic so expired reply blocks never match newer ones. | safety | `monitor.rs` | — | | M9 | `%session-changed` for another session is filtered before reaching the aggregator. | safety | `monitor.rs` | — | | M10 | Resizes retry <=3 times per size after new tmux reports; new sizes reset count; unmonitored sizes re-request. | liveness | `monitor.rs`, `sse.rs` | `monitor.rs › a_window_tmux_did_not_resize…`, `› …not_asked_again…`, `› a_size_tmux_keeps_refusing…`, `› a_new_size_starts…`, `› a_window_with_no_panes…`; `sse.rs › a_size_that_reached_no_monitor…` | | M11 | UI choices re-apply after `source-file` (UI wins), and enforced settings after both. | ordering | `monitor.rs` | `enforced_settings.rs › connecting_enforces_the_settings…` (final values; order is —) | ## X — the tmux connection and the router | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | X1 | Client commands keep pins intact, refuse `resize-window`, and rewrite `new-window` to `splitw ; breakp`. | safety | `command_router.rs` | `command_router.rs › a_pinned_list…`, `› resize_window_is_refused…`, `› new_window_is_rewritten…`; `pinned_split_race.rs › a_pinned_split_survives…` (real tmux) | | X2 | No tmux subprocess runs while control mode is attached; all reads/writes use control mode (`docs/TMUX.md`). | safety | `command_router.rs`, `docs/TMUX.md` | `tests/cli/` (socket flag); ESLint ban on `tmuxExec` in E2E (`docs/TESTS.md`) | | X3 | Control-mode client is never killed with SIGKILL; it detaches with a 3 s grace period. | safety | `connection.rs` | — | | X4 | Attach and create serialize under one lock via `new-session -A`; attach never creates a session. | safety | `connection.rs` | `connection.rs › creating_a_session_attaches…`, `› attaching_never_creates_a_session` (lock is —) | ## S — server stream | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | S1 | Per-session broadcast seqs count from 1 (0 = empty) and the ring preserves concurrent broadcast order. | ordering | `tmuxy-server/src/state.rs` | `state.rs › seqs_are_monotonic…`, `› concurrent_broadcasters_keep…`, `› replay_since_returns…`, `› ring_buffer_drops_oldest…` | | S2 | No duplicates or fillable gaps: connect replays from `Last-Event-Id + 1`; lag replays ring tail (dropping `<= last_replayed`) or leaves one edge gap for C1. | safety | `sse.rs` | `sse.rs › the_first_broadcast…`, `› a_lagged_stream…`, `› a_client_that_stops_reading…`, `› a_client_beyond_the_buffer…` | | S3 | Reconnecting beyond the ring serves no replay so the first live delta triggers a gap refetch (C1). | safety | `sse.rs` | `sse.rs › a_client_beyond_the_buffer…` (`can_replay_from`; connect path is —) | | S4 | Streams end on shutdown; the last client leaving keeps the monitor for a 2 s grace period. | liveness | `sse.rs` | `sse.rs › open_stream_ends…`, `› the_last_client_leaving_keeps…` | | S5 | Monitor reconnect gives up after 5 short runs and broadcasts fatal so clients stop retrying. | liveness | `sse.rs` | — | ## C — client stream stage | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | C1 | Deltas apply only to their base state; missing, misordered, repeated or undecodable deltas drop and refetch full state. | safety | `stateSequencer.ts`, `stateFeed.ts`, `deltaProtocol.ts` | `stateSequencer.test.ts › asks for a resync…`, `› treats a payload…`; `stateFeed.test.ts › a payload…`; `deltaProtocol.test.ts`; `HttpAdapter.test.ts › a state update…` | | C2 | Pre-full-state deltas are ignored; full states/answers carry `seq` so a delta lost after sync triggers a gap. | safety | `stateSequencer.ts`, `deltaProtocol.ts` | `stateSequencer.test.ts › ignores a delta…`, `› a delta lost…`; `stateFeed.props.test.ts › never paints backwards…` (property); `deltaProtocol.test.ts › no gap…` | | C3 | At most one refetch runs in flight; arriving deltas are boundedly held, and failed refetches retry on next delta. | safety | `stateFeed.ts`, `stateSequencer.ts` | `stateFeed.props.test.ts › never paints backwards…` (property); `stateFeed.test.ts › a gap refetches…`; `stateSequencer.test.ts › the deltas…` | | C4 | Post-refetch publish merges the answer, held deltas `> seq`, and queued deltas monotonically; held gaps refetch. | ordering | `stateFeed.ts`, `stateSequencer.ts` | `stateFeed.props.test.ts › never paints backwards…` (property) | | C5 | Once synced, initial-state answers only fill empty panes; before sync or after a gap, they set starting state. | ordering | `deltaProtocol.ts`, `stateSequencer.ts` | `HttpAdapter.test.ts › an initial-state answer…`, `› …does not take back…`, `› after a gap…`; `stateSequencer.test.ts › after a gap…`; `stateFeed.props.test.ts › never paints backwards…` (property) | | C6 | New connections or session switches reset sequencing so old delta seqs never gate new streams. | safety | `HttpAdapter.ts`, `TauriAdapter.ts` | — | | C7 | Transport publishes events in arrival order (including pre-`connect` subscribers); web frame-hold never reorders state ahead of earlier events. | ordering | `TmuxTransport.ts`, `events.ts`, `stateFeed.ts` | `TmuxTransport.test.ts › publishes what the driver pushes…`; `stateFeed.test.ts › publishes a full state…` (examples; per-frame hold is —) | | C8 | States coalesce to one per frame (latest wins); older states never paint after newer ones. | ordering | `stateFeed.ts` | — | | C9 | Each pane id appears at most once in client state, and a delta removes exactly the panes it nulls. | safety | `deltaProtocol.ts` | `deltaProtocol.props.test.ts` (property) | | C10 | Dropped/silent streams retry with backoff, online/visibility/Retry reconnect immediately, refused streams stop, and `EventSource` is singleton. | liveness | `HttpAdapter.ts` | `HttpAdapter.test.ts › keeps retrying…`, `› ten drops…`, `› a pinged stream…`, `› a stream the server refuses…`, `› dedupes concurrent…`; `tests/4-session-connectivity.test.js › six drops…` [nightly] | ## Q — command queue and input | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | Q1 | Mutations serialize in issue order so splits never land in a tab a queued `select-window` is leaving. | ordering | `HttpAdapter.ts`, `TauriAdapter.ts` | `HttpAdapter.test.ts › runs run_tmux_command…` (example; Tauri is —); `tests/4-session-connectivity.test.js › 2 panes → reload…` | | Q2 | Keystroke batches share the mutation queue and non-`send-keys` flush pending batches first. | ordering | `TauriAdapter.ts`, `HttpAdapter.ts`, `keyBatching.ts` | `keyBatching.test.ts › flushAll drains…` (batcher only; mutation interleaving is —) | | Q3 | Lone keystrokes send on leading edge; literal/special order holds; hex-byte sends (`-H`) never batch. | ordering | `keyBatching.ts` | `keyBatching.test.ts › sends an isolated…`, `› preserves literal→special…`, `› preserves special→literal…`, `› never intercepts…`; `tests/1-input-interaction.test.js › Type → backspace…` | | Q4 | Client reads (`query_tmux`) run unserialized; order vs. mutations holds in-band on control mode (A6). | ordering | `HttpAdapter.ts`, `docs/DATA-FLOW.md` | — | ## P — client model | Id | Invariant | Kind | Stated in | Enforced by | | --- | --- | --- | --- | --- | | P1 | Placeholder (predicted) pane or window ids are never sent to tmux as targets — not by a dispatched op, a binding's pin, or a tab switch. | safety | `commands.ts`, `dispatch.ts`, `keyboardActor.ts` (`bindingPin`), `layout.ts` (`layout_selectTab`) | `dispatch.test.ts › leaves the target to tmux…`, `› once tmux has confirmed them…`; `keyboardActor.bindings.test.ts › leaves a placeholder window and pane unpinned…`; `selectTab.test.ts › switches the view to a placeholder tab…` | | P2 | Ops mark in-flight before send and always settle: unacked sweeps at `OP_STALE_TIMEOUT_MS`, acked at `OP_ACKED_STALE_TIMEOUT_MS`, errors roll back. | liveness | `TmuxStore.ts`, `model.ts`, `types.ts` | `model.test.ts › stale ops…`, `› in-flight ops…`; `TmuxStore.test.ts › dispatch rolls back…`, `› TransportError…` | | P3 | Pending ops re-reconcile on a timer so time-based verdicts fire on an idle stream. | liveness | `TmuxStore.ts` | `tests/7-regression-bugs-b.test.js › Creating tabs then switching keeps the rendered tab in sync with tmux` [nightly] (its `waitForSettled` holds while a switch is pending; on an idle terminal only this timer can release it) | | P4 | Confirmed focus ops linger (`FOCUS_CONFIRM_LINGER_MS`), yield to a 3rd pane after `FOCUS_SUPERSEDE_GRACE_MS`, drop on newer focus, and skip dead panes. | ordering | `ops.ts`, `model.ts` | `ops.test.ts › holds a confirmed…`, `› holds through stale…`, `› holds briefly…` (examples; `dropSupersededFocusOps` is —); `tests/2-layout-navigation.test.js › Clicking an inactive tab…` [nightly] | | P5 | In-flight splits never claim the same pane id within a reconcile pass. | safety | `ops.ts` (`reconcile`) | — | | P6 | `NewWindow` placeholders match only tab windows with a pane; the patch hides unmatched newborns. | safety | `windowOps.ts` | `ops.test.ts › stays pending until…`, `› patch hides a tab-typed-but-paneless…` | | P7 | Session switch clears pending ops and committed state before the new session's first state. | ordering | `tmuxStoreActor.ts`, `appMachine.ts` | `workflows.test.ts › TmuxStore — clear…` (store only; order vs. first state is —) | | P8 | `MoveWindow` predicts dropped strip order immediately and holds until server places the window beside its anchor. | ordering | `windowOps.ts`, `tabOverview.ts` | `ops.test.ts › MoveWindow predict…`; `tests/2-layout-navigation.test.js › a tab dragged along the strip…` | | P9 | A focus the client shows is one it asked tmux for: every optimistic active-pane change goes out with an op that makes tmux select that pane, so tmux's next state confirms it rather than taking it back. | safety | `focusAndDrag.ts` (`focusAndDrag_selectPaneGroupTab`, `focusAndDrag_focusPane`) | `AppPaneGroups.stories.tsx › ClickingTheShownMembersTabTakesTheFocus` [v86] (the group tab of the member on screen; other focus paths are —) | ## U and L — UI-mode records, session lifecycle In [INVARIANTS-UI-LIFECYCLE.md](https://github.com/flplima/tmuxy/blob/main/docs/INVARIANTS-UI-LIFECYCLE.md), the ledger's companion. ## What the ledger says today - **Uncovered (`—`)**: A3, A9, M5, M8, M9, X3, S5, C6, C8, Q4, P3, P5. - **Partial**: M7 (timeout), M11 (order), X4 (lock), S3 (connect), C7 (frame hold), Q1 (Tauri), Q2 (interleaving), P4 (`dropSupersededFocusOps`), P7 (first-state order), U2 (2-update hold). - **Example only**: M1, M4, Q1, U1. - **Property**: C3–C5 (`stateFeed.props.test.ts`) and S2 (`sse.rs`). ## Related - `docs/STATE-MANAGEMENT.md`, `docs/DATA-FLOW.md`, `docs/TESTS.md`, `docs/TMUX.md` ======================================================================== docs/NON-GOALS.md https://github.com/flplima/tmuxy/blob/main/docs/NON-GOALS.md ======================================================================== # Non-Goals This document describes what tmuxy intentionally does NOT do. These are conscious decisions, not missing features. ## Target Audience Tmuxy is for people whose work **runs in tmux** — long-lived sessions, agents left working, a laptop picked up from a phone — whether they know tmux's keys yet or not. The README's "beginner-friendly" means a newcomer can drive it with the mouse, the menus and the key hints and learn the keys as they go; it does not mean tmuxy courts people who do not want tmux at all, and it is not a replacement for a standalone terminal emulator. ## Core Philosophy **Tmuxy is a tmux UI, not a terminal emulator.** We delegate terminal emulation to tmux. We render what tmux gives us. We don't re-implement what tmux already handles well. --- ## Non-Goals ### 1. Full Terminal Emulation We do NOT maintain a local terminal state machine (cursor position, SGR attributes, scroll regions). Tmux already does this. ### 2. Live Local Scrollback Buffer We do NOT continuously buffer a pane's output on the client. tmux is the only owner of scrollback history. What we _do_ render client-side is a **view** of it: the mouse wheel in a normal shell opens the client-only scroll view, which renders scrollback **fetched on demand from tmux** (`get_scrollback_cells` → `capture-pane`), lazily in chunks as the user scrolls, and discards it when the view closes; tmux itself is never put into copy mode by the wheel — see [COPY-MODE.md](https://github.com/flplima/tmuxy/blob/main/docs/COPY-MODE.md). In alternate screen (vim, less), the wheel sends arrow keys. So scrolling works and scrollback renders client-side, but the history always comes from tmux at scroll time, never from a buffer we keep in sync with live output. **Why no live buffer?** - Duplicates tmux's work - Unbounded memory growth on the client - State divergence risk against the pane tmux actually holds - Lost on page refresh anyway ### 3. Local Search (Cmd+F / Ctrl+F) We do NOT implement browser-style find-in-page for terminal content. Users search via tmux copy mode (`Prefix + [` then `/` or `?`). **Why?** A find-in-page would have to search history we don't hold — the copy-mode `lines` map covers only the chunks fetched around the current viewport, so a client-side search would silently miss matches outside it. Searching in tmux searches the real buffer. ### 4. Local Echo / Input Prediction We do NOT predict keystrokes locally to reduce perceived latency (like mosh does). Every keystroke round-trips through tmux. **Why?** - Only benefits high-latency connections - Tmuxy is primarily for local/LAN use - Adds significant complexity - Risk of prediction errors ### 5. Binary Protocol / Compression We use JSON over SSE/HTTP, not binary encoding (MessagePack, Protobuf) or compression. **Why?** - For local tmux, bandwidth is not the bottleneck - JSON is debuggable and simple - Premature optimization ### 6. Unicode Width Calculation We do NOT maintain our own `wcwidth` tables for character width calculation. Tmux renders the grid; we display it. Text selection does not need one either: copy mode selects cells of the grid tmux laid out, one cell per column, where a wide glyph's second column is a cell of its own (blank). A selection's columns are therefore tmux's columns, with no width worked out on our side. See [COPY-MODE.md](https://github.com/flplima/tmuxy/blob/main/docs/COPY-MODE.md). ### 7. Canvas/WebGL Rendering We use DOM rendering (spans), not canvas or WebGL. **Why?** - DOM is simpler and works - Native text selection works with DOM - Accessibility (screen readers) works with DOM - Canvas/WebGL is premature optimization ### 8a. Resuming Processes Across a Restart A session snapshot (`tmuxy_core::session_snapshot`) records a session's SHAPE — tabs, splits, floats, sidebars, groups, each pane's directory and the program it ran — and rebuilds that. It does not and cannot resume the processes: no portable mechanism checkpoints a live process with its ptys and sockets, and the one that tries (CRIU) is Linux-only and privileged. A restored pane offers its program at the prompt; a program that wants to come back to where it was says how through `tmuxy pane restore-cmd` (`claude --resume
` | Base64 of any browser-renderable format |
| **APC \_G — Kitty Graphics** | `ESC _ G
▼
GET /api/images/{pane_id}/{image_id}
│ served by tmuxy-server (see api_routes in state.rs)
▼
stored bytes (already in the format the browser expects: PNG/JPEG/WebP)
```
`ImageParser` keeps the raw bytes per `(pane_id, image_id)` so that re-renders, viewport changes, and reconnects all read the same content. Sixel input is converted to PNG once at parse time so the server doesn't repeat the decode on every fetch.
## Placement geometry
Each `ImagePlacement` carries:
- `id` — monotonic per-pane image counter, matches the URL path
- `row`, `col` — top-left cell coordinates (`row` is signed `i32`: `0..height-1` on the live viewport, negative when scrolled into scrollback history)
- `width_cells`, `height_cells` — bounding box in terminal cells
- `protocol` — one of `iterm2`, `kitty`, `sixel`
The frontend positions the `