client: React client on the v3 API, replacing the Vue 2 client (#713)

* client-react: framework-free client core for the v3 API

A new client is built next to the Vue 2 one (client/ keeps shipping until
it is replaced at the end of this series). This first part is the core
library in client-react/src/core: no UI framework, no runtime
dependencies, written against master's protocol (server/pkg/types,
webrtc payloads, openapi.yaml).

- client.ts: NekoClient: login/logout/whoami (token or cookie, ?token=),
  websocket with backoff reconnect and stale-socket detection (the server
  heartbeats every ~10s), WebRTC as answerer (server candidates that
  arrive before signal/provide are buffered), the binary big-endian input
  data channel, control/screen/clipboard/broadcast/unicast messages,
  letterboxed video mount, microphone tracks via renegotiation.
- overlay.ts: input layer over the video: mouse (throttled, last
  position always sent), wheel steps with sensitivity, keyboard through
  the vendored Apache Guacamole keyboard (also used by the Vue 2 client)
  with the macOS key remapping, native touch or touch-as-mouse, file
  drop, and the host's cursor drawn for everyone not in control.
- store.ts: tiny proxy store (subscribe/watch/version) + typed emitter.
- api.ts + api.gen.ts: fetch wrapper; types generated from
  server/openapi.yaml (npm run gen:api), no generated runtime code.
- types.ts: client state and wire types mirroring server/pkg/types.

The root .gitignore ignores "core" (core dumps); client-react/.gitignore
re-includes src/core.

Check: npm run check (types, prettier, store self-check).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* client-react: React GUI with the look and features of the Vue 2 client

React 19 + Vite GUI on top of the core (client-react/src/app). The Vue 2
client's SCSS is reused with the same class names, so it looks the same.

Features: login (display name + password, ?usr=/?pwd= invite links,
OAuth button), video with fullscreen + keyboard lock, picture-in-picture,
resolution menu, clipboard sync and fallback, play/unmute overlays,
members with admin menu, control request/release/give/take, local
controls lock, microphone, volume, emotes, chat, file transfer
(streamed downloads, uploads with progress, multi-select delete),
settings (scroll, autoplay, emotes, chat sound, links in app, keyboard
layout, broadcast), locks for login/controls/file transfer, toasts and
SweetAlert-style dialogs (native <dialog>), all 15 translations,
URL params (embed, cast, volume, scroll, lang, show_side, mute_chat).

Behaviour notes:
- Kick/ban follow common chat-app semantics, documented at actions.kick
  in neko.ts: kick removes the session (may log in again); ban sets
  can_login:false and is only offered when the auth provider stores
  accounts (GET /api/members), with an unban list in Settings. Stale
  "IP ban / restart server" translations are dropped (English fallback).
- Settings keep the Vue 2 localStorage keys and format, so preferences
  carry over; scroll sensitivity uses the v3 steps (new key), ?scroll=
  is mapped onto them.
- Chat markdown (markdown.ts) parses to React elements: no HTML strings,
  links limited to http(s)/mailto. Emoji use the same emoji.json and
  twitter sprite sheet as before.
- Icon controls are labelled buttons with keyboard activation.

Check: npm run check (adds markdown self-check), npm run build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* client-react: two-user end-to-end test

e2e.mjs drives headless Chromium users (Playwright) through a running
server with the legacy API off and the multiuser provider: login + video,
members, chat (incl. markdown rendered safely and the emoji picker),
control hand-off and admin force-release, emotes, file upload/download/
delete, language switch, locks, mute, ban hidden without stored
accounts, keyboard activation, resolution change, keyboard layouts,
about, implicit hosting, clipboard sync, microphone (fake device),
reconnect after a network drop, typing into the desktop, kick, and URL
params (usr/pwd/lang/show_side, embed). With NEKO_FILE_URL pointing at a
file-provider server it also covers ban/unban.

Failing steps save screenshots of both users and log their member lists.

  NEKO_URL=http://host:port/ npm run test:e2e

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* client-react: Dockerfile, dev scripts, emoji tool, README

- Dockerfile (node 24 LTS, npm ci) with the same contract as the Vue 2
  client stage: npm run build -> /src/dist, used by Dockerfile.tmpl.
- dev/ docker wrappers like the Vue 2 client: serve runs vite on :3001
  with /api (incl. websocket) proxied to the backend on :3000 (override
  with NEKO_URL), plus npm/build/exec.
- tools/emoji.ts: the Vue 2 emoji generator ported to ESM (runs on node
  directly); regenerates _emoji.scss byte-identically (npm run
  build:emoji).
- README: core API, development, checks, e2e, behaviour notes.

Check: npm run check; docker build of this directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* client: replace the Vue 2 client with the React client

client-react/ becomes client/. The client uses only the v3 API (it makes
no requests to the legacy compatibility layer, which stays in the server
for other V2 clients), so images built from this tree also work with
NEKO_LEGACY=false. The image build is unchanged: Dockerfile.tmpl still
builds ./client and copies /src/dist to /var/www.

- CI: node 24 LTS (React 19 / Vite 7 need >= 20.19), `npm run check`
  before the build, no browser download for the e2e tool.
- Docs: developer guide (build, development, repository structure) and
  release notes.

Tested on an image built with ./build (xfce, default config, legacy on):
client e2e (28 steps), ban/unban on a file-provider server, no requests
to /ws, /stats, /file or /screenshot.jpg and no V2 data channel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* client: back off and give up when WebRTC keeps failing

When the peer connection failed (no media route: firewall, wrong
NAT1TO1, missing TURN, or a browser that cannot reach the server), the
client immediately asked for a new peer, which failed again: ~9 peer
setups per second on the server (551 in 60s in a test with Firefox).
Retries now back off (1.5s up to 6s) and stop after 10 consecutive
failures with "video connection failed (WebRTC)"; a successful
connection resets the count.

Tested: Firefox (ICE fails in this environment, for the Vue 2 client
too) now makes 11 attempts in 75s and shows the Disconnected dialog;
Chromium e2e 28/28.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* client: tell the user when the browser cannot play the stream's codec

If the browser has no video codec in common with the server (e.g. an
H264 stream and a Firefox without the OpenH264 plugin), it rejects the
video section of the offer (m=video 0). The server then fails with
"unable to start track, codec is not supported by remote" and never
answers ICE, so the client spun and retried until it gave up with a
generic WebRTC error. It now stops right away with "this browser cannot
play H264 video".

Tested with stock Firefox 157 (no OpenH264): H264 server -> message
after 0.3s; VP8 server -> video plays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* client: build on Debian trixie like the rest of the repo

Upstream moved its images to Debian 13 (trixie) in #581; the client
Dockerfile and dev scripts still used node:24-bookworm-slim.

Tested: client/Dockerfile builds on node:24-trixie-slim (Node 24.21)
and produces dist/; dev/build builds dist/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* client: ask for a new peer when the server closes it; mic survives peer changes

When the media path fails, the server notices within about 5 s, destroys
its peer and sends signal/close. The client only closed its own peer: a
locally closed RTCPeerConnection never reports "failed", so the backoff
from 5c9418fb never ran, and the websocket's heartbeats kept the stale
check quiet. Result: a frozen frame with status "connected" until the
page was reloaded. Reproduced by dropping the media port for 12 s with
the websocket up: no signal/request followed, and video.currentTime
stayed frozen for 60 s after the port was open again.

signal/close now takes the same path as a failure the browser noticed:
close, back off, request again. With this the client re-requested 1.5 s
after signal/close and video resumed about 5 s after the port opened.

Recovering exposed the microphone: its RTCRtpSender belonged to the old
peer connection, so removeTrack() on the new one threw InvalidAccessError
before the track was stopped (mic stuck on; releasing control then threw
inside an effect and the error boundary replaced the page). removeTrack()
now ignores senders of a previous connection, and the mic is switched off
whenever the connection is re-established, as the Vue client did on every
disconnect.

The core now runs under plain Node (imports with extensions, no parameter
properties, erasableSyntaxOnly), which makes src/core/client.check.ts
possible: a state-machine check with stub WebSocket/RTCPeerConnection that
fails on the old signal/close handling. It is part of `npm run check`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: encode session ids in REST paths

Session ids on the multiuser and noauth providers start with the name the
user typed at login, and the client interpolated them into URLs as they
were. The browser resolves "..", "?" and "#" before a request leaves, so
a member could choose what an admin's action against them does:

  display name            admin clicks  request actually sent
  x/../../logout?         Mute          POST /api/logout        (admin logged out)
  <someone else's id>#    Kick          DELETE /api/sessions/<someone else>

Both were reproduced against a test server. Every id that is part of a
path now goes through encodeURIComponent; with that the first name no
longer leaves /api/members/ and the second gets the right session kicked.

The e2e test logs in a member named "evil#?x" and checks that the admin's
mute lands on that member and the admin keeps the session. Names with "/"
need the server to build ids from a sanitised name; that is a separate
server change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: chat stays fast past the line limit

Once the chat held CHAT_LIMIT lines, every new line replaced the list
with `chat.slice(-CHAT_LIMIT)`. The elements of that slice are the proxies
the store had handed out, the setter unwraps only the array itself, and
the next read wrapped every element again: one more proxy layer on each
of the 1,000 kept lines per incoming message. Measured in the UI, the
main thread blocked 78 ms per line at the limit and 751 ms eighty lines
later; a read overflowed the stack about 3,000 lines in and the error
boundary replaced the page. Event lines count toward the limit, so a busy
room reached it in an evening.

Store.wrap() now returns a value that is already one of its proxies, and
the chat is trimmed in place. Lines carry an increasing `seq` used as the
React key and by the scroll-to-newest effect, so a trim no longer shifts
spoiler state between messages, re-parses every message, or stops the
scrolling (the length no longer changed once at the limit).

store.check.ts asserts that an element put back into the store keeps its
identity, and that 3,000 such round trips still read in constant depth;
the old store fails both.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: keep one identity per login, keep the session through an outage

Three session-flow defects, all reproduced against a test server:

- An invite link (?usr=&pwd=) opened in a browser that still had a valid
  session raced the autologin: the resumed session opened the websocket,
  the invite's login replaced the token, and connect() was a no-op
  because a connection was already wanted. The websocket then ran as the
  old session and every REST call as the new one (3 of 3 runs). login()
  now closes whatever was resumed before storing the new token.

- After the core gave up reconnecting (about 80 s offline) the app asked
  /whoami and treated any failure as "session gone", including a network
  error, so logout() removed the saved token and the user landed on the
  login form although the session was still valid. The screen written
  for this case (logged in but disconnected, with a Connect button) was
  unreachable. Only a 401 logs out now.

- A wrong ?pwd= was a dead end: the invite was cleared only on success,
  the password field stayed hidden and the URL was already cleaned. The
  invite is now used for the first attempt only, as in the Vue client.

The disconnect dialog also names the common case where a second tab or
window took over the session, instead of the raw "connection replaced".

e2e: a second invite link in a profile with a live session must leave
exactly the new session connected; a long outage must end at the Connect
button, not the login form.

The websocket is closed with status 1000 when the client closes it on
purpose (login, logout, giving up). Without a status the server assumed a
reconnect was coming and kept the session connected, and host, for 5 s;
with it the old session drops at once, which the invite test relies on.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: chat parser cannot be stalled by a message

The chat markdown parser runs during render for every message from
every member. Three inputs made that expensive or fatal:

- A code fence with no closing fence followed by newlines: the fence
  regex backtracked for 4.3 s on a 512-character message that fits the
  normal chat box (the same expression the Vue client used).
- Long runs of backticks: the inline-code regex with a backreference was
  quadratic per position.
- Deep nesting or very long messages: 24,000 asterisks overflowed the
  stack inside render, which ends at the error boundary for everyone.

Fences and inline code are now found with indexOf, the line-start and
previous-word facts a rule needs are computed once per position instead
of by regex over the whole prefix, and the UI parses through parseSafe():
messages above 10,000 characters are shown as plain text, and any parser
failure falls back to plain text as well. The self-check covers the
unclosed fence (must parse in milliseconds), 40,000 asterisks (must not
throw) and the length cap, and keeps the earlier formatting cases.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: muting the sound keeps the desktop usable

The "unmute" overlay was shown whenever the video played muted, so a
user who muted on purpose got a click-catching layer over the desktop:
the first click unmuted instead of reaching the remote, and the only way
to work with the sound off was the volume slider at zero. The Vue client
showed that overlay only until the first unmute.

The core now remembers when it was autoplay that muted the video, and the
overlay appears only in that case. e2e: muting from the controls must
leave no overlay over the player.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: member menu works on narrow screens

The member menu was rendered inside the members bar, which the layout
hides at 768 px and below. Right-clicking a chat author there set the
menu state but nothing appeared, so ignore, mute, kick and give-control
were unreachable on phones. The Vue chat had its own menu.

The menu is position: fixed anyway; it is now rendered by App, outside
the hidden container. e2e: at 700 px wide, the menu opens from a chat
author.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: the video takes the keyboard only while input can flow

Mouse-enter focused the hidden input layer for everyone, so a viewer
typing in the chat lost the focus by brushing the video and the rest of
the sentence went nowhere. The Vue client focused the overlay only while
hosting and unlocked.

The overlay now focuses on mouse-enter only when input is active, and
also when control arrives while the pointer is already over the video.
e2e: a viewer types half a sentence, crosses the video, and the chat box
has the whole sentence.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: emote repeat stops with the bar

Holding an emote sends it every 350 ms; the interval was cleared only by
mouse-up or mouse-leave on the bar. When an admin muted the member the
bar was removed first, so neither event could arrive and the member kept
sending that emote until reload (11 more in the 4 s after releasing the
mouse, then on). The interval now dies with the component, and a muted
member's sendEmote is a no-op, as the Vue client's action was.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: master's keyboard file; typed text as keys; Ctrl+V keeps both clipboards right

The vendored Guacamole keyboard was the 2022 copy from the reference
branch, not the one upstream master ships. It lacked two fixes from
September 2025: typing of indirectly entered text (IME, on-screen
keyboards) as key events (#577) and the GUACAMOLE-1823 Caps Lock fix
for macOS Chrome; it also sent right Alt as AltGr on every platform and
had wrong or missing Japanese keys. The file is now master's, byte for
byte.

With it, IME and phone text is typed as key events instead of going
through control/paste, which overwrote the remote clipboard (and over
https the local one) on every keystroke and did nothing in programs
where Ctrl+V is not paste.

Ctrl+V keeps using the browser's paste event, the reason the reference
client introduced the textarea in 2022: only Chromium over https can
read the clipboard programmatically, a paste event hands the text over
in every browser. Two defects in that path are fixed:

- Ctrl was released to let the paste through and the server's paste
  reset all keys, so with Ctrl still held the next key arrived as a
  plain letter ("xyzxyza"). Ctrl is no longer touched; the remote gets
  its paste keystroke from the overlay, which completes the chord with
  its own Ctrl if the user has let go by then.
- The local text always won, so after copying inside the remote a paste
  on a plain-http site (no clipboard sync) pasted stale local text. Now
  the side that copied most recently wins: the server reports every
  remote change, and if the remote changed since the local clipboard was
  last seen, only the keystroke goes out; otherwise the local text is
  set first and the keystroke follows once the server confirms it.

Verified in a browser on https and on plain http: a fresh session pastes
local text; copy-then-paste inside the remote yields the remote text on
both origins; Ctrl held across V and A selects all instead of typing a.
client.check.ts covers the decision; e2e now asserts what the typing
step delivered and that Ctrl+V inside the remote doubles it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: e2e asserts what its step names promise

- Page errors and console errors now fail the run; they were only
  printed, so a crash to the error boundary passed.
- The typing step checks what arrived in the remote (added with the
  keyboard change) and the resolution step waits for the revert to take
  effect instead of leaving later steps to run mid-resize.
- "cannot log in" for a banned account requires a login error with the
  form still shown, not any open dialog.
- The upload step is named for what it checks (the uploader's own list;
  the documented setup gives plain users no Files tab).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: core fixes from the review's minor list

- A member who may not watch never receives an offer and was left on a
  full-screen spinner for good, chat included. Eight seconds without an
  offer now shows the room without video. e2e: can_watch off, reload,
  the chat is reachable.
- The reconnect lookup (/whoami) is bounded to 5 s; on a black-holed
  network a fetch can otherwise hang for minutes. A close()+connect()
  while that lookup was pending could open a second socket; the decision
  now belongs to one connection generation.
- A closed peer took no retry timer with it, so a websocket reconnect
  could trigger a second signal/request.
- play() treated any rejection as a refused autoplay and muted the
  video; only NotAllowedError does now.
- Upload errors showed xhr.statusText, which is empty under HTTP/2; the
  server's JSON message is used.
- keyboard/map was re-sent, and setxkbmap run on the server, on every
  viewer setting change while hosting; the watcher now follows the
  layout, not the object.
- A click waiting for control under implicit hosting was replayed
  whenever control arrived, even minutes later by an admin giving it;
  only a press from the last two seconds is replayed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: small behaviours the Vue client had

- The side panel, the selected tab and the volume are remembered again,
  under the Vue client's keys (side, tab, volume 0..100). ?show_side=
  and ?volume= still win for the visit they are in; ?volume= is applied
  once, not on every reconnect.
- The autoplay setting is read when a track arrives, so changing it no
  longer needs a reload.
- Control events say who took the controls from whom and who released
  whose, with the strings every locale already had.
- Logging out clears the chat, the ignore list, uploads and the
  broadcast URL for whoever uses the browser next, as the Vue client's
  reset did. The login name and password the Vue client stored in
  localStorage are removed on load; nothing reads them.
- <html lang> gets zh-CN and zh-TW for the two Chinese locales instead of
  the invalid cn and tw.
- ?token= is removed from the address bar after login, like ?pwd=.

e2e opens the side panel instead of toggling it blindly, since the state
now survives a reload.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: small UI defects from the review

- The resolution menu was pinned to the window and sat on the side panel
  when that was open; it is now inside the player. It also no longer
  shares the member menu's class, so the member menu's row styles stop
  leaking into it.
- The file icon lost its class when the list was ported, so the gap
  before the name was gone.
- Choosing the same file twice in a row uploaded nothing: the file input
  is reset after each pick.
- The banned list reloads after a ban or unban from anywhere.
- A slow "can this member be banned" answer could land on another
  member's menu.
- The clipboard box follows what is copied on the remote while it is
  open. Fullscreen and picture-in-picture rejections are no longer
  unhandled. The video's control button toggles through the same
  action as the one in the room bar, with its toast.
- Emote names from other clients are checked against the known list
  before they become class names; link text in chat is not linked
  again, so [https://a](https://b) is one anchor.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: keyboard access to menus and pickers

- Escape closes the member menu, the emote and emoji pickers, the
  resolution and clipboard menus and the About panel; before, only a
  click anywhere did.
- The member menu moves focus to its first item when it opens, and the
  emote picker opened from the keyboard appears at the icon instead of
  the top-left corner.
- The 1,800 emoji were individual tab stops; they are reachable through
  the search box now, where Enter takes the first match.
- The broadcast buttons, the setting switches and the control lock have
  accessible names.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: remove what nothing uses

- The generated OpenAPI types (2,492 lines, a devDependency and a
  script) served two type names in three places; those two shapes are
  now declared in types.ts next to the other hand-written wire types.
  The generated schemas were all-optional and already lacked two
  settings fields, so nothing is lost.
- 47 emote PNGs that no stylesheet referenced (the SVGs are used).
- Styles ported from Vue components that have no element any more: the
  About page's markdown loader and bounce animation (connect.scss has
  the same keyframes), `body > p`, the event line's `<i>`.
- A re-export and a helper that nothing imported, and three locale keys
  nothing reads (one of them added by this branch).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs and build: release notes, Node version, dead build settings

- Release notes cover the behaviour changes of the fix series: media
  recovery, paste semantics and IME typing, the microphone on
  reconnect, the second tab, sanitised session ids.
- The build page said Node 20.19 works; `npm run check` runs TypeScript
  files with Node itself, which needs 22.18 or newer.
- PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD did nothing (Playwright 1.63 has no
  install script); removed from the Dockerfile, the workflow and the dev
  script.
- Three comments used a private convention; reworded as plain notes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: lint the two React hook rules

ESLint with only rules-of-hooks and exhaustive-deps (typescript-eslint
for the parser), run by `npm run check` and therefore in CI. The five
findings were effects that meant to run once but read a hook value:
they now read the stable store instead, or list the dependency.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: e2e can drive a real Chromium-based browser

NEKO_CDP=http://host:port connects the suite to a browser started with
--remote-debugging-port (Brave, Chrome, Edge) instead of launching
Playwright's Chromium, so the same steps run against a browser people
actually use. README: how to start it, the check script also lints, the
typing step works with Xfce's launcher too.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: the on-screen keyboard toggle closes the keyboard

Tapping the keyboard button while the keyboard was open closed and
reopened it: the button is focusable, so the tap took the focus from
the hidden input first (its blur handler cleared the flag), and the
toggle then saw "closed" and focused again. Checked with a Pixel 7
emulation: focus went overlay, overlay, overlay. The button no longer
takes the focus on press, and the toggle decides from its own flag
before the blur handler changes it. Now: overlay, none, overlay.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* server: default video pipeline without the mouse pointer

The client draws the pointer itself from the data channel (cursor image +
position, sent to every peer when its channel opens and on every host
move): CSS cursor for the host, canvas overlay for everyone else. With
the pointer also drawn into the stream, a second pointer trailed the
real one by the stream latency, for the host and for watchers.

Before #691 the default main pipeline had no pointer (the "legacy"
stream carried one for V2 clients, and the V2-config path still forces
show-pointer=false on main). #691 made the default visible; this goes
back to the original default for the main stream. Broadcast and
screencast keep following capture.video.show_pointer (default true),
the legacy stream keeps its pointer, and capture.video.show_pointer=true
puts it back on all video pipelines. Docs: custom pipeline examples use
show-pointer=false, with a note why.

Tested (xfce image built with ./build, default config, compat layer on):
- stream: pointer moved into an empty region changes received frames
  6/6 before (diff ~41k), 0/6 after (diff 0)
- watcher (not in control) sees the host's pointer drawn at the right
  spot 5/5, and still sees it after the host releases control
- client e2e passes

Known gap (from the code): pointer moves that do not come from neko
input (apps warping the pointer, touch-only hosts) are not broadcast, so
watchers see the last position until the next mouse move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* client: an invite link logs in even when a saved session resumes first

Opening a `?usr=&pwd=` link in a browser that still holds a session did
the invite login only if the saved session had not finished resuming by
the time the connect screen mounted. That is a race between the whoami
request and React's first effect; WebKit loses it every time (the page
stayed logged in as the old user, the server never saw the new one) and
Chromium happens to win it. The link now always logs in; login() already
drops a websocket the resumed session may have opened meanwhile.

Found by running e2e.mjs in WebKit (url params step).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: no "connected" without video while the server is silent

When a request for media gets no offer within 8 s, the room is shown
without video (the server declines, e.g. can_watch is off). After a lost
network the same timer ran for the wrong reason: the media path fails
within seconds, the client asks for a new peer into a socket that is
dead but not yet known to be (the stale check needs 25 s), and 8 s later
the connect screen gave way to a frozen picture labelled connected, until
the stale check reconnected. The fallback now applies only while the
server has spoken within the last 16 s (it heartbeats every 10 s);
otherwise it re-arms and leaves the outage to the stale check.

client.check.ts drives the timers by hand and fails on the old code.
Observed in WebKit with the server container paused for 40 s: before,
the connect screen disappeared at +25 s while the server was still
frozen; after, it stays until the real reconnect.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: e2e steps hold in WebKit and Firefox too

The second-invite step gave the server 1.5 s to drop the first session.
Chromium closes the old page's socket with a close code; Firefox closes
it abruptly (1006), and the server then keeps the session for its 5 s
reconnect grace. The step now polls for up to 8 s.

Playwright's offline emulation freezes an open websocket only in
Chromium; WebKit and Firefox keep delivering heartbeats to it (measured:
one frame in 15 s offline), so the two outage steps would wait for
nothing there. They say so and skip outside Chromium; the recovery path
itself was verified in WebKit by pausing the server container.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: @/ path alias in tsconfig; design layer for tokens, reset and fonts

First step of the layout requested in the #713 review: `src/design/` holds the
SCSS tokens (`_variables`), the reset, the Whitney font faces and the global page
styles that `assets/styles/main.scss` used to collect. The `@/` alias that only
Vite knew is now also in `tsconfig.json` (`paths`), so cross-layer imports can
use it everywhere.

The emote and emoji sprite sheets are still under `assets/styles/vendor/` and are
imported from `main.tsx` for now; they move next to their owners (emotes feature,
chat plugin) in the following commits. No CSS rule changes (built CSS compared
rule by rule).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: split neko.ts into state, i18n and a11y modules

`src/app/neko.ts` held the stores, settings, actions, dialogs, chat log, event wiring
and i18n in one module. It is now split by concern:

  state/storage.ts   localStorage helpers (same keys and '1'/'0' booleans as before)
  state/stores.ts    createStore(): every GUI store registers here so one hook can
                     subscribe to all of them (plugins will add their own)
  state/app.ts       the app store and the viewer-settings defaults
  state/client.ts    the NekoClient instance; name(), isMuted(), hostId(), isLocked()
  state/settings.ts  setSetting(), applySettings(), initSettings() (URL parameters)
  state/actions.ts   room actions (login, control, locks, kick/ban, broadcast)
  state/dialogs.ts   ask(), tell(), toast()
  state/api.ts       REST call with a toast on failure
  state/chat.ts      chat log, sendChat(), mute(), openInApp()
  state/files.ts     file transfer actions and types
  state/emoji.ts     emoji data for the picker
  state/emotes.ts    EMOTES, showEmote(), sendEmote()
  state/events.ts    server events -> chat lines and toasts
  state/hooks.ts     useNeko()
  i18n/              t(), setLang(), the locale table (moved from app/locale)
  components/a11y.ts a11y(), closeOn() and the Enter/Space activation listener
  app/boot.ts        start-up order: event wiring, settings, then the connection

Components still live in `app/` and now import from these modules with `@/`.
No behaviour change; built CSS identical.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: GUI split into app, layout, features and components; styles beside their components

The four GUI files (`App.tsx`, `Room.tsx`, `Side.tsx`, `Video.tsx`) and the
`app/styles/` folder become one folder per unit, each with its component and its
`.scss` next to it, imported by the component:

  app/         App (shell), ErrorBoundary, boot, app.scss (the #neko grid)
  layout/      Header, Side, RoomBar (the bar under the video)
  features/    video (+ Resolution, Clipboard), members (+ MemberMenu), controls,
               room-menu, emotes (+ Emote animation, sprite sheet), settings (+ Banned),
               connect, about
  components/  Avatar, Dialog, Toasts, Logo, ContextMenu (the `.context` menu used by
               the member menu and the emote picker), a11y
  plugins/     chat (Chat, Markdown, EmojiPicker, parser, emoji sprite sheet) and
               filetransfer (Files): components and styles only for now, still wired
               statically; the manifest and registry follow in the next commit

Units are imported through their `index.ts` with `@/`. `assets/` keeps only images.
`openMenu()` moved to `state/actions.ts` so the member list and the chat open the
same menu without importing each other. `tools/emoji.ts` and `build:emoji` write
the sprite sheet to its new place. No markup, class-name or behaviour change; built
CSS identical rule for rule.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: plugin manifest and registry; chat and file transfer become plugins

`src/plugins/types.ts` describes what a plugin contributes (sidebar tab, top-bar
items, member-menu items, locale strings, server event handler, init), and
`src/plugins/index.ts` lists the plugins of the build in order: `[chat, filetransfer]`.
The side panel, the header and the member menu render what the registry returns,
and `message` events starting with `<id>/` go to that plugin's `onEvent`. Adding a
plugin is its folder plus one entry in the registry; removing the entry removes its
tab, header item and event handling, and the app still compiles (checked with each
plugin and with both removed).

Each plugin owns its store (`createStore` registers it so useNeko() re-renders on
its changes), its actions, its locale strings (`locale.ts`, merged into the i18n
table at start-up; the merged table is identical to before) and its components:

  plugins/chat          Chat, Markdown, EmojiPicker, emoji data, the mute member-menu
                        item, `chat/init` and `chat/message`, the muted/unmuted lines
  plugins/filetransfer  Files, the header lock (via the shared LockButton), the
                        `filetransfer/update` event, the lock event line, uploads

`chat`, `texts`, `files` and `uploads` left the app store. Room events that the chat
shows as lines ("bob took the controls") go through a small bus in `state/bus.ts`,
so features and plugins never import each other; `logout` is a bus signal too.
`openInApp`, `ignored` and `chat_sound` stay in the app store and settings: they are
read by features (Settings, the member menu, emotes), not only by the chat.

The file drop on the video is unchanged: it is handled in the core (room/upload/drop,
the server's drop into the remote desktop) and the video feature never imported file
transfer. No markup, class-name or behaviour change; built CSS identical rule for rule.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: publish the core as the @m1k1o/neko package

`client/src/core` becomes `client/core`, its own npm package:

  core/package.json   @m1k1o/neko 3.0.0, ESM (`type: module`), `exports` with types,
                      `files: dist + README`, no runtime dependencies, `typescript` as
                      the only dev dependency (own lockfile, so `npm ci && npm run build`
                      works on its own)
  core/tsconfig.json  declaration build into dist/ with `rewriteRelativeImportExtensions`
                      (the sources import `./x.ts` so the self-checks run under plain Node)
  core/build.mjs      after tsc: `.ts` -> `.js` in the .d.ts imports (tsc only rewrites
                      the JavaScript), copies the vendored Guacamole keyboard
  core/src/index.ts   the public entry point: NekoClient, Overlay, Store, Emitter,
                      NekoApi, ApiError and the wire types
  core/README.md      the API reference with usage examples

The GUI imports only `@m1k1o/neko`: for development and the Vite build the name is
aliased to `core/src/index.ts` in tsconfig.json (`paths`) and vite.config.ts, so there
is no workspace and `client/Dockerfile` is unchanged. `npm run build:core` builds the
package; `npm run check` keeps running the core's `*.check.ts` and lints both trees.
`client/.gitignore` re-includes `core/` (the root .gitignore ignores core dumps).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: lint the layering rules and fail on import cycles

`eslint.config.js` gets one `no-restricted-imports` rule set per layer: between folders
only `@/` imports (a folder's own files relatively), features through their `index.ts`,
plugins only through the registry `@/plugins` (and the contract `@/plugins/types`), the
core only from `@m1k1o/neko`, and no layer imports one above it (app -> layout ->
features -> components -> design; state and i18n below all but design). `tools/cycles.mjs`
walks the import graph of `src/` (type-only imports excepted) and fails on a cycle. Both
run in `npm run check`.

Two imports did not fit the rules: the plugin entries used `../types` (now
`@/plugins/types`), and the About dialog read the version from `../../../package.json`;
Vite now defines `__APP_VERSION__` from it, same text in the dialog.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* ci: build the @m1k1o/neko package in the client workflow

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs: client layering, plugin contract and the core package

`client/README.md` documents the layout and its dependency rules, the plugin contract
with the steps to add one, the core package and the scripts. `client/core/README.md` is
the API reference of `@m1k1o/neko` (added in the package commit). The repository
structure page and the release notes name `client/core` and the package.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* core: unit tests for the connection state machine

node:test suites for @m1k1o/neko, run with `npm test` (from client/ or
core/) and as part of `npm run check`. client.test.ts drives NekoClient
headless against fake sockets, peers, fetch and a virtual clock: the
handshake, signal/close and peer failures with backoff and the give-up
after RECONNECT_MAX, the offer timeout, the stale socket check, socket
loss and reconnect, disconnect(), heartbeats, the codec mismatch,
system/disconnect, session/control/screen/clipboard/settings events,
preparePaste precedence, login/authenticate/logout, send() and
sendData(). store.test.ts ports store.check.ts. The *.check.ts
self-checks stay; *.test.ts is excluded from the tsc builds like
*.check.ts, so dist/ and the published package are unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: vitest replaces node:test and the check scripts

One test runner: `npm test` is `vitest run` (core/package.json delegates to
it for core/src). client.test.ts and store.test.ts import the runner from
vitest and keep node:assert; the custom clock in client.test.ts is now
vi.useFakeTimers (setImmediate stays real to settle promises between timers).
The three *.check.ts scripts are folded in: store.check.ts was already
covered by store.test.ts, client.check.ts's stale-sender removeTrack case
moved into client.test.ts, markdown.check.ts became markdown.test.ts. The
tsconfigs, tools/cycles.mjs, dist.check.mjs and the READMEs no longer
mention them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: Video.tsx -> Stage, Player and three hooks

Pure split of the 156-line Video component, no markup or behaviour
change (the view harness shows identical #root HTML for every view):
Stage.tsx keeps the layout, the two popover menus and the play/unmute
overlays; Player.tsx is the mount point the client puts the stream and
the input overlay into; useFullscreen (with the keyboard lock),
useClipboardSync (focus listener + the hover sync, and the
canReadClipboard capability) and usePip (the capability and the
request) carry the browser-API bits out of the component. The feature
index exports Stage; App.tsx renders it in place of Video.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* core: StreamTransport seam, WebRTCTransport, input channels

NekoClient no longer owns the RTCPeerConnection, the data channel or
the <video>: it keeps auth, the websocket, the room state, the stale
check, reconnects and heartbeats, the overlay and the letterboxing,
and hands the stream to a StreamTransport (transport.ts). Modelled on
guacamole-common-js (Client over a Tunnel).

transport.ts: StreamTransport (kind, element, input?, connect(session),
suspend(), close(), attach/detach, setPlaying/setVolume/setMuted,
addTrack?/removeTrack?, on('state'|'stats'|'error'|'cursor.position'|
'cursor.image')), SessionInfo (url, token, the system/init payload,
send/on over the main socket, lastMessage, autoplay) and InputChannel
(send(op, ...fields)). suspend() is what the socket loss did to the
peer (drop it, keep the picture and the backoff count); close() is the
full stop; status 'unavailable' is today's "no offer while the server
keeps talking", shown as connected without resetting the socket
attempt counter, as before.

transport/webrtc.ts: everything peer-related, unchanged in behaviour:
signal/request, provide/offer/answer/restart/candidate/close, trickle
ICE buffering, onnegotiationneeded for the microphone, the offer
timeout, peer-failure backoff with the same constants and messages,
the codec-mismatch check, ontrack -> its own <video> (same element,
attributes and style as before), the autoplay-muted fallback and the
mirrored video state, cursor frames from the data channel as events.

input/datachannel.ts: DataChannelInput, the big-endian encoding that
was sendData() plus the incoming cursor frames. input/websocket.ts:
WebSocketInput, the same opcodes as the server's control/* events
(delta_x/delta_y/control_key, keysym, code, touch_id/x/y/pressure),
moves coalesced on a 16 ms timer and flushed before any other input.
The client uses transport.input ?? new WebSocketInput(client); the
overlay only calls client.input. Not used by default (WebRTC has the
data channel), so nothing changes on the wire.

NekoClient: the `transport` option (instance or factory, default
WebRTCTransport); signal/* is forwarded to the transport and not
handled; connection.status follows socket + transport state as the
joint machine did; play/pause/mute/unmute/setVolume/addTrack/
removeTrack delegate; mount/unmount attach the transport and the
overlay; sendData and the `video` field are gone (transport.element).
App.tsx checks WebRTCTransport.supported() instead of
typeof RTCPeerConnection; the hooks read client.transport.element.

Tests: the 18 state-machine tests run unchanged against the real
WebRTCTransport (sendData moved to input/datachannel.test.ts); the
fake browser moved to test/browser.ts and grew elements, a data
channel and a richer peer; new client.transport.test.ts (the seam with
a fake transport), transport/webrtc.test.ts, input/websocket.test.ts.
build.mjs and tsconfig: the dist now has subfolders, the harness is
excluded from the builds.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* core: document the transport seam, check the new exports

README: Connection and Video and audio follow the transport, Control
and input describes client.input and the two channels, a Transports
section with the interface, SessionInfo, the state it reports and how
a second transport plugs in; the options, state and types tables and
the test notes updated. dist.check.mjs: the new exports, the default
transport and input, a custom transport falling back to WebSocketInput,
and the .d.ts specifier check walks the subfolders.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* core: typed input channel, shareMedia, smaller transport surface

InputChannel is now five typed methods (move, scroll, button, key, touch)
instead of send(op, [bytes, value]...): DataChannelInput encodes them into
the same frames as before (same opcodes, widths, order, big-endian; the OP
table and the field tuple are private to input/datachannel.ts) and
WebSocketInput sends the same control/* events and payloads, with the 16 ms
move coalescing and the flush before a click kept. The overlay's call sites
use the typed methods. The wire format and the control/* payloads are
unchanged.

StreamTransport carries no WebRTC types any more: addTrack/removeTrack
become shareMedia(stream), which returns the function that stops sharing;
NekoClient.shareMedia delegates (and throws "transport cannot send media"
without it); the GUI's microphone holds that function.

The transport surface shrinks to the maintainer's sketch: suspend() is gone
(a socket loss calls close(), a reconnect connect() again; close() drops the
media path, timers and counters, pauses the element and reports it not
playable but keeps the last picture), attach() returns the detach function,
SessionInfo is url/token/init/send/on (no lastMessage, no autoplay),
TransportKind drops 'webtransport'. Autoplay is the client's: setPlaying(true)
when the stream turns playable and the option is on; playback that was
running when the socket was lost is resumed by the client the same way. The
no-offer timeout reports 'unavailable' after OFFER_TIMEOUT_MS on its own;
the stale-socket check stays the guard for a dead socket. A redundant
clearTimeout in the signal/provide handler is dropped (closePeer clears it).

Behaviour changes, both accepted: the peer-failure backoff counter starts
over after a socket reconnect, and 'unavailable' no longer waits for the
server to be heard from. Everything else is unchanged.

Tests: every method's bytes and control/* payload, shareMedia, close and
reconnect, autoplay and resume, mount/unmount (the fake browser gained the
globals the overlay needs); README and dist.check.mjs follow the API.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* core: overlay is a separate control device, Player takes the transport

The keyboard/mouse/touch overlay leaves NekoClient. `Overlay` (already
exported) is now built by the consumer, `new Overlay(client)`, and attached
next to the transport's element with `attach(box)`, which returns the detach
function; it letterboxes the box it is given inside its parent (what the
client's mount() used to do for its container), subscribes to the transport's
cursor events, the store and the DOM itself, and takes everything down on
detach. `mobileKeyboardToggle()` moves onto it. client.ts no longer imports
overlay.ts or the Guacamole keyboard, which stays vendored and used by the
overlay only; dist.check.mjs walks the built client.js import graph to assert
it. `client.mount()/unmount()` and `canvasSize` are gone; `mount(client, el)`
in core/src/mount.ts is the plain-page convenience (the transport, then an
overlay, in a letterboxed box). The GUI builds its one Overlay next to its one
client (src/state/client.ts); `<Player transport={client.transport} />`
attaches the transport and then the overlay into a box it renders, Stage
passes `client.transport`.

onMessage no longer skips `signal/*`: every message already reaches the
transport's subscribers first, and the client acts on none of the signal
events, so they fall through to the `message` event like any other event it
does not handle (the plugin registry ignores unknown prefixes).

Close and outage rule: `WebRTCTransport.close()` is a true close - the peer,
timers, counters go and the element is emptied (`srcObject = null`, paused,
playable false), so a kick, system/disconnect, a give-up or disconnect() show
black behind the Connect screen again. A socket loss leaves the transport
alone: its peer may keep streaming while the socket reconnects, and a peer
that fails meanwhile re-requests itself through session.send, which drops
until the socket is back; the reconnect's system/init calls `connect()` again,
which now replaces whatever the previous session left (peer, retry, failure
count) while the last picture stays until the new stream shows. The
transport's status reports count only while the socket is up (a new
`introduced` flag in the client), so an 'unavailable' verdict during an outage
cannot read as connected.

Markup, CSS and the wire format are unchanged: the views harness shows 27/27
identical DOM dumps, the CSS rule diff is 0, the data-channel probe sends the
same frames.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* e2e: KDE paste check collapses the selection with Home, not End

The step types into KRunner, selects all and copies inside the remote, then
pressed End and Ctrl+V expecting the text twice; the paste never arrived, on
every build, and xdotool inside the container reproduces it without the client.
KRunner's results view (milou ResultsView.qml, navigationKeyHandler) takes End
when the query field's caret is already at the end of the text - which it is
right after Ctrl+A - focuses the results list and jumps to the last result; the
following Ctrl+V reaches the list, whose handler drops control characters, so
nothing is pasted. Home goes to the field (caret not at 0), collapses the
selection and the paste lands at the start, which doubles the text the same way.

The client's paste path is correct: the browser paste event carries the remote
text, no clipboard/set is sent, and the Control_L/v frames arrive in order
(verified in headless Chromium and in Brave over CDP, identical frames).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* e2e: type into KRunner once it has the keyboard, not after a fixed sleep

On a cold container the full suite dropped one character mid-text now and then
("react gu works"). Traced on a fresh container with the server's per-frame
trace and xev attached to KRunner's window: the client sent and the server
injected all 15 characters in order, the KRunner window received the KeyPress
for the missing one - at the same X timestamp as the window's FocusIn. KRunner
is started by the Alt+F2 and takes the keyboard focus only once it is up; under
the suite's load that is later than the 1200 ms the step slept, and the key
that lands on the focus change is lost by the freshly activated window. The
step alone never reproduced it (5/5 cold), the suite did (1 of 2, plus the
coordinator's run).

The sleep is replaced by a readiness probe: type a character, select all and
copy inside the remote, read the remote clipboard, until the character comes
back; then the text is typed over the still selected probe. 15 s deadline, no
retry of the measured typing, assertion unchanged. 5/5 cold full-suite runs.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* e2e: seed the remote clipboard before the hover-sync poll

The first poll of GET /api/room/clipboard ran before the hover sync had set
anything, so xclip found no owner and the server answered 500, which the
browser logged as an error and the suite counted as a page error (exit 1
with every step passing). The step is about the hover sync, so give the X
clipboard an owner first; the assertion still proves the sync replaced it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: replace the proxy store with zustand and selector hooks

The core's hand-written deep-observable Store (proxies, microtask batching, watch/version)
is gone: `client.store` is a zustand vanilla store with subscribeWithSelector and
`client.state` a read-only getter of its current state. Every write is one immutable
setState (what one server message changes is one update, so subscribers never see it
half-applied); the former watch sites are selector subscriptions; selectSession,
selectControlling and selectIsAdmin are exported for UIs. Emitter moved to emitter.ts.
zustand is the package's one runtime dependency.

GUI: the app, chat and filetransfer stores are zustand stores too; useNeko() (subscribe to
everything, return the state whole) is replaced by useStore(store, selector) in every
component, useShallow for several fields, getState/setState outside React. The plugin
contract's visible()/badge() become the hooks useVisible()/useBadge() (member menu:
useVisible(member)), each run in a component of its own (EachHook). A chat message now
re-renders the chat and the badge, not the stage.

Behaviour, markup and CSS are unchanged: same localStorage keys, URL parameters and events;
the DOM views, the e2e (34/34), the input probe and the CSS rules match the previous build.
READMEs, dist.check.mjs and the tests follow (66 tests: the store tests are rewritten for
zustand semantics, every former watch site has a subscription test, and the transport test's
status sequence no longer hides the first 'connecting' in a batched flush).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: locales as JSON per language and namespace, i18next with lazy loading

The strings move from src/i18n/locale/*.ts and the two plugin tables to
src/locales/<lang>/{common,chat,files}.json (15 languages, generated once
from the tables: the same strings, `{x}` is now `{{x}}`). The runtime is
i18next with react-i18next: components use useTranslation(), code outside
components `t` from @/i18n. The files are lazy chunks (import.meta.glob)
that a small backend hands to i18next as it asks for <lang>/<ns>, so only
the active language is downloaded, with en as the fallback; before, all 15
shipped in the main chunk (379 kB -> 372 kB, plus 45 chunks of 0.4-4 kB).

A plugin registers a namespace (`ns`) instead of handing over a language
map; its keys that sat in shared groups (side.chat, context.mute,
notifications.muted, locks.file_transfer.*, files.*, send_a_message) are
`chat:` and `files:` keys now, and a tab carries its own label key. The
language stays the short code in localStorage `lang` and ?lang=, and
<html lang> keeps the zh-CN/zh-TW mapping. A key no language has is logged
in development. src/i18n/i18n.test.ts checks the runtime, that every key
the sources use exists in en, and that each language lacks exactly the
known keys versus en (so a key added to en alone fails); tools/dist.check.mjs
(npm run build) checks the main chunk carries no language. No plural forms
yet (README).

Behaviour and markup unchanged: the same strings in every language, the
same storage keys and URL parameters; a language pick applies once its
chunk has arrived.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: plugin contract: typed events, named slots

Server events are typed through PluginEvents (event name to payload, after
server/internal/plugins/*/types.go): onEvent(...[event, payload]) narrows
the payload on the event, no `any` in the contract. The registry dispatches
through a map from the id prefix to the plugin, built once at initPlugins,
instead of a find per message; an event without a slash reaches nobody, as
before.

tab, topBar and memberMenu go into named slots: initPlugins registers them
in registry order (registerSlot 'side.tab' | 'header.item' | 'member.menu'),
and Side, Header and App read useSlot(name). The manifest keeps the three
fields, so adding a plugin is still a folder and one registry line, and
removing the line removes its slot entries and its event handling
(src/plugins/registry.test.ts).

Behaviour, markup and the bundle unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: e2e selects by test id, not by class or icon name

The e2e used 66 class selectors (15 of them Font Awesome icon classes), which the
styling pass is about to delete. The elements it drives now carry `data-testid`
(member, side, tabs, chat-*, control-request, mic, mute, lock-<resource>, dialog,
emoji-*, emotes-*, file*, resolution-*, about-*, ...) and their state as data
attributes (`data-self`, `data-host`, `data-locked`, `data-on`, `data-active`,
`data-shown`, `data-status`); `e2e.mjs` selects those. `LockButton` takes the id
of its resource for its test id. No style or behaviour change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: Tailwind v4 with the design tokens, cn(), shadcn config

Adds `tailwindcss` 4.3.3 and `@tailwindcss/vite` (the Vite plugin), `src/index.css`
with the theme and the utilities and an `@theme` block holding the 27 tokens of
`src/design/_variables.scss` under their names and values (`--color-text-normal`,
`--color-background-primary`, `--shadow-elevation-high`, `--spacing-side`, ...),
`cn()` in `src/lib/utils.ts` (`clsx` + `tailwind-merge`, tested) and
`components.json` for shadcn/ui (radix base, lucide icons). `lib/` is a bottom
layer like `state/` (eslint, README). The preflight is not imported yet and no
component is migrated: the 27 views render identically (0.00 % pixels differ).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: app shell and layout on Tailwind, room bar without its wrapper divs

`src/index.css` now imports the whole of Tailwind (the preflight replaces
`_reset.scss`), keeps the global rules the preflight does not have (the page's
size and overflow, the 14px/line-height 1 text on body, the focus ring of the icon
controls) and defines the layout's breakpoints as variants (`tablet:` under
1024px, `phone:` under 768px, stacked with Tailwind's `portrait:`/`landscape:`
for the split of the small screen). App, ErrorBoundary, Header, Side and RoomBar
carry their classes; the Logo is written out where it is used (its sizes differ
per place). The room bar's three wrapper divs are gone: the row is a 3-column
grid and the room menu, the controls and the emotes are its cells.

Deleted: src/design/_reset.scss (432), src/app/app.scss (115),
src/app/unsupported.scss (52), src/layout/Header/header.scss (101),
src/layout/Side/side.scss (51). The 27 views: 0.6 % of pixels differ on
average (max 1.8 %, the parts still on legacy scss shifted by the preflight's
border-box, migrated next).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: features on Tailwind

video (Stage, Player, Resolution, Clipboard), controls, members and the member
menu, room menu, emotes, settings, connect and about carry Tailwind classes; so do
Avatar and ContextMenu, whose rules lived in the features' scss. The glyphs the
scss drew with Font Awesome pseudo-elements (the member badges, the lock on the
control switch) are icon elements in the markup. The emote sprites are plain CSS
in Tailwind's components layer (sprites.css, same 47 rules). The shake, loader
and badge animations and the range slider are theme keyframes and an @utility in
src/index.css; Font Awesome's stylesheet is imported there into a layer below the
utilities. The Logo component is written out at its last two call sites.

Deleted: video.scss (100), clipboard.scss (28), resolution.scss (79),
controls.scss (183), members.scss (126), menu.scss (33), emotes.scss (65),
emote.scss (17), sprites.scss (150), settings.scss (232), connect.scss (126),
about.scss (86), avatar.scss (16), context.scss (79), components/Logo (12 + 1).
The 27 views: 0.33 % of pixels differ on average (max 1.5 %, the chat and files
panels still on scss, and the text of form controls, which now inherit the page
font instead of the browser's Arial).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: plugins on Tailwind

The chat (history, messages, markdown, the send box) with its emoji picker and
the file transfer tab carry Tailwind classes. The picker's search glyph and its
group tabs, drawn by Font Awesome pseudo-elements, are icon elements in the
markup. The transfer progress bar is styled (the preflight's border reset drops
Chromium's native theme). The generated emoji sprite sheet stays until the
emoji change.

Deleted: chat.scss (299), emoji.scss (227), files.scss (240). The 27 views: 0.3 %
of pixels differ on average (max 2.2 % in the emoji picker, where the form
controls inherit the page font).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: Dialog and Toasts on Tailwind

The modal dialog (the look of the legacy SweetAlert theme) and the toasts (the
legacy vue-notification palette) carry Tailwind classes; the last component scss
is gone. Named text sizes are written in px throughout (Tailwind's `text-xs` and
friends also set a line height, which the legacy layout did not).

Deleted: dialog.scss (72), toasts.scss (40). What remains of scss: the Whitney
font and the generated emoji sprite sheet. The 27 views: 0.2 % of pixels differ on
average (max 0.9 %, the files panel: the download icon takes the text colour
instead of the browser's link blue, the progress bar is styled).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: dialogs, menus and toasts on Radix (shadcn/ui) and sonner

The kit under src/components/ui/: dialog, dropdown-menu and popover in shadcn's
shape on @radix-ui/react-dialog, -dropdown-menu and -popover, restyled with the
tokens; context-menu on @radix-ui/react-menu (what shadcn's context menu wraps),
anchored to a point, since the member menu is one menu opened from the member
list, a chat author and Shift+F10; sonner's Toaster in the look of the legacy
toasts. They bring focus trapping, roving focus, Escape, outside click and ARIA.

- the modal dialog (ask()/tell(), unchanged promise contract, tested) and About
  are Radix dialogs; a click outside dismisses them too
- the member menu is the context menu; the emote picker and the resolution list
  are dropdown menus from their button; the clipboard textarea and the emoji
  picker are popovers from theirs, so each sits next to the control that opened
  it instead of at the pointer
- toast() maps its kind to sonner (tested); "clear on connected" is dismiss()

Deleted: closeOn() (the setTimeout that avoided the opening click), the toasts
state and the Toasts component, the ContextMenu component. The e2e passes 34/34
(the dialog test id is the mounted content now).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: lucide icons, real buttons, IconButton; a11y.ts and Font Awesome gone

Every icon control is a <button type="button"> through IconButton
(src/components/IconButton.tsx: cva variants plain / header / video, the label as
aria-label and title, tested), so Enter, Space and focus come from the browser;
the text buttons are the kit's Button (variants primary, outline, confirm, cancel);
the side panel's tabs are Radix tabs (@radix-ui/react-tabs, shadcn tabs.tsx);
the files actions are buttons. Icons are lucide-react components (40 of them for
the 40 Font Awesome names, the member badges and the switch knob included; the
GitHub mark is an inline SVG); a plugin tab's `icon` is a lucide component.

Deleted: src/components/a11y.ts (the a11y() spread and the global keydown
listener), LockButton (written out in Header and FileLock),
@fortawesome/fontawesome-free (the dist loses its four webfonts and its CSS).
The e2e passes 34/34; the input probe types and pastes (wsControl=0); the
keyboard and mobile checks (a11y-check.mjs) pass: Tab order, Enter/Space,
Escape on the dialog and the member menu, the touch keyboard toggle, the
clipboard fallback.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: emoji as text, the sprite sheet gone

The chat renders an emoji as its Unicode character (src/plugins/chat/Emoji.tsx,
a 22px box, the browser's emoji font), the custom `neko` as its image.
tools/emoji.ts emits the character of each name into public/emoji.json
(`chars`, from emoji-datasource's unified code points) and no longer writes a
stylesheet; every datasource emoji is listed (eye-in-speech-bubble, which had no
Twitter image, included; the modifier components are not). A test checks the
generated data against emoji-datasource, which is a devDependency now.

Deleted: src/plugins/chat/emoji-sprites.scss (5,426 generated lines) and the
4.2 MB Twitter sheet it embedded in the build (dist: 6.7 MB -> 1.6 MB). The e2e
passes 34/34.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: the last scss, sass, the Whitney font and the one-line barrels gone

- src/design/ is deleted (_variables.scss: the tokens live in src/index.css;
  _font-whitney.scss and the five woff files: the font is Tailwind's system
  stack); the scss block of vite.config.ts and the `sass` devDependency with it
- the 13 one-line index.ts barrels are deleted: a feature is imported by its main
  file (`@/features/video/Stage`), the eslint rule that forced the barrel is
  dropped (the layer rules stay); the single-file folders Avatar, Dialog, Header,
  Side and RoomBar are single files, so their imports do not change
- README: the tree and the layering bullets follow

The dist has no font files left (1.2 MB, from 6.7 MB). The e2e passes 34/34,
the input probe types and pastes, the keyboard and mobile checks pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: README: the design layer

Tailwind v4 and src/index.css (the tokens by their old names, the global rules,
the breakpoint variants, the animations and the slider utility), cn() and cva,
the shadcn/ui kit under src/components/ui/ and what each piece is used for,
IconButton and its variants, lucide icons and their sizes, emoji as text, and
how to add a component or an icon. The scripts table names what build:emoji
writes now.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: tailwindcss and its Vite plugin are build-time dependencies

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: the client through <NekoProvider> and hooks, the singleton gone

src/state/client.ts (the module-level `client` and `overlay`) and src/app/boot.ts
(the import-time start-up) are deleted. createNekoApp() (src/state/neko.ts) makes
one instance of the GUI, a NekoApp (src/state/app.ts): the NekoClient with its
WebRTC transport, the input Overlay, the GUI store and the bus, with the event
wiring (state/events.ts, now wireEvents(neko)) and the settings (initSettings(neko),
the URL parameters included; the ?cast= chat-sound rule moved there from App.tsx)
applied. main.tsx composes the page in the same order as before: the strings,
the instance, the plugins (initPlugins(neko)), then the connection and the first
render under <NekoProvider neko={neko}> (src/state/provider.tsx).

Components read the instance from the context: useNeko(), useClient(), useApp()
(the GUI store) and useActions() (the room actions bound to the instance, one set
per instance through scoped()); everything outside React takes it as its first
argument (setSetting, ask/tell, api, event/line, the emotes). The plugin contract
passes it to every callback (init(neko), onEvent(neko, ...), the member menu's
label(neko, m) and onClick(neko, m)); the chat and file-transfer stores and the
chat's mutedByMe are one per instance (scoped()), the slots stay the build's and
are filled once. The emoji data is still downloaded once per page, each chat's
store gets emojiReady from it.

Behaviour, markup and CSS are unchanged: same localStorage keys, URL parameters,
events and start-up order. Tests: an instance and two of them share nothing
(stores, bus, event wiring, settings, actions, plugin stores and wiring), the
provider's hooks (react-dom/server), the browser faked in src/test/browser.ts;
the three tests that used the singleton or a module store now make their own.
README: Composition (createNekoApp -> NekoProvider -> hooks, embedding) and the
plugin contract's instance argument.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: npm run check runs its scripts in parallel with npm-run-all2

The chain in `check` (tsc, prettier, eslint, tools/cycles.mjs, vitest, the core
build, core/dist.check.mjs) is split into scripts of their own: check:types,
check:format, check:lint, check:cycles (and the existing test), run in parallel
by `run-p` with one label per script and the output of each kept together
(--aggregate-output); the first failure stops the rest and names its script.
build:core and check:dist still follow in sequence (`run-s`), the latter reads
the former's output. `format:check` is now `check:format`. `npm run check`,
`npm run build` and `npm run build:core` keep their names, so CI
(.github/workflows/client_build.yml) is unchanged.

On this machine `npm run check` takes 3.2 s instead of 6.1 s (three runs each).
core/package.json's `check` is left as it is: build -> test -> dist.check is a
dependency chain with nothing to run in parallel. npm-run-all2 is the one new
(dev) dependency. README: the scripts table.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: crypto.randomUUID() for upload ids and emote keys, nextId() gone

The counter nextId() in src/state/dialogs.ts (used for the ids of the upload
list) and the Math.random().toString(36) key of a shown emote are replaced by
the browser's crypto.randomUUID(). Upload.id is a string now (it was a number);
nothing else read it but the React key and the progress updates. Toasts never
had ids of their own (sonner keeps them), so nothing changes there.

Tests: the upload list gives each file a UUID of its own and tracks progress
by it (the core's upload call stubbed); each shown emote gets a UUID key and
hideEmote drops it. README: the behaviour note, with the one constraint of
crypto.randomUUID(): a secure context (https or localhost).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: chat markdown rendered by react-markdown, the hand-written parser gone

Chat messages are other members' text, relayed by the server as they are, so
the 160-line parser of src/plugins/chat/markdown.ts is replaced by react-markdown
(CommonMark to React elements, no HTML string ever reaches the DOM) with
remark-gfm (~~strike~~, tables, bare links; singleTilde off) and remark-breaks
(a newline is a line break, as before). What remains ours in markdown.ts are
four small rehype plugins on the hast tree, 15 lines each at most:

- rehypeSafe: a link to anything but http(s)/mailto ([x](javascript:...),
  <data:...>, a relative path) and an image are replaced by the text they were
  typed as (the node's position in the source); a generated link without a
  position (a GFM footnote's) keeps its text only.
- rehypeUnderline: __x__, which CommonMark reads as strong, is <u> when the
  marker in the source is an underscore.
- rehypeSpoiler: ||x|| in a text node becomes a span Markdown.tsx renders as
  the click-to-reveal Spoiler. Plain text only: formatting inside a spoiler is
  split by the parser first and stays as typed (documented).
- rehypeEmoji: :name: of a name the emoji data knows becomes a span rendered by
  Emoji.tsx (the character, or the image of a custom one). Code is left alone.

Markdown.tsx passes urlTransform: safeUrl (the same whitelist, normalising the
href, a second line behind rehypeSafe), disallowedElements: ['img'] with
unwrapDisallowed (a third, should an image ever get through), and components
for a (target=_blank, rel, open-in-app and its icon as before), the plugins'
spans, code and pre; the other elements are styled from the message's wrapper
(strong, em, blockquote as before; p+p one line apart; lists, headings, table
cells minimal). Raw HTML is text by react-markdown's default. FORMAT_LIMIT
(10,000 characters) is kept, and react-markdown's Markdown, a plain function,
is called in a try/catch so a renderer error (5,000 nested quotes overflow the
hast conversion's stack) shows the text as it is. The component is memo'd: a
new line does not parse the others again.

Dialect: bold, em, underline, strike, code, fences, spoilers, emoji, links,
escapes and line breaks as before. Gone: the Discord `>>>` block quote (no
CommonMark equivalent; `> ` quotes stay, and continue until a blank line as in
CommonMark). New, by CommonMark/GFM: headings, lists, nested quotes, tables,
www. and e-mail autolinks. Images are never rendered.

Tests (src/plugins/chat/markdown.test.tsx, react-dom/server under a
<NekoProvider>): every assertion of the old parser's tests has a successor on
the rendered HTML, plus images, relative and reference links, raw HTML, the
CommonMark extras, open-in-app, and hostile input of every marker at the
10,000-character limit under a time bound. Bundle: +158 kB minified (+48 kB
gzip) in the main chunk. README: the behaviour note.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: list keys from crypto.getRandomValues, which plain http also has

crypto.randomUUID() is undefined outside a secure context, and a neko reached
over plain http on a LAN is not one, so the first upload or emote would have
thrown there. uid() takes 64 random bits from crypto.getRandomValues(), which
every origin has; the two call sites and their tests follow, the README note too.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: vendor chunks, locale chunks named by language

manualChunks splits node_modules into react, radix, markdown, i18n, lucide and vendor;
chunkFileNames names the lazily loaded locale files assets/locales/<lang>-<ns>-<hash>.js.
Before: index 730.17 kB (gzip 232.95) + 45 locale chunks common/chat/files-<hash>.js; total JS 800863 B.
After: react 222.29 (69.21), markdown 158.51 (48.21), index 119.85 (39.78), radix 96.71 (32.24), vendor 65.62 (19.69), i18n 48.97 (16.20), lucide 17.29 (6.23) + 45 locales/<lang>-<ns>; total JS 799935 B, no chunk-size warning, no module in two chunks.
Nothing in src/ changed; tools/dist.check.mjs now also checks the locale chunk names.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: the keyboard stays in the chat box after an emoji pick

Radix returns focus to the popover trigger when it closes, after the
component had focused the textarea, so the first key typed after picking
an emoji went to the Emoji button (found in a real browser; the old
picker kept the textarea). The picker now hands focus back to the message
box on close; the e2e emoji step types after the pick and checks it lands.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: a banned user goes back to the login screen; no mic button where getUserMedia is missing

A closed connection asked /whoami to decide between the login screen and the Connect button, and
only a 401 meant the login screen. A ban changes the profile first: the server disconnects the
session with "profile changed" and /whoami then answers 403 "login is disabled for this session"
(server/internal/api/router.go), which left the banned user on a Connect button that could never
work. 403 now goes back to the login screen too. The two file-provider e2e steps pass.

The microphone button called navigator.mediaDevices.getUserMedia without checking for
mediaDevices, which a plain-http origin does not have, so clicking it raised a raw TypeError. The
button is hidden there, the way the clipboard already picks its fallback by feature.

Found by a real-world pass (file-provider ban, LAN origin), both pre-existing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: chat formats messages up to the server's default length, hostile input stays cheap on CI

The hostile-input test failed on the CI runner: 5000 nested quotes took 1.9 s there (420 ms here),
micromark's cost for nested block quotes growing with the depth squared. The formatting limit was
10,000 characters, a number of our own. It is now 512, the server's default chat.max_length, so a
message only exceeds it on a server configured for longer ones, and the worst hostile case takes
23 ms here. The test gains the bare `>` form, which is the dense one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: the hostile-markdown test has no wall-clock assertions; its timeout is the bound

The second CI run failed on the other bound of the same test: the first render, which pays the
pipeline's warm-up, took 57 ms on the runner against a 50 ms limit. A shared runner is about four
times slower than a workstation and uneven, so any wall-clock assertion there is a flaky test. The
test now asserts what is deterministic (never throws, the fallback to plain text above the limit)
and is bounded by its own 3 s timeout, which a blow-up would hit; all the cases take about 60 ms.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs: release notes and repository structure match the client as it now is

The core is packaged as @m1k1o/neko, not published to npm. The release notes gain the chat dialect
(CommonMark/GFM, no `>>>`, plain text above 512 characters) and the system font and native emoji.
The repository structure drops the deleted design/ folder and names locales/ and lib/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* client: filled icons, phosphor instead of lucide

The v2 client's icons were all filled (FontAwesome solid); lucide is
stroke-only. @phosphor-icons/react has a fill weight, set once through
IconContext in App.tsx, with the same tree-shaken import style, so the
icon components are renamed and nothing else changes. The GitHub mark in
About is phosphor's GithubLogo instead of an inline svg.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: themed focus ring on text fields

The old reset removed outlines globally and this client had no rule for
inputs, so they showed the browser's default ring; now text fields get
the ring the buttons use, hidden where a container is the visual field
(the chat and clipboard textareas).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: remove ban, kick stays

The v2 ban was an IP ban in the legacy layer with no v3 counterpart, so
the ban/unban actions, the Banned list in Settings and their strings go;
member removal belongs with a members tab in a follow-up. The e2e suite
is 33 steps now, the optional NEKO_FILE_URL file-provider run is gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: add npm publish workflow.

* client: phosphor icons by their *Icon names

phosphor 2.1 marks every bare export (`Shield`) deprecated in favour of `ShieldIcon`. `@typescript-eslint/no-deprecated` (type-aware, src and core/src without the tests) now fails the lint on deprecated APIs; it also caught `FormEvent` and `navigator.platform`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs: the core library is published as @m1k1o/neko

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* client: the on-screen clipboard keeps the focus while open

The overlay takes the keyboard focus whenever the host's mouse enters the video; the clipboard popover
(the textarea fallback for browsers that cannot read the local clipboard) closed on that, before
anything could be copied from it. The popover is modal now, so the video gets no pointer events while
it is open. The e2e gets a step for it, with readText removed before the app loads to show the
fallback; the step fails on the previous code ("the clipboard popover closed").

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Miroslav Šedivý <sedivy.miro@gmail.com>

client/core/README.md

@@ -0,0 +1,328 @@
+# @m1k1o/neko
+
+Framework-free client library for the [neko](https://github.com/m1k1o/neko) v3 API: REST, the
+`/api/ws` event protocol, the stream through a pluggable transport (WebRTC with its binary input
+channel, see [Transports](#transports)), and a pluggable control device, with the keyboard/mouse/touch
+overlay (wheel, file drop, the host's cursor) included (see [Control and input](#control-and-input)).
+It is what the React GUI in `../src` is built on, and it can drive a neko server from any framework
+or from a plain page.
+
+ESM, TypeScript declarations included; the one runtime dependency is [zustand](https://github.com/pmndrs/zustand)
+(its vanilla store, ~1 KB), which holds the state.
+
+```sh
+npm install @m1k1o/neko
+```
+
+## Quick start
+
+```ts
+import { NekoClient, mount } from '@m1k1o/neko'
+
+const client = new NekoClient({ autologin: true, autoconnect: true, autoplay: true })
+
+// state: connection, video, control, screen, sessions, settings (a zustand store: select what you need)
+client.store.subscribe(
+  (s) => s.connection.status,
+  (status) => render(client.state),
+)
+// one-off happenings, including plugin events (chat/*, filetransfer/*, ...)
+client.events.on('message', (event, payload) => console.log(event, payload))
+
+// point at a server: resumes a saved session (autologin) or logs in with ?token= in the URL
+client.setUrl('https://neko.example.com/')
+// or log in by hand
+await client.login('name', 'password')
+client.connect()
+
+// the video, with the keyboard/mouse overlay on top (mount() does the two attaches below)
+const unmount = mount(client, document.getElementById('video')!)
+```
+
+The two pieces separately — the stream alone, or with the overlay as a second step:
+
+```ts
+import { Overlay } from '@m1k1o/neko'
+
+const box = document.getElementById('box')! // position: relative, inside the area the video fills
+const detach = client.transport.attach(box) // the stream: a <video> that fills the box
+const overlay = new Overlay(client)
+const detachOverlay = overlay.attach(box) // keyboard, mouse, touch and the host's cursor over it
+```
+
+## `NekoClient`
+
+### Options
+
+| option        | default  |                                                                                     |
+| ------------- | -------- | ----------------------------------------------------------------------------------- |
+| `autologin`   | `false`  | remember the session token in `localStorage` (`neko_session`) and resume it         |
+| `autoconnect` | `false`  | connect as soon as `setUrl()` finds a valid session                                 |
+| `autoplay`    | `false`  | start playback when the stream becomes playable (read each time, so a getter works) |
+| `inputMode`   | `'auto'` | `'touch'` or `'mouse'` instead of detecting it (`isTouchDevice`)                    |
+| `transport`   | WebRTC   | a `StreamTransport` (or a factory for one), see [Transports](#transports)           |
+
+### Authentication
+
+| method                        |                                                                                                          |
+| ----------------------------- | -------------------------------------------------------------------------------------------------------- |
+| `setUrl(url = location.href)` | the server (defaults to the page itself); `?token=` logs in with that token; with `autoconnect` connects |
+| `authenticate(token?)`        | validate a token (or the saved one with `autologin`) with `GET /api/whoami`                              |
+| `login(username, password)`   | `POST /api/login`; keeps the token when the server runs without cookies                                  |
+| `logout()`                    | `POST /api/logout`, forgets the token                                                                    |
+
+### Connection
+
+`connect()` opens the websocket (`/api/ws`) and, once the server has introduced the session
+(`system/init`), starts the transport, which brings up the stream (WebRTC: asks for an offer and
+answers it; the server's ICE candidates are buffered until the offer is applied). The connection
+is kept alive by itself: a socket that goes silent is replaced and the transport started over on the
+new one, a peer the server drops is requested again, and after repeated failures the client backs
+off and finally gives up (`connection.closed` with an error). While the socket is away the stream is
+left alone, so the last picture stays (and keeps streaming while it can) until the reconnect
+replaces it. `disconnect()` closes everything, the stream included (the element goes black); the
+session stays valid.
+
+`connection.status` follows both: `'connecting'` while the socket is being (re)opened or the
+transport has no stream yet, `'connected'` once the transport reports `'connected'` (or
+`'unavailable'`: the server will not stream to this session, the room is shown without video).
+
+| event / state                                        |                                                 |
+| ---------------------------------------------------- | ----------------------------------------------- |
+| `state.connection.status`                            | `'disconnected'`, `'connecting'`, `'connected'` |
+| `events: 'connection.status'`                        | the status changed                              |
+| `events: 'connection.closed'`                        | closed; with an `Error` when not asked for      |
+| `connected`, `session`, `isAdmin`, `implicitControl` | computed from the state (see [State](#state))   |
+| `transport`                                          | the `StreamTransport` in use                    |
+
+### State
+
+`client.store` is a [zustand](https://github.com/pmndrs/zustand) vanilla store (`createStore` with
+`subscribeWithSelector`), and `client.state` is its current state, `client.store.getState()`: a plain
+object that is replaced, never mutated, on every change (the changed slice is a new object, the rest
+keeps its identity). Read it anywhere; subscribe with a selector to be called when that value changes:
+
+```ts
+const off = client.store.subscribe(
+  (s) => s.control.host_id,
+  (hostId, before) => {},
+) // when the selected value changes (Object.is; options: { equalityFn, fireImmediately })
+client.store.subscribe((state, before) => {}) // after every change
+useStore(client.store, (s) => s.video.playing) // React: zustand's hook, re-renders on that value only
+```
+
+`selectSession`, `selectControlling` and `selectIsAdmin` compute what the getters `session`,
+`controlling` and `isAdmin` report, from a state: `client.store.subscribe(selectControlling, (on) => {})`,
+`useStore(client.store, selectIsAdmin)`. The client owns the state: `client.store.setState()` exists
+(zustand, and the tests use it) but nothing a consumer writes is sent to the server.
+
+| field                                      |                                                                                    |
+| ------------------------------------------ | ---------------------------------------------------------------------------------- |
+| `authenticated`                            | a valid session is known                                                           |
+| `connection.{url, token, status}`          |                                                                                    |
+| `video.{playable, playing, volume, muted}` | as the transport reports it; plus `mutedByAutoplay` when the browser refused sound |
+| `control.host_id`                          | who has control (`controlling` compares it with `session_id`)                      |
+| `control.locked`                           | local lock: keep control, send no input                                            |
+| `control.clipboard`                        | the remote clipboard, as the server reports it                                     |
+| `control.{scroll, keyboard, touch}`        | input settings and whether the server takes native touch events                    |
+| `screen.{size, configurations}`            | the current and the available screen sizes                                         |
+| `session_id`, `sessions`                   | this session and all sessions (`Session`: `id`, `profile`, `state`)                |
+| `settings`                                 | room settings (`Settings`: locks, implicit hosting, `plugins`, ...)                |
+| `mobile_keyboard_open`                     | the on-screen keyboard is up (touch devices)                                       |
+
+### Events
+
+`client.events` is a typed `Emitter` (`on`, `off`, `once`; `on` returns the unsubscribe):
+
+| event                    | arguments                                                              |
+| ------------------------ | ---------------------------------------------------------------------- |
+| `session.created`        | `id`                                                                   |
+| `session.updated`        | `id` (profile or state changed)                                        |
+| `session.deleted`        | `id`                                                                   |
+| `room.control.host`      | `hasHost, hostId, by`                                                  |
+| `room.control.request`   | `id` of the member asking for control (hosts only)                     |
+| `room.screen.updated`    | `width, height, rate, by`                                              |
+| `room.settings.updated`  | `settings, by`                                                         |
+| `room.clipboard.updated` | `text`                                                                 |
+| `room.broadcast.status`  | `active, url`                                                          |
+| `receive.broadcast`      | `sender, subject, body` (`sendBroadcast` from another client)          |
+| `receive.unicast`        | `sender, subject, body`                                                |
+| `upload.drop.progress`   | `{ loaded, total }` of a file dropped on the video                     |
+| `upload.drop.finished`   | `error?`                                                               |
+| `overlay.click`          | a click on the video (the GUI hints at the control button with it)     |
+| `message`                | `event, payload`: everything else, i.e. plugin events (`chat/message`) |
+
+### Video and audio
+
+The stream is the transport's: `transport.attach(box)` creates its media element inside `box` (a
+`<video>` filling it, keeping the aspect ratio by itself) and returns what removes it again; the
+media calls below go through the transport. `mount(client, el)` (a plain function, not a method)
+attaches the transport and an `Overlay` into a box it letterboxes inside `el`, and returns the
+function that takes them out again.
+
+| method / field                       |                                                                                                          |
+| ------------------------------------ | -------------------------------------------------------------------------------------------------------- |
+| `transport.attach(box)`              | the media element into `box`; returns the detach function                                                |
+| `transport.element`                  | the transport's media element (a `<video>` for WebRTC), once attached                                    |
+| `play()`, `pause()`                  | when the browser refuses sound, `play()` starts muted (`mutedByAutoplay`) and unmutes on the first click |
+| `mute()`, `unmute()`, `setVolume(v)` | `v` in 0..1                                                                                              |
+| `shareMedia(stream)`                 | send local media (microphone) on a transport that can; returns the function that stops sharing it        |
+
+### Control and input
+
+Input reaches the remote desktop through `client.input`, an `InputChannel` in the server's terms:
+the transport's own channel when it has one (WebRTC: the data channel, `DataChannelInput`, binary
+frames), otherwise `WebSocketInput`, which sends the same input as the server's `control/*`
+websocket events. The client does not care what produces the input: a control device is anything
+that calls these methods, and the one included is the `Overlay`.
+
+```ts
+interface InputChannel {
+  move(x: number, y: number): void // remote screen coordinates
+  scroll(deltaX: number, deltaY: number, controlKey: boolean): void
+  button(code: number, down: boolean): void // X11 button code
+  key(keysym: number, down: boolean): void // X11 keysym
+  touch(phase: 'begin' | 'update' | 'end', id: number, x: number, y: number, pressure: number): void
+}
+```
+
+`Overlay` is the keyboard/mouse/touch device: a transparent layer over the stream that turns DOM
+mouse, wheel, keyboard (Apache Guacamole's keyboard, vendored in `keyboard/`, so keysyms match the
+server's layouts), touch, drop and paste events into `client.input` calls while this member is host
+and not locked, asks for control on a click under implicit hosting, and draws the host's cursor for
+everyone else from the transport's `cursor.*` events. `new Overlay(client)` builds it (it reads the
+client's state and talks to it); `overlay.attach(box)` puts it into the box that already holds the
+transport's element (so the stream is below) and keeps that box letterboxed inside its parent to the
+remote screen's aspect ratio, so that its canvas maps onto the picture; the returned function
+detaches it (a second attach replaces the first). `overlay.mobileKeyboardToggle()` opens / closes
+the on-screen keyboard on touch devices. A stream without input is just `transport.attach(box)`: the
+client never loads the overlay or the keyboard unless they are imported.
+
+Another device (a gamepad, a VR controller, a test driver) does what the overlay does: watch
+`client.controlling` / `client.state.control.locked` to know when input is accepted, call
+`client.request()` to become host, and send through `client.input`:
+
+```ts
+const off = client.store.subscribe(
+  (s) => selectControlling(s) && !s.control.locked,
+  (on) => (on ? gamepad.start() : gamepad.stop()),
+)
+gamepad.onStick = (x, y) => client.input.move(x, y) // in remote screen pixels
+gamepad.onButton = (n, down) => client.input.button(n + 1, down)
+```
+
+| method                                           |                                                                                            |
+| ------------------------------------------------ | ------------------------------------------------------------------------------------------ |
+| `request()`, `release()`                         | ask for / give up control                                                                  |
+| `lock()`, `unlock()`                             | local lock (`control.locked`)                                                              |
+| `setScrollInverse(v)`, `setScrollSensitivity(n)` | wheel direction and v3 scroll steps (-5..5)                                                |
+| `setKeyboard(layout, variant?)`                  | the remote keyboard layout, sent while in control                                          |
+| `setScreenSize(width, height, rate)`             | admins: change the screen                                                                  |
+| `paste(text)`                                    | type text remotely through the server's clipboard                                          |
+| `preparePaste(text)`                             | before a paste keystroke: the side that copied most recently wins (see the source comment) |
+| `uploadDrop({ x, y, files })`                    | drop files into the remote desktop at a position (the overlay does this on drop)           |
+| `input.move(x, y)`, `input.key(keysym, down)`, … | input straight to the remote desktop (the `InputChannel` above)                            |
+
+### Messages
+
+| method                                 |                                                            |
+| -------------------------------------- | ---------------------------------------------------------- |
+| `send(event, payload?)`                | a websocket message, e.g. `send('chat/message', { text })` |
+| `sendBroadcast(subject, body)`         | to every other client (`receive.broadcast` there)          |
+| `sendUnicast(receiver, subject, body)` | to one session                                             |
+
+### REST
+
+`client.api` (`NekoApi`) carries the server URL and the token:
+
+```ts
+const sessions = await client.api.req<Session[]>('GET', '/sessions')
+await client.api.req('POST', `/members/${encodeURIComponent(id)}`, { can_login: false })
+const blob = await client.api.blob('/room/screen/shot.jpg') // admins
+await client.api.upload('/filetransfer', formData, ({ loaded, total }) => {})
+```
+
+Failures throw `ApiError` with `status` and the server's `message`. Session ids can come from
+login names on some member providers, so they go through `encodeURIComponent` in paths.
+
+## Transports
+
+`NekoClient` owns the session (auth, the websocket, the room state) and leaves the stream to a
+`StreamTransport` (`transport.ts`), the way Guacamole's client runs over a tunnel it does not care
+about. WebRTC is the one implemented (`WebRTCTransport`, `transport/webrtc.ts`); others
+(WebCodecs over a websocket, WebTransport, MSE, HLS) plug in the same way:
+
+```ts
+interface StreamTransport {
+  readonly kind: 'webrtc' | 'webcodecs-ws' | 'ws-mse' | 'hls'
+  readonly element: HTMLElement | null // its media element, created by attach()
+  readonly input?: InputChannel // its own input path, if any (the data channel)
+
+  connect(session: SessionInfo): Promise<void> // start streaming; again on every reconnect, replacing what runs
+  close(): void // the stream is over: peer, timers, counters and the picture on the element go
+
+  attach(container: HTMLElement): () => void // create the media element inside; returns what removes it
+
+  setPlaying(on: boolean): Promise<void>
+  setVolume(volume: number): void
+  setMuted(on: boolean): void
+  shareMedia?(stream: MediaStream): () => void // local media (microphone); returns what stops sharing it
+
+  on(event: 'state' | 'stats' | 'error' | 'cursor.position' | 'cursor.image', cb): () => void
+}
+```
+
+`SessionInfo` is what the client hands over on `system/init`: the server `url` and `token`, the
+`init` payload, and `send(event, payload)` / `on(event, cb)` for the transport's own messages over
+the main socket (every message reaches these subscribers; `signal/*` mean nothing to the client
+itself, which passes them on as `message` events like any event it does not handle). A socket loss
+does not touch the transport: `send` drops until the socket is back, and the reconnect's
+`system/init` calls `connect()` again, which replaces whatever the previous session left (its peer,
+retries and failure count) while the last picture stays until the new stream shows; `close()` is
+called only when the connection ends for good (`disconnect()`, a kick, giving up) and empties the element.
+
+A transport reports `'state'` (`{ status, size, video }`: its `status` — `'connecting'`,
+`'connected'`, `'unavailable'` when the server will not stream, `'disconnected'` — the stream's
+size and the media element's `playable`/`playing`/`volume`/`muted`/`mutedByAutoplay`, mirrored
+into `state.video`; its status counts only while the socket is up), `'error'` (the stream is lost
+for good; the client closes the connection with it), the host's `'cursor.position'`/`'cursor.image'`
+when its server feeds them back (the `Overlay` draws them), and `'stats'` (reserved). A transport
+does not start playback by itself: the client calls `setPlaying(true)` when `playable` turns true
+and `autoplay` is on, and a transport keeps playing across its own reconnects. A second transport
+needs: its own connection from the `SessionInfo`, an element it draws into, play/volume/mute, and
+either an `input` channel or nothing (the client then sends input over the websocket). Pass it as
+`new NekoClient({ transport })`; the GUI checks `WebRTCTransport.supported()` before rendering.
+
+## The store and `Emitter`
+
+The state is a zustand store (`createStore` from `zustand/vanilla` with `subscribeWithSelector`
+from `zustand/middleware`; zustand is the package's one runtime dependency): every change is one
+immutable `setState`, listeners run synchronously in it, and a selector subscription fires only when
+its value changes (see [State](#state)). A GUI makes its own stores the same way and selects from all
+of them with one hook (`useStore` from `zustand`). `Emitter<Events>` is a small typed event emitter
+(`on` returns the unsubscribe, `once`, `off`, `emit`), exported for the same reason.
+
+## Types
+
+The wire types mirror `server/pkg/types`: `Session`, `MemberProfile`, `SessionState`, `Settings`,
+`ScreenSize`, `InitPayload`, `LoginResponse`, `MemberData`, `CursorImage`, the client `State` and
+the `NekoEvents` map; the transport seam is `StreamTransport`, `SessionInfo`, `InputChannel`,
+`TransportKind`, `TransportStatus`, `TransportState` and `TransportEvents`.
+
+## Development
+
+```sh
+npm run build   # tsc -> dist/ (ESM + .d.ts), plus the vendored keyboard library
+npm test        # the unit tests in src/**/*.test.ts (vitest, run from ../ where it is installed): the connection
+                # state machine (handshake, reconnects, timeouts, events, clipboard, auth), the transport seam,
+                # the WebRTC transport, the input channels, the overlay and the store, against the fake browser
+                # in src/test/browser.ts (sockets, peers, elements, fake timers)
+npm run check   # build, the tests, then node dist.check.mjs against the built package: exports, the store
+                # (selector subscriptions, immutable updates, zustand resolvable), Emitter, setUrl, .d.ts
+                # specifiers, and that client.js never imports the overlay or the keyboard
+```
+
+The tests and dist.check.mjs run under Node with a few browser globals stubbed, so the connection state
+machine and the store are tested without a browser. The GUI in `../src` consumes this package from source
+through the `@m1k1o/neko` alias in `../tsconfig.json` and `../vite.config.ts`; consumers of the npm
+package get `dist/`.

client/core/build.mjs

@@ -0,0 +1,14 @@
+// After `tsc`: declaration files keep the `.ts` import extensions the sources use (tsc only
+// rewrites them in the JavaScript output), and the vendored keyboard library is plain JS.
+import { readdirSync, readFileSync, writeFileSync, mkdirSync, copyFileSync } from 'node:fs'
+
+for (const f of readdirSync('dist', { recursive: true }).filter((f) => f.endsWith('.d.ts'))) {
+  const src = readFileSync(`dist/${f}`, 'utf8')
+  writeFileSync(
+    `dist/${f}`,
+    src.replace(/(from '\.\.?\/[^']+)\.ts'/g, "$1.js'").replace(/(import\("\.\.?\/[^"]+)\.ts"\)/g, '$1.js")'),
+  )
+}
+
+mkdirSync('dist/keyboard', { recursive: true })
+for (const f of ['guacamole.js', 'guacamole.d.ts']) copyFileSync(`src/keyboard/${f}`, `dist/keyboard/${f}`)

client/core/dist.check.mjs

@@ -0,0 +1,135 @@
+// Runnable self-check of the built package in dist/ (after `npm run build`): `node dist.check.mjs`
+import assert from 'node:assert/strict'
+import { readdirSync, readFileSync } from 'node:fs'
+import { join } from 'node:path'
+
+// just enough of the browser for the core to load and construct (as in src/client.test.ts)
+const g = globalThis
+g.window = g
+g.location = { href: 'http://neko.test/' }
+Object.defineProperty(g, 'navigator', {
+  value: { platform: 'Linux', userAgent: 'check', maxTouchPoints: 0 },
+  configurable: true,
+})
+g.localStorage = { getItem: () => null, setItem() {}, removeItem() {} }
+g.matchMedia = () => ({ matches: false })
+
+const dist = join(import.meta.dirname, 'dist')
+const pkg = await import('./dist/index.js')
+const { WebRTCTransport } = pkg
+
+// the public surface
+for (const name of [
+  'NekoClient',
+  'Overlay',
+  'mount',
+  'Emitter',
+  'selectSession',
+  'selectControlling',
+  'selectIsAdmin',
+  'NekoApi',
+  'ApiError',
+  'WebRTCTransport',
+  'WebSocketInput',
+  'DataChannelInput',
+])
+  assert.equal(typeof pkg[name], 'function', name)
+
+// the client is free of the keyboard/mouse overlay and the Guacamole keyboard: a stream-only
+// consumer that imports NekoClient (and tree-shakes the rest, sideEffects: false) never loads them
+const imports = (file, seen = new Set()) => {
+  if (seen.has(file)) return seen
+  seen.add(file)
+  for (const [, spec] of readFileSync(join(dist, file), 'utf8').matchAll(/(?:from|import) ["'](\.[^"']+)["']/g))
+    imports(join(file, '..', spec), seen)
+  return seen
+}
+const clientGraph = [...imports('client.js')]
+assert.ok(
+  !clientGraph.some((f) => /overlay|keyboard/.test(f)),
+  'client.js must not reach the overlay or the keyboard: ' + clientGraph.join(', '),
+)
+assert.ok(
+  [...imports('overlay.js')].some((f) => f.startsWith('keyboard/')),
+  'the keyboard is still used by the overlay',
+)
+assert.equal(WebRTCTransport.supported(), typeof RTCPeerConnection !== 'undefined')
+assert.ok(!('OP' in pkg), 'the data channel opcodes are not part of the API')
+
+// the store: zustand, the one runtime dependency, resolvable from the built package (client.js keeps
+// the bare specifiers); a selector subscription fires on its value only, updates are immutable, and
+// the proxy store's API (Store, watch, version) is gone
+assert.ok(!('Store' in pkg), 'the proxy Store is gone')
+await import('zustand/vanilla')
+assert.ok(readFileSync(join(dist, 'client.js'), 'utf8').includes("from 'zustand/vanilla'"))
+const probe = new pkg.NekoClient({ autologin: false, autoconnect: false })
+assert.equal(probe.state, probe.store.getState())
+assert.ok(!('watch' in probe.store) && !('version' in probe.store))
+const seen = []
+probe.store.subscribe(
+  (s) => s.control.host_id,
+  (v, old) => seen.push([v, old]),
+)
+probe.store.setState({ mobile_keyboard_open: true })
+assert.deepEqual(seen, [])
+const before = probe.state
+probe.store.setState({ session_id: 'h', control: { ...before.control, host_id: 'h' } })
+assert.deepEqual(seen, [['h', null]])
+assert.notEqual(probe.state, before)
+assert.equal(probe.state.video, before.video, 'untouched slices keep their identity')
+assert.equal(pkg.selectControlling(probe.state), true)
+assert.equal(probe.controlling, true)
+
+// Emitter: on() returns the unsubscribe
+const e = new pkg.Emitter()
+const got = []
+const off = e.on('x', (v) => got.push(v))
+e.emit('x', 1)
+off()
+e.emit('x', 2)
+assert.deepEqual(got, [1])
+
+// NekoClient: constructs without a connection, setUrl normalises the server URL, and the
+// transport seam: WebRTC by default (its data channel as the input), or the one given
+const client = new pkg.NekoClient({ autologin: false, autoconnect: false })
+assert.equal(client.state.connection.status, 'disconnected')
+client.setUrl('http://neko.test/some/path/')
+assert.equal(client.api.url, 'http://neko.test/some/path')
+assert.ok(client.transport instanceof pkg.WebRTCTransport)
+assert.equal(client.transport.kind, 'webrtc')
+assert.ok(client.input instanceof pkg.DataChannelInput)
+const noop = () => {}
+const custom = {
+  kind: 'hls',
+  element: null,
+  connect: async () => {},
+  close: noop,
+  attach: () => noop,
+  setPlaying: async () => {},
+  setVolume: noop,
+  setMuted: noop,
+  on: () => noop,
+}
+const other = new pkg.NekoClient({ transport: custom })
+assert.equal(other.transport, custom)
+assert.ok(other.input instanceof pkg.WebSocketInput, 'no input channel on the transport: the websocket')
+for (const m of ['move', 'scroll', 'button', 'key', 'touch']) assert.equal(typeof other.input[m], 'function', m)
+assert.throws(() => other.shareMedia({}), /transport cannot send media/)
+
+// a stream-only consumer: the transport's element in a box, no overlay (no input, no cursor), and
+// input from a device of its own straight through client.input
+const box = { children: [], append: (el) => box.children.push(el) }
+custom.attach = (el) => (el.append('media'), () => el.children.pop())
+const detach = other.transport.attach(box)
+assert.deepEqual(box.children, ['media'])
+detach()
+assert.deepEqual(box.children, [])
+other.input.key(0xff0d, true) // a gamepad mapped to Return, say; dropped here: no socket
+
+// build.mjs rewrote the `.ts` specifiers the declarations inherit from the sources
+for (const f of readdirSync(dist, { recursive: true }).filter((f) => f.endsWith('.d.ts'))) {
+  const src = readFileSync(join(dist, f), 'utf8')
+  assert.ok(!/from '[^']*\.ts'/.test(src) && !/import\("[^"]*\.ts"\)/.test(src), `${f} still imports a .ts path`)
+}
+
+console.log('dist ok')

client/core/package-lock.json

@@ -0,0 +1,62 @@
+{
+  "name": "@m1k1o/neko",
+  "version": "3.0.0",
+  "lockfileVersion": 3,
+  "requires": true,
+  "packages": {
+    "": {
+      "name": "@m1k1o/neko",
+      "version": "3.0.0",
+      "license": "Apache-2.0",
+      "dependencies": {
+        "zustand": "^5.0.15"
+      },
+      "devDependencies": {
+        "typescript": "~5.8.0"
+      }
+    },
+    "node_modules/typescript": {
+      "version": "5.8.3",
+      "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.8.3.tgz",
+      "integrity": "sha512-p1diW6TqL9L07nNxvRMM7hMMw4c5XOo/1ibL4aAIGmSAt9slTE1Xgw5KWuof2uTOvCg9BY7ZRi+GaF+7sfgPeQ==",
+      "dev": true,
+      "license": "Apache-2.0",
+      "bin": {
+        "tsc": "bin/tsc",
+        "tsserver": "bin/tsserver"
+      },
+      "engines": {
+        "node": ">=14.17"
+      }
+    },
+    "node_modules/zustand": {
+      "version": "5.0.15",
+      "resolved": "https://registry.npmjs.org/zustand/-/zustand-5.0.15.tgz",
+      "integrity": "sha512-MpSEjRiBkA9crSYeOUH32rJC7SVqAbm0Fqcqge/bUi2PPoLcBWKOsG+C8mevmpr8TwXHBVkChbbJiyvkE+i/3A==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=12.20.0"
+      },
+      "peerDependencies": {
+        "@types/react": ">=18.0.0",
+        "immer": ">=9.0.6",
+        "react": ">=18.0.0",
+        "use-sync-external-store": ">=1.2.0"
+      },
+      "peerDependenciesMeta": {
+        "@types/react": {
+          "optional": true
+        },
+        "immer": {
+          "optional": true
+        },
+        "react": {
+          "optional": true
+        },
+        "use-sync-external-store": {
+          "optional": true
+        }
+      }
+    }
+  }
+}

client/core/package.json

@@ -0,0 +1,39 @@
+{
+  "name": "@m1k1o/neko",
+  "version": "3.0.0",
+  "description": "Framework-free client library for the neko v3 API: REST, websocket events, WebRTC with the binary input channel, and the input overlay.",
+  "license": "Apache-2.0",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/m1k1o/neko.git",
+    "directory": "client/core"
+  },
+  "homepage": "https://neko.m1k1o.net/",
+  "type": "module",
+  "sideEffects": false,
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "default": "./dist/index.js"
+    }
+  },
+  "types": "./dist/index.d.ts",
+  "files": [
+    "dist",
+    "README.md"
+  ],
+  "scripts": {
+    "build": "tsc -p tsconfig.json && node build.mjs",
+    "test": "npm --prefix .. test -- core/src",
+    "check": "npm run build && npm test && node dist.check.mjs"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "devDependencies": {
+    "typescript": "~5.8.0"
+  },
+  "dependencies": {
+    "zustand": "^5.0.15"
+  }
+}

client/core/src/api.ts

@@ -0,0 +1,65 @@
+// Thin fetch wrapper over the v3 REST API (server/openapi.yaml). The few response shapes the
+// client reads are declared in types.ts next to the other wire types.
+
+export class ApiError extends Error {
+  readonly status: number
+  constructor(status: number, message: string) {
+    super(message)
+    this.status = status
+  }
+}
+
+export class NekoApi {
+  url = location.href.replace(/[?#].*$/, '').replace(/\/+$/, '')
+  token = ''
+
+  private headers(json: boolean): Record<string, string> {
+    const h: Record<string, string> = {}
+    if (json) h['Content-Type'] = 'application/json'
+    if (this.token) h['Authorization'] = 'Bearer ' + this.token
+    return h
+  }
+
+  async req<T = void>(method: string, path: string, body?: unknown, init?: RequestInit): Promise<T> {
+    const res = await fetch(this.url + '/api' + path, {
+      ...init,
+      method,
+      credentials: 'include',
+      headers: this.headers(body !== undefined),
+      body: body === undefined ? undefined : JSON.stringify(body),
+    })
+    if (!res.ok) {
+      const err = await res.json().catch(() => null)
+      throw new ApiError(res.status, err?.message || res.statusText)
+    }
+    const text = await res.text()
+    return (text ? JSON.parse(text) : undefined) as T
+  }
+
+  async blob(path: string): Promise<Blob> {
+    const res = await fetch(this.url + '/api' + path, { credentials: 'include', headers: this.headers(false) })
+    if (!res.ok) throw new ApiError(res.status, res.statusText)
+    return res.blob()
+  }
+
+  // XHR because fetch cannot report upload progress
+  upload(path: string, form: FormData, onProgress?: (p: { loaded: number; total: number }) => void): Promise<void> {
+    return new Promise((resolve, reject) => {
+      const xhr = new XMLHttpRequest()
+      xhr.open('POST', this.url + '/api' + path)
+      xhr.withCredentials = true
+      for (const [k, v] of Object.entries(this.headers(false))) xhr.setRequestHeader(k, v)
+      if (onProgress) xhr.upload.onprogress = (e) => onProgress({ loaded: e.loaded, total: e.total })
+      xhr.onload = () => {
+        if (xhr.status < 300) return resolve()
+        let message = xhr.statusText // empty under HTTP/2
+        try {
+          message = JSON.parse(xhr.responseText).message || message
+        } catch {}
+        reject(new ApiError(xhr.status, message || 'upload failed'))
+      }
+      xhr.onerror = () => reject(new ApiError(0, 'network error'))
+      xhr.send(form)
+    })
+  }
+}

client/core/src/client.test.ts

@@ -0,0 +1,634 @@
+// Unit tests of the connection state machine (with the real WebRTC transport): `npm test` (vitest).
+// The browser is faked just far enough for the core to load and connect (test/browser.ts), and the
+// clock is vitest's: `tick(ms)` fires the timers that fall due, in order, so nothing waits.
+import { test, vi } from 'vitest'
+import assert from 'node:assert/strict'
+import {
+  tick,
+  settle,
+  reset,
+  sent,
+  sentPayloads,
+  sockets,
+  fetches,
+  net,
+  storage,
+  FakePeer,
+  FakeElement,
+  OFFER,
+  profile,
+  session,
+  settings,
+  init,
+  receive,
+} from './test/browser.ts'
+const { NekoClient, ApiError } = await import('./index.ts')
+type Client = InstanceType<typeof NekoClient>
+type Options = ConstructorParameters<typeof NekoClient>[0]
+
+const setup = (opts?: Options) => {
+  reset()
+  const client = new NekoClient(opts)
+  const closed: (Error | undefined)[] = [] // every connection.closed, with its error
+  client.events.on('connection.closed', (e) => closed.push(e))
+  return { client, closed }
+}
+// a client through the handshake, with `sent` cleared
+const connected = async (opts?: Options) => {
+  const r = setup(opts)
+  r.client.store.setState({ authenticated: true })
+  r.client.connect()
+  receive('system/init', init())
+  receive('signal/provide', { sdp: OFFER })
+  await settle()
+  FakePeer.all[0].transition('connected')
+  await settle()
+  assert.equal(r.client.state.connection.status, 'connected')
+  sent.length = 0
+  return r
+}
+const messages = (closed: (Error | undefined)[]) => closed.map((e) => e?.message)
+
+test('handshake: init -> signal/request -> provide -> signal/answer -> peer connected', async () => {
+  const { client } = setup()
+  client.setUrl('http://neko.test/') // as the GUI does at boot
+  assert.throws(() => client.connect(), /not authenticated/)
+  client.store.setState({ authenticated: true, connection: { ...client.state.connection, token: 'tok' } })
+  const statuses: string[] = []
+  client.events.on('connection.status', (s) => statuses.push(s))
+  client.connect()
+  client.connect() // already wanted: not a second socket
+  assert.equal(sockets.length, 1)
+  assert.equal(sockets[0].url, 'ws://neko.test/api/ws?token=tok')
+  assert.equal(client.state.connection.status, 'connecting')
+
+  receive('system/init', init())
+  assert.deepEqual(sent, ['signal/request'])
+  assert.equal(client.state.session_id, 's1')
+  assert.deepEqual(client.state.screen.size, { width: 1920, height: 1080, rate: 60 })
+  assert.deepEqual(Object.keys(client.state.sessions), ['s1', 's2'])
+  assert.equal(client.session?.profile.name, 'Alice')
+  assert.equal(client.state.control.host_id, 's2')
+  assert.equal(client.state.control.touch, true)
+  assert.equal(client.implicitControl, true)
+
+  receive('signal/candidate', { candidate: 'early' }) // trickled before the offer: held back
+  receive('signal/provide', { sdp: OFFER, iceservers: [] })
+  await settle()
+  assert.deepEqual(sent, ['signal/request', 'signal/answer'])
+  const pc = FakePeer.all[0]
+  assert.deepEqual(pc.remoteDescription, { type: 'offer', sdp: OFFER })
+  assert.deepEqual(pc.candidates, [{ candidate: 'early' }])
+  receive('signal/candidate', { candidate: 'late' })
+  assert.deepEqual(pc.candidates, [{ candidate: 'early' }, { candidate: 'late' }])
+  assert.equal(client.state.connection.status, 'connecting', 'not connected before the media path is')
+
+  pc.transition('connected')
+  await settle()
+  assert.equal(client.state.connection.status, 'connected')
+  assert.equal(client.connected, true)
+  assert.deepEqual(statuses, ['connecting', 'connected'])
+})
+
+test('signal/close: the peer is dropped and a new one requested after the backoff', async () => {
+  const { client } = await connected()
+  const pc = FakePeer.all[0]
+  const stopMic = client.shareMedia({ getTracks: () => [{ kind: 'audio' }] } as unknown as MediaStream)
+  receive('signal/close')
+  assert.ok(pc.closed)
+  assert.equal(client.state.connection.status, 'connecting')
+  receive('signal/close') // no peer any more: nothing to drop, no second request
+  await tick(1499)
+  assert.deepEqual(sent, [], 'waits RECONNECT_BACKOFF_MS before asking again')
+  await tick(1)
+  assert.deepEqual(sent, ['signal/request'])
+  receive('signal/provide', { sdp: OFFER })
+  await settle()
+  assert.equal(FakePeer.all.length, 2)
+  assert.deepEqual(sent, ['signal/request', 'signal/answer'])
+  assert.doesNotThrow(stopMic, 'the media shared on the dropped peer is gone with it')
+  FakePeer.all[1].transition('connected')
+  await settle()
+  assert.equal(client.state.connection.status, 'connected')
+})
+
+test('peer failed: re-requested with growing backoff, given up after RECONNECT_MAX', async () => {
+  const { client, closed } = await connected()
+  FakePeer.all[0].transition('disconnected') // ICE may still recover
+  await settle()
+  assert.equal(client.state.connection.status, 'connecting')
+  assert.equal(FakePeer.all[0].closed, false)
+  for (let n = 1; n <= 10; n++) {
+    sent.length = 0
+    FakePeer.all.at(-1)!.transition('failed')
+    assert.ok(FakePeer.all.at(-1)!.closed)
+    const backoff = 1500 * Math.min(n, 4)
+    await tick(backoff - 1)
+    assert.deepEqual(sent, [], `failure ${n}: waits ${backoff} ms`)
+    await tick(1)
+    assert.deepEqual(sent, ['signal/request'], `failure ${n}`)
+    receive('signal/provide', { sdp: OFFER })
+    await settle()
+  }
+  assert.deepEqual(closed, [])
+  FakePeer.all.at(-1)!.transition('failed')
+  await settle()
+  assert.deepEqual(messages(closed), ['video connection failed (WebRTC)'])
+  assert.equal(client.state.connection.status, 'disconnected')
+  assert.equal(sockets[0].code, 1000)
+  await tick(60_000)
+  assert.equal(sockets.length, 1, 'no reconnect after giving up')
+})
+
+test('peer connected: the failure count starts over', async () => {
+  await connected()
+  for (const backoff of [1500, 3000]) {
+    FakePeer.all.at(-1)!.transition('failed')
+    await tick(backoff)
+    receive('signal/provide', { sdp: OFFER })
+    await settle()
+  }
+  FakePeer.all.at(-1)!.transition('connected')
+  await settle()
+  sent.length = 0
+  FakePeer.all.at(-1)!.transition('failed')
+  await tick(1500)
+  assert.deepEqual(sent, ['signal/request'])
+})
+
+test('no offer after OFFER_TIMEOUT_MS: shown without video; a dead socket is the stale check’s business', async () => {
+  const { client } = await connected()
+  FakePeer.all[0].transition('failed')
+  await tick(1500) // -> signal/request
+  assert.deepEqual(sent, ['signal/request'])
+  await tick(7999)
+  assert.equal(client.state.connection.status, 'connecting')
+  await tick(1) // OFFER_TIMEOUT_MS: the server will not stream (can_watch is off)
+  assert.equal(client.state.connection.status, 'connected')
+  assert.equal(FakePeer.all.length, 1)
+  await tick(25_000) // and the silent socket is still replaced by the stale check
+  assert.equal(client.state.connection.status, 'connecting')
+  assert.ok(sockets[0].closed)
+})
+
+test('stale socket: silent for STALE_MS it is replaced; a message keeps it', async () => {
+  const { client } = await connected()
+  await tick(20_000)
+  receive('system/heartbeat')
+  await tick(20_000)
+  assert.equal(sockets[0].closed, false, '20 s since the last message is not stale')
+  assert.equal(sockets.length, 1)
+  await tick(10_000) // 30 s since
+  const old = sockets[0]
+  assert.ok(old.closed)
+  assert.equal(FakePeer.all[0].closed, false, 'the stream outlives the socket')
+  assert.equal(client.state.connection.status, 'connecting')
+  assert.equal(fetches.at(-1)?.url, 'http://neko.test/api/whoami')
+  await tick(1500)
+  assert.equal(sockets.length, 2)
+  receive('system/init', init()) // the handshake starts over on the new socket, with a new peer
+  assert.deepEqual(sent, ['client/heartbeat', 'signal/request'])
+  assert.ok(FakePeer.all[0].closed)
+  old.onclose!() // the browser reports the closure of the replaced socket: not a second loss
+  await tick(20_000)
+  assert.deepEqual([sockets.length, fetches.length], [2, 1])
+})
+
+test('socket closed: reconnects after the backoff once the session is confirmed', async () => {
+  const { client } = await connected()
+  sockets[0].onclose!()
+  assert.equal(client.state.connection.status, 'connecting')
+  assert.equal(FakePeer.all[0].closed, false)
+  await settle()
+  assert.equal(fetches.at(-1)?.url, 'http://neko.test/api/whoami')
+  await tick(1499)
+  assert.equal(sockets.length, 1)
+  await tick(1)
+  assert.equal(sockets.length, 2)
+  sockets[1].onclose!() // a second loss in a row waits longer
+  await tick(2999)
+  assert.equal(sockets.length, 2)
+  await tick(1)
+  assert.equal(sockets.length, 3)
+  net.respond = () => {
+    throw new TypeError('network') // a lookup that fails for any other reason is still worth a retry
+  }
+  sockets[2].onclose!()
+  await tick(4500)
+  assert.equal(sockets.length, 4)
+  net.respond = () => ({ status: 200 })
+  receive('system/init', init())
+  receive('signal/provide', { sdp: OFFER })
+  await settle()
+  FakePeer.all.at(-1)!.transition('connected') // the attempt count starts over
+  await settle()
+  sockets[3].onclose!()
+  await tick(1500)
+  assert.equal(sockets.length, 5)
+})
+
+test('socket lost: the stream outlives the socket; the reconnect replaces its peer; a final close empties the element', async () => {
+  const { client, closed } = await connected()
+  const box = new FakeElement('div')
+  client.transport.attach(box as any)
+  const video = box.children[0]
+  const stream = { id: 'stream' }
+  FakePeer.all[0].ontrack!({ track: { kind: 'video' }, streams: [stream], receiver: {} })
+  await client.play()
+  video.dispatch('canplaythrough')
+  assert.deepEqual([client.state.video.playable, client.state.video.playing], [true, true])
+
+  sockets[0].onclose!()
+  assert.equal(client.state.connection.status, 'connecting')
+  assert.equal(FakePeer.all[0].closed, false, 'the peer keeps streaming while the socket reconnects')
+  assert.deepEqual([video.srcObject, video.paused], [stream, false], 'the picture stays')
+  FakePeer.all[0].transition('connected') // a report from the stream while the socket is down counts for nothing
+  await settle()
+  assert.equal(client.state.connection.status, 'connecting')
+
+  await tick(1500)
+  receive('system/init', init(), sockets[1])
+  assert.ok(FakePeer.all[0].closed, 'the reconnect starts the transport over: the old peer goes')
+  assert.deepEqual(sent, ['signal/request'])
+  assert.equal(video.srcObject, stream, 'the last picture until the new stream shows')
+  receive('signal/provide', { sdp: OFFER })
+  await settle()
+  assert.equal(FakePeer.all.filter((p) => !p.closed).length, 1, 'exactly one live peer')
+  FakePeer.all[1].ontrack!({ track: { kind: 'video' }, streams: [{ id: 'new' }], receiver: {} })
+  await settle()
+  assert.deepEqual([video.srcObject, video.plays], [{ id: 'new' }, 2], 'kept playing')
+  FakePeer.all[1].transition('connected')
+  await settle()
+  assert.equal(client.state.connection.status, 'connected')
+
+  receive('system/disconnect', { message: 'kicked' }) // a final close: black behind the Connect screen
+  assert.deepEqual(messages(closed), ['kicked'])
+  assert.ok(FakePeer.all[1].closed)
+  assert.deepEqual([video.srcObject, video.paused, client.state.video.playable], [null, true, false])
+})
+
+test('socket lost while the peer fails: it asks again by itself, which waits for the socket; neither that nor its verdict moves the status', async () => {
+  const { client } = await connected()
+  sockets[0].onclose!()
+  FakePeer.all[0].transition('failed')
+  await tick(1500) // the transport's own retry: signal/request goes to session.send, which has no socket
+  assert.deepEqual(sent, [])
+  assert.equal(sockets.length, 2, 'the socket reconnect is on its own clock')
+  await tick(8000) // no offer: the transport calls it unavailable, which would otherwise read as connected
+  assert.equal(client.state.connection.status, 'connecting')
+  receive('system/init', init(), sockets[1])
+  assert.deepEqual(sent, ['signal/request'], 'one request, from the new session')
+  receive('signal/provide', { sdp: OFFER })
+  await settle()
+  assert.equal(FakePeer.all.length, 2)
+  FakePeer.all[1].transition('connected')
+  await settle()
+  assert.equal(client.state.connection.status, 'connected')
+  sent.length = 0
+  FakePeer.all[1].transition('failed') // the count started over with the new session
+  await tick(1499)
+  assert.deepEqual(sent, [])
+  await tick(1)
+  assert.deepEqual(sent, ['signal/request'])
+})
+
+test('socket closed: a deleted session (401) is not retried', async () => {
+  const { client, closed } = await connected()
+  net.respond = () => ({ status: 401, body: { message: 'session not found' } })
+  sockets[0].onclose!()
+  await settle()
+  assert.deepEqual(messages(closed), ['session expired'])
+  assert.equal(client.state.connection.status, 'disconnected')
+  await tick(60_000)
+  assert.equal(sockets.length, 1)
+})
+
+test('socket closed: given up after RECONNECT_MAX attempts', async () => {
+  const { client, closed } = await connected()
+  for (let n = 1; n <= 10; n++) {
+    sockets.at(-1)!.onclose!()
+    await tick(6000)
+    assert.equal(sockets.length, n + 1, `attempt ${n}`)
+  }
+  sockets.at(-1)!.onclose!()
+  await tick(60_000)
+  assert.equal(sockets.length, 11)
+  assert.deepEqual(messages(closed), ['connection lost'])
+  assert.equal(client.state.connection.status, 'disconnected')
+})
+
+test('disconnect: closes socket and peer, clears the room state, cancels every timer', async () => {
+  const { client, closed } = await connected()
+  client.disconnect()
+  assert.equal(sockets[0].code, 1000, 'a normal closure, or the server keeps the session for 5 s')
+  assert.ok(FakePeer.all[0].closed)
+  assert.equal(client.state.connection.status, 'disconnected')
+  assert.equal(client.state.session_id, null)
+  assert.deepEqual(client.state.sessions, {})
+  assert.equal(client.state.control.host_id, null)
+  assert.deepEqual(closed, [undefined])
+  assert.equal(vi.getTimerCount(), 0)
+  sockets[0].onclose!() // the server's side of the closure
+  await tick(60_000)
+  assert.deepEqual([sockets.length, fetches.length], [1, 0])
+
+  client.connect() // and connecting again works
+  assert.equal(sockets.length, 2)
+  sockets[1].onclose!()
+  await settle() // the session lookup came back: a reconnect is scheduled
+  client.disconnect()
+  await tick(60_000)
+  assert.equal(sockets.length, 2, 'a disconnect cancels the pending reconnect')
+
+  client.connect()
+  sockets[2].onclose!()
+  client.disconnect() // while the lookup is still running
+  await tick(60_000)
+  assert.equal(sockets.length, 3)
+
+  client.connect()
+  sockets[3].onclose!()
+  client.disconnect()
+  client.connect() // reconnected by hand while the old lookup is still running
+  assert.equal(sockets.length, 5)
+  await tick(10_000) // the old lookup comes back and would reconnect after 1.5 s
+  assert.equal(sockets.length, 5, 'the old lookup must not replace the new connection')
+  assert.equal(sockets[4].closed, false)
+  assert.deepEqual(closed, [undefined, undefined, undefined, undefined])
+})
+
+test('system/heartbeat is answered with client/heartbeat', async () => {
+  await connected()
+  receive('system/heartbeat')
+  assert.deepEqual(sent, ['client/heartbeat'])
+})
+
+test('codec mismatch: the offer is refused naming the codec, no retry', async () => {
+  const { client, closed } = setup()
+  client.store.setState({ authenticated: true })
+  client.connect()
+  receive('system/init', init())
+  FakePeer.answerSdp = 'v=0\r\nm=audio 9 RTP/AVP 0\r\nm=video 0 RTP/AVP 96\r\n' // the browser rejected the video section
+  receive('signal/provide', { sdp: OFFER.replace('VP8', 'H264') })
+  await settle()
+  assert.deepEqual(sent, ['signal/request'], 'no answer')
+  assert.deepEqual(messages(closed), ['this browser cannot play H264 video'])
+  assert.equal(client.state.connection.status, 'disconnected')
+  assert.equal(sockets[0].code, 1000)
+  await tick(60_000)
+  assert.deepEqual([sockets.length, FakePeer.all.length], [1, 1])
+})
+
+test('system/disconnect (the session opened in another tab): closed with the message, no reconnect', async () => {
+  const { client, closed } = await connected()
+  receive('system/disconnect', { message: 'connection replaced' })
+  assert.deepEqual(messages(closed), ['connection replaced'])
+  assert.equal(client.state.connection.status, 'disconnected')
+  assert.equal(client.state.session_id, null)
+  assert.equal(sockets[0].code, 1000)
+  assert.ok(FakePeer.all[0].closed)
+  sockets[0].onclose!() // the server closes the socket right after
+  await tick(60_000)
+  assert.deepEqual([sockets.length, fetches.length], [1, 0])
+})
+
+test('session, control, screen, clipboard and settings events update the state', async () => {
+  const { client } = await connected()
+  const got: string[] = []
+  const names = [
+    'session.created',
+    'session.updated',
+    'session.deleted',
+    'room.control.host',
+    'room.screen.updated',
+    'room.clipboard.updated',
+    'room.settings.updated',
+    'room.broadcast.status',
+    'receive.unicast',
+    'receive.broadcast',
+    'message',
+  ] as const
+  for (const n of names) client.events.on(n, (...a: unknown[]) => got.push(n + ' ' + JSON.stringify(a)))
+
+  receive('session/created', session('s3', { name: 'Carol' }))
+  assert.equal(client.state.sessions.s3.profile.name, 'Carol')
+  receive('session/profile', { id: 's3', ...profile({ name: 'Caroline' }) })
+  assert.equal(client.state.sessions.s3.profile.name, 'Caroline')
+  receive('session/state', { id: 's3', is_connected: true, is_watching: true })
+  assert.equal(client.state.sessions.s3.state.is_watching, true)
+  receive('session/profile', { id: 'nobody', ...profile() }) // unknown session: ignored
+  receive('session/state', { id: 'nobody', is_connected: true, is_watching: true })
+  assert.equal('nobody' in client.state.sessions, false)
+  receive('session/deleted', { id: 's3' })
+  assert.equal('s3' in client.state.sessions, false)
+  receive('session/profile', { id: 's1', ...profile({ is_admin: true }) }) // our own profile
+  assert.equal(client.isAdmin, true)
+
+  assert.equal(client.controlling, false)
+  receive('control/host', { id: 's1', has_host: true, host_id: 's1' })
+  assert.equal(client.state.control.host_id, 's1')
+  assert.equal(client.controlling, true)
+  await settle()
+  assert.deepEqual(sent, ['keyboard/map'], 'the keyboard layout is sent on taking control')
+  receive('control/host', { id: 's1', has_host: false })
+  assert.equal(client.state.control.host_id, null)
+  receive('control/request', { id: 's2' })
+
+  receive('screen/updated', { id: 's2', width: 1280, height: 720, rate: 30 })
+  assert.deepEqual(client.state.screen.size, { width: 1280, height: 720, rate: 30 })
+  receive('clipboard/updated', { text: 'hi' })
+  assert.deepEqual(client.state.control.clipboard, { text: 'hi' })
+  receive('system/settings', { id: 's2', ...settings({ private_mode: true }) })
+  assert.equal(client.state.settings.private_mode, true)
+  assert.equal(client.privateModeEnabled, false, 'admins see through private mode')
+  receive('system/admin', {
+    screen_sizes_list: [
+      { width: 1280, height: 720, rate: 30 },
+      { width: 1920, height: 1080, rate: 30 },
+      { width: 1920, height: 1080, rate: 60 },
+    ],
+    broadcast_status: { is_active: true, url: 'rtmp://x' },
+  })
+  assert.deepEqual(
+    client.state.screen.configurations.map((c) => `${c.width}x${c.height}@${c.rate}`),
+    ['1920x1080@60', '1920x1080@30', '1280x720@30'],
+  )
+  receive('send/unicast', { sender: 's2', subject: 'hey', body: 1 })
+  receive('send/broadcast', { sender: 's2', subject: 'all', body: [2] })
+  receive('chat/message', { text: 'x' }) // a plugin event
+  assert.deepEqual(got, [
+    'session.created ["s3"]',
+    'session.updated ["s3"]',
+    'session.updated ["s3"]',
+    'session.deleted ["s3"]',
+    'session.updated ["s1"]',
+    'room.control.host [true,"s1","s1"]',
+    'room.control.host [false,null,"s1"]',
+    'room.screen.updated [1280,720,30,"s2"]',
+    'room.clipboard.updated ["hi"]',
+    `room.settings.updated [${JSON.stringify(settings({ private_mode: true }))},"s2"]`,
+    'room.broadcast.status [true,"rtmp://x"]',
+    'receive.unicast ["s2","hey",1]',
+    'receive.broadcast ["s2","all",[2]]',
+    'message ["chat/message",{"text":"x"}]',
+  ])
+})
+
+test('preparePaste: the side that copied most recently wins', async () => {
+  const { client } = await connected()
+  receive('clipboard/updated', { text: 'remote' })
+  for (const text of ['remote', '']) {
+    const p = client.preparePaste(text) // the remote clipboard's own text, or nothing: nothing to do
+    await settle()
+    assert.deepEqual(sent, [], `"${text}" is not sent`)
+    await p
+  }
+
+  let settled = false
+  const p = client.preparePaste('local').then(() => (settled = true)) // newer than anything the remote reported
+  await settle()
+  assert.deepEqual([sent, settled], [['clipboard/set'], false])
+  receive('clipboard/updated', { text: 'local' }) // resolves once the server confirms
+  await p
+
+  await tick(1)
+  receive('clipboard/updated', { text: 'remote2' }) // the remote copied since: the local text is older
+  sent.length = 0
+  const older = client.preparePaste('local')
+  await settle()
+  assert.deepEqual(sent, [])
+  await older
+  const same = client.preparePaste('remote2') // on https the remote text is in the local clipboard too: nothing to send
+  await settle()
+  assert.deepEqual(sent, [])
+  await same
+
+  settled = false
+  client.preparePaste('other').then(() => (settled = true)) // nothing changed remotely since that look
+  await tick(499)
+  assert.deepEqual([sent, settled], [['clipboard/set'], false])
+  await tick(1) // no confirmation: the wait ends after 500 ms
+  assert.equal(settled, true)
+})
+
+test('login, authenticate, logout: the token and the session', async () => {
+  const { client } = setup({ autologin: true })
+  client.setUrl('http://neko.test/')
+  net.respond = (url, init) =>
+    url.endsWith('/api/login') && JSON.parse(init.body as string).password === 'pw'
+      ? { status: 200, body: { ...session('s1'), token: 'tok' } }
+      : { status: 401, body: { message: 'invalid password' } }
+  await assert.rejects(
+    client.login('admin', 'wrong'),
+    (e: unknown) => e instanceof ApiError && e.status === 401 && e.message === 'invalid password',
+  )
+  assert.equal(client.state.authenticated, false)
+  await client.login('admin', 'pw')
+  assert.equal(fetches.at(-1)?.url, 'http://neko.test/api/login')
+  assert.equal(client.state.authenticated, true)
+  assert.equal(client.api.token, 'tok')
+  assert.equal(storage.get('neko_session'), 'tok')
+
+  net.respond = () => ({ status: 200, body: session('s1') })
+  await client.authenticate() // whoami carries the token
+  assert.equal(fetches.at(-1)?.url, 'http://neko.test/api/whoami')
+  assert.equal((fetches.at(-1)?.init.headers as Record<string, string>).Authorization, 'Bearer tok')
+  client.connect()
+  assert.equal(sockets[0].url, 'ws://neko.test/api/ws?token=tok')
+
+  await client.logout()
+  assert.equal(fetches.at(-1)?.url, 'http://neko.test/api/logout')
+  assert.equal(client.state.authenticated, false)
+  assert.equal(client.api.token, '')
+  assert.equal(storage.has('neko_session'), false)
+  assert.equal(sockets[0].code, 1000, 'logging out closes the connection')
+  assert.equal(client.state.connection.status, 'disconnected')
+})
+
+test('authenticate: resumes the saved session, or fails with the ApiError', async () => {
+  const { client } = setup({ autologin: true, autoconnect: true })
+  storage.set('neko_session', 'saved')
+  client.setUrl('http://neko.test/some/path/')
+  await settle()
+  assert.equal(fetches[0].url, 'http://neko.test/some/path/api/whoami')
+  assert.equal((fetches[0].init.headers as Record<string, string>).Authorization, 'Bearer saved')
+  assert.equal(client.state.authenticated, true)
+  assert.equal(sockets[0]?.url, 'ws://neko.test/some/path/api/ws?token=saved', 'autoconnect')
+
+  const expired = setup({ autologin: true }).client
+  storage.set('neko_session', 'expired')
+  net.respond = () => ({ status: 401, body: { message: 'unauthorized' } })
+  await assert.rejects(expired.authenticate(), (e: unknown) => e instanceof ApiError && e.status === 401)
+  assert.equal(expired.state.authenticated, false)
+  assert.throws(() => expired.connect(), /not authenticated/)
+})
+
+test('send: dropped while the socket is not open', async () => {
+  const { client } = setup()
+  client.send('chat/message', { text: 'x' }) // no socket at all
+  client.store.setState({ authenticated: true })
+  client.connect()
+  sockets[0].readyState = 0 // still connecting
+  client.send('chat/message', { text: 'x' })
+  assert.deepEqual(sent, [])
+  sockets[0].readyState = 1
+  client.send('chat/message', { text: 'x' })
+  assert.deepEqual(sent, ['chat/message'])
+  client.disconnect()
+  client.send('chat/message', { text: 'x' })
+  assert.deepEqual(sent, ['chat/message'])
+})
+
+test('subscriptions: the keyboard map goes out when control or the layout changes, once per change, never for the rest', async () => {
+  const { client } = await connected() // Bob (s2) is host
+  sentPayloads.length = 0
+  const statuses: string[] = []
+  client.events.on('connection.status', (s) => statuses.push(s))
+  client.setKeyboard('de')
+  assert.deepEqual(sent, [], 'not host: nothing')
+  receive('control/host', { id: 's1', has_host: true, host_id: 's1' })
+  assert.deepEqual(sent, ['keyboard/map'])
+  assert.deepEqual(sentPayloads.at(-1)?.payload, { layout: 'de', variant: '' })
+  client.setKeyboard('de') // the same layout again: not resent
+  receive('screen/updated', { id: 's2', width: 1280, height: 720, rate: 30 }) // unrelated changes
+  receive('clipboard/updated', { text: 'x' })
+  receive('session/profile', { id: 's1', ...profile({ name: 'Alicia' }) })
+  client.lock()
+  assert.deepEqual(sent, ['keyboard/map'])
+  client.setKeyboard('de', 'nodeadkeys')
+  assert.deepEqual(
+    sentPayloads.map((m) => m.payload),
+    [
+      { layout: 'de', variant: '' },
+      { layout: 'de', variant: 'nodeadkeys' },
+    ],
+  )
+  receive('control/host', { id: 's1', has_host: false })
+  assert.equal(sent.length, 2, 'losing control sends nothing')
+  assert.deepEqual(statuses, [], 'none of it is a status change')
+
+  // what a member list selects: the sessions object is replaced by a session event, by nothing else
+  let lists = 0
+  client.store.subscribe(
+    (s) => s.sessions,
+    () => lists++,
+  )
+  receive('session/created', session('s3', { name: 'Carol' }))
+  receive('session/profile', { id: 's3', ...profile({ name: 'Caroline' }) })
+  receive('session/state', { id: 's3', is_connected: false, is_watching: false })
+  receive('session/deleted', { id: 's3' })
+  assert.equal(lists, 4)
+  receive('control/host', { id: 's2', has_host: true, host_id: 's2' })
+  receive('screen/updated', { id: 's2', width: 1920, height: 1080, rate: 60 })
+  client.unlock()
+  assert.equal(lists, 4)
+
+  // one message that sets several fields at once (system/init on a reconnect that keeps us host:
+  // session_id and host_id together) is observed once, with the final state
+  sockets[0].onclose!()
+  await tick(1500)
+  sent.length = 0
+  receive('system/init', { ...init(), control_host: { has_host: true, host_id: 's1' } }, sockets[1])
+  assert.deepEqual(sent.sort(), ['keyboard/map', 'signal/request'])
+  assert.deepEqual(statuses, ['connecting'], 'the lost socket; init itself does not change the status')
+})

client/core/src/client.transport.test.ts

@@ -0,0 +1,334 @@
+// The seam between NekoClient and a StreamTransport, with a fake transport: what the client hands
+// over, how the connection status follows the transport, and the media calls it delegates.
+import { test } from 'vitest'
+import assert from 'node:assert/strict'
+import { tick, settle, reset, sent, sentPayloads, sockets, init, receive, FakeElement } from './test/browser.ts'
+
+const { NekoClient, Emitter, WebSocketInput, mount } = await import('./index.ts')
+type Options = ConstructorParameters<typeof NekoClient>[0]
+type Transport = NonNullable<Exclude<NonNullable<Options>['transport'], () => unknown>>
+type SessionInfo = Parameters<Transport['connect']>[0]
+type TransportState = Parameters<Parameters<Transport['on']>[1]>[0]
+
+class FakeTransport implements Transport {
+  readonly kind = 'ws-mse' as const
+  element = null
+  input?: Transport['input']
+  shareMedia?: Transport['shareMedia']
+  calls: string[] = []
+  session: SessionInfo | null = null
+  startError: Error | null = null
+  readonly events = new Emitter<any>()
+  video = { playable: false, playing: false, volume: 1, muted: false, mutedByAutoplay: false }
+  async connect(session: SessionInfo) {
+    this.calls.push('connect')
+    this.session = session
+    if (this.startError) throw this.startError
+  }
+  close() {
+    this.calls.push('close')
+  }
+  attach(box: HTMLElement) {
+    this.calls.push('attach')
+    const video = new FakeElement('video')
+    ;(box as unknown as FakeElement).append(video)
+    return () => {
+      this.calls.push('detach')
+      video.remove()
+    }
+  }
+  async setPlaying(on: boolean) {
+    this.calls.push(`setPlaying ${on}`)
+  }
+  setVolume(v: number) {
+    this.calls.push(`setVolume ${v}`)
+  }
+  setMuted(on: boolean) {
+    this.calls.push(`setMuted ${on}`)
+  }
+  on = (event: string, cb: (...a: any[]) => void) => this.events.on(event, cb)
+  // what the transport reports
+  state(status: TransportState['status'], video = this.video) {
+    this.events.emit('state', { status, size: { width: 0, height: 0 }, video })
+  }
+  fail(err: Error) {
+    this.events.emit('error', err)
+  }
+}
+
+const setup = (opts: Options = {}) => {
+  reset()
+  const t = new FakeTransport()
+  const client = new NekoClient({ transport: t, ...opts })
+  client.setUrl('http://neko.test/') // closes whatever was there, as always
+  t.calls.length = 0
+  const closed: (Error | undefined)[] = []
+  client.events.on('connection.closed', (e) => closed.push(e))
+  const statuses: string[] = []
+  client.events.on('connection.status', (s) => statuses.push(s))
+  client.store.setState({ authenticated: true, connection: { ...client.state.connection, token: 'tok' } })
+  return { t, client, closed, statuses }
+}
+
+test('connect: the transport gets the session (url, token, init, send, on) and nothing else', async () => {
+  const { t, client } = setup()
+  client.connect()
+  assert.deepEqual(t.calls, [], 'not before the server introduced the session')
+  receive('system/init', init())
+  assert.deepEqual(t.calls, ['connect'])
+  const s = t.session!
+  assert.deepEqual(Object.keys(s).sort(), ['init', 'on', 'send', 'token', 'url'])
+  assert.equal(s.url, 'http://neko.test')
+  assert.equal(s.token, 'tok')
+  assert.deepEqual(s.init, init())
+  s.send('signal/request', { video: {} })
+  assert.deepEqual(sentPayloads.at(-1), { event: 'signal/request', payload: { video: {} } })
+
+  const got: unknown[] = []
+  const messages: string[] = []
+  client.events.on('message', (e) => messages.push(e))
+  const off = s.on('signal/provide', (p) => got.push(p))
+  const before = JSON.stringify(client.state)
+  receive('signal/provide', { sdp: 'x' })
+  receive('signal/video', { video: 'hd' })
+  assert.deepEqual(got, [{ sdp: 'x' }])
+  assert.equal(JSON.stringify(client.state), before, 'signal/* is the transport’s: the client does not act on it')
+  assert.deepEqual(messages, ['signal/provide', 'signal/video'], 'but passes it on like any event it does not handle')
+  receive('chat/message', { text: 'hi' })
+  assert.deepEqual(messages, ['signal/provide', 'signal/video', 'chat/message'])
+  off()
+  receive('signal/provide', { sdp: 'y' })
+  assert.equal(got.length, 1)
+})
+
+test('status: connected only when the socket and the transport are; a transport error closes', async () => {
+  const { t, client, closed, statuses } = setup()
+  client.connect()
+  receive('system/init', init())
+  assert.equal(client.state.connection.status, 'connecting')
+  t.state('connecting')
+  assert.equal(client.state.connection.status, 'connecting')
+  t.state('connected')
+  assert.equal(client.state.connection.status, 'connected')
+  await settle()
+  t.state('connecting') // the media path hiccups
+  assert.equal(client.state.connection.status, 'connecting')
+  await settle()
+  t.state('unavailable') // the server will not stream: the room without video
+  assert.equal(client.state.connection.status, 'connected')
+  await settle()
+
+  sockets[0].onclose!()
+  assert.deepEqual(t.calls, ['connect'], 'the lost socket leaves the transport alone')
+  assert.equal(client.state.connection.status, 'connecting')
+  t.state('connected') // whatever it reports until the socket is back counts for nothing
+  assert.equal(client.state.connection.status, 'connecting')
+  t.state('disconnected')
+  assert.equal(client.state.connection.status, 'connecting')
+  await tick(1500)
+  receive('system/init', init(), sockets[1])
+  assert.deepEqual(t.calls, ['connect', 'connect'], 'the new socket starts the transport over')
+  t.state('connected')
+  await settle()
+  // every change, as it happens (the proxy store used to merge connect() and the first 'connected')
+  assert.deepEqual(statuses, ['connecting', 'connected', 'connecting', 'connected', 'connecting', 'connected'])
+
+  t.fail(new Error('video connection failed (WebRTC)'))
+  assert.deepEqual(
+    closed.map((e) => e?.message),
+    ['video connection failed (WebRTC)'],
+  )
+  assert.equal(client.state.connection.status, 'disconnected')
+  assert.equal(sockets[1].code, 1000)
+  assert.deepEqual(t.calls.at(-1), 'close')
+  t.state('connected') // a late report from a closed transport changes nothing
+  assert.equal(client.state.connection.status, 'disconnected')
+  await tick(60_000)
+  assert.equal(sockets.length, 2, 'no reconnect after an error')
+})
+
+test('a transport that cannot start closes the connection with its error', async () => {
+  const { t, client, closed } = setup()
+  t.startError = new Error('WebCodecs not supported')
+  client.connect()
+  receive('system/init', init())
+  await settle()
+  assert.deepEqual(
+    closed.map((e) => e?.message),
+    ['WebCodecs not supported'],
+  )
+  assert.equal(client.state.connection.status, 'disconnected')
+})
+
+test('socket attempts start over when the transport connects, not when the server declines media', async () => {
+  const { t, client } = setup()
+  client.connect()
+  for (let n = 1; n <= 10; n++) {
+    sockets.at(-1)!.onclose!()
+    await tick(6000)
+    assert.equal(sockets.length, n + 1, `attempt ${n}`)
+  }
+  receive('system/init', init())
+  t.state('unavailable')
+  assert.equal(client.state.connection.status, 'connected')
+  sockets.at(-1)!.onclose!()
+  await tick(60_000)
+  assert.equal(client.state.connection.status, 'disconnected', 'the 11th loss: given up, as without a transport')
+
+  const second = setup()
+  second.client.connect()
+  for (let n = 1; n <= 10; n++) {
+    sockets.at(-1)!.onclose!()
+    await tick(6000)
+  }
+  receive('system/init', init())
+  second.t.state('connected')
+  sockets.at(-1)!.onclose!()
+  await tick(6000)
+  assert.equal(sockets.length, 12, 'a connected transport reset the count')
+})
+
+test('disconnect closes the transport; the overlay channel is the transport’s input or the websocket', async () => {
+  const { t, client } = setup()
+  client.connect()
+  receive('system/init', init())
+  client.disconnect()
+  assert.deepEqual(t.calls, ['connect', 'close'])
+  assert.ok(client.input instanceof WebSocketInput)
+  client.connect()
+  receive('system/init', init())
+  assert.deepEqual(t.calls, ['connect', 'close', 'connect'])
+
+  reset()
+  const own = new FakeTransport()
+  const keys: [number, boolean][] = []
+  own.input = { move() {}, scroll() {}, button() {}, key: (keysym, down) => keys.push([keysym, down]), touch() {} }
+  const c2 = new NekoClient({ transport: () => own })
+  assert.equal(c2.input, own.input, 'the transport’s own channel wins')
+  assert.equal(c2.transport, own, 'a factory is called once')
+  c2.input.key(0xff0d, true)
+  assert.deepEqual(keys, [[0xff0d, true]])
+  assert.deepEqual(sent, [])
+})
+
+test('play, pause, mute, unmute, volume and shareMedia go through the transport; the video state mirrors it', async () => {
+  const { t, client } = setup()
+  await client.play()
+  client.pause()
+  client.mute()
+  client.unmute()
+  client.setVolume(1.5)
+  client.setVolume(-1)
+  client.setVolume(0.3)
+  assert.deepEqual(t.calls, [
+    'setPlaying true',
+    'setPlaying false',
+    'setMuted true',
+    'setMuted false',
+    'setVolume 1',
+    'setVolume 0',
+    'setVolume 0.3',
+  ])
+  const stream = {} as MediaStream
+  assert.throws(() => client.shareMedia(stream), /transport cannot send media/)
+  t.shareMedia = (s) => {
+    t.calls.push(`shareMedia ${s === stream}`)
+    return () => t.calls.push('stop')
+  }
+  client.shareMedia(stream)()
+  assert.deepEqual(t.calls.slice(7), ['shareMedia true', 'stop'])
+
+  t.state('disconnected', { playable: true, playing: true, volume: 0.3, muted: true, mutedByAutoplay: true })
+  assert.deepEqual(client.state.video, {
+    playable: true,
+    playing: true,
+    volume: 0.3,
+    muted: true,
+    mutedByAutoplay: true,
+  })
+  t.state('disconnected', { playable: true, playing: false, volume: 0.3, muted: false, mutedByAutoplay: false })
+  assert.deepEqual(client.state.video, {
+    playable: true,
+    playing: false,
+    volume: 0.3,
+    muted: false,
+    mutedByAutoplay: false,
+  })
+})
+
+test('autoplay: the client starts playback whenever the transport’s stream becomes playable', async () => {
+  const { t, client } = setup({ autoplay: true })
+  client.connect()
+  receive('system/init', init())
+  const video = { playable: false, playing: false, volume: 1, muted: false, mutedByAutoplay: false }
+  t.state('connected', video)
+  assert.deepEqual(t.calls, ['connect'], 'not before a frame can be shown')
+  t.state('connected', { ...video, playable: true })
+  assert.deepEqual(t.calls, ['connect', 'setPlaying true'])
+  t.state('connected', { ...video, playable: true, playing: true }) // still playable: not again
+  t.state('connected', { ...video, playable: true, playing: false }) // paused by the user: not again
+  assert.deepEqual(t.calls, ['connect', 'setPlaying true'])
+  t.state('connected', video) // a new stream is loading (reconnect)...
+  t.state('connected', { ...video, playable: true })
+  assert.deepEqual(t.calls, ['connect', 'setPlaying true', 'setPlaying true'])
+
+  const off = setup()
+  off.t.state('connected', { ...video, playable: true })
+  assert.deepEqual(off.t.calls, [], 'without the option the user presses play')
+})
+
+test('socket lost: playback that was running resumes on the new stream, autoplay or not', async () => {
+  const { t, client } = setup()
+  const video = { playable: true, playing: true, volume: 1, muted: false, mutedByAutoplay: false }
+  const loading = { ...video, playable: false, playing: false }
+  client.connect()
+  receive('system/init', init())
+  t.state('connected', video)
+  sockets[0].onclose!()
+  t.state('disconnected', loading) // the stream died during the outage (the server dropped the peer)
+  await tick(1500)
+  receive('system/init', init(), sockets[1])
+  t.state('connected', loading)
+  t.state('connected', { ...video, playing: false })
+  assert.deepEqual(t.calls, ['connect', 'connect', 'setPlaying true'])
+  t.state('connected', video)
+
+  sockets[1].onclose!() // and again
+  t.state('disconnected', loading)
+  await tick(1500)
+  receive('system/init', init(), sockets[2])
+  t.state('connected', loading)
+  t.state('connected', { ...video, playing: false })
+  assert.deepEqual(t.calls.slice(3), ['connect', 'setPlaying true'], 'the first stream after the loss')
+  t.state('connected', loading) // a replaced peer brings another stream: the transport's business
+  t.state('connected', { ...video, playing: false })
+  assert.deepEqual(t.calls.slice(5), [], 'a later stream does not')
+
+  t.state('connected', video)
+  sockets[2].onclose!() // lost while playing...
+  t.state('disconnected', loading)
+  client.disconnect() // ...and given up on before the stream came back: the next one waits for play (or autoplay)
+  client.connect()
+  receive('system/init', init(), sockets[3])
+  t.state('connected', loading)
+  t.state('connected', { ...video, playing: false })
+  assert.deepEqual(t.calls.slice(5), ['close', 'connect'], 'the one close is the final one')
+})
+
+test('mount(client, el): attaches the transport, then the overlay, into a box in el; the returned function detaches both', () => {
+  const { t, client } = setup()
+  const el = new FakeElement('div')
+  const unmount = mount(client, el as any)
+  assert.deepEqual(t.calls, ['attach'])
+  assert.equal(el.children.length, 1, 'the letterboxed box')
+  assert.deepEqual(
+    el.children[0].children.map((c) => c.tag),
+    ['video', 'div'],
+    'the stream first, the overlay over it',
+  )
+  unmount()
+  assert.deepEqual(t.calls, ['attach', 'detach'])
+  assert.deepEqual(el.children, [])
+  mount(client, el as any) // and again
+  assert.deepEqual(t.calls, ['attach', 'detach', 'attach'])
+})

client/core/src/client.ts

@@ -0,0 +1,580 @@
+// Framework-free neko client for the v3 API: REST (server/openapi.yaml), the /api/ws event
+// protocol, and the stream through a StreamTransport (WebRTC by default, see transport.ts).
+// Written against master's server/pkg/types. The state lives in a zustand store (`client.store`,
+// immutable updates, subscribe with a selector) and is read as `client.state`; one-off happenings
+// arrive on `client.events`. Input devices (the keyboard/mouse Overlay, or anything else) are
+// attached by the consumer and send through `client.input`.
+import { createStore } from 'zustand/vanilla'
+import { subscribeWithSelector } from 'zustand/middleware'
+import { Emitter } from './emitter.ts'
+import { NekoApi, ApiError } from './api.ts'
+import { WebRTCTransport } from './transport/webrtc.ts'
+import { WebSocketInput } from './input/websocket.ts'
+import type { InputChannel, SessionInfo, StreamTransport, TransportState } from './transport.ts'
+import type { State, Settings, ScreenSize, NekoEvents, LoginResponse } from './types.ts'
+
+const RECONNECT_MAX = 10
+const RECONNECT_BACKOFF_MS = 1500
+// a socket that stays silent this long is dead (the server heartbeats every ~10s)
+const STALE_MS = 25_000
+
+export interface NekoClientOptions {
+  // remember the session token in localStorage
+  autologin?: boolean
+  // connect as soon as setUrl() finds a valid session
+  autoconnect?: boolean
+  autoplay?: boolean
+  inputMode?: 'auto' | 'touch' | 'mouse'
+  // how the stream arrives; WebRTC unless given
+  transport?: StreamTransport | (() => StreamTransport)
+}
+
+export interface Pos {
+  x: number
+  y: number
+}
+
+const defaultSettings = (): Settings => ({
+  private_mode: false,
+  locked_logins: false,
+  locked_controls: false,
+  control_protection: false,
+  implicit_hosting: false,
+  inactive_cursors: false,
+  merciful_reconnect: false,
+})
+
+const initialState = (): State => ({
+  authenticated: false,
+  connection: { url: location.href, status: 'disconnected' },
+  video: { playable: false, playing: false, volume: 1, muted: false, mutedByAutoplay: false },
+  control: {
+    host_id: null,
+    locked: false,
+    clipboard: null,
+    scroll: { inverse: true, sensitivity: 0 },
+    keyboard: { layout: 'us', variant: '' },
+    touch: false,
+  },
+  screen: { size: { width: 1280, height: 720, rate: 30 }, configurations: [] },
+  session_id: null,
+  sessions: {},
+  settings: defaultSettings(),
+  mobile_keyboard_open: false,
+})
+
+// the top-level slices that are objects, for patch()
+type Slice = 'connection' | 'video' | 'control' | 'screen'
+
+// selectors over the state: what the client's getters compute, usable with store.subscribe and
+// a UI's selector hook (`useStore(client.store, selectControlling)`)
+export const selectSession = (s: State) => (s.session_id ? (s.sessions[s.session_id] ?? null) : null)
+export const selectControlling = (s: State) => s.control.host_id !== null && s.control.host_id === s.session_id
+export const selectIsAdmin = (s: State) => !!selectSession(s)?.profile.is_admin
+
+export class NekoClient {
+  readonly store = createStore<State>()(subscribeWithSelector(initialState))
+  readonly api = new NekoApi()
+  readonly events = new Emitter<NekoEvents>()
+  readonly transport: StreamTransport
+  // where input devices send: the transport's own channel, or the websocket
+  readonly input: InputChannel
+
+  private ws: WebSocket | null = null
+  private introduced = false // system/init arrived on the current socket: the transport's reports count
+  // every websocket message, for the transport's signalling
+  private readonly messages = new Emitter<Record<string, (payload: any) => void>>()
+  private wanted = false // user asked to be connected
+  private attempts = 0
+  private reconnectTimer = 0
+  private reconnectGen = 0 // a close()+connect() while a reconnect was deciding must win
+  private staleTimer = 0
+  private lastMessage = 0
+  private resume = false // playback was running when the socket was lost: pick it up on the new stream
+  private clipboardAt = 0 // when the remote clipboard last changed
+  private localSeen = { text: '', at: 0 } // the local clipboard text last handed over, and when
+
+  private readonly opts: NekoClientOptions
+
+  constructor(opts: NekoClientOptions = {}) {
+    this.opts = opts
+    const t = opts.transport
+    this.transport = typeof t === 'function' ? t() : (t ?? new WebRTCTransport())
+    this.input = this.transport.input ?? new WebSocketInput(this)
+    this.transport.on('state', (s) => this.onTransportState(s))
+    this.transport.on('error', (err) => this.close(err))
+
+    const { store } = this
+    store.subscribe(selectControlling, () => this.sendKeyboardMap())
+    store.subscribe(
+      (s) => s.control.keyboard.layout + '/' + s.control.keyboard.variant,
+      () => this.sendKeyboardMap(),
+    )
+    store.subscribe(
+      (s) => s.connection.status,
+      (s) => this.events.emit('connection.status', s),
+    )
+  }
+
+  /////////////////////////////
+  // state
+  /////////////////////////////
+
+  // the current state; a new object after every change, never mutated
+  get state() {
+    return this.store.getState()
+  }
+
+  // an immutable update of one slice: patch('control', { locked: true })
+  private patch<K extends Slice>(key: K, part: Partial<State[K]>) {
+    this.store.setState((s) => ({ ...s, [key]: { ...s[key], ...part } }))
+  }
+
+  get connected() {
+    return this.state.connection.status === 'connected'
+  }
+
+  get controlling() {
+    return selectControlling(this.state)
+  }
+
+  get session() {
+    return selectSession(this.state)
+  }
+
+  get isAdmin() {
+    return selectIsAdmin(this.state)
+  }
+
+  get privateModeEnabled() {
+    return this.state.settings.private_mode && !this.isAdmin
+  }
+
+  get implicitControl() {
+    return this.state.settings.implicit_hosting && !!this.session?.profile.can_host
+  }
+
+  get isTouchDevice() {
+    if (this.opts.inputMode) return this.opts.inputMode === 'touch'
+    return (
+      ('ontouchstart' in window || navigator.maxTouchPoints > 0) &&
+      !matchMedia('(pointer:fine)').matches &&
+      !matchMedia('(hover:hover)').matches
+    )
+  }
+
+  /////////////////////////////
+  // auth
+  /////////////////////////////
+
+  // point at a server (defaults to this page); `?token=` in the url logs in with that token
+  setUrl(url = location.href) {
+    const u = new URL(url)
+    const token = u.searchParams.get('token') || undefined
+    const http = (u.origin + u.pathname).replace(/^ws/, 'http').replace(/\/(api\/ws\/?)?$/, '')
+
+    this.close()
+    this.api.url = http
+    this.store.setState({ connection: { ...this.state.connection, url: http, token }, authenticated: false })
+
+    if (this.opts.autoconnect) {
+      this.authenticate()
+        .then(() => this.connect())
+        .catch(() => {})
+    }
+  }
+
+  async authenticate(token = this.state.connection.token) {
+    if (!token && this.opts.autologin) token = localStorage.getItem('neko_session') ?? undefined
+    if (token) this.setToken(token)
+    await this.api.req('GET', '/whoami')
+    this.store.setState({ authenticated: true })
+  }
+
+  async login(username: string, password: string) {
+    const res = await this.api.req<LoginResponse>('POST', '/login', { username, password })
+    // a session resumed meanwhile (autologin racing an invite link) must not keep its websocket
+    this.close()
+    // token is only returned when the server does not use cookies
+    if (res.token) this.setToken(res.token)
+    this.store.setState({ authenticated: true })
+  }
+
+  async logout() {
+    this.close()
+    try {
+      await this.api.req('POST', '/logout')
+    } finally {
+      this.setToken('')
+      this.store.setState({ authenticated: false })
+    }
+  }
+
+  private setToken(token: string) {
+    this.api.token = token
+    this.patch('connection', { token: token || undefined })
+    if (!this.opts.autologin) return
+    try {
+      token ? localStorage.setItem('neko_session', token) : localStorage.removeItem('neko_session')
+    } catch {}
+  }
+
+  /////////////////////////////
+  // connection: websocket + transport
+  /////////////////////////////
+
+  connect() {
+    if (!this.state.authenticated) throw new Error('client not authenticated')
+    if (this.wanted) return
+    this.wanted = true
+    this.attempts = 0
+    this.patch('connection', { status: 'connecting' })
+    this.openSocket()
+  }
+
+  disconnect() {
+    this.close()
+  }
+
+  // stop everything, the stream included; emits connection.closed if we were connected or connecting
+  private close(error?: Error) {
+    const was = this.wanted
+    this.wanted = false
+    this.introduced = false
+    this.resume = false
+    clearTimeout(this.reconnectTimer)
+    clearInterval(this.staleTimer)
+    this.reconnectGen++
+    const ws = this.ws
+    this.ws = null
+    // a normal closure: without a status code the server assumes a reconnect is coming and
+    // keeps the session connected (and host) for another 5 s
+    ws?.close(1000)
+    this.transport.close()
+    this.clear()
+    if (was) this.events.emit('connection.closed', error)
+  }
+
+  private openSocket() {
+    const { url, token } = this.state.connection
+    const ws = (this.ws = new WebSocket(
+      url.replace(/^http/, 'ws') + '/api/ws' + (token ? '?token=' + encodeURIComponent(token) : ''),
+    ))
+    this.lastMessage = Date.now()
+    ws.onmessage = (e) => {
+      this.lastMessage = Date.now()
+      const { event, payload } = JSON.parse(e.data)
+      this.onMessage(event, payload)
+    }
+    ws.onclose = () => this.ws === ws && this.onSocketLost()
+    // a dropped network often never closes the socket; give up on it and reconnect
+    clearInterval(this.staleTimer)
+    this.staleTimer = window.setInterval(() => {
+      if (this.ws === ws && Date.now() - this.lastMessage > STALE_MS) {
+        this.ws = null
+        ws.close()
+        this.onSocketLost()
+      }
+    }, 5000)
+  }
+
+  // the transport is left alone: its stream may well outlive the socket (the last picture stays
+  // while the socket reconnects, and a peer that fails meanwhile re-requests itself through
+  // session.send, which drops until the socket is back); the reconnect's system/init then
+  // starts it over with transport.connect()
+  private onSocketLost() {
+    this.ws = null
+    this.introduced = false
+    this.resume ||= this.state.video.playing
+    if (!this.wanted) return
+    this.patch('connection', { status: 'connecting' })
+    if (++this.attempts > RECONNECT_MAX) return this.close(new Error('connection lost'))
+
+    // a deleted session never comes back; anything else is worth retrying. The lookup is bounded:
+    // on a black-holed network a fetch can otherwise hang for minutes
+    const gen = this.reconnectGen
+    this.api
+      .req('GET', '/whoami', undefined, { signal: AbortSignal.timeout(5000) })
+      .then(
+        () => true,
+        (err) => !(err instanceof ApiError && err.status === 401),
+      )
+      .then((retry) => {
+        if (!this.wanted || gen !== this.reconnectGen) return
+        if (!retry) return this.close(new Error('session expired'))
+        this.reconnectTimer = window.setTimeout(
+          () => this.openSocket(),
+          RECONNECT_BACKOFF_MS * Math.min(this.attempts, 4),
+        )
+      })
+  }
+
+  send(event: string, payload?: any) {
+    if (this.ws?.readyState === WebSocket.OPEN) this.ws.send(JSON.stringify({ event, payload }))
+  }
+
+  // the stream: connected only once the transport is; its status counts while the socket is up
+  private onTransportState(s: TransportState) {
+    const playable = s.video.playable && !this.state.video.playable // a stream arrived (or a new one)
+    this.patch('video', s.video)
+    if (playable) {
+      if (this.opts.autoplay || this.resume) this.transport.setPlaying(true).catch(() => {})
+      this.resume = false
+    }
+    if (!this.wanted || !this.introduced) return
+    if (s.status === 'connected') this.attempts = 0
+    this.patch('connection', {
+      status: s.status === 'connected' || s.status === 'unavailable' ? 'connected' : 'connecting',
+    })
+  }
+
+  // what the transport gets of this session, once the server has introduced it (system/init)
+  private sessionInfo(init: SessionInfo['init']): SessionInfo {
+    return {
+      url: this.state.connection.url,
+      token: this.state.connection.token,
+      init,
+      send: (event, payload) => this.send(event, payload),
+      on: (event, cb) => this.messages.on(event, cb),
+    }
+  }
+
+  /////////////////////////////
+  // websocket events -> state
+  /////////////////////////////
+
+  // every message goes to the transport's subscribers first (signal/*); the rest is the room.
+  // What one message changes is one setState, so subscribers never see it half-applied.
+  private onMessage(event: string, p: any) {
+    this.messages.emit(event, p)
+    const { state, store, events } = this
+    switch (event) {
+      case 'system/init':
+        store.setState({
+          session_id: p.session_id,
+          control: {
+            ...state.control,
+            touch: !!p.touch_events,
+            host_id: p.control_host?.has_host ? p.control_host.host_id : null,
+          },
+          screen: { ...state.screen, size: p.screen_size },
+          sessions: p.sessions ?? {},
+          settings: p.settings,
+        })
+        this.introduced = true
+        this.transport.connect(this.sessionInfo(p)).catch((err) => this.close(err))
+        break
+      case 'system/admin':
+        this.patch('screen', {
+          configurations: [...(p.screen_sizes_list ?? [])].sort(
+            (a: ScreenSize, b: ScreenSize) => b.width - a.width || b.height - a.height || b.rate - a.rate,
+          ),
+        })
+        events.emit('room.broadcast.status', p.broadcast_status?.is_active, p.broadcast_status?.url)
+        break
+      case 'system/settings': {
+        const { id, ...settings } = p
+        store.setState({ settings })
+        events.emit('room.settings.updated', settings, id)
+        break
+      }
+      case 'system/disconnect':
+        this.close(new Error(p.message))
+        break
+      case 'system/heartbeat':
+        // lets proxies see client->server traffic; the server ignores the payload
+        this.send('client/heartbeat')
+        break
+
+      case 'session/created':
+        store.setState({ sessions: { ...state.sessions, [p.id]: p } })
+        events.emit('session.created', p.id)
+        break
+      case 'session/deleted': {
+        const sessions = { ...state.sessions }
+        delete sessions[p.id]
+        store.setState({ sessions })
+        events.emit('session.deleted', p.id)
+        break
+      }
+      case 'session/profile': {
+        const { id, ...profile } = p
+        if (!state.sessions[id]) return
+        store.setState({ sessions: { ...state.sessions, [id]: { ...state.sessions[id], profile } } })
+        events.emit('session.updated', id)
+        break
+      }
+      case 'session/state': {
+        const { id, ...st } = p
+        if (!state.sessions[id]) return
+        store.setState({ sessions: { ...state.sessions, [id]: { ...state.sessions[id], state: st } } })
+        events.emit('session.updated', id)
+        break
+      }
+      case 'session/cursors':
+        break // inactive cursors: not shown by this client
+
+      case 'control/host':
+        this.patch('control', { host_id: p.has_host ? p.host_id : null })
+        events.emit('room.control.host', p.has_host, p.host_id, p.id)
+        break
+      case 'control/request':
+        events.emit('room.control.request', p.id)
+        break
+
+      case 'screen/updated': {
+        const { id, ...size } = p
+        this.patch('screen', { size })
+        events.emit('room.screen.updated', size.width, size.height, size.rate, id)
+        break
+      }
+      case 'clipboard/updated':
+        this.patch('control', { clipboard: { text: p.text } })
+        this.clipboardAt = Date.now()
+        navigator.clipboard?.writeText(p.text).catch(() => {}) // only over https
+        events.emit('room.clipboard.updated', p.text)
+        break
+      case 'broadcast/status':
+        events.emit('room.broadcast.status', p.is_active, p.url)
+        break
+      case 'send/unicast':
+        events.emit('receive.unicast', p.sender, p.subject, p.body)
+        break
+      case 'send/broadcast':
+        events.emit('receive.broadcast', p.sender, p.subject, p.body)
+        break
+
+      default:
+        events.emit('message', event, p)
+    }
+  }
+
+  private clear() {
+    const { state } = this
+    this.store.setState({
+      connection: { ...state.connection, status: 'disconnected' },
+      control: { ...state.control, host_id: null, clipboard: null },
+      screen: { ...state.screen, configurations: [] },
+      session_id: null,
+      sessions: {},
+      settings: defaultSettings(),
+    })
+  }
+
+  /////////////////////////////
+  // control
+  /////////////////////////////
+
+  request() {
+    this.send('control/request')
+  }
+
+  release() {
+    this.send('control/release')
+  }
+
+  // local lock: keep control but stop sending input
+  lock() {
+    this.patch('control', { locked: true })
+  }
+
+  unlock() {
+    this.patch('control', { locked: false })
+  }
+
+  // types text remotely via the server's clipboard
+  paste(text: string) {
+    this.send('control/paste', { text })
+  }
+
+  // Before a paste keystroke, with the text the browser's own paste event handed over: the side
+  // that copied most recently wins. The remote reports every change of its clipboard; the local
+  // clipboard can only be looked at when the browser hands it over, so if the remote changed since
+  // the local text was last seen, the remote is taken as fresher and nothing is sent (on https the
+  // remote text is also written to the local clipboard, so the two then match). Otherwise the
+  // local text becomes the remote clipboard, and this resolves once the server has confirmed it.
+  async preparePaste(text: string) {
+    const stale = this.localSeen.at < this.clipboardAt
+    const fresh = text !== '' && text !== this.state.control.clipboard?.text && !stale
+    this.localSeen = { text, at: Date.now() }
+    if (!fresh) return
+    this.send('clipboard/set', { text })
+    await new Promise<void>((done) => {
+      const off = this.events.on('room.clipboard.updated', () => (off(), done()))
+      setTimeout(() => (off(), done()), 500)
+    })
+  }
+
+  setScrollInverse(value = true) {
+    this.patch('control', { scroll: { ...this.state.control.scroll, inverse: value } })
+  }
+
+  setScrollSensitivity(value: number) {
+    this.patch('control', { scroll: { ...this.state.control.scroll, sensitivity: value } })
+  }
+
+  setKeyboard(layout: string, variant = '') {
+    this.patch('control', { keyboard: { layout, variant } })
+  }
+
+  private sendKeyboardMap() {
+    if (this.controlling && this.state.control.keyboard.layout) this.send('keyboard/map', this.state.control.keyboard)
+  }
+
+  setScreenSize(width: number, height: number, rate: number) {
+    this.send('screen/set', { width, height, rate })
+  }
+
+  sendBroadcast(subject: string, body: any) {
+    this.send('send/broadcast', { subject, body })
+  }
+
+  sendUnicast(receiver: string, subject: string, body: any) {
+    this.send('send/unicast', { receiver, subject, body })
+  }
+
+  async uploadDrop({ x, y, files }: { x: number; y: number; files: File[] }) {
+    const form = new FormData()
+    form.append('x', String(x))
+    form.append('y', String(y))
+    for (const f of files) form.append('files', f)
+    try {
+      await this.api.upload('/room/upload/drop', form, (p) => this.events.emit('upload.drop.progress', p))
+      this.events.emit('upload.drop.finished')
+    } catch (err: any) {
+      this.events.emit('upload.drop.finished', err)
+    }
+  }
+
+  /////////////////////////////
+  // media (through the transport)
+  /////////////////////////////
+
+  // when the browser refuses sound, playback starts muted (video.mutedByAutoplay) and unmutes on
+  // the first click
+  play() {
+    return this.transport.setPlaying(true)
+  }
+
+  pause() {
+    this.transport.setPlaying(false)
+  }
+
+  mute() {
+    this.transport.setMuted(true)
+  }
+
+  unmute() {
+    this.transport.setMuted(false)
+  }
+
+  setVolume(value: number) {
+    this.transport.setVolume(Math.max(0, Math.min(1, value)))
+  }
+
+  // share local media (microphone), on transports that can; returns what stops sharing it
+  shareMedia(stream: MediaStream): () => void {
+    if (!this.transport.shareMedia) throw new Error('transport cannot send media')
+    return this.transport.shareMedia(stream)
+  }
+}

client/core/src/emitter.ts

@@ -0,0 +1,23 @@
+// Minimal typed event emitter (on/off/once/emit).
+export class Emitter<E extends { [K in keyof E]: (...args: any[]) => void }> {
+  private handlers = new Map<keyof E, Set<Function>>()
+
+  on<K extends keyof E>(event: K, fn: E[K]) {
+    if (!this.handlers.has(event)) this.handlers.set(event, new Set())
+    this.handlers.get(event)!.add(fn)
+    return () => this.off(event, fn)
+  }
+
+  off<K extends keyof E>(event: K, fn: E[K]) {
+    this.handlers.get(event)?.delete(fn)
+  }
+
+  once<K extends keyof E>(event: K, fn: E[K]) {
+    const wrap = ((...args: any[]) => (this.off(event, wrap), fn(...args))) as E[K]
+    return this.on(event, wrap)
+  }
+
+  emit<K extends keyof E>(event: K, ...args: Parameters<E[K]>) {
+    for (const fn of [...(this.handlers.get(event) ?? [])]) fn(...args)
+  }
+}

client/core/src/index.ts

@@ -0,0 +1,27 @@
+// @m1k1o/neko: everything needed to talk to a neko server, usable from any framework.
+// The GUI in ../src imports only from here.
+export {
+  NekoClient,
+  selectSession,
+  selectControlling,
+  selectIsAdmin,
+  type NekoClientOptions,
+  type Pos,
+} from './client.ts'
+export { Overlay } from './overlay.ts'
+export { mount } from './mount.ts'
+export { Emitter } from './emitter.ts'
+export { NekoApi, ApiError } from './api.ts'
+export type {
+  StreamTransport,
+  SessionInfo,
+  InputChannel,
+  TransportKind,
+  TransportStatus,
+  TransportState,
+  TransportEvents,
+} from './transport.ts'
+export { WebRTCTransport } from './transport/webrtc.ts'
+export { DataChannelInput } from './input/datachannel.ts'
+export { WebSocketInput } from './input/websocket.ts'
+export * from './types.ts'

client/core/src/input/datachannel.test.ts

@@ -0,0 +1,75 @@
+// The data channel input: the overlay's input as big-endian frames out, the host's cursor in.
+import { test } from 'vitest'
+import assert from 'node:assert/strict'
+import { FakeChannel } from '../test/browser.ts'
+
+const { DataChannelInput } = await import('../index.ts')
+
+test('every method is one frame: opcode, u16 payload length, big-endian fields of the server’s widths', () => {
+  const input = new DataChannelInput()
+  const dc = new FakeChannel()
+  input.bind(dc as any)
+  assert.equal(dc.binaryType, 'arraybuffer')
+  dc.readyState = 'open'
+  input.move(100, 200)
+  input.move(65535, 1)
+  input.scroll(-1, 1, true)
+  input.scroll(3, 0, false)
+  input.button(1, true)
+  input.button(3, false)
+  input.key(0xff0d, true)
+  input.key(0xff0d, false)
+  input.key(0x1008ff11, true) // XF86AudioLowerVolume: a keysym above u16
+  input.touch('begin', 7, -5, 6, 255)
+  input.touch('update', 7, 10, 20, 128)
+  input.touch('end', 7, 10, 20, 0)
+  assert.deepEqual(dc.frames, [
+    [1, 0, 4, 0, 100, 0, 200], // MOVE: u16 x, u16 y
+    [1, 0, 4, 255, 255, 0, 1],
+    [2, 0, 5, 255, 255, 0, 1, 1], // SCROLL: i16 delta_x, i16 delta_y, u8 control_key
+    [2, 0, 5, 0, 3, 0, 0, 0],
+    [5, 0, 4, 0, 0, 0, 1], // BTN_DOWN: u32 code
+    [6, 0, 4, 0, 0, 0, 3], // BTN_UP
+    [3, 0, 4, 0, 0, 0xff, 0x0d], // KEY_DOWN: u32 keysym
+    [4, 0, 4, 0, 0, 0xff, 0x0d], // KEY_UP
+    [3, 0, 4, 0x10, 0x08, 0xff, 0x11],
+    [8, 0, 13, 0, 0, 0, 7, 255, 255, 255, 251, 0, 0, 0, 6, 255], // TOUCH_BEGIN: u32 id, i32 x, i32 y, u8 pressure
+    [9, 0, 13, 0, 0, 0, 7, 0, 0, 0, 10, 0, 0, 0, 20, 128], // TOUCH_UPDATE
+    [10, 0, 13, 0, 0, 0, 7, 0, 0, 0, 10, 0, 0, 0, 20, 0], // TOUCH_END
+  ])
+})
+
+test('nothing is sent while the channel is not open: unbound, connecting, or gone', () => {
+  const input = new DataChannelInput()
+  input.move(1, 2) // no channel at all
+  const dc = new FakeChannel()
+  input.bind(dc as any)
+  input.move(1, 2) // still connecting
+  input.key(1, true)
+  assert.deepEqual(dc.frames, [])
+  dc.readyState = 'open'
+  input.move(1, 2)
+  assert.equal(dc.frames.length, 1)
+  input.bind(null) // the peer is gone
+  input.move(1, 2)
+  input.button(1, true)
+  assert.equal(dc.frames.length, 1)
+})
+
+test('receive: cursor position and image frames reach the callbacks', () => {
+  const input = new DataChannelInput()
+  const got: unknown[] = []
+  input.onCursorPosition = (p) => got.push(p)
+  input.onCursorImage = (img) => got.push(img)
+  const dc = new FakeChannel()
+  input.bind(dc as any)
+  const frame = (bytes: number[]) => dc.onmessage!({ data: new Uint8Array(bytes).buffer })
+  frame([1, 0, 4, 0x01, 0x02, 0x03, 0x04]) // CURSOR_POSITION x=258 y=772
+  frame([2, 0, 12, 0, 16, 0, 20, 0, 3, 0, 5, 0x89, 0x50, 0x4e, 0x47]) // CURSOR_IMAGE 16x20 hotspot 3,5 + png bytes
+  frame([9, 0, 0]) // unknown: ignored
+  assert.deepEqual(got[0], { x: 258, y: 772 })
+  const img = got[1] as { width: number; height: number; x: number; y: number; uri: string }
+  assert.deepEqual([img.width, img.height, img.x, img.y], [16, 20, 3, 5])
+  assert.match(img.uri, /^blob:/)
+  assert.equal(got.length, 2)
+})

client/core/src/input/datachannel.ts

@@ -0,0 +1,97 @@
+// Input over the WebRTC data channel: big-endian binary messages (server/internal/webrtc/payload),
+// and the host's cursor coming back the same way.
+import type { InputChannel } from '../transport.ts'
+import type { CursorImage } from '../types.ts'
+
+// opcodes (server/internal/webrtc/payload)
+const OP = {
+  // client -> server
+  MOVE: 1,
+  SCROLL: 2,
+  KEY_DOWN: 3,
+  KEY_UP: 4,
+  BTN_DOWN: 5,
+  BTN_UP: 6,
+  TOUCH_BEGIN: 8,
+  TOUCH_UPDATE: 9,
+  TOUCH_END: 10,
+  // server -> client
+  CURSOR_POSITION: 1,
+  CURSOR_IMAGE: 2,
+}
+const TOUCH_OP = { begin: OP.TOUCH_BEGIN, update: OP.TOUCH_UPDATE, end: OP.TOUCH_END }
+
+// a field of a message: its width in bytes (2 = u16, 4 = u32, -2 = i16, -4 = i32, 1 = u8) and value
+type Field = [bytes: number, value: number]
+
+export class DataChannelInput implements InputChannel {
+  private dc: RTCDataChannel | null = null
+
+  onCursorPosition?: (pos: { x: number; y: number }) => void
+  onCursorImage?: (image: CursorImage) => void
+
+  // the channel the server opened on the peer connection; null once the peer is gone
+  bind(dc: RTCDataChannel | null) {
+    if (!dc) {
+      this.dc = null
+      return
+    }
+    dc.binaryType = 'arraybuffer'
+    dc.onmessage = (e) => {
+      const v = new DataView(e.data)
+      const op = v.getUint8(0)
+      if (op === OP.CURSOR_POSITION) {
+        this.onCursorPosition?.({ x: v.getUint16(3), y: v.getUint16(5) })
+      } else if (op === OP.CURSOR_IMAGE) {
+        const uri = URL.createObjectURL(new Blob([e.data.slice(11)], { type: 'image/png' }))
+        this.onCursorImage?.({
+          width: v.getUint16(3),
+          height: v.getUint16(5),
+          x: v.getUint16(7),
+          y: v.getUint16(9),
+          uri,
+        })
+      }
+    }
+    this.dc = dc
+  }
+
+  move(x: number, y: number) {
+    this.send(OP.MOVE, [2, x], [2, y])
+  }
+
+  scroll(deltaX: number, deltaY: number, controlKey: boolean) {
+    this.send(OP.SCROLL, [-2, deltaX], [-2, deltaY], [1, controlKey ? 1 : 0])
+  }
+
+  button(code: number, down: boolean) {
+    this.send(down ? OP.BTN_DOWN : OP.BTN_UP, [4, code])
+  }
+
+  key(keysym: number, down: boolean) {
+    this.send(down ? OP.KEY_DOWN : OP.KEY_UP, [4, keysym])
+  }
+
+  touch(phase: 'begin' | 'update' | 'end', id: number, x: number, y: number, pressure: number) {
+    this.send(TOUCH_OP[phase], [4, id], [-4, x], [-4, y], [1, pressure])
+  }
+
+  // opcode, u16 payload length, the fields; dropped while the channel is not open
+  private send(op: number, ...fields: Field[]) {
+    if (this.dc?.readyState !== 'open') return
+    const len = fields.reduce((n, [b]) => n + Math.abs(b), 0)
+    const v = new DataView(new ArrayBuffer(3 + len))
+    v.setUint8(0, op)
+    v.setUint16(1, len)
+    let o = 3
+    for (const [b, val] of fields) {
+      if (b === 1) v.setUint8(o, val)
+      else if (b === 2) v.setUint16(o, val)
+      else if (b === -2) v.setInt16(o, val)
+      else if (b === 4) v.setUint32(o, val)
+      else v.setInt32(o, val)
+      o += Math.abs(b)
+    }
+    this.dc.send(v.buffer)
+  }
+}

client/core/src/input/websocket.test.ts

@@ -0,0 +1,100 @@
+// Input over the websocket: the overlay's input as the server's control/* events.
+import { test } from 'vitest'
+import assert from 'node:assert/strict'
+import { tick, reset, sent, sentPayloads, sockets, init, receive } from '../test/browser.ts'
+
+const { WebSocketInput, NekoClient } = await import('../index.ts')
+
+const recorder = () => {
+  const out: { event: string; payload: unknown }[] = []
+  return { out, send: (event: string, payload?: unknown) => out.push({ event, payload }) }
+}
+
+test('every method becomes the matching control/* event with the server’s payload fields', async () => {
+  reset()
+  const r = recorder()
+  const input = new WebSocketInput(r)
+  input.scroll(-1, 2, true)
+  input.scroll(3, 0, false)
+  input.key(0xff0d, true)
+  input.key(0xff0d, false)
+  input.button(1, true)
+  input.button(1, false)
+  input.touch('begin', 7, 10, 20, 128)
+  input.touch('update', 7, 11, 21, 129)
+  input.touch('end', 7, 12, 22, 0)
+  assert.deepEqual(r.out, [
+    { event: 'control/scroll', payload: { delta_x: -1, delta_y: 2, control_key: true } },
+    { event: 'control/scroll', payload: { delta_x: 3, delta_y: 0, control_key: false } },
+    { event: 'control/keydown', payload: { keysym: 0xff0d } },
+    { event: 'control/keyup', payload: { keysym: 0xff0d } },
+    { event: 'control/buttondown', payload: { code: 1 } },
+    { event: 'control/buttonup', payload: { code: 1 } },
+    { event: 'control/touchbegin', payload: { touch_id: 7, x: 10, y: 20, pressure: 128 } },
+    { event: 'control/touchupdate', payload: { touch_id: 7, x: 11, y: 21, pressure: 129 } },
+    { event: 'control/touchend', payload: { touch_id: 7, x: 12, y: 22, pressure: 0 } },
+  ])
+  await tick(100)
+  assert.equal(r.out.length, 9)
+})
+
+test('moves within 16 ms are coalesced, the last one wins; a click flushes the move first', async () => {
+  reset()
+  const r = recorder()
+  const input = new WebSocketInput(r)
+  input.move(10, 20)
+  input.move(11, 21)
+  assert.deepEqual(r.out, [])
+  await tick(15)
+  assert.deepEqual(r.out, [])
+  await tick(1)
+  assert.deepEqual(r.out, [{ event: 'control/move', payload: { x: 11, y: 21 } }])
+  await tick(100)
+  assert.equal(r.out.length, 1, 'a flushed move is not sent again')
+
+  input.move(30, 40)
+  input.button(1, true) // the press lands where the pointer is now
+  assert.deepEqual(r.out.slice(1), [
+    { event: 'control/move', payload: { x: 30, y: 40 } },
+    { event: 'control/buttondown', payload: { code: 1 } },
+  ])
+  await tick(16)
+  assert.equal(r.out.length, 3)
+
+  input.move(50, 60) // and the timer works again after a flush
+  await tick(16)
+  assert.deepEqual(r.out.at(-1), { event: 'control/move', payload: { x: 50, y: 60 } })
+})
+
+test('nothing reaches the server while the socket is not open', async () => {
+  reset()
+  const transport = {
+    kind: 'hls',
+    element: null,
+    connect: async () => {},
+    close() {},
+    attach: () => () => {},
+    setPlaying: async () => {},
+    setVolume() {},
+    setMuted() {},
+    on: () => () => {},
+  } as const
+  const client = new NekoClient({ transport })
+  assert.ok(client.input instanceof WebSocketInput, 'no input channel on the transport: the websocket')
+  client.input.key(1, true) // no socket at all
+  client.store.setState({ authenticated: true })
+  client.connect()
+  sockets[0].readyState = 0 // still connecting
+  client.input.key(1, true)
+  client.input.move(1, 2)
+  await tick(16)
+  assert.deepEqual(sent, [])
+  sockets[0].readyState = 1
+  receive('system/init', init())
+  client.input.key(1, true)
+  assert.deepEqual(sent, ['control/keydown'])
+  assert.deepEqual(sentPayloads.at(-1), { event: 'control/keydown', payload: { keysym: 1 } })
+  client.disconnect()
+  client.input.key(1, false)
+  assert.deepEqual(sent, ['control/keydown'])
+})

client/core/src/input/websocket.ts

@@ -0,0 +1,50 @@
+// Input over the main websocket, for transports without an input path of their own: the v3
+// control/* events the server handles (server/internal/websocket/handler/control.go, payloads in
+// server/pkg/types/message).
+import type { InputChannel } from '../transport.ts'
+
+const MOVE_COALESCE_MS = 16
+
+export class WebSocketInput implements InputChannel {
+  private readonly socket: { send(event: string, payload?: unknown): void }
+  private pending: { x: number; y: number } | null = null
+  private moveTimer = 0
+
+  constructor(socket: { send(event: string, payload?: unknown): void }) {
+    this.socket = socket
+  }
+
+  // one move per frame, the latest wins
+  move(x: number, y: number) {
+    this.pending = { x, y }
+    if (!this.moveTimer) this.moveTimer = window.setTimeout(() => this.flush(), MOVE_COALESCE_MS)
+  }
+
+  scroll(deltaX: number, deltaY: number, controlKey: boolean) {
+    this.flush()
+    this.socket.send('control/scroll', { delta_x: deltaX, delta_y: deltaY, control_key: controlKey })
+  }
+
+  button(code: number, down: boolean) {
+    this.flush() // a click lands where the pointer is now
+    this.socket.send(down ? 'control/buttondown' : 'control/buttonup', { code })
+  }
+
+  key(keysym: number, down: boolean) {
+    this.flush()
+    this.socket.send(down ? 'control/keydown' : 'control/keyup', { keysym })
+  }
+
+  touch(phase: 'begin' | 'update' | 'end', id: number, x: number, y: number, pressure: number) {
+    this.flush()
+    this.socket.send(`control/touch${phase}`, { touch_id: id, x, y, pressure })
+  }
+
+  private flush() {
+    clearTimeout(this.moveTimer)
+    this.moveTimer = 0
+    if (!this.pending) return
+    this.socket.send('control/move', this.pending)
+    this.pending = null
+  }
+}

client/core/src/keyboard/guacamole.d.ts

@@ -0,0 +1,11 @@
+// Apache Guacamole keyboard, the same file as upstream master's client/src/utils/guacamole-keyboard.js
+export default class GuacamoleKeyboard {
+  constructor(element?: Element)
+  // return true to let the browser handle the key, false to prevent default
+  onkeydown?: (keysym: number) => boolean
+  onkeyup?: (keysym: number) => void
+  press(keysym: number): boolean
+  release(keysym: number): void
+  reset(): void
+  listenTo(element: Element | Document): void
+}

client/core/src/mount.ts

@@ -0,0 +1,18 @@
+// The non-React convenience: the stream with the keyboard/mouse overlay on top fill `el`,
+// letterboxed to the remote screen. Returns what takes them out again. A stream without input is
+// just `client.transport.attach(el)`; a UI framework does these two attaches itself (../src/features/video/Player.tsx).
+import { Overlay } from './overlay.ts'
+import type { NekoClient } from './client.ts'
+
+export function mount(client: NekoClient, el: HTMLElement): () => void {
+  const box = document.createElement('div')
+  box.style.position = 'relative'
+  const detach = client.transport.attach(box)
+  el.append(box)
+  const detachOverlay = new Overlay(client).attach(box)
+  return () => {
+    detachOverlay()
+    detach()
+    box.remove()
+  }
+}

client/core/src/overlay.test.ts

@@ -0,0 +1,369 @@
+// The keyboard/mouse/touch overlay on its own: what attach() puts into the box and detach() takes
+// out, DOM events reaching client.input, the host's cursor drawn from the transport's events,
+// and the mount() helper around it.
+import { test } from 'vitest'
+import assert from 'node:assert/strict'
+import { tick, settle, reset, init, patch, FakeElement, resizeObservers, windowListeners } from './test/browser.ts'
+
+const { NekoClient, Overlay, mount } = await import('./index.ts')
+type Options = ConstructorParameters<typeof NekoClient>[0]
+type Transport = NonNullable<Exclude<NonNullable<Options>['transport'], () => unknown>>
+
+const LAYER = 'position:absolute;inset:0;width:100%;height:100%;'
+
+// a transport with a recording input channel and events a test can emit
+const fakeTransport = () => {
+  const calls: unknown[][] = []
+  const listeners = new Map<string, Set<(...a: any[]) => void>>()
+  const noop = () => {}
+  const record =
+    (m: string) =>
+    (...a: unknown[]) =>
+      calls.push([m, ...a])
+  const t = {
+    kind: 'ws-mse' as const,
+    element: null,
+    input: {
+      move: record('move'),
+      scroll: record('scroll'),
+      button: record('button'),
+      key: record('key'),
+      touch: record('touch'),
+    },
+    connect: async () => {},
+    close: noop,
+    attach: (box: HTMLElement) => {
+      const video = new FakeElement('video')
+      ;(box as unknown as FakeElement).append(video)
+      return () => video.remove()
+    },
+    setPlaying: async () => {},
+    setVolume: noop,
+    setMuted: noop,
+    on: (ev: string, cb: (...a: any[]) => void) => {
+      ;(listeners.get(ev) ?? listeners.set(ev, new Set()).get(ev)!).add(cb)
+      return () => listeners.get(ev)!.delete(cb)
+    },
+    emit: (ev: string, ...a: unknown[]) => listeners.get(ev)?.forEach((cb) => cb(...a)),
+    subscribers: (ev: string) => listeners.get(ev)?.size ?? 0,
+  }
+  return { t: t as unknown as Transport, calls, emit: t.emit, subscribers: t.subscribers }
+}
+
+// a client in a connected room (s1 = us, s2 = Bob the host) with an area of 1000x500 holding a
+// box with the transport's element; the overlay is not attached yet
+const setup = () => {
+  reset()
+  const { t, calls, emit, subscribers } = fakeTransport()
+  const client = new NekoClient({ transport: t })
+  const p = init()
+  const s = client.state
+  client.store.setState({
+    session_id: p.session_id,
+    sessions: p.sessions,
+    settings: p.settings,
+    screen: { ...s.screen, size: p.screen_size },
+    control: { ...s.control, host_id: 's2', touch: true },
+    connection: { ...s.connection, status: 'connected' },
+  })
+  const area = new FakeElement('div')
+  area.offsetWidth = 1000
+  area.offsetHeight = 500
+  const box = new FakeElement('div')
+  area.append(box)
+  t.attach(box as any)
+  const overlay = new Overlay(client)
+  const wrap = () => box.children[1]
+  const canvas = () => wrap().children[0]
+  const textarea = () => wrap().children[1]
+  return { client, calls, emit, subscribers, area, box, overlay, wrap, canvas, textarea }
+}
+const host = async (client: InstanceType<typeof NekoClient>) => {
+  patch(client, 'control', { host_id: 's1' })
+  await settle()
+  assert.equal(client.controlling, true)
+}
+// the canvas as laid out: 960x540 at (10, 20), half the 1920x1080 remote screen
+const laidOut = (canvas: FakeElement) => (canvas.rect = { left: 10, top: 20, width: 960, height: 540 })
+
+test('attach: cursor canvas and textarea after the transport’s element, as the client used to make them; detach takes them out and unsubscribes', async () => {
+  const { client, box, overlay, wrap, canvas, textarea, subscribers } = setup()
+  assert.equal(box.children.length, 1, 'only the stream before attach')
+  const detach = overlay.attach(box as any)
+  assert.deepEqual(
+    box.children.map((c) => c.tag),
+    ['video', 'div'],
+  )
+  assert.deepEqual(
+    wrap().children.map((c) => c.tag),
+    ['canvas', 'textarea'],
+  )
+  assert.equal(canvas().style.cssText, LAYER)
+  assert.equal(
+    textarea().style.cssText,
+    LAYER +
+      'font-size:16px;resize:none;caret-color:transparent;outline:0;border:0;color:transparent;background:transparent;',
+  )
+  assert.equal(textarea().spellcheck, false)
+  assert.deepEqual(textarea().attrs, { autocapitalize: 'off', autocomplete: 'off', 'data-gramm': 'false' })
+  assert.deepEqual(
+    [wrap().style.display, wrap().style.pointerEvents],
+    ['', 'auto'],
+    'shown, and can_host takes the mouse',
+  )
+
+  // the box is letterboxed inside the area (1000x500 for 16:9 -> 888.9x500, centred), the canvas sized with it
+  const w = (500 * 1920) / 1080
+  assert.deepEqual(box.style, { width: `${w}px`, height: '500px', marginTop: '0px', marginLeft: `${(1000 - w) / 2}px` })
+  assert.deepEqual([(canvas() as any).width, (canvas() as any).height], [w, 500], 'the canvas backing store follows')
+  assert.equal(resizeObservers.at(-1)?.observed, box.parent, 'the area is observed')
+  patch(client, 'screen', { size: { width: 1000, height: 1000, rate: 30 } }) // a square screen: 500x500 in the middle
+  await settle()
+  assert.deepEqual(box.style, { width: '500px', height: '500px', marginTop: '0px', marginLeft: '250px' })
+  box.parent!.offsetWidth = 400 // the area shrinks: the observer reports it
+  resizeObservers.at(-1)!.cb()
+  assert.deepEqual(box.style, { width: '400px', height: '400px', marginTop: '50px', marginLeft: '0px' })
+
+  // private mode hides it from users, a local lock or no can_host takes the mouse away
+  patch(client, 'settings', { private_mode: true })
+  await settle()
+  assert.equal(wrap().style.display, 'none')
+  patch(client, 'settings', { private_mode: false })
+  patch(client, 'control', { locked: true })
+  await settle()
+  assert.deepEqual([wrap().style.display, wrap().style.pointerEvents], ['', 'none'])
+  patch(client, 'control', { locked: false })
+  await settle()
+
+  assert.equal(windowListeners.mouseup?.size, 1)
+  assert.deepEqual([subscribers('cursor.position'), subscribers('cursor.image')], [1, 1])
+  const w1 = wrap()
+  detach()
+  assert.deepEqual(
+    box.children.map((c) => c.tag),
+    ['video'],
+  )
+  assert.equal(windowListeners.mouseup?.size, 0)
+  assert.deepEqual([subscribers('cursor.position'), subscribers('cursor.image')], [0, 0])
+  assert.ok(resizeObservers.at(-1)!.disconnected)
+  patch(client, 'control', { locked: true }) // the store subscriptions are gone too
+  patch(client, 'screen', { size: { width: 1920, height: 1080, rate: 60 } })
+  await settle()
+  assert.equal(w1.style.pointerEvents, 'auto')
+  assert.equal(box.style.width, '400px')
+  detach() // twice: nothing left to do
+})
+
+test('attaching twice replaces the first attachment: one set of elements, listeners and subscriptions', () => {
+  const { box, overlay, wrap, subscribers } = setup()
+  overlay.attach(box as any)
+  const first = resizeObservers.length
+  overlay.attach(box as any)
+  assert.deepEqual(
+    box.children.map((c) => c.tag),
+    ['video', 'div'],
+  )
+  assert.equal(wrap().children.length, 2)
+  assert.equal(windowListeners.mouseup.size, 1)
+  assert.deepEqual([subscribers('cursor.position'), subscribers('cursor.image')], [1, 1])
+  assert.ok(resizeObservers[first - 1].disconnected, 'the first observer is gone')
+  assert.equal(resizeObservers.at(-1)!.disconnected, false)
+  const other = new FakeElement('div')
+  box.parent!.append(other)
+  const detach = overlay.attach(other as any) // and it can move to another box
+  assert.deepEqual([box.children.length, other.children.length], [1, 1])
+  detach()
+  assert.deepEqual([box.children.length, other.children.length], [1, 0])
+  assert.equal(windowListeners.mouseup.size, 0)
+})
+
+test('keyboard: the browser’s keydown/keypress/keyup on the textarea reach client.input.key as the X11 keysym, only while hosting and unlocked', async () => {
+  const { client, calls, box, overlay, textarea } = setup()
+  overlay.attach(box as any)
+  // what the browser fires for the letter a
+  const press = () => {
+    textarea().dispatch('keydown', { keyCode: 65, key: 'a', location: 0 })
+    textarea().dispatch('keypress', { keyCode: 97, charCode: 97, key: 'a', location: 0 })
+  }
+  const release = () => textarea().dispatch('keyup', { keyCode: 65, key: 'a', location: 0 })
+  press()
+  release()
+  assert.deepEqual(calls, [], 'a viewer’s keys go nowhere')
+  await host(client)
+  press()
+  assert.deepEqual(calls, [['key', 0x61, true]])
+  release()
+  assert.deepEqual(calls, [
+    ['key', 0x61, true],
+    ['key', 0x61, false],
+  ])
+  client.lock()
+  await settle()
+  press()
+  release()
+  assert.equal(calls.length, 2, 'locked: nothing is sent')
+})
+
+test('mouse: a mousedown reaches client.input.move and .button with the position in remote screen coordinates; the release may land anywhere', async () => {
+  const { client, calls, box, overlay, canvas, textarea } = setup()
+  overlay.attach(box as any)
+  laidOut(canvas())
+  const press = { clientX: 10 + 480, clientY: 20 + 270, button: 0 }
+  textarea().dispatch('mousedown', press)
+  assert.deepEqual(calls, [], 'not hosting: the press asks for control (implicit hosting) and waits')
+  await host(client) // ...to be replayed once we are host
+  assert.deepEqual(calls, [
+    ['move', 960, 540],
+    ['button', 1, true],
+  ])
+  for (const fn of windowListeners.mouseup) fn({ clientX: 10 + 960 + 50, clientY: 20 - 50, button: 0 }) // outside: clamped
+  assert.deepEqual(calls.slice(2), [
+    ['move', 1920, 0],
+    ['button', 1, false],
+  ])
+  for (const fn of windowListeners.mouseup) fn(press) // a release without a press on the video: nothing
+  assert.equal(calls.length, 4)
+  textarea().dispatch('mousedown', { ...press, button: 2 }) // hosting: straight through
+  assert.deepEqual(calls.slice(4), [
+    ['move', 960, 540],
+    ['button', 3, true],
+  ])
+})
+
+test('touch: a touchstart reaches client.input.touch with the id, position and pressure', async () => {
+  const { client, calls, box, overlay, canvas, textarea } = setup()
+  overlay.attach(box as any)
+  laidOut(canvas())
+  await host(client)
+  textarea().dispatch('touchstart', {
+    changedTouches: [{ identifier: 7, clientX: 10 + 96, clientY: 20 + 54, force: 0.5 }],
+  })
+  assert.deepEqual(calls, [['touch', 'begin', 7, 192, 108, 128]])
+  patch(client, 'control', { touch: false }) // a server without touch events: the first finger is the left button
+  textarea().dispatch('touchend', { changedTouches: [{ identifier: 7, clientX: 10 + 96, clientY: 20 + 54, force: 0 }] })
+  assert.deepEqual(calls.slice(1), [
+    ['move', 192, 108],
+    ['button', 1, false],
+  ])
+})
+
+test('cursor: the host’s cursor from the transport is drawn on the canvas with their name, cleared on null, and is our own pointer while hosting', async () => {
+  const { client, box, overlay, canvas, textarea, emit } = setup()
+  overlay.attach(box as any)
+  const ctx = canvas().ctx
+  const drawn = () => ctx.calls.filter((c: unknown[]) => c[0] !== 'setTransform').map((c: unknown[]) => c[0])
+  const image = { width: 16, height: 16, x: 1, y: 2, uri: 'blob:cursor' }
+  emit('cursor.image', image)
+  assert.equal(textarea().style.cursor, 'default', 'a viewer keeps their own pointer')
+  emit('cursor.position', { x: 960, y: 540 })
+  await tick(16) // the next frame
+  assert.deepEqual(drawn(), ['clearRect', 'drawImage', 'strokeText', 'fillText'])
+  const w = (500 * 1920) / 1080
+  const [, img, x, y, dw, dh] = ctx.calls.find((c: unknown[]) => c[0] === 'drawImage')!
+  assert.equal((img as { src: string }).src, 'blob:cursor')
+  assert.deepEqual([x, y, dw, dh], [Math.round(w / 2) - 1, 250 - 2, 16, 16])
+  assert.deepEqual(ctx.calls.find((c: unknown[]) => c[0] === 'fillText')!.slice(1), [
+    'Bob',
+    Math.round(w / 2) + 16,
+    250 + 16,
+  ])
+
+  ctx.calls.length = 0
+  emit('cursor.position', null) // the stream dropped its peer
+  await tick(16)
+  assert.deepEqual(drawn(), ['clearRect'])
+
+  emit('cursor.position', { x: 1, y: 1 })
+  await host(client)
+  await tick(16)
+  assert.equal(textarea().style.cursor, 'url(blob:cursor) 1 2, default')
+  assert.equal(drawn().includes('drawImage'), false, 'the host sees it as their real pointer, not drawn')
+})
+
+test('mobileKeyboardToggle: focuses the textarea to open the on-screen keyboard, blurs it to close', () => {
+  const { client, box, overlay, textarea } = setup()
+  overlay.attach(box as any)
+  overlay.mobileKeyboardToggle()
+  assert.deepEqual([textarea().focused, client.state.mobile_keyboard_open], [true, true])
+  overlay.mobileKeyboardToggle()
+  assert.deepEqual([textarea().focused, client.state.mobile_keyboard_open], [false, false])
+  textarea().focused = true
+  client.store.setState({ mobile_keyboard_open: true })
+  textarea().dispatch('blur') // the keyboard was dismissed by the user
+  assert.equal(client.state.mobile_keyboard_open, false)
+})
+
+test('mount(client, el): the stream and the overlay in a letterboxed box inside el; the returned function takes it all out', () => {
+  const { client, area, box } = setup()
+  box.remove()
+  const unmount = mount(client, area as any)
+  assert.equal(area.children.length, 1)
+  const mounted = area.children[0]
+  assert.equal(mounted.style.position, 'relative')
+  assert.deepEqual(
+    mounted.children.map((c) => c.tag),
+    ['video', 'div'],
+  )
+  assert.deepEqual(
+    mounted.children[1].children.map((c) => c.tag),
+    ['canvas', 'textarea'],
+  )
+  assert.equal(mounted.style.width, `${(500 * 1920) / 1080}px`)
+  assert.equal(windowListeners.mouseup.size, 1)
+  unmount()
+  assert.deepEqual(area.children, [])
+  assert.deepEqual(mounted.children, [], 'the transport’s element and the overlay are detached, not just the box')
+  assert.equal(windowListeners.mouseup.size, 0)
+})
+
+test('store subscriptions: resize on the screen size, sync on visibility and interactivity, control and focus on hosting, a redraw on the host; once per change, not for the rest, and one update with several fields once', async () => {
+  const { client, box, overlay } = setup()
+  const calls = { resize: 0, sync: 0, control: [] as boolean[], focus: 0, draw: 0 }
+  const o = overlay as any
+  o.onResize = () => calls.resize++
+  o.sync = () => calls.sync++
+  o.onControl = (c: boolean) => calls.control.push(c)
+  o.focusIfActive = () => calls.focus++
+  o.draw = () => calls.draw++ // before attach: the host subscription takes the function itself
+  overlay.attach(box as any)
+  Object.assign(calls, { resize: 0, sync: 0, draw: 0 }) // attach syncs and sizes once by itself
+
+  patch(client, 'control', { clipboard: { text: 'x' } }) // not watched
+  client.store.setState({ mobile_keyboard_open: true, authenticated: true })
+  patch(client, 'settings', { locked_logins: true })
+  assert.deepEqual(calls, { resize: 0, sync: 0, control: [], focus: 0, draw: 0 })
+
+  patch(client, 'screen', { size: { width: 800, height: 600, rate: 30 } })
+  assert.deepEqual(calls, { resize: 1, sync: 0, control: [], focus: 0, draw: 0 })
+  patch(client, 'settings', { private_mode: true }) // hidden from a user
+  assert.deepEqual(calls, { resize: 1, sync: 1, control: [], focus: 0, draw: 0 })
+  patch(client, 'control', { host_id: 's1' }) // we are host: control, focus, and the drawn cursor goes
+  assert.deepEqual(calls, { resize: 1, sync: 1, control: [true], focus: 1, draw: 1 })
+  client.lock() // interactive and active flip, controlling does not
+  assert.deepEqual(calls, { resize: 1, sync: 2, control: [true], focus: 2, draw: 1 })
+  client.unlock()
+  patch(client, 'control', { host_id: 's2' })
+  assert.deepEqual(calls, { resize: 1, sync: 3, control: [true, false], focus: 4, draw: 2 })
+
+  // system/init changes the screen size, the sessions (we become admin), the settings (private
+  // mode on) and the host in one update: every subscriber runs once, on the final state. Visible
+  // stays true (an admin sees through private mode), so sync does not run at all; applied field
+  // by field, settings before sessions, it would flip to false and back.
+  patch(client, 'settings', { private_mode: false }) // visible again
+  const before = JSON.parse(JSON.stringify(calls))
+  client.connect()
+  const p = init()
+  p.sessions.s1.profile.is_admin = true
+  p.settings.private_mode = true
+  p.control_host = { has_host: true, host_id: 's1' }
+  p.screen_size = { width: 640, height: 480, rate: 30 }
+  ;(client as any).onMessage('system/init', p)
+  assert.deepEqual(calls, {
+    resize: before.resize + 1,
+    sync: before.sync,
+    control: [...before.control, true],
+    focus: before.focus + 1,
+    draw: before.draw + 1,
+  })
+  assert.deepEqual([o.visible, o.interactive, o.active], [true, true, true])
+})

client/core/src/overlay.ts

@@ -0,0 +1,518 @@
+// The keyboard/mouse/touch control device: an input layer over the stream (mouse, wheel, keyboard,
+// touch, file drop) that sends through `client.input`, and the host's cursor drawn for everyone
+// who is not controlling. Attached by the consumer next to the transport's element (see attach);
+// the client itself knows nothing of it, so a stream-only consumer never loads it or the keyboard.
+import GuacamoleKeyboard from './keyboard/guacamole.js'
+import { selectControlling, type NekoClient, type Pos } from './client.ts'
+import type { CursorImage } from './types.ts'
+
+const WHEEL_STEP = 53 // px of wheel delta per scroll step
+const WHEEL_LINE_HEIGHT = 19 // px per line when the browser reports lines
+const MOVE_THROTTLE_MS = 1000 / 60
+const LAYER = 'position:absolute;inset:0;width:100%;height:100%;'
+
+// X11 keysyms
+const XK = {
+  v: 0x0076,
+  Control_L: 0xffe3,
+  Control_R: 0xffe4,
+  Meta_L: 0xffe7,
+  Alt_L: 0xffe9,
+  Alt_R: 0xffea,
+  Super_L: 0xffeb,
+  Super_R: 0xffec,
+  Mode_switch: 0xff7e,
+  ISO_Level3_Shift: 0xfe03,
+}
+
+// macOS: Cmd acts as Ctrl and Option as AltGr on the remote (same as noVNC / the legacy client)
+const isMac = /Mac|iPhone|iPod|iPad/i.test(navigator.userAgent)
+const MAC_REMAP: Record<number, number> = {
+  [XK.Meta_L]: XK.Control_L,
+  [XK.Super_L]: XK.Alt_L,
+  [XK.Super_R]: XK.Super_L,
+  [XK.Alt_L]: XK.Mode_switch,
+  [XK.Alt_R]: XK.ISO_Level3_Shift,
+}
+const remap = (key: number) => (isMac && MAC_REMAP[key]) || key
+const isCtrl = (key: number) => key === XK.Control_L || key === XK.Control_R
+
+const TOUCH_PHASE: Record<string, 'begin' | 'update' | 'end'> = {
+  touchstart: 'begin',
+  touchmove: 'update',
+  touchend: 'end',
+  touchcancel: 'end',
+}
+
+export class Overlay {
+  readonly wrap = document.createElement('div')
+  private readonly canvas = document.createElement('canvas')
+  private readonly ctx = this.canvas.getContext('2d')!
+  private readonly input = document.createElement('textarea')
+  private readonly keyboard = new GuacamoleKeyboard()
+  private readonly cleanup: (() => void)[] = []
+  private box: HTMLElement | null = null
+  private size = { width: 0, height: 0 } // of the box, in CSS px
+
+  private cursor: CursorImage | null = null
+  private readonly cursorImg = new Image()
+  private cursorPos: Pos | null = null
+  private drawQueued = false
+
+  private wheel = { x: 0, y: 0, t: 0 }
+  private lastMove = 0
+  private moveTimer = 0
+  private mouseDown = false
+  private pendingPress: { e: MouseEvent; at: number } | null = null // implicit hosting: replayed once we are host
+  private ctrlDown = 0
+  private readonly noKeyUp = new Set<number>()
+
+  private readonly client: NekoClient
+
+  constructor(client: NekoClient) {
+    this.client = client
+    this.canvas.style.cssText = LAYER
+    // a transparent textarea receives keys, IME composition and the mobile keyboard
+    this.input.style.cssText =
+      LAYER +
+      'font-size:16px;resize:none;caret-color:transparent;outline:0;border:0;color:transparent;background:transparent;'
+    this.input.spellcheck = false
+    this.input.setAttribute('autocapitalize', 'off')
+    this.input.setAttribute('autocomplete', 'off')
+    this.input.setAttribute('data-gramm', 'false')
+    this.wrap.append(this.canvas, this.input)
+
+    this.keyboard.onkeydown = (key) => this.onKeyDown(remap(key))
+    this.keyboard.onkeyup = (key) => this.onKeyUp(remap(key))
+    this.keyboard.listenTo(this.input)
+    this.cursorImg.onload = this.draw
+  }
+
+  // `box` is a positioned element that already holds the transport's element (so the stream is
+  // below): the overlay fills it, and keeps it letterboxed inside its parent to the remote screen's
+  // aspect ratio so that its canvas maps onto the picture (a stream alone needs no letterboxing, the
+  // <video> keeps the aspect ratio by itself). Returns what detaches it again; a second attach
+  // replaces the first.
+  attach(box: HTMLElement): () => void {
+    this.detach()
+    this.box = box
+    box.append(this.wrap)
+    const { client, input } = this
+
+    const on = (el: EventTarget, ev: string, fn: (e: any) => void, opts?: AddEventListenerOptions) => {
+      el.addEventListener(ev, fn, opts)
+      this.cleanup.push(() => el.removeEventListener(ev, fn, opts))
+    }
+    const stop = (fn: (e: any) => void) => (e: Event) => {
+      e.preventDefault()
+      e.stopPropagation()
+      fn(e)
+    }
+    on(
+      input,
+      'click',
+      stop((e) => client.events.emit('overlay.click', e)),
+    )
+    on(
+      input,
+      'contextmenu',
+      stop(() => {}),
+    )
+    on(
+      input,
+      'wheel',
+      stop((e) => this.onWheel(e)),
+      { passive: false },
+    )
+    on(
+      input,
+      'mousemove',
+      stop((e) => this.onMove(e)),
+    )
+    on(
+      input,
+      'mousedown',
+      stop((e) => this.onButton(e, true)),
+    )
+    // mouseup anywhere: the button may be released outside the video
+    on(window, 'mouseup', (e) => this.onButton(e, false), { capture: true })
+    on(input, 'mouseenter', (e) => this.onEnter(e))
+    on(input, 'mouseleave', () => {
+      this.hovering = false
+      this.keyboard.reset()
+    })
+    for (const ev of Object.keys(TOUCH_PHASE)) on(input, ev, (e) => this.onTouch(e), { passive: false })
+    on(
+      input,
+      'dragover',
+      stop((e) => this.onMove(e)),
+    )
+    on(
+      input,
+      'drop',
+      stop((e) => this.onDrop(e)),
+    )
+    // text that arrives as input or composition events (IME, on-screen keyboards) is typed as
+    // key events by the Guacamole keyboard; the textarea only has to be emptied afterwards
+    on(input, 'input', (e) => !e.isComposing && (this.input.value = ''))
+    on(input, 'compositionend', () => (this.input.value = ''))
+    on(
+      input,
+      'paste',
+      stop((e: ClipboardEvent) => this.onPaste(e.clipboardData?.getData('text/plain') ?? '')),
+    )
+    on(input, 'blur', () => client.store.setState({ mobile_keyboard_open: false }))
+
+    // the box follows its parent and the remote screen
+    const observer = new ResizeObserver(() => this.onResize())
+    if (box.parentElement) observer.observe(box.parentElement)
+    this.cleanup.push(
+      () => observer.disconnect(),
+      client.store.subscribe(
+        (s) => s.screen.size,
+        () => this.onResize(),
+      ),
+      client.store.subscribe(
+        () => this.visible,
+        () => this.sync(),
+      ),
+      client.store.subscribe(
+        () => this.interactive,
+        () => this.sync(),
+      ),
+      client.store.subscribe(selectControlling, (c) => this.onControl(c)),
+      client.store.subscribe((s) => s.control.host_id, this.draw),
+      client.store.subscribe(
+        () => this.active,
+        () => this.focusIfActive(),
+      ),
+      client.transport.on('cursor.position', (p) => (p ? this.onCursorPosition(p) : this.clearCursor())),
+      client.transport.on('cursor.image', (img) => this.onCursorImage(img)),
+    )
+    this.sync()
+    this.onResize()
+    return () => this.detach()
+  }
+
+  private detach() {
+    if (!this.box) return
+    clearTimeout(this.moveTimer)
+    this.cleanup.splice(0).forEach((fn) => fn())
+    this.keyboard.reset()
+    this.wrap.remove()
+    this.box = null
+  }
+
+  // shown unless the room is private (to non-admins) or there is no connection; takes the mouse
+  // only while this member may host and has not locked themselves out
+  private get visible() {
+    return !this.client.privateModeEnabled && this.client.state.connection.status !== 'disconnected'
+  }
+
+  private get interactive() {
+    return !this.client.state.control.locked && !!this.client.session?.profile.can_host
+  }
+
+  private sync() {
+    this.wrap.style.display = this.visible ? '' : 'none'
+    this.wrap.style.pointerEvents = this.interactive ? 'auto' : 'none'
+  }
+
+  // letterbox the box to the remote screen's aspect ratio inside its parent
+  private onResize() {
+    const area = this.box?.parentElement
+    if (!this.box || !area) return
+    const { width, height } = this.client.state.screen.size
+    const { offsetWidth: W, offsetHeight: H } = area
+    const w = Math.min(W, (H * width) / height)
+    const h = Math.min(H, (W * height) / width)
+    Object.assign(this.box.style, {
+      width: `${w}px`,
+      height: `${h}px`,
+      marginTop: `${(H - h) / 2}px`,
+      marginLeft: `${(W - w) / 2}px`,
+    })
+    this.size = { width: w, height: h }
+    this.resize()
+  }
+
+  // input only flows while we are host and have not locked ourselves out
+  private get active() {
+    return this.client.controlling && !this.client.state.control.locked
+  }
+
+  private pos(e: { clientX: number; clientY: number }): Pos {
+    const r = this.canvas.getBoundingClientRect()
+    const { width, height } = this.client.state.screen.size
+    const clamp = (v: number, max: number) => Math.max(0, Math.min(Math.round(v), max))
+    return {
+      x: clamp((width / r.width) * (e.clientX - r.left), width),
+      y: clamp((height / r.height) * (e.clientY - r.top), height),
+    }
+  }
+
+  private move(p: Pos) {
+    this.client.input.move(p.x, p.y)
+  }
+
+  private button(code: number, down: boolean) {
+    this.client.input.button(code, down)
+  }
+
+  /////////////////////////////
+  // mouse
+  /////////////////////////////
+
+  // throttled to MOVE_THROTTLE_MS, but the last position always goes out (trailing send),
+  // otherwise the remote pointer stops short of where the local one came to rest
+  private onMove(e: MouseEvent) {
+    if (!this.active) return
+    const pos = this.pos(e)
+    clearTimeout(this.moveTimer)
+    const wait = MOVE_THROTTLE_MS - (performance.now() - this.lastMove)
+    if (wait <= 0) return this.sendMove(pos)
+    this.moveTimer = window.setTimeout(() => this.sendMove(pos), wait)
+  }
+
+  private sendMove(pos: Pos) {
+    this.lastMove = performance.now()
+    if (this.active) this.move(pos)
+  }
+
+  private onButton(e: MouseEvent, down: boolean) {
+    if (down) this.mouseDown = true
+    else if (!this.mouseDown)
+      return // only releases of presses that started on the video
+    else this.mouseDown = false
+
+    if (!this.client.controlling) {
+      if (down && this.client.implicitControl) {
+        this.pendingPress = { e, at: performance.now() }
+        this.client.request()
+      }
+      return
+    }
+    if (this.client.state.control.locked) return
+    this.move(this.pos(e))
+    this.button(e.button + 1, down)
+  }
+
+  private onWheel(e: WheelEvent) {
+    if (!this.active) return
+
+    // the first event of a gesture scrolls immediately, then one step per WHEEL_STEP px
+    const first = e.timeStamp - this.wheel.t > 250
+    if (first) this.wheel = { x: 0, y: 0, t: e.timeStamp }
+    const k = e.deltaMode !== 0 ? WHEEL_LINE_HEIGHT : 1
+    this.wheel.x += e.deltaX * k
+    this.wheel.y += e.deltaY * k
+
+    // sensitivity -5..5: negative widens the step, positive scrolls more per step
+    const { sensitivity, inverse } = this.client.state.control.scroll
+    const step = sensitivity < 0 ? WHEEL_STEP * (1 - sensitivity) : WHEEL_STEP
+    const amount = (sensitivity > 0 ? sensitivity + 1 : 1) * (inverse ? -1 : 1)
+    const axis = (d: 'x' | 'y') => {
+      const v = this.wheel[d]
+      if (!first && Math.abs(v) < step) return 0
+      if (!first) this.wheel[d] = 0
+      return Math.sign(v) * amount
+    }
+    const dx = axis('x')
+    const dy = axis('y')
+    if (dx || dy) this.client.input.scroll(dx, dy, e.ctrlKey)
+  }
+
+  private hovering = false
+
+  // keys go to the hidden input only while it is useful: a viewer typing in the chat must not lose
+  // the focus by brushing the video (touch devices: focusing would pop up the on-screen keyboard)
+  private focusIfActive() {
+    if (this.hovering && this.active && !this.client.isTouchDevice) this.input.focus()
+  }
+
+  private onEnter(e: MouseEvent) {
+    this.hovering = true
+    this.focusIfActive()
+    if (this.client.controlling) {
+      this.client.send('keyboard/modifiers', {
+        capslock: e.getModifierState('CapsLock'),
+        numlock: e.getModifierState('NumLock'),
+      })
+    }
+  }
+
+  private onDrop(e: DragEvent) {
+    if (!this.client.controlling && !this.client.implicitControl) return
+    // dropped folders are skipped; walking webkitGetAsEntry() would be needed for them
+    const files = [...(e.dataTransfer?.files ?? [])]
+    if (files.length) this.client.uploadDrop({ ...this.pos(e), files })
+  }
+
+  /////////////////////////////
+  // keyboard
+  /////////////////////////////
+
+  private onKeyDown(key: number): boolean {
+    if (!this.active) {
+      this.noKeyUp.add(key)
+      return true
+    }
+    // Ctrl+V: the keystroke stays with the browser, whose paste event hands over the local
+    // clipboard text (see onPaste). Ctrl itself was sent and stays held, so the next key has it.
+    if (this.ctrlDown && key === XK.v) {
+      this.noKeyUp.add(key)
+      return true
+    }
+    if (isCtrl(key)) this.ctrlDown = key
+    this.client.input.key(key, true)
+    return isCtrl(key) // ctrl must reach the browser for the paste shortcut above
+  }
+
+  private onKeyUp(key: number) {
+    if (this.noKeyUp.delete(key)) return
+    if (isCtrl(key)) this.ctrlDown = 0
+    this.client.input.key(key, false)
+  }
+
+  // the remote gets its paste keystroke once the clipboard is settled. By then the user may have
+  // let go of Ctrl, so the chord is completed with our own Ctrl when needed
+  private async onPaste(text: string) {
+    if (!this.active) return
+    await this.client.preparePaste(text)
+    if (!this.active) return
+    const ctrl = this.ctrlDown
+    if (!ctrl) this.client.input.key(XK.Control_L, true)
+    this.client.input.key(XK.v, true)
+    this.client.input.key(XK.v, false)
+    if (!ctrl) this.client.input.key(XK.Control_L, false)
+  }
+
+  mobileKeyboardToggle() {
+    const open = this.client.state.mobile_keyboard_open
+    if (open)
+      this.input.blur() // the blur handler clears the flag; set it from `open`, not from the flag
+    else this.input.focus()
+    this.client.store.setState({ mobile_keyboard_open: !open })
+  }
+
+  /////////////////////////////
+  // touch
+  /////////////////////////////
+
+  private onTouch(e: TouchEvent) {
+    if (!this.client.controlling) {
+      if (e.type === 'touchstart' && this.client.implicitControl) this.client.request()
+      return // let the page scroll
+    }
+    e.preventDefault()
+    e.stopPropagation()
+    if (this.client.state.control.locked) return
+
+    if (this.client.state.control.touch) {
+      for (const t of e.changedTouches) {
+        const p = this.pos(t)
+        this.client.input.touch(TOUCH_PHASE[e.type], t.identifier, p.x, p.y, Math.round(t.force * 255))
+      }
+      return
+    }
+    // server without touch support: the first finger drives the left mouse button
+    const t = e.changedTouches[0]
+    this.move(this.pos(t))
+    if (e.type === 'touchstart') this.button(1, true)
+    else if (e.type !== 'touchmove') this.button(1, false)
+  }
+
+  /////////////////////////////
+  // host changes + remote cursor
+  /////////////////////////////
+
+  private onControl(controlling: boolean) {
+    this.ctrlDown = 0
+    this.noKeyUp.clear()
+    this.updateCursorStyle()
+    this.draw()
+
+    const pending = this.pendingPress
+    this.pendingPress = null
+    // only a press from just now: control that arrives minutes later by other means (an admin
+    // giving it) must not replay an old click
+    if (controlling && pending && performance.now() - pending.at < 2000) {
+      const e = pending.e
+      this.move(this.pos(e))
+      this.button(e.button + 1, true)
+      if (!this.mouseDown) this.button(e.button + 1, false) // released while we waited
+    }
+  }
+
+  private onCursorImage(img: CursorImage) {
+    const old = this.cursor
+    this.cursor = img
+    this.cursorImg.src = img.uri
+    this.updateCursorStyle()
+    if (old) URL.revokeObjectURL(old.uri)
+  }
+
+  private onCursorPosition(p: Pos) {
+    this.cursorPos = p
+    this.draw()
+  }
+
+  private clearCursor() {
+    this.cursorPos = null
+    this.draw()
+  }
+
+  // the host sees the remote cursor as their real mouse pointer
+  private updateCursorStyle() {
+    const c = this.cursor
+    this.input.style.cursor = this.client.controlling && c ? `url(${c.uri}) ${c.x} ${c.y}, default` : 'default'
+  }
+
+  // devicePixelRatio is read on resize only; listen to matchMedia(resolution) if
+  // cursors look blurry after dragging the window to another monitor
+  private resize() {
+    const { width, height } = this.size
+    this.canvas.width = width * devicePixelRatio
+    this.canvas.height = height * devicePixelRatio
+    this.draw()
+  }
+
+  private draw = () => {
+    if (this.drawQueued) return
+    this.drawQueued = true
+    requestAnimationFrame(() => {
+      this.drawQueued = false
+      this.paint()
+    })
+  }
+
+  // everyone but the host sees the host's cursor drawn here, tagged with their name
+  private paint() {
+    const { ctx, cursor, cursorPos } = this
+    const { width, height } = this.size
+    ctx.setTransform(devicePixelRatio, 0, 0, devicePixelRatio, 0, 0)
+    ctx.clearRect(0, 0, width, height)
+    if (this.client.controlling || !cursor || !cursorPos || (cursor.width <= 1 && cursor.height <= 1)) return
+
+    const screen = this.client.state.screen.size
+    let x = Math.round((cursorPos.x / screen.width) * width)
+    let y = Math.round((cursorPos.y / screen.height) * height)
+    ctx.drawImage(this.cursorImg, x - cursor.x, y - cursor.y, cursor.width, cursor.height)
+
+    const hostId = this.client.state.control.host_id
+    const name = hostId ? this.client.state.sessions[hostId]?.profile.name : ''
+    if (!name) return
+    x += cursor.width
+    y += cursor.height
+    ctx.font = '14px Arial, sans-serif'
+    ctx.textBaseline = 'top'
+    ctx.shadowColor = 'black'
+    ctx.shadowBlur = 2
+    ctx.lineWidth = 2
+    ctx.fillStyle = 'black'
+    ctx.strokeText(name, x, y)
+    ctx.shadowBlur = 0
+    ctx.fillStyle = 'white'
+    ctx.fillText(name, x, y)
+  }
+}

client/core/src/store.test.ts

@@ -0,0 +1,95 @@
+// Unit tests of the client's store (zustand, immutable updates, selector subscriptions) and the
+// emitter: `npm test` (vitest)
+import { test } from 'vitest'
+import assert from 'node:assert/strict'
+import { reset, patch, session } from './test/browser.ts'
+import { Emitter } from './emitter.ts'
+const { NekoClient, selectControlling, selectSession, selectIsAdmin } = await import('./index.ts')
+
+const fresh = () => {
+  reset()
+  return new NekoClient({ autologin: false, autoconnect: false })
+}
+
+test('a selector subscription fires only when the selected value changes, with the value before', () => {
+  const client = fresh()
+  const seen: unknown[] = []
+  const off = client.store.subscribe(
+    (s) => s.video.playing,
+    (v, old) => seen.push([v, old]),
+  )
+  // writes elsewhere in the state: not seen (the cursor never enters the store at all, it is a
+  // transport event the overlay draws)
+  patch(client, 'control', { clipboard: { text: 'x' } })
+  client.store.setState({ mobile_keyboard_open: true })
+  patch(client, 'video', { volume: 0.5 }) // the same slice, another field: a new object, the same value
+  assert.deepEqual(seen, [])
+  patch(client, 'video', { playing: true })
+  assert.deepEqual(seen, [[true, false]])
+  patch(client, 'video', { playing: true }) // no change
+  assert.deepEqual(seen, [[true, false]])
+  off()
+  patch(client, 'video', { playing: false })
+  assert.equal(seen.length, 1, 'unsubscribed')
+})
+
+test('updates are immutable: the changed slice is a new object, untouched slices keep their identity', () => {
+  const client = fresh()
+  const before = client.state
+  client.lock()
+  const after = client.state
+  assert.notEqual(after, before)
+  assert.notEqual(after.control, before.control)
+  assert.equal(after.control.locked, true)
+  assert.equal(before.control.locked, false, 'the state before is not touched')
+  for (const k of ['connection', 'video', 'screen', 'sessions', 'settings'] as const)
+    assert.equal(after[k], before[k], `${k} keeps its identity`)
+  client.setScrollInverse(false)
+  assert.equal(
+    client.state.control.keyboard,
+    after.control.keyboard,
+    'the sibling of a nested write keeps its identity',
+  )
+  assert.notEqual(client.state.control.scroll, after.control.scroll)
+})
+
+test('client.state is the store’s current state; a plain listener sees every update', () => {
+  const client = fresh()
+  assert.equal(client.state, client.store.getState())
+  let runs = 0
+  const off = client.store.subscribe(() => runs++)
+  client.store.setState({ authenticated: true })
+  assert.equal(client.state, client.store.getState())
+  assert.deepEqual([client.state.authenticated, runs], [true, 1])
+  patch(client, 'video', { volume: 0.5 })
+  assert.equal(runs, 2)
+  off()
+  assert.ok(!('watch' in client.store) && !('version' in client.store), 'the proxy store’s API is gone')
+})
+
+test('selectors: session, controlling and isAdmin from the state, as the getters report them', () => {
+  const client = fresh()
+  const s1 = session('s1', { is_admin: true })
+  assert.deepEqual(
+    [selectSession(client.state), selectControlling(client.state), selectIsAdmin(client.state)],
+    [null, false, false],
+  )
+  client.store.setState({ session_id: 's1', sessions: { s1 } })
+  assert.deepEqual([selectSession(client.state), client.session], [s1, s1])
+  assert.deepEqual([selectIsAdmin(client.state), client.isAdmin], [true, true])
+  patch(client, 'control', { host_id: 's2' })
+  assert.deepEqual([selectControlling(client.state), client.controlling], [false, false])
+  patch(client, 'control', { host_id: 's1' })
+  assert.deepEqual([selectControlling(client.state), client.controlling], [true, true])
+})
+
+test('emitter: on returns the unsubscribe, once fires one time', () => {
+  const e = new Emitter<{ x: (v: number) => void }>()
+  const got: number[] = []
+  const off = e.on('x', (v) => got.push(v))
+  e.once('x', (v) => got.push(v * 10))
+  e.emit('x', 1)
+  off()
+  e.emit('x', 2)
+  assert.deepEqual(got, [1, 10])
+})

client/core/src/test/browser.ts

@@ -0,0 +1,342 @@
+// The browser, faked just far enough for the core to load and run under vitest: globals, a few
+// elements, a websocket, a peer connection and fetch. Imported first; the core is then imported
+// dynamically, after the globals exist. The clock is vitest's (see tick/settle).
+import { vi } from 'vitest'
+import type { NekoClient } from '../client.ts'
+import type { State } from '../types.ts'
+
+const g = globalThis as any
+
+// elements: enough of a <video>, <div>, <canvas> and <textarea> for the transport and the overlay
+export class FakeElement {
+  readonly tag: string
+  children: FakeElement[] = []
+  parent: FakeElement | null = null
+  style: Record<string, string> = {}
+  playsInline = false
+  spellcheck = true
+  value = ''
+  attrs: Record<string, string> = {}
+  paused = true
+  playError: Error | null = null // what play() rejects with, once
+  plays = 0 // how often play() was called
+  videoWidth = 0
+  videoHeight = 0
+  offsetWidth = 0
+  offsetHeight = 0
+  rect = { left: 0, top: 0, width: 0, height: 0 } // what getBoundingClientRect reports
+  focused = false
+  // a <canvas> context that records what is drawn
+  readonly ctx = new Proxy({ calls: [] as unknown[][] } as Record<string, any>, {
+    get: (t, k: string) => (k in t ? t[k] : (...a: unknown[]) => t.calls.push([k, ...a])),
+  })
+  private listeners = new Map<string, Set<(e: any) => void>>()
+  private _muted = false
+  private _volume = 1
+  private _srcObject: unknown = null
+
+  constructor(tag: string) {
+    this.tag = tag
+  }
+  get muted() {
+    return this._muted
+  }
+  set muted(v: boolean) {
+    this._muted = v
+    this.dispatch('volumechange')
+  }
+  get volume() {
+    return this._volume
+  }
+  set volume(v: number) {
+    this._volume = v
+    this.dispatch('volumechange')
+  }
+  get srcObject() {
+    return this._srcObject
+  }
+  set srcObject(v: unknown) {
+    const had = this._srcObject
+    this._srcObject = v
+    if (had && !v) this.dispatch('emptied')
+  }
+  addEventListener(ev: string, fn: (e: any) => void) {
+    if (!this.listeners.has(ev)) this.listeners.set(ev, new Set())
+    this.listeners.get(ev)!.add(fn)
+  }
+  removeEventListener(ev: string, fn: (e: any) => void) {
+    this.listeners.get(ev)?.delete(fn)
+  }
+  // one event object for every listener, as the DOM does; a test's fields override the defaults
+  dispatch(ev: string, e: any = {}) {
+    const event = { type: ev, preventDefault() {}, stopPropagation() {}, ...e }
+    for (const fn of [...(this.listeners.get(ev) ?? [])]) fn(event)
+  }
+  get parentElement() {
+    return this.parent
+  }
+  append(...els: FakeElement[]) {
+    for (const el of els) {
+      el.parent = this
+      this.children.push(el)
+    }
+  }
+  remove() {
+    if (this.parent) this.parent.children = this.parent.children.filter((c) => c !== this)
+    this.parent = null
+  }
+  setAttribute(k: string, v: string) {
+    this.attrs[k] = v
+  }
+  async play() {
+    this.plays++
+    if (this.playError) {
+      const err = this.playError
+      this.playError = null
+      throw err
+    }
+    this.paused = false
+    this.dispatch('playing')
+  }
+  pause() {
+    this.paused = true
+    this.dispatch('pause')
+  }
+  getContext() {
+    return this.ctx
+  }
+  getBoundingClientRect() {
+    return this.rect
+  }
+  focus() {
+    this.focused = true
+  }
+  blur() {
+    this.focused = false
+  }
+}
+
+g.window = g
+g.location = { href: 'http://neko.test/' }
+Object.defineProperty(g, 'navigator', {
+  value: { platform: 'Linux', userAgent: 'test', maxTouchPoints: 0 },
+  configurable: true,
+})
+export const storage = new Map<string, string>()
+g.localStorage = {
+  getItem: (k: string) => storage.get(k) ?? null,
+  setItem: (k: string, v: string) => storage.set(k, v),
+  removeItem: (k: string) => storage.delete(k),
+}
+g.matchMedia = () => ({ matches: false })
+// what the overlay needs: observers and window listeners are recorded so tests can fire and count them
+export const resizeObservers: { cb: () => void; observed: FakeElement | null; disconnected: boolean }[] = []
+g.ResizeObserver = class {
+  private readonly rec: (typeof resizeObservers)[number]
+  constructor(cb: () => void) {
+    resizeObservers.push((this.rec = { cb, observed: null, disconnected: false }))
+  }
+  observe(el: FakeElement) {
+    this.rec.observed = el
+  }
+  disconnect() {
+    this.rec.disconnected = true
+  }
+}
+g.Image = class {}
+g.devicePixelRatio = 1
+g.requestAnimationFrame = (fn: () => void) => setTimeout(fn, 16)
+export const windowListeners: Record<string, Set<(e: any) => void>> = {}
+g.addEventListener = (ev: string, fn: (e: any) => void) => (windowListeners[ev] ??= new Set()).add(fn)
+g.removeEventListener = (ev: string, fn: (e: any) => void) => windowListeners[ev]?.delete(fn)
+export const documentListeners: Record<string, ((e: any) => void)[]> = {}
+g.document = {
+  createElement: (tag: string) => new FakeElement(tag),
+  addEventListener: (ev: string, fn: (e: any) => void) => (documentListeners[ev] ??= []).push(fn),
+  removeEventListener() {},
+  hasFocus: () => true,
+}
+
+// the clock: timers and Date are fake, setImmediate stays real so promise chains can be drained
+vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout', 'setInterval', 'clearInterval', 'Date'] })
+// one macrotask turn drains every pending promise chain (the handshake, a fetch)
+export const settle = () => new Promise<void>((r) => setImmediate(r))
+export const tick = async (ms: number) => {
+  await settle() // whatever is in flight may still schedule a timer
+  await vi.advanceTimersByTimeAsync(ms) // fires the due timers in order, settling after each
+}
+
+// the network
+export const sent: string[] = []
+export const sentPayloads: { event: string; payload: unknown }[] = []
+export const sockets: FakeSocket[] = []
+export class FakeSocket {
+  static OPEN = 1
+  readyState = 1
+  url: string
+  closed = false
+  code?: number
+  onmessage?: (e: { data: string }) => void
+  onclose?: () => void
+  constructor(url: string) {
+    this.url = url
+    sockets.push(this)
+  }
+  send(data: string) {
+    const m = JSON.parse(data)
+    sent.push(m.event)
+    sentPayloads.push(m)
+  }
+  close(code?: number) {
+    this.closed = true
+    this.code = code
+  }
+}
+g.WebSocket = FakeSocket
+
+export const OFFER =
+  'v=0\r\nm=audio 9 RTP/AVP 0\r\na=rtpmap:0 opus/48000\r\nm=video 9 RTP/AVP 96\r\na=rtpmap:96 VP8/90000\r\n'
+export const ANSWER = 'v=0\r\nm=audio 9 RTP/AVP 0\r\nm=video 9 RTP/AVP 96\r\n'
+export class FakePeer {
+  static all: FakePeer[] = []
+  static answerSdp = ANSWER
+  config: unknown
+  connectionState = 'new'
+  signalingState = 'stable'
+  remoteDescription: unknown = null
+  localDescription: { sdp: string } | null = null
+  candidates: unknown[] = []
+  senders: unknown[] = []
+  closed = false
+  onconnectionstatechange?: () => void
+  ontrack?: (e: unknown) => void
+  ondatachannel?: (e: { channel: unknown }) => void
+  onicecandidate?: (e: { candidate: { toJSON(): unknown } | null }) => void
+  onnegotiationneeded?: () => Promise<void>
+  constructor(config?: unknown) {
+    this.config = config
+    FakePeer.all.push(this)
+  }
+  async setRemoteDescription(d: unknown) {
+    this.remoteDescription = d
+  }
+  async createAnswer() {
+    return { type: 'answer', sdp: FakePeer.answerSdp }
+  }
+  async createOffer() {
+    return { type: 'offer', sdp: 'local-offer' }
+  }
+  async setLocalDescription(d: { sdp: string }) {
+    this.localDescription = d
+  }
+  async addIceCandidate(c: unknown) {
+    this.candidates.push(c)
+  }
+  addTrack(track: unknown) {
+    const sender = { track }
+    this.senders.push(sender)
+    return sender
+  }
+  removeTrack(sender: unknown) {
+    if (!this.senders.includes(sender)) throw new Error('InvalidAccessError')
+    this.senders = this.senders.filter((s) => s !== sender)
+  }
+  getSenders() {
+    return this.senders
+  }
+  close() {
+    this.closed = true
+  }
+  // what the browser's ICE agent would report
+  transition(s: string) {
+    this.connectionState = s
+    this.onconnectionstatechange?.()
+  }
+}
+g.RTCPeerConnection = FakePeer
+
+export const fetches: { url: string; init: RequestInit }[] = []
+// what fetch answers; tests replace net.respond
+export const net = {
+  respond: (_url: string, _init: RequestInit): { status: number; body?: unknown } => ({ status: 200 }),
+}
+g.fetch = async (url: string, init: RequestInit) => {
+  fetches.push({ url, init })
+  const { status, body } = net.respond(url, init)
+  const text = body === undefined ? '' : JSON.stringify(body)
+  return {
+    ok: status < 300,
+    status,
+    statusText: 'status ' + status,
+    json: async () => JSON.parse(text),
+    text: async () => text,
+  }
+}
+
+// a data channel as the peer would hand it over
+export class FakeChannel {
+  readyState = 'connecting'
+  binaryType = ''
+  frames: number[][] = []
+  onmessage?: (e: { data: ArrayBuffer }) => void
+  send(b: ArrayBuffer) {
+    this.frames.push([...new Uint8Array(b)])
+  }
+}
+
+// server payloads
+export const profile = (over = {}) => ({
+  name: 'Alice',
+  is_admin: false,
+  can_login: true,
+  can_connect: true,
+  can_watch: true,
+  can_host: true,
+  can_share_media: false,
+  can_access_clipboard: true,
+  sends_inactive_cursor: false,
+  can_see_inactive_cursors: false,
+  ...over,
+})
+export const session = (id: string, over = {}) => ({
+  id,
+  profile: profile(over),
+  state: { is_connected: true, is_watching: false },
+})
+export const settings = (over = {}) => ({
+  private_mode: false,
+  locked_logins: false,
+  locked_controls: false,
+  control_protection: false,
+  implicit_hosting: true,
+  inactive_cursors: false,
+  merciful_reconnect: true,
+  ...over,
+})
+export const init = () => ({
+  session_id: 's1',
+  sessions: { s1: session('s1'), s2: session('s2', { name: 'Bob' }) },
+  settings: settings(),
+  screen_size: { width: 1920, height: 1080, rate: 60 },
+  control_host: { has_host: true, host_id: 's2' },
+  touch_events: true,
+})
+export const receive = (event: string, payload?: unknown, ws = sockets.at(-1)!) =>
+  ws.onmessage!({ data: JSON.stringify({ event, payload }) })
+
+// a clean slate for a test
+export const reset = () => {
+  sent.length = sentPayloads.length = sockets.length = fetches.length = FakePeer.all.length = 0
+  resizeObservers.length = 0
+  for (const k of Object.keys(documentListeners)) delete documentListeners[k]
+  for (const k of Object.keys(windowListeners)) delete windowListeners[k]
+  vi.clearAllTimers()
+  storage.clear()
+  net.respond = () => ({ status: 200 })
+  FakePeer.answerSdp = ANSWER
+}
+
+// a nested write of the client state, one slice at a time (the store takes immutable updates)
+type Slice = 'connection' | 'video' | 'control' | 'screen' | 'settings'
+export const patch = <K extends Slice>(client: NekoClient, key: K, part: Partial<State[K]>) =>
+  client.store.setState((s) => ({ ...s, [key]: { ...s[key], ...part } }))

client/core/src/transport.ts

@@ -0,0 +1,86 @@
+// The seam between the session (NekoClient: auth, the main websocket, room state) and the media
+// stream. A transport owns its media element and its own connection; it talks to the server only
+// through the SessionInfo the client hands it. Modelled on guacamole-common-js, where Guacamole.Client
+// runs over a Guacamole.Tunnel that may be HTTP or WebSocket.
+import type { CursorImage, InitPayload } from './types.ts'
+
+export type TransportKind = 'webrtc' | 'webcodecs-ws' | 'ws-mse' | 'hls'
+
+export type TransportStatus =
+  | 'disconnected'
+  | 'connecting'
+  | 'connected'
+  // the server will not stream to this session (e.g. can_watch is off): the room works without video
+  | 'unavailable'
+
+export interface TransportState {
+  status: TransportStatus
+  // the stream's own size once a frame has arrived (letterboxing uses the server's screen size)
+  size: { width: number; height: number }
+  video: {
+    playable: boolean
+    playing: boolean
+    volume: number
+    muted: boolean
+    // the browser refused to autoplay with sound, so playback started muted
+    mutedByAutoplay: boolean
+  }
+}
+
+export interface TransportEvents {
+  state: (state: TransportState) => void
+  // bitrate, fps, rtt, ... (nothing emits it yet)
+  stats: (stats: Record<string, number>) => void
+  // the stream is lost for good; the client closes the connection with it
+  error: (error: Error) => void
+  // the host's cursor, for transports whose server feeds it back (the WebRTC data channel);
+  // null when the stream is dropped
+  'cursor.position': (pos: { x: number; y: number } | null) => void
+  'cursor.image': (image: CursorImage) => void
+}
+
+// what a transport gets from the client: where the server is, who we are, and the main websocket
+// for signalling
+export interface SessionInfo {
+  url: string // http(s) origin and path of the server
+  token?: string
+  init: InitPayload // the system/init payload
+  send(event: string, payload?: unknown): void
+  on(event: string, cb: (payload: any) => void): () => void
+}
+
+// input to the remote desktop, in the server's terms (keysyms, button codes, touch ids); how it
+// travels is the channel's business
+export interface InputChannel {
+  move(x: number, y: number): void
+  scroll(deltaX: number, deltaY: number, controlKey: boolean): void
+  button(code: number, down: boolean): void
+  key(keysym: number, down: boolean): void
+  touch(phase: 'begin' | 'update' | 'end', id: number, x: number, y: number, pressure: number): void
+}
+
+export interface StreamTransport {
+  readonly kind: TransportKind
+  // the media element created by attach(), for Picture-in-Picture and the like
+  readonly element: HTMLElement | null
+  // the transport's own input path, when it has one (the WebRTC data channel)
+  readonly input?: InputChannel
+
+  // start streaming for a session; called again on every reconnect, also while the previous
+  // session's stream is still up: it is replaced (its last picture stays until the new one shows)
+  connect(session: SessionInfo): Promise<void>
+  // stop for good: the media path, timers, counters and the picture on the element go
+  close(): void
+
+  // create the media element inside the container; returns what removes it again
+  attach(container: HTMLElement): () => void
+
+  setPlaying(on: boolean): Promise<void>
+  setVolume(volume: number): void
+  setMuted(on: boolean): void
+
+  // share local media (microphone), where the transport can; returns what stops sharing it
+  shareMedia?(stream: MediaStream): () => void
+
+  on<K extends keyof TransportEvents>(event: K, cb: TransportEvents[K]): () => void
+}

client/core/src/transport/webrtc.test.ts

@@ -0,0 +1,349 @@
+// The WebRTC transport on its own: its element, the signalling it does over the SessionInfo, the
+// state it reports, and what close() leaves behind.
+import { test, vi } from 'vitest'
+import assert from 'node:assert/strict'
+import {
+  tick,
+  settle,
+  reset,
+  FakePeer,
+  FakeElement,
+  FakeChannel,
+  OFFER,
+  init,
+  documentListeners,
+} from '../test/browser.ts'
+
+const { WebRTCTransport, DataChannelInput, Emitter } = await import('../index.ts')
+type Transport = InstanceType<typeof WebRTCTransport>
+type State = Parameters<Parameters<Transport['on']>[1]>[0]
+
+// a SessionInfo as the client builds it, with the socket recorded
+const sessionInfo = () => {
+  const out: { event: string; payload: unknown }[] = []
+  const messages = new Emitter<Record<string, (p: any) => void>>()
+  const s = {
+    url: 'http://neko.test',
+    token: 'tok',
+    init: init(),
+    send: (event: string, payload?: unknown) => out.push({ event, payload }),
+    on: (event: string, cb: (p: any) => void) => messages.on(event, cb),
+  }
+  return { s, out, events: () => out.map((m) => m.event), receive: (e: string, p?: unknown) => messages.emit(e, p) }
+}
+const setup = () => {
+  reset()
+  const t = new WebRTCTransport()
+  const states: State[] = []
+  const errors: Error[] = []
+  const cursors: unknown[] = []
+  t.on('state', (s) => states.push(s))
+  t.on('error', (e) => errors.push(e))
+  t.on('cursor.position', (p) => cursors.push(p))
+  t.on('cursor.image', (i) => cursors.push(i))
+  const container = new FakeElement('div')
+  const detach = t.attach(container as any)
+  const video = container.children[0]
+  return { t, states, errors, cursors, container, video, detach, status: () => states.at(-1)?.status }
+}
+const connected = async () => {
+  const r = setup()
+  const si = sessionInfo()
+  await r.t.connect(si.s)
+  si.receive('signal/provide', { sdp: OFFER })
+  await settle()
+  FakePeer.all[0].transition('connected')
+  si.out.length = 0
+  return { ...r, ...si }
+}
+const micStream = () => ({ getTracks: () => [{ kind: 'audio' }] }) as unknown as MediaStream
+
+test('supported: a static check for RTCPeerConnection', () => {
+  assert.equal(WebRTCTransport.supported(), true)
+  const g = globalThis as any
+  const pc = g.RTCPeerConnection
+  delete g.RTCPeerConnection
+  assert.equal(WebRTCTransport.supported(), false)
+  g.RTCPeerConnection = pc
+})
+
+test('attach: the <video> the client used to create, inside the container; the returned detach removes it', () => {
+  const { t, container, video, states, detach } = setup()
+  assert.equal(t.kind, 'webrtc')
+  assert.equal(container.children.length, 1)
+  assert.equal(video.tag, 'video')
+  assert.equal(video.playsInline, true)
+  assert.equal(video.style.cssText, 'position:absolute;inset:0;width:100%;height:100%;background:transparent')
+  assert.equal(t.element, video)
+  assert.deepEqual(states.at(-1), {
+    status: 'disconnected',
+    size: { width: 0, height: 0 },
+    video: { playable: false, playing: false, volume: 1, muted: false, mutedByAutoplay: false },
+  })
+  detach()
+  assert.deepEqual(container.children, [])
+  assert.equal(t.element, null)
+  t.attach(container as any) // and again
+  assert.equal(container.children.length, 1)
+  t.attach(container as any) // a second attach replaces the element
+  assert.equal(container.children.length, 1)
+  detach() // the first one's: long gone, and not the current element
+  assert.equal(container.children.length, 1)
+  assert.equal(t.element, container.children[0])
+})
+
+test('connect: signal/* over the session, the track on the element, state with status and size', async () => {
+  const { t, video, states, status } = setup()
+  const { s, out, events, receive } = sessionInfo()
+  await t.connect(s)
+  assert.deepEqual(out, [{ event: 'signal/request', payload: { video: {}, audio: {} } }])
+  assert.equal(status(), 'connecting')
+  receive('signal/candidate', { candidate: 'early' })
+  receive('signal/provide', { sdp: OFFER, iceservers: [{ urls: 'stun:x' }] })
+  await settle()
+  const pc = FakePeer.all[0]
+  assert.deepEqual(pc.config, { iceServers: [{ urls: 'stun:x' }] })
+  assert.deepEqual(pc.remoteDescription, { type: 'offer', sdp: OFFER })
+  assert.deepEqual(pc.candidates, [{ candidate: 'early' }])
+  assert.deepEqual(events(), ['signal/request', 'signal/answer'])
+  pc.onicecandidate!({ candidate: { toJSON: () => ({ candidate: 'mine' }) } })
+  pc.onicecandidate!({ candidate: null })
+  assert.deepEqual(out.at(-1), { event: 'signal/candidate', payload: { candidate: 'mine' } })
+  receive('signal/answer', { sdp: 'renegotiated' })
+  assert.deepEqual(pc.remoteDescription, { type: 'answer', sdp: 'renegotiated' })
+
+  const stream = { id: 'stream' }
+  const receiver: Record<string, unknown> = { jitterBufferTarget: 100 }
+  pc.ontrack!({ track: { kind: 'audio' }, streams: [stream], receiver })
+  assert.equal(video.srcObject, null, 'the audio track rides on the video stream')
+  pc.ontrack!({ track: { kind: 'video' }, streams: [stream], receiver })
+  assert.equal(video.srcObject, stream)
+  assert.equal(receiver.jitterBufferTarget, 0, 'the smallest jitter buffer')
+  await settle()
+  assert.equal(video.paused, true, 'starting playback is the client’s call (setPlaying)')
+
+  pc.transition('connected')
+  assert.equal(status(), 'connected')
+  video.videoWidth = 1920
+  video.videoHeight = 1080
+  video.dispatch('resize')
+  assert.deepEqual(states.at(-1)!.size, { width: 1920, height: 1080 })
+  pc.transition('disconnected')
+  assert.equal(status(), 'connecting')
+})
+
+test('playback: a refused play starts muted, reports mutedByAutoplay and unmutes on the first click', async () => {
+  const { t, video, states } = setup()
+  const { s, receive } = sessionInfo()
+  await t.connect(s)
+  receive('signal/provide', { sdp: OFFER })
+  await settle()
+  FakePeer.all[0].ontrack!({ track: { kind: 'video' }, streams: [{}], receiver: {} })
+  await settle()
+  assert.equal(video.plays, 0)
+  video.playError = Object.assign(new Error('blocked'), { name: 'NotAllowedError' })
+  await t.setPlaying(true)
+  assert.equal(video.paused, false)
+  assert.equal(video.muted, true)
+  assert.deepEqual(states.at(-1)!.video, {
+    playable: false,
+    playing: true,
+    volume: 1,
+    muted: true,
+    mutedByAutoplay: true,
+  })
+  documentListeners.click[0]({})
+  assert.equal(video.muted, false)
+  assert.equal(states.at(-1)!.video.mutedByAutoplay, false)
+
+  video.dispatch('canplaythrough')
+  assert.equal(states.at(-1)!.video.playable, true)
+  await t.setPlaying(false)
+  assert.equal(video.paused, true)
+  assert.equal(states.at(-1)!.video.playing, false)
+  t.setVolume(0.5)
+  t.setMuted(true)
+  assert.deepEqual([states.at(-1)!.video.volume, states.at(-1)!.video.muted], [0.5, true])
+  video.playError = Object.assign(new Error('nope'), { name: 'AbortError' })
+  await assert.rejects(t.setPlaying(true), /nope/, 'other failures are the caller’s')
+  video.playError = Object.assign(new Error('blocked'), { name: 'NotAllowedError' })
+  await assert.rejects(t.setPlaying(true), /blocked/, 'already muted: nothing more to try')
+})
+
+test('no offer within OFFER_TIMEOUT_MS: "unavailable", the room without video', async () => {
+  const { t, status } = setup()
+  const { s } = sessionInfo()
+  await t.connect(s)
+  await tick(7999)
+  assert.equal(status(), 'connecting')
+  await tick(1)
+  assert.equal(status(), 'unavailable', 'the server will not stream to this session')
+  assert.equal(vi.getTimerCount(), 0)
+
+  const again = setup()
+  const si = sessionInfo()
+  await again.t.connect(si.s)
+  si.receive('signal/provide', { sdp: OFFER })
+  await settle()
+  assert.equal(vi.getTimerCount(), 0, 'an offer that arrived stops the clock')
+  await tick(8000)
+  assert.equal(again.status(), 'connecting')
+})
+
+test('a new track resumes playback that was running (reconnect), autoplay or not', async () => {
+  const { t, video } = await connected()
+  const pc = FakePeer.all[0]
+  pc.ontrack!({ track: { kind: 'video' }, streams: [{ id: 1 }], receiver: {} })
+  await settle()
+  assert.equal(video.plays, 0, 'the transport does not start playback by itself')
+  await t.setPlaying(true)
+  assert.equal(video.plays, 1)
+  pc.ontrack!({ track: { kind: 'video' }, streams: [{ id: 2 }], receiver: {} }) // the peer was replaced
+  await settle()
+  assert.equal(video.plays, 2, 'kept playing')
+  await t.setPlaying(false)
+  pc.ontrack!({ track: { kind: 'video' }, streams: [{ id: 3 }], receiver: {} })
+  await settle()
+  assert.equal(video.plays, 2, 'paused by the user: stays paused')
+})
+
+test('input: a DataChannelInput bound to the channel the peer opens; cursor frames become events', async () => {
+  const { t, cursors } = await connected()
+  assert.ok(t.input instanceof DataChannelInput)
+  const dc = new FakeChannel()
+  FakePeer.all[0].ondatachannel!({ channel: dc })
+  dc.readyState = 'open'
+  t.input.move(1, 2)
+  assert.deepEqual(dc.frames, [[1, 0, 4, 0, 1, 0, 2]])
+  dc.onmessage!({ data: new Uint8Array([1, 0, 4, 0, 7, 0, 9]).buffer }) // CURSOR_POSITION
+  assert.deepEqual(cursors.at(-1), { x: 7, y: 9 })
+  dc.onmessage!({ data: new Uint8Array([2, 0, 9, 0, 2, 0, 3, 0, 0, 0, 1, 0]).buffer }) // CURSOR_IMAGE
+  assert.deepEqual((cursors.at(-1) as { width: number }).width, 2)
+  t.close()
+  assert.equal(cursors.at(-1), null, 'the dropped peer takes the cursor with it')
+  t.input.move(1, 2)
+  assert.equal(dc.frames.length, 1, 'nothing is sent on the old channel')
+})
+
+test('close: peer, timers, failure count, playback and the picture go; it can be connected again', async () => {
+  const old = await connected()
+  const { t, video, errors, states, status } = old
+  const stream = { id: 'stream' }
+  FakePeer.all[0].ontrack!({ track: { kind: 'video' }, streams: [stream], receiver: {} })
+  await t.setPlaying(true)
+  video.dispatch('canplaythrough')
+  assert.deepEqual([states.at(-1)!.video.playable, states.at(-1)!.video.playing], [true, true])
+  FakePeer.all[0].transition('failed') // one failure, a retry is pending
+  assert.equal(vi.getTimerCount(), 1)
+  t.close()
+  assert.ok(FakePeer.all[0].closed)
+  assert.equal(vi.getTimerCount(), 0)
+  assert.equal(status(), 'disconnected')
+  assert.equal(video.srcObject, null, 'the element is emptied: black')
+  assert.equal(video.paused, true)
+  assert.deepEqual([states.at(-1)!.video.playable, states.at(-1)!.video.playing], [false, false])
+  await tick(60_000)
+  assert.equal(FakePeer.all.length, 1, 'nothing is requested while closed')
+  old.receive('signal/provide', { sdp: OFFER }) // the old session's messages are not its business any more
+  await settle()
+  assert.equal(FakePeer.all.length, 1)
+
+  // connected again: the count started over (the first failure waits the first step of the backoff)
+  const si = sessionInfo()
+  await t.connect(si.s)
+  si.receive('signal/provide', { sdp: OFFER })
+  await settle()
+  FakePeer.all[1].ontrack!({ track: { kind: 'video' }, streams: [{ id: 'new' }], receiver: {} })
+  await settle()
+  assert.deepEqual(video.srcObject, { id: 'new' })
+  assert.equal(video.plays, 1, 'starting again is the client’s call (it remembers what was playing)')
+  si.out.length = 0
+  FakePeer.all[1].transition('failed')
+  await tick(1499)
+  assert.deepEqual(si.events(), [])
+  await tick(1)
+  assert.deepEqual(si.events(), ['signal/request'])
+  assert.deepEqual(errors, [])
+})
+
+test('connect while a stream is up (the socket reconnected): the old peer, its retry and count go, the picture stays until the new stream', async () => {
+  const old = await connected()
+  const { t, video, status } = old
+  const stream = { id: 'stream' }
+  FakePeer.all[0].ontrack!({ track: { kind: 'video' }, streams: [stream], receiver: {} })
+  await t.setPlaying(true)
+  video.dispatch('canplaythrough')
+  for (const backoff of [1500, 3000]) {
+    // two failures: the third would wait 4500 ms
+    FakePeer.all.at(-1)!.transition('failed')
+    await tick(backoff)
+    old.receive('signal/provide', { sdp: OFFER })
+    await settle()
+  }
+  FakePeer.all.at(-1)!.transition('failed') // and a retry pending
+  assert.equal(vi.getTimerCount(), 1)
+
+  const si = sessionInfo()
+  await t.connect(si.s)
+  assert.ok(
+    FakePeer.all.every((p) => p.closed),
+    'no peer of the old session survives',
+  )
+  assert.equal(vi.getTimerCount(), 1, 'its retry is gone; only the new offer timeout runs')
+  assert.deepEqual(si.events(), ['signal/request'])
+  assert.deepEqual([video.srcObject, video.paused], [stream, false], 'the last picture stays, still playing')
+  assert.equal(status(), 'connecting')
+  old.receive('signal/provide', { sdp: OFFER }) // the old session's provide: ignored
+  await settle()
+  assert.equal(FakePeer.all.length, 3)
+  si.receive('signal/provide', { sdp: OFFER })
+  await settle()
+  assert.equal(FakePeer.all.filter((p) => !p.closed).length, 1, 'exactly one live peer')
+  si.out.length = 0
+  FakePeer.all.at(-1)!.transition('failed') // the count started over: the first step of the backoff, not the fourth
+  await tick(1499)
+  assert.deepEqual(si.events(), [])
+  await tick(1)
+  assert.deepEqual(si.events(), ['signal/request'])
+  si.receive('signal/provide', { sdp: OFFER })
+  await settle()
+  const pc = FakePeer.all.at(-1)!
+  pc.ontrack!({ track: { kind: 'video' }, streams: [{ id: 'new' }], receiver: {} })
+  await settle()
+  assert.deepEqual(
+    [video.srcObject, video.plays],
+    [{ id: 'new' }, 2],
+    'the new stream replaces the picture and keeps playing',
+  )
+  pc.transition('connected')
+  assert.equal(status(), 'connected')
+})
+
+test('errors: a codec mismatch and the give-up are reported, the transport stays usable', async () => {
+  const { t, errors } = setup()
+  const { s, out, receive } = sessionInfo()
+  await t.connect(s)
+  FakePeer.answerSdp = 'v=0\r\nm=audio 9 RTP/AVP 0\r\nm=video 0 RTP/AVP 96\r\n'
+  receive('signal/provide', { sdp: OFFER.replace('VP8', 'AV1') })
+  await settle()
+  assert.deepEqual(
+    errors.map((e) => e.message),
+    ['this browser cannot play AV1 video'],
+  )
+  assert.equal(out.length, 1, 'no answer')
+})
+
+test('microphone: shareMedia adds the tracks and renegotiates through the session; stop removes them', async () => {
+  const { t, out } = await connected()
+  assert.throws(() => new WebRTCTransport().shareMedia(micStream()), /not connected/)
+  const pc = FakePeer.all[0]
+  const stream = micStream()
+  const stop = t.shareMedia(stream)
+  assert.deepEqual(pc.senders, [{ track: stream.getTracks()[0] }])
+  await pc.onnegotiationneeded!()
+  assert.deepEqual(out, [{ event: 'signal/offer', payload: { sdp: 'local-offer' } }])
+  stop()
+  assert.deepEqual(pc.senders, [])
+  stop() // already gone: ignored, not thrown (the fake peer throws for an unknown sender)
+  assert.deepEqual(pc.senders, [])
+})

client/core/src/transport/webrtc.ts

@@ -0,0 +1,261 @@
+// WebRTC: the server's offer arrives over the main socket (signal/*), media on an
+// RTCPeerConnection, input and the host's cursor on its data channel.
+import { Emitter } from '../emitter.ts'
+import { DataChannelInput } from '../input/datachannel.ts'
+import type { SessionInfo, StreamTransport, TransportEvents, TransportState, TransportStatus } from '../transport.ts'
+
+const RECONNECT_MAX = 10
+const RECONNECT_BACKOFF_MS = 1500
+// no offer within this long after asking for one: the server will not send media (e.g. can_watch
+// is off), so the room is shown without video instead of a spinner for ever; a dead socket is the
+// client's stale check's business
+const OFFER_TIMEOUT_MS = 8_000
+
+export class WebRTCTransport implements StreamTransport {
+  readonly kind = 'webrtc' as const
+  readonly input = new DataChannelInput()
+
+  static supported() {
+    return typeof RTCPeerConnection !== 'undefined'
+  }
+
+  private readonly events = new Emitter<TransportEvents>()
+  private readonly state: TransportState = {
+    status: 'disconnected',
+    size: { width: 0, height: 0 },
+    video: { playable: false, playing: false, volume: 1, muted: false, mutedByAutoplay: false },
+  }
+  private video: HTMLVideoElement | null = null
+  private session: SessionInfo | null = null
+  private unsubs: (() => void)[] = []
+  private pc: RTCPeerConnection | null = null
+  // the server trickles its candidates before signal/provide; hold them until the offer is applied
+  private candidates: RTCIceCandidateInit[] = []
+  private failures = 0
+  private retryTimer = 0
+  private offerTimer = 0
+
+  constructor() {
+    this.input.onCursorPosition = (p) => this.events.emit('cursor.position', p)
+    this.input.onCursorImage = (img) => this.events.emit('cursor.image', img)
+  }
+
+  get element() {
+    return this.video
+  }
+
+  on<K extends keyof TransportEvents>(event: K, cb: TransportEvents[K]) {
+    return this.events.on(event, cb)
+  }
+
+  /////////////////////////////
+  // signalling + peer
+  /////////////////////////////
+
+  // again on every reconnect: whatever a previous session left (its peer, retries, the count)
+  // goes, and its last picture stays on the element until the new stream replaces it
+  async connect(session: SessionInfo) {
+    this.unsubscribe()
+    this.closePeer()
+    this.failures = 0
+    this.session = session
+    const on = (event: string, cb: (p: any) => void) => this.unsubs.push(session.on(event, cb))
+    on('signal/provide', (p) => this.onProvide(p).catch((err) => console.error('[neko] webrtc setup failed', err)))
+    const renegotiate = (p: { sdp: string }) =>
+      this.onOffer(p.sdp).catch((err) => console.error('[neko] webrtc renegotiation failed', err))
+    on('signal/offer', renegotiate)
+    on('signal/restart', renegotiate)
+    on('signal/answer', (p) => this.pc?.setRemoteDescription({ type: 'answer', sdp: p.sdp }).catch(() => {}))
+    on('signal/candidate', (p) => {
+      if (this.pc?.remoteDescription) this.pc.addIceCandidate(p).catch(() => {})
+      else this.candidates.push(p)
+    })
+    // the server dropped its peer (it saw the media path fail first): ask for a new one,
+    // with the same backoff as a failure the browser noticed itself
+    on('signal/close', () => this.pc && this.onPeerState('failed'))
+    this.setStatus('connecting')
+    this.requestPeer()
+  }
+
+  // the stream is over: the peer, timers, counters and the picture go (a reconnect does not come
+  // through here but through connect(), which keeps the picture)
+  close() {
+    this.unsubscribe()
+    this.session = null
+    this.closePeer()
+    this.failures = 0
+    if (this.video) {
+      this.video.pause()
+      this.video.srcObject = null
+    }
+    this.state.video.playable = false
+    this.setStatus('disconnected')
+  }
+
+  private unsubscribe() {
+    this.unsubs.splice(0).forEach((off) => off())
+  }
+
+  private requestPeer() {
+    this.candidates = []
+    this.session?.send('signal/request', { video: {}, audio: {} })
+    clearTimeout(this.offerTimer)
+    this.offerTimer = window.setTimeout(
+      () => this.session && !this.pc && this.setStatus('unavailable'),
+      OFFER_TIMEOUT_MS,
+    )
+  }
+
+  private closePeer() {
+    clearTimeout(this.retryTimer)
+    clearTimeout(this.offerTimer)
+    this.pc?.close()
+    this.pc = null
+    this.input.bind(null)
+    this.events.emit('cursor.position', null)
+  }
+
+  private async onProvide({ sdp, iceservers }: { sdp: string; iceservers?: RTCIceServer[] }) {
+    this.closePeer()
+    const pc = (this.pc = new RTCPeerConnection({ iceServers: iceservers ?? [] }))
+    pc.onicecandidate = (e) => e.candidate && this.session?.send('signal/candidate', e.candidate.toJSON())
+    pc.ontrack = (e) => this.onTrack(e)
+    pc.ondatachannel = (e) => this.input.bind(e.channel)
+    pc.onconnectionstatechange = () => this.pc === pc && this.onPeerState(pc.connectionState)
+    // only fires after a local addTrack (microphone), the server offers everything else
+    pc.onnegotiationneeded = async () => {
+      if (pc.signalingState !== 'stable' || !pc.remoteDescription) return
+      await pc.setLocalDescription(await pc.createOffer())
+      this.session?.send('signal/offer', { sdp: pc.localDescription!.sdp })
+    }
+    await this.onOffer(sdp)
+  }
+
+  private async onOffer(sdp: string) {
+    const pc = this.pc
+    if (!pc) return
+    await pc.setRemoteDescription({ type: 'offer', sdp })
+    for (const c of this.candidates.splice(0)) pc.addIceCandidate(c).catch(() => {})
+    await pc.setLocalDescription(await pc.createAnswer())
+    // no video codec in common (e.g. an H264 stream and a Firefox without the OpenH264 plugin):
+    // the browser rejects the video section, the server cannot start the track, retrying won't help
+    if (/^m=video 0 /m.test(pc.localDescription!.sdp)) {
+      const codec = sdp.match(/^m=video[\s\S]*?a=rtpmap:\d+ ([\w-]+)/m)?.[1] ?? 'the stream'
+      return this.events.emit('error', new Error(`this browser cannot play ${codec} video`))
+    }
+    this.session?.send('signal/answer', { sdp: pc.localDescription!.sdp })
+  }
+
+  private onPeerState(s: RTCPeerConnectionState) {
+    if (s === 'connected') {
+      this.failures = 0
+      this.setStatus('connected')
+    } else if (s === 'disconnected') {
+      this.setStatus('connecting') // ICE may still recover
+    } else if (s === 'failed') {
+      this.setStatus('connecting')
+      this.closePeer()
+      // no media route (firewall, NAT1TO1, missing TURN): retry with backoff, then give up
+      // instead of asking the server for a new peer several times a second
+      if (++this.failures > RECONNECT_MAX)
+        return this.events.emit('error', new Error('video connection failed (WebRTC)'))
+      clearTimeout(this.retryTimer)
+      this.retryTimer = window.setTimeout(() => this.requestPeer(), RECONNECT_BACKOFF_MS * Math.min(this.failures, 4))
+    }
+  }
+
+  private onTrack({ track, streams, receiver }: RTCTrackEvent) {
+    // ask the browser for the smallest jitter buffer: interactive desktop, not a movie
+    if ('jitterBufferTarget' in receiver) (receiver as any).jitterBufferTarget = 0
+    if (track.kind !== 'video' || !this.video) return
+    this.video.srcObject = streams[0]
+    // a replaced peer keeps playing; starting (autoplay, after a socket loss) is the client's call
+    if (this.state.video.playing) this.play().catch(() => {})
+  }
+
+  // share local media (microphone); renegotiation happens via onnegotiationneeded
+  shareMedia(stream: MediaStream) {
+    const pc = this.pc
+    if (!pc) throw new Error('not connected')
+    const senders = stream.getTracks().map((track) => pc.addTrack(track, stream))
+    return () => {
+      // senders of a previous peer connection are already gone; removing them from the current one throws
+      for (const s of senders) if (this.pc?.getSenders().includes(s)) this.pc.removeTrack(s)
+    }
+  }
+
+  /////////////////////////////
+  // the element
+  /////////////////////////////
+
+  attach(container: HTMLElement) {
+    this.video?.remove() // a second attach replaces the element
+    const video = (this.video = document.createElement('video'))
+    video.playsInline = true
+    video.style.cssText = 'position:absolute;inset:0;width:100%;height:100%;background:transparent'
+    const v = this.state.video
+    const sync = () => this.emitState()
+    video.addEventListener('canplaythrough', () => ((v.playable = true), sync()))
+    video.addEventListener('playing', () => ((v.playing = true), sync()))
+    video.addEventListener('pause', () => ((v.playing = false), sync()))
+    video.addEventListener('emptied', () => ((v.playable = v.playing = false), sync()))
+    video.addEventListener('volumechange', () => ((v.muted = video.muted), (v.volume = video.volume), sync()))
+    video.addEventListener('resize', () => {
+      this.state.size = { width: video.videoWidth, height: video.videoHeight }
+      sync()
+    })
+    v.muted = video.muted
+    v.volume = video.volume
+    container.append(video)
+    this.emitState()
+    return () => {
+      video.remove()
+      if (this.video === video) this.video = null
+    }
+  }
+
+  setPlaying(on: boolean) {
+    if (on) return this.play()
+    this.video?.pause()
+    return Promise.resolve()
+  }
+
+  setVolume(volume: number) {
+    if (this.video) this.video.volume = volume
+  }
+
+  setMuted(on: boolean) {
+    if (this.video) this.video.muted = on
+    if (!on) {
+      this.state.video.mutedByAutoplay = false
+      this.emitState()
+    }
+  }
+
+  private async play() {
+    const video = this.video
+    if (!video) return
+    try {
+      await video.play()
+    } catch (err) {
+      if (video.muted || (err as DOMException).name !== 'NotAllowedError') throw err
+      // autoplay with sound is blocked until the user interacts: play muted,
+      // unmute on the first click anywhere
+      video.muted = true
+      this.state.video.mutedByAutoplay = true
+      this.emitState()
+      await video.play()
+      document.addEventListener('click', () => this.setMuted(false), { once: true })
+    }
+  }
+
+  private setStatus(status: TransportStatus) {
+    this.state.status = status
+    this.emitState()
+  }
+
+  private emitState() {
+    const { status, size, video } = this.state
+    this.events.emit('state', { status, size: { ...size }, video: { ...video } })
+  }
+}

client/core/src/types.ts

@@ -0,0 +1,136 @@
+// Client state + wire types, mirroring server/pkg/types (master).
+
+export interface MemberProfile {
+  name: string
+  avatar?: string
+  is_admin: boolean
+  can_login: boolean
+  can_connect: boolean
+  can_watch: boolean
+  can_host: boolean
+  can_share_media: boolean
+  can_access_clipboard: boolean
+  sends_inactive_cursor: boolean
+  can_see_inactive_cursors: boolean
+  plugins?: Record<string, any>
+}
+
+export interface SessionState {
+  is_connected: boolean
+  connected_since?: string
+  not_connected_since?: string
+  is_watching: boolean
+  watching_since?: string
+  not_watching_since?: string
+}
+
+export interface Session {
+  id: string
+  profile: MemberProfile
+  state: SessionState
+}
+
+// REST shapes the client reads (server/internal/api)
+export interface LoginResponse extends Session {
+  token?: string // only when the server does not use cookies
+}
+
+export interface MemberData {
+  id: string
+  profile: MemberProfile
+}
+
+export interface Settings {
+  private_mode: boolean
+  locked_logins: boolean
+  locked_controls: boolean
+  control_protection: boolean
+  implicit_hosting: boolean
+  inactive_cursors: boolean
+  merciful_reconnect: boolean
+  heartbeat_interval?: number
+  plugins?: Record<string, any>
+}
+
+export interface ScreenSize {
+  width: number
+  height: number
+  rate: number
+}
+
+// the system/init payload (message.SystemInit)
+export interface InitPayload {
+  session_id: string
+  control_host: { has_host: boolean; host_id?: string }
+  screen_size: ScreenSize
+  sessions: Record<string, Session>
+  settings: Settings
+  touch_events: boolean
+  screencast_enabled?: boolean
+  webrtc?: { videos?: string[] }
+}
+
+export interface State {
+  authenticated: boolean
+  connection: {
+    url: string
+    token?: string
+    status: 'disconnected' | 'connecting' | 'connected'
+  }
+  video: {
+    playable: boolean
+    playing: boolean
+    volume: number
+    muted: boolean
+    // the browser refused to autoplay with sound, so playback started muted
+    mutedByAutoplay: boolean
+  }
+  control: {
+    host_id: string | null
+    // local lock: stop sending input without giving up control
+    locked: boolean
+    clipboard: { text: string } | null
+    scroll: { inverse: boolean; sensitivity: number }
+    keyboard: { layout: string; variant: string }
+    // server supports native touch events (otherwise touch emulates the mouse)
+    touch: boolean
+  }
+  screen: {
+    size: ScreenSize
+    configurations: ScreenSize[]
+  }
+  session_id: string | null
+  sessions: Record<string, Session>
+  settings: Settings
+  mobile_keyboard_open: boolean
+}
+
+export interface CursorImage {
+  width: number
+  height: number
+  x: number
+  y: number
+  uri: string
+}
+
+// events emitted by NekoClient.events
+export interface NekoEvents {
+  'connection.status': (status: State['connection']['status']) => void
+  'connection.closed': (error?: Error) => void
+  'session.created': (id: string) => void
+  'session.deleted': (id: string) => void
+  'session.updated': (id: string) => void
+  'room.control.host': (hasHost: boolean, hostId: string | undefined, by: string) => void
+  'room.control.request': (id: string) => void
+  'room.screen.updated': (width: number, height: number, rate: number, by: string) => void
+  'room.settings.updated': (settings: Settings, by: string) => void
+  'room.clipboard.updated': (text: string) => void
+  'room.broadcast.status': (active: boolean, url?: string) => void
+  'receive.unicast': (sender: string, subject: string, body: any) => void
+  'receive.broadcast': (sender: string, subject: string, body: any) => void
+  'upload.drop.progress': (p: { loaded: number; total: number }) => void
+  'upload.drop.finished': (error?: Error) => void
+  'overlay.click': (e: MouseEvent) => void
+  // anything the core does not handle itself (plugins: chat/*, filetransfer/*, ...)
+  message: (event: string, payload: any) => void
+}

client/core/tsconfig.json

@@ -0,0 +1,19 @@
+{
+  "compilerOptions": {
+    "target": "ES2022",
+    "lib": ["ES2022", "DOM", "DOM.Iterable"],
+    "module": "ESNext",
+    "moduleResolution": "bundler",
+    "strict": true,
+    "skipLibCheck": true,
+    "isolatedModules": true,
+    "erasableSyntaxOnly": true,
+    "allowImportingTsExtensions": true,
+    "rewriteRelativeImportExtensions": true,
+    "declaration": true,
+    "outDir": "dist",
+    "rootDir": "src"
+  },
+  "include": ["src"],
+  "exclude": ["src/**/*.test.ts", "src/test"]
+}