watchparty/docs/architecture.md

# Architecture

## Goals

- Watch the same title, in sync, with other users of the same Emby server.
- Use Emby as the media source (catalog + streaming) without requiring
  changes to Emby itself or its official clients.
- Access gated to people with an Emby account on this server — no anonymous
  use, no separate identity system.

## Non-goals

- Chat/reactions — deliberately out of scope.
- OIDC/SSO — an OIDC provider (LemonLDAP::NG) is available in principle, but
  the explicit choice was Emby-only auth: simpler, no extra infra dependency,
  and everyone already has an Emby account.
- Full library navigation (Shows → Seasons → Episodes) in v1 — search-based
  catalog only, to keep the first version achievable. Revisit if browsing
  turns out to matter more than searching in practice.

## Why this replaced an Emby plugin

The first attempt (still in [`src/WatchParty/`](../src/WatchParty/), kept for
reference) was a real Emby Server plugin: sync via `ISessionManager`'s
existing remote-control commands, a settings page via `IHasWebPages`, and —
after a fair amount of reverse-engineering — a real user-facing page
integrated into Emby's own web client menu (`PluginPageInfo.EnableInUserMenu`,
undocumented but confirmed working; required a `data-controller` AMD module
to satisfy Emby's modern `ViewManager`, found by reading the target server's
actual `viewmanager.js`).

That plugin builds and installs correctly. What it hit a hard wall on: **a
button inside the video player itself**, or on an item's detail page. An
Emby developer confirmed on their own forum that plugins have no API for
this ("It's currently not possible for a plugin to do this"), and nothing
found in research since suggests that's changed. The only way to get a
button into Emby's own player would be patching Emby's shipped web client
files directly inside the Docker container — outside the plugin entirely,
guaranteed to break on every Emby image update, and requiring manual
reapplication each time. That trade-off wasn't worth it.

Building a standalone app sidesteps the question entirely: it's our own
player, so there's no "how do I add a button to someone else's UI" problem
in the first place, and sync can be tighter than nudging Emby's
remote-control commands ever allowed (see below).

Full trace of the plugin investigation (real errors, real fixes, sources
consulted) is in the git history of this file and of `src/WatchParty/` —
worth reading if a future Emby SDK update ever makes in-player integration
possible and this gets revisited.

## Components

```
server/   Node.js + Express + ws. Owns party/room state and the realtime
          sync channel. Never touches a media byte - see "Media never
          flows through our backend" below.

client/   Static frontend, plain JS + ES modules, no build step (same bias
          as the earlier plugin's control panel). Talks to two different
          things and must not confuse them:
            - Emby directly (embyApi.js): catalog, images, streaming.
            - Our own backend (party.js): create/join/leave a party, and
              the WebSocket for realtime sync.
```

### Auth and Emby access: the server proxies everything

Revised after the first version: the browser used to call Emby directly
(catalog, images, streaming) and only used `server/` for party coordination.
That hit real CORS problems in practice (Emby/its reverse proxy don't
answer cross-origin preflights the way that needed), and separately wasn't
the wanted shape anyway — the browser holding a real Emby access token at
all was more exposure than necessary. Current design:

- The browser sends a username/password **once**, to
  `POST /api/auth/login` on **our own backend** — never to Emby directly.
- Our backend (`server/src/embyAuth.js`) calls Emby's
  `POST /Users/AuthenticateByName` itself, using `EMBY_SERVER_URL` from its
  own environment (not something the user ever types — see `.env.example`).
  On success it mints its own opaque session id, stores
  `{embyToken, userId, userName}` server-side (in-memory, like party state),
  and hands the browser only that session id, as an `httpOnly` cookie
  (`wp_session`). The browser never sees or holds Emby's real access token.
- Every other request — catalog search, item details, images, HLS
  streaming, party actions — goes to **our own origin only**
  (`/api/...`), authenticated by that same cookie. `server/src/embyProxy.js`
  is what actually forwards these to Emby server-side and streams the
  response back, attaching the session's real Emby token itself.
- The `/api/catalog/stream/:itemId/master.m3u8` and `.../raw` routes proxy
  HLS specifically: the master playlist is fetched from Emby server-side,
  and every URI line in it is rewritten to point back through `.../raw`
  (whether Emby wrote it as relative or absolute) — otherwise hls.js would
  follow the playlist's own URLs straight to Emby the moment it tried to
  fetch a segment, defeating the whole point. See `rewritePlaylist` in
  `embyProxy.js`.
- Images (`<img src>`) and video (`hls.js`/`<video>`) still can't carry a
  bearer header, but that's fine now: they're same-origin requests to our
  own `/api/...` paths, so the session cookie rides along automatically —
  no token in any URL anywhere, unlike the first version's `?api_key=...`.

This also means the client no longer needs to ask for a server URL at all —
one less field, one less way to get it wrong. `EMBY_SERVER_URL` lives only
in the backend's environment.

### Control model: host drives, everyone else follows

Now that we own the player, the classic watch-party model (Teleparty,
Amazon Watch Party, ...) fits better than the old plugin's "everyone keeps
their own controls, shared reference position" — that design was specifically
a concession to nudging *pre-existing, independently-running* Emby app
sessions we didn't control. Here, there's only one player implementation,
so:

- The party creator is the host. Their player has normal controls.
- Everyone else's `<video>` has controls hidden; a small notice says the
  host is in charge.
- Host → server → everyone else, over the WebSocket
  (`server/src/wsHandler.js`):
  - Event-driven: every `play`/`pause`/`seeked` on the host's video sends
    `{type:"playstate", isPaused, positionTicks}` immediately.
  - Heartbeat: also sent every 3s regardless, so latecomers/drift get
    corrected even without a discrete host action.
  - Followers apply pause/unpause immediately, and only seek if drift
    exceeds 1.5s (`DRIFT_TOLERANCE_SECONDS` in `client/js/views/player.js`)
    — avoids fighting small timing noise with constant re-seeks.
- If the host leaves, the server promotes another member (same
  fallback logic the old plugin used for its "reference" member).

### Party lifecycle

In-memory only on the backend (`server/src/partyManager.js`) — a restart
clears every party, which is fine: a party is meant to last one watch
session, not persist for weeks. `removeIdleParties` sweeps anything with no
activity for 4 hours.

## Assumptions to verify

**Confirmed against the real server** (not a guess — actually exercised,
see session history): `POST /Users/AuthenticateByName` really is reachable
from Node's server-side `fetch` with no CORS issue at all (server-to-server
has no CORS concept), returns a real 401 for bad credentials, and the whole
login round-trip (browser → our backend → real Emby → real rejection → our
error message) renders correctly end to end. The `--theme-primary-color-*`
etc. custom properties `client/css/app.css` is built from were read directly
off this exact server's live dark theme, not guessed.

**Not yet exercised against the real server** — a successful login hasn't
happened yet (no real credentials used in testing), so nothing past that
point is confirmed:

- **`/Users/{userId}` for token validation** was dropped — the proxy
  redesign no longer needs it at all (sessions are validated once at login
  and cached server-side, not re-checked against Emby per-request).
- **`/Videos/{id}/master.m3u8`** for HLS playback, with
  `VideoCodec=h264&AudioCodec=aac` forcing a broadly-compatible transcode
  target. Works for basically anything Emby can transcode, but is never the
  most efficient path (no direct play, no bitrate/quality negotiation) —
  fine for v1, worth revisiting once the basic flow works.
- **HLS playlist rewriting** (`rewritePlaylist` in `embyProxy.js`) — the
  logic is straightforward (rewrite every non-`#` line) but has only been
  reasoned about, never run against a playlist Emby actually generated. If
  video loads the manifest but segments 404 or hang, start here.
- **Response field names** (`AccessToken`, `User.Id`, `User.Name`,
  `Items[].Id/Name/Overview/ProductionYear`) — long-standing Emby/Jellyfin
  convention, not independently re-confirmed for this Emby version.
- **`X-Emby-Authorization` header format** — same caveat as above.

## Known limitations

- Search-only catalog (see Non-goals) — no way to browse a show's episode
  list yet.
- One item per party; no shared queue.
- A party member needs a browser tab open and connected for sync to apply —
  there's no "catch up automatically when you come back" beyond whatever
  the next heartbeat corrects.
- No persistence: server restart or long idle time ends every party.