watchparty/docs/architecture.md

# Architecture

## Goals

- Watch the same title, at (roughly) the same time, as other users on the
  same Emby server.
- Don't require anyone to change how they watch — no new client, no browser
  extension, no modified apps. People keep using whatever Emby app they
  already use.
- Ship as a single Emby Server plugin.

## Non-goals (for v1)

- Frame-accurate sync. A few seconds of drift between corrections is
  acceptable — that's inherent to the "generic remote control" approach (see
  below), and is a conscious tradeoff, not an oversight.
- A shared/forced "host controls everyone" model. The chosen model (see
  "Control model" below) is: everyone keeps their own play/pause/seek
  controls; the party only shares a reference position to resync against.
- Cross-server parties, chat, reactions. Possible later, not in v1 — see
  Roadmap in the README.

## The key idea: reuse Emby's existing remote-control commands

Emby already has an authoritative-session model for "cast" / "remote control
another device" style features: `ISessionManager` tracks every active
session's playback state, and a session can be commanded via
`SendPlaystateCommand` (`Play`, `Pause`, `Unpause`, `Seek`, ...) or
`SendGeneralCommand`. This already exists in Emby Server and, to whatever
extent each client supports being remote-controlled at all (most official
clients do, since it's how casting/remote-play works), in the clients too.

So instead of building a new synchronized player, this plugin is mostly a
coordinator sitting on top of that existing mechanism:

1. Track which sessions belong to which "party".
2. Watch each member's reported position/pause state (the same data Emby's
   own "Now Playing" / dashboard already reads).
3. When a member drifts too far from the party's reference position, or its
   pause state disagrees with the reference, send the same kind of
   Seek/Pause/Unpause command Emby would send for ordinary remote control.

Sources consulted while designing this:
- [How to build a Server Plugin](https://github.com/MediaBrowser/Emby/wiki/How-to-build-a-Server-Plugin) — project setup, `BasePlugin<T>`, `IServerEntryPoint`.
- [Creating Api Endpoints](https://github.com/MediaBrowser/Emby/wiki/Creating-Api-Endpoints) — the `IService`/`[Route]`/`IReturn<T>` REST pattern.
- [Emby plugin dev docs](https://dev.emby.media/doc/plugins/dev/index.html) — confirms `netstandard2.0`, .NET Core/Mono runtimes.
- [`MediaBrowser.Server.Core` on NuGet](https://www.nuget.org/packages/mediabrowser.server.core) — package to reference; latest published at research time was `4.9.1.90`.
- Emby community thread on [remote-controlling seek](https://emby.media/community/topic/119432-emby-remote-control-another-client-to-seek-tracks/), and Jellyfin's (Emby-derived) [`SessionManager.cs`](https://github.com/jellyfin/jellyfin/blob/master/Emby.Server.Implementations/Session/SessionManager.cs) — shape of `SendPlaystateCommand(controllingSessionId, sessionId, PlaystateRequest, CancellationToken)` and `PlaystateRequest { Command, SeekPositionTicks, ControllingUserId }`.

## Control model

Design decision: **each participant keeps their own player and controls; the
party only shares a reference position to resync against.** Concretely:

- Nobody's play/pause/seek is disabled. You control your own playback
  exactly like you always do.
- One member is the *reference* (the host, by default — falls back to
  whoever's left if the host leaves). The server treats the reference
  session's reported position/pause state as ground truth for the party.
- Other members get corrected against the reference:
  - **Pause/unpause is mirrored immediately** (configurable —
    `SyncPausePlay`, on by default): if the reference pauses, everyone gets a
    `Pause` command; if it unpauses, everyone gets `Unpause`.
  - **Position drift beyond a threshold is corrected with a `Seek`**, on a
    cooldown so a shaky connection doesn't cause a stream of seek commands
    (`DriftThresholdSeconds`, `ResyncCooldownSeconds`).
  - Anyone can also hit **"Resync me now"** in the control panel for an
    immediate correction, bypassing the cooldown.

This deliberately does *not* try to stop someone from pausing or seeking
locally — it just keeps pulling things back toward the reference. That fits
"loose sync via existing remote-control commands" much better than trying to
fight every local action.

## Sync algorithm

Runs on a timer (`SyncTickSeconds`, default 5s) inside `PartySyncEntryPoint`,
per active party:

```
reference = party.Members[party.ReferenceUserId]
referencePos = reference.EstimatedPositionTicks(now)   # extrapolated if playing

for member in party.Members, excluding reference:
    if SyncPausePlay and member.IsPaused != reference.IsPaused:
        send Pause/Unpause to member's session
        continue

    if now - member.LastResyncUtc < ResyncCooldownSeconds:
        continue   # give the last correction time to take effect

    drift = abs(member.EstimatedPositionTicks(now) - referencePos)
    if drift > DriftThresholdSeconds:
        send Seek(referencePos) to member's session
```

`EstimatedPositionTicks` extrapolates from the last known position using
elapsed wall-clock time when playing, so members aren't marked "drifted" just
because their last progress event was a few seconds ago.

Position/pause state itself comes from Emby's own session events
(`ISessionManager.PlaybackStart` / `PlaybackProgress` / `PlaybackStopped`) —
the plugin doesn't poll clients itself.

## Which session is "mine"?

The trickiest part of gluing "a user" to "a specific playback session",
since one user can have several active sessions (phone + TV + browser tab).

v1's blunt heuristic (`WatchPartyService.FindCallerSession`): when you
create/join a party, the server looks at your currently active sessions,
keeps the ones actually playing something (and, when joining, playing the
*same item* as the party), and picks the most recently active one. Good
enough for "one person, one device, actively watching" — the expected case —
but it's a real limitation if you're doing something else on a second device
at the same time. Documented here rather than hidden.

## Components

```
src/WatchParty/
  Plugin.cs           BasePlugin<PluginConfiguration> + admin settings page
  Configuration/       PluginConfiguration (defaults) + configPage.html (dashboard UI)
  Parties/
    Party.cs            party id, join code, item id, host/reference user, members
    PartyMember.cs       per-member session id + last known position/pause + EstimatedPositionTicks
    PartyManager.cs      in-memory store (ConcurrentDictionary); create/join/leave/lookup
  Sync/
    PartySyncEntryPoint.cs   IServerEntryPoint: session event wiring + the tick loop above
  Api/
    WatchPartyService.cs     REST endpoints (create/join/leave/get/resync) +
                             serves web/control-panel's static files
web/control-panel/           plain HTML/JS/CSS control panel (sign in, create/join,
                             member list with a sync-status dot, resync/leave)
```

Parties are **in-memory only** — a server restart clears them. That's fine
for v1: a party is meant to live for one watch session, not persist for
weeks.

The control panel is served *by the plugin itself* (`GET /WatchParty/app`,
`/app.js`, `/app.css`, reading embedded resources) rather than injected into
Emby's main web client. Emby doesn't document a stable way to inject a
user-facing (non-admin) page/script into the main web client, and hand-patching
the client's install directory is a well-known but fragile hack that breaks
on every Emby update — not worth it for v1. The tradeoff: opening the control
panel is a separate step from opening Emby, rather than a button inside the
Emby UI. It authenticates with your normal Emby username/password (via Emby's
standard `/Users/AuthenticateByName`) and only ever talks to your own server.

## Assumptions to verify

Update: the .NET SDK is now installed and `src/WatchParty` **builds clean
(0 errors, 0 warnings) against the real `MediaBrowser.Server.Core` /
`MediaBrowser.Common` 4.9.1.90 assemblies**, restored from NuGet. Most of the
list below turned out right on the first guess; a few didn't, and were fixed
by reading the actual assemblies (via .NET reflection) and then letting the
compiler confirm. Keeping the history here since it's a useful map of where
this SDK surprises you.

**Confirmed correct as originally written** (compiled with no changes needed):
- `BasePlugin<T>` constructor shape, `IServerEntryPoint`, `netstandard2.0` +
  `MediaBrowser.Server.Core` package — all from the official wiki.
- `IHasWebPages` (not `IHasUIPages` — that name does exist too, in
  `MediaBrowser.Model.Plugins.UI`, but `IHasWebPages` in
  `MediaBrowser.Model.Plugins` is the one that's actually wired up here) and
  `PluginPageInfo.Name`/`.EmbeddedResourcePath`.
- `ISessionManager.SendPlaystateCommand(string controllingSessionId, string
  sessionId, PlaystateRequest, CancellationToken)`, `SendGeneralCommand`, and
  `PlaystateRequest { Command, SeekPositionTicks }` / `PlaystateCommand.{Pause,
  Unpause, Seek}`.
- `ILogManager.GetLogger(name)` / `ILogger.Debug(...)` /
  `.ErrorException(...)` — still the live pattern on 4.9.x, not replaced by
  `Microsoft.Extensions.Logging`.
- `IRequiresRequest` / `IService` / `[Route]` / `IReturn<T>` for the REST
  endpoints.
- `SessionInfo.PlayState.PositionTicks` (`long?`) and `.IsPaused` (`bool`).

**Wrong on the first guess, fixed once the real assemblies were available:**
- **Namespaces**, not names: `PlaybackProgressEventArgs`/`PlaybackStopEventArgs`
  live in `MediaBrowser.Controller.Library`, not `...Session`.
  `IAuthorizationContext`/`AuthorizationInfo` live in `MediaBrowser.Controller.Net`.
- **ID types**: item/user/session ids (`SessionInfo.Id`, `SessionInfo.UserId`,
  `BaseItemDto.Id`) are plain **`string`**, not `Guid`, on this SDK version.
  `Party`/`PartyMember` were changed from `Guid` to `string` accordingly.
  (`Party.Id` itself stays a `Guid` — that's an id this plugin mints, not an
  Emby one.)
- **Reading the calling user is a two-step, not one property**:
  `IAuthorizationContext.GetAuthorizationInfo(Request).UserId` is a `long`
  (an internal row id — not the id used anywhere else). The one that matches
  `SessionInfo.UserId` etc. is `.User.Id` — except that one comes back as a
  `Guid`, so `WatchPartyService.CallerUserId()` reformats it with
  `.ToString("N")` to match the string convention used elsewhere. See the
  comment there.

**Still only compile-checked, not verified against a running server**
(nothing here fails to build — these are runtime behaviors the compiler
can't check):
- Whether `.ToString("N")` (no dashes) is really the right string format to
  match `SessionInfo.UserId` — plausible, matches Emby/Jellyfin convention
  elsewhere, not observed directly.
- HTTP error mapping — endpoints throw plain
  `ArgumentException`/`InvalidOperationException`; Emby likely has a more
  specific exception type that maps to a proper 404/400 instead of a
  generic 500.
- Static file content types — `GET /WatchParty/app` etc. return a bare
  string; `IHasResultFactory` looks like the right extension point for a
  real `Content-Type`/caching headers, not wired up yet.
- `controllingSessionId` passed as `null` to `SendPlaystateCommand` — compiles,
  unknown whether Emby's runtime wants a real session/user there instead.
- `X-Emby-Authorization` header fields and `/Users/AuthenticateByName`'s `Pw`
  field name in `web/control-panel/app.js` — it's JS, nothing compiles it;
  long-standing Emby/Jellyfin convention, not exercised against a real server.
- The dashboard settings page (`configPage.html`) actually rendering/saving
  correctly — embedding compiles, but the page's own JS (`ApiClient`,
  `Dashboard` globals) only exists inside a running Emby dashboard.

The remaining way to close these out: install the plugin DLL on a real Emby
Server 4.9.x and exercise it with two accounts.

## Known limitations

- Sync is best-effort and loose by design (see "Non-goals"). Don't expect
  frame-accuracy.
- Depends on the member's *own* client actually honoring remote Seek/Pause
  commands smoothly — most official Emby apps do (it's how casting/remote
  control works today), but behavior can vary by platform, and a Seek can
  cause a visible re-buffer.
- One item per party; no shared "up next" queue.
- No chat/reactions in v1 (kept out on purpose — see the design conversation
  that led to this repo).
- A user with two simultaneously active sessions on the same item is
  resolved by "most recently active", which can guess wrong.