diff --git a/docs/docs/technical-docs/Connection-Based-Routing.md b/docs/docs/technical-docs/Connection-Based-Routing.md index 4ba902db..f5ff3be0 100644 --- a/docs/docs/technical-docs/Connection-Based-Routing.md +++ b/docs/docs/technical-docs/Connection-Based-Routing.md @@ -12,12 +12,18 @@ When routes are to be executed, Essentials will use this connection graph to dec ### Classes Referenced -* `PepperDash.Essentials.Core.Routing.IRoutingSource` -* `PepperDash.Essentials.Core.Routing.IRoutingOutputs` -* `PepperDash.Essentials.Core.Routing.IRoutingInputs` -* `PepperDash.Essentials.Core.Routing.IRoutingInputsOutputs` -* `PepperDash.Essentials.Core.Routing.IRoutingSinkNoSwitching` -* `PepperDash.Essentials.Core.Routing.IRoutingSinkWithSwitching` +* `PepperDash.Essentials.Core.IRoutingSource` +* `PepperDash.Essentials.Core.IRoutingOutputs` +* `PepperDash.Essentials.Core.IRoutingInputs` +* `PepperDash.Essentials.Core.IRoutingMidpoint` +* `PepperDash.Essentials.Core.IRoutingMidpointWithFeedback` +* `PepperDash.Essentials.Core.IRoutingSinkWithFeedback` + +> **Note:** As of the v3 routing refactor, the older `IRoutingInputsOutputs`, `IRouting`, +> `IRoutingWithFeedback`, `IRoutingWithClear`, `IRoutingSink`, `IRoutingSinkWithInputPort`, +> `IRoutingSinkNoSwitching`, `IRoutingSinkWithSwitching`, and `IRoutingSinkWithSwitchingWithInputPort` +> interfaces have been removed and consolidated - see [Routing interfaces (v3)](#routing-interfaces-v3) +> below for the current mapping. ## Example system, a simple presentation system @@ -56,11 +62,12 @@ Source devices in Essentials must implement `IRoutingOutputs` or `IRoutingSource A midpoint is a device in the middle of the signal chain. Typically a switcher, matrix or otherwise. Examples: DM chassis; DM-TX; DM-RMC; A video codec. These devices will have input and output ports. -Midpoint devices must implement `IRoutingInputsOutputs`. Midpoints with switching must implement `IRouting`. +Passthrough midpoints (no active switching) must implement `IRoutingMidpoint`. Midpoints that actively +switch and report their current routes must implement `IRoutingMidpointWithFeedback`. #### Sink -A sink is a device at the end of a full signal path. For example, a display, amplifier, encoder, etc. Sinks typically contain only input ports. They may or may not have switching, like a display with several inputs. Classes defining sink devices must implement `IRoutingSinkNoSwitching` or `IRoutingSinkWithSwitching`. +A sink is a device at the end of a full signal path. For example, a display, amplifier, encoder, etc. Sinks typically contain only input ports. They may or may not have switching, like a display with several inputs. Classes defining sink devices must implement `IRoutingSinkWithFeedback`. #### Tie-line @@ -78,4 +85,95 @@ A tie-line is a logical representation of a physical cable connection between tw ### Interfaces -Todo: Define Interfaces IRouting, IRoutingOutputs, IRoutingInputs +#### Routing interfaces (v3) + +The v3 routing refactor consolidated several older, overlapping interfaces into two feedback-aware +interfaces. If you're updating a device implemented against the pre-v3 API, use this table to find +its replacement: + +| Old interface (pre-v3, removed) | Current interface | +| --- | --- | +| `IRoutingInputsOutputs` | `IRoutingMidpoint` | +| `IRouting`, `IRoutingWithFeedback`, `IRoutingWithClear` | `IRoutingMidpointWithFeedback` | +| `IRoutingSink`, `IRoutingSinkWithInputPort`, `IRoutingSinkWithSwitching`, `IRoutingSinkWithSwitchingWithInputPort` | `IRoutingSinkWithFeedback` | + +**Base building-block interfaces** (unchanged): + +* `IRoutingSource` - marker interface for a device that originates a signal path; extends `IRoutingOutputs`. +* `IRoutingOutputs` - exposes `RoutingPortCollection OutputPorts`. +* `IRoutingInputs` - exposes `RoutingPortCollection InputPorts`. + +**Midpoint interfaces:** + +* `IRoutingMidpoint` - a passthrough device with both input and output ports but no active switching. Extends `IRoutingInputs` and `IRoutingOutputs`. +* `IRoutingMidpointWithFeedback` - a midpoint that actively switches and reports feedback. Extends `IRoutingMidpoint` and adds: + * `void ExecuteSwitch(object inputSelector, object outputSelector, eRoutingSignalType signalType)` + * `void ClearRoute(object outputSelector, eRoutingSignalType signalType)` + * `List CurrentRoutes { get; }` + * `event RouteChangedEventHandler RouteChanged` + +**Sink interface:** + +* `IRoutingSinkWithFeedback` - a routing endpoint device that can switch inputs and provides feedback. Extends `IRoutingInputs`, `IKeyName`, and `ICurrentSources` (see below), and adds: + * `void ExecuteSwitch(object inputSelector)` + * `RoutingInputPort CurrentInputPort { get; }` + * `event InputChangedEventHandler InputChanged` + +#### Current source tracking (`ICurrentSources`) + +`ICurrentSources` is a new interface (implemented by `IRoutingSinkWithFeedback`) that reports which +source is actually feeding a sink right now, per signal type: + +* `Dictionary CurrentSources { get; }` +* `Dictionary CurrentSourceKeys { get; }` +* `event EventHandler CurrentSourcesChanged` + +A new `RoutingFeedbackManager` subscribes to `RouteChanged` on every `IRoutingMidpointWithFeedback` +and `InputChanged` on every `IRoutingSinkWithFeedback` in the system, traces each change back through +the tie-line graph to the originating source (debounced ~500ms), and keeps each sink's +`ICurrentSources` state up to date automatically - developers no longer need to manually wire up +source tracking per device. + +#### Multiview layout support + +For decoders that can display several independently-routable tiles at once (e.g. a WyreStorm +NetworkHD-style multiview decoder), two additional interfaces are available: + +* `IRoutingSinkWithLayouts` - extends `IRoutingSource` and exposes `Dictionary WindowTileSinks`, one per-tile "virtual sink" that can be routed independently like any other `IRoutingSinkWithFeedback`. +* `IRoutingSinkWithLayoutState` - extends `IRoutingSinkWithLayouts` and additionally reports the current canvas/tile geometry via `MultiviewLayoutState CurrentLayout { get; }` and `event EventHandler LayoutChanged`. + +`MultiviewLayoutState` is a JSON-serializable, product-agnostic model of the canvas: `CanvasWidth`, +`CanvasHeight`, and a list of `MultiviewTileState` entries (`TileNumber`, `TileSinkKey`, `X`, `Y`, +`Width`, `Height`, `ZOrder`, `SourceDeviceKey`) - suitable for driving a routing diagram UI. + +#### Real-time routing feedback over WebSocket + +Essentials Core includes a built-in WebSocket service, `RoutingFeedbackWebsocket` +(`PepperDash.Essentials.Core.Web`), that broadcasts routing state changes to any connected client in +real time - this is what powers the live routing diagram in the +[Essentials Web Config App](https://github.com/PepperDash/essentials-web-config-app) and +[essentials-devtools](https://github.com/PepperDash/essentials-devtools). + +A client starts (or reuses) a session by sending an HTTP `GET` to the `routingFeedbackSession` API +route (e.g. `https://[processor-ip]/cws/[appId]/api/routingFeedbackSession`), which starts the +WebSocket server on a random high port (if not already running), forwards that port on the CS LAN +adapter if present, and returns a JSON body with the connection URL(s): + +```json +{ "url": "wss://[processor-ip]:[port]/routing/join/", "fallbackUrl": "wss://..." } +``` + +Once connected, the client receives a `snapshot` message describing every midpoint route, sink +input, and multiview layout currently active, followed by incremental update messages (each +debounced ~200ms) as things change: + +| Message `type` | Sent when | Payload | +| --- | --- | --- | +| `snapshot` | On connect | `midpointRoutes`, `sinkRoutes`, `layouts` for the whole system | +| `midpointRouteChanged` | An `IRoutingMidpointWithFeedback`'s `RouteChanged` fires | `deviceKey`, `routes: [{ inputPortKey, outputPortKey, signalType }]` | +| `sinkInputChanged` | An `IRoutingSinkWithFeedback`'s `InputChanged` fires | `deviceKey`, `inputPortKey`, `sourceDeviceKey`, `signalType` | +| `layoutChanged` | An `IRoutingSinkWithLayoutState`'s `LayoutChanged` fires | `deviceKey`, `layout` (a `MultiviewLayoutState`) | + +Per-tile sinks belonging to an `IRoutingSinkWithLayouts` device are reported to clients as qualified +input ports on their parent device's node, rather than as separate graph nodes, so a routing diagram +can render a multiview decoder as a single device with per-tile inputs. diff --git a/docs/docs/technical-docs/Mobile-Control.md b/docs/docs/technical-docs/Mobile-Control.md new file mode 100644 index 00000000..0215f897 --- /dev/null +++ b/docs/docs/technical-docs/Mobile-Control.md @@ -0,0 +1,240 @@ +# Mobile Control: architecture and client communication + +## TL;DR + +Mobile Control is a WebSocket-based messaging layer built into Essentials. Devices expose small +"messenger" classes that register JSON message paths (e.g. `/device/{key}/startMeeting`) with a +central `MobileControlSystemController`. Clients (typically a React app built on +[`@pepperdash/mobile-control-react-app-core`](https://github.com/PepperDash/mobile-control-react-app-core)) +connect over WebSocket with a per-room token, send the same JSON envelope to invoke device methods, +and receive unsolicited state-update messages that a Redux store uses to keep device/room state in +sync in real time. + +## Server-side architecture + +### Projects involved + +* `PepperDash.Essentials.MobileControl` - the `MobileControlSystemController` device, its + WebSocket server (`WebSocketServer/MobileControlWebsocketServer.cs`), room bridges, and + touchpanel controller. +* `PepperDash.Essentials.MobileControl.Messengers` (namespace `PepperDash.Essentials.AppServer.Messengers`) - + the `MessengerBase` class and the library of built-in messengers for common device interfaces + (`IHasStartMeetingMessenger`, `IHasMeetingInfoMessenger`, `ICommunicationMonitorMessenger`, + `IRoutingMidpointWithFeedbackMessenger`, etc.), plus the message envelope types. +* `PepperDash.Essentials.Core.DeviceTypeInterfaces` - the public contracts: + `IMobileControl`, `IMobileControlMessenger`, `IMobileControlMessengerWithSubscriptions`, + `IMobileControlRoomMessenger`, `IMobileControlAction`, `IMobileControlMessage`, and the + touchpanel-controller interfaces. + +### Direct Server vs. Edge Server + +Mobile Control supports two (non-exclusive) transport modes, configured on the `MobileControlConfig` +device: + +* **Direct Server** - `MobileControlWebsocketServer` runs an HTTPS WebSocket server directly on the + processor hardware. Clients on the same network connect straight to the processor - lowest + latency, no external dependency. This is what `mobile-control-react-app-core`'s local dev flow + and the deployed `mcUserApp` React app use. +* **Edge Server** (API server / cloud gateway) - the processor instead makes an *outbound* WebSocket + connection to an external Mobile Control server, which relays messages between it and remote + clients. Useful when clients can't reach the processor directly (e.g. no local network access). + +Both modes can be enabled simultaneously; outgoing messages are queued and sent down whichever +transport(s) are active. + +### The WebSocket endpoint (Direct Server) + +`MobileControlWebsocketServer` (`WebSocketServer/MobileControlWebsocketServer.cs`) hosts the +Direct Server: + +* **Path**: `/mc/api/ui/join/` (`_wsPath`) +* **Port**: `50000 + ` by default (e.g. program slot 2 → port `50002`), or a + custom port from config +* **User app**: static files are served from `/user/programX/mcUserApp` (`_appPath`) at the base + href `/mc/app` (`_userAppBaseHref`) - this is exactly the `mcUserApp` deployment directory used + when [deploying](../Get-started.md) a built React app. + +### Connecting and authentication (tokens) + +A client doesn't connect straight to the WebSocket - it first needs a **token** identifying which +room/UI-client slot it's joining: + +1. **Get a token** - on the processor console, run: + ``` + mobileinfo:[programSlot] + ``` + This prints the Direct Server port, any existing UI-client tokens, and full connect URLs (e.g. + `http://[ip]:[port]/mc/app?token=[token]`). New tokens can also be added dynamically + (`mobileadduiclient:[programSlot] [roomKey]`) - see the console command's help text for details. +2. **Join a room** - the client calls `GET {apiPath}/ui/joinroom?token={token}` over HTTPS, which + validates the token and returns room data including a generated `clientId`. +3. **Open the WebSocket** - the client connects to + `wss://[processor-ip]:[port]/mc/api/ui/join/{token}?clientId={clientId}`. + +Each connected client is tracked as a `UiClient`; disconnecting removes its subscriptions from +every messenger it was subscribed to. + +### The message envelope + +Every message sent in either direction is the same simple JSON envelope +(`MobileControlMessage`, namespace `PepperDash.Essentials.AppServer.Messengers`): + +```json +{ + "type": "/device/zoomRoom1/startMeeting", + "clientId": "abc-123", + "content": { "value": 30 } +} +``` + +* `type` - a path identifying the target device/room and action (client → server), or the + device/room whose state is being reported (server → client). +* `clientId` - which client sent the message, or (server → client) which single client a targeted + reply is meant for; omitted/ignored for broadcasts to all subscribers. +* `content` - a `JToken` payload. For simple values, `MobileControlSimpleContent` wraps a single + `value` property (as in the example above); for state updates, `content` is the device's full + serialized state object. + +### The messenger pattern + +Each device that wants to be controllable/observable over Mobile Control gets a small **messenger** +class deriving from `MessengerBase` (`PepperDash.Essentials.AppServer.Messengers`): + +```csharp +public class IHasStartMeetingMessenger : MessengerBase +{ + // ... + protected override void RegisterActions() + { + AddAction("/fullStatus", (id, content) => SendFullStatus(id)); + + AddAction("/startMeeting", (id, content) => + { + var msg = content.ToObject>(); + _startMeeting.StartMeeting(msg?.Value ?? _startMeeting.DefaultMeetingDurationMin); + }); + + AddAction("/leaveMeeting", (id, content) => _startMeeting.LeaveMeeting()); + } +} +``` + +* The messenger is constructed with a **base `MessagePath`** (e.g. `/device/zoomRoom1`) and calls + `AddAction(subPath, handler)` for each supported action, relative to that base path. +* `RegisterWithAppServer(IMobileControl appServerController)` registers the messenger's + `HandleMessage` callback with the system controller via `IMobileControl.AddAction(...)`. +* When an incoming message's `type` starts with the messenger's `MessagePath`, `HandleMessage` + strips that prefix (leaving e.g. `/startMeeting`) and dispatches to the matching registered + action. The sending client is automatically added to that messenger's subscriber list, so it + receives future unsolicited feedback from it. +* To push state out, messengers call `PostStatusMessage(DeviceStateMessageBase message, clientId)` - + omitting `clientId` broadcasts to every subscribed client; passing one targets a single client + (e.g. replying to a `/fullStatus` request). + +### How devices opt in + +`EssentialsDevice.CustomActivate()` calls a virtual `CreateMobileControlMessengers()` hook once all +devices have activated. A device (or a room) overrides this method to look up the `IMobileControl` +device and construct/register whichever messengers it needs: + +```csharp +protected override void CreateMobileControlMessengers() +{ + var mc = DeviceManager.AllDevices.OfType().FirstOrDefault(); + if (mc == null) return; + + var messenger = new IHasStartMeetingMessenger("zoomRoom1-startMeeting", "/device/zoomRoom1", this); + messenger.RegisterWithAppServer(mc); +} +``` + +Rooms typically implement `IMobileControlRoomMessenger` and register a room-level messenger at a +`/room/{roomKey}` path (via a `MobileControlBridgeBase`/`MobileControlEssentialsRoomBridge`), in +addition to any per-device messengers. + +## Client-side architecture (React) + +`@pepperdash/mobile-control-react-app-core` provides the client building blocks used by apps like +[beincourt-pv2-react-app](https://github.com/pepperdash-beincourt/beincourt-pv2-react-app): + +### Connection lifecycle + +The real work happens in a Redux middleware (`src/lib/store/middleware/websocketMiddleware.ts`); +`WebsocketProvider`/`WebsocketContext` are a thin, backward-compatible React Context wrapper around +it that simply dispatches Redux actions (`wsConnect`, `wsSendMessage`, `wsReconnect`, ...). + +1. On mount, `wsConnect()` is dispatched. +2. The middleware reads `apiPath` from the app's local config (`_config.local.json`/ + `_config.default.json`) and the connection `token` (from the URL's `?token=` query param). +3. It calls `GET {apiPath}/ui/joinroom?token={token}` to validate the token and get room data + (including a `clientId`). +4. It opens a WebSocket at `{apiPath with ws(s) scheme}/ui/join/{token}?clientId={clientId}`. +5. On specific close codes (e.g. `4000` user code changed, `4002` room combination changed) it stops + auto-reconnecting and surfaces an error; otherwise it automatically retries. + +### Sending messages + +```ts +const { sendMessage } = useWebsocketContext(); +sendMessage('/device/zoomRoom1/startMeeting', { value: 30 }); +``` + +`sendMessage` serializes `{ type, clientId, content }` (the same envelope described above) and +sends it over the open WebSocket. + +### Receiving messages / state hydration + +Incoming messages are routed by their `type` prefix: + +| Prefix | Handling | +| --- | --- | +| `/system/*` | Internal system messages (user code, touchpanel key, room-combination/device-interface changes, initial sync complete, ...) | +| `/event/*` | Dispatched to any handlers registered via `addEventHandler(eventType, key, callback)` | +| `/room/*` | `dispatch(roomsActions.setRoomState(message))` | +| `/device/*` | `dispatch(devicesActions.setDeviceState(message))` | + +Device/room state lives in Redux slices keyed by device/room key, so any component can read the +latest known state for a given key. + +### The hook pattern + +Each supported device interface has a small hook that combines reading state with calling actions, +e.g. `useIHasStartMeeting`: + +```ts +export function useIHasStartMeeting(key: string): IHasStartMeetingReturn | undefined { + const { sendMessage } = useWebsocketContext(); + const state = useGetDevice(key); // reads devices[key] from Redux + + return useMemo(() => { + if (!state) return undefined; + const path = `/device/${key}`; + return { + state, + startMeeting: (durationMin?: number) => sendMessage(`${path}/startMeeting`, { value: durationMin }), + leaveMeeting: () => sendMessage(`${path}/leaveMeeting`, null), + }; + }, [key, sendMessage, state]); +} +``` + +Components use these hooks to read live device state and call device actions without needing to +know anything about the WebSocket transport underneath. + +## Full round-trip example: starting a meeting + +1. **UI**: user clicks "Start Meeting"; component calls `startMeeting(30)` from `useIHasStartMeeting('zoomRoom1')`. +2. **Client → server**: `sendMessage('/device/zoomRoom1/startMeeting', { value: 30 })` sends + `{"type":"/device/zoomRoom1/startMeeting","clientId":"abc-123","content":{"value":30}}` over the WebSocket. +3. **Server routing**: the system controller matches `/device/zoomRoom1` against the registered + `IHasStartMeetingMessenger`'s `MessagePath`, strips the prefix to get `/startMeeting`, and invokes + its registered action, which calls `StartMeeting(30)` on the underlying device. +4. **Device fires feedback**: the device's meeting-started feedback fires. +5. **Server → clients**: the messenger calls `PostStatusMessage(...)`, broadcasting + `{"type":"/device/zoomRoom1","clientId":null,"content":{ ...state }}` to every client subscribed + to that messenger. +6. **Client receives**: the message's `type` starts with `/device/`, so + `dispatch(devicesActions.setDeviceState(message))` merges the new state into Redux under + `devices['zoomRoom1']`. +7. **UI updates**: `useIHasStartMeeting('zoomRoom1')`'s `state` reflects the new meeting info on the + next render. diff --git a/docs/docs/toc.yml b/docs/docs/toc.yml index 820de992..9ec7ee87 100644 --- a/docs/docs/toc.yml +++ b/docs/docs/toc.yml @@ -38,6 +38,8 @@ href: technical-docs/Plugins.md - name: Communication Basics href: technical-docs/Communication-Basics.md + - name: Mobile Control + href: technical-docs/Mobile-Control.md - name: Debugging href: technical-docs/Debugging.md - name: Feedback Classes