feat: add Mobile Control documentation and update table of contents

This commit is contained in:
Neil Dorin 2026-07-23 22:16:55 -06:00
parent 5a183e7333
commit 61b5ca6297
3 changed files with 349 additions and 9 deletions

View file

@ -12,12 +12,18 @@ When routes are to be executed, Essentials will use this connection graph to dec
### Classes Referenced
* `PepperDash.Essentials.Core.Routing.IRoutingSource`
* `PepperDash.Essentials.Core.Routing.IRoutingOutputs`
* `PepperDash.Essentials.Core.Routing.IRoutingInputs`
* `PepperDash.Essentials.Core.Routing.IRoutingInputsOutputs`
* `PepperDash.Essentials.Core.Routing.IRoutingSinkNoSwitching`
* `PepperDash.Essentials.Core.Routing.IRoutingSinkWithSwitching`
* `PepperDash.Essentials.Core.IRoutingSource`
* `PepperDash.Essentials.Core.IRoutingOutputs`
* `PepperDash.Essentials.Core.IRoutingInputs`
* `PepperDash.Essentials.Core.IRoutingMidpoint`
* `PepperDash.Essentials.Core.IRoutingMidpointWithFeedback`
* `PepperDash.Essentials.Core.IRoutingSinkWithFeedback`
> **Note:** As of the v3 routing refactor, the older `IRoutingInputsOutputs`, `IRouting`,
> `IRoutingWithFeedback`, `IRoutingWithClear`, `IRoutingSink`, `IRoutingSinkWithInputPort`,
> `IRoutingSinkNoSwitching`, `IRoutingSinkWithSwitching`, and `IRoutingSinkWithSwitchingWithInputPort`
> interfaces have been removed and consolidated - see [Routing interfaces (v3)](#routing-interfaces-v3)
> below for the current mapping.
## Example system, a simple presentation system
@ -56,11 +62,12 @@ Source devices in Essentials must implement `IRoutingOutputs` or `IRoutingSource
A midpoint is a device in the middle of the signal chain. Typically a switcher, matrix or otherwise. Examples: DM chassis; DM-TX; DM-RMC; A video codec. These devices will have input and output ports.
Midpoint devices must implement `IRoutingInputsOutputs`. Midpoints with switching must implement `IRouting`.
Passthrough midpoints (no active switching) must implement `IRoutingMidpoint`. Midpoints that actively
switch and report their current routes must implement `IRoutingMidpointWithFeedback`.
#### Sink
A sink is a device at the end of a full signal path. For example, a display, amplifier, encoder, etc. Sinks typically contain only input ports. They may or may not have switching, like a display with several inputs. Classes defining sink devices must implement `IRoutingSinkNoSwitching` or `IRoutingSinkWithSwitching`.
A sink is a device at the end of a full signal path. For example, a display, amplifier, encoder, etc. Sinks typically contain only input ports. They may or may not have switching, like a display with several inputs. Classes defining sink devices must implement `IRoutingSinkWithFeedback`.
#### Tie-line
@ -78,4 +85,95 @@ A tie-line is a logical representation of a physical cable connection between tw
### Interfaces
Todo: Define Interfaces IRouting, IRoutingOutputs, IRoutingInputs
#### Routing interfaces (v3)
The v3 routing refactor consolidated several older, overlapping interfaces into two feedback-aware
interfaces. If you're updating a device implemented against the pre-v3 API, use this table to find
its replacement:
| Old interface (pre-v3, removed) | Current interface |
| --- | --- |
| `IRoutingInputsOutputs` | `IRoutingMidpoint` |
| `IRouting`, `IRoutingWithFeedback`, `IRoutingWithClear` | `IRoutingMidpointWithFeedback` |
| `IRoutingSink`, `IRoutingSinkWithInputPort`, `IRoutingSinkWithSwitching`, `IRoutingSinkWithSwitchingWithInputPort` | `IRoutingSinkWithFeedback` |
**Base building-block interfaces** (unchanged):
* `IRoutingSource` - marker interface for a device that originates a signal path; extends `IRoutingOutputs`.
* `IRoutingOutputs` - exposes `RoutingPortCollection<RoutingOutputPort> OutputPorts`.
* `IRoutingInputs` - exposes `RoutingPortCollection<RoutingInputPort> InputPorts`.
**Midpoint interfaces:**
* `IRoutingMidpoint` - a passthrough device with both input and output ports but no active switching. Extends `IRoutingInputs` and `IRoutingOutputs`.
* `IRoutingMidpointWithFeedback` - a midpoint that actively switches and reports feedback. Extends `IRoutingMidpoint` and adds:
* `void ExecuteSwitch(object inputSelector, object outputSelector, eRoutingSignalType signalType)`
* `void ClearRoute(object outputSelector, eRoutingSignalType signalType)`
* `List<RouteSwitchDescriptor> CurrentRoutes { get; }`
* `event RouteChangedEventHandler RouteChanged`
**Sink interface:**
* `IRoutingSinkWithFeedback` - a routing endpoint device that can switch inputs and provides feedback. Extends `IRoutingInputs`, `IKeyName`, and `ICurrentSources` (see below), and adds:
* `void ExecuteSwitch(object inputSelector)`
* `RoutingInputPort CurrentInputPort { get; }`
* `event InputChangedEventHandler InputChanged`
#### Current source tracking (`ICurrentSources`)
`ICurrentSources` is a new interface (implemented by `IRoutingSinkWithFeedback`) that reports which
source is actually feeding a sink right now, per signal type:
* `Dictionary<eRoutingSignalType, IRoutingSource> CurrentSources { get; }`
* `Dictionary<eRoutingSignalType, string> CurrentSourceKeys { get; }`
* `event EventHandler<CurrentSourcesChangedEventArgs> CurrentSourcesChanged`
A new `RoutingFeedbackManager` subscribes to `RouteChanged` on every `IRoutingMidpointWithFeedback`
and `InputChanged` on every `IRoutingSinkWithFeedback` in the system, traces each change back through
the tie-line graph to the originating source (debounced ~500ms), and keeps each sink's
`ICurrentSources` state up to date automatically - developers no longer need to manually wire up
source tracking per device.
#### Multiview layout support
For decoders that can display several independently-routable tiles at once (e.g. a WyreStorm
NetworkHD-style multiview decoder), two additional interfaces are available:
* `IRoutingSinkWithLayouts` - extends `IRoutingSource` and exposes `Dictionary<int, IRoutingSinkWithFeedback> WindowTileSinks`, one per-tile "virtual sink" that can be routed independently like any other `IRoutingSinkWithFeedback`.
* `IRoutingSinkWithLayoutState` - extends `IRoutingSinkWithLayouts` and additionally reports the current canvas/tile geometry via `MultiviewLayoutState CurrentLayout { get; }` and `event EventHandler<MultiviewLayoutStateEventArgs> LayoutChanged`.
`MultiviewLayoutState` is a JSON-serializable, product-agnostic model of the canvas: `CanvasWidth`,
`CanvasHeight`, and a list of `MultiviewTileState` entries (`TileNumber`, `TileSinkKey`, `X`, `Y`,
`Width`, `Height`, `ZOrder`, `SourceDeviceKey`) - suitable for driving a routing diagram UI.
#### Real-time routing feedback over WebSocket
Essentials Core includes a built-in WebSocket service, `RoutingFeedbackWebsocket`
(`PepperDash.Essentials.Core.Web`), that broadcasts routing state changes to any connected client in
real time - this is what powers the live routing diagram in the
[Essentials Web Config App](https://github.com/PepperDash/essentials-web-config-app) and
[essentials-devtools](https://github.com/PepperDash/essentials-devtools).
A client starts (or reuses) a session by sending an HTTP `GET` to the `routingFeedbackSession` API
route (e.g. `https://[processor-ip]/cws/[appId]/api/routingFeedbackSession`), which starts the
WebSocket server on a random high port (if not already running), forwards that port on the CS LAN
adapter if present, and returns a JSON body with the connection URL(s):
```json
{ "url": "wss://[processor-ip]:[port]/routing/join/", "fallbackUrl": "wss://..." }
```
Once connected, the client receives a `snapshot` message describing every midpoint route, sink
input, and multiview layout currently active, followed by incremental update messages (each
debounced ~200ms) as things change:
| Message `type` | Sent when | Payload |
| --- | --- | --- |
| `snapshot` | On connect | `midpointRoutes`, `sinkRoutes`, `layouts` for the whole system |
| `midpointRouteChanged` | An `IRoutingMidpointWithFeedback`'s `RouteChanged` fires | `deviceKey`, `routes: [{ inputPortKey, outputPortKey, signalType }]` |
| `sinkInputChanged` | An `IRoutingSinkWithFeedback`'s `InputChanged` fires | `deviceKey`, `inputPortKey`, `sourceDeviceKey`, `signalType` |
| `layoutChanged` | An `IRoutingSinkWithLayoutState`'s `LayoutChanged` fires | `deviceKey`, `layout` (a `MultiviewLayoutState`) |
Per-tile sinks belonging to an `IRoutingSinkWithLayouts` device are reported to clients as qualified
input ports on their parent device's node, rather than as separate graph nodes, so a routing diagram
can render a multiview decoder as a single device with per-tile inputs.

View file

@ -0,0 +1,240 @@
# Mobile Control: architecture and client communication
## TL;DR
Mobile Control is a WebSocket-based messaging layer built into Essentials. Devices expose small
"messenger" classes that register JSON message paths (e.g. `/device/{key}/startMeeting`) with a
central `MobileControlSystemController`. Clients (typically a React app built on
[`@pepperdash/mobile-control-react-app-core`](https://github.com/PepperDash/mobile-control-react-app-core))
connect over WebSocket with a per-room token, send the same JSON envelope to invoke device methods,
and receive unsolicited state-update messages that a Redux store uses to keep device/room state in
sync in real time.
## Server-side architecture
### Projects involved
* `PepperDash.Essentials.MobileControl` - the `MobileControlSystemController` device, its
WebSocket server (`WebSocketServer/MobileControlWebsocketServer.cs`), room bridges, and
touchpanel controller.
* `PepperDash.Essentials.MobileControl.Messengers` (namespace `PepperDash.Essentials.AppServer.Messengers`) -
the `MessengerBase` class and the library of built-in messengers for common device interfaces
(`IHasStartMeetingMessenger`, `IHasMeetingInfoMessenger`, `ICommunicationMonitorMessenger`,
`IRoutingMidpointWithFeedbackMessenger`, etc.), plus the message envelope types.
* `PepperDash.Essentials.Core.DeviceTypeInterfaces` - the public contracts:
`IMobileControl`, `IMobileControlMessenger`, `IMobileControlMessengerWithSubscriptions`,
`IMobileControlRoomMessenger`, `IMobileControlAction`, `IMobileControlMessage`, and the
touchpanel-controller interfaces.
### Direct Server vs. Edge Server
Mobile Control supports two (non-exclusive) transport modes, configured on the `MobileControlConfig`
device:
* **Direct Server** - `MobileControlWebsocketServer` runs an HTTPS WebSocket server directly on the
processor hardware. Clients on the same network connect straight to the processor - lowest
latency, no external dependency. This is what `mobile-control-react-app-core`'s local dev flow
and the deployed `mcUserApp` React app use.
* **Edge Server** (API server / cloud gateway) - the processor instead makes an *outbound* WebSocket
connection to an external Mobile Control server, which relays messages between it and remote
clients. Useful when clients can't reach the processor directly (e.g. no local network access).
Both modes can be enabled simultaneously; outgoing messages are queued and sent down whichever
transport(s) are active.
### The WebSocket endpoint (Direct Server)
`MobileControlWebsocketServer` (`WebSocketServer/MobileControlWebsocketServer.cs`) hosts the
Direct Server:
* **Path**: `/mc/api/ui/join/` (`_wsPath`)
* **Port**: `50000 + <program slot number>` by default (e.g. program slot 2 → port `50002`), or a
custom port from config
* **User app**: static files are served from `/user/programX/mcUserApp` (`_appPath`) at the base
href `/mc/app` (`_userAppBaseHref`) - this is exactly the `mcUserApp` deployment directory used
when [deploying](../Get-started.md) a built React app.
### Connecting and authentication (tokens)
A client doesn't connect straight to the WebSocket - it first needs a **token** identifying which
room/UI-client slot it's joining:
1. **Get a token** - on the processor console, run:
```
mobileinfo:[programSlot]
```
This prints the Direct Server port, any existing UI-client tokens, and full connect URLs (e.g.
`http://[ip]:[port]/mc/app?token=[token]`). New tokens can also be added dynamically
(`mobileadduiclient:[programSlot] [roomKey]`) - see the console command's help text for details.
2. **Join a room** - the client calls `GET {apiPath}/ui/joinroom?token={token}` over HTTPS, which
validates the token and returns room data including a generated `clientId`.
3. **Open the WebSocket** - the client connects to
`wss://[processor-ip]:[port]/mc/api/ui/join/{token}?clientId={clientId}`.
Each connected client is tracked as a `UiClient`; disconnecting removes its subscriptions from
every messenger it was subscribed to.
### The message envelope
Every message sent in either direction is the same simple JSON envelope
(`MobileControlMessage`, namespace `PepperDash.Essentials.AppServer.Messengers`):
```json
{
"type": "/device/zoomRoom1/startMeeting",
"clientId": "abc-123",
"content": { "value": 30 }
}
```
* `type` - a path identifying the target device/room and action (client → server), or the
device/room whose state is being reported (server → client).
* `clientId` - which client sent the message, or (server → client) which single client a targeted
reply is meant for; omitted/ignored for broadcasts to all subscribers.
* `content` - a `JToken` payload. For simple values, `MobileControlSimpleContent<T>` wraps a single
`value` property (as in the example above); for state updates, `content` is the device's full
serialized state object.
### The messenger pattern
Each device that wants to be controllable/observable over Mobile Control gets a small **messenger**
class deriving from `MessengerBase` (`PepperDash.Essentials.AppServer.Messengers`):
```csharp
public class IHasStartMeetingMessenger : MessengerBase
{
// ...
protected override void RegisterActions()
{
AddAction("/fullStatus", (id, content) => SendFullStatus(id));
AddAction("/startMeeting", (id, content) =>
{
var msg = content.ToObject<MobileControlSimpleContent<uint>>();
_startMeeting.StartMeeting(msg?.Value ?? _startMeeting.DefaultMeetingDurationMin);
});
AddAction("/leaveMeeting", (id, content) => _startMeeting.LeaveMeeting());
}
}
```
* The messenger is constructed with a **base `MessagePath`** (e.g. `/device/zoomRoom1`) and calls
`AddAction(subPath, handler)` for each supported action, relative to that base path.
* `RegisterWithAppServer(IMobileControl appServerController)` registers the messenger's
`HandleMessage` callback with the system controller via `IMobileControl.AddAction<T>(...)`.
* When an incoming message's `type` starts with the messenger's `MessagePath`, `HandleMessage`
strips that prefix (leaving e.g. `/startMeeting`) and dispatches to the matching registered
action. The sending client is automatically added to that messenger's subscriber list, so it
receives future unsolicited feedback from it.
* To push state out, messengers call `PostStatusMessage(DeviceStateMessageBase message, clientId)` -
omitting `clientId` broadcasts to every subscribed client; passing one targets a single client
(e.g. replying to a `/fullStatus` request).
### How devices opt in
`EssentialsDevice.CustomActivate()` calls a virtual `CreateMobileControlMessengers()` hook once all
devices have activated. A device (or a room) overrides this method to look up the `IMobileControl`
device and construct/register whichever messengers it needs:
```csharp
protected override void CreateMobileControlMessengers()
{
var mc = DeviceManager.AllDevices.OfType<IMobileControl>().FirstOrDefault();
if (mc == null) return;
var messenger = new IHasStartMeetingMessenger("zoomRoom1-startMeeting", "/device/zoomRoom1", this);
messenger.RegisterWithAppServer(mc);
}
```
Rooms typically implement `IMobileControlRoomMessenger` and register a room-level messenger at a
`/room/{roomKey}` path (via a `MobileControlBridgeBase`/`MobileControlEssentialsRoomBridge`), in
addition to any per-device messengers.
## Client-side architecture (React)
`@pepperdash/mobile-control-react-app-core` provides the client building blocks used by apps like
[beincourt-pv2-react-app](https://github.com/pepperdash-beincourt/beincourt-pv2-react-app):
### Connection lifecycle
The real work happens in a Redux middleware (`src/lib/store/middleware/websocketMiddleware.ts`);
`WebsocketProvider`/`WebsocketContext` are a thin, backward-compatible React Context wrapper around
it that simply dispatches Redux actions (`wsConnect`, `wsSendMessage`, `wsReconnect`, ...).
1. On mount, `wsConnect()` is dispatched.
2. The middleware reads `apiPath` from the app's local config (`_config.local.json`/
`_config.default.json`) and the connection `token` (from the URL's `?token=` query param).
3. It calls `GET {apiPath}/ui/joinroom?token={token}` to validate the token and get room data
(including a `clientId`).
4. It opens a WebSocket at `{apiPath with ws(s) scheme}/ui/join/{token}?clientId={clientId}`.
5. On specific close codes (e.g. `4000` user code changed, `4002` room combination changed) it stops
auto-reconnecting and surfaces an error; otherwise it automatically retries.
### Sending messages
```ts
const { sendMessage } = useWebsocketContext();
sendMessage('/device/zoomRoom1/startMeeting', { value: 30 });
```
`sendMessage` serializes `{ type, clientId, content }` (the same envelope described above) and
sends it over the open WebSocket.
### Receiving messages / state hydration
Incoming messages are routed by their `type` prefix:
| Prefix | Handling |
| --- | --- |
| `/system/*` | Internal system messages (user code, touchpanel key, room-combination/device-interface changes, initial sync complete, ...) |
| `/event/*` | Dispatched to any handlers registered via `addEventHandler(eventType, key, callback)` |
| `/room/*` | `dispatch(roomsActions.setRoomState(message))` |
| `/device/*` | `dispatch(devicesActions.setDeviceState(message))` |
Device/room state lives in Redux slices keyed by device/room key, so any component can read the
latest known state for a given key.
### The hook pattern
Each supported device interface has a small hook that combines reading state with calling actions,
e.g. `useIHasStartMeeting`:
```ts
export function useIHasStartMeeting(key: string): IHasStartMeetingReturn | undefined {
const { sendMessage } = useWebsocketContext();
const state = useGetDevice<IHasStartMeetingState>(key); // reads devices[key] from Redux
return useMemo(() => {
if (!state) return undefined;
const path = `/device/${key}`;
return {
state,
startMeeting: (durationMin?: number) => sendMessage(`${path}/startMeeting`, { value: durationMin }),
leaveMeeting: () => sendMessage(`${path}/leaveMeeting`, null),
};
}, [key, sendMessage, state]);
}
```
Components use these hooks to read live device state and call device actions without needing to
know anything about the WebSocket transport underneath.
## Full round-trip example: starting a meeting
1. **UI**: user clicks "Start Meeting"; component calls `startMeeting(30)` from `useIHasStartMeeting('zoomRoom1')`.
2. **Client → server**: `sendMessage('/device/zoomRoom1/startMeeting', { value: 30 })` sends
`{"type":"/device/zoomRoom1/startMeeting","clientId":"abc-123","content":{"value":30}}` over the WebSocket.
3. **Server routing**: the system controller matches `/device/zoomRoom1` against the registered
`IHasStartMeetingMessenger`'s `MessagePath`, strips the prefix to get `/startMeeting`, and invokes
its registered action, which calls `StartMeeting(30)` on the underlying device.
4. **Device fires feedback**: the device's meeting-started feedback fires.
5. **Server → clients**: the messenger calls `PostStatusMessage(...)`, broadcasting
`{"type":"/device/zoomRoom1","clientId":null,"content":{ ...state }}` to every client subscribed
to that messenger.
6. **Client receives**: the message's `type` starts with `/device/`, so
`dispatch(devicesActions.setDeviceState(message))` merges the new state into Redux under
`devices['zoomRoom1']`.
7. **UI updates**: `useIHasStartMeeting('zoomRoom1')`'s `state` reflects the new meeting info on the
next render.

View file

@ -38,6 +38,8 @@
href: technical-docs/Plugins.md
- name: Communication Basics
href: technical-docs/Communication-Basics.md
- name: Mobile Control
href: technical-docs/Mobile-Control.md
- name: Debugging
href: technical-docs/Debugging.md
- name: Feedback Classes