Message Flow

This page traces the complete path of messages through the CritterWatch system, from service state changes to browser updates and from operator actions back to services.

Telemetry Flow (Service → Browser)

1. Runtime Event Occurs

A monitored service triggers a Wolverine runtime event: a node starts, an agent is assigned, a projection shard advances, a circuit breaker trips. The Wolverine runtime calls the corresponding IWolverineObserver callback.

2. Observer Accumulates Changes

CritterWatchObserver receives the callback and writes a WolverineChange record to its BatchingChannel — a thread-safe, non-blocking queue. The callback returns immediately.

3. Periodic Publish (every 1 second)

A timer in the observer fires every second and:

1. Drains all accumulated changes from the BatchingChannel
2. Collects current endpoint states and shard states
3. Assembles a ServiceUpdates packet
4. Publishes to RabbitMQ via Wolverine's message bus

Even when no changes have occurred, a heartbeat ServiceUpdates is published to confirm the service is alive.

4. CritterWatch Receives

The CritterWatch server's Wolverine listener receives the ServiceUpdates message from the critterwatch RabbitMQ queue. The queue is configured as Sequential() to preserve ordering within a service.

5. Handler Pipeline

The ServiceUpdatesHandler processes the packet:

public static async Task Handle(
    ServiceUpdates updates,
    IDocumentSession session,
    IMessageBus bus)
{
    // Translate changes to domain events
    var events = TranslateToEvents(updates);

    // Append to the service's event stream in Marten
    session.Events.Append(updates.ServiceName, events);
    await session.SaveChangesAsync();

    // Relay to SignalR for live browser updates
    await bus.PublishAsync(new RelayServiceUpdate(updates));
}

6. Marten Projection

The ServiceSummaryProjection applies the appended events to the ServiceSummary snapshot document. This happens inline (synchronous) for ServiceSummary updates, ensuring queries always return up-to-date data.

7. SignalR Relay

A relay handler sends the update to all connected SignalR clients:

public static async Task Handle(
    RelayServiceUpdate relay,
    IHubContext<CommunicationHub> hub)
{
    await hub.Clients.All.SendAsync("serviceUpdated", relay.Updates);
}

8. Browser Updates

The Vue frontend's SignalR client receives the serviceUpdated message. The relayToStore.ts module routes it to the appropriate Pinia store action, which updates reactive state. Vue's reactivity system updates the UI automatically.

Service event → Observer (< 1ms) → BatchingChannel → Publish (≤ 1s)
→ RabbitMQ → CritterWatch handler → Marten + SignalR relay
→ Browser (< 100ms network)
→ Total: typically 1–2 seconds end-to-end

Command Flow (Browser → Service)

1. Operator Triggers Command

The operator clicks a button in the CritterWatch UI (e.g., "Replay Dead Letters"). The Vue component calls the HTTP API:

POST /api/critterwatch/dead-letters/replay
{ "serviceName": "trip-service", "messageType": "BookTrip" }

2. HTTP Handler Validates and Dispatches

The Wolverine HTTP handler validates the request and dispatches a command message to the target service's queue:

[WolverinePost("/api/critterwatch/dead-letters/replay")]
public static async Task<IResult> Replay(
    ReplayRequest request,
    IMessageBus bus)
{
    await bus.SendAsync(new ReplayMessages(
        request.ServiceName,
        request.MessageType));
    return Results.Accepted();
}

The HTTP handler returns 202 Accepted immediately — it does not wait for the service to process the command.

3. RabbitMQ Routes Command

Wolverine routes the ReplayMessages command to the RabbitMQ exchange/queue configured for the target service. The routing is based on the service name registered when AddCritterWatchMonitoring() was called.

4. Service Receives Command

The target service's Wolverine listener receives the ReplayMessages command. The Wolverine.CritterWatch library has registered handlers for all command types. The replay handler queries the dead-letter store and re-enqueues matching messages.

5. Feedback via Telemetry

The service does not send a direct response to CritterWatch. Instead, the result of the command is visible through the normal telemetry stream:

• DLQ count drops in the next ServiceUpdates publish
• Timeline entry appears ("Replayed N messages of type BookTrip")
• Alert auto-resolves if the DLQ count drops below threshold

This asynchronous feedback loop is a deliberate design choice — it means the UI stays responsive and commands are fire-and-forget.

Sequence Diagram