Tracing, metrics, and health

Lily lets you observe listener, dispatcher, DI, and backplane state. Taking a snapshot does not publish it as an HTTP endpoint; your application supplies the probe or monitoring bridge.

Tracing choices#

Owned tracing is installed before eager Singleton services so their instruments bind to the selected provider. The external owner shuts down an external runtime. When using TraceConfig, add trace to the lilyrs feature list and import lilyrs::trace::TraceConfig. The feature includes the console default; file/OTLP exporters are selected through runtime configuration. There are no trace-file or trace-otlp features. Shared runtime setup is explained in HTTP observability.

Snapshots and dependency registration#

let dependencies = app.required_dependency_health();
dependencies.register_required("orders.store")?;
let health = app.health_snapshot()?;
let metrics = app.metrics_snapshot();
let local_connections = app.active_connection_count().await;

The application example demonstrates this order in a complete function. Complete required dependency registrations before start/close lifecycle ownership is claimed. A registration is not initially healthy; update it with mark_healthy, mark_degraded, or mark_unhealthy after a real probe result arrives.

Existing status updates may continue after new registration closes. The application updater is restricted to the dependency. prefix; it cannot mark the framework's listener or dispatcher checks healthy on the application's behalf.

Expose health#

Connect health_snapshot() to a separate HTTP route in the enclosing application or to a probe bridge. The WS builder does not automatically add /health, /ready, or Prometheus routes. Explicitly establish stop/join ownership for probe work as well; a raw spawned task is not automatically a managed background service. A periodic probe can be a builder-registered worker. Starting that worker does not automatically mark its dependency healthy.

A Required backplane outage affects readiness. Existing sockets are not necessarily closed solely for this reason; observe readiness separately from live connection processing. Backplane lists these cases.

Interpret counters correctly#

Metrics report connection, message, and backplane results through bounded counters. A cancellation request, actual pipeline result, exceeded deadline, and queued output are different events. Do not turn payloads, tokens, URLs, or user identities into unbounded metric labels.

active_connection_count() is the local count across all namespaces; closing connections remain visible until removed from the registry. It is not a global presence count. Tracing parentage passes from Upgrade into identity and scoped work.

Observe shutdown#

There is no public app.shutdown_report() API. Internal shutdown events are emitted under the lily_websocket::shutdown tracing target; facade imports do not rename that target. Worker/scope counters are included in the pre-flush checkpoint. An external subscriber may be needed to see the final outcome after owned telemetry closes; exporter delivery is not the same guarantee as task termination. Map observation metrics to the evidence milestones on the shutdown page.