StreamTrace BETA
Documentation menu

Using Stream Trace

Throughput & tracing

Analyze stream and consumer throughput, and follow messages across services with traces.

Throughput

Stream Trace periodically samples stream and consumer rates and stores the samples so you can chart throughput and spot trends over time.

  • Time windows - view the last 15m, 1h, 4h, 1d, 2d, 7d, and longer, up to the retention limit.
  • Rates - message ingress and egress, with a per-consumer breakdown that respects each consumer’s filter subjects.
  • Percentiles - p50–p99 message-size percentiles to understand payload distribution, not just averages.
  • Trends - Stream Trace flags whether throughput is growing or stable using a configurable tolerance (ST_THROUGHPUT_TREND_TOLERANCE_PERCENT).

Sampling cadence and retention are configurable - see ST_THROUGHPUT_SAMPLE_INTERVAL_MS and ST_THROUGHPUT_SAMPLE_RETENTION_MS in the Configuration reference.

Distributed tracing

When your messages carry a correlation ID, Stream Trace can correlate matching message occurrences across streams and consumers into a single trace.

Application change required. Publishers must attach and propagate one stable trace ID using a header in ST_TRACE_ID_HEADERS. Stream Trace does not inject or propagate tracing headers. Set application-consumer SampleFrequency / sample_freq when you also want sampled ACK and latency enrichment. Follow the application integration checklist before relying on traces.

  • Trace headers - Stream Trace reads trace IDs from the headers listed in ST_TRACE_ID_HEADERS (default Nats-Trace-Id,X-Trace-Id).
  • Trace view - enter a known trace ID to see up to 500 correlated spans, the streams and consumers involved, and available ACK, NAK, max-delivery, terminated, or unknown observations.
  • From a message - messages that carry a trace ID link straight to their trace.
  • Retention - trace observations are kept for ST_TRACE_OBSERVATION_RETENTION_MS (default 7 days).

Traces are filtered by access: you only see spans for streams your teams can access. This is application-level correlation by a known ID, not NATS network-hop tracing, automatic trace discovery, or a service topology map. Consumer enrichment depends on JetStream ACK metric sampling and may be unavailable.