Logging and Observability

Lesson 3 of 7

Metrics and Time-Series Data

Logs tell you about individual events. Metrics aggregate numbers over time, letting you see trends, set alerts, and build dashboards without drowning in individual log lines.

The three basic metric types

TypeBehaviorExample
CounterOnly ever goes up (until reset)Total requests served
GaugeGoes up or down freelyCurrent memory usage, active connections
HistogramBuckets observations to compute distributionsRequest latency (so you can ask for p50/p95/p99)

A counter for http_requests_total doesn't tell you the request rate, but a monitoring system can compute the rate of change over time from a raw counter, which is usually what you actually want to graph.

Prometheus-style metrics

A very common pattern (used by Prometheus and compatible with most modern monitoring stacks) is exposing metrics as plain text on an HTTP endpoint that a collector periodically scrapes:

# HELP http_requests_total Total HTTP requests
# TYPE http_requests_total counter
http_requests_total{method="GET",status="200"} 84213
http_requests_total{method="GET",status="500"} 12

# HELP http_request_duration_seconds Request latency
# TYPE http_request_duration_seconds histogram
http_request_duration_seconds_bucket{le="0.1"} 79000
http_request_duration_seconds_bucket{le="0.5"} 83900
http_request_duration_seconds_bucket{le="+Inf"} 84225

The {method="GET",status="200"} part is a set of labels, letting you slice the same metric by dimensions (which method, which status code) without defining a separate metric name for every combination.

The cardinality trap

Labels are powerful, but each unique combination of label values creates a separate time series stored and tracked individually. Adding a label like userId to a metric with millions of users creates millions of time series, this is called a cardinality explosion, and it can silently overwhelm a monitoring system's storage and query performance.

WARNING

Never put unbounded, high-cardinality values (user IDs, email addresses, raw URLs with query strings, request IDs) directly into a metric's labels. Those belong in logs and traces instead, metric labels should stay to a bounded, relatively small set of values (a handful of environments, a handful of status codes, a handful of regions).

📝 Metrics Quiz

Passing score: 70%
  1. 1.Which metric type only ever increases (until it resets)?

  2. 2.Which metric type lets you compute percentiles like p95 latency?

  3. 3.A gauge metric can go both up and down, like current memory usage.

  4. 4.A set of key-value pairs attached to a metric, like method and status, letting you slice it by dimension, is called a ____.

  5. 5.Adding a high-cardinality label like userId to a metric is a safe, recommended practice.

  6. 6.Adding unbounded label values that create millions of separate time series is called a cardinality ____.