Observability¶

github-runner provides structured logging, Prometheus metrics, and health endpoints for production monitoring.

Logging¶

Configuration¶

[global]
log_level = "info"     # debug, info, warn, error
log_format = "json"    # json, text

CLI flags override the config file:

github-runner start --log-level debug --log-format text

Structured logging¶

All logging uses Go's log/slog package with consistent field names:

Example JSON log output:

{
  "time": "2026-03-03T12:00:00Z",
  "level": "INFO",
  "msg": "job completed",
  "job_id": 12345,
  "repository": "nficano/github-runner",
  "workflow": "CI",
  "duration": "45.2s",
  "component": "worker"
}

Component loggers¶

Each subsystem creates a child logger with its component name:

logger := log.WithComponent(parentLogger, "poller")
logger := log.WithJobContext(parentLogger, jobID, repo, workflow)

Secret masking in logs¶

The MaskingHandler wraps the slog handler to redact secret values from log output. It masks:

• Log message text
• String attribute values
• Attributes within nested groups

This is enabled automatically when secrets are registered with the masker via log.SetupWithMask().

Metrics¶

Endpoint¶

Prometheus metrics are served on a configurable address:

[global]
metrics_listen = "127.0.0.1:9252"

Scrape the /metrics path:

curl http://127.0.0.1:9252/metrics

Available metrics¶

Job metrics¶

Step metrics¶

Cache metrics¶

Infrastructure metrics¶

Runtime metrics¶

The runtime collector exports Go process metrics:

Custom registry¶

Metrics use a private Prometheus registry to avoid polluting the global default registry. This prevents conflicts when embedding github-runner as a library.

Prometheus scrape configuration¶

scrape_configs:
  - job_name: "github-runner"
    static_configs:
      - targets: ["127.0.0.1:9252"]
    scrape_interval: 15s

Grafana dashboard¶

Key panels to create:

• Job throughput — rate(github_runner_jobs_total[5m]) by pool and status
• Active jobs — github_runner_jobs_active by pool
• Job duration p99 — histogram_quantile(0.99, github_runner_job_duration_seconds)
• Error rate — rate(github_runner_job_errors_total[5m])
• Poll latency — histogram_quantile(0.95, github_runner_poll_duration)
• Cache hit rate — github_runner_cache_hit_ratio
• Goroutines — github_runner_goroutines
• Memory — github_runner_heap_alloc_bytes

Health endpoints¶

Configuration¶

[global]
health_listen = "127.0.0.1:8484"

Liveness: /healthz¶

Returns 200 OK if the process is running. Use this for Kubernetes liveness probes.

curl http://127.0.0.1:8484/healthz

{
  "status": "ok"
}

Readiness: /readyz¶

Returns 200 OK when the runner is ready to accept jobs. Returns 503 Service Unavailable during startup or shutdown drain.

curl http://127.0.0.1:8484/readyz

{
  "status": "ok",
  "checks": {
    "github_api": "ok",
    "disk_space": "ok",
    "executor": "ok"
  }
}

Registered checks¶

Kubernetes probes¶

livenessProbe:
  httpGet:
    path: /healthz
    port: 8484
  initialDelaySeconds: 5
  periodSeconds: 10

readinessProbe:
  httpGet:
    path: /readyz
    port: 8484
  initialDelaySeconds: 10
  periodSeconds: 5

Readiness lifecycle¶

1. Startup — Health server starts, readiness is false.
2. Ready — After pools are initialised, SetReady(true) is called.
3. Shutdown — On receiving SIGTERM/SIGINT, SetReady(false) is called immediately to stop receiving traffic while in-flight jobs drain.

pprof¶

The metrics server includes pprof endpoints for runtime profiling at /debug/pprof/. These are useful for diagnosing performance issues:

go tool pprof http://127.0.0.1:9252/debug/pprof/heap
go tool pprof http://127.0.0.1:9252/debug/pprof/goroutine

Alerting recommendations¶