Metrics, logs and traces Commercial Edition

This page lists what each Plane service exports once OpenTelemetry is enabled, and the queries that turn it into dashboards and answers. Metric names are given both as OpenTelemetry names and as the Prometheus names you see after the collector's prometheus exporter has rewritten them. Log and trace queries assume Loki and Tempo as deployed in Deploy a monitoring stack, but the field names are the same in any backend.

Service names in this page are the OTEL_SERVICE_NAME values the plane-enterprise chart sets per workload. If you deploy another way, substitute your own.

What each service exports

Service	Traces	Metrics	Logs	Transport
api, external-api	Yes	Yes	JSON on stdout, with trace IDs	OTLP gRPC or HTTP
worker, worker-importers, beat-worker	Yes	Yes	JSON on stdout, with trace IDs	OTLP gRPC or HTTP
automation-consumer, agent-consumer, webhook-consumer, outbox-poller	Partial	Yes	JSON on stdout, with trace IDs	OTLP gRPC or HTTP
live, live-exporter, silo	Client spans only	Runtime and outgoing HTTP	JSON on stdout, no trace IDs	OTLP gRPC or HTTP
space (server side)	Yes	No	Over OTLP	OTLP HTTP only
pi-api, pi-worker, pi-beat (Plane AI)	No	No	Over OTLP	OTLP HTTP only
plane-web, plane-admin, plane-space (browser, optional)	Yes	No	Over OTLP	OTLP HTTP only

Notes on the table:

• external-api runs the same code as api, so it emits the same metrics and spans under its own service name.
• The consumers and outbox-poller read from RabbitMQ, and the RabbitMQ client is not instrumented. Their database, Redis and HTTP spans still arrive, but as separate root spans rather than one span per message.
• live and silo are instrumented for traces, metrics and logs, but on current builds (Plane v4.0.1) only the outgoing side shows up: client spans and http.client.request.duration, plus the Node runtime gauges. Server spans, server request metrics, database pool metrics and the OTLP log bridge do not activate. Their logs reach Loki through Alloy like the Python services', without trace IDs.
• Only the Plane AI services export logs over OTLP. Every other service writes JSON to stdout, which is why the stack guide tails the containers with Alloy.
• Plane's product telemetry (plane_instance_* and plane_workspace_* gauges) goes to Plane's own endpoint on a separate pipeline. It does not reach your collector and is covered on Data collection and usage.

Metrics

The collector's prometheus exporter turns http.server.duration with unit ms into http_server_duration_milliseconds, adds _bucket, _sum and _count for histograms, _total for counters, and converts attribute keys from dots to underscores. With resource_to_telemetry_conversion on, every series also carries service_name, deployment_environment_name and whatever you put in observability.otel.resourceAttributes.

The Python and Node SDKs follow different versions of the HTTP semantic conventions. Python reports the older names in milliseconds. Node reports the stable names in seconds. The two never collide, but a dashboard that covers both needs two queries.

To see the exact names your instance produces, read the collector's endpoint from inside the cluster. The collector binds to the pod IP, so kubectl port-forward to it does not work:

kubectl run otel-curl --rm -i --restart=Never -n observability --image=curlimages/curl:8.11.1 \
  -- sh -c "curl -s otel-collector.observability.svc:8889/metrics | grep -E '^[a-z_]+' | cut -d'{' -f1 | sort -u"

API and workers (Python)

OpenTelemetry name	Prometheus name	Type	Labels	Emitted by
http.server.duration	http_server_duration_milliseconds_*	Histogram (ms)	http_method, http_status_code, http_target, http_scheme, http_flavor, http_host, http_server_name, net_host_name, net_host_port	api, external-api
http.server.active_requests	http_server_active_requests	Gauge	http_method, http_scheme, http_flavor, http_host, http_server_name	api, external-api
http.client.duration	http_client_duration_milliseconds_*	Histogram (ms)	http_method, http_status_code, http_host, net_peer_name, net_peer_port, http_scheme, http_flavor	Every Python service that calls out
flower.task.runtime.seconds	flower_task_runtime_seconds_*	Histogram (s)	task, worker	worker, worker-importers

http_target holds the Django route pattern, for example api/workspaces/<str:slug>/projects/, not the requested path. That keeps the label set small and makes per-route queries possible. The health endpoints /api/live/, /api/ready/ and /api/health/ are included, so a probe every 10 seconds shows up as traffic. Requests under /dav/ are excluded.

flower_task_runtime_seconds is recorded when a task finishes, whether it succeeded or failed. Failures are visible in traces and logs, not in this metric.

live and silo (Node)

OpenTelemetry name	Prometheus name	Type	Labels	Emitted by
http.client.request.duration	http_client_request_duration_seconds_*	Histogram (s)	http_request_method, http_response_status_code, server_address, server_port, error_type	live, silo
nodejs.eventloop.delay.p50, .p90, .p99, .min, .max, .mean, .stddev	nodejs_eventloop_delay_p99_seconds and so on	Gauge (s)	none	live, silo
nodejs.eventloop.utilization	nodejs_eventloop_utilization_ratio	Gauge (0 to 1)	none	live, silo
nodejs.eventloop.time	nodejs_eventloop_time_seconds_total	Counter (s)	nodejs_eventloop_state (active or idle)	live, silo
v8js.gc.duration	v8js_gc_duration_seconds_*	Histogram (s)	v8js_gc_type	live, silo
v8js.memory.heap.used, .limit	v8js_memory_heap_used_bytes, v8js_memory_heap_limit_bytes	Gauge (bytes)	v8js_heap_space_name	live, silo

Incoming request metrics (http.server.request.duration) and the pg pool metrics do not appear on current builds, even though the instrumentations are loaded. Latency for live and silo therefore comes from the event loop gauges above, from the container metrics below, or from span metrics on their client spans. Neither service emits process CPU or memory metrics.

Pods and nodes (kube-prometheus-stack)

These come from kube-state-metrics and the kubelet, not from Plane, and the queries further down rely on them:

Prometheus name	What it tells you
container_cpu_usage_seconds_total	CPU per container, as a rate
container_memory_working_set_bytes	Memory the kernel will not reclaim, the OOM basis
kube_pod_container_resource_limits	The limits you set, to compare against the two above
kube_pod_container_status_restarts_total	Restart count per container
kube_pod_container_status_last_terminated_reason	OOMKilled, Error and similar

Span metrics (Tempo)

With Tempo's metrics generator on, every span also becomes a series in Prometheus. These cover the parts of Plane that have spans but no native metric, such as Celery tasks and the consumers:

Prometheus name	Labels
traces_spanmetrics_calls_total	service, span_name, span_kind, status_code
traces_spanmetrics_latency_*	Same, as a histogram in milliseconds
traces_service_graph_request_total	client, server, used by the Grafana service map

Span metrics only count sampled traces. With the chart's default always_on sampler that is every trace. If you lower samplerArg, these series become an estimate.

What Plane does not emit

Plane has no metrics for queue depth, Celery task failures, database pool usage on the Python side, or anything about Postgres, Redis and RabbitMQ themselves. For those, add the standard exporters next to the dependencies: postgres_exporter, redis_exporter and the RabbitMQ Prometheus plugin all work with the Prometheus deployed in the stack guide.

Logs

API and workers

Every line is one JSON object with these fields:

Field	Value
levelname	DEBUG, INFO, WARNING, ERROR or CRITICAL
asctime	Timestamp
name	Logger name, see the table below
module	Python module that logged
message	The message
service_name	The service's OTEL_SERVICE_NAME. Present only when OpenTelemetry is enabled.
trace_id	32 hex characters, or empty outside a trace. Present only when OpenTelemetry is enabled.
span_id	16 hex characters, or empty. Present only when OpenTelemetry is enabled.
trace_flags	1 when the trace was sampled, else 0.

The logger name tells you which part of Plane spoke:

Logger	What logs there	Default level
plane.api.request	One line per HTTP request, see below	INFO
plane.api	View and serializer messages	INFO
plane.worker	Celery task messages	INFO
plane.exception	Unhandled exceptions with stack traces	ERROR
plane.webhook	Webhook delivery	INFO
plane.authentication	Sign-in, SSO and session events	INFO
plane.payments	License and billing	INFO
plane.silo, plane.external	Calls to silo and to external services	INFO
plane.migrations	Database migrations	INFO
celery	Celery itself (worker start, task received)	WARNING
django	Django itself	ERROR

LOG_LEVEL on the service changes the level of every plane.* logger. VERBOSE_LOGGING=1 raises celery to INFO and django to WARNING.

The request log line on plane.api.request is the most useful line Plane writes. Its message is METHOD path status, and it carries these extra fields:

Field	Value
method	HTTP method
path	Requested path
status_code	Response status
duration_ms	Wall time for the request
query_count	Number of SQL queries the request ran
user_id	Authenticated user, if any
remote_addr	Client address as seen by Django
user_agent	Client user agent
response_size	Response body size in bytes
plane_log_id	Unique ID for the line
request_body, response_body	Present on 4xx and 5xx only, with secrets redacted
exception_type	Present on 5xx

Its level encodes the outcome: ERROR for a 5xx, WARNING when the request took more than 1 second, otherwise INFO. GET / is not logged. The gunicorn access log is plain text, not JSON, and duplicates the same requests. Filter on name="plane.api.request" to get only the structured lines.

live and silo

Both use pino. Each line is JSON with level (a number: 30 info, 40 warn, 50 error, 60 fatal), time, msg, pid and hostname. Request lines have req.method, req.url, res.statusCode and responseTime. Errors are under err with message and stack. On current builds the lines carry no trace_id, and they are not exported over OTLP, so they reach Loki through Alloy with the namespace, pod, container and service_name labels and need | json to filter on a field.

Silo's import and sync workers add jobId and a traceId field of the form workspace:<id>:<provider>:import:<jobId>. That traceId is a job key, not an OpenTelemetry trace ID. Use jobId to follow one import.

Plane AI, space and the browser apps

pi-api, pi-worker and pi-beat send their Python log records over OTLP. Under the Helm chart each reports under its own service name (pi-api, pi-worker, pi-beat) with the message as the line body, severity_text in upper case (INFO, ERROR) and a plane.pi.process attribute. These are the only Plane logs that arrive with k8s_namespace_name and deployment_environment_name labels rather than Alloy's namespace and pod.

The space server and the three browser apps emit one log record per router event rather than free-form lines. The body is a short phrase such as middleware root (412ms) or loader routes/project-issues (88ms), the event name is on the record, and the attributes carry the detail:

Event	Attributes
react_router.navigate, .loader, .action, .middleware, .lazy, .fetch (browser)	react_router.route.id, react_router.route.pattern, react_router.url.from, react_router.url.to, duration_ms
react_router.request (space server)	http.request.method, url.full, duration_ms
Any of the above on failure	exception.type, exception.message, exception.stacktrace, severity ERROR

react_router.route.pattern with a duration_ms is the page-load timing for one route, as the user experienced it.

Labels in Loki

Logs reach Loki by two routes, and the labels differ:

• Lines tailed by Alloy carry namespace, pod, container and service_name. Everything in the JSON line is available after | json.
• Lines sent over OTLP (Plane AI, and the browser apps if enabled) carry service_name, k8s_namespace_name and deployment_environment_name as labels. The other attributes are structured metadata and can be filtered without a parser, for example | severity_text="ERROR".

service_name works for both, which is why the queries below use it.

Traces

API and workers

Span	Name	Key attributes
One server span per HTTP request	GET api/workspaces/<str:slug>/projects/	http.method, http.route, http.status_code, http.target, http.url, net.peer.ip
Task enqueued (producer)	apply_async/<task name>	celery.action, celery.task_name, messaging.message.id
Task executed (consumer)	run/<task name>	celery.action, celery.task_name, celery.state, celery.hostname, celery.delivery_info
Each SQL statement	SELECT, INSERT, UPDATE, ...	db.system = postgresql, db.statement, db.name, net.peer.name
Each Redis command	GET, SET, HGETALL, ...	db.system = redis, db.statement
Each outgoing HTTP call	GET, POST, ...	http.method, http.url, http.status_code

The span name uses the Django route pattern, so every request to one endpoint shares a name. Trace context travels through Celery message headers, so a request that enqueues a task and the task's execution appear in one trace. The SQL statement is recorded without parameter values.

live, silo and space

On current builds live and silo emit client spans only: one per outgoing HTTP call, named by method, with server.address and url.full. The incoming request, Express, Redis, Postgres and RabbitMQ spans the auto-instrumentation is meant to add do not appear. A trace that starts in the browser or the API therefore ends at the call into live or silo rather than continuing inside it. The space server emits one span per server-rendered request, named GET /spaces/<path>.

Browser apps

Browser tracing sends one client span per fetch call, named by method, plus CORS Preflight spans. Page loads and user interactions are not traced. Plane's own API client uses XHR rather than fetch, so most API calls from the web app do not appear as browser spans. Use the react_router.* log events above for page timings.

Plane AI

The pier process traces each AI run as a pier.run span with pier.run_id, pier.chat_id, pier.mode, pier.agent, pier.model and pier.workspace_id, and on completion pier.run.duration_ms and pier.run.tool_calls. Model and tool calls appear as child spans that follow the GenAI semantic conventions. Prompt content is not exported.

Queries for dashboards and debugging

Each query below is a panel you can paste into Grafana. The first group is the API's request rate, error rate and duration, which is the dashboard to build first.

API request rate, errors and latency (PromQL)

Request rate per second:

sum(rate(http_server_duration_milliseconds_count{service_name="api"}[5m]))

Error rate as a percentage. Use [45].. for client and server errors together, or 5.. for server errors only:

100 * sum(rate(http_server_duration_milliseconds_count{service_name="api", http_status_code=~"5.."}[5m]))
  / sum(rate(http_server_duration_milliseconds_count{service_name="api"}[5m]))

Latency percentiles in milliseconds. Repeat with 0.50 and 0.99:

histogram_quantile(0.95, sum by (le) (rate(http_server_duration_milliseconds_bucket{service_name="api"}[5m])))

p95 per route, for a table or a time series with one line per endpoint:

histogram_quantile(0.95, sum by (le, http_target) (rate(http_server_duration_milliseconds_bucket{service_name="api"}[5m])))

The 15 slowest routes right now:

topk(15, histogram_quantile(0.95, sum by (le, http_target, http_method) (rate(http_server_duration_milliseconds_bucket{service_name="api"}[5m]))))

Requests by status code:

sum by (http_status_code) (rate(http_server_duration_milliseconds_count{service_name="api"}[5m]))

Requests in flight, which climbs when the database or a downstream service stalls:

sum(http_server_active_requests{service_name="api"})

Outgoing calls from the API and workers by destination, to spot a slow webhook target or identity provider:

histogram_quantile(0.95, sum by (le, net_peer_name) (rate(http_client_duration_milliseconds_bucket[5m])))

Workers (PromQL)

Tasks completed per second by task name:

sum by (task) (rate(flower_task_runtime_seconds_count[5m]))

p95 task runtime by task name:

histogram_quantile(0.95, sum by (le, task) (rate(flower_task_runtime_seconds_bucket[5m])))

Failed tasks per second, from span metrics:

sum by (span_name) (rate(traces_spanmetrics_calls_total{service="worker", span_name=~"run/.*", status_code="STATUS_CODE_ERROR"}[5m]))

live and silo (PromQL)

p95 of their outgoing calls in seconds, by destination. For silo that is the integration provider, for live the Plane API:

histogram_quantile(0.95, sum by (le, service_name, server_address) (rate(http_client_request_duration_seconds_bucket{service_name=~"live|silo"}[5m])))

Event loop delay at p99. Sustained values above 100 ms mean the process is CPU bound and real-time collaboration will feel slow:

nodejs_eventloop_delay_p99_seconds{service_name=~"live|silo"}

Heap usage as a share of the limit. Above 0.9 the process is close to an out-of-memory crash:

sum by (service_name) (v8js_memory_heap_used_bytes) / sum by (service_name) (v8js_memory_heap_limit_bytes)

Pods (PromQL)

CPU per container as a share of its limit, for the Plane namespace:

sum by (container) (rate(container_cpu_usage_seconds_total{namespace="plane", container!=""}[5m]))
  / sum by (container) (kube_pod_container_resource_limits{namespace="plane", resource="cpu"})

Memory per container as a share of its limit:

sum by (container) (container_memory_working_set_bytes{namespace="plane", container!=""})
  / sum by (container) (kube_pod_container_resource_limits{namespace="plane", resource="memory"})

Restarts in the last hour, with the reason:

sum by (container) (increase(kube_pod_container_status_restarts_total{namespace="plane"}[1h])) > 0

kube_pod_container_status_last_terminated_reason{namespace="plane"} == 1

Logs (LogQL)

Slow API requests, with the trace ID to click through:

{service_name="api"} | json | name="plane.api.request" | duration_ms > 1000
  | line_format "{{.duration_ms}}ms {{.status_code}} {{.method}} {{.path}} queries={{.query_count}} trace={{.trace_id}}"

Server errors:

{service_name="api"} | json | name="plane.api.request" | status_code >= 500
  | line_format "{{.status_code}} {{.method}} {{.path}} {{.exception_type}} trace={{.trace_id}}"

Requests that ran many SQL queries. These are the N+1 candidates and the usual cause of a slow list view:

{service_name="api"} | json | name="plane.api.request" | query_count > 50
  | line_format "queries={{.query_count}} {{.duration_ms}}ms {{.method}} {{.path}} trace={{.trace_id}}"

Error rate per logger across the Python services, as a time series:

sum by (name) (rate({service_name=~"api|worker|worker-importers|beat-worker"} | json | levelname="ERROR" [5m]))

Failed Celery tasks with their stack traces:

{service_name=~"worker|worker-importers"} | json | levelname="ERROR" | name=~"plane.worker|plane.exception|celery"

Everything one import job did in silo:

{service_name="silo"} | jobId="<job id>"

Errors from the Node services, which arrive through Alloy as pino JSON:

{service_name=~"live|silo|live-exporter"} | json | level >= 50

Errors from Plane AI, which arrive over OTLP:

{service_name=~"pi-api|pi-worker|pi-beat"} | severity_text="ERROR"

Every log line from one trace, across every service:

{service_name=~".+"} |= "<trace id>"

Page-load time at p95 for one route, from the browser's root middleware event, which spans the whole transition:

quantile_over_time(0.95, {service_name="plane-web"} |= "middleware root"
  | react_router_route_pattern=":workspaceSlug/projects/:projectId/issues" | unwrap duration_ms [5m])

Traces (TraceQL)

API requests slower than two seconds:

{ resource.service.name = "api" && kind = server && duration > 2s }

All traces for one endpoint, by route pattern:

{ resource.service.name = "api" && span.http.route = "api/workspaces/<str:slug>/projects/" }

Celery tasks that failed:

{ resource.service.name = "worker" && name =~ "run/.*" && status = error }

Slow SQL statements, with the statement text in the span:

{ span.db.system = "postgresql" && duration > 500ms }

Requests that ran more than 50 SQL statements, which pairs with the query_count log query above:

{ resource.service.name = "api" && kind = server } | count() > 50

Slow calls from silo to an integration provider:

{ resource.service.name = "silo" && kind = client && duration > 5s }

AI runs for one workspace:

{ resource.service.name = "pier" && span.pier.workspace_id = "<workspace id>" }

From symptom to cause

Symptom	Look at	Usual cause
Pages load slowly	p95 per route, then the query_count > 50 log query, then the trace for one slow request	One endpoint doing many queries on a large project, or the database itself under load
5xx errors after an upgrade	The status_code >= 500 log query, the exception_type field, then plane.exception for the stack trace	A failed migration or a changed environment variable
Error rate climbs with no deploy	http_server_active_requests, then the pod memory and restart queries	A dependency (Postgres, Redis, RabbitMQ) is slow or restarting
Emails, notifications or webhooks do not arrive	Worker task rate, then the failed-task log query, then http_client_duration_milliseconds by net_peer_name	Worker not running, SMTP or webhook target rejecting, or a task raising
Imports or integration syncs are stuck	Silo logs by jobId, then silo http_client_request_duration_seconds by server_address	The external provider rate-limiting or returning errors
Real-time collaboration stops syncing	nodejs_eventloop_delay_p99_seconds and heap share for live, then live error logs	live CPU bound or out of memory, or Redis unreachable
AI features time out	pi-api and pi-worker error logs, then pier.run traces for the workspace	The model endpoint slow or rejecting, or the embedding store unavailable
A pod restarts in a loop	The restart and termination-reason queries, then that container's logs just before the restart	Memory limit too low for the workload, or a dependency missing at start

Starter alert rules

These rules use only the metrics above. Load them as a PrometheusRule in the observability namespace. The release: kps label is what the Prometheus Operator installed by the stack guide matches on. Adjust thresholds after a week of looking at your own numbers.

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: plane
  namespace: observability
  labels:
    release: kps
spec:
  groups:
    - name: plane
      rules:
        - alert: PlaneApiHighErrorRate
          expr: |
            100 * sum(rate(http_server_duration_milliseconds_count{service_name="api", http_status_code=~"5.."}[5m]))
              / sum(rate(http_server_duration_milliseconds_count{service_name="api"}[5m])) > 5
          for: 10m
          labels:
            severity: critical
          annotations:
            summary: "Plane API 5xx rate above 5% for 10 minutes"

        - alert: PlaneApiHighLatency
          expr: |
            histogram_quantile(0.95, sum by (le) (rate(http_server_duration_milliseconds_bucket{service_name="api"}[5m]))) > 2000
          for: 10m
          labels:
            severity: warning
          annotations:
            summary: "Plane API p95 latency above 2 s for 10 minutes"

        - alert: PlaneWorkerTaskFailures
          expr: |
            sum(rate(traces_spanmetrics_calls_total{service="worker", span_name=~"run/.*", status_code="STATUS_CODE_ERROR"}[10m])) > 0.1
          for: 10m
          labels:
            severity: warning
          annotations:
            summary: "Plane workers failing more than 6 tasks a minute"

        - alert: PlaneLiveEventLoopLag
          expr: |
            nodejs_eventloop_delay_p99_seconds{service_name="live"} > 0.5
          for: 5m
          labels:
            severity: warning
          annotations:
            summary: "Plane live event loop p99 delay above 500 ms"

        - alert: PlanePodRestarting
          expr: |
            increase(kube_pod_container_status_restarts_total{namespace="plane"}[30m]) > 2
          for: 0m
          labels:
            severity: warning
          annotations:
            summary: "A Plane container restarted more than twice in 30 minutes"

        - alert: PlaneTelemetryDown
          expr: |
            up{job="plane"} == 0
          for: 5m
          labels:
            severity: warning
          annotations:
            summary: "Prometheus cannot scrape the OpenTelemetry Collector"

PlaneWorkerTaskFailures depends on Tempo's metrics generator. Drop it if you run traces elsewhere.

Deploy a monitoring stack

Prometheus, Grafana, Loki and Tempo next to Plane in under an hour.

Health checks

Liveness and readiness endpoints for probes and uptime monitors.