PostHog for AI observability: traces, errors, and session replay in one place

Your copilot fails for one customer. You need the prompt and model output, the exception stack, and what the user clicked before they reported "it broke." That usually means three tools: an LLM tracer, an error tracker, and session replay, plus product analytics if you want funnel or retention context.

PostHog puts those pieces in one product. AI observability (traces, generations, sessions, users, errors) sits next to error tracking and session replay, with the same person and session identity you already use for product analytics.

What PostHog gives you for LLM features

PostHog's AI observability is built around events your backend (or an SDK wrapper) emits for model work. The concepts match how production teams already debug copilots and agents:

You also get PostHog's usual product surface on the same data: trends, funnels, dashboards, and alerts. So "copilot error rate by tenant" does not need a separate BI project.

Ease of use: why consolidation helps

Specialized LLM platforms are strong at prompt versioning and eval datasets. The cost shows up when debugging a production incident means opening three UIs:

1. Langfuse (or equivalent) for the generation
2. Sentry for the exception
3. FullStory, LogRocket, or another replay tool for the session
4. Hoping requestId, sessionId, and userId were correlated everywhere

With PostHog, correlation is shorter:

• Same person across product events, AI events, and errors
• Same session from replay into AI traces (pass $session_id or PostHog's AI session fields from the API request into your middleware)
• Error Tracking linked to AI traces via $ai_trace_id on captured exceptions
• One query and dashboard model your eng team may already use for funnels

Setup is familiar if you already ship posthog-js and a server SDK. AI events are normal PostHog events ($ai_generation, spans, embeddings), not a separate store. Autocaptured properties (cost, latency, tokens, model) show up in the AI Observability tabs without a custom admin.

Keep instrumentation in middleware. Browser-only tracing is incomplete (no server context assembly, no tool results) and risky (keys, prompt leakage). Frontend PostHog owns replay and client identity; backend PostHog owns generations and spans.

Use cases

Support escalation

A user reports a wrong draft. Filter generations by distinct_id or tenant property, open the trace, then jump to session replay for the same session. You see both the model I/O and the UI state. That loop is PostHog's strongest argument versus an LLM-only tracer.

Agent and tool failures

For tool-calling agents, emit spans for each tool (name, redacted args, permission outcome, latency) under one $ai_trace_id. When Error Tracking fires on a schema validation failure, link $ai_trace_id so on-call lands on the full tool chain instead of a bare stack trace. Pair with prompt injection and tool security so denials show up in the same product as exceptions.

Quality and reliability without a second vendor

Use the Errors tab for normalized LLM failure messages, trends for $ai_is_error rate by feature or model, and alerts when error rate or latency spikes. For unit economics (tokens and cost per successful outcome), the same event properties feed PostHog insights. See the patterns in Monitoring LLM costs in production; you can implement them with PostHog events instead of, or alongside, Grafana.

Product and AI in one funnel

"Users who opened copilot, got an error, then churned" is awkward across three tools. In PostHog it is a funnel over product events and AI error events on the same persons. Useful when AI is a product surface, not an internal batch job.

Teams already on PostHog for analytics

If product engineering already lives in PostHog, adding AI observability extends the existing source of truth. You avoid a new login, billing line, and metadata schema. For small teams, that often beats a best-of-breed LLM tracer.

Comparison to tools from our other guides

We have written about Langfuse, the broader observability stack, and cost monitoring. Here is how PostHog fits.

PostHog vs Langfuse

Choose PostHog when the bottleneck is incident debugging and product context: who was the user, what did they click, what exception fired, what did the model return. Especially if you do not want three subscriptions for that path.

Choose Langfuse when the bottleneck is LLM engineering workflow: managed prompt versions, human annotation queues, golden datasets, and CI-style eval runs. Langfuse remains a solid default for that kind of team (our Langfuse guide).

You can dual-write, but avoid tool sprawl. Pick one place your team opens daily for AI debugging. If that place is PostHog, invest in consistent $ai_* properties and middleware tags. If you need Langfuse-grade evals later, add them on purpose. Do not run three half-configured LLM platforms (beyond Langfuse).

PostHog vs Sentry (or Datadog) alone

Sentry and Datadog are good for exceptions and infrastructure. They answer "the route threw" and "p95 is up." They do not replace a generation view with prompt, completion, token cost, and tool spans. PostHog's advantage is LLM-shaped events plus errors plus replay. It is not a reason to rip out APM if SRE already lives in Datadog.

PostHog vs Braintrust, Phoenix, Helicone

How to instrument it

Architecture stays the same as our other observability guides:

1. Client to your authenticated API (PostHog.js for replay / session id)
2. LLM middleware: auth, rate limits, context assembly, model call; emit AI events here
3. Model provider

Minimum tags on every generation (same schema we recommend for Langfuse):

• feature: copilot, search-assist, support-agent
• tenantId: multi-tenant cost and quality
• $ai_trace_id: required grouping
• $ai_session_id: multi-turn threads (when applicable)
• Model, token counts, latency, and cost fields your SDK documents
• On failure: $ai_is_error, $ai_error, and captureException with $ai_trace_id

Pass the PostHog session id from the client (automatic header injection or explicit body field) into middleware so generations link to replay. Redact prompts and tool outputs before emit. Traces are debugging artifacts, not a second copy of customer data.

When PostHog is enough, and when it is not

PostHog is not a substitute for property-based eval gates in CI. Use eval pipelines whether your production tracer is PostHog or Langfuse.

Common mistakes

Tracing only from the browser. Incomplete context and leaked keys. Middleware owns AI events.

Skipping session linkage. You get generations without replay, which undercuts the point of consolidation.

No shared metadata. If feature and tenantId are missing, PostHog becomes another unfilterable firehose.

Expecting Langfuse-level prompt CMS. Use code or your own store for prompts, or adopt Langfuse when that workflow is the product.

Tool sprawl. PostHog plus Langfuse plus Braintrust plus Helicone, with none configured deeply. Pick one primary tool and go deep.

How 475 Cumulus uses this

We do not resell PostHog. On our stack and on integration projects we:

• Instrument server-side middleware so AI events, auth, and rate limits share one path
• Prefer PostHog when the team wants AI, errors, replay, and analytics in one product
• Still recommend Langfuse when prompt management and eval datasets are the primary workflow
• Keep redaction, sampling, and tenant tags non-negotiable regardless of vendor

---

HTTP latency alone is not enough for LLM features, and a debug path that needs four tabs will not get used. Putting AI traces next to errors, users, sessions, and replay keeps that loop short. Describe your copilot or agent and we will map middleware instrumentation and which layer belongs in PostHog versus a specialized eval tool.