Tracing

Overview

Every request passing through nginx is automatically traced to Langfuse via a Lua script (nginx/lua/langfuse.lua) running in the log_by_lua_file phase. No changes to the client or vLLM are required.

Client → nginx (port 8100) → vLLM
                ↓ (async, log phase)
           Langfuse (port 3010)

Traces are sent asynchronously using ngx.timer.at so they never block the request path.

---

What Gets Traced

LLM endpoints (/v1/chat/completions, /v1/completions)

Each request produces two Langfuse events:

Event	Type	Contents
Trace	trace-create	name (first 80 chars of user message or prompt), input, output, metadata
Generation	generation-create	model name, input messages/prompt, output choice, token usage

Token usage (prompt_tokens, completion_tokens, total_tokens) is extracted directly from the vLLM response.

All other endpoints

Each request produces:

Event	Type	Contents
Trace	trace-create	METHOD /path, request body (if any), response body
Span	span-create	same as trace, with startTime / endTime

---

Trace Structure

Every trace carries a metadata object:

{
  "method": "POST",
  "uri": "/v1/chat/completions",
  "status_code": 200,
  "request_time": 0.843,
  "remote_addr": "localhost",
  "user_agent": "python-requests/"
}

Traces are tagged with the URI and HTTP status code for easy filtering in the Langfuse UI.

---

Trace ID

The trace ID is derived from the id field in the vLLM response (e.g. chatcmpl-abc123). If unavailable, it falls back to nginx's $request_id, then ngx.now().

---

Configuration

Tracing requires these variables set in .env:

LANGFUSE_PUBLIC_KEY=pk-...
LANGFUSE_SECRET_KEY=sk-...
LANGFUSE_BASE_URL=http://localhost:3010

If LANGFUSE_PUBLIC_KEY or LANGFUSE_SECRET_KEY is empty, tracing is silently skipped.

---

Viewing Traces

Open the Langfuse UI at http://<host>:${LANGFUSE_PORT:-3010} and navigate to Traces. Use the tag filter to narrow by endpoint (e.g. /v1/chat/completions) or status code.

---

Disabling Tracing

To disable tracing without removing the Lua script, leave LANGFUSE_PUBLIC_KEY or LANGFUSE_SECRET_KEY empty in .env. The script exits early when either is unset.

---

Trace Retention (TTL)

Traces are automatically deleted after a configurable TTL. The default is 7 days:

LANGFUSE_DEFAULT_TTL_DAYS=7   # default
LANGFUSE_DEFAULT_TTL_DAYS=0   # disable TTL, keep forever

TTL applies to trace data stored in ClickHouse and MinIO event blobs.

---

Storage Backend

Langfuse persists trace data across three services:

Service	Role
PostgreSQL	Users, projects, API keys, config
ClickHouse	Time-series trace/event data
MinIO	Raw event blobs (langfuse-events), media (langfuse-media), batch exports (langfuse-exports)