Langfuse Integration

Setup

Configure Langfuse in the admin console under your account settings. You'll need:

Langfuse keys are securely stored and never appear in logs or the database.

Trace structure

Each proxied request creates a trace with three child spans that map to the NoPII pipeline:

nopii-proxy (root span)
├── sanitize       - PII detection + tokenization
├── llm-call       - Forwarded request to the LLM provider (generation)
└── desanitize     - Response detokenization

Root span metadata

Sanitize span

Records the PII detection and tokenization step. Output metadata includes entity_count - the number of PII entities detected and tokenized.

LLM call generation

Recorded as a Langfuse generation (not a span), which enables Langfuse's built-in model cost tracking. Includes:

• Sanitized input (request body with PII replaced by tokens)
• Sanitized output (LLM response before detokenization - tokens, not original PII)
• Model name
• Token usage (prompt and completion tokens)
• HTTP status code

Desanitize span

Records the response detokenization step, where tokens in the LLM response are replaced with original PII values.

Streaming vs. non-streaming

Both streaming and non-streaming requests log the full sanitized input and sanitized output in the generation. For streaming requests, the output is assembled from the streamed chunks before being sent to Langfuse. In both cases, all content is tokenized - tokens, not original PII.

Distributed tracing

NoPII supports the W3C Trace Context standard via the traceparent header. This allows you to connect NoPII traces to your existing distributed tracing pipeline.

Format

traceparent: 00-{trace_id(32hex)}-{parent_span_id(16hex)}-{flags(2hex)}

Example

import openai
import uuid

# Generate a trace context for your application
trace_id = uuid.uuid4().hex  # 32 hex characters
span_id = uuid.uuid4().hex[:16]  # 16 hex characters
traceparent = f"00-{trace_id}-{span_id}-01"

client = openai.OpenAI(base_url="https://api.nopii.co")

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarize the case for John Smith"}],
    extra_headers={"traceparent": traceparent}
)

# The NoPII trace in Langfuse will appear as a child of your trace

When a valid traceparent header is present, NoPII creates its trace as a child of the specified parent span via OpenTelemetry context propagation. This lets you see the full request lifecycle - from your application through NoPII to the LLM - in a single trace view.

Performance and reliability

Tracing never blocks or slows down your proxy requests. If Langfuse is unreachable or returns an error, NoPII continues normally - your requests are never affected by tracing issues.

When Langfuse is not configured or disabled:

• No performance impact
• No errors if credentials are invalid or the service is down
• You can enable and disable Langfuse at any time without any disruption