Observability

CoquiTitle integrates with Langfuse for LLM observability, providing distributed tracing across all Lambda stages.

Architecture: OTEL Context Propagation

The system uses OpenTelemetry native context propagation to maintain trace hierarchy across Lambda invocations.

Trace Hierarchy

All pipeline spans are nested in a call-chain hierarchy:

coquititle.pipeline (root)
└── coquititle.extractor
    ├── extractor.property (generation)
    ├── extractor.ownership (generation)
    ├── extractor.gravamenes (generation)
    ├── extractor.acq[0], acq[1], ... (generations)
    └── coquititle.title_state_builder
        └── coquititle.evidence
            ├── evidence.llm_fallback (generation)
            └── coquititle.report
                ├── report.header (generation)
                ├── report.prose (generation)
                └── report.annotate_* (generations)

Key Components

Function	Purpose
bootstrap_trace(run_id, case_id, tags)	Creates root pipeline span
lambda_trace(run_id, name, otel_carrier)	Creates Lambda span, attaches upstream context
generation_span(name, model, otel_ctx)	Creates LLM generation span
inject_otel_carrier()	Injects context into W3C carrier
extract_otel_context(carrier)	Extracts context from carrier

Lambda Payload Format

Each Lambda invocation includes an otel field:

{
  "case_id": "uuid",
  "run_id": "uuid",
  "otel": {
    "traceparent": "00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01",
    "tracestate": "langfuse=..."
  }
}

Evaluation Scores

Each stage emits Langfuse scores for quality tracking:

Stage	Score Name	Description
Extractor	extraction_evidence_count	Evidence fields extracted
Extractor	extraction_entity_count	Total entities
Title State	title_state_confidence	Derivation confidence
Evidence	evidence_match_rate	Successful match percentage
Evidence	evidence_llm_fallbacks	LLM fallback count
Report	report_evidence_coverage	Fields with evidence

Configuration

Langfuse is enabled via environment variables:

Variable	Description
LANGFUSE_ENABLED	"true" to enable tracing
LANGFUSE_SECRET_KEY_ARN	AWS Secrets Manager ARN

Secrets JSON format:

{
  "public_key": "pk-lf-...",
  "secret_key": "sk-lf-...",
  "base_url": "https://us.cloud.langfuse.com"
}

Viewing Traces

Access traces at Langfuse Cloud:

• Filter by trace name: coquititle.pipeline
• Filter by tag: coquititle
• Search by case_id or run_id in metadata

---

Related Pages

• System Overview - High-level architecture
• Extraction Pipeline - Extraction stage tracing
• Evidence Resolution - Evidence resolution tracing
• Report Generation - Report generation tracing
• Data Model - Database schema