Dapr Distributed Tracing Overview - Summary

Overview

Dapr provides distributed tracing capabilities using OpenTelemetry (OTEL) and Zipkin protocols. OTEL is the industry standard and recommended approach, with support from most observability tools.

Key Concepts

• Primary Use Cases: Service invocation and pub/sub APIs
• Trace Context Flow: Enables trace propagation between services using these APIs
• Integration: Works with multiple observability tools through standardized protocols

Two Main Scenarios

Scenario 1: Dapr-Generated Trace Context

Most common scenario - Dapr automatically creates trace headers

Single Service Call

• Simple flow: Service A → Service B
• Dapr generates trace headers in Service A and propagates to Service B
• No additional work required

Sequential Service Calls

• Flow: Service A → Service B → Service C (and beyond)
• Dapr generates initial trace headers in Service A
• Developer responsibility: Propagate headers from Service B to Service C and subsequent services
• Rule: Applications must propagate trace headers to Dapr when continuing the chain

External Requests

• Flow: Gateway/Ingress → Service A → Service B
• External gateway calls Dapr, which generates trace headers
• Same propagation rules apply for subsequent service calls

Pub/Sub Messages

• Dapr automatically generates trace headers in published message topics
• Headers propagate to all services listening on that topic
• Support for traceparent header in rawPayload messages

Multiple Parallel Service Calls

• When one service calls multiple different services sequentially
• Trace headers returned in response must be manually propagated to subsequent calls
• Dapr doesn't automatically reuse headers across different service calls

Scenario 2: User-Generated Trace Context

Less common scenario - When you generate your own trace headers

Three Methods:

1. OpenTelemetry SDK (Preferred)

   • Use standard OpenTelemetry SDKs to generate trace headers
   • Industry standard approach

2. Vendor SDK

   • Use vendor-specific SDKs that support W3C trace headers

3. Manual W3C Trace Context

   • Handcraft trace context following W3C specifications
   • Most complex but fully customizable approach