Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Span Link support for compound header tag extractions with conflicting trace IDs #2948

Open
wants to merge 28 commits into
base: main
Choose a base branch
from

Conversation

mhlidd
Copy link
Contributor

@mhlidd mhlidd commented Oct 24, 2024

What does this PR do?

Adds support for span links when trace propagators extract headers with conflicting trace IDs.

  • Under the above condition, the (*chainedPropagator).Extract function now appends spanLinks to the returned SpanContext
  • All integrations that invoke tracer.Extract now check for spanLinks on the returned SpanContext; if spanLinks is not nil, pass tracer.WithSpanLinks as a StartSpanOption.

Motivation

Make distrubuted tracing with span links consistent across Datadog libraries according the following RFC.

APMAPI-762

Reviewer's Checklist

  • Changed code has unit tests for its functionality at or near 100% coverage.
  • System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
  • There is a benchmark for any new code, or changes to existing code.
  • If this interacts with the agent in a new way, a system test has been added.
  • Add an appropriate team label so this PR gets put in the right place for the release notes.
  • Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.

Unsure? Have a question? Request a review!

@pr-commenter
Copy link

pr-commenter bot commented Oct 24, 2024

Benchmarks

Benchmark execution time: 2024-11-25 16:17:18

Comparing candidate commit 036efda in PR branch mhlidd/tracecontext_span_links with baseline commit d7cc13a in branch main.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 58 metrics, 0 unstable metrics.

scenario:BenchmarkTracerAddSpans-24

  • 🟥 execution_time [+78.693ns; +153.907ns] or [+2.049%; +4.008%]

@mhlidd mhlidd force-pushed the mhlidd/tracecontext_span_links branch from 3a74a18 to b6e05e6 Compare October 25, 2024 18:42
ddtrace/tracer/option.go Outdated Show resolved Hide resolved
ddtrace/tracer/spancontext.go Outdated Show resolved Hide resolved
ddtrace/tracer/stats_payload_msgp.go Outdated Show resolved Hide resolved
@mhlidd mhlidd force-pushed the mhlidd/tracecontext_span_links branch from 0aec80a to 4f7ef39 Compare November 4, 2024 19:26
Copy link
Contributor

@mtoffl01 mtoffl01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting various changes, lmk if anything unclear!

ddtrace/tracer/span.go Show resolved Hide resolved
ddtrace/tracer/spancontext.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
.vscode/launch.json Outdated Show resolved Hide resolved
contrib/IBM/sarama.v1/sarama.go Outdated Show resolved Hide resolved
Copy link
Contributor

@mtoffl01 mtoffl01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting various changes, lmk if anything unclear!

Copy link
Contributor

@mtoffl01 mtoffl01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly nits...
Just be sure all the existing extract tests pass, like in textmap_test.go etc.

ddtrace/tracer/context_test.go Outdated Show resolved Hide resolved
ddtrace/tracer/context_test.go Outdated Show resolved Hide resolved
contrib/IBM/sarama.v1/sarama.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap_test.go Outdated Show resolved Hide resolved
@mhlidd mhlidd requested a review from mtoffl01 November 7, 2024 21:05
@github-actions github-actions bot added the apm:ecosystem contrib/* related feature requests or bugs label Nov 7, 2024
@mhlidd mhlidd changed the title initial code for adding spanlinks for span context Adding Span Link creation due to compound header tag extractions for invalid traces Nov 7, 2024
@mhlidd mhlidd changed the title Adding Span Link creation due to compound header tag extractions for invalid traces Adding Span Link support for compound header tag extractions with invalid traces Nov 7, 2024
@mhlidd mhlidd marked this pull request as ready for review November 7, 2024 21:10
@mhlidd mhlidd requested review from a team as code owners November 7, 2024 21:10
@mhlidd mhlidd requested a review from darccio November 7, 2024 21:11
contrib/IBM/sarama.v1/sarama.go Outdated Show resolved Hide resolved
ddtrace/ddtrace.go Outdated Show resolved Hide resolved
ddtrace/ddtrace.go Outdated Show resolved Hide resolved
ddtrace/tracer/context_test.go Outdated Show resolved Hide resolved
ddtrace/tracer/context_test.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap_test.go Outdated Show resolved Hide resolved
@mhlidd mhlidd force-pushed the mhlidd/tracecontext_span_links branch from 8a32649 to 538d25a Compare November 18, 2024 19:14
contrib/IBM/sarama.v1/sarama.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
Copy link
Contributor

@mtoffl01 mtoffl01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to simplify the Extract method a bit; will push up the changes we discussed to assist (and moving this to draft in the meantime)

ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
@mtoffl01 mtoffl01 marked this pull request as draft November 20, 2024 15:07
}
} else { // Trace IDs do not match - create span links
link := ddtrace.SpanLink{TraceID: extractedCtx2.TraceID(), SpanID: extractedCtx2.SpanID(), TraceIDHigh: extractedCtx2.TraceIDUpper(), Attributes: map[string]string{"reason": "terminated_context", "context_headers": "tracecontext"}}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mtoffl01 FYI the context_headers field is supposed to hold whichever propagator type was used to extract the violating context. For instance, if the propagator type was Datadog and we have violating trace IDs, we want context_headers to have the value of datadog

@mtoffl01 mtoffl01 marked this pull request as ready for review November 21, 2024 18:05
@mtoffl01 mtoffl01 self-requested a review November 21, 2024 18:05
Copy link
Contributor

@mtoffl01 mtoffl01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mhlidd , make sure you update the expected "context_headers" in TestSpanLinks::Links_on_divergent_trace_IDs test!

Copy link
Contributor Author

@mhlidd mhlidd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few small comments!

ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
Comment on lines 288 to 290
extractedCtx2, ok1 := extractedCtx.(*spanContext)
ctx2, ok2 := ctx.(*spanContext)
// If we can't cast to SpanContextW3C, we can't propgate tracestate or create span links
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for casting to spanContext struct? We had conversation in the past to use SpanContextW3C instead of spanContext so I'm wondering what the motivation to change it back is.

PS: Should update the comment below if the decision is to cast to spanContext

Copy link
Contributor

@mtoffl01 mtoffl01 Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To my memory, I think the cast to SpanContextW3C was never even used except to further cast to spanContext.
(Like for example, pW3C.propagateTracestate requires spanContext type)

I do remember that casting directly to spanContext made this code a lot easier to read.

@mtoffl01 mtoffl01 changed the title Adding Span Link support for compound header tag extractions with invalid traces Adding Span Link support for compound header tag extractions with conflicting trace IDs Nov 25, 2024
Copy link
Contributor Author

@mhlidd mhlidd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't approve my own PR but LGTM! The flow is a lot clearer, thanks for helping out!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
apm:ecosystem contrib/* related feature requests or bugs do-not-merge/WIP
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants