Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

contrib/net/http: refactor tracing #2921

Merged
merged 7 commits into from
Oct 21, 2024

Conversation

rarguelloF
Copy link
Contributor

@rarguelloF rarguelloF commented Oct 9, 2024

What does this PR do?

  • Refactor contrib/net/http.TraceAndServe to extract the logic ran before executing the provided handler into a separate function: contrib/internal/httptrace.BeforeHandle.
  • BeforeHandle returns the modified ResponseWriter and Request types with the required tracing information included. It also provides an afterHandle function that should be executed after the Handler runs, and a handled bool that instructs the original handler should not be called if true (this was required for the appsec logic).
  • Move around some of the contrib/net/http types to contrib/internal/httptrace (motivated by the changes described above), adding type aliases in contrib/net/http for the exported ones.
  • Similar changes described above for contrib/julienschmidt/httprouter, adding a new internal/tracing package with a BeforeHandle function that uses generics to avoid importing the original package (and therefore be reusable in orchestrion).
  • Refactor in internal/appsec/emitter/httpsec motivated by the refactor described above.

Motivation

Orchestrion support.

Reviewer's Checklist

  • Changed code has unit tests for its functionality at or near 100% coverage.
  • System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
  • There is a benchmark for any new code, or changes to existing code.
  • If this interacts with the agent in a new way, a system test has been added.
  • Add an appropriate team label so this PR gets put in the right place for the release notes.
  • Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.

Unsure? Have a question? Request a review!

@github-actions github-actions bot added the apm:ecosystem contrib/* related feature requests or bugs label Oct 9, 2024
@pr-commenter
Copy link

pr-commenter bot commented Oct 9, 2024

Benchmarks

Benchmark execution time: 2024-10-15 11:23:28

Comparing candidate commit cbcce81 in PR branch rarguelloF/APPSEC-55047/refactor-trace-http with baseline commit 93311db in branch main.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 57 metrics, 1 unstable metrics.

scenario:BenchmarkHttpServeTrace-24

  • 🟥 allocations [+2; +2] or [+2.597%; +2.597%]

Comment on lines +41 to +45
if w.status != 0 {
return
}
w.ResponseWriter.WriteHeader(status)
w.status = status
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "regular" ResponseWriter implementation has some logging code when it's called multiple times; and I feel like this conditional might break this semantics...

I think I'd probably change this to:

Suggested change
if w.status != 0 {
return
}
w.ResponseWriter.WriteHeader(status)
w.status = status
w.ResponseWriter.WriteHeader(status)
if w.status == 0 {
w.status = status
}

contrib/internal/httptrace/before_handle.go Show resolved Hide resolved
Comment on lines 23 to 26
serviceName string
spanOpts []ddtrace.StartSpanOption
analyticsRate float64
headerTags *internal.LockMap
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest front-loading pointers to make the GC's life easier

Suggested change
serviceName string
spanOpts []ddtrace.StartSpanOption
analyticsRate float64
headerTags *internal.LockMap
headerTags *internal.LockMap
spanOpts []ddtrace.StartSpanOption
serviceName string
analyticsRate float64

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! even though this is a "just" a refactor PR and I wanted to avoid extra changes, since this is a pretty-safe to make change I will just update here.

GetValue() string
}

func BeforeHandle[T any, WT Router](cfg *Config, router T, wrapRouter func(T) WT, w http.ResponseWriter, req *http.Request) (http.ResponseWriter, *http.Request, func(), bool) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as before, this could use documentation commentary

Comment on lines +39 to +41
for _, param := range ps {
route = strings.Replace(route, param.GetValue(), ":"+param.GetKey(), 1)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, this could replace the wrong part of the URI, if a parameter's value is actually a substring of any other segment of the URI. I'm also not sure how the contraption behaves in presence of URL-encoded characters (will the param.GetValue() return the un-escaped version?)

Copy link
Contributor Author

@rarguelloF rarguelloF Oct 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch, I'll open a subsequent PR adding some extra test-cases and try to do this replacement in a smarter way.

Comment on lines 111 to 112
func BeforeHandle(w http.ResponseWriter, r *http.Request, span ddtrace.Span, pathParams map[string]string, opts *Config) (
http.ResponseWriter, *http.Request, func(), bool) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as before, documentation would be welcome.

Also, I'm not a huge fan of the line break after the results parenthesis... I'm a little grossed out... For these I prefer the following (which I reckon gofmt isn't against):

Suggested change
func BeforeHandle(w http.ResponseWriter, r *http.Request, span ddtrace.Span, pathParams map[string]string, opts *Config) (
http.ResponseWriter, *http.Request, func(), bool) {
func BeforeHandle(
w http.ResponseWriter,
r *http.Request,
span ddtrace.Span,
pathParams map[string]string,
opts *Config,
) (http.ResponseWriter, *http.Request, func(), bool) {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah agree this looks much better

Comment on lines +133 to +135
if res, ok := w.(interface{ Status() int }); ok {
statusCode = res.Status()
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding an else that prints a debug statement indicating the writer somehow misses the Status() method (+ mention its actual type).

Suggested change
if res, ok := w.(interface{ Status() int }); ok {
statusCode = res.Status()
}
if res, ok := w.(interface{ Status() int }); ok {
statusCode = res.Status()
} else {
log.Debugf("http.ResponseWriter (%T) lacks Statsus()int method", w)
}

Comment on lines +157 to +158
blockPtr.Handler.ServeHTTP(w, tr)
blockPtr.Handler = nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is making me feel a little unsafe (@eliottness can you have a look?).

Basically, the blockPtr comes from an atomic pointer, suggesting it may be concurrently modified... And then we're changing one value in there that was checked above... so we have a possible race condition here...

We may need to also use an atomic pointer here, and atomic-swap it with a nil?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took note of the issues you mention in appsec code. Although the atomic pointer was us being overzealous in the first place, let's be consistent and replace the atomic.Load() calls to atomic.CompareAndSwap in another PR so we get rid of this race

@rarguelloF rarguelloF marked this pull request as ready for review October 16, 2024 08:44
@rarguelloF rarguelloF requested review from a team as code owners October 16, 2024 08:44
@rarguelloF rarguelloF changed the title WIP: contrib/net/http: refactor tracing contrib/net/http: refactor tracing Oct 16, 2024
@darccio darccio merged commit 020a9da into main Oct 21, 2024
165 of 166 checks passed
@darccio darccio deleted the rarguelloF/APPSEC-55047/refactor-trace-http branch October 21, 2024 09:35
darccio added a commit that referenced this pull request Oct 23, 2024
github-merge-queue bot pushed a commit to DataDog/orchestrion that referenced this pull request Nov 5, 2024
Adds support for `github.com/julienschmidt/httprouter`

Requires DataDog/dd-trace-go#2921
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
apm:ecosystem contrib/* related feature requests or bugs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants