Memory leak in Micrometer code when exceptions are returned from the method

[adpaste]

We've identified a memory leak in ThreadLocal when the application endpoint returns with an exception. E.g.:

@GET
@Path("/404")
public String notFound() {
    throw new NotFoundException("Not found");
}

It happens because in case of exceptions, the observation is started twice in:

jersey/ext/micrometer/src/main/java/org/glassfish/jersey/micrometer/server/ObservationRequestEventListener.java

Lines 67 to 72 in 619ebef

	case ON_EXCEPTION:
	if (!isNotFoundException(event)) {
	break;
	}
	startObservation(event);
	break;

And:

jersey/ext/micrometer/src/main/java/org/glassfish/jersey/micrometer/server/ObservationRequestEventListener.java

Lines 73 to 75 in 619ebef

	case REQUEST_MATCHED:
	startObservation(event);
	break;

This is an issue because Micrometer's SimpleObservationRegistry keeps the observation scope inside a ThreadLocal variable in SimpleObservationRegistry.java#L30:

private static final ThreadLocal<Observation.Scope> localObservationScope = new ThreadLocal<>();

The instance of Scope class contains a pointer to a previous scope SimpleObservation.java#L285:

static class SimpleScope implements Scope {
    @Nullable
    final Scope previousObservationScope;

And these 2 things are updated in SimpleObservation.java#L290-L291

this.previousObservationScope = registry.getCurrentObservationScope();
this.registry.setCurrentObservationScope(this);

Now, each request that ends with an exception causes this linked list of observations to grow, because the code in:

jersey/ext/micrometer/src/main/java/org/glassfish/jersey/micrometer/server/ObservationRequestEventListener.java

Lines 83 to 91 in 619ebef

	case FINISHED:
	ObservationScopeAndContext finishedObservation = observations.remove(containerRequest);
	if (finishedObservation != null) {
	finishedObservation.jerseyContext.setRequestEvent(event);
	Observation.Scope observationScope = finishedObservation.observationScope;
	observationScope.close();
	observationScope.getCurrentObservation().stop();
	}
	break;

is closing only one observation.

Here is an example test that reproduces the issue: 2.x...adpaste:jersey:memory_leak