Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flaky instrumentation-runtime-node test #1960

Closed
trentm opened this issue Feb 26, 2024 · 1 comment · Fixed by #1965
Closed

flaky instrumentation-runtime-node test #1960

trentm opened this issue Feb 26, 2024 · 1 comment · Fixed by #1965
Assignees
Labels
bug Something isn't working pkg:instrumentation-runtime-node priority:p4 Bugs and spec inconsistencies which do not fall into a higher prioritization

Comments

@trentm
Copy link
Contributor

trentm commented Feb 26, 2024

I believe the 'should export event loop utilization metrics after eventLoopUtilizationMeasurementInterval' test case in

assert.strictEqual(metrics[0].dataPoints[0].value < 1, true);
may be flaky.

In the following unit-test run for PR #1938, that test assert failed:

    assert.strictEqual(metrics[0].dataPoints[0].value < 1, true);

https://github.com/open-telemetry/opentelemetry-js-contrib/actions/runs/8026238361/job/21928362946?pr=1938

> @opentelemetry/[email protected] test
> nyc ts-mocha -p tsconfig.json 'test/**/*.test.ts'



  nodejs.event_loop.utilization
    ✓ should stop exporting metrics when disabled
    ✓ should not export immediately after enable
    ✓ can use default eventLoopUtilizationMeasurementInterval
    1) should export event loop utilization metrics after eventLoopUtilizationMeasurementInterval


  3 passing (84ms)
  1 failing

  1) nodejs.event_loop.utilization
       should export event loop utilization metrics after eventLoopUtilizationMeasurementInterval:

      AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:

false !== true

      + expected - actual

      -false
      +true
      
      at Context.<anonymous> (test/event_loop_utilization.test.ts:94:12)
      at runNextTicks (internal/process/task_queues.js:60:5)
      at processTimers (internal/timers.js:497:9)

So I wonder if the new nodejs.event_loop.utilization metric can unexpectedly be higher than 1.
I have not been able to reproduce this failing locally.

I have only seen the failure that one time in CI. Mostly I'm just noting it for now. I'll have a PR to add some more output in those asserts in case that might help; but otherwise, I think we can wait and see if the test fails at all frequently before spending a lot of time on it.

/cc @d4nyll

@trentm trentm added the bug Something isn't working label Feb 26, 2024
trentm added a commit to trentm/opentelemetry-js-contrib that referenced this issue Feb 26, 2024
This is to give more detail if those asserts fail. I suspect the
'val < 1' test is flaky.

Refs: open-telemetry#1960
pichlermarc pushed a commit that referenced this issue Feb 26, 2024
This is to give more detail if those asserts fail. I suspect the
'val < 1' test is flaky.

Refs: #1960
@pichlermarc pichlermarc self-assigned this Feb 26, 2024
@pichlermarc
Copy link
Member

I'll look into it.

@pichlermarc pichlermarc added the priority:p4 Bugs and spec inconsistencies which do not fall into a higher prioritization label Feb 28, 2024
@pichlermarc pichlermarc changed the title flaky instrumentation-perf-tools test flaky instrumentation-runtime-node test Mar 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working pkg:instrumentation-runtime-node priority:p4 Bugs and spec inconsistencies which do not fall into a higher prioritization
Projects
None yet
2 participants