Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: abp_nvsmi_detection doesn't detect any anomalous rows #2002

Closed
2 tasks done
dagardner-nv opened this issue Oct 28, 2024 · 2 comments
Closed
2 tasks done

[BUG]: abp_nvsmi_detection doesn't detect any anomalous rows #2002

dagardner-nv opened this issue Oct 28, 2024 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@dagardner-nv
Copy link
Contributor

Version

24.10

Which installation method(s) does this occur on?

Source

Describe the bug.

24.06 found 188 anomalous rows. I've reproduced this using triton launched from a 24.06 repo, which leads me to believe this is a code issue and not an issue with the model.

$ wc -l detections.jsonlines 
1242 detections.jsonlines
$ grep false detections.jsonlines | wc -l
1242

Minimum reproducible example

Follow example as documented
examples/abp_nvsmi_detection/README.md

Relevant log output

Click here to see error details

[Paste the error here, it will be hidden by default]

Full env printout

Click here to see environment details

[Paste the results of print_env.sh here, it will be hidden by default]

Other/Misc.

No response

Code of Conduct

  • I agree to follow Morpheus' Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report
@dagardner-nv dagardner-nv added the bug Something isn't working label Oct 28, 2024
@dagardner-nv dagardner-nv added this to the 24.10 - Release milestone Oct 28, 2024
@dagardner-nv
Copy link
Contributor Author

dagardner-nv commented Oct 28, 2024

Results of the examples/abp_pcap_detection don't match 24.06 results (24.06 results match 24.03 results).

$ ./scripts/compare_data_files.py ../morpheus-24.06.01/pcap_out.jsonlines pcap_out.jsonlines 

Results do not match. Diff 437241/537241 (81.386380 %). First 10 mismatched rows:

@dagardner-nv
Copy link
Contributor Author

log_parsing also yields mis-matched rows.
24.10 compared to 24.06 yields 96.8% yields
24.06 compared to 24.03 yields 2.6% yields

@dagardner-nv dagardner-nv moved this from Todo to In Progress in Morpheus Boards Oct 28, 2024
@dagardner-nv dagardner-nv self-assigned this Oct 28, 2024
@morpheus-bot-test morpheus-bot-test bot moved this from In Progress to Review - Ready for Review in Morpheus Boards Oct 28, 2024
dagardner-nv added a commit to dagardner-nv/Morpheus that referenced this issue Oct 29, 2024
rapids-bot bot pushed a commit that referenced this issue Oct 29, 2024
…2006)

Closes #2002

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)
  - Christopher Harris (https://github.com/cwharris)

URL: #2006
dagardner-nv added a commit to dagardner-nv/Morpheus that referenced this issue Oct 29, 2024
@github-project-automation github-project-automation bot moved this from Review - Ready for Review to Done in Morpheus Boards Oct 29, 2024
rapids-bot bot pushed a commit that referenced this issue Oct 30, 2024
…e correct rows for each batch (#2015)

* `DeserializeStage` now performs a copy of the rows needed for each batch, ensuring each output `ControlMessage` now has a unique `MessageMeta` and underlying `DataFrame`
* Works-around issue where calling `SlicedMessageMeta::get_mutable_info().checkout_obj()` returns the entire `DataFrame`
* This unfortunately means that `DeserializeStage` will require the GIL.
* Remove unused `make_output_message` method
* Avoid some redundant code in `SlicedMessageMeta` (unrelated but I was in this part of the code).

Closes #2002

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Christopher Harris (https://github.com/cwharris)

URL: #2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant