Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: stop() called twice for each stage on Control-C #1477

Closed
2 tasks done
efajardo-nv opened this issue Jan 24, 2024 · 0 comments · Fixed by #1479
Closed
2 tasks done

[BUG]: stop() called twice for each stage on Control-C #1477

efajardo-nv opened this issue Jan 24, 2024 · 0 comments · Fixed by #1479
Assignees
Labels
bug Something isn't working

Comments

@efajardo-nv
Copy link
Contributor

efajardo-nv commented Jan 24, 2024

Version

23.07

Which installation method(s) does this occur on?

Conda

Describe the bug.

When using Control-C to stop a pipeline, the stop() method on each stage is being called twice.

Minimum reproducible example

  1. Set up debug environment for Production DFP pipeline examples:
    https://github.com/nv-morpheus/Morpheus/tree/branch-24.03/examples/digital_fingerprinting/production

  2. Add breakpoint here:
    https://github.com/nv-morpheus/Morpheus/blob/branch-24.03/morpheus/pipeline/stage_base.py#L448

  3. Run any of the example pipelines.

  4. Send SIGINT (Ctrl-C) to pipeline process.

Observe with breakpoint that stages are looped through twice to call stop().

Relevant log output

No response

Full env printout

No response

Other/Misc.

No response

Code of Conduct

  • I agree to follow Morpheus' Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report
@efajardo-nv efajardo-nv added the bug Something isn't working label Jan 24, 2024
@efajardo-nv efajardo-nv self-assigned this Jan 25, 2024
@jarmak-nv jarmak-nv moved this from Todo to In Progress in Morpheus Boards Jan 25, 2024
@jarmak-nv jarmak-nv moved this from In Progress to Review - Ready for Review in Morpheus Boards Feb 9, 2024
rapids-bot bot pushed a commit that referenced this issue Feb 15, 2024
… all stages (#1479)

1. Removes the `_is_built`, `_is_started` and `_is_stopped` flags and replaces with single member which holds onto the state enum for: INITIALIZED, BUILT, STARTED, STOPPED, COMPLETED
1. Changes the meaning of `stop()` and the meaning of `join()` for stages
   1. `stop()` called 0 or 1 times. Only way it can get called is if `pipeline.stop()` was called indicating the pipeline should try to shut down gracefully. 
      1. Users should only implement this method if they have a source stage (or sources in their stage)
   1. `join()` called exactly 1 time. Only called when the pipeline is complete and all stages are shut down. This is where users should implement any cleanup code
1. Tests for handling all of these scenarios with the pipeline.

Closes #1477

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Eli Fajardo (https://github.com/efajardo-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #1479
@github-project-automation github-project-automation bot moved this from Review - Ready for Review to Done in Morpheus Boards Feb 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Done
1 participant