[BUG]: Many Python source stages do not stop if the user hits Ctrl+C #1837
Assignees
Labels
bug
Something isn't working
Comments
dagardner-nv
changed the title
HttpServerSourceStage deadlocks if the user hits Ctrl+C or another stage raises an exception
dagardner-nv
changed the title
HttpServerSourceStage deadlocks if the user hits Ctrl+C or another stage raises an exception
|
Related to #1838 |
morpheus-bot-test
bot
moved this from Todo
to Review - Ready for Review
in Morpheus Boards
2 tasks
rapids-bot bot
pushed a commit
that referenced
this issue
Aug 14, 2024
* Ensure that Python sources which run indefinitely actually shutdown when requested by the pipline (ex: when the user hits Ctrl+C). * The `KafkaSourceStage` already handled this properly, move the `stop` method from this class into the `SingleOutputSource`. * Add `request_stop` and `is_stop_requested` methods to `SingleOutputSource`. * Update all existing source stages which run indefinitely. * Add new `should_stop_fn` constructor argument to `DirectoryWatcher`, sources which use the watcher (`AutoencoderSourceStage` & `AppShieldSourceStage`) pass in their `is_stop_requested` method allowing the watcher to know when to shut down. * Move the RSS source generator method from the RSS source module into the `RSSController`, and update the `RSSSourceStage` to use the `RSSController` directly rather than the RSS module. This avoids the problem where modules require all config values to be JSON serializable, preventing a callback function from being passed to a module. Closes #1837 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #1839
github-project-automation
bot
moved this from Review - Ready for Review
to Done
in Morpheus Boards
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Version
24.10
Which installation method(s) does this occur on?
Source
Describe the bug.
This primarily impacts sources which run indefinitely (
HttpClientSourceStage,AppShieldSourceStage,RSSSourceStage). The Kafka source stage currently does the correct thing and shuts down when requested, fix should just be to promote that shutdown code to the base class.Minimum reproducible example
Relevant log output
Click here to see error details
Full env printout
Click here to see environment details
Other/Misc.
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: