Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable batch parallelization for CommonAlertsFilters #28

Conversation

kkovary
Copy link
Contributor

@kkovary kkovary commented Nov 14, 2024

Changelogs

Closes #27

  • Added optional batch_size arg to CommonAlertsFilter.__call__ that conditionally uses dm.parallelized_with_batches

Checklist:

  • Was this PR discussed in an issue? It is recommended to first discuss a new feature into a GitHub issue before opening a PR.
  • Add tests to cover the fixed bug(s) or the new introduced feature(s) (if appropriate).
  • Update the API documentation is a new function is added, or an existing one is deleted.
  • Write concise and explanatory changelogs above.
  • If possible, assign one of the following labels to the PR: feature, fix or test (or ask a maintainer to do it for you).

discussion related to that PR

@kkovary kkovary requested a review from maclandrol as a code owner November 14, 2024 05:38
Copy link
Member

@maclandrol maclandrol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kkovary that's very nice. Could you address the lint ? I can merge after.

@kkovary
Copy link
Contributor Author

kkovary commented Nov 15, 2024

@maclandrol made the lint fixes and updated pyproject.toml to get rid of a deprecation warning from ruff and to exclude Jupyter notebooks.

@kkovary kkovary changed the title calling common alerts filters can do batch parallelization Enable batch parallelization for CommonAlertsFilters Nov 15, 2024
@maclandrol
Copy link
Member

maclandrol commented Nov 18, 2024

merging. Will address the black issue in another PR

@maclandrol maclandrol merged commit a17bc12 into datamol-io:main Nov 18, 2024
3 of 4 checks passed
@maclandrol maclandrol mentioned this pull request Nov 18, 2024
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CommonAlertsFilters slow when running parallel with large numbers of mols
2 participants