Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

♻️ reroute user services restart via dynamic-scheduler #6943

Conversation

GitHK
Copy link
Contributor

@GitHK GitHK commented Dec 11, 2024

What do these changes do?

Rerouted user services restart command via dynamic-scheduler.

Related issue/s

How to test

Dev-ops checklist

@GitHK GitHK changed the title ♻️ reroute user services restart ♻️ reroute inputs retrieval via dynamic-scheduler Dec 11, 2024
Copy link

codecov bot commented Dec 11, 2024

Codecov Report

Attention: Patch coverage is 72.00000% with 7 lines in your changes missing coverage. Please review.

Project coverage is 86.93%. Comparing base (acd677a) to head (4e397da).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6943      +/-   ##
==========================================
- Coverage   87.06%   86.93%   -0.14%     
==========================================
  Files        1611     1256     -355     
  Lines       63621    53925    -9696     
  Branches     2026     1001    -1025     
==========================================
- Hits        55392    46880    -8512     
+ Misses       7895     6851    -1044     
+ Partials      334      194     -140     
Flag Coverage Δ
integrationtests 64.91% <50.00%> (-2.07%) ⬇️
unittests 85.25% <72.00%> (-0.10%) ⬇️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library ∅ <ø> (∅)
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration ∅ <ø> (∅)
pkg_service_library 74.14% <0.00%> (-0.06%) ⬇️
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 85.38% <ø> (ø)
agent 96.82% <ø> (ø)
api_server 90.13% <ø> (ø)
autoscaling 96.09% <ø> (ø)
catalog 90.57% <ø> (ø)
clusters_keeper 99.48% <ø> (ø)
dask_sidecar 91.26% <ø> (ø)
datcore_adapter 93.18% <ø> (ø)
director 76.40% <ø> (ø)
director_v2 91.43% <ø> (ø)
dynamic_scheduler 97.14% <100.00%> (+0.04%) ⬆️
dynamic_sidecar 89.75% <ø> (ø)
efs_guardian 90.12% <ø> (ø)
invitations 93.44% <ø> (ø)
osparc_gateway_server ∅ <ø> (∅)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.58% <ø> (+0.26%) ⬆️
storage 89.54% <ø> (ø)
webclient ∅ <ø> (∅)
webserver 84.46% <50.00%> (+0.06%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update acd677a...4e397da. Read the comment docs.

@GitHK GitHK changed the title ♻️ reroute inputs retrieval via dynamic-scheduler ♻️ reroute user services restart via dynamic-scheduler Dec 11, 2024
@GitHK GitHK self-assigned this Dec 11, 2024
@GitHK GitHK added a:webserver issue related to the webserver service a:dynamic-scheduler labels Dec 11, 2024
@GitHK GitHK added this to the Event Horizon milestone Dec 11, 2024
@GitHK GitHK marked this pull request as ready for review December 11, 2024 12:19
@GitHK GitHK requested a review from giancarloromeo December 11, 2024 12:19
Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this PR basically replaces the first part of this long trace to "restart a user-service" bu adding a buffer (i.e. rabbit) and on extra service (dynamic_scheduler)

Before: (3 services)

  • web-server ->(REST)-> director-v2 -> (REST) -> dynamic-sidecar -> (docker REST)

Now: (5 services)

  • web-server -(RPC)->rabbitMQ -> (RPC) -> dynamic_scheduler ->(REST)-> director-v2 -> (REST) -> dynamic-sidecar -> (docker REST)

Probably i am missing something but besides the buffer (which i think is good) I do not totally understand the role of the dynamic-scheduler in there.

⚠️ This is a good example to check with opentelemetry tracing... and see the impact of this design decision :-) !

@GitHK
Copy link
Contributor Author

GitHK commented Dec 13, 2024

So this PR basically replaces the first part of this long trace to "restart a user-service" bu adding a buffer (i.e. rabbit) and on extra service (dynamic_scheduler)

Before: (3 services)

  • web-server ->(REST)-> director-v2 -> (REST) -> dynamic-sidecar -> (docker REST)

Now: (5 services)

  • web-server -(RPC)->rabbitMQ -> (RPC) -> dynamic_scheduler ->(REST)-> director-v2 -> (REST) -> dynamic-sidecar -> (docker REST)

Probably i am missing something but besides the buffer (which i think is good) I do not totally understand the role of the dynamic-scheduler in there.

⚠️ This is a good example to check with opentelemetry tracing... and see the impact of this design decision :-) !

@pcrespov

There is one main reason for doing all this. In the context of moving away the dynamic scheduling logic form director-v2 (to the dynamic-scheduler), I'm trying to support two scheduling modes: via director-v2 and via dynamic-scheduler.

To achieve it, there needs to be a point of switching. The dynamic-scheduler, in this case, which decides which scheduler to use.

Another advantage is that in the end all scheduling logic will be placed inside the dynamic-scheduler and we are already rerouting all the calls from webserver to dynamic-scheduler in the context of dynamic services (only).

So it's required now and also in the future. Which is why I'm doing all this refactoring.

Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx for the explanation.

@GitHK GitHK enabled auto-merge (squash) December 17, 2024 11:47
@GitHK GitHK merged commit 38109f8 into ITISFoundation:master Dec 17, 2024
89 of 93 checks passed
@GitHK GitHK deleted the pr-osparc-redirect-service-restart-containers branch December 17, 2024 12:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:dynamic-scheduler a:webserver issue related to the webserver service
Projects
None yet
Development

Successfully merging this pull request may close these issues.

redirect service_restart_containers call via dynamic-scheduler
5 participants