Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support pool.sync_updates from remote_pool repo #6108

Open
wants to merge 4 commits into
base: feature/easier-pool-join
Choose a base branch
from

Conversation

gangj
Copy link
Contributor

@gangj gangj commented Nov 8, 2024

CP-50787 CP-51347: Support pool.sync_updates from remote_pool repo

When a remote_pool type repository, which points to the enabled
repository in the remote pool coordinator, is set as the enabled
repository of the pool, updates can be synced from it with API
pool.sync_updates.

The username password of the remote pool coordinator is required as
parameters for pool.sync_updates to login the remote pool.

And the remote pool coordinator's host server certificate needs to be
configured in the remote_pool repository, it will be used to verify the
remote end when sending out username passwords.

A new yum/dnf plugin "xapitoken" is introduced to set xapi token as HTTP
cookie: "session_id" for each HTTP request which downloads files from the
remote_pool repository.

CP-52245: Temp disable repo_gpgcheck when syncing from remote_pool repo

Will re-enable repo_gpgcheck after CP-51429 is done by reverting this
commit.

@gangj gangj force-pushed the private/gangj/CP-50787_CP-51347 branch 2 times, most recently from ebb463a to 841c232 Compare November 8, 2024 09:10
python3/dnf_plugins/xapitoken.py Outdated Show resolved Hide resolved
python3/dnf_plugins/xapitoken.py Show resolved Hide resolved
@gangj gangj force-pushed the private/gangj/CP-50787_CP-51347 branch 5 times, most recently from 3ff9d75 to a2f74fd Compare November 11, 2024 08:40
ocaml/xapi/repository.ml Outdated Show resolved Hide resolved
ocaml/xapi/repository_helpers.ml Outdated Show resolved Hide resolved
ocaml/idl/datamodel_pool.ml Outdated Show resolved Hide resolved
ocaml/xapi/helpers.ml Outdated Show resolved Hide resolved
ocaml/xapi/repository.ml Outdated Show resolved Hide resolved
ocaml/xapi/repository_helpers.ml Outdated Show resolved Hide resolved
ocaml/xapi/repository.ml Outdated Show resolved Hide resolved
ocaml/xapi/xapi_pool.ml Outdated Show resolved Hide resolved
let sync ~__context ~self ~token ~token_id =
let ext_host_verified_rpc ~__context ~cert host_address xml =
try Helpers.make_external_host_verified_rpc ~__context host_address cert xml
with Xmlrpc_client.Connection_reset ->
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now when the certificate is invalid, the expected ssl_verify_error is not raised, instead this Xmlrpc_client.Connection_reset is caught here. I think this problem can be resolved with another ticket later.

@@ -1919,6 +1919,16 @@ let _ =
"If the bundle repository or remote_pool repository is enabled, it \
should be the only one enabled repository of the pool."
() ;
error Api_errors.update_syncing_remote_pool_coordinator_connection_failed []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the context of the error is clear as it would happen when a pool.sync_updates. It could be as simple as "CANNOT_CONTACT_HOST".
But the authentication (incorrect password) may fail with this error also. So the possible causes should be mentioned in the message.

Copy link
Contributor Author

@gangj gangj Nov 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The authentication (incorrect password) failure will fail with another error: update_syncing_remote_pool_coordinator_service_failed.
And we have a corresponding error pool_joining_host_connection_failed.
If we change it to a general one, I think we will need to provide more info like the address and port of the host which failed to be contacted, like the other existing one: error Api_errors.tls_connection_failed ["address"; "port"]

ocaml/xapi/xapi_pool.ml Outdated Show resolved Hide resolved
ocaml/xapi/repository_helpers.ml Outdated Show resolved Hide resolved
ocaml/xapi/repository.ml Outdated Show resolved Hide resolved
ocaml/xapi/repository.ml Outdated Show resolved Hide resolved
ocaml/xapi/repository.ml Outdated Show resolved Hide resolved
ocaml/idl/datamodel_repository.ml Outdated Show resolved Hide resolved
ocaml/xapi/repository.ml Outdated Show resolved Hide resolved
ocaml/xapi/helpers.ml Outdated Show resolved Hide resolved
"There was an error connecting to the remote pool coordinator while \
syncing updates from it."
() ;
error Api_errors.update_syncing_remote_pool_coordinator_service_failed []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be useful to the user, how about to use "HOST_OFFLINE"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error corresponds to the error during pool join: pool_joining_host_service_failed, I think it is not host offline, it is the HTTP service failed to respond.

@gangj gangj force-pushed the private/gangj/CP-50787_CP-51347 branch from 04f1cec to 9a9fa40 Compare December 10, 2024 17:19
@gangj gangj force-pushed the private/gangj/CP-50787_CP-51347 branch 2 times, most recently from 642e1ac to 61c948b Compare December 12, 2024 05:06
When a remote_pool type repository, which points to the enabled
repository in the remote pool coordinator, is set as the enabled
repository of the pool, updates can be synced from it with API
pool.sync_updates.

The username password of the remote pool coordinator is required as
parameters for pool.sync_updates to login the remote pool.

And the remote pool coordinator's host server certificate needs to be
configured in the remote_pool repository, it will be used to verify the
remote end when sending out username passwords and syncing updates from
it.

A new yum/dnf plugin "xapitoken" is introduced to set xapi token as HTTP
cookie: "session_id" for each HTTP request which downloads files from the
remote_pool repository.

Signed-off-by: Gang Ji <[email protected]>
Will re-enable repo_gpgcheck by reverting this commit after CP-51429 is done.

Signed-off-by: Gang Ji <[email protected]>
@gangj gangj force-pushed the private/gangj/CP-50787_CP-51347 branch from 61c948b to cf7af87 Compare December 12, 2024 05:11
@gangj gangj requested a review from minglumlu December 12, 2024 05:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants