Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Client API #253

Open
johanhenriksson opened this issue Jan 25, 2021 · 0 comments
Open

Improve Client API #253

johanhenriksson opened this issue Jan 25, 2021 · 0 comments
Assignees
Labels
enhancement Improve existing features high High Priority

Comments

@johanhenriksson
Copy link
Collaborator

johanhenriksson commented Jan 25, 2021

The CLI currently contains a lot of code that could be useful for interacting with Cowait from other Python applications. We'd like to extract this into an API client that can be used on its own. Among other things, this is required to implement #287

For the API client to be truly useful it should be able to interact with RemoteTask references similar to what is possible when creating subtasks. A key observation is that all the information available to parent tasks through the websocket connection is also available through the structured log output. This means that we can subscribe to a task through Docker or Kubernetes by attaching to the container logs and parsing them. While its not a perfect solution, it provides a means to retrieve status updates, errors and return values from tasks. RPC calls can not be supported this way.

RemoteTask can then properly implement __await__() and become API-compatible with Task.spawn(). For use cases where asyncio is not desired, a RemoteTask.wait() method could be implemented. It should block until the task completes and either return the result or throw an error.

Ideally, attaching and following container logs should be done in an asynchronous fashion so that multiple tasks could run in parallel. Docker/Kubernetes SDKs provide limited options for asyncio, which means we can either implement our own websocket logs functionality for them - or use additional dependencies. aiodocker seems like a good option for Docker, kubernetes_asyncio could be an option for kubernetes. Both of them have relatively few dependencies (most of which are already dependencies of Cowait), but would still bloat the library considerably.

Other considerations:

  • Should the client respect cowait.yml configuration by default? Is it the clients' responsibility to load configuration?
@johanhenriksson johanhenriksson added enhancement Improve existing features high High Priority labels Jan 25, 2021
@johanhenriksson johanhenriksson self-assigned this Apr 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improve existing features high High Priority
Projects
None yet
Development

No branches or pull requests

1 participant