Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor to clean contracts for task input/ouput chain #25

Open
ghost opened this issue Dec 27, 2013 · 0 comments
Open

Refactor to clean contracts for task input/ouput chain #25

ghost opened this issue Dec 27, 2013 · 0 comments

Comments

@ghost
Copy link

ghost commented Dec 27, 2013

Currently the task input is a file. This does not work well with the bucket-idiom introduced with the API. Also, for downloaded payloads it currently blocks the interface.

We need to refactor the task (as well as web and API views) to accept a uniform payload identifier as input for a task. It then can be used to trigger further actions. For example if the payload id is bucket://1234 then a folder in the storage backend named 123 is used and analyzed, model identified, etc.. If url is specified download into a bucket should occur, etc.

This input resource identifier als allows for easy chaining of tasks. For exmaple it could first be passed as input to a task which determines that it needs to download a file first, which then is downloaded, decompressed into a bucket and the actual processing task is executed with that newly created bucket URI. etc. etc.

We need to define a standard for output format of a task, so this chaining/onion model is possible.

This was referenced Dec 27, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

0 participants