Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache non-finished pipeline models to prevent loss of data when crash #385

Closed
Yalz opened this issue Oct 27, 2023 · 1 comment
Closed

Cache non-finished pipeline models to prevent loss of data when crash #385

Yalz opened this issue Oct 27, 2023 · 1 comment
Assignees
Labels
component: ldio Issues related to Linked Data Interactions enhancement New feature or request move to JIRA

Comments

@Yalz
Copy link
Collaborator

Yalz commented Oct 27, 2023

Is your feature request related to a problem? Please describe.
To prevent data loss, a solution needs to be put into place that caches models that are inside the LDIO pipelines

Describe the solution you'd like
Usage of Redis DB that will, on ingest save the member whilst on output delete the member again.
Should first replay members that are in cache on reboot.

@Yalz Yalz added enhancement New feature or request component: ldio Issues related to Linked Data Interactions labels Oct 27, 2023
@github-project-automation github-project-automation bot moved this to 📋 Backlog in VSDS Backlog Oct 27, 2023
@jeflaenen
Copy link
Contributor

https://vlaamseoverheid.atlassian.net/browse/VSDSPUB-1063 is logged in the JIRA backlog to implement this request.

@github-project-automation github-project-automation bot moved this from 📋 Backlog to 👀 In review in VSDS Backlog Dec 4, 2023
@Yalz Yalz moved this from 👀 In review to ✅ Done in VSDS Backlog Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: ldio Issues related to Linked Data Interactions enhancement New feature or request move to JIRA
Projects
Archived in project
Development

No branches or pull requests

2 participants