Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardize client iterator #31

Open
pietercolpaert opened this issue Apr 11, 2023 · 3 comments
Open

Standardize client iterator #31

pietercolpaert opened this issue Apr 11, 2023 · 3 comments

Comments

@pietercolpaert
Copy link
Member

While the design of some LDES-es can be really simple, just keeping a history log of all pages and members processed to keep the state will not scale to really big LDES-es.

In earlier work, we have introduced the ldes:timestampPath to point at a property that can be used to keep the state in a different way: just resume from that timestamp.

If would be good to have a standard algorithm described in the spec to explain how a client must pause and resume from a certain state.

Coined by @sandervd

@sandervd
Copy link

To make this work efficiently, I think we would need a way to communicate to the client that the view uses a unidirectional linked list as pagination model between fragments.
If this is the case, the client can keep the last consumed fragment uri and the timestamp of the last processed member as state, and can continue from there on?

@pietercolpaert
Copy link
Member Author

Not convinced a unidirectional fragmentation is needed: the relations however do have to use the ldes:timestampPath as tree:path.

@sandervd
Copy link

sandervd commented May 4, 2023

If we go with this idea of 'profiles', that allow the client to discover how a view can be consumed, we could define the minimal relations that should be present in said profile?
e.g. tree:GreaterThanRelation is required in each fragment?
Having someting with even more simplistic semantics like nextOrderOfArrivalFragment would be nice to have, as in theory a fragment could contain multiple GreaterThanRelation relations?
If we want to keep the itterator logic as simple as possible, being able to have a explicit order beween fragments becomes important.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants