Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compatibility to Linked Data Event Streams #8

Open
pietercolpaert opened this issue Aug 17, 2021 · 1 comment
Open

Compatibility to Linked Data Event Streams #8

pietercolpaert opened this issue Aug 17, 2021 · 1 comment

Comments

@pietercolpaert
Copy link

We launched the Linked Data Event Streams (LDES) specification with a focus on making third parties replicate the full history of a dataset, and stay in-sync with it. It is available at https://w3id.org/ldes/specification

It has a different set-up as the delta-notifier’s v0.0.1 format, which is based on additions/updates/deletions of triples. LDES advocates defining a collection of entities (e.g., a collection of mandates). LDES requires you to add version identifiers to the mandates (e.g., ex:MayorofGhent2019v1) and add all triples that you want to expose in this particular collection (you can document this in a shacl shape). We thus only add triples to the system: triples describing a new version of the entity. In order to keep that manageable, you can describe a retention policy and that way remove older data or previous versions from disk.

Some examples of feeds already available:

Components that already exist that allows you to work with LDES:

I wonder whether the delta-notifier could also work with LDES by:

  1. Subscribing to an LDES using the LDES client https://github.com/TREEcg/event-stream-client/tree/main/packages/actor-init-ldes-client
  2. Sending the new version objects to the right mu-semtech microservice

It is then a responsibility of that microservice to understand what it needs to do with that version object. One option could be to just add triples as-is in a store (this is similar to what we do in LDES2Service). Another option could be to create a version materialization (making the subject’s IRI the IRI of the dcterms:isVersionOf object, and changing dcterms:created into dcterms:modified), removing all triples of the previous version and loading the new version in the store.

@madnificent
Copy link
Member

This did not receive a public response yet but: thank you for the issue!

The delta-notifier is a bit more detailed in what it communicates. Constructing LDES feeds is therefore done on a per-case basis with a shared common ground. We may later construct something more generic. Leaving this issue open for the time being to keep that in sight. Related repositories:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants