v3.3.0

What's Changed

Fragmentation revamp: Bucketisation & Pagination 📇

Although no breaking changes on the front end have been introduced, you might see in the logs that there is one fragmentation job that will run periodically.
This job will, for each view, first define the buckets in the bucketisation step. After that, the pagination step will assign the bucketised members to pages.
This was done in terms of performance to decouple the ingestion process from fragmentation.
Note that further releases will improve on performance of these processes.

Fragmentation will check every 30 seconds for unprocessed members.
If you wish to use a specific interval, update the ldes-server.fragmentation-cron spring value with a valid spring cron.

Fragmentation metrics 📈

In terms of observability, we also provide an update for all you Prometheus fans!
You will now have 3 metrics to follow up the internal processes of your LDES Server:

• ldes_server_ingested_members_count : Will keep track of the ingested members per collection.
• ldes_server_bucket_members_count : Will keep track of all bucketised members per view.
• ldes_server_pagination_members_count : Will keep track of all paginated members per view.

Skolemization ♻️

As per request of one of our onboarders, we introduced skolemization.
This feature allows the LDES Server to replace blank nodes with skolemized URIs.
For more details, check out the new documentation

Known Issues 🚧

Compaction vs Fragmentation

Whilst preparing for this release, we noticed how the newly created compaction URIs conflict with our pagination.
This issue is being tracked as part of #1365
We advice you to turn off compaction. This can be done by setting the compaction cron to -.

Change Notes

• chore: finalise 3.2.0 by @Yalz in #1332
• doc: update event source documentation by @jeflaenen in #1333
• Use latest cache actions by @rorlic in #1345
• chore: #1346: auto enable Spring Batch table creation + filter out ac… by @Yalz in #1349
• feat: skolemization by @jobulcke in #1351
• chore: improve ci pipeline by @Yalz in #1352
• feat: data model: retention + fragmentation by @Yalz in #1343
• fix: missing skolemization domain shacl uri validation by @jobulcke in #1354
• feat: include expanded metrics + general fragmentation fixes by @Yalz in #1356
• fix compaction by @Yalz in #1358
• feat: run fragmentation jobs per view in parallel by @Yalz in #1359
• fix: broken needs_pagination db view by @jobulcke in #1360
• chore: make fragmentation jobs parallel by @Yalz in #1361
• chore: removed incorrect unique constraint and added some indexes by @Yalz in #1362
• chore: finalise 3.3.0 release by @Yalz in #1363
• docs: skolemization updates by @Yalz in #1367

New Contributors

• @rorlic made their first contribution in #1345

Full Changelog: v3.0.0...v3.3.0