diff --git a/README.md b/README.md index 0981b27..ec7524d 100644 --- a/README.md +++ b/README.md @@ -77,6 +77,7 @@ You can define bucketizers as follows: js:outputStreamId . ``` + #### Example of a time-based fragmentation ```turtle @@ -110,6 +111,31 @@ The members need to be arrived in order of their timestamps. When a member arrives, all buckets that hold members with a timestamp older than the new member's timestamp will be made immutable and no new members can be added to them. +#### Example of a timebucket based fragmentation + +```turtle + a tree:TimeBucketFragmentation; + tree:timestampPath ; + tree:buffer 5000; # members can arrive 5 seconds out of sync () + tree:level ( [ # Create 5 levels, resulting uri's //// + tree:range "year"; + tree:maxSize 0; # place no members at this level + ] [ + tree:range "month"; + tree:maxSize 0; # place no members at this level + ] [ + tree:range "day-of-month"; + tree:maxSize 1000; # place at most 1000 members at this level + ] [ + tree:range "hour"; + tree:maxSize 1000; # place at most 1000 members at this level + ] [ + tree:range "minute"; + tree:maxSize 10000; # place at most 10000 members at this level, this is the last level thus excess members are also put in this level + ] ). +``` + + ### [`js:Ldesify`](https://github.com/rdf-connect/sds-processors/blob/master/configs/ldesify.ttl#L10) This processor takes a stream of raw entities (e.g., out from a RML transformation process) and creates versioned entities appending the current timestamp to the entity IRI to make it unique. It is capable of keeping a state so that unmodified entities are filtered.