Smarter state with timestampPath

[pietercolpaert]

The state can be kept more efficiently, even without immutable caching headers, when the timestampPath is set:

The state then may become:

• The last processed timestamp, and the IRI of all members on that timestamp
• The rootnode and its expiration timestamp, or the fact the root is immutable
• The border node’s expiration or the fact they became immutable
  • All visited nodes that were reachable from a non bounded relation (i.e. there’s a greaterThanRelation, but not a lessThanRelation when traversing in ascending order), and also don’t match the bounds of their parents,
  • non-timestamp-based relations, and also don’t match the bounds of their parents, and
  • all currently on-going nodes (lessThan timestamps did not reach the time of our iterator yet)

All relations that point at earlier data can be forgotten.

[pietercolpaert]

A more concise rewrite of the above:

Given that:

1. an ldes:timestampPath is set,
2. a relation uses that timestampPath in the relation, and
3. we’re processing chronologically (ascending)

Then, when the relation points at something that is earlier than the current time we’re processing, a client can ignore the relation and not include it in the state.

The state however needs to be extended with:

1. The fact timestampPath can be taken into account and the last processed timestamp
2. The IRIs of all members on that timestamp that were already processed (otherwise, a member on exactly that timestamp might be emitted a second time when resuming)