move durable-Kind nextInstanceID out to separate vatstore key

[warner]

What is the Problem Being Solved?

Currently, for each virtual Kind, we store a single vatstore key named vom.vkind.${kindID}, with a metadata record named the "Kind Descriptor". This contains { kindID, tag }, and either unfaceted: true or facets: [facetName1, facetName2, ..].

Durable Kinds get the same thing, except in a key named vom.dkind.${kindID}, and they get an additional property named nextInstanceID (an integer, incremented each time an instance of this kind is created).

Once #7338 is implemented, the descriptors will also include a serialized stateShape capdata record (with { body, slots }). For durable Kinds, this may be allowed to change during subsequent buildRootObject calls (subject to some innovation on shape-compatibility -checking functions).

For virtual kinds, the descriptor is written to the vatstore during buildRootObject (or some later crank, whichever one calls defineDurableKind), and never read again, because it is also remembered in RAM for the remainder of the incarnation. It ought to be deleted during upgrade, since no virtual (non-durable) objects survive an upgrade, but don't currently delete the vkind records. We write this descriptor for completeness, and to support debugging/recovery of merely-virtual data. It is not needed for operational purposes.

For durable kinds, we write the initial descriptor during the crank that creates the Kind, and again each time nextInstanceID is incremented. The nextInstanceID counter is also kept in RAM (and hence we don't do a vatstore read on each new allocation), but we write the record out after each allocation so the DB form is accurate: a subsequent incarnation needs to allocate new instances from the right starting point, and not assign duplicate IDs to distinct objects. After each upgrade, we'll read the descriptor during buildRootObject to compare the facet/unfaceted "facetiousness" against the earlier version, to ensure that it hasn't changed (userspace is required to redefine all durable Kinds, but is not allowed to change the facetiousness or change the names or number of the facets).

As raised in #7338 (comment) :

The KindDescriptor also holds the next instance ID, so it's a bit too "hot" to hold large cold data like state shapes. So we might want to split it into separate vatstore keys, a hot counter and a cold metadata one. Given the disruption of that, we might want to do that now, rather than later, even if we don't need the extra metadata for a while yet.

Description of the Design

We'll move the Kind Descriptor to vom.dkind.${kindID}.descriptor. However we'll remove nextInstanceID from that descriptor, and record it in a separate key, vom.dkind.${kindID}.nextID.

During the first crank that calls defineDurableKind, we'll write out both records:

• vom.dkind.${kindID}.descriptor: { kindID, tag, unfaceted | facets, stateShape }
• vom.dkind.${kindID}.nextID: 1

Both values are held in RAM, so we won't read these keys again for the remainder of the incarnation.

Each time a new instance is allocated, in addition to updating the in-RAM counter, we'll write a new copy of the counter out to vom.dkind.${kindID}.nextID.

At the start of the next incarnation, during buildRootObject (when defineDurableKind is mandated to be called for all existing durable kinds), we'll read both records, to initialize the RAM values. We may also write out a new Kind Descriptor, if/when we allow the stateShape to change, or if we start to include versioning metadata.

Security Considerations

When building our vatstore keys (or kvStore keys in general), we usually refrain from putting static fields as a suffix to variable fields, to reduce the danger of format/parsing confusion and colliding keys. To follow this guideline, we'd need to create a new prefix for state data, and use vatstore keys more like:

• vom.dkind.descriptor.${kindID}
• vom.dkind.nextID.${kindID}

However I don't want to do that here, because I want the two records for any given kindID to appear next to each other in a sorted dump of the DB. And, I know that ${kindID} is always an integer (and the only user influence is the fact that they are incremented sequentially, by makeKindHandle).

In the longer run, we're hoping to get away from using string encoding to build vatstore keys, and get structured data like this stored in SQL more directly (#6677). So we might wind up with a dedicated SQL table for Kinds, with (kindID, durable, facets, stateShapeCapData, nextID) as columns, and let SQLite be responsible for efficient caching and hot/cold-ness of different fields.

Scaling Considerations

This will slightly increase the space needed per Kind, but remember that Kinds are low-cardinality.

This should slightly improve the speed of creating new Kind instances, because we're writing less data to the vatstore each time (just the new nextID counter, rather than re-writing the entire descriptor record).

Test Plan

Update unit tests to check that the new keys are in place.