Presence

[curran]

I may be going out on a limb here, but what's your take on presence?

I've embarked on adding presence to json0.

Related proposal for OT spec changes: ottypes/docs#27

[josephg]

I like it and think its important, but for the JSON OT type I haven't given it much thought yet.

In a sense, we need the ability to create a cursor / marker (which might just be a path into some place in the document), and for that cursor to be moved around by other operations. So (marker, op) => marker. In the case of JSON operations, the marker is an array (potentially with another opaque object embedded at the end in the case of embedded child types).

I'm no longer convinced that we need the idea of a selection range. (Just use 2 cursors). And I'm not convinced that it makes sense for the OT code to care about your operations vs the operations of others. (Those differences should probably be handled in surrounding code).

[curran]

I really appreciate your input here. Thank you for giving this some thought.

I love your idea of the marker being an array. My original ideas were much more complicated.

Using the array idea, a marker might look something like this:

[
  'some', 'path', // Path of the presence.
  'ot-rich-text', // Subtype of the presence (a registered subtype).
  {               // Opaque presence object (subtype-specific structure).
    u: '123',     // An example of an ot-rich-text presence object.
    c: 8,
    s: [ [ 1, 1 ], [ 5, 7 ]]
  }
]

[curran]

FWIW, re: the OT code caring about your operations vs the operations of others, this informations ends up getting passed through to transformPosition. For the case of plain text and rich text, of the op inserts at the same position as a presence cursor, this information (isOwnOp) determines whether or not the presence cursor is transformed (corresponding in text0 to the side argument of transformCursor, and the insertAfter argument of transformPosition). At least, this is what I have observed by studying the ot-rich-text transformPresence implementation by @gkubisa.

Though it's implemented that way, I'm also not convinced that it's necessary. The need for OT code caring about your operations vs the operations of others could possibly be avoided if, after you submit an op, you also emit a change in your own presence. In this case, how you transform your own presence by an op you emit yourself would be a moot point (because the transformed presence would be overwritten anyway). Although, there may be cases I haven't considered here.

[gkubisa]

The need for OT code caring about your operations vs the operations of others could possibly be avoided if, after you submit an op, you also emit a change in your own presence. In this case, how you transform your own presence by an op you emit yourself would be a moot point (because the transformed presence would be overwritten anyway). Although, there may be cases I haven't considered here.

That's right, if you always submitted presence after an op, you would not need isOwnOp, however, transformPresence or transformCursor might be used in other contexts too, where transforming against own op is what's needed. Or you simply might not want to submit presence unnecessarily, if you know that your peers are going to transform it correctly. IMHO, keeping that param makes the API complete, as it allows the client code to control conflict resolution in a way similar to the side param in transform.

[curran]

By the way, there's been a new presence implementation added in ShareDB.

Here's an example of an OT type extended to support presence, compatible with ShareDB: ottypes/rich-text@ce14c8f#diff-4e2abe5a12642efc78b26087a61a8e9f

The task at hand is to come up with a transformPresence function.

FWIW I got presence working with json0 and ShareDB without transformations using the following:

import { type } from 'ot-json0';

type.transformPresence = function (presence, op, isOwnOp) {
  if (!presence) {
    return null;
  }
  return presence;
};

export { type };

Here's a snippet that interfaces with CodeMirror to generate the presence object:

const handleCursorActivity = () => {
  const from = codeMirror.getCursor(true);
  const to = codeMirror.getCursor(false);

  const doc = codeMirror.getDoc();
  const fromIndex = doc.indexFromPos(from);
  const toIndex = doc.indexFromPos(to);

  const presenceObject = {
    path,
    index: fromIndex,
    length: toIndex - fromIndex,
    userId: me.id,
  };

  localPresence.submit(presenceObject);
};
codeMirror.on('cursorActivity', handleCursorActivity);

[josephg]

Cool.

Yeah so - right now we have a working transformPosition function. This takes in a path (eg ['users', 3, 'address']) and an op, and returns the path after the operation has been applied. Eg:

const op = removeOp(['users', 1]) // remove the second user
const newPos = transformPosition(['users', 3, 'address'], op)
//  -> [ 'users', 2, 'address' ]

Its awkward to specify a selection range with json1, because I'm not sure what we should do we do if the start or end of the range moves somewhere else in the document.

In general though, its been a long time since I've thought about presence.

What do we need to make it work well for sharedb? What should the logic look like? I think that transformPosition method already does the heavy lifting. What else do we need to add / change to make it useful?

[curran]

I think what's needed is to:

• Decide on the shape of the presence object. I propose (strawman) path, index, and length.
• Expose a function called type.transformPresence that expects that shape and invokes transformPosition.
• Consider how this should work with nested OT types such as text-unicode (perhaps the string-specific things like index and length would be best suited as a nested text-unicode presence object, within the json1 presence object.)

[josephg]

Yep sounds good. I'm not sure what index and length refer to in this case - though that makes sense in the context of embedded types.

Wanna spearhead / lead this @curran?

[curran]

I kind of do.

Recycling some ideas from earlier for a more nuanced proposed shape for json1 + text-unicode presence:

{
  path: ['some', 'path'],  // Path of the presence.
  subtype: 'text-unicode', // Subtype of the presence (a registered subtype).
  subtypePresence: {       // Opaque presence object (subtype-specific structure).
    selections: [          // Let's say we support multiple selections in text-unicode
      {
        index: 4,          // The index in the string where the start of the selection is.
        length: 3          // The length of the selection (number of characters long).
      },
      { start: 9, length: 2 }
    ]
  }
]

I suppose tackling presence in text-unicode would be a nice sub-problem.

Opened a new issue to tackle text-unicode presence in isolation: ottypes/text-unicode#5

[samiur]

@curran @josephg Very excited to see movement on this! Let me know If I can help in anyway. We're currently using quill + sharedb with ot/rich-text, but there's a bunch of different problems that we think using ot/json1 as our root type would solve.

Side note: if either of you have any recommendations or examples on how to get started on creating a document structure using ot/json1 where rich-text subtypes are editable by quill, that'd be great!

[curran]

@samiur Excellent! Indeed, a working example of ot/json1 where rich-text subtypes are editable by quill would be amazing. Actually, that might be a better first step than adding presence to text-unicode, as presence is already implemented in rich-text.

This might be a good example for you to copy & modify to swap out json0 with json1: https://github.com/share/sharedb/tree/master/examples/rich-text-presence

Another thought - subtype and subtypePresence should probably be optional, as presence may be associated with just a path in a JSON data structure, not necessarily within a text or rich text value. So maybe a prototype of something could be built that shows presence in a JSON tree somehow.

[curran]

I haven't forgot about this and I do intend to work on it one day!

[curran]

I got it working over in this fork https://github.com/vizhub-core/json1-presence