Send output item data as VSBuffer

[jrieken]

Currently output items send data as number array. That's not ideal and use send things a VSBuffer for which the IPC layer is fast. This is a bigger change because it requires to stop using IOutputItemDto from extHost.protocol.ts

[jrieken]

kinda blocked by #125668

[RandomFractals]

@jrieken even the current output data array is chopped off: RandomFractals/vscode-data-table#30 (comment)

would be nice to read the whole binary array data from cell output. Are you truncating it for text data display only?

[mjbvz]

I looked into this a little since I'd done something similar for webviews. I think there are two potential approaches to optimize NotebookOutputItemDto:

Pass the the output data buffers as top level arguments to ipc calls

For any ipc call that deals with an cell output, pass the list of output VSBuffers as top level arguments:

$tryApplyEdits(editorId: string, modelVersionId: number, cellEdits: ICellEditOperationDto[], ...outputBuffers: VSBuffer[]): Promise<boolean>

This ensures that they are efficiently serialized by our existing ipc implementation

Then for NotebookOutputItemDto, we use a reference to tell us which output buffer from the list of buffers to use:

export interface NotebookOutputItemDto {
	readonly mime: string;
	readonly bufferIndex: number;
}

Making this change is sort of a pain because it requires updating a lot of to/from conversation functions to take an array of output buffers

Add ipc support for efficiently serializing VSBuffers inside of objects

First update NotebookOutputItemDto to use a VS Buffer

export interface NotebookOutputItemDto {
	readonly mime: string;
	readonly buffer: VSBuffer;
}

On its own, this would result in the buffer being very inefficiently serialized. So we'd also need update our IPC implementation to deeply detect and VSBuffers inside of passed in arguments and then serialize them more efficiently

This efficiently serialization would likely be similar to what we do for webviews:

• While serializing the arguments, walk each object to extract all buffers and replace them with references (just indexes into the list of extracted buffers)
• The write the serialized object and buffers to the wire
• On the receiving side while deserializing objects, walk though them again and replace all buffer references with the corresponding buffer

I feel this approach is probably better since it not specific to notebooks. However I'm not sure if it would significantly degrade performance for all ipc calls. I believe we already walk objects during serialization/deserialization to handle URIs so it may be ok. We could also make the behavior opt-in and only enable it for specific calls that need it

[RandomFractals]

I have not looked at what constitutes a VSBuffer. Would it be possible to instead use application/octet-stream mime type for cell outputs with binary data, and allow renderers to read them with a standard ReadableStream? See Using Readable Streams and Using Writable Streams.

NodeJS has much better documentation on streams. I assume VSBuffer does some of it. However, I don't like the proposed api above. I'd prefer to see cell outputs and renderers use standard JS APIs for data streaming and piping data to make them easier to test, etc.

I liked that current cell output API exposes blob(), but I was not able to read the whole data blob either in my tests.

This goes back to the original discussion on cell data outputs we started here: #123884 (comment)