Custom data types and schemas?

[andrew-otiv]

Does zenoh have any current or planned support for custom data types with schemas in some IDL, with codegen for multiple languages, similar to how DDS and ROS work?

If not, you probably have a good reason; do you have a link to a chat or document where the decision was discussed? It would be interesting if this were mentioned somewhere in the high level documentation or on the website, to help potential users contextualize zenoh vs. dds.

Thanks!

[JEnoch]

Since the very beginning of "what would become Zenoh" (was named Yaks for the record), it was decided to not tie the protocol and the API to any data encoding. Amongst the reasons were:

• Give the user the freedom to pick the data encoding that best suits his use case. Some might need self-descriptive encodings (such as JSON), while some would like more wire-efficient encodings (such as Protobuf or DDS' CDR)
• Support storages such as no-SQL databases (or filesystem) that could store heterogeneous data representations, without the need to transcode, as that's complex to implement and has impact on performances
• Keep the APIs more lightweight, not embedding serialization/deserialization code - This is important for embedded platforms
• Be able to efficiently integrate or bridge other protocols (such as DDS) just routing the data payload as a byte buffer

Therefore, the data representation is out of the scope of Zenoh protocol and API. Zenoh just deals with byte buffers as user data.
It's up to the user to choose and apply the data encoding he wants before giving the encoded data as a byte buffer to Zenoh.

In order to help user to deal with possibly multiple encodings in the system, Zenoh provides the possibility for the user to attach an encoding description to its data, in the form of a mime type (sent with optimisations in the Zenoh protocol). This allows custom data types to be used.
In addition, for few basic types and encoding (numbers, string, json), the APIs provide typed operations. But we're not keen to add support to other data types, if this requires more code or dependencies in the various APIs. We would prefer to see such support in an extension library over Zenoh.

[p-avital]

To complement @JEnoch's answer, if you wish to make serialization "transparent", I suggest implementing Into<Value> (or equivalently, inheriting from IntoValue in Python), as this will allow you to put the serialization code (as well as set the appropriate encoding) in a single spot.

[andrew-otiv]

OK, thanks for the response! It will be interesting to see how this plays out in terms of adoption and network effects.