Domain model / data schema in Scala, Protobuf and Avro

[SemanticBeeng]

Looking to develop a combination of gRPC services with mu-RPC and REST endpoints which share a common domain model.

Would like to reduce boiler plate of maintaining multiple versions of the model and wondering if skeumorph can help. This provides more context for the motivation https://medium.com/@sugarpirate/exploring-the-strongly-typed-graph-31fc27512326.

Do you see a problem in principle with attempting this?

Is skeumorph mature enough to make possible
1, maintaining the truth model in protobuf
2. generating Scala domain classes and @message classes (through a few custom morphisms)
3. generating Avro version of same model from 1.

Or some combination of the above.

[pepegar]

Hey @SemanticBeeng , glad to see you're interested in Skeuomorph!

Everything you've mentioned right there is possible to do right now using Skeuomorph, however, maybe you're more interested in https://github.com/higherkindness/compendium.

Compendium is, more or less, what the post you've mentioned above calls a schema registry. Right now is in a very early stage, but we're planning to make it a nice utility providing the following:

• act as a source of truth of schemas
• convert between IDL representations
• provide build tool plugins as integrations, see https://github.com/higherkindness/sbt-compendium
• perform syntactic analysis on schemas to avoid errors
• perform backward/foward compatibility validations on schema changes, and provide migrations, thanks in part to Introduce schema comparison #82

Right now, I repeat, is in an early stage, but we welcome new contributors, and beta testers too once it's usable.

[SemanticBeeng]

"convert between IDL representations"

For that it relies fully on skewmorph, correct ?

Or will there be some reusable morphisms in here?
Trying to understand if the user starts from scratch with the schema transformations or will there be any library of reusable ones. It would be great to see a few non-trivial morphism examples.

[SemanticBeeng]

Side question : json schema is sorely missing from skewmorph https://github.com/fge/json-schema-avro

What is the reason?

I stand corrected, just found https://github.com/higherkindness/skeuomorph/blob/master/src/test/scala/higherkindness/skeuomorph/openapi/OpenApiSchemaSpec.scala.
Main doc did not mention.

[SemanticBeeng]

Related topics:

01 How will skewmorph morphisms compare with matryoshka ?
Asking because I see "hylo ".

02 This work seems reusable https://github.com/vil1/recursion-schemes-cookbook/blob/master/schemas/README.md.

03 https://github.com/scalaz/scalaz-schema is similar but meant to Big Data applications.

Would be great to have an unifying way to manage data schema for all.
Thoughts about rationale: https://gist.github.com/SemanticBeeng/2edad7a5e2cd7cd6af5cde824b4e0da0

04 https://github.com/spartanz/schemaz

"generic representation of algebraic data structures and combinators that turn such schema into a generic computation over arbitrary data"
"Skeumorph is also a possible competitor, however it is focused on providing translation between different formats (Avro, Protobuf and Mu)."

Then it would be good to have support for reshaping between bounded contexts.
For example, service apis types may need to be mapped to (more rich/structured) domain model used to implement the functionality.

Currently there are:
01 https://beyondthelines.net/programming/fluent-the-seamless-translation-layer/

02 https://scalalandio.github.io/chimney/

Wondering if these morphisms can also be specified at the schema level?