COMPENDIUM EXAMPLE

This project contains some sbt-compendium examples.

Before starting: requirements and configuration

Required:

First you'll need a working compendium server, check the compendium documentation. Please notice compendium requires a postgres database to save schemas so it offers a docker image to run a PostgreSQL database and a Compendium instance connected to that database. You can also install PostgreSQL on your local, create the required databases, compendium_metadata and compendium_protocols, and point compendium out to that PostgreSQL instance.

Second, in order to add new schemas in our compendium server, we'll have to make some http requests. Some call samples are provided below.

Set up configuration:

• Postgres in local.

By default: pointing to localhost:5432 and database postgres. Configurable on environment var: COMPENDIUM_METADATA_STORAGE_JDBC_URL by default: "jdbc:postgresql://localhost:5432/postgres"

• Docker

  More details on compendium microsite.

• Postman

  Import collection on postman/compendium.postman_collection.json. Please notice this has localhost:8080 as a default host for compendium server. POST calls has to give a string with the whole schema. It wil be parsed internally.

[Avro] Model example

Warning: Before compiling the project, make sure you have all the schemas in postgres.

To show this example run:

sbt "project avroExample" run

• The log traces will show some data.
• In target/scala-2.12/src_managed will appear scala files with all the case classes.
• On plugin/src/main/resources there are some csv files with data that correspond to its proper case class.

Let's suppose some data with the following structure:

name	fields
supplier	id_supplier, name, email, phone
sale	client*, product**
client*	id_client, name, surname, email
product**	id_prod, description, color, size
material	name, code, shipId

In postgres, schemas will be saved as a full string with the POST call:

name	schema
supplier	"{"type":"record","name":"Supplier","namespace":"higherkindness.compendiumtest","fields":[{"name":"id_supplier","type":"string"},{"name":"name","type":"string"},{"name":"email","type":"string"},{"name":"phone","type":"string"}]}"
sale	"{"type":"record","name":"Sale","namespace":"higherkindness.compendiumtest","fields":[{"name":"client","type":{"type":"record","name":"Client","fields":[{"name":"id_client","type":"string"},{"name":"name","type":"string"},{"name":"surname","type":"string"},{"name":"email","type":"string"}]}},{"name":"product","type":{"type":"record","name":"Product","fields":[{"name":"id_prod","type":"string"},{"name":"description","type":"string"},{"name":"color","type":"string"},{"name":"size","type":"string"}]}}]}"
client*	Generated by the nested schema in sale
product**	Generated by the nested schema in sale
material	"{"type":"record","name":"Material","namespace":"higherkindness.compendiumtest","fields":[{"name":"name","type":"string"},{"name":"code","type":"string"},{"name":"shipId","type":"string"}]}"

• Schemas have been flatted using From json to string Converter
• Data has been generated using Mockaroo.

Please, notice that in POST calls the identifiers need to be added to the compendiumProtocolIdentifiers list in the build.sbt file.

Avro files in JSON format

This section provides, in json format, the schemas presented previously. Also it contains new models and versions to test the behaviour of compendium. Remember, compendium always provides the last version saved unless otherwise stated.

Supplier schema:

 {
  "type": "record",
  "name": "Supplier",
  "namespace": "higherkindness.compendiumtest",
  "fields": [
    {
        "name": "id_supplier",
        "type": "string"
    }, 
    {
        "name": "name",
        "type": "string"
    },
    {
        "name": "email",
        "type": "string"
    },
    {
        "name": "phone",
        "type": "string"
    }
  ]
}

Sale schema:

{
  "type": "record",
  "name": "Sale",
  "namespace": "higherkindness.compendiumtest",
  "fields": [
    {
        "name": "client",
        "type": "higherkindness.compendiumtest.Client"
    },
    {
        "name": "product",
        "type": "higherkindness.compendiumtest.Product"
    }
  ]
}

Client schema [used in nested supplier schema]:

{
  "type": "record",
  "name": "Client",
  "namespace": "higherkindness.compendiumtest",
  "fields": [
    {
        "name": "id_client",
        "type": "string"
    },  
    {
        "name": "name",
        "type": "string"
    },
    {
        "name": "surname",
        "type": "string"
    },
    {
        "name": "email",
        "type": "string"
    }
  ]
}

Product schema [used in nested Supplier schema]:

{
  "type": "record",
  "name": "Product",
  "namespace": "higherkindness.compendiumtest",
  "fields": [
    {
        "name": "id_prod",
        "type": "string"
    },
    {
        "name": "description",
        "type": "string"
    },    
    {
        "name": "color",
        "type": "string"
    },
    {
        "name": "size",
        "type": "string"
    }
  ]
}

Material schema (first version):

{
  "type": "record",
  "name": "Material",
  "namespace": "higherkindness.compendiumtest",
  "fields": [
    {
        "name": "name",
        "type": "string"
    },
    {
        "name": "code",
        "type": "string"
    },
    {
        "name": "shipId",
        "type": "string"
    }
  ]
}

Future iterations

You can add a new version of each schema adding new fields.

Product schema (second version):

{
  "type": "record",
  "name": "Product",
  "namespace": "higherkindness.compendiumtest",
  "fields": [
     {
        "name": "id_prod",
        "type": "string"
    },
    {
        "name": "name",
        "type": "string"
    },
    {
        "name": "color",
        "type": "string"
    },
    {
        "name": "size",
        "type": "string"
    },
    {
        "name": "soldDate",
        "type": "string"
    }
  ]
}

Material schema (second version):

{
  "type": "record",
  "name": "Material",
  "namespace": "higherkindness.compendiumtest",
  "fields": [
    {
        "name": "name",
        "type": "string"
    },
    {
        "name": "code",
        "type": "string"
    },
    {
        "name": "originIdentifier",
        "type": "higherkindness.compendiumtest.Supplier"
    },    
    {
        "name": "shipId",
        "type": "string"
    },    
    {
        "name": "color",
        "type": "string"
    }
  ]
}

Process version:

{
  "type": "record",
  "name": "Process",
  "namespace": "higherkindness.compendiumtest",
  "fields": [
    {
        "name": "name",
        "type": "string"
    },
    {
        "name": "mat",
        "type": {
            "type": "array",
            "items": "higherkindness.compendiumtest.Material"
        }
    },
    {
        "name": "prod",
        "type": "higherkindness.compendiumtest.Product"
    }
  ]
}

[Proto] Model example

Warning: Before compiling the project, make sure you have all the schemas in postgres.

The following dependency is mandatory for protobuf type:

 "com.47deg"    %% "pbdirect" % "0.4.1"

To show this example run:

sbt "project protoExample" run

• The log traces will show some data.
• In target/scala-2.12/src_managed will appear scala files with all the case classes.
• In this case, each class will be created inside a compendium package object. To access to a certain class you'll need to import import [actually.your.package].compendium

Structure

Let's suppose some data with the following structure:

name	fields
supplier	id_supplier, name, email, phone
sale	client*, product**
client*	id_client, name, surname, email
product**	id_prod, description, color, size
material	name, code, shipId

In postgres, schemas will be saved as a full string with the POST call:

"syntax = \"proto3\";\n\npackage higherkindness.compendiumtest;\n\nmessage Supplier {\n  string id_supplier = 1;\n  string name = 2;\n  string email = 3;\n  string phone = 4;\n}\n\nmessage Client {\n  string id_client = 1;\n  string name = 2;\n  string surname = 3;\n  string email = 4;\n}\n\nmessage Product {\n  string id_prod = 1;\n  string description = 2;\n  string color = 3;\n  string size = 4;\n}\n\nmessage Sale {\n Client client = 1;\n Product product = 2;\n}\n\n\nservice SearchOps {\n  rpc FindProducts (Client) returns (Product);\n  rpc FindClients (Product) returns (Client);\n}"

Full schema

syntax = "proto3";

package higherkindness.compendiumtest;

message Supplier {
  string id_supplier = 1;
  string name = 2;
  string email = 3;
  string phone = 4;
}

message Client {
  string id_client = 1;
  string name = 2;
  string surname = 3;
  string email = 4;
}

message Product {
  string id_prod = 1;
  string description = 2;
  string color = 3;
  string size = 4;
}

message Sale {
  Client client = 1;
  Product product = 2;
}


service SearchOps {
  rpc FindProducts (Client) returns (Product);
  rpc FindClients (Product) returns (Client);
}