diff --git a/.github/workflows/docs-site-deploy.yaml b/.github/workflows/docs-site-deploy.yaml index 8aa8578..e44ffea 100644 --- a/.github/workflows/docs-site-deploy.yaml +++ b/.github/workflows/docs-site-deploy.yaml @@ -1,6 +1,8 @@ name: docs-site-deploy on: push: + tags: + - * branches: - main permissions: @@ -26,10 +28,8 @@ jobs: path: .cache restore-keys: | mkdocs-material- - - run: pip install mkdocs-open-in-new-tab mike - - run: pip install mkdocs-material - - run: pip install "mkdocs-material[imaging]" - - run: bash script/build_docs.sh + - run: pip install mkdocs-material mkdocs-open-in-new-tab "mkdocs-material[imaging]" mkdocs-awesome-pages-plugin mike + - run: bash src/script/build_docs.sh - run: | latest_tag=$(git describe --tags --abbrev=0) mike deploy --push --update-aliases "$latest_tag" latest diff --git a/.github/workflows/validate-examples.yaml b/.github/workflows/validate-examples.yaml new file mode 100644 index 0000000..9f66220 --- /dev/null +++ b/.github/workflows/validate-examples.yaml @@ -0,0 +1,18 @@ +name: validate-examples +on: + push: + branches: ["*"] + +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: latest + - name: Install ajv + run: | + npm i -g ajv-cli ajv-formats + - name: Validate examples + run: bash src/script/validate-examples.sh diff --git a/.gitignore b/.gitignore index 10b3150..a0889dc 100644 --- a/.gitignore +++ b/.gitignore @@ -3,3 +3,10 @@ .idea site *.iml +.cache + +docs/changelog.md +docs/contributing.md +docs/home.md +docs/vendors.md +docs/examples/**/*.md \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md index 83a3485..cb5a62c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,76 @@ image: "https://raw.githubusercontent.com/bitol-io/artwork/main/horizontal/color This document tracks the history and evolution of the **Open Data Contract Standard**. +# v3.0.0 - 2024-10-05 - IN REVIEW + +* **New section**: Support & communication channels. +* **New section**: Servers. +* **Changes** to fundamentals : + * Rename `uuid` to `id`. + * Add `name`. + * Rename `quantumName` to `dataProduct` and make it optional. + * Rename `datasetDomain` to `domain` (we avoid the dataset prefix). + * Drop `datasetKind` (example: `virtualDataset`, was optional, have not seen any usage). + * Drop `userConsumptionMode` (examples: `analytical`, was optional, already deprecated in v2.). + * Drop `sourceSystem` (example: `bigQuery`, information will be encoded in servers). + * Drop `sourcePlatform` (example: `googleCloudPlatform`, information will be encoded in servers). + * Drop `productSlackChannel` (will move to support channels). + * Drop `productFeedbackUrl` (will move to support channels). + * Drop `productDl` (will move to support channels). + * Drop `username` (credentials should not be stored in the data contract). + * Drop `password` (credentials should not be stored in the data contract). + * Drop `driverVersion` (will move to servers if needed). + * Drop `driver` (will move to servers if needed). + * Drop `server` (will move to servers if needed). + * Drop `project` (BigQuery-specific, will move to servers). + * Drop `datasetName` (BigQuery-specific, will move to servers). + * Drop `database` (BigQuery-specific, will move to servers). + * Drop `schedulerAppName` (not part of the contract). +* **Changes** to Schema: + * Major changes, check spec. + * Adds support for non table formats, hierarchies, and arrays. + * `name` is a new field + * `items` is a new field + * `priorTableName` is not supported anymore, if needed, consider a custom property. + * `table` is not supported anymore, if needed, consider using `name`. + * `columns` is now `properties` + * `dataGranularity` is now `dataGranularityDescription`. + * `encryptedColumnName`is now `encryptedName`. + * `partitionStatus` is now `partitioned`. + * `clusterStatus` is not supported anymore, if needed, consider a custom property. + * `clusterKeyPosition` is not supported anymore, if needed, consider a custom property. + * `sampleValues` is now `examples`. + * `isNullable` is now `required`. + * `isUnique` is now `unique`. + * `isPrimaryKey` is now `primaryKey`. + * `criticalDataElementStatus` is now `criticalDataElement`. + * `clusterKeyPosition` is not supported anymore, if needed, consider a custom property. + * `transformSourceTables` is now `transformSourceObjects` + * Restrict `schema.*.logicalType` to be one of `string`, `date`, `number`, `integer`, `object`, `array`, `boolean`. + * Add `schema.*.logicalTypeOptions`. +* **Changes** to Data Quality: + * Significant changes have been applied to support more tools and use cases. Please review the new section. + * If needed, `templateName` is a custom property. + * `toolName` is obsolete, replaced by `type=custom; engine: `. + * `scheduleCronExpression` is replaced by `schedule` and `scheduler`. `scheduleCronExpression: 0 20 * * *` becomes `schedule: 0 20 * * *` and `scheduler: cron`. +* Pricing: + * No changes. +* **Changes** to team (fka stakeholders): + * Replaces `stakeholders`. Content stays the same. +* **Changes** to Role: + * Added `description` + * Changed `access` is not required anymore +* Security: + * No changes. +* **Changes** to SLA: + * Starting with v3, the schema is not purely tables and columns, hence minor modifications: columns are now elements. + * `slaDefaultColumn` is now `slaDefaultElement`. + * `column` is now `element`. + * Explicit reference to Data QoS. +* **Changes** to custom and other properties: + * `systemInstance` is not supported anymore, if needed, consider a custom property. + + # v2.2.2 - 2024-05-23 - APPROVED * In JSON schema validation: @@ -14,12 +84,13 @@ This document tracks the history and evolution of the **Open Data Contract Stand * Change `price.priceAmount` data type from `string` to `number`. * Change `slaProperties.value` data type from `string` to `oneOf[string, number]`. * Change `slaProperties.valueExt` data type from `string` to `oneOf[string, number]`. -* Update [examples](docs/examples) to adhere to JSON schema. -* Full example from README directs to [full-example.yaml](docs/examples/all/full-example.yaml). +* Update [examples](docs/examples/README.md) to adhere to JSON schema. +* Full example from README directs to [full-example.yaml](docs/examples/all/full-example.odcs.yaml). * Add in mkdocs for creating a [documentation website](https://bitol-io.github.io/open-data-contract-standard/). Check [building-doc.md](building-doc.md). * Add vendors page [vendors.md](vendors.md). Feel free to add anyone there. -# v2.2.1 - 2023-12-18 - APPROVED + +# v2.2.1 - 2023-12-18 - REPLACED BY V2.2.2 * Reformat quality examples to be valid YAML. * Type of definition for authority have standard values: `businessDefinition`, `transformationImplementation`, `videoTutorial`, `tutorial`, and `implementation`. @@ -28,7 +99,8 @@ This document tracks the history and evolution of the **Open Data Contract Stand * Integrated as part of [Bitol](https://lfaidata.foundation/projects/bitol/). * Reformat Markdown tables. -# v2.2.0 - 2023-07-27 - APPROVED + +# v2.2.0 - 2023-07-27 - REPLACED BY V2.2.1 * New name to Open Data Contract Standard. * `templateName` is now called `standardVersion`, v2.2.0 parsers should account for this change and support both to avoid a breaking change. @@ -37,12 +109,14 @@ This document tracks the history and evolution of the **Open Data Contract Stand * Various improvements and typo corrections. * Finalization of fork under AIDA User Group. -# v2.1.1 - 2023-04-26 - APPROVED + +# v2.1.1 - 2023-04-26 - REPLACED BY V2.2.0 * Open source version. * Additional value field `valueExt` in SLA. -# v2.1.0 - 2023-03-23 - APPROVED + +# v2.1.0 - 2023-03-23 - REPLACED BY V2.1.1 ## Data Quality The data contract adds elements specifically for interfacing with the Data Quality tooling. @@ -68,7 +142,7 @@ The service-level agreements not previously used are more detailed to follow the ## Other Removed the weight for system ratings from the data contract. Their default values remain. -# v2.0.0 - SUPERSEED BY V2.1.0 +# v2.0.0 - REPLACED BY V2.1.0 ## Guidelines & Evolution * [Type case](https://google.github.io/styleguide/jsoncstyleguide.xml?showone=Property_Name_Format#Property_Name_Format) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 022d4c2..10c36c4 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -4,88 +4,7 @@ description: "How you can contribute to the Open Data Contract Standard (ODCS)." image: "https://raw.githubusercontent.com/bitol-io/artwork/main/horizontal/color/Bitol_Logo_color.svg" --- -# Open Data Contract Standard +# Contributing to Open Data Contract Standard -## Executive summary -First off, thanks for taking the time to contribute! ❤️ +Thank you for your interest in contributing to Open Data Contract Standard (ODCS). Please refer to the [TSC contributing guidelines](https://github.com/bitol-io/tsc/blob/main/CONTRIBUTING.md). -All types of contributions are encouraged and valued. See the [Table of Contents](#table-of-contents) for different ways to help and details about how this project handles them. Please make sure to read the relevant section before making your contribution. It will make it a lot easier for us maintainers and smooth out the experience for all involved. The community looks forward to your contributions. 🎉 - -You do not have to be a member of AIDA User Group to contribute, although becoming a member is free. Strength is always in the number. Check [it out](https://aidausergroup.org/join/). - -> And if you like the project, but just don't have time to contribute, that's fine. There are other easy ways to support the project and show your appreciation, which we would also be very happy about: -> - Star the project. -> - Tweet about it. -> - Refer this project in your project's readme. -> - Mention the project at local meetups and tell your friends/colleagues. - - -## Table of Contents - -- [Code of Conduct](#code-of-conduct) -- [I Have a Question](#i-have-a-question) -- [I Want To Contribute](#i-want-to-contribute) -- [Suggesting Enhancements](#suggesting-enhancements) -- [Improving The Documentation](#improving-the-documentation) -- [Join The Project Team](#join-the-project-team) - - -## Code of Conduct - -This project and everyone participating in it is governed by the -[Open Data Contract Standard Code of Conduct](blob/master/CODE_OF_CONDUCT.md). -By participating, you are expected to uphold this code. Please report unacceptable behavior -to [@jgperrin](https://github.com/jgperrin). - - -## I Have a Question - -** New ** - -AIDA User Group also opened its Slack for Data Contract discussion. It is an alternate way of contributing to this project. The Slack channel is now [available](https://aidaug.slack.com/archives/C05UZRSBKLY). - -You have to be a member of AIDA User Group (it's free) to have access to our Slack channel. All the details are [here](https://aidausergroup.org/welcome/). - -> If you want to ask a question, we assume that you have read the available [Documentation](https://github.com/AIDAUserGroup/open-data-contract-standard). - -Before you ask a question, it is best to search for existing [Issues](https://github.com/bitol-io/open-data-contract-standard/issues) that might help you. In case you have found a suitable issue and still need clarification, you can write your question in this issue. It is also advisable to search the internet for answers first. - -If you then still feel the need to ask a question and need clarification, we recommend the following: - -- Open a [New Issue](https://github.com/bitol-io/open-data-contract-standard/issues/new). -- Provide as much context as you can about what you're running into. - -We will then take care of the issue as soon as possible. - -## I Want To Contribute - -> ### Legal Notice -> When contributing to this project, you must agree that you have authored 100% of the content, that you have the necessary rights to the content and that the content you contribute may be provided under the project license. - -### Suggesting Enhancements - -This section guides you through submitting an enhancement suggestion for Data Contract Template, **including completely new features and minor improvements to existing functionality**. Following these guidelines will help maintainers and the community to understand your suggestion and find related suggestions. - - -#### Before Submitting an Enhancement - -- Make sure that you are using the latest version. -- Read the [documentation](https://github.com/AIDAUserGroup/open-data-contract-standard) carefully and find out if the functionality is already covered. -- Perform a [search](https://github.com/bitol-io/open-data-contract-standard/issues) to see if the enhancement has already been suggested. If it has, add a comment to the existing issue instead of opening a new one. -- Find out whether your idea fits with the scope and aims of the project. It's up to you to make a strong case to convince the project's developers of the merits of this feature. Keep in mind that we want features that will be useful to the majority of our users and not just a small subset. - - -#### How Do I Submit a Good Enhancement Suggestion? - -Enhancement suggestions are tracked as [GitHub issues](https://github.com/bitol-io/open-data-contract-standard/issues). - -- Use a **clear and descriptive title** for the issue to identify the suggestion. -- Provide a **step-by-step description of the suggested enhancement** in as many details as possible. -- **Describe the current behavior** and **explain which behavior you expected to see instead** and why. At this point you can also tell which alternatives do not work for you. -- **Explain why this enhancement would be useful** to most Open Data Contract Standard users. You may also want to point out the other projects that solved it better and which could serve as inspiration. - -### Improving The Documentation -Please contact [@jgperrin](https://github.com/jgperrin). Examples are always welcome. - -## Join The Project Team -Please contact [@jgperrin](https://github.com/jgperrin). diff --git a/README.md b/README.md index 85b78db..7d1fa58 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ Welcome! Thanks for your interest and for taking the time to come here! ❤️ ## Executive summary -This standard describes a structure for a **data contract**. Its current version is v2.2.2. It is available for you as an Apache 2.0 license. Contributions are welcome! +This standard describes a structure for a **data contract**. Its current version is v3.0.0. It is available for you as an Apache 2.0 license. Contributions are welcome! ## Discover the open standard A reader-friendly version of the standard can be found on its [dedicated site](https://bitol-io.github.io/open-data-contract-standard/). @@ -25,14 +25,18 @@ Discover the [Open Data Contract Standard](docs/README.md). This file contains s ### The basics of a data contract A data contract defines the agreement between a data producer and consumers. A data contract contains several sections: -* [Fundamentals](docs/README.md#demographics). -* [Schema](docs/README.md#dataset-and-schema). -* [Data quality](docs/README.md#data-quality-). -* [Service-level agreement (SLA)](docs/README.md#service-level-agreement). -* [Security & stakeholders](docs/README.md#stakeholders). -* [Custom properties](docs/README.md#other-properties). +* [Fundamentals](docs/README.md#fundamentals). +* [Schema](docs/README.md#schema). +* [Data quality](docs/README.md#data-quality). +* [Support & communication channels](docs/README.md#support-and-communication-channels) +* [Pricing](docs/README.md#pricing) +* [Team](docs/README.md#team) +* [Roles](docs/README.md#roles) +* [Service-level agreement (SLA)](docs/README.md#service-level-agreement-sla). +* [Infrastructures & servers](docs/README.md#infrastructure-and-servers) +* [Custom properties](docs/README.md#custom-properties). -![Data contract schema](docs/img/data-contract-v2.2.1-schema.svg "Data contract schema") +![Data contract schema](docs/img/data-contract-diagram-latest.svg "Data contract schema") *Figure 1: illustration of a data contract, its principal contributors, sections, and usage.* @@ -44,30 +48,11 @@ validation of your YAML files. Links below show how you can import the schema: - [IntelliJ](https://www.jetbrains.com/help/idea/json.html#ws_json_schema_add_custom) - [VS Code](https://code.visualstudio.com/docs/languages/json#_json-schemas-and-settings) -## Contributing to the project -Check out the [CONTRIBUTING](./CONTRIBUTING.md) file. - ## Articles and Other Resources +Check out the [resources](resources.md) page. -* 2024-07-17 - [Data Contracts in Action: Testing](https://medium.com/@pflooky/data-contracts-in-action-testing-111631338657) -* 2024-06-12 - [The Future of Data Management: An Enabler of AI Development? A Basic Illustration with RAG, Open Standards, and Data Contracts](https://blog.owulveryck.info/2024/06/12/the-future-of-data-management-an-enabler-of-ai-development-a-basic-illustration-with-rag-open-standards-and-data-contracts.html) -* 2024-05-30 - [ODCS Roadmap](https://medium.com/abeadata/odcs-roadmap-9b9a17367af4) -* 2024-05-25 - [Conceptual model of Data Quality of Service as Code by Jarkko Moilanen](https://aidausergroup.org/2024/05/25/aida-user-group-forecaster-pi-day-highlights-data-quality-whats-new/) -* 2024-02-06 - [Getting started with ODCS](https://medium.com/abeadata/getting-started-with-odcs-3ba790707879) -* 2023-12-08 - [Why the Need for Standardizing Data Contracts?](https://medium.com/abeadata/why-the-need-for-standardizing-data-contracts-133bc3491148) -* 2023-11-30 - [Linux Foundation AI & Data - Bitol Joins LF AI & Data as New Sandbox Project](https://lfaidata.foundation/blog/2023/11/30/bitol-joins-lf-ai-data-as-new-sandbox-project/) -* 2023-11-30 - [AIDAUG - Bitol Joins LF AI & Data as New Sandbox Project](https://aidausergroup.org/2023/11/30/bitol-joins-lf-ai-data-as-new-sandbox-project/) -* 2023-11-22 - [What is, and what isn’t, a data contract](https://datacreation.substack.com/p/what-is-and-what-isnt-a-data-contract) -* 2023-10-01 - [Data Contracts: A Bridge Connecting Two Worlds](https://medium.com/@atanas.iliev.ai/data-contracts-a-bridge-connecting-two-worlds-404eff1d970d) -* 2023-09-10 - [Data Contracts 101](https://medium.com/p/568a9adbf9a9) -* 2023-08-10 - [Welcome to the Open Data Contract Standard](https://jgp.ai/2023/08/09/welcome-to-the-open-data-contract-standard/) -* 2023-05-11 - [Data Contracts – Everything You Need to Know](https://www.montecarlodata.com/blog-data-contracts-explained/) -* 2023-05-07 - [Data Engineering Weekly #130 - Data Contract in the Wild with PayPal’s Data Contract Template](https://www.dataengineeringweekly.com/p/data-engineering-weekly-130) -* 2023-05-06 - [PayPal เปิด Data Contract เป็น Open Source Template ให้ไปใช้งานกัน](https://discuss.dataengineercafe.io/t/paypal-data-contract-open-source-template/581/1) -* 2023-05-05 - [Jonathan Neo (j__neo ) on Reddit](https://www.reddit.com/r/dataengineering/comments/137glbo/comment/jixw5hj/?utm_source=reddit&utm_medium=web2x&context=3) -* 2023-05-01 - [PayPal open sources its data contract template](https://jgp.ai/2023/05/01/paypal-open-sources-its-data-contract-template/) - -If you spot an article about the Open Data Contract Standard, make a pull request! +## Contributing to the project +Check out the [CONTRIBUTING](./CONTRIBUTING.md) page. ## More diff --git a/docs/README.md b/docs/README.md index 4bb77c4..26d0b69 100644 --- a/docs/README.md +++ b/docs/README.md @@ -6,184 +6,159 @@ image: "https://raw.githubusercontent.com/bitol-io/artwork/main/horizontal/color # Open Data Contract Standard -## Executive summary -This document describes the keys and values expected in a YAML data contract, per the **Open Data Contract Standard**. It is divided in multiple sections: [demographics](#demographics), [dataset & schema](#dataset-and-schema), [data quality](#data-quality), [pricing](#pricing), [stakeholders](#stakeholders), [roles](#roles), [service-level agreement](#service-level-agreement), and [other properties](#other-properties). Each section starts with at least an example followed by definition of each field/key. +## Executive Summary +This document describes the keys and values expected in a YAML data contract, per the **Open Data Contract Standard**. +It is divided in multiple sections: [fundamentals (fka demographics)](#fundamentals), [schema](#schema), +[data quality](#data-quality), [Support & communication channels](#support-and-communication-channels), [pricing](#pricing), [team](#team), +[roles](#roles), [service-level agreement](#service-level-agreement-sla), [Infrastructures & servers](#infrastructure-and-servers) and +[other/custom properties](#custom-properties). Each section starts with at least an example followed by definition of +each field/key. + ## Table of content -* [Fundamentals & demographics](#demographics) -* [Datasets & schema](#dataset-and-schema) -* [Data quality](#data-quality) -* [Pricing](#pricing) -* [Stakeholders](#stakeholders) -* [Roles](#roles) -* [Service-level agreement](#service-level-agreement) -* [Other properties](#other-properties) -* [Full example](#full-example) +1. [Fundamentals (fka demographics)](#fundamentals) +1. [Schema](#schema) +1. [Data quality](#data-quality) +1. [Support & communication channels](#support-and-communication-channels) +1. [Pricing](#pricing) +1. [Team](#team) +1. [Roles](#roles) +1. [Service-level agreement](#service-level-agreement-sla) +1. [Infrastructures & servers](#infrastructure-and-servers) +1. [Custom & other properties](#custom-properties) +1. [Examples](#full-example-1) + ## Notes * This contract is containing example values, we reviewed very carefully the consistency of those values, but we cannot guarantee that there are no errors. If you spot one, please raise an [issue](https://github.com/AIDAUserGroup/open-data-contract-standard/issues). * Some fields have `null` value: even if it is equivalent to not having the field in the contract, we wanted to have the field for illustration purpose. -* This contract leverages BigQuery but should be **platform agnostic**. If you think it is not the case, please raise an [issue](https://github.com/AIDAUserGroup/open-data-contract-standard/issues). +* This contract should be **platform agnostic**. If you think it is not the case, please raise an [issue](https://github.com/AIDAUserGroup/open-data-contract-standard/issues). + -## Demographics +## Fundamentals This section contains general information about the contract. ### Example ```YAML -# What's this data contract about? -datasetDomain: seller # Domain -quantumName: my quantum # Data product name -userConsumptionMode: Analytical -version: 1.1.0 # Version (follows semantic versioning) -status: current -uuid: 53581432-6c55-4ba2-a65f-72344a91553a - -# Lots of information +apiVersion: 3.0.0 # Standard version +kind: DataContract + +id: 53581432-6c55-4ba2-a65f-72344a91553a +name: seller_payments_v1 +version: 1.1.0 # Data Contract Version +status: production +domain: seller +dataProduct: payments +tenant: ClimateQuantumInc + description: purpose: Views built on top of the seller tables. limitations: null usage: null -tenant: ClimateQuantumInc -# Getting support -productDl: product-dl@ClimateQuantum.org -productSlackChannel: '#product-help' -productFeedbackUrl: null +tags: null +``` + +### Definitions -# Physical parts / GCP / BigQuery specific -sourcePlatform: googleCloudPlatform -sourceSystem: bigQuery -project: edw # BQ dataset -datasetName: access_views # BQ dataset +| Key | UX label | Required | Description | +|-------------------------|------------------|----------|------------------------------------------------------------------------------------------------| +| apiVersion | Standard version | Yes | Version of the standard used to build data contract. Default value is `v3.0.0`. | +| kind | Kind | Yes | The kind of file this is. Valid value is `DataContract`. | +| id | ID | Yes | A unique identifier used to reduce the risk of dataset name collisions, such as a UUID. | +| name | Name | No | Name of the data contract. | +| version | Version | Yes | Current version of the data contract. | +| status | Status | Yes | Current status of the data contract. Valid values are `production`, `test`, or `development`. | +| tenant | Tenant | No | Indicates the property the data is primarily associated with. Value is case insensitive. | +| domain | Domain | No | Name of the logical data domain. | +| dataProduct | Data Product | No | Name of the data product. | +| description | Description | No | Object containing the descriptions. | +| description.purpose | Purpose | No | Intended purpose for the provided data. | +| description.limitations | Limitations | No | Technical, compliance, and legal limitations for data use. | +| description.usage | Usage | No | Recommended usage of the data. | -kind: DataContract -apiVersion: 2.3.0 # Standard version (follows semantic versioning, previously known as templateVersion) -type: tables +## Schema +This section describes the schema of the data contract. It is the support for data quality, which is detailed in the next section. Schema supports both a business representation of your data and a physical implementation. It allows to tie them together. -# Physical access -driver: null -driverVersion: null -server: null -database: pypl-edw.pp_access_views -username: '${env.username}' -password: '${env.password}' -schedulerAppName: name_coming_from_scheduler # NEW 2.1.0 Required if you want to schedule stuff, comes from DataALM. +In ODCS v3, the schema has evolved from the table and column representation, therefore the schema introduces a new terminology: -# Data Quality -quality: null # See more information below +* **Objects** are a structure of data: a table in a RDBMS system, a document in a NoSQL database, and so on. +* **Properties** are attributes of an object: a column in a table, a field in a payload, and so on. +* **Elements** are either an object or a property. -# Tags -tags: null -``` +Figure 1 illustrates those terms with a basic relational database. -### Definitions + -| Key | UX label | Required | Description | -|-------------------------|--------------------------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| datasetDomain | Dataset domain | No | Name of the logical domain dataset the contract describes. This field is only required for output data contracts. Examples: `imdb_ds_aggregate`, `receiver_profile_out`, `transaction_profile_out`. | -| quantumName | Quantum name | Yes | The name of the data quantum or data product. | -| userConsumptionMode | Consumption mode | No | List of data modes for which the dataset may be used. Expected sample values might be `analytical` or `operational`.
Note: in the future, this will probably be replaced by ports. | -| version | Version | Yes | Current version of the data contract. | -| status | Status | Yes | Current status of the dataset. Valid values are `production`, `test`, or `development`. | -| uuid | Identifier | Yes | A unique identifier used to reduce the risk of dataset name collisions; initially the UUID will be created using a UUID generator tool ([example](https://www.uuidgenerator.net/)). However, we may want to develop a method that accepts a seed value using a combination of fields–such as name, kind and source–to create a repeatable value. | -| description | Description | No | Object. | -| description.purpose | Purpose | No | Purpose of the dataset, table, or column (depending on the level); the key may appear at the dataset, table, or column level. | -| description.limitations | Limitations | No | Limitations of the dataset, table, or column (depending on the level); the key may appear at the dataset, table, or column level. | -| description.usage | Usage | No | Intended usage of the dataset, table, or column (depending on the level); the key may appear at the dataset, table, or column level. | -| tenant | Tenant | No | Indicates the property the data is primarily associated with. Value is case insensitive. | -| productDl | E-mail distribution list | No | The email distribution list (DL) of the persons or team responsible for maintaining the dataset. | -| productSlackChannel | Support Slack channel | No | Slack channel of the team responsible for maintaining the dataset. | -| productFeedbackUrl | Feedback URL | No | The URL for submitting feedback to the team responsible for maintaining the dataset. | -| sourcePlatform | Source platform | No | The platform where the dataset resides. Expected value is GoogleCloudPlatform, IBMCloud, Azure... | -| sourceSystem | Source system | No | The system where the dataset resides. Expected value can be BigQuery. | -| project | Project | No | Associated cloud project name, can be used for billing or administrative purpose. Used to be datasetProject. | -| datasetName | Dataset name | No | May be required in cloud instance like GCP BigQuery dataset name. | -| server | Server | No | The server where the dataset resides. | -| kind | Kind | Yes | The kind of file this is. Valid value is `DataContract`. | -| apiVersion | Standard version | No | Version of the standard used to build data contract. Default value is v2.2.2. | -| type | Type | Yes | Identifies the types of objects in the dataset. For BigQuery or any other database, the expected value would be Tables. | -| driver | Driver | No | The connection driver required to connect to the dataset. | -| driverVersion | Driver version | No | The version of the connection driver to be used to connect to the dataset. | -| username | Username | No | User credentials for connecting to the dataset; how the credentials will be stored/passed is outside of the scope of the contract. | -| password | Password | No | User credentials for connecting to the dataset; how the credentials will be stored/passed is out of the scope of this contract. | -| schedulerAppName | Scheduler App Name | No | Required if you want to schedule stuff. The name of the application used to schedule jobs | - -## Dataset and schema -This section describes the dataset and the schema of the data contract. It is the support for data quality, which is detailed in the next section. +*Figure 1: elements of the schema in ODCS v3.* -### Example +### Examples + +#### Complete schema ```YAML -dataset: - - table: tbl - physicalName: tbl_1 # NEW in v2.1.0, Optional, default value is table name + version separated by underscores, as table_1_2_0 - priorTableName: null # if needed +schema: + - name: tbl + logicalType: object + physicalType: table + physicalName: tbl_1 description: Provides core payment metrics - authoritativeDefinitions: # NEW in v2.2.0, inspired by the column-level authoritative links + authoritativeDefinitions: - url: https://catalog.data.gov/dataset/air-quality type: businessDefinition - url: https://youtu.be/jbY1BKFj9ec type: videoTutorial tags: null - dataGranularity: Aggregation on columns txn_ref_dt, pmt_txn_id - columns: - - column: txn_ref_dt - isPrimaryKey: false # NEW in v2.1.0, Optional, default value is false, indicates whether the column is primary key in the table. - primaryKeyPosition: -1 + dataGranularityDescription: Aggregation on columns txn_ref_dt, pmt_txn_id + properties: + - name: txn_ref_dt businessName: transaction reference date logicalType: date physicalType: date - isNullable: false description: null - partitionStatus: true + partitioned: true partitionKeyPosition: 1 - clusterStatus: false - clusterKeyPosition: -1 - criticalDataElementStatus: false + criticalDataElement: false tags: [] classification: public - transformSourceTables: + transformSourceObjects: - table_name_1 - table_name_2 - table_name_3 transformLogic: sel t1.txn_dt as txn_ref_dt from table_name_1 as t1, table_name_2 as t2, table_name_3 as t3 where t1.txn_dt=date-3 transformDescription: Defines the logic in business terms. - sampleValues: + examples: - 2022-10-03 - 2020-01-28 - - column: rcvr_id - isPrimaryKey: true # NEW in v2.1.0, Optional, default value is false, indicates whether the column is primary key in the table. + - name: rcvr_id + primaryKey: true primaryKeyPosition: 1 businessName: receiver id logicalType: string physicalType: varchar(18) - isNullable: false + required: false description: A description for column rcvr_id. - partitionStatus: false + partitioned: false partitionKeyPosition: -1 - clusterStatus: true - clusterKeyPosition: 1 - criticalDataElementStatus: false + criticalDataElement: false tags: [] classification: restricted - encryptedColumnName: enc_rcvr_id - - column: rcvr_cntry_code - isPrimaryKey: false # NEW in v2.1.0, Optional, default value is false, indicates whether the column is primary key in the table. + encryptedName: enc_rcvr_id + - name: rcvr_cntry_code + primaryKey: false primaryKeyPosition: -1 businessName: receiver country code logicalType: string physicalType: varchar(2) - isNullable: false + required: false description: null - partitionStatus: false + partitioned: false partitionKeyPosition: -1 - clusterStatus: false - clusterKeyPosition: -1 - criticalDataElementStatus: false + criticalDataElement: false tags: [] classification: public authoritativeDefinitions: @@ -193,163 +168,393 @@ dataset: type: transformationImplementation - url: jdbc:postgresql://localhost:5432/adventureworks/tbl_1/rcvr_cntry_code type: implementation - encryptedColumnName: rcvr_cntry_code_encrypted + encryptedName: rcvr_cntry_code_encrypted +``` + +#### Simple Array + +```yaml +schema: + - name: AnObject + logicalType: object + properties: + - name: street_lines + logicalType: array + items: + logicalType: string +``` + +#### Array of Objects + +```yaml +schema: + - name: AnotherObject + logicalType: object + properties: + - name: x + logicalType: array + items: + logicalType: object + properties: + - name: id + logicalType: string + physicalType: VARCHAR(40) + - name: zip + logicalType: string + physicalType: VARCHAR(15) ``` ### Definitions +#### Schema (top level) + | Key | UX label | Required | Description | |--------------------------------------------------------|------------------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| dataset | Dataset | Yes | Array. A list of tables within the dataset to be cataloged. | -| dataset.table | Table | Yes | Name of the table being cataloged; the value should only contain the table name. Do not include the project or dataset name in the value. | -| dataset.table.physicalName | Physical Name | No | Physical name of the table, default value is table name + version separated by underscores, as `table_1_2_0`. | -| dataset.table.priorTableName | Prior Table Name | No | Name of the previous version of the dataset, if applicable. | -| dataset.table.description | Description | No | Description of the dataset. | -| dataset.table.authoritativeDefinitions | Authoritative Definitions | No | List of links to sources that provide more details on the table; examples would be a link to an external definition, a training video, a GitHub repo, Collibra, or another tool. Authoritative definitions follow the same structure in the standard. | -| dataset.table.dataGranularity | Data Granularity | No | Granular level of the data in the table. Example would be `pmt_txn_id`. | -| dataset.table.columns | Columns | Yes | Array. A list of columns in the table. | -| dataset.table.columns.column | Column | Yes | The name of the column. | -| dataset.table.columns.column.isPrimaryKey | Primary Key | No | Boolean value specifying whether the column is primary or not. Default is false. | -| dataset.table.columns.column.primaryKeyPosition | Primary Key Position | No | If column is a primary key, the position of the primary key column. Starts from 1. Example of `account_id, name` being primary key columns, `account_id` has primaryKeyPosition 1 and `name` primaryKeyPosition 2. Default to -1. | -| dataset.table.columns.column.businessName | Business Name | No | The business name of the column. | -| dataset.table.columns.column.logicalType | Logical Type | Yes | The logical column datatype. | -| dataset.table.columns.column.physicalType | Physical Type | Yes | The physical column datatype. | -| dataset.table.columns.column.description | Description | No | Description of the column. | -| dataset.table.columns.column.isNullable | Nullable | No | Indicates if the column may contain Null values; possible values are true and false. Default is false. | -| dataset.table.columns.column.isUnique | Unique | No | Indicates if the column contains unique values; possible values are true and false. Default is false. | -| dataset.table.columns.column.partitionStatus | Partition Status | No | Indicates if the column is partitioned; possible values are true and false. | -| dataset.table.columns.column.partitionKeyPosition | Partition Key Position | No | If column is used for partitioning, the position of the partition column. Starts from 1. Example of `country, year` being partition columns, `country` has partitionKeyPosition 1 and `year` partitionKeyPosition 2. Default to -1. | -| dataset.table.columns.column.clusterStatus | Cluster Status | No | Indicates of the column is clustered; possible values are true and false. | -| dataset.table.columns.column.clusterKeyPosition | Cluster Key Position | No | If column is used for clustering, the position of the cluster column. Starts from 1. Example of `year, date` being cluster columns, `year` has clusterKeyPosition 1 and `date` clusterKeyPosition 2. Default to -1. | -| dataset.table.columns.column.classification | Classification | No | Can be anything, like confidential, restricted, and public to more advanced categorization. Some companies like PayPal, use data classification indicating the class of data in the column; expected values are 1, 2, 3, 4, or 5. | -| dataset.table.columns.column.authoritativeDefinitions | Authoritative Definitions | No | List of links to sources that provide more detail on column logic or values; examples would be URL to a GitHub repo, Collibra, on another tool. | -| dataset.table.columns.column.encryptedColumnName | Encrypted Column Name | No | The column name within the table that contains the encrypted column value. For example, unencrypted column `email_address` might have an encryptedColumnName of `email_address_encrypt`. | -| dataset.table.columns.column.transformSourceTables | Transform Source Tables | No | List of sources used in column transformation. | -| dataset.table.columns.column.transformLogic | Transform Logic | No | Logic used in the column transformation. | -| dataset.table.columns.column.transformDescription | Transform Description | No | Describes the transform logic in very simple terms. | -| dataset.table.columns.column.sampleValues | Sample Values | No | List of sample column values. | -| dataset.table.columns.column.criticalDataElementStatus | Critical Data Element Status | No | True or false indicator; If element is considered a critical data element (CDE) then true else false. | -| dataset.table.columns.column.tags | Tags | No | A list of tags that may be assigned to the dataset, table or column; the tags keyword may appear at any level. | - | - -### Authorative definitions - -Updated in ODCS (Open Data Contract Standard) v2.2.1. - -| Key | UX label | Required | Description | -|------|-------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| type | Definition type | Yes | Type of definition for authority: v2.2.1 adds standard values: `businessDefinition`, `transformationImplementation`, `videoTutorial`, `tutorial`, and `implementation`. | -| url | URL to definition | Yes | URL to the authority. | - -## Data quality +| schema | schema | Yes | Array. A list of elements within the schema to be cataloged. | + +#### Applicable to Elements (either Objects or Properties) + +| Key | UX label | Required | Description | +|--------------------------|------------------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| name | Name | Yes | Name of the element. | +| physicalName | Physical Name | No | Physical name. | +| description | Description | No | Description of the element. | +| businessName | Business Name | No | The business name of the element. | +| authoritativeDefinitions | Authoritative Definitions | No | List of links to sources that provide more details on the table; examples would be a link to an external definition, a training video, a GitHub repo, Collibra, or another tool. See `authoritativeDefinitions` below. | +| quality | Quality | No | List of data quality attributes. | +| tags | Tags | No | A list of tags that may be assigned to the elements (object or property); the tags keyword may appear at any level. | +| customProperties | Custom Properties | No | Custom properties that are not part of the standard. | + +#### Applicable to Objects + +| Key | UX label | Required | Description | +|--------------------------------------------------------|------------------------------|----------|--------------------------------------------------------------------------------------| +| dataGranularityDescription | Data Granularity | No | Granular level of the data in the object. Example would be "Aggregation by country." | + +#### Applicable to Properties + +Some keys are more applicable when the described property is a column. + +| Key | UX label | Required | Description | +|--------------------------|------------------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| primaryKey | Primary Key | No | Boolean value specifying whether the field is primary or not. Default is false. | +| primaryKeyPosition | Primary Key Position | No | If field is a primary key, the position of the primary key element. Starts from 1. Example of `account_id, name` being primary key columns, `account_id` has primaryKeyPosition 1 and `name` primaryKeyPosition 2. Default to -1. | +| logicalType | Logical Type | Yes | The logical field datatype. One of `string`, `date`, `number`, `integer`, `object`, `array` or `boolean`. | +| logicalTypeOptions | Logical Type Options | No | Additional optional metadata to describe the logical type. See [here](#logical-type-options) for more details about supported options for each `logicalType`. | +| physicalType | Physical Type | Yes | The physical element data type in the data source. For example, VARCHAR(2), DOUBLE, INT. | +| description | Description | No | Description of the element. | +| required | Required | No | Indicates if the element may contain Null values; possible values are true and false. Default is false. | +| unique | Unique | No | Indicates if the element contains unique values; possible values are true and false. Default is false. | +| partitioned | Partitioned | No | Indicates if the element is partitioned; possible values are true and false. | +| partitionKeyPosition | Partition Key Position | No | If element is used for partitioning, the position of the partition element. Starts from 1. Example of `country, year` being partition columns, `country` has partitionKeyPosition 1 and `year` partitionKeyPosition 2. Default to -1. | +| classification | Classification | No | Can be anything, like confidential, restricted, and public to more advanced categorization. | +| authoritativeDefinitions | Authoritative Definitions | No | List of links to sources that provide more detail on element logic or values; examples would be URL to a git repo, documentation, a data catalog or another tool. | +| encryptedName | Encrypted Name | No | The element name within the dataset that contains the encrypted element value. For example, unencrypted element `email_address` might have an encryptedName of `email_address_encrypt`. | +| transformSourceObjects | Transform Sources | No | List of objects in the data source used in the transformation. | +| transformLogic | Transform Logic | No | Logic used in the column transformation. | +| transformDescription | Transform Description | No | Describes the transform logic in very simple terms. | +| examples | Example Values | No | List of sample element values. | +| criticalDataElement | Critical Data Element Status | No | True or false indicator; If element is considered a critical data element (CDE) then true else false. | +| items | Items | No | List of items in an array (only applicable when `logicalType: array`) | + +### Logical Type Options + +Additional metadata options to more accurately define the data type. + +| Data Type | Key | UX Label | Required | Description | +|----------------|------------------|--------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| array | maxItems | Maximum Items | No | Maximum number of items. | +| array | minItems | Minimum Items | No | Minimum number of items. | +| array | uniqueItems | Unique Items | No | If set to true, all items in the array are unique. | +| date | format | Format | No | Format of the date. Follows the format as prescribed by [JDK DateTimeFormatter](https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html). For example, format 'yyyy-MM-dd'. | +| date | exclusiveMaximum | Exclusive Maximum | No | If set to true, all values are strictly less than the maximum value (values < maximum). Otherwise, less than or equal to the maximum value (values <= maximum). | +| date | exclusiveMinimum | Exclusive Minimum | No | If set to true, all values are strictly greater than the minimum value (values > minimum). Otherwise, greater than or equal to the minimum value (values >= minimum). | +| date | maximum | Maximum | No | All date values are less than or equal to this value (values <= maximum). | +| date | minimum | Minimum | No | All date values are greater than or equal to this value (values >= minimum). | +| integer/number | exclusiveMaximum | Exclusive Maximum | No | If set to true, all values are strictly less than the maximum value (values < maximum). Otherwise, less than or equal to the maximum value (values <= maximum). | +| integer/number | exclusiveMinimum | Exclusive Minimum | No | If set to true, all values are strictly greater than the minimum value (values > minimum). Otherwise, greater than or equal to the minimum value (values >= minimum). | +| integer/number | format | Format | No | Format of the value in terms of how many bits of space it can use and whether it is signed or unsigned (follows the Rust integer types). | +| integer/number | maximum | Maximum | No | All values are less than or equal to this value (values <= maximum). | +| integer/number | minimum | Minimum | No | All values are greater than or equal to this value (values >= minimum). | +| integer/number | multipleOf | Multiple Of | No | Values must be multiples of this number. For example, multiple of 5 has valid values 0, 5, 10, -5. | +| object | maxProperties | Maximum Properties | No | Maximum number of properties. | +| object | minProperties | Minimum Properties | No | Minimum number of properties. | +| object | required | Required | No | Property names that are required to exist in the object. | +| string | format | Format | No | Provides extra context about what format the string follows. For example, password, byte, binary, email, uuid, uri, hostname, ipv4, ipv6. | +| string | maxLength | Maximum Length | No | Maximum length of the string. | +| string | minLength | Minimum Length | No | Minimum length of the string. | +| string | pattern | Pattern | No | Regular expression pattern to define valid value. Follows regular expression syntax from ECMA-262 (https://262.ecma-international.org/5.1/#sec-15.10.1). | + +### Authoritative definitions + +Reference to an external definition on element logic or values. + +| Key | UX label | Required | Description | +|------|-------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------| +| type | Definition type | Yes | Type of definition for authority. Valid values are: `businessDefinition`, `transformationImplementation`, `videoTutorial`, `tutorial`, and `implementation`. | +| url | URL to definition | Yes | URL to the authority. | + + +## Data quality This section describes data quality rules & parameters. They are tightly linked to the schema described in the previous section. -### Example of data quality at the dataset level +Data quality rules support different levels/stages of data quality attributes: -Note: -* This example relies on a data quality tool called Elevate. It should be easily transformed to any other tool. If you have questions, please raise an [issue](https://github.com/AIDAUserGroup/open-data-contract-standard/issues). -* The Open Data Contract Standard has a provision for supporting multiple data quality tools. + - __Text__: A human-readable text that describes the quality of the data. + - __Library__ rules: A maintained library of commonly-used predefined quality attributes such as `rowCount`, `unique`, `freshness`, and more. + - __SQL__: An individual SQL query that returns a value that can be compared. Can be extended to `Python` or other. + - __Custom__: Quality attributes that are vendor-specific, such as Soda, Great Expectations, dbt tests, or Montecarlo monitors. -```YAML -dataset: - - table: tab1 -# ... - quality: - - code: countCheck # Required, name of the rule - templateName: CountCheck # NEW in v2.1.0 Required - description: Ensure row count is within expected volume range # Optional - toolName: Elevate # Required - toolRuleName: DQ.rw.tab1.CountCheck # NEW in v2.1.0 Optional (Available only to the users who can change in source code edition) - dimension: completeness # Optional - type: reconciliation # Optional NEW in v2.1.0 default value for column level check - dataQuality and for table level reconciliation - severity: error # Optional NEW in v2.1.0, default value is error - businessImpact: operational # Optional NEW in v2.1.0 - scheduleCronExpression: 0 20 * * * # Optional NEW in v2.1.0 default schedule - every day 10 a.m. UTC - - code: distinctCheck - description: enforce distinct values - toolName: Elevate - templateName: DistinctCheck - dimension: completeness - toolRuleName: DQ.rw.tab1.DistinctCheck - columns: - - txn_ref_dt - - rcvr_id - column: null - type: reconciliation - severity: error - businessImpact: operational - scheduleCronExpression: 0 20 * * * - customProperties: - - property: FIELD_NAME - value: rcvr_id - - property: FILTER_CONDITIONS - value: 1>0 - - code: piiCheck - description: null - toolName: Elevate - templateName: PIICheck - toolRuleName: DQ.rw.tab1_2_0_0.piiCheck - type: dataQuality - severity: error - businessImpact: operational - scheduleCronExpression: 0 20 * * * - customProperties: - - property: FIELD_NAME - value: - - property: FILTER_CONDITIONS - value: +### Text +A human-readable text that describes the quality of the data. Later in the development process, these might be translated into an executable check (such as `sql`), a library rule, or checked through an AI engine. + +```yaml +quality: + - type: text + description: The email address was verified by the system. ``` -### Example of data quality at the column level +### Library +ODCS will provide a set of predefined rules commonly used in data quality checks, designed to be compatible with all major data quality engines. This simplifies the work for data engineers by eliminating the need to manually write SQL queries. -```YAML -dataset: - - table: tab1 - - column: rcvr_id - isPrimaryKey: true # NEW in v2.1.0, Optional, default value is false, indicates whether the column is primary key in the table. - businessName: receiver id -# ... - quality: - - code: nullCheck - templateName: NullCheck - description: column should not contain null values - toolName: Elevate - toolRuleName: DQ.rw.tab1_2_0_0.rcvr_cntry_code.NullCheck - dimension: completeness # dropdown 7 values - type: dataQuality - severity: error - businessImpact: operational - scheduleCronExpression: 0 20 * * * - customProperties: - - property: FIELD_NAME - value: - - property: COMPARE_TO - value: - - property: COMPARISON_TYPE - value: Greater than +#### Property-level +Those examples apply at the property level, such as column, field, etc. + +##### Duplicate count on rows +No more than 10 duplicate names. + +```yaml +quality: +- type: library # optional and default value for data quality rules + rule: duplicateCount + mustBeLessThan: 10 + name: Fewer than 10 duplicate names + unit: rows +``` + +##### Duplicate count on % +Duplicates should be less than 1%. + +```yaml +quality: +- rule: duplicateCount + mustBeLessThan: 1 + unit: percent +``` + +##### Valid values +Valid values from a static list. + +```yaml +quality: +- rule: validValues + validValues: ['pounds'] +``` + +#### Object-level +This example applies at the object level (like a table or a view). + +##### Row count +The number of rows must be between 100 and 120. + +```yaml +quality: + - rule: rowCount + mustBeBetween: [100, 120] + name: Verify row count range +``` + +### SQL +A single SQL query that returns either a numeric or boolean value for comparison. The query must be written in the SQL dialect specific to the provided server. `${object}` and `${property}` are automatically replaced by the current object (in the case of SQL on a relational database, the table or view name) and the current property name (in the case of SQL on a relational database, the column). + +```yaml +quality: + - type: sql + query: | + SELECT COUNT(*) FROM ${object} WHERE ${property} IS NOT NULL + mustBeLessThan: 3600 +``` + +### Custom +Custom rules allow for vendor-specific checks, including tools like Soda, Great Expectations, dbt-tests, Montecarlo, and others. Any format for properties is acceptable, whether it's written in YAML, JSON, XML, or even uuencoded binary. They are an intermediate step before the vendor accepts ODCS natively. + +#### Soda Example + +```yaml +quality: +- type: custom + engine: soda + implementation: | + type: duplicate_percent # Block + columns: # passed as-is + - carrier # to the tool + - shipment_numer # (Soda in this situation) + must_be_less_than: 1.0 # +``` + +#### Great Expectation Example + +```yaml +quality: +- type: custom + engine: greatExpectations + implementation: | + type: expect_table_row_count_to_be_between # Block + kwargs: # passed as-is + minValue: 10000 # to the tool + maxValue: 50000 # (Great Expectations in this situation) +``` + +### Scheduling +The data contract can contain scheduling information for executing the rules. You can use `schedule` and `scheduler` for those operation. In previous versions of ODCS, the only allowed scheduler was cron and its syntax was `scheduleCronExpression`. + +```yaml +quality: + - type: sql + query: | + SELECT COUNT(*) FROM ${object} WHERE ${property} IS NOT NULL + mustBeLessThan: 3600 + scheduler: cron + schedule: 0 20 * * * +``` + + +### Definitions + +Acronyms: +* DQ: data quality. + +| Key | UX label | Required | Description | +|----------------------------------|----------------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| quality | Quality | No | Quality tag with all the relevant information for rule setup and execution. | +| quality.name | Name | No | A short name for the rule. | +| quality.description | Description | No | Describe the quality check to be completed. | +| quality.type | Type | No | Type of DQ rule. Valid values are `library` (default), `text`, `sql`, and `custom`. | +| quality.rule | Rule name | No | Required for `library` DQ rules: the name of the rule to be executed. | +| quality.\ | See below | No | Multiple values are allowed for the **property**, the value is the one to compare to. | +| quality.unit | Unit | No | Unit the rule is using, popular values are `rows` or `percent`, but any value is allowed. | +| quality.validValues | Valid values | No | Static list of valid values. | +| quality.query | SQL Query | No | Required for `sql` DQ rules: the SQL query to be executed. Note that it should match the target SQL engine/database, no transalation service are provided here. | +| quality.engine | Third-party DQ Engine | No | Required for `custom` DQ rule: name of the third-party engine being used. Any value is authorized here but common values are `soda`, `greatExpectations`, `montecarlo`, etc. | +| quality.implementation | Third-party Implementation | No | A text (non-parsed) block of code required for the third-party DQ engine to run. | +| quality.dimension | Dimension | No | The key performance indicator (KPI) or dimension for data quality. Valid values are listed after the table. | +| quality.method | Method | No | Values are open and include `reconciliation`. | +| quality.severity | Severity | No | The severity of the DQ rule. | +| quality.businessImpact | Business Impact | No | Consequences of the rule failure. | +| quality.customProperties | Custom Properties | No | Additional properties required for rulee execution. Follows the same structure as any custom properties block. | +| quality.tags | Tags | No | Tags. Follows the same structure as any tags property. | +| quality.authoritativeDefinitions | Authoritative Definitions | No | Authoritative definitions indicate the link to external definition. Follows the same structure as any authoritative definitions block. | +| quality.scheduler | Scheduler | No | Name of the scheduler, can be `cron` or any tool your organization support. | +| quality.schedule | Scheduler Configuration | No | Configuration information for the scheduling tool, for `cron` a possible value is `0 20 * * *`. | + +#### Valid Values for Dimension +Those data quality dimensions are used for classification and reporting in data quality. Valid values are: + + * `accuracy` (synonym `ac`), + * `completeness` (synonym `cp`), + * `conformity` (synonym `cf`), + * `consistency` (synonym `cs`), + * `coverage` (synonym `cv`), + * `timeliness` (synonym `tm`), + * `uniqueness` (synonym `uq`). + +#### Valid Properties for Operator +The operator specifies the condition to validate the rule. + +| Operator | Expected Value | Math Symbol | Example | +|--------------------------|---------------------|-------------|------------------------------| +| `mustBe` | number | `=` | `mustBe: 5` | +| `mustNotBe` | number | `<>`, `≠` | `mustNotBe: 3.14` | +| `mustBeGreaterThan` | number | `>` | `mustBeGreaterThan: 59` | +| `mustBeGreaterOrEqualTo` | number | `>=`, `≥` | `mustBeGreaterOrEqualTo: 60` | +| `mustBeLessThan` | number | `<` | `mustBeLessThan: 1000` | +| `mustBeLessOrEqualTo` | number | `<=`, `≤` | `mustBeLessOrEqualTo: 999` | +| `mustBeBetween` | list of two numbers | `⊂` | `mustBeBetween: [0, 100]` | +| `mustNotBeBetween` | list of two numbers | `⊄` | `mustNotBeBetween: [0, 100]` | + +`mustBeBetween` is the equivalent to `mustBeGreaterThan` and `mustBeLessThan`. + +```yaml +quality: + - type: sql + query: | + SELECT COUNT(*) FROM ${table} WHERE ${column} IS NOT NULL + mustBeBetween: [0, 100] +``` + +is equivalent to: + +```yaml +quality: + - type: sql + query: | + SELECT COUNT(*) FROM ${table} WHERE ${column} IS NOT NULL + mustBeGreaterThan: 0 + mustBeLessThan: 100 +``` + + +#### Library Rules +Bitol has the ambition of creating a library of common data quality rules. Join the working group around [RFC #0012](https://github.com/bitol-io/tsc/blob/main/rfcs/0012-implicit-dq-rules.md). + + +## Support and Communication Channels +Support and communication channels help consumers find help regarding their use of the data contract. + +### Examples + +#### Minimal example + +```yaml +support: + - channel: channel-name-or-identifier # Simple Slack communication channel + url: https://aidaug.slack.com/archives/C05UZRSBKLY + - channel: channel-name-or-identifier # Simple distribution list + url: mailto:datacontract-ann@bitol.io +``` + +#### Full example + +```yaml +support: + - channel: channel-name-or-identifier + tool: teams + scope: interactive + url: https://bitol.io/teams/channel/my-data-contract-interactive + - channel: channel-name-or-identifier + tool: teams + scope: announcements + url: https://bitol.io/teams/channel/my-data-contract-announcements + invitationUrl: https://bitol.io/teams/channel/my-data-contract-announcements-invit + - channel: channel-name-or-identifier-for-all-announcement + description: All announcement for all data contracts + tool: teams + scope: announcements + url: https://bitol.io/teams/channel/all-announcements + - channel: channel-name-or-identifier + tool: email + scope: announcements + url: mailto:datacontract-ann@bitol.io + - channel: channel-name-or-identifier + tool: ticket + url: https://bitol.io/ticket/my-product ``` ### Definitions -| Key | UX label | Required | Description | -|--------------------------------|---------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| quality | Quality | No | Quality tag with all the relevant information for rule setup and execution. | -| quality.code | Quality Code | No | The Rosewall data quality code(s) indicating which quality checks need to be performed at the dataset, table, or column level. The quality keyword may appear at any level. Some quality checks require parameters, so the check can be completed (e.g., a list of fields used to identify a distinct row). Therefore, some quality checks may be followed by a single value or an array. See the appendix for a link to quality checks. | -| quality.templateName | Template Name | Yes | The template name that indicates what is the equivalent template from the tool. | -| quality.description | Description | No | Describe the quality check to be completed. | -| quality.toolName | Tool Name | Yes | Name of the tool used to complete the quality check; Most will be Elevate initially. | -| quality.toolRuleName | Tool Rule Name | No | Name of the quality tool's rule created to complete the quality check. | -| quality.dimension | Dimension | No | The key performance indicator (KPI) or dimension for data quality. | -| quality.columns | Columns | No | List of columns to be used in the quality check. | -| quality.column | Column | No | To be used in lieu of quality.columns when only a single column is required for the quality check. | -| quality.type | Type | No | The type of quality check. | -| quality.severity | Severity | No | The severity of the quality rule. | -| quality.businessImpact | Business Impact | No | Consequences of the rule failure. | -| quality.scheduleCronExpression | Schedule Expression | No | Rule execution schedule details. | -| quality.customProperties | Custom Properties | No | Additional properties required for rule execution. | +| Key | UX label | Required | Description | +|-----------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------------| +| support | Support | No | Top level for support channels. | +| support.channel | Channel | Yes | Channel name or identifier. | +| support.url | Channel URL | Yes | Access URL using normal [URL scheme](https://en.wikipedia.org/wiki/URL#Syntax) (https, mailto, etc.). | +| support.description | Description | No | Description of the channel, free text. | +| support.tool | Tool | No | Name of the tool, value can be `email`, `slack`, `teams`, `discord`, `ticket`, or `other`. | +| support.scope | Scope | No | Scope can be: `interactive`, `announcements`, `issues`. | +| support.invitationUrl | Invitation URL | No | Some tools uses invitation URL for requesting or subscribing. Follows the [URL scheme](https://en.wikipedia.org/wiki/URL#Syntax). | + ## Pricing -This section covers pricing when you bill your customer for using this data product. Pricing is experimental in v2.1.1 of the data contract. +This section covers pricing when you bill your customer for using this data product. ### Example @@ -370,12 +575,12 @@ price: | price.priceUnit | Price Unit | No | The unit of measure for calculating cost. Examples megabyte, gigabyte. | -## Stakeholders -This section lists stakeholders and the history of their relation with this data contract. +## Team +This section lists team members and the history of their relation with this data contract. In v2.x, this section was called stakeholders. ### Example ```YAML -stakeholders: +team: - username: ceastwood role: Data Scientist dateIn: 2022-08-02 @@ -387,21 +592,21 @@ stakeholders: - username: daustin role: Owner comment: Keeper of the grail + name: David Austin dateIn: 2022-10-01 ``` ### Definitions -The UX label is the label used in the UI and other user experiences. It is not limited to BlueRacket. - -| Key | UX label | Required | Description | -|---------------------------------|----------------------------|----------|---------------------------------------------------------------------------------------------------| -| stakeholders | Stakeholders | No | Array | -| stakeholders.username | Username | No | The stakeholder's username or email. | -| stakeholders.role | Role | No | The stakeholder's job role; Examples might be owner, data steward. There is no limit on the role. | -| stakeholders.dateIn | Date In | No | The date when the user became a stakeholder. | -| stakeholders.dateOut | Date Out | No | The date when the user ceased to be a stakeholder | -| stakeholders.replacedByUsername | Replaced By Username | No | The username of the user who replaced the stakeholder | +The UX label is the label used in the UI and other user experiences. +| Key | UX label | Required | Description | +|-------------------------|----------------------|----------|--------------------------------------------------------------------------------------------| +| team | Team | No | Object | +| team.username | Username | No | The user's username or email. | +| team.role | Role | No | The user's job role; Examples might be owner, data steward. There is no limit on the role. | +| team.dateIn | Date In | No | The date when the user joined the team. | +| team.dateOut | Date Out | No | The date when the user ceased to be part of the team. | +| team.replacedByUsername | Replaced By Username | No | The username of the user who replaced the previous user. | ## Roles This section lists the roles that a consumer may need to access the dataset depending on the type of access they require. @@ -430,30 +635,32 @@ roles: ### Definitions -| Key | UX label | Required | Description | -|----------------------------|---------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------| -| roles | Roles | Yes | Array. A list of roles that will provide user access to the dataset. | -| roles.role | Role | Yes | Name of the IAM role that provides access to the dataset; the value will generally come directly from the "BQ dataset to IAM roles mapping" document. | -| roles.access | Access | Yes | The type of access provided by the IAM role; the value will generally come directly from the "BQ dataset to IAM roles mapping" document. | -| roles.firstLevelApprovers | 1st Level Approvers | No | The name(s) of the first-level approver(s) of the role; the value will generally come directly from the "BQ dataset to IAM roles mapping" document. | -| roles.secondLevelApprovers | 2nd Level Approvers | No | The name(s) of the second-level approver(s) of the role; the value will generally come directly from the "BQ dataset to IAM roles mapping" document. | +| Key | UX label | Required | Description | +|----------------------------|---------------------|----------|----------------------------------------------------------------------| +| roles | Roles | No | Array. A list of roles that will provide user access to the dataset. | +| roles.role | Role | Yes | Name of the IAM role that provides access to the dataset. | +| roles.description | Description | No | Description of the IAM role and its permissions. | +| roles.access | Access | No | The type of access provided by the IAM role. | +| roles.firstLevelApprovers | 1st Level Approvers | No | The name(s) of the first-level approver(s) of the role. | +| roles.secondLevelApprovers | 2nd Level Approvers | No | The name(s) of the second-level approver(s) of the role. | + -## Service-level agreement -This section describes the service-level agreements (SLA). SLA have been extended in version v2.1.0. +## Service-Level Agreement (SLA) +This section describes the service-level agreements (SLA). -* Use the `Table.Column` to indicate the number to do the checks on, as in `SELECT txn_ref_dt FROM tab1`. -* Separate multiple table.columns by a comma, as in `table1.col1`, `table2.col1`, `table1.col2`. -* If there is only one table in the contract, the table name is not required. +* Use the `Object.Element` to indicate the number to do the checks on, as in `SELECT txn_ref_dt FROM tab1`. +* Separate multiple object.element by a comma, as in `table1.col1`, `table2.col1`, `table1.col2`. +* If there is only one object in the contract, the object name is not required. ### Example ```YAML -slaDefaultColumn: tab1.txn_ref_dt # Optional, default value is partitionColumn. +slaDefaultElement: tab1.txn_ref_dt # Optional, default value is partitionColumn. slaProperties: - property: latency # Property, see list of values in DP QoS value: 4 unit: d # d, day, days for days; y, yr, years for years - column: tab1.txn_ref_dt # This would not be needed as it is the same table.column as the default one + element: tab1.txn_ref_dt # This would not be needed as it is the same table.column as the default one - property: generalAvailability value: 2022-05-12T09:30:10-08:00 - property: endOfSupport @@ -463,37 +670,389 @@ slaProperties: - property: retention value: 3 unit: y - column: tab1.txn_ref_dt + element: tab1.txn_ref_dt - property: frequency value: 1 valueExt: 1 unit: d - column: tab1.txn_ref_dt + element: tab1.txn_ref_dt - property: timeOfAvailability value: 09:00-08:00 - column: tab1.txn_ref_dt + element: tab1.txn_ref_dt driver: regulatory # Describes the importance of the SLA: [regulatory|analytics|operational|...] - property: timeOfAvailability value: 08:00-08:00 - column: tab1.txn_ref_dt + element: tab1.txn_ref_dt driver: analytics ``` ### Definitions -| Key | UX label | Required | Description | -|------------------------|-----------------------|--------------------------------|----------------------------------------------------------------------------------------------------------------------------| -| slaDefaultColumn | Default SLA column(s) | No | Columns (using the Table.Column notation) to do the checks on. By default, it is the partition column. | -| slaProperties | SLA | No | A list of key/value pairs for SLA specific properties. There is no limit on the type of properties (more details to come). | -| slaProperties.property | Property | Yes | Specific property in SLA, check the periodic table. May requires units (more details to come). | -| slaProperties.value | Value | Yes | Agreement value. The label will change based on the property itself. | -| slaProperties.valueExt | Extended value | No - unless needed by property | Extended agreement value. The label will change based on the property itself. | -| slaProperties.unit | Unit | No - unless needed by property | **d**, day, days for days; **y**, yr, years for years, etc. Units use the ISO standard. | -| slaProperties.column | Column(s) | No | Column(s) to check on. Multiple columns should be extremely rare and, if so, separated by commas. | -| slaProperties.driver | Driver | No | Describes the importance of the SLA from the list of: `regulatory`, `analytics`, or `operational`. | - -## Other properties -This section covers other properties you may find in a data contract. +| Key | UX label | Required | Description | +|------------------------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------| +| slaDefaultElement | Default SLA element(s) | No | Element (using the element path notation) to do the checks on. | +| slaProperties | SLA | No | A list of key/value pairs for SLA specific properties. There is no limit on the type of properties. | +| slaProperties.property | Property | Yes | Specific property in SLA, check the Data QoS periodic table. May requires units. | +| slaProperties.value | Value | Yes | Agreement value. The label will change based on the property itself. | +| slaProperties.valueExt | Extended value | No - unless needed by property | Extended agreement value. The label will change based on the property itself. | +| slaProperties.unit | Unit | No - unless needed by property | **d**, day, days for days; **y**, yr, years for years, etc. Units use the ISO standard. | +| slaProperties.element | Element(s) | No | Element(s) to check on. Multiple elements should be extremely rare and, if so, separated by commas. | +| slaProperties.driver | Driver | No | Describes the importance of the SLA from the list of: `regulatory`, `analytics`, or `operational`. | + +## Infrastructure and Servers + +The `servers` element describes where the data protected by this data contract is *physically* located. That metadata helps to know where the data is so that a data consumer can discover the data and a platform engineer can automate access. + +An entry in `servers` describes a single dataset on a specific environment and a specific technology. The `servers` element can contain multiple servers, each with its own configuration. + +The typical ways of using the top level `servers` element are as follows: +- **Single Server:** The data contract protects a specific dataset at a specific location. *Example:* a CSV file on an SFTP server. +- **Multiple Environments:** The data contract makes sure that the data is protected in all environments. *Example:* a data product with data in a dev(elopment), UAT, and prod(uction) environment on Databricks. +- **Different Technologies:** The data contract makes sure that regardless of the offered technology, it still holds. *Example:* a data product offers its data in a Kafka topic and in a BigQuery table that should have the same structure and content. +- **Different Technologies and Multiple Environments:** The data contract makes sure that regardless of the offered technology and environment, it still holds. *Example:* a data product offers its data in a Kafka topic and in a BigQuery table that should have the same structure and content in dev(elopment), UAT, and prod(uction). + +### General Server Structure + +Each server in the schema has the following structure: + +```yaml +servers: + - type: + description: + environment: + # according to the server type + roles: + - + customProperties: + - +``` + +#### Common Server Properties + +- **type**: The type of server. Valid values include various server technologies like `athena`, `bigquery`, `postgresql`, etc. +- **description**: A description of the server. +- **environment**: The environment where the server operates (e.g., `prod`, `dev`, `uat`). There are no set values. +- **roles**: An optional array of roles that have access to the server. +- **customProperties**: Any additional custom properties specific to the server. + +### Specific Server Properties + +Each server type can be customized with different properties such as `host`, `port`, `database`, and `schema`, depending on the server technology in use. Refer to the specific documentation for each server type for additional configurations. + +### Specific Server Properties & Types + +If your server is not in the list, please use [custom](#custom-server) and suggest it as an improvement. Possible values for `type` are: + +- [api](#api-server) +- [athena](#amazon-athena-server) +- [azure](#azure-server) +- [bigquery](#google-bigquery) +- [clickhouse](#clickhouse-server) +- [databricks](#databricks-server) +- [db2](#ibm-db2-server) +- [denodo](#denodo-server) +- [dremio](#dremio-server) +- [duckdb](#duckdb-server) +- [glue](#amazon-glue) +- [cloudsql](#google-cloud-sql) +- [informix](#ibm-informix-and-hcl-informix) +- [kafka](#kafka-server) +- [kinesis](#amazon-kinesis) +- [local](#local-files) +- [mysql](#mysql-server) +- [oracle](#oracle) +- [postgresql](#postgresql) +- [presto](#presto-server) +- [pubsub](#google-pubsub) +- [redshift](#amazon-redshift-server) +- [s3](#amazon-s3-server-and-compatible-servers) +- [sftp](#sftp-server) +- [snowflake](#snowflake) +- [sqlserver](#microsoft-sql-server) +- [synapse](#synapse-server) +- [trino](#trino-server) +- [vertica](#vertica-server) + +#### API Server + +| Key | UX Label | Required | Description | +|----------------|------------|------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| **location** | Location | Yes | URL to the API | + + +#### Amazon Athena Server +[Amazon Athena](https://docs.aws.amazon.com/athena/latest/ug/what-is.html) is an interactive query service that makes it easy to analyze data directly in Amazon Simple Storage Service (Amazon S3) using standard SQL. With a few actions in the AWS Management Console, you can point Athena at your data stored in Amazon S3 and begin using standard SQL to run ad-hoc queries and get results in seconds. + +| Key | UX Label | Required | Description | +|------------|-------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| schema | Schema | Yes | Identify the schema in the data source in which your tables exist. | +| stagingDir | Staging Directory | No | Amazon Athena automatically stores query results and metadata information for each query that runs in a query result location that you can specify in Amazon S3. | +| catalog | Catalog | No | Identify the name of the Data Source, also referred to as a Catalog. | +| regionName | Region Name | No | The region your AWS account uses. | + +#### Azure Server + +| Key | UX Label | Required | Description | +|-----------|-----------|----------|-----------------------------------------------------------------------------------------------| +| location | Location | Yes | Fully qualified path to Azure Blob Storage or Azure Data Lake Storage (ADLS), supports globs. | +| format | Format | Yes | File format. | +| delimiter | Delimiter | No | Only for format = json. How multiple json documents are delimited within one file | + +#### Google BigQuery +[BigQuery](https://cloud.google.com/bigquery) is a fully managed, AI-ready data analytics platform that helps you maximize value from your data and is designed to be multi-engine, multi-format, and multi-cloud. + +| Key | UX Label | Required | Description | +|---------|----------|----------|-----------------------------------------------| +| project | Project | Yes | The Google Cloud Platform (GCP) project name. | +| dataset | Dataset | Yes | The GCP dataset name. | + +#### ClickHouse Server +[ClickHouse](https://clickhouse.com/) is an open-source column-oriented database management system that allows generating analytical data reports in real-time. + +| Key | UX Label | Required | Description | +|----------|----------|----------|------------------------------------| +| host | Host | Yes | The host of the ClickHouse server. | +| port | Port | Yes | The port to the ClickHouse server. | +| database | Database | Yes | The name of the database. | + +#### Google Cloud SQL +[Google Cloud SQL](https://cloud.google.com/sql) is a fully managed, cost-effective relational database service for PostgreSQL, MySQL, and SQL Server. + +| Key | UX Label | Required | Description | +|----------|----------|----------|------------------------------------------| +| host | Host | Yes | The host of the Google Cloud SQL server. | +| port | Port | Yes | The port of the Google Cloud SQL server. | +| database | Database | Yes | The name of the database. | +| schema | Schema | Yes | The name of the schema. | + +#### Databricks Server + +| Key | UX Label | Required | Description | +|---------|----------|----------|---------------------------------------| +| catalog | Catalog | Yes | The name of the Hive or Unity catalog | +| schema | Schema | Yes | The schema name in the catalog | +| host | Host | No | The Databricks host | + +#### IBM Db2 Server + +| Key | UX Label | Required | Description | +|----------|----------|----------|---------------------------------| +| host | Host | Yes | The host of the IBM DB2 server. | +| port | Port | Yes | The port of the IBM DB2 server. | +| database | Database | Yes | The name of the database. | +| schema | Schema | No | The name of the schema. | + +#### Denodo Server + +| Key | UX Label | Required | Description | +|----------|----------|----------|--------------------------------| +| host | Host | Yes | The host of the Denodo server. | +| port | Port | Yes | The port of the Denodo server. | +| database | Database | No | The name of the database. | + +#### Dremio Server + +| Key | UX Label | Required | Description | +|--------|----------|----------|--------------------------------| +| host | Host | Yes | The host of the Dremio server. | +| port | Port | Yes | The port of the Dremio server. | +| schema | Schema | No | The name of the schema. | + +#### DuckDB Server +[DuckDB](https://duckdb.org/) supports a feature-rich SQL dialect complemented with deep integrations into client APIs. + +| Key | UX Label | Required | Description | +|----------|----------|----------|-------------------------------| +| database | Database | Yes | Path to duckdb database file. | +| schema | Schema | No | The name of the schema. | + +#### Amazon Glue + +| Key | UX Label | Required | Description | +|----------|----------|----------|------------------------------------------------| +| account | Account | Yes | The AWS Glue account | +| database | Database | Yes | The AWS Glue database name | +| location | Location | No | The AWS S3 path. Must be in the form of a URL. | +| format | Format | No | The format of the files | + +#### IBM Informix and HCL Informix +[IBM Informix](https://www.ibm.com/products/informix) is a high performance, always-on, highly scalable and easily embeddable enterprise-class database optimized for the most demanding transactional and analytics workloads. As an object-relational engine, IBM Informix seamlessly integrates the best of relational and object-oriented capabilities enabling the flexible modeling of complex data structures and relationships. + +| Key | UX Label | Required | Description | +|--------------|-----------------|------------|------------------------------------------------------------| +| host | Host | Yes | The host to the Informix server. | +| port | Port | No | The port to the Informix server. Defaults to 9088. | +| database | Database | Yes | The name of the database. | + +#### Kafka Server + +| Key | UX Label | Required | Description | +|--------|----------|----------|--------------------------------------------| +| host | Host | Yes | The bootstrap server of the kafka cluster. | +| format | Format | No | The format of the messages. | + +#### Amazon Kinesis + +| Key | UX Label | Required | Description | +|--------|----------|----------|--------------------------------------| +| stream | Stream | Yes | The name of the Kinesis data stream. | +| region | Region | No | AWS region. | +| format | Format | No | The format of the record | + +#### Local Files + +| Key | UX Label | Required | Description | +|--------|----------|----------|----------------------------------------------------| +| path | Path | Yes | The relative or absolute path to the data file(s). | +| format | Format | Yes | The format of the file(s) | + +#### MySQL Server + +| Key | UX Label | Required | Description | +|----------|----------|----------|-------------------------------------------------| +| host | Host | Yes | The host of the MySql server. | +| port | Port | No | The port of the MySql server. Defaults to 3306. | +| database | Database | Yes | The name of the database. | + +#### Oracle + +| Key | UX Label | Required | Description | +|-------------|--------------|----------|--------------------------------| +| host | Host | Yes | The host to the Oracle server | +| port | Port | Yes | The port to the Oracle server. | +| serviceName | Service Name | Yes | The name of the service. | + +#### PostgreSQL +[PostgreSQL](https://www.postgresql.org/) is a powerful, open source object-relational database system with over 35 years of active development that has earned it a strong reputation for reliability, feature robustness, and performance. + +| Key | UX Label | Required | Description | +|----------|----------|----------|------------------------------------------------------| +| host | Host | Yes | The host to the PostgreSQL server | +| port | Port | No | The port to the PostgreSQL server. Defaults to 5432. | +| database | Database | Yes | The name of the database. | +| schema | Schema | No | The name of the schema in the database. | + +#### Presto Server + +| Key | UX Label | Required | Description | +|---------|----------|----------|-------------------------------| +| host | Host | Yes | The host to the Presto server | +| catalog | Catalog | No | The name of the catalog. | +| schema | Schema | No | The name of the schema. | + +#### Google Pub/Sub +[Google Cloud](https://cloud.google.com/pubsub) service to Ingest events for streaming into BigQuery, data lakes or operational databases. + +| Key | UX Label | Required | Description | +|---------|----------|----------|-----------------------| +| project | Project | Yes | The GCP project name. | + +#### Amazon Redshift Server +[Amazon Redshift](https://aws.amazon.com/redshift/) is a power data driven decisions with the best price-performance cloud data warehouse. + +| Key | UX Label | Required | Description | +|----------|----------|----------|-------------------------------------------| +| database | Database | Yes | The name of the database. | +| schema | Schema | Yes | The name of the schema. | +| host | Host | No | An optional string describing the server. | +| region | Region | No | AWS region of Redshift server. | +| account | Account | No | The account used by the server. | + +#### Amazon S3 Server and Compatible Servers +[Amazon Simple Storage Service (Amazon S3)](https://aws.amazon.com/s3/) is an object storage service offering industry-leading scalability, data availability, security, and performance. Millions of customers of all sizes and industries store, manage, analyze, and protect any amount of data for virtually any use case, such as data lakes, cloud-native applications, and mobile apps. Other vendors have implemented a compatible implementation of S3. + +| Key | UX Label | Required | Description | +|-------------|--------------|----------|-----------------------------------------------------------------------------------| +| location | Location | Yes | S3 URL, starting with `s3://` | +| endpointUrl | Endpoint URL | No | The server endpoint for S3-compatible servers. | +| format | Format | No | File format. | +| delimiter | Delimiter | No | Only for format = json. How multiple json documents are delimited within one file | + +#### SFTP Server +Secure File Transfer Protocol (SFTP) is a network protocol that enables secure and encrypted file transfers between a client and a server. + +| Key | UX Label | Required | Description | +|-----------|-----------|----------|-----------------------------------------------------------------------------------| +| location | Location | Yes | SFTP URL, starting with `sftp://`. The URL should include the port number. | +| format | Format | No | File format. | +| delimiter | Delimiter | No | Only for format = json. How multiple json documents are delimited within one file | + +#### Snowflake + +| Key | UX Label | Required | Description | +|-----------|-----------|----------|-----------------------------------------------------------------------------| +| host | Host | Yes | The host to the Snowflake server | +| port | Port | Yes | The port to the Snowflake server. | +| account | Account | Yes | The Snowflake account used by the server. | +| database | Database | Yes | The name of the database. | +| warehouse | Warehouse | Yes | The name of the cluster of resources that is a Snowflake virtual warehouse. | +| schema | Schema | Yes | The name of the schema. | + +#### Microsoft SQL Server +[Microsoft SQL Server](https://www.microsoft.com/en-us/sql-server/sql-server-downloads) is a proprietary relational database management system developed by Microsoft. + +| Key | UX Label | Required | Description | +|----------|----------|----------|----------------------------------------------------| +| host | Host | Yes | The host to the database server | +| port | Port | No | The port to the database server. Defaults to 1433. | +| database | Database | Yes | The name of the database. | +| schema | Schema | Yes | The name of the schema in the database. | + +#### Synapse Server + +| Key | UX Label | Required | Description | +|----------|----------|----------|---------------------------------| +| host | Host | Yes | The host of the Synapse server. | +| port | Port | Yes | The port of the Synapse server. | +| database | Database | Yes | The name of the database. | + +#### Trino Server + +| Key | UX Label | Required | Description | +|---------|----------|----------|-----------------------------------------| +| host | Host | Yes | The Trino host URL. | +| port | Port | Yes | The Trino port. | +| catalog | Catalog | Yes | The name of the catalog. | +| schema | Schema | Yes | The name of the schema in the database. | + +#### Vertica Server + +| Key | UX Label | Required | Description | +|----------|----------|----------|---------------------------------| +| host | Host | Yes | The host of the Vertica server. | +| port | Port | Yes | The port of the Vertica server. | +| database | Database | Yes | The name of the database. | +| schema | Schema | Yes | The name of the schema. | + +#### Custom Server + +| Key | UX Label | Required | Description | +|-------------|-------------------|----------|---------------------------------------------------------------------| +| account | Account | No | Account used by the server. | +| catalog | Catalog | No | Name of the catalog. | +| database | Database | No | Name of the database. | +| dataset | Dataset | No | Name of the dataset. | +| delimiter | Delimiter | No | Delimiter. | +| endpointUrl | Endpoint URL | No | Server endpoint. | +| format | Format | No | File format. | +| host | Host | No | Host name or IP address. | +| location | Location | No | A URL to a location. | +| path | Path | No | Relative or absolute path to the data file(s). | +| port | Port | No | Port to the server. No default value is assumed for custom servers. | +| project | Project | No | Project name. | +| region | Region | No | Cloud region. | +| regionName | Region Name | No | Region name. | +| schema | Schema | No | Name of the schema. | +| serviceName | Service Name | No | Name of the service. | +| stagingDir | Staging Directory | No | Staging directory. | +| stream | Stream | No | Name of the data stream. | +| warehouse | Warehouse | No | Name of the cluster or warehouse. | + +If you need another property, use [custom properties](#custom-properties). + + +## Custom Properties +This section covers custom properties you may find in a data contract. ### Example @@ -505,12 +1064,8 @@ customProperties: value: property.value - property: dataprocClusterName # Used for specific applications like Elevate value: [cluster name] - -systemInstance: instance.ClimateQuantum.org -contractCreatedTs: 2022-11-15 02:59:43 ``` - ### Definitions | Key | UX label | Required | Description | @@ -518,10 +1073,27 @@ contractCreatedTs: 2022-11-15 02:59:43 | customProperties | Custom Properties | No | A list of key/value pairs for custom properties. Initially created to support the REF ruleset property. | | customProperties.property | Property | No | The name of the key. Names should be in camel case–the same as if they were permanent properties in the contract. | | customProperties.value | Value | No | The value of the key. | -| systemInstance | System Instance | No | System Instance name where the dataset resides. | -| contractCreatedTs | Contract Created UTC | No | Timestamp in UTC of when the data contract was created. | +## Other Properties +This section covers other properties you may find in a data contract. + +### Example + +```YAML +contractCreatedTs: 2024-09-17T11:58:08Z +``` + + +### Other properties definition + +| Key | UX label | Required | Description | +|---------------------------|----------------------|----------|-------------------------------------------------------------------------| +| contractCreatedTs | Contract Created UTC | No | Timestamp in UTC of when the data contract was created, using ISO 8601. | + ## Full example [Check full example here.](examples/all/full-example.odcs.yaml) + + +All trademarks are the property of their respective owners. diff --git a/docs/examples/README.md b/docs/examples/README.md index 12fd420..fb75e18 100644 --- a/docs/examples/README.md +++ b/docs/examples/README.md @@ -1,7 +1,7 @@ # Examples of Data Contracts ## Executive summary -This folder contains mainly excerpt of data contracts to illustrate specific sections & behaviors. +This folder contains mainly excerpts of data contracts to illustrate specific sections & behaviors. ## Table of content * [Full example](#full-example) @@ -9,6 +9,7 @@ This folder contains mainly excerpt of data contracts to illustrate specific sec * [Datasets & schema](#dataset-and-schema) * [Data quality](#data-quality) * [Pricing](#pricing) +* [Server](#server) * [Stakeholders](#stakeholders) * [Roles](#roles) * [Service-level agreement](#service-level-agreement) @@ -25,6 +26,8 @@ This folder contains mainly excerpt of data contracts to illustrate specific sec ## Dataset and schema +- [All schema types](schema/all-schema-types.odcs.yaml) +- [Data types](data-types/all-data-types.odcs.yaml) - [Table with single column](schema/table-column.odcs.yaml) - [Table with columns and partitioning](schema/table-columns-with-partition.odcs.yaml) @@ -32,10 +35,16 @@ This folder contains mainly excerpt of data contracts to illustrate specific sec - [Column accuracy](quality/column-accuracy.odcs.yaml) - [Column completeness](quality/column-completeness.odcs.yaml) +- [Column custom](quality/column-custom.odcs.yaml) - [Column validity](quality/column-validity.odcs.yaml) +## Server + +- [Azure server](server/azure-server.odcs.yaml) +- [Kafka server](server/kafka-server.odcs.yaml) + ## Pricing -This section covers pricing when you bill your customer for using this data product. Pricing is experimental in v2.2.0 of the data contract. +This section covers pricing when you bill your customer for using this data product. ## Stakeholders diff --git a/docs/examples/all/full-example.odcs.yaml b/docs/examples/all/full-example.odcs.yaml index a9db806..3dfffe2 100644 --- a/docs/examples/all/full-example.odcs.yaml +++ b/docs/examples/all/full-example.odcs.yaml @@ -1,10 +1,9 @@ # What's this data contract about? -datasetDomain: seller # Domain -quantumName: my quantum # Data product name -userConsumptionMode: Analytical +domain: seller # Domain +dataProduct: my quantum # Data product name version: 1.1.0 # Version (follows semantic versioning) status: current -uuid: 53581432-6c55-4ba2-a65f-72344a91553a +id: 53581432-6c55-4ba2-a65f-72344a91553a # Lots of information description: @@ -13,98 +12,82 @@ description: usage: Predict sales over time tenant: ClimateQuantumInc -# Getting support -productDl: product-dl@ClimateQuantum.org -productSlackChannel: '#product-help' -productFeedbackUrl: https://product-feedback.com/sellers - -# Physical parts / GCP / BigQuery specific -sourcePlatform: googleCloudPlatform -sourceSystem: bigQuery -datasetProject: edw # BQ dataset -datasetName: access_views # BQ dataset - kind: DataContract -apiVersion: v2.2.2 # Standard version (follows semantic versioning, previously known as templateVersion) - -type: tables +apiVersion: v3.0.0 # Standard version (follows semantic versioning) -# Physical access -driver: jdbc:postgresql -driverVersion: 1.0.0 -server: localhost:5432 -database: pypl-edw.pp_access_views -username: '${env.username}' -password: '${env.password}' -schedulerAppName: name_coming_from_scheduler # NEW 2.1.0 Required if you want to schedule stuff, comes from DataALM. +# Infrastructure & servers +servers: + - server: my-postgres + type: postgres + host: localhost + port: 5432 + database: pypl-edw + schema: pp_access_views # Dataset, schema and quality -dataset: - - table: tbl - physicalName: tbl_1 # NEW in v2.1.0, Optional, default value is table name + version separated by underscores, as table_1_2_0 - priorTableName: seller_table # if needed +schema: + - name: tbl + physicalName: tbl_1 + physicalType: table description: Provides core payment metrics - authoritativeDefinitions: # NEW in v2.2.0, inspired by the column-level authoritative links + authoritativeDefinitions: - url: https://catalog.data.gov/dataset/air-quality type: businessDefinition - url: https://youtu.be/jbY1BKFj9ec type: videoTutorial - tags: null - dataGranularity: Aggregation on columns txn_ref_dt, pmt_txn_id - columns: - - column: txn_ref_dt - isPrimaryKey: false # NEW in v2.1.0, Optional, default value is false, indicates whether the column is primary key in the table. + tags: [ ] + dataGranularityDescription: Aggregation on columns txn_ref_dt, pmt_txn_id + properties: + - name: txn_ref_dt + primaryKey: false primaryKeyPosition: -1 businessName: transaction reference date logicalType: date physicalType: date - isNullable: false + required: false description: Reference date for transaction - partitionStatus: true + partitioned: true partitionKeyPosition: 1 - clusterStatus: false - clusterKeyPosition: -1 - criticalDataElementStatus: false - tags: [] + criticalDataElement: false + tags: [ ] classification: public - transformSourceTables: + transformSourceObjects: - table_name_1 - table_name_2 - table_name_3 transformLogic: sel t1.txn_dt as txn_ref_dt from table_name_1 as t1, table_name_2 as t2, table_name_3 as t3 where t1.txn_dt=date-3 transformDescription: defines the logic in business terms; logic for dummies - sampleValues: - - 2022-10-03 - - 2020-01-28 - - column: rcvr_id - isPrimaryKey: true # NEW in v2.1.0, Optional, default value is false, indicates whether the column is primary key in the table. + examples: + - "2022-10-03" + - "2020-01-28" + customProperties: + - property: anonymizationStrategy + value: none + - name: rcvr_id + primaryKey: true primaryKeyPosition: 1 businessName: receiver id logicalType: string physicalType: varchar(18) - isNullable: false + required: false description: A description for column rcvr_id. - partitionStatus: false + partitioned: false partitionKeyPosition: -1 - clusterStatus: true - clusterKeyPosition: 1 - criticalDataElementStatus: false - tags: [] + criticalDataElement: false + tags: [ ] classification: restricted - - column: rcvr_cntry_code - isPrimaryKey: false # NEW in v2.1.0, Optional, default value is false, indicates whether the column is primary key in the table. + - name: rcvr_cntry_code + primaryKey: false primaryKeyPosition: -1 businessName: receiver country code logicalType: string physicalType: varchar(2) - isNullable: false + required: false description: Country code - partitionStatus: false + partitioned: false partitionKeyPosition: -1 - clusterStatus: false - clusterKeyPosition: -1 - criticalDataElementStatus: false - tags: [] + criticalDataElement: false + tags: [ ] classification: public authoritativeDefinitions: - url: https://collibra.com/asset/742b358f-71a5-4ab1-bda4-dcdba9418c25 @@ -113,18 +96,16 @@ dataset: type: transformationImplementation - url: jdbc:postgresql://localhost:5432/adventureworks/tbl_1/rcvr_cntry_code type: implementation - encryptedColumnName: rcvr_cntry_code_encrypted + encryptedName: rcvr_cntry_code_encrypted quality: - - code: nullCheck - templateName: NullCheck + - rule: nullCheck description: column should not contain null values - toolName: Elevate - toolRuleName: DQ.rw.tab1_2_0_0.rcvr_cntry_code.NullCheck dimension: completeness # dropdown 7 values - type: dataQuality + type: library severity: error businessImpact: operational - scheduleCronExpression: 0 20 * * * + schedule: 0 20 * * * + scheduler: cron customProperties: - property: FIELD_NAME value: @@ -133,16 +114,21 @@ dataset: - property: COMPARISON_TYPE value: Greater than quality: - - code: countCheck # Required, name of the rule - templateName: CountCheck # NEW in v2.1.0 Required - description: Ensure row count is within expected volume range # Optional - toolName: Elevate # Required - toolRuleName: DQ.rw.tab1.CountCheck # NEW in v2.1.0 Optional (Available only to the users who can change in source code edition) - dimension: completeness # Optional - type: reconciliation # Optional NEW in v2.1.0 default value for column level check - dataQuality and for table level reconciliation - severity: error # Optional NEW in v2.1.0, default value is error - businessImpact: operational # Optional NEW in v2.1.0 - scheduleCronExpression: 0 20 * * * # Optional NEW in v2.1.0 default schedule - every day 10 a.m. UTC + - rule: countCheck + type: library + description: Ensure row count is within expected volume range + dimension: completeness + method: reconciliation + severity: error + businessImpact: operational + schedule: 0 20 * * * + scheduler: cron + customProperties: + - property: business-key + value: + - txn_ref_dt + - rcvr_id + # Pricing price: @@ -150,20 +136,22 @@ price: priceCurrency: USD priceUnit: megabyte -# Stakeholders -stakeholders: + +# Team +team: - username: ceastwood role: Data Scientist - dateIn: 2022-08-02 - dateOut: 2022-10-01 + dateIn: "2022-08-02" + dateOut: "2022-10-01" replacedByUsername: mhopper - username: mhopper role: Data Scientist - dateIn: 2022-10-01 + dateIn: "2022-10-01" - username: daustin role: Owner comment: Keeper of the grail - dateIn: 2022-10-01 + dateIn: "2022-10-01" + # Roles roles: @@ -185,40 +173,54 @@ roles: secondLevelApprovers: 'mickey' # SLA -slaDefaultColumn: tab1.txn_ref_dt # Optional, default value is partitionColumn. +slaDefaultElement: tab1.txn_ref_dt slaProperties: - property: latency # Property, see list of values in DP QoS value: 4 unit: d # d, day, days for days; y, yr, years for years - column: tab1.txn_ref_dt # This would not be needed as it is the same table.column as the default one + element: tab1.txn_ref_dt # This would not be needed as it is the same table.column as the default one - property: generalAvailability - value: 2022-05-12T09:30:10-08:00 + value: "2022-05-12T09:30:10-08:00" - property: endOfSupport - value: 2032-05-12T09:30:10-08:00 + value: "2032-05-12T09:30:10-08:00" - property: endOfLife - value: 2042-05-12T09:30:10-08:00 + value: "2042-05-12T09:30:10-08:00" - property: retention value: 3 unit: y - column: tab1.txn_ref_dt + element: tab1.txn_ref_dt - property: frequency value: 1 valueExt: 1 unit: d - column: tab1.txn_ref_dt + element: tab1.txn_ref_dt - property: timeOfAvailability value: 09:00-08:00 - column: tab1.txn_ref_dt + element: tab1.txn_ref_dt driver: regulatory # Describes the importance of the SLA: [regulatory|analytics|operational|...] - property: timeOfAvailability value: 08:00-08:00 - column: tab1.txn_ref_dt + element: tab1.txn_ref_dt driver: analytics + +# Support +support: + - channel: '#product-help' # Simple Slack communication channel + tool: slack + url: https://aidaug.slack.com/archives/C05UZRSBKLY + - channel: datacontract-ann # Simple distribution list + tool: email + url: mailto:datacontract-ann@bitol.io + - channel: Feedback # Product Feedback + description: General Product Feedback (Public) + url: https://product-feedback.com + # Tags tags: - transactions + # Custom properties customProperties: - property: refRulesetName @@ -226,7 +228,6 @@ customProperties: - property: somePropertyName value: property.value - property: dataprocClusterName # Used for specific applications like Elevate - value: [cluster name] + value: [ cluster name ] -systemInstance: instance.ClimateQuantum.org -contractCreatedTs: 2022-11-15 02:59:43 +contractCreatedTs: "2022-11-15T02:59:43+00:00" diff --git a/docs/examples/all/postgresql-adventureworks-contract.odcs.yaml b/docs/examples/all/postgresql-adventureworks-contract.odcs.yaml index 9092916..73950a6 100644 --- a/docs/examples/all/postgresql-adventureworks-contract.odcs.yaml +++ b/docs/examples/all/postgresql-adventureworks-contract.odcs.yaml @@ -1,5719 +1,5324 @@ version: "1.0.0" -standardVersion: "2.2.0" -uuid: "6aeafdc1-ed62-4c8f-bf0a-da1061c98cdb" -username: "postgres" -type: "tables" +apiVersion: "v3.0.0" +id: "6aeafdc1-ed62-4c8f-bf0a-da1061c98cdb" status: "Development" -server: "localhost" -kind: "managedDataset" -driverVersion: "42.6.0" -driver: "org.postgresql.Driver" +kind: "DataContract" description: {} -database: "adventureworks" -dataset: - tables: - - table: "department" +schema: + - name: "department" physicalName: "department" + physicalType: "table" description: "Lookup table containing the departments within the Adventure Works\ \ Cycles company." - columns: - - column: "departmentid" - logicalType: "Numeric" + properties: + - name: "departmentid" + logicalType: "number" physicalType: "serial" description: "Primary key for Department records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/department/departmentid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Name of the department." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/department/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "groupname" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "groupname" + logicalType: "object" physicalType: "Name" description: "Name of the group to which the department belongs." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/department/groupname" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/department/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "employee" + criticalDataElement: false + primaryKey: false + required: false + - name: "employee" physicalName: "employee" + physicalType: "table" description: "Employee information such as salary, department, and title." - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Primary key for Employee records. Foreign key to BusinessEntity.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "nationalidnumber" - logicalType: "String" + criticalDataElement: false + primaryKey: true + required: false + - name: "nationalidnumber" + logicalType: "string" physicalType: "varchar[15]" description: "Unique national identification number such as a social security\ \ number." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/nationalidnumber" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "loginid" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "loginid" + logicalType: "string" physicalType: "varchar[256]" description: "Network login." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/loginid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "jobtitle" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "jobtitle" + logicalType: "string" physicalType: "varchar[50]" description: "Work title such as Buyer or Sales Representative." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/jobtitle" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "birthdate" - logicalType: "Date" + criticalDataElement: false + primaryKey: false + required: false + - name: "birthdate" + logicalType: "date" physicalType: "date" description: "Date of birth." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/birthdate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "maritalstatus" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "maritalstatus" + logicalType: "string" physicalType: "bpchar" description: "M = Married, S = Single" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/maritalstatus" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "gender" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "gender" + logicalType: "string" physicalType: "bpchar" description: "M = Male, F = Female" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/gender" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "hiredate" - logicalType: "Date" + criticalDataElement: false + primaryKey: false + required: false + - name: "hiredate" + logicalType: "date" physicalType: "date" description: "Employee hired on this date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/hiredate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "salariedflag" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "salariedflag" + logicalType: "object" physicalType: "Flag" description: "Job classification. 0 = Hourly, not exempt from collective bargaining.\ \ 1 = Salaried, exempt from collective bargaining." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/salariedflag" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "vacationhours" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "vacationhours" + logicalType: "number" physicalType: "int2" description: "Number of available vacation hours." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/vacationhours" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "sickleavehours" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "sickleavehours" + logicalType: "number" physicalType: "int2" description: "Number of available sick leave hours." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/sickleavehours" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "currentflag" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "currentflag" + logicalType: "object" physicalType: "Flag" description: "0 = Inactive, 1 = Active" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/currentflag" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "organizationnode" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "organizationnode" + logicalType: "string" physicalType: "varchar[2147483647]" description: "Where the employee is located in corporate hierarchy." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employee/organizationnode" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "employeedepartmenthistory" + criticalDataElement: false + primaryKey: false + required: false + - name: "employeedepartmenthistory" physicalName: "employeedepartmenthistory" + physicalType: "table" description: "Employee department transfers." - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Employee identification number. Foreign key to Employee.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employeedepartmenthistory/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "departmentid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "departmentid" + logicalType: "number" physicalType: "int2" description: "Department in which the employee worked including currently. Foreign\ \ key to Department.DepartmentID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employeedepartmenthistory/departmentid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "shiftid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "shiftid" + logicalType: "number" physicalType: "int2" description: "Identifies which 8-hour shift the employee works. Foreign key\ \ to Shift.Shift.ID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employeedepartmenthistory/shiftid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "startdate" - logicalType: "Date" + criticalDataElement: false + primaryKey: true + required: false + - name: "startdate" + logicalType: "date" physicalType: "date" description: "Date the employee started work in the department." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employeedepartmenthistory/startdate" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "enddate" - logicalType: "Date" + criticalDataElement: false + primaryKey: true + required: false + - name: "enddate" + logicalType: "date" physicalType: "date" description: "Date the employee left the department. NULL = Current department." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employeedepartmenthistory/enddate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employeedepartmenthistory/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "employeepayhistory" + criticalDataElement: false + primaryKey: false + required: false + - name: "employeepayhistory" physicalName: "employeepayhistory" + physicalType: "table" description: "Employee pay history." - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Employee identification number. Foreign key to Employee.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employeepayhistory/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "ratechangedate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "ratechangedate" + logicalType: "date" physicalType: "timestamp" description: "Date the change in pay is effective" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employeepayhistory/ratechangedate" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "rate" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "rate" + logicalType: "number" physicalType: "numeric" description: "Salary hourly rate." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employeepayhistory/rate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "payfrequency" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "payfrequency" + logicalType: "number" physicalType: "int2" description: "1 = Salary received monthly, 2 = Salary received biweekly" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employeepayhistory/payfrequency" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/employeepayhistory/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "jobcandidate" + criticalDataElement: false + primaryKey: false + required: false + - name: "jobcandidate" physicalName: "jobcandidate" + physicalType: "table" description: "Résumés submitted to Human Resources by job applicants." - columns: - - column: "jobcandidateid" - logicalType: "Numeric" + properties: + - name: "jobcandidateid" + logicalType: "number" physicalType: "serial" description: "Primary key for JobCandidate records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/jobcandidate/jobcandidateid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "businessentityid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Employee identification number if applicant was hired. Foreign\ \ key to Employee.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/jobcandidate/businessentityid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "resume" - logicalType: "XML" + criticalDataElement: false + primaryKey: false + required: false + - name: "resume" + logicalType: "string" physicalType: "xml" description: "Résumé in XML format." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/jobcandidate/resume" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/jobcandidate/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "shift" + criticalDataElement: false + primaryKey: false + required: false + - name: "shift" physicalName: "shift" + physicalType: "table" description: "Work shift lookup table." - columns: - - column: "shiftid" - logicalType: "Numeric" + properties: + - name: "shiftid" + logicalType: "number" physicalType: "serial" description: "Primary key for Shift records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shift/shiftid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Shift description." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shift/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "starttime" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "starttime" + logicalType: "date" physicalType: "time" description: "Shift start time." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shift/starttime" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "endtime" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "endtime" + logicalType: "date" physicalType: "time" description: "Shift end time." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shift/endtime" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shift/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "address" + criticalDataElement: false + primaryKey: false + required: false + - name: "address" physicalName: "address" + physicalType: "table" description: "Street address information for customers, employees, and vendors." - columns: - - column: "addressid" - logicalType: "Numeric" + properties: + - name: "addressid" + logicalType: "number" physicalType: "serial" description: "Primary key for Address records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/address/addressid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "addressline1" - logicalType: "String" + criticalDataElement: false + primaryKey: true + required: false + - name: "addressline1" + logicalType: "string" physicalType: "varchar[60]" description: "First street address line." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/address/addressline1" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "addressline2" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "addressline2" + logicalType: "string" physicalType: "varchar[60]" description: "Second street address line." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/address/addressline2" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "city" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "city" + logicalType: "string" physicalType: "varchar[30]" description: "Name of the city." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/address/city" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "stateprovinceid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "stateprovinceid" + logicalType: "number" physicalType: "int4" description: "Unique identification number for the state or province. Foreign\ \ key to StateProvince table." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/address/stateprovinceid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "postalcode" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "postalcode" + logicalType: "string" physicalType: "varchar[15]" description: "Postal code for the street address." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/address/postalcode" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "spatiallocation" - logicalType: "Binary" + criticalDataElement: false + primaryKey: false + required: false + - name: "spatiallocation" + logicalType: "string" physicalType: "bytea" description: "Latitude and longitude of this address." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/address/spatiallocation" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/address/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/address/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "addresstype" + criticalDataElement: false + primaryKey: false + required: false + - name: "addresstype" physicalName: "addresstype" + physicalType: "table" description: "Types of addresses stored in the Address table." - columns: - - column: "addresstypeid" - logicalType: "Numeric" + properties: + - name: "addresstypeid" + logicalType: "number" physicalType: "serial" description: "Primary key for AddressType records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/addresstype/addresstypeid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Address type description. For example, Billing, Home, or Shipping." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/addresstype/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/addresstype/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/addresstype/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "businessentity" + criticalDataElement: false + primaryKey: false + required: false + - name: "businessentity" physicalName: "businessentity" + physicalType: "table" description: "Source of the ID that connects vendors, customers, and employees\ \ with address and contact information." - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "serial" description: "Primary key for all customers, vendors, and employees." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/businessentity/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: true + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/businessentity/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/businessentity/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "businessentityaddress" + criticalDataElement: false + primaryKey: false + required: false + - name: "businessentityaddress" physicalName: "businessentityaddress" + physicalType: "table" description: "Cross-reference table mapping customers, vendors, and employees\ \ to their addresses." - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to BusinessEntity.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/businessentityaddress/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "addressid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "addressid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to Address.AddressID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/businessentityaddress/addressid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "addresstypeid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "addresstypeid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to AddressType.AddressTypeID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/businessentityaddress/addresstypeid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: true + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/businessentityaddress/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/businessentityaddress/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "businessentitycontact" + criticalDataElement: false + primaryKey: false + required: false + - name: "businessentitycontact" physicalName: "businessentitycontact" + physicalType: "table" description: "Cross-reference table mapping stores, vendors, and employees to\ \ people" - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to BusinessEntity.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/businessentitycontact/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "personid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "personid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to Person.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/businessentitycontact/personid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "contacttypeid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "contacttypeid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to ContactType.ContactTypeID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/businessentitycontact/contacttypeid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: true + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/businessentitycontact/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/businessentitycontact/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "contacttype" + criticalDataElement: false + primaryKey: false + required: false + - name: "contacttype" physicalName: "contacttype" + physicalType: "table" description: "Lookup table containing the types of business entity contacts." - columns: - - column: "contacttypeid" - logicalType: "Numeric" + properties: + - name: "contacttypeid" + logicalType: "number" physicalType: "serial" description: "Primary key for ContactType records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/contacttype/contacttypeid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Contact type description." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/contacttype/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/contacttype/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "countryregion" + criticalDataElement: false + primaryKey: false + required: false + - name: "countryregion" physicalName: "countryregion" + physicalType: "table" description: "Lookup table containing the ISO standard codes for countries and\ \ regions." - columns: - - column: "countryregioncode" - logicalType: "String" + properties: + - name: "countryregioncode" + logicalType: "string" physicalType: "varchar[3]" description: "ISO standard code for countries and regions." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/countryregion/countryregioncode" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Country or region name." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/countryregion/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/countryregion/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "emailaddress" + criticalDataElement: false + primaryKey: false + required: false + - name: "emailaddress" physicalName: "emailaddress" + physicalType: "table" description: "Where to send a person email." - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Primary key. Person associated with this email address. Foreign\ \ key to Person.BusinessEntityID" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/emailaddress/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "emailaddressid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "emailaddressid" + logicalType: "number" physicalType: "serial" description: "Primary key. ID of this email address." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/emailaddress/emailaddressid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "emailaddress" - logicalType: "String" + criticalDataElement: false + primaryKey: true + required: false + - name: "emailaddress" + logicalType: "string" physicalType: "varchar[50]" description: "E-mail address for the person." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/emailaddress/emailaddress" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/emailaddress/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/emailaddress/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "password" + criticalDataElement: false + primaryKey: false + required: false + - name: "password" physicalName: "password" + physicalType: "table" description: "One way hashed authentication information" - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/password/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "passwordhash" - logicalType: "String" + criticalDataElement: false + primaryKey: true + required: false + - name: "passwordhash" + logicalType: "string" physicalType: "varchar[128]" description: "Password for the e-mail account." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/password/passwordhash" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "passwordsalt" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "passwordsalt" + logicalType: "string" physicalType: "varchar[10]" description: "Random value concatenated with the password string before the\ \ password is hashed." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/password/passwordsalt" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/password/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/password/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "person" + criticalDataElement: false + primaryKey: false + required: false + - name: "person" physicalName: "person" + physicalType: "table" description: "Human beings involved with AdventureWorks: employees, customer contacts,\ \ and vendor contacts." - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Primary key for Person records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/person/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "persontype" - logicalType: "String" + criticalDataElement: false + primaryKey: true + required: false + - name: "persontype" + logicalType: "string" physicalType: "bpchar" description: "Primary type of person: SC = Store Contact, IN = Individual (retail)\ \ customer, SP = Sales person, EM = Employee (non-sales), VC = Vendor contact,\ \ GC = General contact" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/person/persontype" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "namestyle" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "namestyle" + logicalType: "object" physicalType: "NameStyle" description: "0 = The data in FirstName and LastName are stored in western style\ \ (first name, last name) order. 1 = Eastern style (last name, first name)\ \ order." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/person/namestyle" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "title" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "title" + logicalType: "string" physicalType: "varchar[8]" description: "A courtesy title. For example, Mr. or Ms." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/person/title" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "firstname" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "firstname" + logicalType: "object" physicalType: "Name" description: "First name of the person." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/person/firstname" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "middlename" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "middlename" + logicalType: "object" physicalType: "Name" description: "Middle name or middle initial of the person." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/person/middlename" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "lastname" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "lastname" + logicalType: "object" physicalType: "Name" description: "Last name of the person." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/person/lastname" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "suffix" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "suffix" + logicalType: "string" physicalType: "varchar[10]" description: "Surname suffix. For example, Sr. or Jr." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/person/suffix" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "emailpromotion" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "emailpromotion" + logicalType: "number" physicalType: "int4" description: "0 = Contact does not wish to receive e-mail promotions, 1 = Contact\ \ does wish to receive e-mail promotions from AdventureWorks, 2 = Contact\ \ does wish to receive e-mail promotions from AdventureWorks and selected\ \ partners." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/person/emailpromotion" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "additionalcontactinfo" - logicalType: "XML" + criticalDataElement: false + primaryKey: false + required: false + - name: "additionalcontactinfo" + logicalType: "string" physicalType: "xml" description: "Additional contact information about the person stored in xml\ \ format." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/person/additionalcontactinfo" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "demographics" - logicalType: "XML" + criticalDataElement: false + primaryKey: false + required: false + - name: "demographics" + logicalType: "string" physicalType: "xml" description: "Personal information such as hobbies, and income collected from\ \ online shoppers. Used for sales analysis." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/person/demographics" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/person/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/person/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "personphone" + criticalDataElement: false + primaryKey: false + required: false + - name: "personphone" physicalName: "personphone" + physicalType: "table" description: "Telephone number and type of a person." - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Business entity identification number. Foreign key to Person.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/personphone/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "phonenumber" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "phonenumber" + logicalType: "object" physicalType: "Phone" description: "Telephone number identification number." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/personphone/phonenumber" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "phonenumbertypeid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "phonenumbertypeid" + logicalType: "number" physicalType: "int4" description: "Kind of phone number. Foreign key to PhoneNumberType.PhoneNumberTypeID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/personphone/phonenumbertypeid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/personphone/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "phonenumbertype" + criticalDataElement: false + primaryKey: false + required: false + - name: "phonenumbertype" physicalName: "phonenumbertype" + physicalType: "table" description: "Type of phone number of a person." - columns: - - column: "phonenumbertypeid" - logicalType: "Numeric" + properties: + - name: "phonenumbertypeid" + logicalType: "number" physicalType: "serial" description: "Primary key for telephone number type records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/phonenumbertype/phonenumbertypeid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Name of the telephone number type" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/phonenumbertype/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/phonenumbertype/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "stateprovince" + criticalDataElement: false + primaryKey: false + required: false + - name: "stateprovince" physicalName: "stateprovince" + physicalType: "table" description: "State and province lookup table." - columns: - - column: "stateprovinceid" - logicalType: "Numeric" + properties: + - name: "stateprovinceid" + logicalType: "number" physicalType: "serial" description: "Primary key for StateProvince records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/stateprovince/stateprovinceid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "stateprovincecode" - logicalType: "String" + criticalDataElement: false + primaryKey: true + required: false + - name: "stateprovincecode" + logicalType: "string" physicalType: "bpchar" description: "ISO standard state or province code." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/stateprovince/stateprovincecode" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "countryregioncode" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "countryregioncode" + logicalType: "string" physicalType: "varchar[3]" description: "ISO standard country or region code. Foreign key to CountryRegion.CountryRegionCode." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/stateprovince/countryregioncode" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "isonlystateprovinceflag" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "isonlystateprovinceflag" + logicalType: "object" physicalType: "Flag" description: "0 = StateProvinceCode exists. 1 = StateProvinceCode unavailable,\ \ using CountryRegionCode." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/stateprovince/isonlystateprovinceflag" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "State or province description." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/stateprovince/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "territoryid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "territoryid" + logicalType: "number" physicalType: "int4" description: "ID of the territory in which the state or province is located.\ \ Foreign key to SalesTerritory.SalesTerritoryID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/stateprovince/territoryid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/stateprovince/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/stateprovince/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "billofmaterials" + criticalDataElement: false + primaryKey: false + required: false + - name: "billofmaterials" physicalName: "billofmaterials" + physicalType: "table" description: "Items required to make bicycles and bicycle subassemblies. It identifies\ \ the heirarchical relationship between a parent product and its components." - columns: - - column: "billofmaterialsid" - logicalType: "Numeric" + properties: + - name: "billofmaterialsid" + logicalType: "number" physicalType: "serial" description: "Primary key for BillOfMaterials records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/billofmaterials/billofmaterialsid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "productassemblyid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "productassemblyid" + logicalType: "number" physicalType: "int4" description: "Parent product identification number. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/billofmaterials/productassemblyid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "componentid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "componentid" + logicalType: "number" physicalType: "int4" description: "Component identification number. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/billofmaterials/componentid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "startdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "startdate" + logicalType: "date" physicalType: "timestamp" description: "Date the component started being used in the assembly item." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/billofmaterials/startdate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "enddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "enddate" + logicalType: "date" physicalType: "timestamp" description: "Date the component stopped being used in the assembly item." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/billofmaterials/enddate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "unitmeasurecode" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "unitmeasurecode" + logicalType: "string" physicalType: "bpchar" description: "Standard code identifying the unit of measure for the quantity." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/billofmaterials/unitmeasurecode" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "bomlevel" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "bomlevel" + logicalType: "number" physicalType: "int2" description: "Indicates the depth the component is from its parent (AssemblyID)." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/billofmaterials/bomlevel" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "perassemblyqty" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "perassemblyqty" + logicalType: "number" physicalType: "numeric" description: "Quantity of the component needed to create the assembly." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/billofmaterials/perassemblyqty" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/billofmaterials/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "culture" + criticalDataElement: false + primaryKey: false + required: false + - name: "culture" physicalName: "culture" + physicalType: "table" description: "Lookup table containing the languages in which some AdventureWorks\ \ data is stored." - columns: - - column: "cultureid" - logicalType: "String" + properties: + - name: "cultureid" + logicalType: "string" physicalType: "bpchar" description: "Primary key for Culture records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/culture/cultureid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Culture description." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/culture/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/culture/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "document" + criticalDataElement: false + primaryKey: false + required: false + - name: "document" physicalName: "document" + physicalType: "table" description: "Product maintenance documents." - columns: - - column: "title" - logicalType: "String" + properties: + - name: "title" + logicalType: "string" physicalType: "varchar[50]" description: "Title of the document." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/document/title" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "owner" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "owner" + logicalType: "number" physicalType: "int4" description: "Employee who controls the document. Foreign key to Employee.BusinessEntityID" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/document/owner" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "folderflag" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "folderflag" + logicalType: "object" physicalType: "Flag" description: "0 = This is a folder, 1 = This is a document." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/document/folderflag" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "filename" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "filename" + logicalType: "string" physicalType: "varchar[400]" description: "File name of the document" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/document/filename" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "fileextension" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "fileextension" + logicalType: "string" physicalType: "varchar[8]" description: "File extension indicating the document type. For example, .doc\ \ or .txt." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/document/fileextension" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "revision" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "revision" + logicalType: "string" physicalType: "bpchar" description: "Revision number of the document." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/document/revision" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "changenumber" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "changenumber" + logicalType: "number" physicalType: "int4" description: "Engineering change approval number." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/document/changenumber" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "status" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "status" + logicalType: "number" physicalType: "int2" description: "1 = Pending approval, 2 = Approved, 3 = Obsolete" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/document/status" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "documentsummary" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "documentsummary" + logicalType: "string" physicalType: "text" description: "Document abstract." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/document/documentsummary" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "document" - logicalType: "Binary" + criticalDataElement: false + primaryKey: false + required: false + - name: "document" + logicalType: "string" physicalType: "bytea" description: "Complete document." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/document/document" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" description: "ROWGUIDCOL number uniquely identifying the record. Required for\ \ FileStream." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/document/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/document/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "documentnode" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "documentnode" + logicalType: "string" physicalType: "varchar[2147483647]" description: "Primary key for Document records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/document/documentnode" - criticalDataElementStatus: false - primary: true - nullable: false - - table: "illustration" + criticalDataElement: false + primaryKey: true + required: false + - name: "illustration" physicalName: "illustration" + physicalType: "table" description: "Bicycle assembly diagrams." - columns: - - column: "illustrationid" - logicalType: "Numeric" + properties: + - name: "illustrationid" + logicalType: "number" physicalType: "serial" description: "Primary key for Illustration records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/illustration/illustrationid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "diagram" - logicalType: "XML" + criticalDataElement: false + primaryKey: true + required: false + - name: "diagram" + logicalType: "string" physicalType: "xml" description: "Illustrations used in manufacturing instructions. Stored as XML." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/illustration/diagram" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/illustration/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "location" + criticalDataElement: false + primaryKey: false + required: false + - name: "location" physicalName: "location" + physicalType: "table" description: "Product inventory and manufacturing locations." - columns: - - column: "locationid" - logicalType: "Numeric" + properties: + - name: "locationid" + logicalType: "number" physicalType: "serial" description: "Primary key for Location records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/location/locationid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Location description." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/location/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "costrate" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "costrate" + logicalType: "number" physicalType: "numeric" description: "Standard hourly cost of the manufacturing location." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/location/costrate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "availability" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "availability" + logicalType: "number" physicalType: "numeric" description: "Work capacity (in hours) of the manufacturing location." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/location/availability" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/location/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "product" + criticalDataElement: false + primaryKey: false + required: false + - name: "product" physicalName: "product" + physicalType: "table" description: "Products sold or used in the manfacturing of sold products." - columns: - - column: "productid" - logicalType: "Numeric" + properties: + - name: "productid" + logicalType: "number" physicalType: "serial" description: "Primary key for Product records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/productid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Name of the product." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "productnumber" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "productnumber" + logicalType: "string" physicalType: "varchar[25]" description: "Unique product identification number." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/productnumber" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "makeflag" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "makeflag" + logicalType: "object" physicalType: "Flag" description: "0 = Product is purchased, 1 = Product is manufactured in-house." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/makeflag" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "finishedgoodsflag" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "finishedgoodsflag" + logicalType: "object" physicalType: "Flag" description: "0 = Product is not a salable item. 1 = Product is salable." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/finishedgoodsflag" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "color" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "color" + logicalType: "string" physicalType: "varchar[15]" description: "Product color." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/color" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "safetystocklevel" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "safetystocklevel" + logicalType: "number" physicalType: "int2" description: "Minimum inventory quantity." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/safetystocklevel" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "reorderpoint" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "reorderpoint" + logicalType: "number" physicalType: "int2" description: "Inventory level that triggers a purchase order or work order." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/reorderpoint" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "standardcost" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "standardcost" + logicalType: "number" physicalType: "numeric" description: "Standard cost of the product." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/standardcost" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "listprice" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "listprice" + logicalType: "number" physicalType: "numeric" description: "Selling price." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/listprice" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "size" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "size" + logicalType: "string" physicalType: "varchar[5]" description: "Product size." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/size" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "sizeunitmeasurecode" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "sizeunitmeasurecode" + logicalType: "string" physicalType: "bpchar" description: "Unit of measure for Size column." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/sizeunitmeasurecode" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "weightunitmeasurecode" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "weightunitmeasurecode" + logicalType: "string" physicalType: "bpchar" description: "Unit of measure for Weight column." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/weightunitmeasurecode" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "weight" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "weight" + logicalType: "number" physicalType: "numeric" description: "Product weight." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/weight" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "daystomanufacture" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "daystomanufacture" + logicalType: "number" physicalType: "int4" description: "Number of days required to manufacture the product." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/daystomanufacture" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "productline" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "productline" + logicalType: "string" physicalType: "bpchar" description: "R = Road, M = Mountain, T = Touring, S = Standard" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/productline" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "class" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "class" + logicalType: "string" physicalType: "bpchar" description: "H = High, M = Medium, L = Low" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/class" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "style" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "style" + logicalType: "string" physicalType: "bpchar" description: "W = Womens, M = Mens, U = Universal" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/style" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "productsubcategoryid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "productsubcategoryid" + logicalType: "number" physicalType: "int4" description: "Product is a member of this product subcategory. Foreign key to\ \ ProductSubCategory.ProductSubCategoryID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/productsubcategoryid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "productmodelid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "productmodelid" + logicalType: "number" physicalType: "int4" description: "Product is a member of this product model. Foreign key to ProductModel.ProductModelID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/productmodelid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "sellstartdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "sellstartdate" + logicalType: "date" physicalType: "timestamp" description: "Date the product was available for sale." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/sellstartdate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "sellenddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "sellenddate" + logicalType: "date" physicalType: "timestamp" description: "Date the product was no longer available for sale." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/sellenddate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "discontinueddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "discontinueddate" + logicalType: "date" physicalType: "timestamp" description: "Date the product was discontinued." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/discontinueddate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/product/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "productcategory" + criticalDataElement: false + primaryKey: false + required: false + - name: "productcategory" physicalName: "productcategory" + physicalType: "table" description: "High-level product categorization." - columns: - - column: "productcategoryid" - logicalType: "Numeric" + properties: + - name: "productcategoryid" + logicalType: "number" physicalType: "serial" description: "Primary key for ProductCategory records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productcategory/productcategoryid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Category description." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productcategory/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productcategory/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productcategory/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "productcosthistory" + criticalDataElement: false + primaryKey: false + required: false + - name: "productcosthistory" physicalName: "productcosthistory" + physicalType: "table" description: "Changes in the cost of a product over time." - columns: - - column: "productid" - logicalType: "Numeric" + properties: + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Product identification number. Foreign key to Product.ProductID" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productcosthistory/productid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "startdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "startdate" + logicalType: "date" physicalType: "timestamp" description: "Product cost start date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productcosthistory/startdate" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "enddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "enddate" + logicalType: "date" physicalType: "timestamp" description: "Product cost end date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productcosthistory/enddate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "standardcost" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "standardcost" + logicalType: "number" physicalType: "numeric" description: "Standard cost of the product." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productcosthistory/standardcost" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productcosthistory/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "productdescription" + criticalDataElement: false + primaryKey: false + required: false + - name: "productdescription" physicalName: "productdescription" + physicalType: "table" description: "Product descriptions in several languages." - columns: - - column: "productdescriptionid" - logicalType: "Numeric" + properties: + - name: "productdescriptionid" + logicalType: "number" physicalType: "serial" description: "Primary key for ProductDescription records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productdescription/productdescriptionid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "description" - logicalType: "String" + criticalDataElement: false + primaryKey: true + required: false + - name: "description" + logicalType: "string" physicalType: "varchar[400]" description: "Description of the product." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productdescription/description" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productdescription/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productdescription/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "productdocument" + criticalDataElement: false + primaryKey: false + required: false + - name: "productdocument" physicalName: "productdocument" + physicalType: "table" description: "Cross-reference table mapping products to related product documents." - columns: - - column: "productid" - logicalType: "Numeric" + properties: + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Product identification number. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productdocument/productid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productdocument/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "documentnode" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "documentnode" + logicalType: "string" physicalType: "varchar[2147483647]" description: "Document identification number. Foreign key to Document.DocumentNode." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productdocument/documentnode" - criticalDataElementStatus: false - primary: true - nullable: false - - table: "productinventory" + criticalDataElement: false + primaryKey: true + required: false + - name: "productinventory" physicalName: "productinventory" + physicalType: "table" description: "Product inventory information." - columns: - - column: "productid" - logicalType: "Numeric" + properties: + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Product identification number. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productinventory/productid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "locationid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "locationid" + logicalType: "number" physicalType: "int2" description: "Inventory location identification number. Foreign key to Location.LocationID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productinventory/locationid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "shelf" - logicalType: "String" + criticalDataElement: false + primaryKey: true + required: false + - name: "shelf" + logicalType: "string" physicalType: "varchar[10]" description: "Storage compartment within an inventory location." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productinventory/shelf" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "bin" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "bin" + logicalType: "number" physicalType: "int2" description: "Storage container on a shelf in an inventory location." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productinventory/bin" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "quantity" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "quantity" + logicalType: "number" physicalType: "int2" description: "Quantity of products in the inventory location." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productinventory/quantity" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productinventory/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productinventory/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "productlistpricehistory" + criticalDataElement: false + primaryKey: false + required: false + - name: "productlistpricehistory" physicalName: "productlistpricehistory" + physicalType: "table" description: "Changes in the list price of a product over time." - columns: - - column: "productid" - logicalType: "Numeric" + properties: + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Product identification number. Foreign key to Product.ProductID" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productlistpricehistory/productid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "startdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "startdate" + logicalType: "date" physicalType: "timestamp" description: "List price start date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productlistpricehistory/startdate" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "enddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "enddate" + logicalType: "date" physicalType: "timestamp" description: "List price end date" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productlistpricehistory/enddate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "listprice" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "listprice" + logicalType: "number" physicalType: "numeric" description: "Product list price." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productlistpricehistory/listprice" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productlistpricehistory/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "productmodel" + criticalDataElement: false + primaryKey: false + required: false + - name: "productmodel" physicalName: "productmodel" + physicalType: "table" description: "Product model classification." - columns: - - column: "productmodelid" - logicalType: "Numeric" + properties: + - name: "productmodelid" + logicalType: "number" physicalType: "serial" description: "Primary key for ProductModel records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productmodel/productmodelid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Product model description." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productmodel/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "catalogdescription" - logicalType: "XML" + criticalDataElement: false + primaryKey: false + required: false + - name: "catalogdescription" + logicalType: "string" physicalType: "xml" description: "Detailed product catalog information in xml format." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productmodel/catalogdescription" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "instructions" - logicalType: "XML" + criticalDataElement: false + primaryKey: false + required: false + - name: "instructions" + logicalType: "string" physicalType: "xml" description: "Manufacturing instructions in xml format." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productmodel/instructions" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productmodel/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productmodel/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "productmodelillustration" + criticalDataElement: false + primaryKey: false + required: false + - name: "productmodelillustration" physicalName: "productmodelillustration" + physicalType: "table" description: "Cross-reference table mapping product models and illustrations." - columns: - - column: "productmodelid" - logicalType: "Numeric" + properties: + - name: "productmodelid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to ProductModel.ProductModelID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productmodelillustration/productmodelid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "illustrationid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "illustrationid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to Illustration.IllustrationID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productmodelillustration/illustrationid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productmodelillustration/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "productmodelproductdescriptionculture" + criticalDataElement: false + primaryKey: false + required: false + - name: "productmodelproductdescriptionculture" physicalName: "productmodelproductdescriptionculture" + physicalType: "table" description: "Cross-reference table mapping product descriptions and the language\ \ the description is written in." - columns: - - column: "productmodelid" - logicalType: "Numeric" + properties: + - name: "productmodelid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to ProductModel.ProductModelID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productmodelproductdescriptionculture/productmodelid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "productdescriptionid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "productdescriptionid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to ProductDescription.ProductDescriptionID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productmodelproductdescriptionculture/productdescriptionid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "cultureid" - logicalType: "String" + criticalDataElement: false + primaryKey: true + required: false + - name: "cultureid" + logicalType: "string" physicalType: "bpchar" description: "Culture identification number. Foreign key to Culture.CultureID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productmodelproductdescriptionculture/cultureid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productmodelproductdescriptionculture/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "productphoto" + criticalDataElement: false + primaryKey: false + required: false + - name: "productphoto" physicalName: "productphoto" + physicalType: "table" description: "Product images." - columns: - - column: "productphotoid" - logicalType: "Numeric" + properties: + - name: "productphotoid" + logicalType: "number" physicalType: "serial" description: "Primary key for ProductPhoto records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productphoto/productphotoid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "thumbnailphoto" - logicalType: "Binary" + criticalDataElement: false + primaryKey: true + required: false + - name: "thumbnailphoto" + logicalType: "string" physicalType: "bytea" description: "Small image of the product." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productphoto/thumbnailphoto" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "thumbnailphotofilename" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "thumbnailphotofilename" + logicalType: "string" physicalType: "varchar[50]" description: "Small image file name." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productphoto/thumbnailphotofilename" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "largephoto" - logicalType: "Binary" + criticalDataElement: false + primaryKey: false + required: false + - name: "largephoto" + logicalType: "string" physicalType: "bytea" description: "Large image of the product." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productphoto/largephoto" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "largephotofilename" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "largephotofilename" + logicalType: "string" physicalType: "varchar[50]" description: "Large image file name." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productphoto/largephotofilename" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productphoto/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "productproductphoto" + criticalDataElement: false + primaryKey: false + required: false + - name: "productproductphoto" physicalName: "productproductphoto" + physicalType: "table" description: "Cross-reference table mapping products and product photos." - columns: - - column: "productid" - logicalType: "Numeric" + properties: + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Product identification number. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productproductphoto/productid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "productphotoid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "productphotoid" + logicalType: "number" physicalType: "int4" description: "Product photo identification number. Foreign key to ProductPhoto.ProductPhotoID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productproductphoto/productphotoid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "primary" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "primary" + logicalType: "object" physicalType: "Flag" description: "0 = Photo is not the principal image. 1 = Photo is the principal\ \ image." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productproductphoto/primary" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productproductphoto/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "productreview" + criticalDataElement: false + primaryKey: false + required: false + - name: "productreview" physicalName: "productreview" + physicalType: "table" description: "Customer reviews of products they have purchased." - columns: - - column: "productreviewid" - logicalType: "Numeric" + properties: + - name: "productreviewid" + logicalType: "number" physicalType: "serial" description: "Primary key for ProductReview records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productreview/productreviewid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "productid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Product identification number. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productreview/productid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "reviewername" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "reviewername" + logicalType: "object" physicalType: "Name" description: "Name of the reviewer." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productreview/reviewername" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "reviewdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "reviewdate" + logicalType: "date" physicalType: "timestamp" description: "Date review was submitted." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productreview/reviewdate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "emailaddress" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "emailaddress" + logicalType: "string" physicalType: "varchar[50]" description: "Reviewer's e-mail address." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productreview/emailaddress" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rating" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "rating" + logicalType: "number" physicalType: "int4" description: "Product rating given by the reviewer. Scale is 1 to 5 with 5 as\ \ the highest rating." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productreview/rating" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "comments" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "comments" + logicalType: "string" physicalType: "varchar[3850]" description: "Reviewer's comments" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productreview/comments" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productreview/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "productsubcategory" + criticalDataElement: false + primaryKey: false + required: false + - name: "productsubcategory" physicalName: "productsubcategory" + physicalType: "table" description: "Product subcategories. See ProductCategory table." - columns: - - column: "productsubcategoryid" - logicalType: "Numeric" + properties: + - name: "productsubcategoryid" + logicalType: "number" physicalType: "serial" description: "Primary key for ProductSubcategory records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productsubcategory/productsubcategoryid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "productcategoryid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "productcategoryid" + logicalType: "number" physicalType: "int4" description: "Product category identification number. Foreign key to ProductCategory.ProductCategoryID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productsubcategory/productcategoryid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Subcategory description." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productsubcategory/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productsubcategory/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productsubcategory/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "scrapreason" + criticalDataElement: false + primaryKey: false + required: false + - name: "scrapreason" physicalName: "scrapreason" + physicalType: "table" description: "Manufacturing failure reasons lookup table." - columns: - - column: "scrapreasonid" - logicalType: "Numeric" + properties: + - name: "scrapreasonid" + logicalType: "number" physicalType: "serial" description: "Primary key for ScrapReason records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/scrapreason/scrapreasonid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Failure description." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/scrapreason/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/scrapreason/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "transactionhistory" + criticalDataElement: false + primaryKey: false + required: false + - name: "transactionhistory" physicalName: "transactionhistory" + physicalType: "table" description: "Record of each purchase order, sales order, or work order transaction\ \ year to date." - columns: - - column: "transactionid" - logicalType: "Numeric" + properties: + - name: "transactionid" + logicalType: "number" physicalType: "serial" description: "Primary key for TransactionHistory records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistory/transactionid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "productid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Product identification number. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistory/productid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "referenceorderid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "referenceorderid" + logicalType: "number" physicalType: "int4" description: "Purchase order, sales order, or work order identification number." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistory/referenceorderid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "referenceorderlineid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "referenceorderlineid" + logicalType: "number" physicalType: "int4" description: "Line number associated with the purchase order, sales order, or\ \ work order." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistory/referenceorderlineid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "transactiondate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "transactiondate" + logicalType: "date" physicalType: "timestamp" description: "Date and time of the transaction." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistory/transactiondate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "transactiontype" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "transactiontype" + logicalType: "string" physicalType: "bpchar" description: "W = WorkOrder, S = SalesOrder, P = PurchaseOrder" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistory/transactiontype" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "quantity" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "quantity" + logicalType: "number" physicalType: "int4" description: "Product quantity." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistory/quantity" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "actualcost" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "actualcost" + logicalType: "number" physicalType: "numeric" description: "Product cost." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistory/actualcost" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistory/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "transactionhistoryarchive" + criticalDataElement: false + primaryKey: false + required: false + - name: "transactionhistoryarchive" physicalName: "transactionhistoryarchive" + physicalType: "table" description: "Transactions for previous years." - columns: - - column: "transactionid" - logicalType: "Numeric" + properties: + - name: "transactionid" + logicalType: "number" physicalType: "int4" description: "Primary key for TransactionHistoryArchive records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistoryarchive/transactionid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "productid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Product identification number. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistoryarchive/productid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "referenceorderid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "referenceorderid" + logicalType: "number" physicalType: "int4" description: "Purchase order, sales order, or work order identification number." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistoryarchive/referenceorderid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "referenceorderlineid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "referenceorderlineid" + logicalType: "number" physicalType: "int4" description: "Line number associated with the purchase order, sales order, or\ \ work order." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistoryarchive/referenceorderlineid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "transactiondate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "transactiondate" + logicalType: "date" physicalType: "timestamp" description: "Date and time of the transaction." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistoryarchive/transactiondate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "transactiontype" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "transactiontype" + logicalType: "string" physicalType: "bpchar" description: "W = Work Order, S = Sales Order, P = Purchase Order" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistoryarchive/transactiontype" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "quantity" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "quantity" + logicalType: "number" physicalType: "int4" description: "Product quantity." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistoryarchive/quantity" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "actualcost" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "actualcost" + logicalType: "number" physicalType: "numeric" description: "Product cost." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistoryarchive/actualcost" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/transactionhistoryarchive/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "unitmeasure" + criticalDataElement: false + primaryKey: false + required: false + - name: "unitmeasure" physicalName: "unitmeasure" + physicalType: "table" description: "Unit of measure lookup table." - columns: - - column: "unitmeasurecode" - logicalType: "String" + properties: + - name: "unitmeasurecode" + logicalType: "string" physicalType: "bpchar" description: "Primary key." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/unitmeasure/unitmeasurecode" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Unit of measure description." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/unitmeasure/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/unitmeasure/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "workorder" + criticalDataElement: false + primaryKey: false + required: false + - name: "workorder" physicalName: "workorder" + physicalType: "table" description: "Manufacturing work orders." - columns: - - column: "workorderid" - logicalType: "Numeric" + properties: + - name: "workorderid" + logicalType: "number" physicalType: "serial" description: "Primary key for WorkOrder records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorder/workorderid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "productid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Product identification number. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorder/productid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "orderqty" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "orderqty" + logicalType: "number" physicalType: "int4" description: "Product quantity to build." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorder/orderqty" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "scrappedqty" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "scrappedqty" + logicalType: "number" physicalType: "int2" description: "Quantity that failed inspection." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorder/scrappedqty" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "startdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "startdate" + logicalType: "date" physicalType: "timestamp" description: "Work order start date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorder/startdate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "enddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "enddate" + logicalType: "date" physicalType: "timestamp" description: "Work order end date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorder/enddate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "duedate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "duedate" + logicalType: "date" physicalType: "timestamp" description: "Work order due date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorder/duedate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "scrapreasonid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "scrapreasonid" + logicalType: "number" physicalType: "int2" description: "Reason for inspection failure." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorder/scrapreasonid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorder/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "workorderrouting" + criticalDataElement: false + primaryKey: false + required: false + - name: "workorderrouting" physicalName: "workorderrouting" + physicalType: "table" description: "Work order details." - columns: - - column: "workorderid" - logicalType: "Numeric" + properties: + - name: "workorderid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to WorkOrder.WorkOrderID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorderrouting/workorderid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "productid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorderrouting/productid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "operationsequence" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "operationsequence" + logicalType: "number" physicalType: "int2" description: "Primary key. Indicates the manufacturing process sequence." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorderrouting/operationsequence" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "locationid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "locationid" + logicalType: "number" physicalType: "int2" description: "Manufacturing location where the part is processed. Foreign key\ \ to Location.LocationID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorderrouting/locationid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "scheduledstartdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "scheduledstartdate" + logicalType: "date" physicalType: "timestamp" description: "Planned manufacturing start date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorderrouting/scheduledstartdate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "scheduledenddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "scheduledenddate" + logicalType: "date" physicalType: "timestamp" description: "Planned manufacturing end date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorderrouting/scheduledenddate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "actualstartdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "actualstartdate" + logicalType: "date" physicalType: "timestamp" description: "Actual start date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorderrouting/actualstartdate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "actualenddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "actualenddate" + logicalType: "date" physicalType: "timestamp" description: "Actual end date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorderrouting/actualenddate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "actualresourcehrs" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "actualresourcehrs" + logicalType: "number" physicalType: "numeric" description: "Number of manufacturing hours used." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorderrouting/actualresourcehrs" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "plannedcost" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "plannedcost" + logicalType: "number" physicalType: "numeric" description: "Estimated manufacturing cost." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorderrouting/plannedcost" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "actualcost" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "actualcost" + logicalType: "number" physicalType: "numeric" description: "Actual manufacturing cost." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorderrouting/actualcost" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/workorderrouting/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "productvendor" + criticalDataElement: false + primaryKey: false + required: false + - name: "productvendor" physicalName: "productvendor" + physicalType: "table" description: "Cross-reference table mapping vendors with the products they supply." - columns: - - column: "productid" - logicalType: "Numeric" + properties: + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productvendor/productid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "businessentityid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to Vendor.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productvendor/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "averageleadtime" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "averageleadtime" + logicalType: "number" physicalType: "int4" description: "The average span of time (in days) between placing an order with\ \ the vendor and receiving the purchased product." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productvendor/averageleadtime" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "standardprice" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "standardprice" + logicalType: "number" physicalType: "numeric" description: "The vendor's usual selling price." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productvendor/standardprice" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "lastreceiptcost" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "lastreceiptcost" + logicalType: "number" physicalType: "numeric" description: "The selling price when last purchased." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productvendor/lastreceiptcost" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "lastreceiptdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "lastreceiptdate" + logicalType: "date" physicalType: "timestamp" description: "Date the product was last received by the vendor." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productvendor/lastreceiptdate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "minorderqty" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "minorderqty" + logicalType: "number" physicalType: "int4" description: "The maximum quantity that should be ordered." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productvendor/minorderqty" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "maxorderqty" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "maxorderqty" + logicalType: "number" physicalType: "int4" description: "The minimum quantity that should be ordered." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productvendor/maxorderqty" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "onorderqty" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "onorderqty" + logicalType: "number" physicalType: "int4" description: "The quantity currently on order." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productvendor/onorderqty" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "unitmeasurecode" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "unitmeasurecode" + logicalType: "string" physicalType: "bpchar" description: "The product's unit of measure." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productvendor/unitmeasurecode" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/productvendor/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "purchaseorderdetail" + criticalDataElement: false + primaryKey: false + required: false + - name: "purchaseorderdetail" physicalName: "purchaseorderdetail" + physicalType: "table" description: "Individual products associated with a specific purchase order. See\ \ PurchaseOrderHeader." - columns: - - column: "purchaseorderid" - logicalType: "Numeric" + properties: + - name: "purchaseorderid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to PurchaseOrderHeader.PurchaseOrderID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderdetail/purchaseorderid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "purchaseorderdetailid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "purchaseorderdetailid" + logicalType: "number" physicalType: "serial" description: "Primary key. One line number per purchased product." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderdetail/purchaseorderdetailid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "duedate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "duedate" + logicalType: "date" physicalType: "timestamp" description: "Date the product is expected to be received." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderdetail/duedate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "orderqty" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "orderqty" + logicalType: "number" physicalType: "int2" description: "Quantity ordered." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderdetail/orderqty" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "productid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Product identification number. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderdetail/productid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "unitprice" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "unitprice" + logicalType: "number" physicalType: "numeric" description: "Vendor's selling price of a single product." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderdetail/unitprice" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "receivedqty" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "receivedqty" + logicalType: "number" physicalType: "numeric" description: "Quantity actually received from the vendor." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderdetail/receivedqty" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rejectedqty" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "rejectedqty" + logicalType: "number" physicalType: "numeric" description: "Quantity rejected during inspection." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderdetail/rejectedqty" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderdetail/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "purchaseorderheader" + criticalDataElement: false + primaryKey: false + required: false + - name: "purchaseorderheader" physicalName: "purchaseorderheader" + physicalType: "table" description: "General purchase order information. See PurchaseOrderDetail." - columns: - - column: "purchaseorderid" - logicalType: "Numeric" + properties: + - name: "purchaseorderid" + logicalType: "number" physicalType: "serial" description: "Primary key." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderheader/purchaseorderid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "revisionnumber" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "revisionnumber" + logicalType: "number" physicalType: "int2" description: "Incremental number to track changes to the purchase order over\ \ time." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderheader/revisionnumber" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "status" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "status" + logicalType: "number" physicalType: "int2" description: "Order current status. 1 = Pending; 2 = Approved; 3 = Rejected;\ \ 4 = Complete" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderheader/status" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "employeeid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "employeeid" + logicalType: "number" physicalType: "int4" description: "Employee who created the purchase order. Foreign key to Employee.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderheader/employeeid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "vendorid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "vendorid" + logicalType: "number" physicalType: "int4" description: "Vendor with whom the purchase order is placed. Foreign key to\ \ Vendor.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderheader/vendorid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "shipmethodid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "shipmethodid" + logicalType: "number" physicalType: "int4" description: "Shipping method. Foreign key to ShipMethod.ShipMethodID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderheader/shipmethodid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "orderdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "orderdate" + logicalType: "date" physicalType: "timestamp" description: "Purchase order creation date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderheader/orderdate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "shipdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "shipdate" + logicalType: "date" physicalType: "timestamp" description: "Estimated shipment date from the vendor." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderheader/shipdate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "subtotal" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "subtotal" + logicalType: "number" physicalType: "numeric" description: "Purchase order subtotal. Computed as SUM(PurchaseOrderDetail.LineTotal)for\ \ the appropriate PurchaseOrderID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderheader/subtotal" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "taxamt" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "taxamt" + logicalType: "number" physicalType: "numeric" description: "Tax amount." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderheader/taxamt" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "freight" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "freight" + logicalType: "number" physicalType: "numeric" description: "Shipping cost." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderheader/freight" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/purchaseorderheader/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "shipmethod" + criticalDataElement: false + primaryKey: false + required: false + - name: "shipmethod" physicalName: "shipmethod" + physicalType: "table" description: "Shipping company lookup table." - columns: - - column: "shipmethodid" - logicalType: "Numeric" + properties: + - name: "shipmethodid" + logicalType: "number" physicalType: "serial" description: "Primary key for ShipMethod records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shipmethod/shipmethodid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Shipping company name." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shipmethod/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "shipbase" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "shipbase" + logicalType: "number" physicalType: "numeric" description: "Minimum shipping charge." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shipmethod/shipbase" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "shiprate" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "shiprate" + logicalType: "number" physicalType: "numeric" description: "Shipping charge per pound." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shipmethod/shiprate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shipmethod/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shipmethod/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "vendor" + criticalDataElement: false + primaryKey: false + required: false + - name: "vendor" physicalName: "vendor" + physicalType: "table" description: "Companies from whom Adventure Works Cycles purchases parts or other\ \ goods." - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Primary key for Vendor records. Foreign key to BusinessEntity.BusinessEntityID" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/vendor/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "accountnumber" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "accountnumber" + logicalType: "object" physicalType: "AccountNumber" description: "Vendor account (identification) number." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/vendor/accountnumber" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Company name." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/vendor/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "creditrating" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "creditrating" + logicalType: "number" physicalType: "int2" description: "1 = Superior, 2 = Excellent, 3 = Above average, 4 = Average, 5\ \ = Below average" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/vendor/creditrating" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "preferredvendorstatus" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "preferredvendorstatus" + logicalType: "object" physicalType: "Flag" description: "0 = Do not use if another vendor is available. 1 = Preferred over\ \ other vendors supplying the same product." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/vendor/preferredvendorstatus" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "activeflag" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "activeflag" + logicalType: "object" physicalType: "Flag" description: "0 = Vendor no longer used. 1 = Vendor is actively used." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/vendor/activeflag" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "purchasingwebserviceurl" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "purchasingwebserviceurl" + logicalType: "string" physicalType: "varchar[1024]" description: "Vendor URL." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/vendor/purchasingwebserviceurl" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/vendor/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "countryregioncurrency" + criticalDataElement: false + primaryKey: false + required: false + - name: "countryregioncurrency" physicalName: "countryregioncurrency" + physicalType: "table" description: "Cross-reference table mapping ISO currency codes to a country or\ \ region." - columns: - - column: "countryregioncode" - logicalType: "String" + properties: + - name: "countryregioncode" + logicalType: "string" physicalType: "varchar[3]" description: "ISO code for countries and regions. Foreign key to CountryRegion.CountryRegionCode." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/countryregioncurrency/countryregioncode" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "currencycode" - logicalType: "String" + criticalDataElement: false + primaryKey: true + required: false + - name: "currencycode" + logicalType: "string" physicalType: "bpchar" description: "ISO standard currency code. Foreign key to Currency.CurrencyCode." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/countryregioncurrency/currencycode" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/countryregioncurrency/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "creditcard" + criticalDataElement: false + primaryKey: false + required: false + - name: "creditcard" physicalName: "creditcard" + physicalType: "table" description: "Customer credit card information." - columns: - - column: "creditcardid" - logicalType: "Numeric" + properties: + - name: "creditcardid" + logicalType: "number" physicalType: "serial" description: "Primary key for CreditCard records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/creditcard/creditcardid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "cardtype" - logicalType: "String" + criticalDataElement: false + primaryKey: true + required: false + - name: "cardtype" + logicalType: "string" physicalType: "varchar[50]" description: "Credit card name." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/creditcard/cardtype" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "cardnumber" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "cardnumber" + logicalType: "string" physicalType: "varchar[25]" description: "Credit card number." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/creditcard/cardnumber" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "expmonth" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "expmonth" + logicalType: "number" physicalType: "int2" description: "Credit card expiration month." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/creditcard/expmonth" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "expyear" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "expyear" + logicalType: "number" physicalType: "int2" description: "Credit card expiration year." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/creditcard/expyear" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/creditcard/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "currency" + criticalDataElement: false + primaryKey: false + required: false + - name: "currency" physicalName: "currency" + physicalType: "table" description: "Lookup table containing standard ISO currencies." - columns: - - column: "currencycode" - logicalType: "String" + properties: + - name: "currencycode" + logicalType: "string" physicalType: "bpchar" description: "The ISO code for the Currency." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/currency/currencycode" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Currency name." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/currency/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/currency/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "currencyrate" + criticalDataElement: false + primaryKey: false + required: false + - name: "currencyrate" physicalName: "currencyrate" + physicalType: "table" description: "Currency exchange rates." - columns: - - column: "currencyrateid" - logicalType: "Numeric" + properties: + - name: "currencyrateid" + logicalType: "number" physicalType: "serial" description: "Primary key for CurrencyRate records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/currencyrate/currencyrateid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "currencyratedate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "currencyratedate" + logicalType: "date" physicalType: "timestamp" description: "Date and time the exchange rate was obtained." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/currencyrate/currencyratedate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "fromcurrencycode" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "fromcurrencycode" + logicalType: "string" physicalType: "bpchar" description: "Exchange rate was converted from this currency code." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/currencyrate/fromcurrencycode" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "tocurrencycode" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "tocurrencycode" + logicalType: "string" physicalType: "bpchar" description: "Exchange rate was converted to this currency code." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/currencyrate/tocurrencycode" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "averagerate" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "averagerate" + logicalType: "number" physicalType: "numeric" description: "Average exchange rate for the day." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/currencyrate/averagerate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "endofdayrate" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "endofdayrate" + logicalType: "number" physicalType: "numeric" description: "Final exchange rate for the day." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/currencyrate/endofdayrate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/currencyrate/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "customer" + criticalDataElement: false + primaryKey: false + required: false + - name: "customer" physicalName: "customer" + physicalType: "table" description: "Current customer information. Also see the Person and Store tables." - columns: - - column: "customerid" - logicalType: "Numeric" + properties: + - name: "customerid" + logicalType: "number" physicalType: "serial" description: "Primary key." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/customer/customerid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "personid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "personid" + logicalType: "number" physicalType: "int4" description: "Foreign key to Person.BusinessEntityID" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/customer/personid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "storeid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "storeid" + logicalType: "number" physicalType: "int4" description: "Foreign key to Store.BusinessEntityID" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/customer/storeid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "territoryid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "territoryid" + logicalType: "number" physicalType: "int4" description: "ID of the territory in which the customer is located. Foreign\ \ key to SalesTerritory.SalesTerritoryID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/customer/territoryid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/customer/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/customer/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "personcreditcard" + criticalDataElement: false + primaryKey: false + required: false + - name: "personcreditcard" physicalName: "personcreditcard" + physicalType: "table" description: "Cross-reference table mapping people to their credit card information\ \ in the CreditCard table." - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Business entity identification number. Foreign key to Person.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/personcreditcard/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "creditcardid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "creditcardid" + logicalType: "number" physicalType: "int4" description: "Credit card identification number. Foreign key to CreditCard.CreditCardID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/personcreditcard/creditcardid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/personcreditcard/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "salesorderdetail" + criticalDataElement: false + primaryKey: false + required: false + - name: "salesorderdetail" physicalName: "salesorderdetail" + physicalType: "table" description: "Individual products associated with a specific sales order. See\ \ SalesOrderHeader." - columns: - - column: "salesorderid" - logicalType: "Numeric" + properties: + - name: "salesorderid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to SalesOrderHeader.SalesOrderID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderdetail/salesorderid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "salesorderdetailid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "salesorderdetailid" + logicalType: "number" physicalType: "serial" description: "Primary key. One incremental unique number per product sold." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderdetail/salesorderdetailid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "carriertrackingnumber" - logicalType: "String" + criticalDataElement: false + primaryKey: true + required: false + - name: "carriertrackingnumber" + logicalType: "string" physicalType: "varchar[25]" description: "Shipment tracking number supplied by the shipper." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderdetail/carriertrackingnumber" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "orderqty" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "orderqty" + logicalType: "number" physicalType: "int2" description: "Quantity ordered per product." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderdetail/orderqty" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "productid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Product sold to customer. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderdetail/productid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "specialofferid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "specialofferid" + logicalType: "number" physicalType: "int4" description: "Promotional code. Foreign key to SpecialOffer.SpecialOfferID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderdetail/specialofferid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "unitprice" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "unitprice" + logicalType: "number" physicalType: "numeric" description: "Selling price of a single product." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderdetail/unitprice" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "unitpricediscount" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "unitpricediscount" + logicalType: "number" physicalType: "numeric" description: "Discount amount." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderdetail/unitpricediscount" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderdetail/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderdetail/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "salesorderheader" + criticalDataElement: false + primaryKey: false + required: false + - name: "salesorderheader" physicalName: "salesorderheader" + physicalType: "table" description: "General sales order information." - columns: - - column: "salesorderid" - logicalType: "Numeric" + properties: + - name: "salesorderid" + logicalType: "number" physicalType: "serial" description: "Primary key." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/salesorderid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "revisionnumber" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "revisionnumber" + logicalType: "number" physicalType: "int2" description: "Incremental number to track changes to the sales order over time." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/revisionnumber" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "orderdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "orderdate" + logicalType: "date" physicalType: "timestamp" description: "Dates the sales order was created." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/orderdate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "duedate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "duedate" + logicalType: "date" physicalType: "timestamp" description: "Date the order is due to the customer." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/duedate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "shipdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "shipdate" + logicalType: "date" physicalType: "timestamp" description: "Date the order was shipped to the customer." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/shipdate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "status" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "status" + logicalType: "number" physicalType: "int2" description: "Order current status. 1 = In process; 2 = Approved; 3 = Backordered;\ \ 4 = Rejected; 5 = Shipped; 6 = Cancelled" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/status" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "onlineorderflag" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "onlineorderflag" + logicalType: "object" physicalType: "Flag" description: "0 = Order placed by sales person. 1 = Order placed online by customer." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/onlineorderflag" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "purchaseordernumber" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "purchaseordernumber" + logicalType: "object" physicalType: "OrderNumber" description: "Customer purchase order number reference." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/purchaseordernumber" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "accountnumber" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "accountnumber" + logicalType: "object" physicalType: "AccountNumber" description: "Financial accounting number reference." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/accountnumber" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "customerid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "customerid" + logicalType: "number" physicalType: "int4" description: "Customer identification number. Foreign key to Customer.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/customerid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "salespersonid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "salespersonid" + logicalType: "number" physicalType: "int4" description: "Sales person who created the sales order. Foreign key to SalesPerson.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/salespersonid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "territoryid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "territoryid" + logicalType: "number" physicalType: "int4" description: "Territory in which the sale was made. Foreign key to SalesTerritory.SalesTerritoryID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/territoryid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "billtoaddressid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "billtoaddressid" + logicalType: "number" physicalType: "int4" description: "Customer billing address. Foreign key to Address.AddressID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/billtoaddressid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "shiptoaddressid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "shiptoaddressid" + logicalType: "number" physicalType: "int4" description: "Customer shipping address. Foreign key to Address.AddressID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/shiptoaddressid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "shipmethodid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "shipmethodid" + logicalType: "number" physicalType: "int4" description: "Shipping method. Foreign key to ShipMethod.ShipMethodID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/shipmethodid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "creditcardid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "creditcardid" + logicalType: "number" physicalType: "int4" description: "Credit card identification number. Foreign key to CreditCard.CreditCardID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/creditcardid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "creditcardapprovalcode" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "creditcardapprovalcode" + logicalType: "string" physicalType: "varchar[15]" description: "Approval code provided by the credit card company." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/creditcardapprovalcode" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "currencyrateid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "currencyrateid" + logicalType: "number" physicalType: "int4" description: "Currency exchange rate used. Foreign key to CurrencyRate.CurrencyRateID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/currencyrateid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "subtotal" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "subtotal" + logicalType: "number" physicalType: "numeric" description: "Sales subtotal. Computed as SUM(SalesOrderDetail.LineTotal)for\ \ the appropriate SalesOrderID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/subtotal" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "taxamt" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "taxamt" + logicalType: "number" physicalType: "numeric" description: "Tax amount." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/taxamt" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "freight" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "freight" + logicalType: "number" physicalType: "numeric" description: "Shipping cost." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/freight" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "totaldue" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "totaldue" + logicalType: "number" physicalType: "numeric" description: "Total due from customer. Computed as Subtotal + TaxAmt + Freight." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/totaldue" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "comment" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "comment" + logicalType: "string" physicalType: "varchar[128]" description: "Sales representative comments." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/comment" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheader/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "salesorderheadersalesreason" + criticalDataElement: false + primaryKey: false + required: false + - name: "salesorderheadersalesreason" physicalName: "salesorderheadersalesreason" + physicalType: "table" description: "Cross-reference table mapping sales orders to sales reason codes." - columns: - - column: "salesorderid" - logicalType: "Numeric" + properties: + - name: "salesorderid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to SalesOrderHeader.SalesOrderID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheadersalesreason/salesorderid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "salesreasonid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "salesreasonid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to SalesReason.SalesReasonID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheadersalesreason/salesreasonid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesorderheadersalesreason/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "salesperson" + criticalDataElement: false + primaryKey: false + required: false + - name: "salesperson" physicalName: "salesperson" + physicalType: "table" description: "Sales representative current information." - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Primary key for SalesPerson records. Foreign key to Employee.BusinessEntityID" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesperson/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "territoryid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "territoryid" + logicalType: "number" physicalType: "int4" description: "Territory currently assigned to. Foreign key to SalesTerritory.SalesTerritoryID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesperson/territoryid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "salesquota" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "salesquota" + logicalType: "number" physicalType: "numeric" description: "Projected yearly sales." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesperson/salesquota" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "bonus" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "bonus" + logicalType: "number" physicalType: "numeric" description: "Bonus due if quota is met." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesperson/bonus" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "commissionpct" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "commissionpct" + logicalType: "number" physicalType: "numeric" description: "Commision percent received per sale." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesperson/commissionpct" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "salesytd" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "salesytd" + logicalType: "number" physicalType: "numeric" description: "Sales total year to date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesperson/salesytd" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "saleslastyear" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "saleslastyear" + logicalType: "number" physicalType: "numeric" description: "Sales total of previous year." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesperson/saleslastyear" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesperson/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesperson/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "salespersonquotahistory" + criticalDataElement: false + primaryKey: false + required: false + - name: "salespersonquotahistory" physicalName: "salespersonquotahistory" + physicalType: "table" description: "Sales performance tracking." - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Sales person identification number. Foreign key to SalesPerson.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salespersonquotahistory/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "quotadate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "quotadate" + logicalType: "date" physicalType: "timestamp" description: "Sales quota date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salespersonquotahistory/quotadate" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "salesquota" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "salesquota" + logicalType: "number" physicalType: "numeric" description: "Sales quota amount." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salespersonquotahistory/salesquota" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salespersonquotahistory/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salespersonquotahistory/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "salesreason" + criticalDataElement: false + primaryKey: false + required: false + - name: "salesreason" physicalName: "salesreason" + physicalType: "table" description: "Lookup table of customer purchase reasons." - columns: - - column: "salesreasonid" - logicalType: "Numeric" + properties: + - name: "salesreasonid" + logicalType: "number" physicalType: "serial" description: "Primary key for SalesReason records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesreason/salesreasonid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Sales reason description." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesreason/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "reasontype" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "reasontype" + logicalType: "object" physicalType: "Name" description: "Category the sales reason belongs to." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesreason/reasontype" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesreason/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "salestaxrate" + criticalDataElement: false + primaryKey: false + required: false + - name: "salestaxrate" physicalName: "salestaxrate" + physicalType: "table" description: "Tax rate lookup table." - columns: - - column: "salestaxrateid" - logicalType: "Numeric" + properties: + - name: "salestaxrateid" + logicalType: "number" physicalType: "serial" description: "Primary key for SalesTaxRate records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salestaxrate/salestaxrateid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "stateprovinceid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "stateprovinceid" + logicalType: "number" physicalType: "int4" description: "State, province, or country/region the sales tax applies to." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salestaxrate/stateprovinceid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "taxtype" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "taxtype" + logicalType: "number" physicalType: "int2" description: "1 = Tax applied to retail transactions, 2 = Tax applied to wholesale\ \ transactions, 3 = Tax applied to all sales (retail and wholesale) transactions." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salestaxrate/taxtype" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "taxrate" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "taxrate" + logicalType: "number" physicalType: "numeric" description: "Tax rate amount." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salestaxrate/taxrate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: false + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Tax rate description." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salestaxrate/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salestaxrate/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salestaxrate/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "salesterritory" + criticalDataElement: false + primaryKey: false + required: false + - name: "salesterritory" physicalName: "salesterritory" + physicalType: "table" description: "Sales territory lookup table." - columns: - - column: "territoryid" - logicalType: "Numeric" + properties: + - name: "territoryid" + logicalType: "number" physicalType: "serial" description: "Primary key for SalesTerritory records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritory/territoryid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Sales territory description" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritory/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "countryregioncode" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "countryregioncode" + logicalType: "string" physicalType: "varchar[3]" description: "ISO standard country or region code. Foreign key to CountryRegion.CountryRegionCode." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritory/countryregioncode" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "group" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "group" + logicalType: "string" physicalType: "varchar[50]" description: "Geographic area to which the sales territory belong." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritory/group" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "salesytd" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "salesytd" + logicalType: "number" physicalType: "numeric" description: "Sales in the territory year to date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritory/salesytd" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "saleslastyear" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "saleslastyear" + logicalType: "number" physicalType: "numeric" description: "Sales in the territory the previous year." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritory/saleslastyear" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "costytd" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "costytd" + logicalType: "number" physicalType: "numeric" description: "Business costs in the territory year to date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritory/costytd" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "costlastyear" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "costlastyear" + logicalType: "number" physicalType: "numeric" description: "Business costs in the territory the previous year." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritory/costlastyear" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritory/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritory/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "salesterritoryhistory" + criticalDataElement: false + primaryKey: false + required: false + - name: "salesterritoryhistory" physicalName: "salesterritoryhistory" + physicalType: "table" description: "Sales representative transfers to other sales territories." - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Primary key. The sales rep. Foreign key to SalesPerson.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritoryhistory/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "territoryid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "territoryid" + logicalType: "number" physicalType: "int4" description: "Primary key. Territory identification number. Foreign key to SalesTerritory.SalesTerritoryID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritoryhistory/territoryid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "startdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "startdate" + logicalType: "date" physicalType: "timestamp" description: "Primary key. Date the sales representive started work in the territory." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritoryhistory/startdate" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "enddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: true + required: false + - name: "enddate" + logicalType: "date" physicalType: "timestamp" description: "Date the sales representative left work in the territory." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritoryhistory/enddate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritoryhistory/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/salesterritoryhistory/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "shoppingcartitem" + criticalDataElement: false + primaryKey: false + required: false + - name: "shoppingcartitem" physicalName: "shoppingcartitem" + physicalType: "table" description: "Contains online customer orders until the order is submitted or\ \ cancelled." - columns: - - column: "shoppingcartitemid" - logicalType: "Numeric" + properties: + - name: "shoppingcartitemid" + logicalType: "number" physicalType: "serial" description: "Primary key for ShoppingCartItem records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shoppingcartitem/shoppingcartitemid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "shoppingcartid" - logicalType: "String" + criticalDataElement: false + primaryKey: true + required: false + - name: "shoppingcartid" + logicalType: "string" physicalType: "varchar[50]" description: "Shopping cart identification number." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shoppingcartitem/shoppingcartid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "quantity" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "quantity" + logicalType: "number" physicalType: "int4" description: "Product quantity ordered." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shoppingcartitem/quantity" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "productid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Product ordered. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shoppingcartitem/productid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "datecreated" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "datecreated" + logicalType: "date" physicalType: "timestamp" description: "Date the time the record was created." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shoppingcartitem/datecreated" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/shoppingcartitem/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "specialoffer" + criticalDataElement: false + primaryKey: false + required: false + - name: "specialoffer" physicalName: "specialoffer" + physicalType: "table" description: "Sale discounts lookup table." - columns: - - column: "specialofferid" - logicalType: "Numeric" + properties: + - name: "specialofferid" + logicalType: "number" physicalType: "serial" description: "Primary key for SpecialOffer records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialoffer/specialofferid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "description" - logicalType: "String" + criticalDataElement: false + primaryKey: true + required: false + - name: "description" + logicalType: "string" physicalType: "varchar[255]" description: "Discount description." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialoffer/description" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "discountpct" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "discountpct" + logicalType: "number" physicalType: "numeric" description: "Discount precentage." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialoffer/discountpct" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "type" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "type" + logicalType: "string" physicalType: "varchar[50]" description: "Discount type category." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialoffer/type" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "category" - logicalType: "String" + criticalDataElement: false + primaryKey: false + required: false + - name: "category" + logicalType: "string" physicalType: "varchar[50]" description: "Group the discount applies to such as Reseller or Customer." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialoffer/category" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "startdate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "startdate" + logicalType: "date" physicalType: "timestamp" description: "Discount start date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialoffer/startdate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "enddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "enddate" + logicalType: "date" physicalType: "timestamp" description: "Discount end date." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialoffer/enddate" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "minqty" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "minqty" + logicalType: "number" physicalType: "int4" description: "Minimum discount percent allowed." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialoffer/minqty" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "maxqty" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "maxqty" + logicalType: "number" physicalType: "int4" description: "Maximum discount percent allowed." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialoffer/maxqty" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialoffer/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialoffer/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "specialofferproduct" + criticalDataElement: false + primaryKey: false + required: false + - name: "specialofferproduct" physicalName: "specialofferproduct" + physicalType: "table" description: "Cross-reference table mapping products to special offer discounts." - columns: - - column: "specialofferid" - logicalType: "Numeric" + properties: + - name: "specialofferid" + logicalType: "number" physicalType: "int4" description: "Primary key for SpecialOfferProduct records." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialofferproduct/specialofferid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "productid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: true + required: false + - name: "productid" + logicalType: "number" physicalType: "int4" description: "Product identification number. Foreign key to Product.ProductID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialofferproduct/productid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: true + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialofferproduct/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/specialofferproduct/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false - - table: "store" + criticalDataElement: false + primaryKey: false + required: false + - name: "store" physicalName: "store" + physicalType: "table" description: "Customers (resellers) of Adventure Works products." - columns: - - column: "businessentityid" - logicalType: "Numeric" + properties: + - name: "businessentityid" + logicalType: "number" physicalType: "int4" description: "Primary key. Foreign key to Customer.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/store/businessentityid" - criticalDataElementStatus: false - primary: true - nullable: false - - column: "name" - logicalType: "Object" + criticalDataElement: false + primaryKey: true + required: false + - name: "name" + logicalType: "object" physicalType: "Name" description: "Name of the store." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/store/name" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "salespersonid" - logicalType: "Numeric" + criticalDataElement: false + primaryKey: false + required: false + - name: "salespersonid" + logicalType: "number" physicalType: "int4" description: "ID of the sales person assigned to the customer. Foreign key to\ \ SalesPerson.BusinessEntityID." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/store/salespersonid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "demographics" - logicalType: "XML" + criticalDataElement: false + primaryKey: false + required: false + - name: "demographics" + logicalType: "string" physicalType: "xml" description: "Demographic informationg about the store such as the number of\ \ employees, annual sales and store type." - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/store/demographics" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "rowguid" - logicalType: "UUID" + criticalDataElement: false + primaryKey: false + required: false + - name: "rowguid" + logicalType: "string" physicalType: "uuid" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/store/rowguid" - criticalDataElementStatus: false - primary: false - nullable: false - - column: "modifieddate" - logicalType: "Timestamp" + criticalDataElement: false + primaryKey: false + required: false + - name: "modifieddate" + logicalType: "date" physicalType: "timestamp" - partitionStatus: false - clusterStatus: false + partitioned: false authoritativeDefinitions: - type: "implementation" url: "jdbc:postgresql://localhost:5432/adventureworks/store/modifieddate" - criticalDataElementStatus: false - primary: false - nullable: false -contractCreatedTs: "2023-09-28 20:24:49.331 UTC" + criticalDataElement: false + primaryKey: false + required: false +contractCreatedTs: "2023-09-28T20:24:49.331+00:00" diff --git a/docs/examples/data-types/all-data-types.odcs.yaml b/docs/examples/data-types/all-data-types.odcs.yaml new file mode 100644 index 0000000..beddd6b --- /dev/null +++ b/docs/examples/data-types/all-data-types.odcs.yaml @@ -0,0 +1,66 @@ +version: 1.0.0 +kind: DataContract +id: 53581432-6c55-4ba2-a65f-72344a91553a +status: current +name: my_table +dataProduct: my_quantum +apiVersion: v3.0.0 +schema: + - name: transactions_tbl + description: Provides core payment metrics + dataGranularityDescription: Aggregation on names txn_ref_dt, pmt_txn_id + physicalType: table + properties: + - name: account_id + physicalType: string + logicalType: string + logicalTypeOptions: + minLength: 11 + maxLength: 11 + pattern: ACC[0-9]{8} + - name: txn_ref_date + physicalType: date + logicalType: date + logicalTypeOptions: + minimum: "2020-01-01" + maximum: "2021-01-01" + format: yyyy-MM-dd + - name: txn_timestamp + physicalType: timestamp + logicalType: date + logicalTypeOptions: + minimum: "2020-01-01 00:00:00" + maximum: "2021-01-01 00:00:00" + format: "yyyy-MM-dd HH:mm:ss" + - name: amount + physicalType: double + logicalType: number + logicalTypeOptions: + minimum: 0 + format: f32 + - name: age + physicalType: int + logicalType: integer + logicalTypeOptions: + minimum: 18 + maximum: 100 + exclusiveMaximum: true + format: i64 + - name: is_open + physicalType: bool + logicalType: boolean + - name: latest_txns + physicalType: list + logicalType: array + logicalTypeOptions: + minItems: 0 + maxItems: 3 + uniqueItems: true + - name: customer_details + physicalType: json + logicalType: object + logicalTypeOptions: + required: + - num_children + - date_of_birth + maxProperties: 5 diff --git a/docs/examples/fundamentals/table-column-description.odcs.yaml b/docs/examples/fundamentals/table-column-description.odcs.yaml index 99ed466..57a1576 100644 --- a/docs/examples/fundamentals/table-column-description.odcs.yaml +++ b/docs/examples/fundamentals/table-column-description.odcs.yaml @@ -1,20 +1,20 @@ version: 1.0.0 kind: DataContract -uuid: 53581432-6c55-4ba2-a65f-72344a91553a -type: tables +id: 53581432-6c55-4ba2-a65f-72344a91553a status: current -datasetName: my_table -quantumName: my_quantum -dataset: - - table: tbl +name: my_quantum +apiVersion: v3.0.0 +schema: + - name: tbl description: Provides core payment metrics - dataGranularity: Aggregation on columns txn_ref_dt, pmt_txn_id - columns: - - column: txn_ref_dt + dataGranularityDescription: Aggregation on columns txn_ref_dt, pmt_txn_id + physicalType: table + properties: + - name: txn_ref_dt businessName: Transaction reference date logicalType: date physicalType: date description: Reference date for the transaction. Use this date in reports and aggregation rather than txn_mystical_dt, as it is slightly too mystical. - sampleValues: + examples: - 2022-10-03 - 2025-01-28 diff --git a/docs/examples/quality/column-accuracy.odcs.yaml b/docs/examples/quality/column-accuracy.odcs.yaml index d9ec23a..fb0a705 100644 --- a/docs/examples/quality/column-accuracy.odcs.yaml +++ b/docs/examples/quality/column-accuracy.odcs.yaml @@ -1,26 +1,27 @@ version: 1.0.0 kind: DataContract -uuid: 53581432-6c55-4ba2-a65f-72344a91553a -type: tables +id: 53581432-6c55-4ba2-a65f-72344a91553a status: current -datasetName: my_table -quantumName: my_quantum -dataset: - - table: Air_Quality +name: my_table +dataProduct: my_quantum +apiVersion: v3.0.0 +schema: + - name: Air_Quality description: Air quality of the city of New York authoritativeDefinitions: - url: https://catalog.data.gov/dataset/air-quality type: Reference definition on Data.gov - dataGranularity: Raw records - columns: - - column: DataValue + dataGranularityDescription: Raw records + physicalType: table + properties: + - name: DataValue businessName: Measured value logicalType: number physicalType: float(3,2) quality: - - templateName: RangeCheck - toolName: ClimateQuantumDataQualityPackage - description: 'This column should contain positive values under 500' + - type: library + rule: between + description: 'This name should contain positive values under 500' dimension: accuracy severity: error businessImpact: operational diff --git a/docs/examples/quality/column-completeness.odcs.yaml b/docs/examples/quality/column-completeness.odcs.yaml index 6397fd7..f137d28 100644 --- a/docs/examples/quality/column-completeness.odcs.yaml +++ b/docs/examples/quality/column-completeness.odcs.yaml @@ -1,27 +1,27 @@ version: 1.0.0 +apiVersion: v3.0.0 kind: DataContract -uuid: 53581432-6c55-4ba2-a65f-72344a91553a -type: tables +id: 53581432-6c55-4ba2-a65f-72344a91553a status: current -datasetName: my_table -quantumName: my_quantum -dataset: - - table: Air_Quality +name: my_table +dataProduct: my_quantum +schema: + - name: Air_Quality description: Air quality of the city of New York authoritativeDefinitions: - url: https://catalog.data.gov/dataset/air-quality type: Reference definition on Data.gov - dataGranularity: Raw records - columns: - - column: UniqueID - isPrimaryKey: true + dataGranularityDescription: Raw records + physicalType: table + properties: + - name: UniqueID + primaryKey: true businessName: Unique identifier logicalType: number physicalType: int quality: - - templateName: NullCheck - toolName: ClimateQuantumDataQualityPackage - description: This column should not contain null values + - description: This column should not contain null values dimension: completeness severity: error + rule: nullCheck businessImpact: operational \ No newline at end of file diff --git a/docs/examples/quality/column-custom.odcs.yaml b/docs/examples/quality/column-custom.odcs.yaml new file mode 100644 index 0000000..32d1fff --- /dev/null +++ b/docs/examples/quality/column-custom.odcs.yaml @@ -0,0 +1,45 @@ +version: 1.0.0 +apiVersion: v3.0.0 +kind: DataContract +id: 53581432-6c55-4ba2-a65f-72344a91553a +status: current +name: my_table +dataProduct: my_quantum +schema: + - name: Air_Quality + description: Air quality of the city of New York + authoritativeDefinitions: + - url: https://catalog.data.gov/dataset/air-quality + type: Reference definition on Data.gov + dataGranularityDescription: Raw records + physicalType: table + properties: + - name: UniqueID + primaryKey: true + businessName: Unique identifier + logicalType: number + physicalType: int + quality: + - type: custom + description: This column should not contain values under 100000 + dimension: conformity + severity: error + businessImpact: operational + engine: soda + implementation: | + type: duplicate_percent + columns: + - carrier + - shipment_numer + must_be_less_than: 1.0 + - type: custom + dimension: uniqueness + severity: error + businessImpact: operational + engine: soda + implementation: + type: duplicate_percent + columns: + - carrier + - shipment_numer + must_be_less_than: 1.0 diff --git a/docs/examples/quality/column-validity.odcs.yaml b/docs/examples/quality/column-validity.odcs.yaml index 29c8f7a..99db211 100644 --- a/docs/examples/quality/column-validity.odcs.yaml +++ b/docs/examples/quality/column-validity.odcs.yaml @@ -1,28 +1,28 @@ version: 1.0.0 +apiVersion: v3.0.0 kind: DataContract -uuid: 53581432-6c55-4ba2-a65f-72344a91553a -type: tables +id: 53581432-6c55-4ba2-a65f-72344a91553a status: current -datasetName: my_table -quantumName: my_quantum -dataset: - - table: Air_Quality +name: my_table +dataProduct: my_quantum +schema: + - name: Air_Quality description: Air quality of the city of New York authoritativeDefinitions: - url: https://catalog.data.gov/dataset/air-quality type: Reference definition on Data.gov - dataGranularity: Raw records - columns: - - column: UniqueID - isPrimaryKey: true + dataGranularityDescription: Raw records + physicalType: table + properties: + - name: UniqueID + primaryKey: true businessName: Unique identifier logicalType: number physicalType: int quality: - - templateName: RangeCheck - toolName: ClimateQuantumDataQualityPackage + - rule: minCheck description: This column should not contain values under 100000 - dimension: validity + dimension: conformity severity: error businessImpact: operational customProperties: diff --git a/docs/examples/roles/service-and-operational-roles.odcs.yaml b/docs/examples/roles/service-and-operational-roles.odcs.yaml index dd34e88..b51a0a3 100644 --- a/docs/examples/roles/service-and-operational-roles.odcs.yaml +++ b/docs/examples/roles/service-and-operational-roles.odcs.yaml @@ -1,11 +1,11 @@ version: 1.0.0 kind: DataContract -uuid: 53581432-6c55-4ba2-a65f-72344a91553a -type: tables +id: 53581432-6c55-4ba2-a65f-72344a91553a status: current -datasetName: my_table -quantumName: my_quantum -dataset: [] +name: my_table +dataProduct: my_quantum +schema: [] +apiVersion: v3.0.0 roles: - role: microstrategy_user_opr access: read diff --git a/docs/examples/schema/all-schema-types.odcs.yaml b/docs/examples/schema/all-schema-types.odcs.yaml new file mode 100644 index 0000000..45b02fc --- /dev/null +++ b/docs/examples/schema/all-schema-types.odcs.yaml @@ -0,0 +1,102 @@ +version: 1.0.0 +kind: DataContract +id: 53581432-6c55-4ba2-a65f-72344a91553a +status: current +name: my_quantum +apiVersion: v3.0.0 +schema: + - name: tbl + logicalType: object + physicalType: table + physicalName: tbl_1 + description: Provides core payment metrics + authoritativeDefinitions: + - url: https://catalog.data.gov/dataset/air-quality + type: businessDefinition + - url: https://youtu.be/jbY1BKFj9ec + type: videoTutorial + tags: [] + dataGranularityDescription: Aggregation on columns txn_ref_dt, pmt_txn_id + properties: + - name: txn_ref_dt + primaryKey: false + primaryKeyPosition: -1 + businessName: transaction reference date + logicalType: date + physicalType: date + required: false + description: transaction ref date + partitioned: true + partitionKeyPosition: 1 + criticalDataElement: false + tags: [] + classification: public + transformSourceObjects: + - table_name_1 + - table_name_2 + - table_name_3 + transformLogic: sel t1.txn_dt as txn_ref_dt from table_name_1 as t1, table_name_2 as t2, table_name_3 as t3 where t1.txn_dt=date-3 + transformDescription: Defines the logic in business terms. + examples: + - "2022-10-03" + - "2020-01-28" + - name: rcvr_id + primaryKey: true + primaryKeyPosition: 1 + businessName: receiver id + logicalType: string + physicalType: varchar(18) + required: false + description: A description for column rcvr_id. + partitioned: false + partitionKeyPosition: -1 + criticalDataElement: false + tags: [] + classification: restricted + encryptedName: enc_rcvr_id + - name: rcvr_cntry_code + primaryKey: false + primaryKeyPosition: -1 + businessName: receiver country code + logicalType: string + physicalType: varchar(2) + required: false + description: country code + partitioned: false + partitionKeyPosition: -1 + criticalDataElement: false + tags: [] + classification: public + authoritativeDefinitions: + - url: https://collibra.com/asset/742b358f-71a5-4ab1-bda4-dcdba9418c25 + type: businessDefinition + - url: https://github.com/myorg/myrepo + type: transformationImplementation + - url: jdbc:postgresql://localhost:5432/adventureworks/tbl_1/rcvr_cntry_code + type: implementation + encryptedName: rcvr_cntry_code_encrypted + - name: AnObject + logicalType: object + properties: + - name: street_lines + logicalType: array + physicalType: array + items: + physicalType: string + logicalType: string + - name: AnotherObject + logicalType: object + properties: + - name: x + logicalType: array + physicalType: array + items: + logicalType: object + physicalType: object + properties: + - name: id + logicalType: string + physicalType: VARCHAR(40) + - name: zip + logicalType: string + physicalType: VARCHAR(15) \ No newline at end of file diff --git a/docs/examples/schema/table-column.odcs.yaml b/docs/examples/schema/table-column.odcs.yaml index 7ca324b..183807c 100644 --- a/docs/examples/schema/table-column.odcs.yaml +++ b/docs/examples/schema/table-column.odcs.yaml @@ -1,14 +1,15 @@ version: 1.0.0 kind: DataContract -uuid: 53581432-6c55-4ba2-a65f-72344a91553a -type: tables +id: 53581432-6c55-4ba2-a65f-72344a91553b status: current -datasetName: my_table -quantumName: my_quantum -dataset: - - table: tbl - columns: - - column: rcvr_cntry_code +name: my_table +dataProduct: my_quantum +apiVersion: v3.0.0 +schema: + - name: tbl + physicalType: table + properties: + - name: rcvr_cntry_code businessName: Receiver country code logicalType: string physicalType: varchar(2) diff --git a/docs/examples/schema/table-columns-with-partition.odcs.yaml b/docs/examples/schema/table-columns-with-partition.odcs.yaml index 5e487ac..25a0cc1 100644 --- a/docs/examples/schema/table-columns-with-partition.odcs.yaml +++ b/docs/examples/schema/table-columns-with-partition.odcs.yaml @@ -1,43 +1,39 @@ version: 1.0.0 kind: DataContract -uuid: 53581432-6c55-4ba2-a65f-72344a91553a -type: tables +id: 53581432-6c55-4ba2-a65f-72344a91553c status: current -datasetName: my_table -quantumName: my_quantum -dataset: - - table: tbl - columns: - - column: rcvr_cntry_code - businessName: Receiver country code - logicalType: string - physicalType: varchar(2) - isPrimaryKey: true - primaryKeyPosition: 1 - partitionStatus: true - partitionKeyPosition: 1 - clusterStatus: false - - column: rcvr_id - businessName: Receiver identification number - logicalType: string - physicalType: varchar(20) - isPrimaryKey: true - primaryKeyPosition: 2 - partitionStatus: false - clusterStatus: false - - column: year - businessName: Year of transaction - logicalType: integer - physicalType: int - isPrimaryKey: false - partitionStatus: true - partitionKeyPosition: 2 - clusterStatus: true - clusterKeyPosition: 1 - - column: amount - businessName: Transaction amount - logicalType: double - physicalType: double - isPrimaryKey: false - partitionStatus: false - clusterStatus: false +name: my_table +dataProduct: my_quantum +apiVersion: v3.0.0 +schema: + - name: tbl + physicalType: table + properties: + - name: rcvr_cntry_code + businessName: Receiver country code + logicalType: string + physicalType: varchar(2) + primaryKey: true + primaryKeyPosition: 1 + partitioned: true + partitionKeyPosition: 1 + - name: rcvr_id + businessName: Receiver identification number + logicalType: string + physicalType: varchar(20) + primaryKey: true + primaryKeyPosition: 2 + partitioned: false + - name: year + businessName: Year of transaction + logicalType: integer + physicalType: int + primaryKey: false + partitioned: true + partitionKeyPosition: 2 + - name: amount + businessName: Transaction amount + logicalType: number + physicalType: double + primaryKey: false + partitioned: false diff --git a/docs/examples/server/azure-server.odcs.yaml b/docs/examples/server/azure-server.odcs.yaml new file mode 100644 index 0000000..9d3c23d --- /dev/null +++ b/docs/examples/server/azure-server.odcs.yaml @@ -0,0 +1,10 @@ +version: 1.0.0 +apiVersion: v3.0.0 +kind: DataContract +id: abc123 +status: pending +servers: + - server: my-azure + type: azure + location: az://my-data-location + format: parquet diff --git a/docs/examples/server/kafka-server.odcs.yaml b/docs/examples/server/kafka-server.odcs.yaml new file mode 100644 index 0000000..0dc94dc --- /dev/null +++ b/docs/examples/server/kafka-server.odcs.yaml @@ -0,0 +1,14 @@ +version: 1.0.0 +apiVersion: v3.0.0 +kind: DataContract +id: abc123 +status: pending +schema: + - name: Orders + physicalName: orders.avro.v1 + physicalType: topic +servers: + - server: my-kafka + type: kafka + host: pkc-7xoy1.eu-central-1.aws.confluent.cloud:9092 + format: avro diff --git a/docs/examples/sla/database-table-sla.odcs.yaml b/docs/examples/sla/database-table-sla.odcs.yaml index 643c290..9348ab2 100644 --- a/docs/examples/sla/database-table-sla.odcs.yaml +++ b/docs/examples/sla/database-table-sla.odcs.yaml @@ -1,28 +1,28 @@ version: 1.0.0 +apiVersion: v3.0.0 kind: DataContract -uuid: 53581432-6c55-4ba2-a65f-72344a91553a -type: tables +id: 53581432-6c55-4ba2-a65f-72344a91553a status: current -datasetName: my_table -quantumName: my_quantum -dataset: [] +name: my_table +dataProduct: my_quantum +schema: [] # SLA -slaDefaultColumn: tab1.txn_ref_dt # Optional, default value is partitionColumn. +slaDefaultElement: tab1.txn_ref_dt # Optional, default value is partitionColumn. slaProperties: - property: latency # Property, see list of values in DP QoS value: 4 unit: d # d, day, days for days; y, yr, years for years - column: tab1.txn_ref_dt # This would not be needed as it is the same table.column as the default one + element: tab1.txn_ref_dt # This would not be needed as it is the same table.column as the default one - property: generalAvailability - value: 2022-05-12T09:30:10-08:00 + value: "2022-05-12T09:30:10-08:00" - property: endOfSupport - value: 2032-05-12T09:30:10-08:00 + value: "2032-05-12T09:30:10-08:00" - property: endOfLife - value: 2042-05-12T09:30:10-08:00 + value: "2042-05-12T09:30:10-08:00" - property: retention value: 3 unit: y - column: tab1.txn_ref_dt + element: tab1.txn_ref_dt - property: frequency value: 1 valueExt: 1 @@ -30,9 +30,9 @@ slaProperties: column: tab1.txn_ref_dt - property: timeOfAvailability value: 09:00-08:00 - column: tab1.txn_ref_dt + element: tab1.txn_ref_dt driver: regulatory # Describes the importance of the SLA: [regulatory|analytics|operational|...] - property: timeOfAvailability value: 08:00-08:00 - column: tab1.txn_ref_dt - driver: analytics \ No newline at end of file + element: tab1.txn_ref_dt + driver: analytics diff --git a/docs/examples/stakeholders/basic-four-dpo.odcs.yaml b/docs/examples/stakeholders/basic-four-dpo.odcs.yaml index 6eec799..cc870ac 100644 --- a/docs/examples/stakeholders/basic-four-dpo.odcs.yaml +++ b/docs/examples/stakeholders/basic-four-dpo.odcs.yaml @@ -1,28 +1,28 @@ version: 1.0.0 +apiVersion: v3.0.0 kind: DataContract -uuid: 53581432-6c55-4ba2-a65f-72344a91553a -type: tables +id: 53581432-6c55-4ba2-a65f-72344a91553a status: current -datasetName: my_table -quantumName: my_quantum -dataset: [] -stakeholders: +name: my_table +dataProduct: my_quantum +schema: [ ] +team: - username: ceastwood role: dpo - dateIn: 2014-08-02 - dateOut: 2014-10-01 + dateIn: "2014-08-02" + dateOut: "2014-10-01" replacedByUsername: jwayne - username: jwayne role: dpo - dateIn: 2014-10-01 - dateOut: 2019-03-14 + dateIn: "2014-10-01" + dateOut: "2019-03-14" replacedByUsername: cjane - username: cjane role: dpo - dateIn: 2019-03-14 + dateIn: "2019-03-14" comment: Minor interruption due to sabbatical, will be back by end of April 2021 - dateOut: 2021-04-01 + dateOut: "2021-04-01" replacedByUsername: bkid - username: bkid role: dpo - dateIn: 2021-04-01 + dateIn: "2021-04-01" diff --git a/docs/img/data-contract-diagram-latest.svg b/docs/img/data-contract-diagram-latest.svg new file mode 100644 index 0000000..152b428 --- /dev/null +++ b/docs/img/data-contract-diagram-latest.svg @@ -0,0 +1,450 @@ + + + + + + + + + + + + + + + + + Data Contract Vert - v3 + + bg + + + + + + Layer 1 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Applications + + + + + + Monitoring + + + + + + Observability + + + + + + Notification + + + + + + Tools + + + + + + Enterprise Data + Governance + + + + + + Enterprise Data + Catalog + + + + + + Enterprise Security + & Audit + + + + + + Enterprise Ops + + + + + + Open Data Contract Standard v3 + + + + + + Data Engineers + + + + + + Data Scientists + + + + + + Data Product + Owners + + + + + + Automation Tools + + + + + + + + Name, + Version, + Descriptions… + + + + + + + + History of team + members + + + + + + + + Latency, retention, + frequency… + + + + + + + + Data quality rules, + Data governance + policies + + + + + + + + Logical + representation & + physical + implementation + + + + + + + + Servers, environment, + and storage + + + + + + + + Internal or external + costs associated to + usage + + + + + + + + Roles + + + + + + Fundamentals + + + + + + Schema* + + + + + + SLA + + + + + + Data Quality* + + + + + + Team + + + + + + Infrastructure** + + + + + + Pricing + + + + + + Security + + + + + + + + Enterprise-level consumers & + contributors + + + + + + + + Contributors to the data + contract + + + + + + + + Consumers of the data + contract + + + + + + + + + + + + + + + + + + + + + + + Data QoS applied to + your business needs + + + + + + Business rules*** + + + + + + + + Room for + custom needs + + + + + + Custom + + + + + + + + * Major evolution in v3 + ** New in v3 + *** Scheduled for v3.1 + + + + + + diff --git a/docs/img/data-contract-diagram-v3.pdf b/docs/img/data-contract-diagram-v3.pdf new file mode 100644 index 0000000..53392f2 Binary files /dev/null and b/docs/img/data-contract-diagram-v3.pdf differ diff --git a/docs/img/data-contract-diagram-v3.png b/docs/img/data-contract-diagram-v3.png new file mode 100644 index 0000000..c219a97 Binary files /dev/null and b/docs/img/data-contract-diagram-v3.png differ diff --git a/docs/img/data-contract-diagram-v3.svg b/docs/img/data-contract-diagram-v3.svg new file mode 100644 index 0000000..152b428 --- /dev/null +++ b/docs/img/data-contract-diagram-v3.svg @@ -0,0 +1,450 @@ + + + + + + + + + + + + + + + + + Data Contract Vert - v3 + + bg + + + + + + Layer 1 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Applications + + + + + + Monitoring + + + + + + Observability + + + + + + Notification + + + + + + Tools + + + + + + Enterprise Data + Governance + + + + + + Enterprise Data + Catalog + + + + + + Enterprise Security + & Audit + + + + + + Enterprise Ops + + + + + + Open Data Contract Standard v3 + + + + + + Data Engineers + + + + + + Data Scientists + + + + + + Data Product + Owners + + + + + + Automation Tools + + + + + + + + Name, + Version, + Descriptions… + + + + + + + + History of team + members + + + + + + + + Latency, retention, + frequency… + + + + + + + + Data quality rules, + Data governance + policies + + + + + + + + Logical + representation & + physical + implementation + + + + + + + + Servers, environment, + and storage + + + + + + + + Internal or external + costs associated to + usage + + + + + + + + Roles + + + + + + Fundamentals + + + + + + Schema* + + + + + + SLA + + + + + + Data Quality* + + + + + + Team + + + + + + Infrastructure** + + + + + + Pricing + + + + + + Security + + + + + + + + Enterprise-level consumers & + contributors + + + + + + + + Contributors to the data + contract + + + + + + + + Consumers of the data + contract + + + + + + + + + + + + + + + + + + + + + + + Data QoS applied to + your business needs + + + + + + Business rules*** + + + + + + + + Room for + custom needs + + + + + + Custom + + + + + + + + * Major evolution in v3 + ** New in v3 + *** Scheduled for v3.1 + + + + + + diff --git a/docs/img/elements-of-schema-odcs-v3.png b/docs/img/elements-of-schema-odcs-v3.png new file mode 100644 index 0000000..3f7dce8 Binary files /dev/null and b/docs/img/elements-of-schema-odcs-v3.png differ diff --git a/docs/img/elements-of-schema-odcs-v3.svg b/docs/img/elements-of-schema-odcs-v3.svg new file mode 100644 index 0000000..c84e921 --- /dev/null +++ b/docs/img/elements-of-schema-odcs-v3.svg @@ -0,0 +1,195 @@ + + + + + + 2♠︎ - + + + Backgound + + + + + + Layer 1 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + logicalType: object + physicalType: table + name: Orders + physicalName: order + + + + + logicalType: object + physicalType: table + name: OrdersLines + physicalName: or_line + + + + + logicalType: number + physicalType: float(10) + name: Quantity + physicalName: qty + + + + + logicalType: integer + physicalType: serial + name: Id + physicalName: id + + + + + properties + + + + + + + + + + + + elements + + + + + + + + + + + objects + + + + + + Orders + + + + + + OrderLines + + + + + + date + + + + + + customer_id + + + + + + delivery_ctry + + + + + + … + + + + + + id + + + + + + id + + + + + + order_id + + + + + + qty + + + + + + product_id + + + + + + … + + + + + diff --git a/mkdocs.yml b/mkdocs.yml index 0cd0fed..3e583d7 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -7,6 +7,7 @@ plugins: - open-in-new-tab - search - social + - awesome-pages theme: name: material @@ -34,7 +35,9 @@ theme: nav: - Home: 'home.md' - Standard: 'README.md' - - Examples: 'examples/README.md' + - Examples: + - 'examples/README.md' + - '...' - Changelog: 'changelog.md' - Contributing: 'contributing.md' - Vendors: 'vendors.md' diff --git a/resources.md b/resources.md new file mode 100644 index 0000000..e8f8d49 --- /dev/null +++ b/resources.md @@ -0,0 +1,35 @@ +# Articles & Other Resources + +## Articles + +* 2024-09-30 - [Data Contracts in Action: Tools](https://medium.com/data-engineer-things/data-contracts-in-action-tools-303bc7fbceb5) +* 2024-07-17 - [Data Contracts in Action: Testing](https://medium.com/@pflooky/data-contracts-in-action-testing-111631338657) +* 2024-06-12 - [The Future of Data Management: An Enabler of AI Development? A Basic Illustration with RAG, Open Standards, and Data Contracts](https://blog.owulveryck.info/2024/06/12/the-future-of-data-management-an-enabler-of-ai-development-a-basic-illustration-with-rag-open-standards-and-data-contracts.html) +* 2024-05-30 - [ODCS Roadmap](https://medium.com/abeadata/odcs-roadmap-9b9a17367af4) +* 2024-05-25 - [Conceptual model of Data Quality of Service as Code by Jarkko Moilanen](https://aidausergroup.org/2024/05/25/aida-user-group-forecaster-pi-day-highlights-data-quality-whats-new/) +* 2024-02-06 - [Getting started with ODCS](https://medium.com/abeadata/getting-started-with-odcs-3ba790707879) +* 2023-12-08 - [Why the Need for Standardizing Data Contracts?](https://medium.com/abeadata/why-the-need-for-standardizing-data-contracts-133bc3491148) +* 2023-11-30 - [Linux Foundation AI & Data - Bitol Joins LF AI & Data as New Sandbox Project](https://lfaidata.foundation/blog/2023/11/30/bitol-joins-lf-ai-data-as-new-sandbox-project/) +* 2023-11-30 - [AIDAUG - Bitol Joins LF AI & Data as New Sandbox Project](https://aidausergroup.org/2023/11/30/bitol-joins-lf-ai-data-as-new-sandbox-project/) +* 2023-11-22 - [What is, and what isn’t, a data contract](https://datacreation.substack.com/p/what-is-and-what-isnt-a-data-contract) +* 2023-10-01 - [Data Contracts: A Bridge Connecting Two Worlds](https://medium.com/@atanas.iliev.ai/data-contracts-a-bridge-connecting-two-worlds-404eff1d970d) +* 2023-09-10 - [Data Contracts 101](https://medium.com/p/568a9adbf9a9) +* 2023-08-10 - [Welcome to the Open Data Contract Standard](https://jgp.ai/2023/08/09/welcome-to-the-open-data-contract-standard/) +* 2023-05-11 - [Data Contracts – Everything You Need to Know](https://www.montecarlodata.com/blog-data-contracts-explained/) +* 2023-05-07 - [Data Engineering Weekly #130 - Data Contract in the Wild with PayPal’s Data Contract Template](https://www.dataengineeringweekly.com/p/data-engineering-weekly-130) +* 2023-05-06 - [PayPal เปิด Data Contract เป็น Open Source Template ให้ไปใช้งานกัน](https://discuss.dataengineercafe.io/t/paypal-data-contract-open-source-template/581/1) +* 2023-05-05 - [Jonathan Neo (j__neo ) on Reddit](https://www.reddit.com/r/dataengineering/comments/137glbo/comment/jixw5hj/?utm_source=reddit&utm_medium=web2x&context=3) +* 2023-05-01 - [PayPal open sources its data contract template](https://jgp.ai/2023/05/01/paypal-open-sources-its-data-contract-template/) + +If you spot an article about the Open Data Contract Standard, make a pull request! + +## Books + +* 2024-09-01 - **Implementing Data Mesh**, [O'Reilly](https://www.oreilly.com/library/view/implementing-data-mesh/9781098156213/), [Amazon](https://amzn.to/3ysN3Jf): in Chapter 5, ODCS is extensively used & described. +* 2024-01-20 - **Data Contracts for all ages**, [Amazon](https://amzn.to/3Wl1My1). + +If you spot a book about the Open Data Contract Standard, make a pull request! + +## Videos + +If you spot a video about the Open Data Contract Standard, make a pull request! diff --git a/schema/README.md b/schema/README.md index 82fd56a..aa2fcc3 100644 --- a/schema/README.md +++ b/schema/README.md @@ -19,7 +19,7 @@ following section: ```json { - "name": "Open Data Contract Standard (ODCS))", + "name": "Open Data Contract Standard (ODCS)", ..., "versions": { "": "https://github.com/bitol-io/open-data-contract-standard/blob/main/schema/odcs-json-schema-.json", diff --git a/schema/odcs-json-schema-latest.json b/schema/odcs-json-schema-latest.json index b72086c..1ecb880 100644 --- a/schema/odcs-json-schema-latest.json +++ b/schema/odcs-json-schema-latest.json @@ -1,6 +1,6 @@ { "$schema": "https://json-schema.org/draft/2019-09/schema", - "title": "Open Data Contract Standard (OCDS)", + "title": "Open Data Contract Standard (ODCS)", "description": "An open data contract specification to establish agreement between data producers and consumers.", "type": "object", "properties": { @@ -16,90 +16,40 @@ }, "apiVersion": { "type": "string", - "default": "v2.2.2", - "description": "Version of the standard used to build data contract. Default value is v2.2.2.", - "pattern": "^v[0-9]+\\.[0-9]+\\.[0-9]+" + "default": "v3.0.0", + "description": "Version of the standard used to build data contract. Default value is v3.0.0.", + "enum": ["v3.0.0", "v2.2.2", "v2.2.1", "v2.2.0"] }, - "uuid": { + "id": { "type": "string", - "description": "A unique identifier used to reduce the risk of dataset name collisions; initially the UUID will be created using a UUID generator tool ([example](https://www.uuidgenerator.net/)). However, we may want to develop a method that accepts a seed value using a combination of fields–such as name, kind and source–to create a repeatable value." + "description": "A unique identifier used to reduce the risk of dataset name collisions, such as a UUID." }, - "datasetKind": { + "name": { "type": "string", - "description": "The kind of dataset being cataloged; Expected values are `virtualDataset` or `managedDataset`.", - "examples": ["virtualDataset", "managedDataset"] - }, - "userConsumptionMode": { - "type": "string", - "description": "List of data modes for which the dataset may be used. Expected sample values might be `analytical` or `operational`.
Note: in the future, this will probably be replaced by ports.", - "examples": ["analytical", "operational"] - }, - "type": { - "type": "string", - "description": "Identifies the types of objects in the dataset. For BigQuery or any other database, the expected value would be Tables.", - "examples": ["Tables"] + "description": "Name of the data contract." }, "tenant": { "type": "string", "description": "Indicates the property the data is primarily associated with. Value is case insensitive." }, "tags": { - "type": "array", - "description": "A list of tags that may be assigned to the dataset, table or column; the `tags` keyword may appear at any level.", - "items": { - "type": "string" - } + "$ref": "#/$defs/Tags" }, "status": { "type": "string", "description": "Current status of the dataset. Valid values are `production`, `test`, or `development`.", "examples": ["production", "test", "development"] }, - "sourceSystem": { - "type": "string", - "description": "The system where the dataset resides. Expected value can be BigQuery.", - "examples": ["BigQuery"] - }, - "sourcePlatform": { - "type": "string", - "description": "The platform where the dataset resides. Expected value is GoogleCloudPlatform, IBMCloud, Azure...", - "examples": ["GoogleCloudPlatform", "IBMCloud", "Azure", "AWS"] - }, - "server": { - "type": "string", - "description": "The server where the dataset resides." - }, - "quantumName": { - "type": "string", - "description": "The name of the data quantum or data product." - }, - "productSlackChannel": { - "type": "string", - "description": "Slack channel of the team responsible for maintaining the dataset." - }, - "productFeedbackUrl": { - "type": "string", - "description": "The URL for submitting feedback to the team responsible for maintaining the dataset." - }, - "productDl": { - "type": "string", - "description": "The email distribution list (DL) of the persons or team responsible for maintaining the dataset." - }, - "username": { - "type": "string", - "description": "User credentials for connecting to the dataset; how the credentials will be stored/passed is outside of the scope of the contract." - }, - "password": { - "type": "string", - "description": "User credentials for connecting to the dataset; how the credentials will be stored/passed is out of the scope of this contract." - }, - "driverVersion": { - "type": "string", - "description": "The version of the connection driver to be used to connect to the dataset." + "servers": { + "type": "array", + "description": "List of servers where the datasets reside.", + "items": { + "$ref": "#/$defs/Server" + } }, - "driver": { + "dataProduct": { "type": "string", - "description": "The connection driver required to connect to the dataset." + "description": "The name of the data product." }, "description": { "type": "object", @@ -119,36 +69,28 @@ } } }, - "project": { - "type": "string", - "description": "Associated project name, can be used for billing or administrative purpose. Used to be datasetProject." - }, - "datasetName": { + "domain": { "type": "string", - "description": "May be required in cloud instance like GCP BigQuery dataset name." - }, - "datasetDomain": { - "type": "string", - "description": "Name of the logical domain dataset the contract describes. This field is only required for output data contracts.", + "description": "Name of the logical data domain.", "examples": ["imdb_ds_aggregate", "receiver_profile_out", "transaction_profile_out"] }, - "database": { - "type": "string", - "description": "The database where the dataset resides." - }, - "dataset": { + "schema": { "type": "array", + "description": "A list of elements within the schema to be cataloged.", "items": { - "$ref": "#/$defs/Dataset" + "$ref": "#/$defs/SchemaObject" } }, + "support": { + "$ref": "#/$defs/Support" + }, "price": { "$ref": "#/$defs/Pricing" }, - "stakeholders": { + "team": { "type": "array", "items": { - "$ref": "#/$defs/Stakeholder" + "$ref": "#/$defs/Team" } }, "roles": { @@ -158,240 +100,2013 @@ "$ref": "#/$defs/Role" } }, - "slaDefaultColumn": { - "type": "string", - "description": "Columns (using the Table.Column notation) to do the checks on. By default, it is the partition column." - }, - "slaProperties": { - "type": "array", - "description": "A list of key/value pairs for SLA specific properties. There is no limit on the type of properties (more details to come).", - "items": { - "$ref": "#/$defs/ServiceLevelAgreementProperty" - } - } - }, - "required": ["version", "kind", "uuid", "type", "status", "dataset", "datasetName", "quantumName"], - "$defs": { - "Dataset": { + "slaDefaultElement": { + "type": "string", + "description": "Element (using the element path notation) to do the checks on." + }, + "slaProperties": { + "type": "array", + "description": "A list of key/value pairs for SLA specific properties. There is no limit on the type of properties (more details to come).", + "items": { + "$ref": "#/$defs/ServiceLevelAgreementProperty" + } + }, + "customProperties": { + "$ref": "#/$defs/CustomProperties" + }, + "contractCreatedTs": { + "type": "string", + "format": "date-time", + "description": "Timestamp in UTC of when the data contract was created." + } + }, + "required": ["version", "apiVersion", "kind", "id", "status"], + "additionalProperties": false, + "$defs": { + "Server": { + "type": "object", + "description": "Data source details of where data is physically stored.", + "properties": { + "server": { + "type": "string", + "description": "Identifier of the server." + }, + "type": { + "type": "string", + "description": "Type of the server.", + "enum": [ + "api", "athena", "azure", "bigquery", "clickhouse", "databricks", "denodo", "dremio", + "duckdb", "glue", "cloudsql", "db2", "informix", "kafka", "kinesis", "local", + "mysql", "oracle", "postgresql", "postgres", "presto", "pubsub", + "redshift", "s3", "sftp", "snowflake", "sqlserver", "synapse", "trino", "vertica", "custom" + ] + }, + "description": { + "type": "string", + "description": "Description of the server." + }, + "environment": { + "type": "string", + "description": "Environment of the server.", + "examples": ["prod", "preprod", "dev", "uat"] + }, + "roles": { + "type": "array", + "description": "List of roles that have access to the server.", + "items": { + "$ref": "#/$defs/Role" + } + }, + "customProperties": { + "$ref": "#/$defs/CustomProperties" + } + }, + "allOf": [ + { + "if": { + "properties": { + "type": { + "const": "api" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/ApiServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "athena" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/AthenaServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "azure" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/AzureServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "bigquery" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/BigQueryServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "clickhouse" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/ClickHouseServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "databricks" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/DatabricksServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "denodo" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/DenodoServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "dremio" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/DremioServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "duckdb" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/DuckdbServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "glue" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/GlueServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "cloudsql" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/GoogleCloudSqlServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "db2" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/IBMDB2Server" + } + }, + { + "if": { + "properties": { + "type": { + "const": "informix" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/InformixServer" + } + }, + + { + "if": { + "properties": { + "type": { + "const": "custom" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/CustomServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "kafka" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/KafkaServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "kinesis" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/KinesisServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "local" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/LocalServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "mysql" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/MySqlServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "oracle" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/OracleServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "postgresql" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/PostgresServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "postgres" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/PostgresServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "presto" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/PrestoServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "pubsub" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/PubSubServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "redshift" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/RedshiftServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "s3" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/S3Server" + } + }, + { + "if": { + "properties": { + "type": { + "const": "sftp" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/SftpServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "snowflake" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/SnowflakeServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "sqlserver" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/SqlserverServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "synapse" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/SynapseServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "trino" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/TrinoServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "vertica" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/VerticaServer" + } + } + ], + "required": ["server", "type"] + }, + "ServerSource": { + "ApiServer": { + "type": "object", + "title": "AthenaServer", + "properties": { + "location": { + "type": "string", + "format": "uri", + "description": "The url to the API.", + "examples": [ + "https://api.example.com/v1" + ] + } + }, + "required": [ + "location" + ] + }, + "AthenaServer": { + "type": "object", + "title": "AthenaServer", + "properties": { + "stagingDir": { + "type": "string", + "format": "uri", + "description": "Amazon Athena automatically stores query results and metadata information for each query that runs in a query result location that you can specify in Amazon S3.", + "examples": [ + "s3://my_storage_account_name/my_container/path" + ] + }, + "schema": { + "type": "string", + "description": "Identify the schema in the data source in which your tables exist." + }, + "catalog": { + "type": "string", + "description": "Identify the name of the Data Source, also referred to as a Catalog.", + "default": "awsdatacatalog" + }, + "regionName": { + "type": "string", + "description": "The region your AWS account uses.", + "examples": ["eu-west-1"] + } + }, + "required": [ + "staging_dir", + "schema" + ] + }, + "AzureServer": { + "type": "object", + "title": "AzureServer", + "properties": { + "location": { + "type": "string", + "format": "uri", + "description": "Fully qualified path to Azure Blob Storage or Azure Data Lake Storage (ADLS), supports globs.", + "examples": [ + "az://my_storage_account_name.blob.core.windows.net/my_container/path/*.parquet", + "abfss://my_storage_account_name.dfs.core.windows.net/my_container_name/path/*.parquet" + ] + }, + "format": { + "type": "string", + "enum": [ + "parquet", + "delta", + "json", + "csv" + ], + "description": "File format." + }, + "delimiter": { + "type": "string", + "enum": [ + "new_line", + "array" + ], + "description": "Only for format = json. How multiple json documents are delimited within one file" + } + }, + "required": [ + "location", + "format" + ] + }, + "BigQueryServer": { + "type": "object", + "title": "BigQueryServer", + "properties": { + "project": { + "type": "string", + "description": "The GCP project name." + }, + "dataset": { + "type": "string", + "description": "The GCP dataset name." + } + }, + "required": [ + "project", + "dataset" + ] + }, + "ClickHouseServer": { + "type": "object", + "title": "ClickHouseServer", + "properties": { + "host": { + "type": "string", + "description": "The host of the ClickHouse server." + }, + "port": { + "type": "integer", + "description": "The port to the ClickHouse server." + }, + "database": { + "type": "string", + "description": "The name of the database." + } + }, + "required": [ + "host", + "port", + "database" + ] + }, + "DatabricksServer": { + "type": "object", + "title": "DatabricksServer", + "properties": { + "host": { + "type": "string", + "description": "The Databricks host", + "examples": [ + "dbc-abcdefgh-1234.cloud.databricks.com" + ] + }, + "catalog": { + "type": "string", + "description": "The name of the Hive or Unity catalog" + }, + "schema": { + "type": "string", + "description": "The schema name in the catalog" + } + }, + "required": [ + "catalog", + "schema" + ] + }, + "DenodoServer": { + "type": "object", + "title": "DenodoServer", + "properties": { + "host": { + "type": "string", + "description": "The host of the Denodo server." + }, + "port": { + "type": "integer", + "description": "The port of the Denodo server." + }, + "database": { + "type": "string", + "description": "The name of the database." + } + }, + "required": [ + "host", + "port" + ] + }, + "DremioServer": { + "type": "object", + "title": "DremioServer", + "properties": { + "host": { + "type": "string", + "description": "The host of the Dremio server." + }, + "port": { + "type": "integer", + "description": "The port of the Dremio server." + }, + "schema": { + "type": "string", + "description": "The name of the schema." + } + }, + "required": [ + "host", + "port" + ] + }, + "DuckdbServer": { + "type": "object", + "title": "DuckdbServer", + "properties": { + "database": { + "type": "string", + "description": "Path to duckdb database file." + }, + "schema": { + "type": "integer", + "description": "The name of the schema." + } + }, + "required": [ + "database" + ] + }, + "GlueServer": { + "type": "object", + "title": "GlueServer", + "properties": { + "account": { + "type": "string", + "description": "The AWS Glue account", + "examples": [ + "1234-5678-9012" + ] + }, + "database": { + "type": "string", + "description": "The AWS Glue database name", + "examples": [ + "my_database" + ] + }, + "location": { + "type": "string", + "format": "uri", + "description": "The AWS S3 path. Must be in the form of a URL.", + "examples": [ + "s3://datacontract-example-orders-latest/data/{model}" + ] + }, + "format": { + "type": "string", + "description": "The format of the files", + "examples": [ + "parquet", + "csv", + "json", + "delta" + ] + } + }, + "required": [ + "account", + "database" + ] + }, + "GoogleCloudSqlServer": { + "type": "object", + "title": "GoogleCloudSqlServer", + "properties": { + "host": { + "type": "string", + "description": "The host of the Google Cloud Sql server." + }, + "port": { + "type": "integer", + "description": "The port of the Google Cloud Sql server." + }, + "database": { + "type": "string", + "description": "The name of the database." + }, + "schema": { + "type": "string", + "description": "The name of the schema." + } + }, + "required": [ + "host", + "port", + "database", + "schema" + ] + }, + "IBMDB2Server": { + "type": "object", + "title": "IBMDB2Server", + "properties": { + "host": { + "type": "string", + "description": "The host of the IBM DB2 server." + }, + "port": { + "type": "integer", + "description": "The port of the IBM DB2 server." + }, + "database": { + "type": "string", + "description": "The name of the database." + }, + "schema": { + "type": "string", + "description": "The name of the schema." + } + }, + "required": [ + "host", + "port", + "database" + ] + }, + "InformixServer": { + "type": "object", + "title": "InformixServer", + "properties": { + "host": { + "type": "string", + "description": "The host to the Informix server. " + }, + "port": { + "type": "integer", + "description": "The port to the Informix server. Defaults to 9088." + }, + "database": { + "type": "string", + "description": "The name of the database." + } + }, + "required": [ + "host", + "database" + ] + }, + "CustomServer": { + "type": "object", + "title": "CustomServer", + "properties": { + "account": { + "type": "string", + "description": "Account used by the server." + }, + "catalog": { + "type": "string", + "description": "Name of the catalog." + }, + "database": { + "type": "string", + "description": "Name of the database." + }, + "dataset": { + "type": "string", + "description": "Name of the dataset." + }, + "delimiter": { + "type": "string", + "description": "Delimiter." + }, + "endpointUrl": { + "type": "string", + "description": "Server endpoint.", + "format": "uri" + }, + "format": { + "type": "string", + "description": "File format." + }, + "host": { + "type": "string", + "description": "Host name or IP address." + }, + "location": { + "type": "string", + "description": "A URL to a location.", + "format": "uri" + }, + "path": { + "type": "string", + "description": "Relative or absolute path to the data file(s)." + }, + "port": { + "type": "integer", + "description": "Port to the server. No default value is assumed for custom servers." + }, + "project": { + "type": "string", + "description": "Project name." + }, + "region": { + "type": "string", + "description": "Cloud region." + }, + "regionName": { + "type": "string", + "description": "Region name." + }, + "schema": { + "type": "string", + "description": "Name of the schema." + }, + "serviceName": { + "type": "string", + "description": "Name of the service." + }, + "stagingDir": { + "type": "string", + "description": "Staging directory." + }, + "warehouse": { + "type": "string", + "description": "Name of the cluster or warehouse." + } + } + }, + "KafkaServer": { + "type": "object", + "title": "KafkaServer", + "description": "Kafka Server", + "properties": { + "host": { + "type": "string", + "description": "The bootstrap server of the kafka cluster." + }, + "format": { + "type": "string", + "description": "The format of the messages.", + "examples": ["json", "avro", "protobuf", "xml"], + "default": "json" + } + }, + "required": [ + "host" + ] + }, + "KinesisServer": { + "type": "object", + "title": "KinesisDataStreamsServer", + "description": "Kinesis Data Streams Server", + "properties": { + "region": { + "type": "string", + "description": "AWS region.", + "examples": [ + "eu-west-1" + ] + }, + "format": { + "type": "string", + "description": "The format of the record", + "examples": [ + "json", + "avro", + "protobuf" + ] + } + } + }, + "LocalServer": { + "type": "object", + "title": "LocalServer", + "properties": { + "path": { + "type": "string", + "description": "The relative or absolute path to the data file(s).", + "examples": [ + "./folder/data.parquet", + "./folder/*.parquet" + ] + }, + "format": { + "type": "string", + "description": "The format of the file(s)", + "examples": [ + "json", + "parquet", + "delta", + "csv" + ] + } + }, + "required": [ + "path", + "format" + ] + }, + "MySqlServer": { + "type": "object", + "title": "MySqlServer", + "properties": { + "host": { + "type": "string", + "description": "The host of the MySql server." + }, + "port": { + "type": "integer", + "description": "The port of the MySql server." + }, + "database": { + "type": "string", + "description": "The name of the database." + } + }, + "required": [ + "host", + "port", + "database" + ] + }, + "OracleServer": { + "type": "object", + "title": "OracleServer", + "properties": { + "host": { + "type": "string", + "description": "The host to the oracle server", + "examples": [ + "localhost" + ] + }, + "port": { + "type": "integer", + "description": "The port to the oracle server.", + "examples": [ + 1523 + ] + }, + "serviceName": { + "type": "string", + "description": "The name of the service.", + "examples": [ + "service" + ] + } + }, + "required": [ + "host", + "port", + "serviceName" + ] + }, + "PostgresServer": { + "type": "object", + "title": "PostgresServer", + "properties": { + "host": { + "type": "string", + "description": "The host to the Postgres server" + }, + "port": { + "type": "integer", + "description": "The port to the Postgres server." + }, + "database": { + "type": "string", + "description": "The name of the database." + }, + "schema": { + "type": "string", + "description": "The name of the schema in the database." + } + }, + "required": [ + "host", + "port", + "database", + "schema" + ] + }, + "PrestoServer": { + "type": "object", + "title": "PrestoServer", + "properties": { + "host": { + "type": "string", + "description": "The host to the Presto server", + "examples": [ + "localhost:8080" + ] + }, + "catalog": { + "type": "string", + "description": "The name of the catalog.", + "examples": [ + "postgres" + ] + }, + "schema": { + "type": "string", + "description": "The name of the schema.", + "examples": [ + "public" + ] + } + }, + "required": [ + "host" + ] + }, + "PubSubServer": { + "type": "object", + "title": "PubSubServer", + "properties": { + "project": { + "type": "string", + "description": "The GCP project name." + } + }, + "required": [ + "project" + ] + }, + "RedshiftServer": { + "type": "object", + "title": "RedshiftServer", + "properties": { + "host": { + "type": "string", + "description": "An optional string describing the server." + }, + "database": { + "type": "string", + "description": "The name of the database." + }, + "schema": { + "type": "string", + "description": "The name of the schema." + }, + "region": { + "type": "string", + "description": "AWS region of Redshift server.", + "examples": ["us-east-1"] + }, + "account": { + "type": "string", + "description": "The account used by the server." + } + }, + "required": [ + "database", + "schema" + ] + }, + "S3Server": { + "type": "object", + "title": "S3Server", + "properties": { + "location": { + "type": "string", + "format": "uri", + "description": "S3 URL, starting with `s3://`", + "examples": [ + "s3://datacontract-example-orders-latest/data/{model}/*.json" + ] + }, + "endpointUrl": { + "type": "string", + "format": "uri", + "description": "The server endpoint for S3-compatible servers.", + "examples": ["https://minio.example.com"] + }, + "format": { + "type": "string", + "enum": [ + "parquet", + "delta", + "json", + "csv" + ], + "description": "File format." + }, + "delimiter": { + "type": "string", + "enum": [ + "new_line", + "array" + ], + "description": "Only for format = json. How multiple json documents are delimited within one file" + } + }, + "required": [ + "location" + ] + }, + "SftpServer": { + "type": "object", + "title": "SftpServer", + "properties": { + "location": { + "type": "string", + "format": "uri", + "pattern": "^sftp://.*", + "description": "SFTP URL, starting with `sftp://`", + "examples": [ + "sftp://123.123.12.123/{model}/*.json" + ] + }, + "format": { + "type": "string", + "enum": [ + "parquet", + "delta", + "json", + "csv" + ], + "description": "File format." + }, + "delimiter": { + "type": "string", + "enum": [ + "new_line", + "array" + ], + "description": "Only for format = json. How multiple json documents are delimited within one file" + } + }, + "required": [ + "location" + ] + }, + "SnowflakeServer": { + "type": "object", + "title": "SnowflakeServer", + "properties": { + "host": { + "type": "string", + "description": "The host to the Snowflake server" + }, + "port": { + "type": "integer", + "description": "The port to the Snowflake server." + }, + "account": { + "type": "string", + "description": "The Snowflake account used by the server." + }, + "database": { + "type": "string", + "description": "The name of the database." + }, + "schema": { + "type": "string", + "description": "The name of the schema." + }, + "warehouse": { + "type": "string", + "description": "The name of the cluster of resources that is a Snowflake virtual warehouse." + } + }, + "required": [ + "account", + "database", + "schema" + ] + }, + "SqlserverServer": { + "type": "object", + "title": "SqlserverServer", + "properties": { + "host": { + "type": "string", + "description": "The host to the database server", + "examples": [ + "localhost" + ] + }, + "port": { + "type": "integer", + "description": "The port to the database server.", + "default": 1433, + "examples": [ + 1433 + ] + }, + "database": { + "type": "string", + "description": "The name of the database.", + "examples": [ + "database" + ] + }, + "schema": { + "type": "string", + "description": "The name of the schema in the database.", + "examples": [ + "dbo" + ] + } + }, + "required": [ + "host", + "database", + "schema" + ] + }, + "SynapseServer": { + "type": "object", + "title": "SynapseServer", + "properties": { + "host": { + "type": "string", + "description": "The host of the Synapse server." + }, + "port": { + "type": "integer", + "description": "The port of the Synapse server." + }, + "database": { + "type": "string", + "description": "The name of the database." + } + }, + "required": [ + "host", + "port", + "database" + ] + }, + "TrinoServer": { + "type": "object", + "title": "TrinoServer", + "properties": { + "host": { + "type": "string", + "description": "The Trino host URL.", + "examples": [ + "localhost" + ] + }, + "port": { + "type": "integer", + "description": "The Trino port." + }, + "catalog": { + "type": "string", + "description": "The name of the catalog.", + "examples": [ + "hive" + ] + }, + "schema": { + "type": "string", + "description": "The name of the schema in the database.", + "examples": [ + "my_schema" + ] + } + }, + "required": [ + "host", + "port", + "catalog", + "schema" + ] + }, + "VerticaServer": { + "type": "object", + "title": "VerticaServer", + "properties": { + "host": { + "type": "string", + "description": "The host of the Vertica server." + }, + "port": { + "type": "integer", + "description": "The port of the Vertica server." + }, + "database": { + "type": "string", + "description": "The name of the database." + }, + "schema": { + "type": "string", + "description": "The name of the schema." + } + }, + "required": [ + "host", + "port", + "database", + "schema" + ] + } + }, + "SchemaElement": { "type": "object", "properties": { - "table": { + "name": { "type": "string", - "description": "Name of the table being cataloged; the value should only contain the table name. Do not include the project or dataset name in the value." + "description": "Name of the element." }, - "physicalName": { + "physicalType": { "type": "string", - "description": "Physical name of the table, default value is table name + version separated by underscores, as `table_1_2_0`.", - "examples": ["table_1_2_0"] + "description": "The physical element data type in the data source.", + "examples": ["table", "view", "topic", "file"] }, - "priorTableName": { + "description": { "type": "string", - "description": "Name of the previous version of the dataset, if applicable." + "description": "Description of the element." }, - "description": { + "businessName": { "type": "string", - "description": "Description of the dataset." + "description": "The business name of the element." }, "authoritativeDefinitions": { "$ref": "#/$defs/AuthoritativeDefinitions" }, - "dataGranularity": { + "tags": { + "$ref": "#/$defs/Tags" + }, + "customProperties": { + "$ref": "#/$defs/CustomProperties" + } + } + }, + "SchemaObject": { + "type": "object", + "properties": { + "logicalType": { + "type": "string", + "description": "The logical element data type.", + "enum": ["object"] + }, + "physicalName": { + "type": "string", + "description": "Physical name.", + "examples": ["table_1_2_0"] + }, + "dataGranularityDescription": { "type": "string", - "description": "Granular level of the data in the table. Example would be `pmt_txn_id`.", - "examples": ["pmt_txn_id"] + "description": "Granular level of the data in the object.", + "examples": ["Aggregation by country"] }, - "columns": { + "properties": { "type": "array", - "description": "Array. A list of columns in the table.", + "description": "A list of properties for the object.", "items": { - "$ref": "#/$defs/Column" + "$ref": "#/$defs/SchemaProperty" } }, "quality": { - "type": "array", - "description": "Data quality rules with all the relevant information for rule setup and execution.", - "items": { - "$ref": "#/$defs/DataQuality" - } + "$ref": "#/$defs/DataQualityChecks" } }, - "required": ["table"] + "allOf": [ + { + "$ref": "#/$defs/SchemaElement" + } + ], + "required": ["name"], + "unevaluatedProperties": false }, - "Column": { + "SchemaBaseProperty": { "type": "object", "properties": { - "column": { - "type": "string", - "description": "The name of the column." - }, - "isPrimaryKey": { + "primaryKey": { "type": "boolean", - "description": "Boolean value specifying whether the column is primary or not. Default is false." + "description": "Boolean value specifying whether the element is primary or not. Default is false." }, "primaryKeyPosition": { "type": "integer", "default": -1, - "description": "If column is a primary key, the position of the primary key column. Starts from 1. Example of `account_id, name` being primary key columns, `account_id` has primaryKeyPosition 1 and `name` primaryKeyPosition 2. Default to -1." - }, - "businessName": { - "type": "string", - "description": "The business name of the column." + "description": "If element is a primary key, the position of the primary key element. Starts from 1. Example of `account_id, name` being primary key columns, `account_id` has primaryKeyPosition 1 and `name` primaryKeyPosition 2. Default to -1." }, "logicalType": { "type": "string", - "description": "The logical column datatype." + "description": "The logical element data type.", + "enum": ["string", "date", "number", "integer", "object", "array", "boolean"] }, - "physicalType": { - "type": "string", - "description": "The physical column datatype." + "logicalTypeOptions": { + "type": "object", + "description": "Additional optional metadata to describe the logical type." }, - "description": { + "physicalType": { "type": "string", - "description": "Description of the column." + "description": "The physical element data type in the data source. For example, VARCHAR(2), DOUBLE, INT." }, - "isNullable": { + "required": { "type": "boolean", "default": false, - "description": "Indicates if the column may contain Null values; possible values are true and false. Default is false." + "description": "Indicates if the element may contain Null values; possible values are true and false. Default is false." }, - "isUnique": { + "unique": { "type": "boolean", "default": false, - "description": "Indicates if the column contains unique values; possible values are true and false. Default is false." + "description": "Indicates if the element contains unique values; possible values are true and false. Default is false." }, - "partitionStatus": { + "partitioned": { "type": "boolean", "default": false, - "description": "Indicates if the column is partitioned; possible values are true and false." + "description": "Indicates if the element is partitioned; possible values are true and false." }, "partitionKeyPosition": { "type": "integer", "default": -1, - "description": "If column is used for partitioning, the position of the partition column. Starts from 1. Example of `country, year` being partition columns, `country` has partitionKeyPosition 1 and `year` partitionKeyPosition 2. Default to -1." - }, - "clusterStatus": { - "type": "boolean", - "default": false, - "description": "Indicates of the column is clustered; possible values are true and false." - }, - "clusterKeyPosition": { - "type": "integer", - "default": -1, - "description": "If column is used for clustering, the position of the cluster column. Starts from 1. Example of `year, date` being cluster columns, `year` has clusterKeyPosition 1 and `date` clusterKeyPosition 2. Default to -1." + "description": "If element is used for partitioning, the position of the partition element. Starts from 1. Example of `country, year` being partition columns, `country` has partitionKeyPosition 1 and `year` partitionKeyPosition 2. Default to -1." }, "classification": { "type": "string", - "description": "Can be anything, like confidential, restricted, and public to more advanced categorization. Some companies like PayPal, use data classification indicating the class of data in the column; expected values are 1, 2, 3, 4, or 5.", + "description": "Can be anything, like confidential, restricted, and public to more advanced categorization. Some companies like PayPal, use data classification indicating the class of data in the element; expected values are 1, 2, 3, 4, or 5.", "examples": ["confidential", "restricted", "public"] }, - "authoritativeDefinitions": { - "$ref": "#/$defs/AuthoritativeDefinitions" - }, - "encryptedColumnName": { + "encryptedName": { "type": "string", - "description": "The column name within the table that contains the encrypted column value. For example, unencrypted column `email_address` might have an encryptedColumnName of `email_address_encrypt`." + "description": "The element name within the dataset that contains the encrypted element value. For example, unencrypted element `email_address` might have an encryptedName of `email_address_encrypt`." }, - "transformSourceTables": { + "transformSourceObjects": { "type": "array", - "description": "List of sources used in column transformation.", + "description": "List of objects in the data source used in the transformation.", "items": { "type": "string" } }, "transformLogic": { "type": "string", - "description": "Logic used in the column transformation." + "description": "Logic used in the element transformation." }, "transformDescription": { "type": "string", "description": "Describes the transform logic in very simple terms." }, - "sampleValues": { + "examples": { "type": "array", - "description": "List of sample column values.", + "description": "List of sample element values.", "items": { - "type": "string" + "$ref": "#/$defs/AnyType" } }, - "criticalDataElementStatus": { + "criticalDataElement": { "type": "boolean", "default": false, "description": "True or false indicator; If element is considered a critical data element (CDE) then true else false." }, - "tags": { + "quality": { + "$ref": "#/$defs/DataQualityChecks" + } + }, + "allOf": [ + { + "$ref": "#/$defs/SchemaElement" + }, + { + "if": { + "properties": { + "logicalType": { + "const": "string" + } + } + }, + "then": { + "properties": { + "logicalTypeOptions": { + "type": "object", + "properties": { + "minLength": { + "type": "integer", + "minimum": 0, + "description": "Minimum length of the string." + }, + "maxLength": { + "type": "integer", + "minimum": 0, + "description": "Maximum length of the string." + }, + "pattern": { + "type": "string", + "description": "Regular expression pattern to define valid value. Follows regular expression syntax from ECMA-262 (https://262.ecma-international.org/5.1/#sec-15.10.1)." + }, + "format": { + "type": "string", + "examples": ["password", "byte", "binary", "email", "uuid", "uri", "hostname", "ipv4", "ipv6"], + "description": "Provides extra context about what format the string follows." + } + }, + "additionalProperties": false + } + } + } + }, + { + "if": { + "properties": { + "logicalType": { + "const": "date" + } + } + }, + "then": { + "properties": { + "logicalTypeOptions": { + "type": "object", + "properties": { + "format": { + "type": "string", + "examples": ["yyyy-MM-dd", "yyyy-MM-dd HH:mm:ss", "HH:mm:ss"], + "description": "Format of the date. Follows the format as prescribed by [JDK DateTimeFormatter](https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html). For example, format 'yyyy-MM-dd'." + }, + "exclusiveMaximum": { + "type": "boolean", + "default": false, + "description": "If set to true, all values are strictly less than the maximum value (values < maximum). Otherwise, less than or equal to the maximum value (values <= maximum)." + }, + "maximum": { + "type": "string", + "description": "All date values are less than or equal to this value (values <= maximum)." + }, + "exclusiveMinimum": { + "type": "boolean", + "default": false, + "description": "If set to true, all values are strictly greater than the minimum value (values > minimum). Otherwise, greater than or equal to the minimum value (values >= minimum)." + }, + "minimum": { + "type": "string", + "description": "All date values are greater than or equal to this value (values >= minimum)." + } + }, + "additionalProperties": false + } + } + } + }, + { + "if": { + "anyOf": [ + { + "properties": { + "logicalType": { + "const": "integer" + } + } + } + ] + }, + "then": { + "properties": { + "logicalTypeOptions": { + "type": "object", + "properties": { + "multipleOf": { + "type": "number", + "exclusiveMinimum": 0, + "description": "Values must be multiples of this number. For example, multiple of 5 has valid values 0, 5, 10, -5." + }, + "maximum": { + "type": "number", + "description": "All values are less than or equal to this value (values <= maximum)." + }, + "exclusiveMaximum": { + "type": "boolean", + "default": false, + "description": "If set to true, all values are strictly less than the maximum value (values < maximum). Otherwise, less than or equal to the maximum value (values <= maximum)." + }, + "minimum": { + "type": "number", + "description": "All values are greater than or equal to this value (values >= minimum)." + }, + "exclusiveMinimum": { + "type": "boolean", + "default": false, + "description": "If set to true, all values are strictly greater than the minimum value (values > minimum). Otherwise, greater than or equal to the minimum value (values >= minimum)." + }, + "format": { + "type": "string", + "default": "i32", + "description": "Format of the value in terms of how many bits of space it can use and whether it is signed or unsigned (follows the Rust integer types).", + "enum": ["i8", "i16", "i32", "i64", "i128", "u8", "u16", "u32", "u64", "u128"] + } + }, + "additionalProperties": false + } + } + } + }, + { + "if": { + "anyOf": [ + { + "properties": { + "logicalType": { + "const": "number" + } + } + } + ] + }, + "then": { + "properties": { + "logicalTypeOptions": { + "type": "object", + "properties": { + "multipleOf": { + "type": "number", + "exclusiveMinimum": 0, + "description": "Values must be multiples of this number. For example, multiple of 5 has valid values 0, 5, 10, -5." + }, + "maximum": { + "type": "number", + "description": "All values are less than or equal to this value (values <= maximum)." + }, + "exclusiveMaximum": { + "type": "boolean", + "default": false, + "description": "If set to true, all values are strictly less than the maximum value (values < maximum). Otherwise, less than or equal to the maximum value (values <= maximum)." + }, + "minimum": { + "type": "number", + "description": "All values are greater than or equal to this value (values >= minimum)." + }, + "exclusiveMinimum": { + "type": "boolean", + "default": false, + "description": "If set to true, all values are strictly greater than the minimum value (values > minimum). Otherwise, greater than or equal to the minimum value (values >= minimum)." + }, + "format": { + "type": "string", + "default": "i32", + "description": "Format of the value in terms of how many bits of space it can use (follows the Rust float types).", + "enum": ["f32", "f64"] + } + }, + "additionalProperties": false + } + } + } + }, + { + "if": { + "properties": { + "logicalType": { + "const": "object" + } + } + }, + "then": { + "properties": { + "logicalTypeOptions": { + "type": "object", + "properties": { + "maxProperties": { + "type": "integer", + "minimum": 0, + "description": "Maximum number of properties." + }, + "minProperties": { + "type": "integer", + "minimum": 0, + "default": 0, + "description": "Minimum number of properties." + }, + "required": { + "type": "array", + "items": { + "type": "string" + }, + "minItems": 1, + "uniqueItems": true, + "description": "Property names that are required to exist in the object." + } + }, + "additionalProperties": false + }, + "properties": { + "type": "array", + "description": "A list of properties for the object.", + "items": { + "$ref": "#/$defs/SchemaProperty" + } + } + } + } + }, + { + "if": { + "properties": { + "logicalType": { + "const": "array" + } + } + }, + "then": { + "properties": { + "logicalTypeOptions": { + "type": "object", + "properties": { + "maxItems": { + "type": "integer", + "minimum": 0, + "description": "Maximum number of items." + }, + "minItems": { + "type": "integer", + "minimum": 0, + "default": 0, + "description": "Minimum number of items" + }, + "uniqueItems": { + "type": "boolean", + "default": false, + "description": "If set to true, all items in the array are unique." + } + }, + "additionalProperties": false + }, + "items": { + "$ref": "#/$defs/SchemaItemProperty", + "description": "List of items in an array (only applicable when `logicalType: array`)." + } + } + } + } + ] + }, + "SchemaProperty": { + "type": "object", + "$ref": "#/$defs/SchemaBaseProperty", + "required": ["name"], + "unevaluatedProperties": false + }, + "SchemaItemProperty": { + "type": "object", + "$ref": "#/$defs/SchemaBaseProperty", + "properties": { + "properties": { "type": "array", - "description": "A list of tags that may be assigned to the dataset, table or column; the tags keyword may appear at any level.", + "description": "A list of properties for the object.", "items": { - "type": "string" + "$ref": "#/$defs/SchemaProperty" } } }, - "required": ["column", "logicalType", "physicalType"] + "unevaluatedProperties": false + }, + "Tags": { + "type": "array", + "description": "A list of tags that may be assigned to the elements (object or property); the tags keyword may appear at any level.", + "items": { + "type": "string" + } }, "DataQuality": { "type": "object", "properties": { - "code": { - "type": "string", - "description": "The Rosewall data quality code(s) indicating which quality checks need to be performed at the dataset, table or column level; The quality keyword may appear at any level; Some quality checks require parameters such so the check can be completed (eg, list of fields used to identify a distinct row) therefore some quality checks may be followed by a single value or an array; See appendix for link to quality checks." + "authoritativeDefinitions": { + "$ref": "#/$defs/AuthoritativeDefinitions" }, - "templateName": { + "businessImpact": { "type": "string", - "description": "The template name which indicates what is the equivalent template from the tool." + "description": "Consequences of the rule failure.", + "examples": ["operational", "regulatory"] + }, + "customProperties": { + "type": "array", + "description": "Additional properties required for rule execution.", + "items": { + "$ref": "#/$defs/CustomProperty" + } }, "description": { "type": "string", "description": "Describe the quality check to be completed." }, - "toolName": { + "dimension": { "type": "string", - "description": "Name of the tool used to complete the quality check; Most will be Elevate initially." + "description": "The key performance indicator (KPI) or dimension for data quality.", + "enum": ["accuracy", "completeness", "conformity", "consistency", "coverage", "timeliness", "uniqueness"] }, - "toolRuleName": { + "method": { "type": "string", - "description": "Name of the quality tool's rule created to complete the quality check." + "examples": ["reconciliation"] }, - "dimension": { + "name": { "type": "string", - "description": "The key performance indicator (KPI) or dimension for data quality." + "description": "Name of the data quality check." }, - "columns": { + "schedule": { "type": "string", - "description": "List of columns to be used in the quality check." + "description": "Rule execution schedule details.", + "examples": ["0 20 * * *"] }, - "column": { + "scheduler": { "type": "string", - "description": "To be used in lieu of quality.columns when only a single column is required for the quality check." + "description": "The name or type of scheduler used to start the data quality check.", + "examples": ["cron"] }, - "type": { + "severity": { "type": "string", - "description": "The type of quality check." + "description": "The severance of the quality rule.", + "examples": ["info", "warning", "error"] }, - "severity": { + "tags": { + "$ref": "#/$defs/Tags" + }, + "type": { "type": "string", - "description": "The severance of the quality rule." + "description": "The type of quality check. 'text' is human-readable text that describes the quality of the data. 'library' is a set of maintained predefined quality attributes such as row count or unique. 'sql' is an individual SQL query that returns a value that can be compared. 'custom' is quality attributes that are vendor-specific, such as Soda or Great Expectations.", + "enum": ["text", "library", "sql", "custom"], + "default": "library" }, - "businessImpact": { + "unit": { "type": "string", - "description": "Consequences of the rule failure." + "description": "Unit the rule is using, popular values are `rows` or `percent`, but any value is allowed.", + "examples": ["rows", "percent"] + } + }, + "allOf": [ + { + "if": { + "properties": { + "type": { + "const": "library" + } + } + }, + "then": { + "$ref": "#/$defs/DataQualityLibrary" + } + }, + { + "if": { + "properties": { + "type": { + "const": "sql" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/DataQualitySql" + } }, - "scheduleCronExpression": { + { + "if": { + "properties": { + "type": { + "const": "custom" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/DataQualityCustom" + } + } + ] + }, + "DataQualityChecks": { + "type": "array", + "description": "Data quality rules with all the relevant information for rule setup and execution.", + "items": { + "$ref": "#/$defs/DataQuality" + } + }, + "DataQualityLibrary": { + "type": "object", + "properties": { + "rule": { "type": "string", - "description": "Rule execution schedule details." + "description": "Define a data quality check based on the predefined rules as per ODCS.", + "examples": ["duplicateCount", "validValues", "rowCount"] }, - "customProperties": { + "mustBe": { + "description": "Must be equal to the value to be valid. When using numbers, it is equivalent to '='." + }, + "mustNotBe": { + "description": "Must not be equal to the value to be valid. When using numbers, it is equivalent to '!='." + }, + "mustBeGreaterThan": { + "type": "number", + "description": "Must be greater than the value to be valid. It is equivalent to '>'." + }, + "mustBeGreaterOrEqualTo": { + "type": "number", + "description": "Must be greater than or equal to the value to be valid. It is equivalent to '>='." + }, + "mustBeLessThan": { + "type": "number", + "description": "Must be less than the value to be valid. It is equivalent to '<'." + }, + "mustBeLessOrEqualTo": { + "type": "number", + "description": "Must be less than or equal to the value to be valid. It is equivalent to '<='." + }, + "mustBeBetween": { "type": "array", - "description": "Additional properties required for rule execution.", + "description": "Must be between the two numbers to be valid. Smallest number first in the array.", + "minItems": 2, + "maxItems": 2, + "uniqueItems": true, "items": { - "$ref": "#/$defs/CustomProperty" + "type": "number" + } + }, + "mustNotBeBetween": { + "type": "array", + "description": "Must not be between the two numbers to be valid. Smallest number first in the array.", + "minItems": 2, + "maxItems": 2, + "uniqueItems": true, + "items": { + "type": "number" } } }, - "required": ["templateName", "toolName"] + "required": ["rule"] + }, + "DataQualitySql": { + "type": "object", + "properties": { + "query": { + "type": "string", + "description": "Query string that adheres to the dialect of the provided server.", + "examples": ["SELECT COUNT(*) FROM ${table} WHERE ${column} IS NOT NULL"] + } + }, + "required": ["query"] + }, + "DataQualityCustom": { + "type": "object", + "properties": { + "engine": { + "type": "string", + "description": "Name of the engine which executes the data quality checks.", + "examples": ["soda", "great-expectations", "monte-carlo", "dbt"] + }, + "implementation": { + "oneOf": [ + { + "type": "string" + }, + { + "type": "object" + } + ] + } + }, + "required": ["engine", "implementation"] }, "AuthoritativeDefinitions": { "type": "array", - "description": "List of links to sources that provide more details on the table; examples would be a link to an external definition, a training video, a GitHub repo, Collibra, or another tool. Authoritative definitions follow the same structure in the standard.", + "description": "List of links to sources that provide more details on the dataset; examples would be a link to an external definition, a training video, a GitHub repo, Collibra, or another tool. Authoritative definitions follow the same structure in the standard.", "items": { "type": "object", "properties": { @@ -408,6 +2123,45 @@ "required": ["url", "type"] } }, + "Support": { + "type": "array", + "description": "Top level for support channels.", + "items": { + "$ref": "#/$defs/SupportItem" + } + }, + "SupportItem": { + "type": "object", + "properties": { + "channel": { + "type": "string", + "description": "Channel name or identifier." + }, + "url": { + "type": "string", + "description": "Access URL using normal [URL scheme](https://en.wikipedia.org/wiki/URL#Syntax) (https, mailto, etc.)." + }, + "description": { + "type": "string", + "description": "Description of the channel, free text." + }, + "tool": { + "type": "string", + "description": "Name of the tool, value can be `email`, `slack`, `teams`, `discord`, `ticket`, or `other`.", + "examples": ["email", "slack", "teams", "discord", "ticket", "other"] + }, + "scope": { + "type": "string", + "description": "Scope can be: `interactive`, `announcements`, `issues`.", + "examples": ["interactive", "announcements", "issues"] + }, + "invitationUrl": { + "type": "string", + "description": "Some tools uses invitation URL for requesting or subscribing. Follows the [URL scheme](https://en.wikipedia.org/wiki/URL#Syntax)." + } + }, + "required": ["channel", "url"] + }, "Pricing": { "type": "object", "properties": { @@ -425,28 +2179,30 @@ } } }, - "Stakeholder": { + "Team": { "type": "object", "properties": { "username": { "type": "string", - "description": "The stakeholder's username or email." + "description": "The user's username or email." }, "role": { "type": "string", - "description": "The stakeholder's job role; Examples might be owner, data steward. There is no limit on the role." + "description": "The user's job role; Examples might be owner, data steward. There is no limit on the role." }, "dateIn": { "type": "string", - "description": "The date when the user became a stakeholder." + "format": "date", + "description": "The date when the user joined the team." }, "dateOut": { "type": "string", - "description": "The date when the user ceased to be a stakeholder" + "format": "date", + "description": "The date when the user ceased to be part of the team." }, "replacedByUsername": { "type": "string", - "description": "The username of the user who replaced the stakeholder" + "description": "The username of the user who replaced the previous user." } } }, @@ -455,22 +2211,26 @@ "properties": { "role": { "type": "string", - "description": "Name of the IAM role that provides access to the dataset; the value will generally come directly from the \"BQ dataset to IAM roles mapping\" document." + "description": "Name of the IAM role that provides access to the dataset." + }, + "description": { + "type": "string", + "description": "Description of the IAM role and its permissions." }, "access": { "type": "string", - "description": "The type of access provided by the IAM role; the value will generally come directly from the \"BQ dataset to IAM roles mapping\" document." + "description": "The type of access provided by the IAM role." }, "firstLevelApprovers": { "type": "string", - "description": "The name(s) of the first level approver(s) of the role; the value will generally come directly from the \"BQ dataset to IAM roles mapping\" document." + "description": "The name(s) of the first-level approver(s) of the role." }, "secondLevelApprovers": { "type": "string", - "description": "The name(s) of the second level approver(s) of the role; the value will generally come directly from the \"BQ dataset to IAM roles mapping\" document." + "description": "The name(s) of the second-level approver(s) of the role." } }, - "required": ["role", "access"] + "required": ["role"] }, "ServiceLevelAgreementProperty": { "type": "object", @@ -480,34 +2240,36 @@ "description": "Specific property in SLA, check the periodic table. May requires units (more details to come)." }, "value": { - "oneOf": [ + "anyOf": [ { "type": "string" }, { "type": "number" - } - ], - "description": "Agreement value. The label will change based on the property itself." - }, - "valueExt": { - "oneOf": [ + }, { - "type": "string" + "type": "integer" }, { - "type": "number" + "type": "boolean" + }, + { + "type": "null" } ], + "description": "Agreement value. The label will change based on the property itself." + }, + "valueExt": { + "$ref": "#/$defs/AnyNonCollectionType", "description": "Extended agreement value. The label will change based on the property itself." }, "unit": { "type": "string", "description": "**d**, day, days for days; **y**, yr, years for years, etc. Units use the ISO standard." }, - "column": { + "element": { "type": "string", - "description": "Column(s) to check on. Multiple columns should be extremely rare and, if so, separated by commas." + "description": "Element(s) to check on. Multiple elements should be extremely rare and, if so, separated by commas." }, "driver": { "type": "string", @@ -517,6 +2279,13 @@ }, "required": ["property", "value"] }, + "CustomProperties": { + "type": "array", + "description": "A list of key/value pairs for custom properties.", + "items": { + "$ref": "#/$defs/CustomProperty" + } + }, "CustomProperty": { "type": "object", "properties": { @@ -525,10 +2294,54 @@ "description": "The name of the key. Names should be in camel case–the same as if they were permanent properties in the contract." }, "value": { - "type": "object", + "$ref": "#/$defs/AnyType", "description": "The value of the key." } } + }, + "AnyType": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "number" + }, + { + "type": "integer" + }, + { + "type": "boolean" + }, + { + "type": "null" + }, + { + "type": "array" + }, + { + "type": "object" + } + ] + }, + "AnyNonCollectionType": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "number" + }, + { + "type": "integer" + }, + { + "type": "boolean" + }, + { + "type": "null" + } + ] } } -} \ No newline at end of file +} diff --git a/schema/odcs-json-schema-v2.2.1.json b/schema/odcs-json-schema-v2.2.1.json new file mode 100644 index 0000000..0c01006 --- /dev/null +++ b/schema/odcs-json-schema-v2.2.1.json @@ -0,0 +1,523 @@ +{ + "$schema": "https://json-schema.org/draft/2019-09/schema", + "title": "Open Data Contract Standard (OCDS)", + "description": "An open data contract specification to establish agreement between data producers and consumers.", + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Current version of the data contract." + }, + "kind": { + "type": "string", + "default": "DataContract", + "description": "The kind of file this is. Valid value is `DataContract`.", + "enum": ["DataContract"] + }, + "apiVersion": { + "type": "string", + "default": "v2.3.0", + "description": "Version of the standard used to build data contract. Default value is v2.3.0.", + "pattern": "^v[0-9]+\\.[0-9]+\\.[0-9]+" + }, + "uuid": { + "type": "string", + "description": "A unique identifier used to reduce the risk of dataset name collisions; initially the UUID will be created using a UUID generator tool ([example](https://www.uuidgenerator.net/)). However, we may want to develop a method that accepts a seed value using a combination of fields–such as name, kind and source–to create a repeatable value." + }, + "datasetKind": { + "type": "string", + "description": "The kind of dataset being cataloged; Expected values are `virtualDataset` or `managedDataset`.", + "examples": ["virtualDataset", "managedDataset"] + }, + "userConsumptionMode": { + "type": "string", + "description": "List of data modes for which the dataset may be used. Expected sample values might be `analytical` or `operational`.
Note: in the future, this will probably be replaced by ports.", + "examples": ["analytical", "operational"] + }, + "type": { + "type": "string", + "description": "Identifies the types of objects in the dataset. For BigQuery or any other database, the expected value would be Tables.", + "examples": ["Tables"] + }, + "tenant": { + "type": "string", + "description": "Indicates the property the data is primarily associated with. Value is case insensitive." + }, + "tags": { + "type": "array", + "description": "A list of tags that may be assigned to the dataset, table or column; the `tags` keyword may appear at any level.", + "items": { + "type": "string" + } + }, + "status": { + "type": "string", + "description": "Current status of the dataset. Valid values are `production`, `test`, or `development`.", + "examples": ["production", "test", "development"] + }, + "sourceSystem": { + "type": "string", + "description": "The system where the dataset resides. Expected value can be BigQuery.", + "examples": ["BigQuery"] + }, + "sourcePlatform": { + "type": "string", + "description": "The platform where the dataset resides. Expected value is GoogleCloudPlatform, IBMCloud, Azure...", + "examples": ["GoogleCloudPlatform", "IBMCloud", "Azure", "AWS"] + }, + "server": { + "type": "string", + "description": "The server where the dataset resides." + }, + "quantumName": { + "type": "string", + "description": "The name of the data quantum or data product." + }, + "productSlackChannel": { + "type": "string", + "description": "Slack channel of the team responsible for maintaining the dataset." + }, + "productFeedbackUrl": { + "type": "string", + "description": "The URL for submitting feedback to the team responsible for maintaining the dataset." + }, + "productDl": { + "type": "string", + "description": "The email distribution list (DL) of the persons or team responsible for maintaining the dataset." + }, + "username": { + "type": "string", + "description": "User credentials for connecting to the dataset; how the credentials will be stored/passed is outside of the scope of the contract." + }, + "password": { + "type": "string", + "description": "User credentials for connecting to the dataset; how the credentials will be stored/passed is out of the scope of this contract." + }, + "driverVersion": { + "type": "string", + "description": "The version of the connection driver to be used to connect to the dataset." + }, + "driver": { + "type": "string", + "description": "The connection driver required to connect to the dataset." + }, + "description": { + "type": "object", + "description": "High level description of the dataset.", + "properties": { + "usage": { + "type": "string", + "description": "Intended usage of the dataset." + }, + "purpose": { + "type": "string", + "description": "Purpose of the dataset." + }, + "limitations": { + "type": "string", + "description": "Limitations of the dataset." + } + } + }, + "project": { + "type": "string", + "description": "Associated project name, can be used for billing or administrative purpose. Used to be datasetProject." + }, + "datasetName": { + "type": "string", + "description": "May be required in cloud instance like GCP BigQuery dataset name." + }, + "datasetDomain": { + "type": "string", + "description": "Name of the logical domain dataset the contract describes. This field is only required for output data contracts.", + "examples": ["imdb_ds_aggregate", "receiver_profile_out", "transaction_profile_out"] + }, + "database": { + "type": "string", + "description": "The database where the dataset resides." + }, + "dataset": { + "type": "array", + "items": { + "$ref": "#/$defs/Dataset" + } + }, + "price": { + "$ref": "#/$defs/Pricing" + }, + "stakeholders": { + "type": "array", + "items": { + "$ref": "#/$defs/Stakeholder" + } + }, + "roles": { + "type": "array", + "description": "A list of roles that will provide user access to the dataset.", + "items": { + "$ref": "#/$defs/Role" + } + }, + "slaDefaultColumn": { + "type": "string", + "description": "Columns (using the Table.Column notation) to do the checks on. By default, it is the partition column." + }, + "slaProperties": { + "type": "array", + "description": "A list of key/value pairs for SLA specific properties. There is no limit on the type of properties (more details to come).", + "items": { + "$ref": "#/$defs/ServiceLevelAgreementProperty" + } + } + }, + "required": ["version", "kind", "uuid", "type", "status", "dataset", "datasetName", "quantumName"], + "$defs": { + "Dataset": { + "type": "object", + "properties": { + "table": { + "type": "string", + "description": "Name of the table being cataloged; the value should only contain the table name. Do not include the project or dataset name in the value." + }, + "physicalName": { + "type": "string", + "description": "Physical name of the table, default value is table name + version separated by underscores, as `table_1_2_0`.", + "examples": ["table_1_2_0"] + }, + "priorTableName": { + "type": "string", + "description": "Name of the previous version of the dataset, if applicable." + }, + "description": { + "type": "array", + "description": "List of links to sources that provide more detail on column logic or values; examples would be URL to a GitHub repo, Collibra, on another tool.", + "items": { + "type": "string" + } + }, + "authoritativeDefinitions": { + "$ref": "#/$defs/AuthoritativeDefinitions" + }, + "dataGranularity": { + "type": "string", + "description": "Granular level of the data in the table. Example would be `pmt_txn_id`.", + "examples": ["pmt_txn_id"] + }, + "columns": { + "type": "array", + "description": "Array. A list of columns in the table.", + "items": { + "$ref": "#/$defs/Column" + } + }, + "quality": { + "type": "array", + "description": "Data quality rules with all the relevant information for rule setup and execution.", + "items": { + "$ref": "#/$defs/DataQuality" + } + } + }, + "required": ["table"] + }, + "Column": { + "type": "object", + "properties": { + "column": { + "type": "string", + "description": "The name of the column." + }, + "isPrimaryKey": { + "type": "string", + "description": "Boolean value specifying whether the column is primary or not. Default is false." + }, + "primaryKeyPosition": { + "type": "integer", + "default": -1, + "description": "If column is a primary key, the position of the primary key column. Starts from 1. Example of `account_id, name` being primary key columns, `account_id` has primaryKeyPosition 1 and `name` primaryKeyPosition 2. Default to -1." + }, + "businessName": { + "type": "string", + "description": "The business name of the column." + }, + "logicalType": { + "type": "string", + "description": "The logical column datatype." + }, + "physicalType": { + "type": "string", + "description": "The physical column datatype." + }, + "description": { + "type": "string", + "description": "Description of the column." + }, + "isNullable": { + "type": "boolean", + "default": false, + "description": "Indicates if the column may contain Null values; possible values are true and false. Default is false." + }, + "isUnique": { + "type": "boolean", + "default": false, + "description": "Indicates if the column contains unique values; possible values are true and false. Default is false." + }, + "partitionStatus": { + "type": "boolean", + "default": false, + "description": "Indicates if the column is partitioned; possible values are true and false." + }, + "partitionKeyPosition": { + "type": "integer", + "default": -1, + "description": "If column is used for partitioning, the position of the partition column. Starts from 1. Example of `country, year` being partition columns, `country` has partitionKeyPosition 1 and `year` partitionKeyPosition 2. Default to -1." + }, + "clusterStatus": { + "type": "boolean", + "default": false, + "description": "Indicates of the column is clustered; possible values are true and false." + }, + "clusterKeyPosition": { + "type": "integer", + "default": -1, + "description": "If column is used for clustering, the position of the cluster column. Starts from 1. Example of `year, date` being cluster columns, `year` has clusterKeyPosition 1 and `date` clusterKeyPosition 2. Default to -1." + }, + "classification": { + "type": "string", + "description": "Can be anything, like confidential, restricted, and public to more advanced categorization. Some companies like PayPal, use data classification indicating the class of data in the column; expected values are 1, 2, 3, 4, or 5.", + "examples": ["confidential", "restricted", "public"] + }, + "authoritativeDefinitions": { + "$ref": "#/$defs/AuthoritativeDefinitions" + }, + "encryptedColumnName": { + "type": "string", + "description": "The column name within the table that contains the encrypted column value. For example, unencrypted column `email_address` might have an encryptedColumnName of `email_address_encrypt`." + }, + "transformSourceTables": { + "type": "array", + "description": "List of sources used in column transformation.", + "items": { + "type": "string" + } + }, + "transformLogic": { + "type": "string", + "description": "Logic used in the column transformation." + }, + "transformDescription": { + "type": "string", + "description": "Describes the transform logic in very simple terms." + }, + "sampleValues": { + "type": "array", + "description": "List of sample column values.", + "items": { + "type": "string" + } + }, + "criticalDataElementStatus": { + "type": "boolean", + "default": false, + "description": "True or false indicator; If element is considered a critical data element (CDE) then true else false." + }, + "tags": { + "type": "array", + "description": "A list of tags that may be assigned to the dataset, table or column; the tags keyword may appear at any level.", + "items": { + "type": "string" + } + } + }, + "required": ["column", "logicalType", "physicalType"] + }, + "DataQuality": { + "type": "object", + "properties": { + "code": { + "type": "string", + "description": "The Rosewall data quality code(s) indicating which quality checks need to be performed at the dataset, table or column level; The quality keyword may appear at any level; Some quality checks require parameters such so the check can be completed (eg, list of fields used to identify a distinct row) therefore some quality checks may be followed by a single value or an array; See appendix for link to quality checks." + }, + "templateName": { + "type": "string", + "description": "The template name which indicates what is the equivalent template from the tool." + }, + "description": { + "type": "string", + "description": "Describe the quality check to be completed." + }, + "toolName": { + "type": "string", + "description": "Name of the tool used to complete the quality check; Most will be Elevate initially." + }, + "toolRuleName": { + "type": "string", + "description": "Name of the quality tool's rule created to complete the quality check." + }, + "dimension": { + "type": "string", + "description": "The key performance indicator (KPI) or dimension for data quality." + }, + "columns": { + "type": "string", + "description": "List of columns to be used in the quality check." + }, + "column": { + "type": "string", + "description": "To be used in lieu of quality.columns when only a single column is required for the quality check." + }, + "type": { + "type": "string", + "description": "The type of quality check." + }, + "severity": { + "type": "string", + "description": "The severance of the quality rule." + }, + "businessImpact": { + "type": "string", + "description": "Consequences of the rule failure." + }, + "scheduleCronExpression": { + "type": "string", + "description": "Rule execution schedule details." + }, + "customProperties": { + "type": "array", + "description": "Additional properties required for rule execution.", + "items": { + "$ref": "#/$defs/CustomProperty" + } + } + }, + "required": ["templateName", "toolName"] + }, + "AuthoritativeDefinitions": { + "type": "array", + "description": "List of links to sources that provide more details on the table; examples would be a link to an external definition, a training video, a GitHub repo, Collibra, or another tool. Authoritative definitions follow the same structure in the standard.", + "items": { + "type": "object", + "properties": { + "url": { + "type": "string", + "description": "URL to the authority." + }, + "type": { + "type": "string", + "description": "Type of definition for authority: v2.3 adds standard values: `businessDefinition`, `transformationImplementation`, `videoTutorial`, `tutorial`, and `implementation`.", + "examples": ["businessDefinition", "transformationImplementation", "videoTutorial", "tutorial", "implementation"] + } + }, + "required": ["url", "type"] + } + }, + "Pricing": { + "type": "object", + "properties": { + "priceAmount": { + "type": "string", + "description": "Subscription price per unit of measure in `priceUnit`." + }, + "priceCurrency": { + "type": "string", + "description": "Currency of the subscription price in `price.priceAmount`." + }, + "priceUnit": { + "type": "string", + "description": "The unit of measure for calculating cost. Examples megabyte, gigabyte." + } + } + }, + "Stakeholder": { + "type": "object", + "properties": { + "username": { + "type": "string", + "description": "The stakeholder's username or email." + }, + "role": { + "type": "string", + "description": "The stakeholder's job role; Examples might be owner, data steward. There is no limit on the role." + }, + "dateIn": { + "type": "string", + "description": "The date when the user became a stakeholder." + }, + "dateOut": { + "type": "string", + "description": "The date when the user ceased to be a stakeholder" + }, + "replacedByUsername": { + "type": "string", + "description": "The username of the user who replaced the stakeholder" + } + } + }, + "Role": { + "type": "object", + "properties": { + "role": { + "type": "string", + "description": "Name of the IAM role that provides access to the dataset; the value will generally come directly from the \"BQ dataset to IAM roles mapping\" document." + }, + "access": { + "type": "string", + "description": "The type of access provided by the IAM role; the value will generally come directly from the \"BQ dataset to IAM roles mapping\" document." + }, + "firstLevelApprovers": { + "type": "string", + "description": "The name(s) of the first level approver(s) of the role; the value will generally come directly from the \"BQ dataset to IAM roles mapping\" document." + }, + "secondLevelApprovers": { + "type": "string", + "description": "The name(s) of the second level approver(s) of the role; the value will generally come directly from the \"BQ dataset to IAM roles mapping\" document." + } + }, + "required": ["role", "access"] + }, + "ServiceLevelAgreementProperty": { + "type": "object", + "properties": { + "property": { + "type": "string", + "description": "Specific property in SLA, check the periodic table. May requires units (more details to come)." + }, + "value": { + "type": "string", + "description": "Agreement value. The label will change based on the property itself." + }, + "valueExt": { + "type": "string", + "description": "Extended agreement value. The label will change based on the property itself." + }, + "unit": { + "type": "string", + "description": "**d**, day, days for days; **y**, yr, years for years, etc. Units use the ISO standard." + }, + "column": { + "type": "string", + "description": "Column(s) to check on. Multiple columns should be extremely rare and, if so, separated by commas." + }, + "driver": { + "type": "string", + "description": "Describes the importance of the SLA from the list of: `regulatory`, `analytics`, or `operational`.", + "examples": ["regulatory", "analytics", "operational"] + } + }, + "required": ["property", "value"] + }, + "CustomProperty": { + "type": "object", + "properties": { + "property": { + "type": "string", + "description": "The name of the key. Names should be in camel case–the same as if they were permanent properties in the contract." + }, + "value": { + "type": "object", + "description": "The value of the key." + } + } + } + } +} \ No newline at end of file diff --git a/schema/odcs-json-schema-v3.0.0.json b/schema/odcs-json-schema-v3.0.0.json new file mode 100644 index 0000000..1ecb880 --- /dev/null +++ b/schema/odcs-json-schema-v3.0.0.json @@ -0,0 +1,2347 @@ +{ + "$schema": "https://json-schema.org/draft/2019-09/schema", + "title": "Open Data Contract Standard (ODCS)", + "description": "An open data contract specification to establish agreement between data producers and consumers.", + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Current version of the data contract." + }, + "kind": { + "type": "string", + "default": "DataContract", + "description": "The kind of file this is. Valid value is `DataContract`.", + "enum": ["DataContract"] + }, + "apiVersion": { + "type": "string", + "default": "v3.0.0", + "description": "Version of the standard used to build data contract. Default value is v3.0.0.", + "enum": ["v3.0.0", "v2.2.2", "v2.2.1", "v2.2.0"] + }, + "id": { + "type": "string", + "description": "A unique identifier used to reduce the risk of dataset name collisions, such as a UUID." + }, + "name": { + "type": "string", + "description": "Name of the data contract." + }, + "tenant": { + "type": "string", + "description": "Indicates the property the data is primarily associated with. Value is case insensitive." + }, + "tags": { + "$ref": "#/$defs/Tags" + }, + "status": { + "type": "string", + "description": "Current status of the dataset. Valid values are `production`, `test`, or `development`.", + "examples": ["production", "test", "development"] + }, + "servers": { + "type": "array", + "description": "List of servers where the datasets reside.", + "items": { + "$ref": "#/$defs/Server" + } + }, + "dataProduct": { + "type": "string", + "description": "The name of the data product." + }, + "description": { + "type": "object", + "description": "High level description of the dataset.", + "properties": { + "usage": { + "type": "string", + "description": "Intended usage of the dataset." + }, + "purpose": { + "type": "string", + "description": "Purpose of the dataset." + }, + "limitations": { + "type": "string", + "description": "Limitations of the dataset." + } + } + }, + "domain": { + "type": "string", + "description": "Name of the logical data domain.", + "examples": ["imdb_ds_aggregate", "receiver_profile_out", "transaction_profile_out"] + }, + "schema": { + "type": "array", + "description": "A list of elements within the schema to be cataloged.", + "items": { + "$ref": "#/$defs/SchemaObject" + } + }, + "support": { + "$ref": "#/$defs/Support" + }, + "price": { + "$ref": "#/$defs/Pricing" + }, + "team": { + "type": "array", + "items": { + "$ref": "#/$defs/Team" + } + }, + "roles": { + "type": "array", + "description": "A list of roles that will provide user access to the dataset.", + "items": { + "$ref": "#/$defs/Role" + } + }, + "slaDefaultElement": { + "type": "string", + "description": "Element (using the element path notation) to do the checks on." + }, + "slaProperties": { + "type": "array", + "description": "A list of key/value pairs for SLA specific properties. There is no limit on the type of properties (more details to come).", + "items": { + "$ref": "#/$defs/ServiceLevelAgreementProperty" + } + }, + "customProperties": { + "$ref": "#/$defs/CustomProperties" + }, + "contractCreatedTs": { + "type": "string", + "format": "date-time", + "description": "Timestamp in UTC of when the data contract was created." + } + }, + "required": ["version", "apiVersion", "kind", "id", "status"], + "additionalProperties": false, + "$defs": { + "Server": { + "type": "object", + "description": "Data source details of where data is physically stored.", + "properties": { + "server": { + "type": "string", + "description": "Identifier of the server." + }, + "type": { + "type": "string", + "description": "Type of the server.", + "enum": [ + "api", "athena", "azure", "bigquery", "clickhouse", "databricks", "denodo", "dremio", + "duckdb", "glue", "cloudsql", "db2", "informix", "kafka", "kinesis", "local", + "mysql", "oracle", "postgresql", "postgres", "presto", "pubsub", + "redshift", "s3", "sftp", "snowflake", "sqlserver", "synapse", "trino", "vertica", "custom" + ] + }, + "description": { + "type": "string", + "description": "Description of the server." + }, + "environment": { + "type": "string", + "description": "Environment of the server.", + "examples": ["prod", "preprod", "dev", "uat"] + }, + "roles": { + "type": "array", + "description": "List of roles that have access to the server.", + "items": { + "$ref": "#/$defs/Role" + } + }, + "customProperties": { + "$ref": "#/$defs/CustomProperties" + } + }, + "allOf": [ + { + "if": { + "properties": { + "type": { + "const": "api" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/ApiServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "athena" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/AthenaServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "azure" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/AzureServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "bigquery" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/BigQueryServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "clickhouse" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/ClickHouseServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "databricks" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/DatabricksServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "denodo" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/DenodoServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "dremio" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/DremioServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "duckdb" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/DuckdbServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "glue" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/GlueServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "cloudsql" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/GoogleCloudSqlServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "db2" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/IBMDB2Server" + } + }, + { + "if": { + "properties": { + "type": { + "const": "informix" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/InformixServer" + } + }, + + { + "if": { + "properties": { + "type": { + "const": "custom" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/CustomServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "kafka" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/KafkaServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "kinesis" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/KinesisServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "local" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/LocalServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "mysql" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/MySqlServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "oracle" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/OracleServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "postgresql" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/PostgresServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "postgres" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/PostgresServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "presto" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/PrestoServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "pubsub" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/PubSubServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "redshift" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/RedshiftServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "s3" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/S3Server" + } + }, + { + "if": { + "properties": { + "type": { + "const": "sftp" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/SftpServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "snowflake" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/SnowflakeServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "sqlserver" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/SqlserverServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "synapse" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/SynapseServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "trino" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/TrinoServer" + } + }, + { + "if": { + "properties": { + "type": { + "const": "vertica" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/ServerSource/VerticaServer" + } + } + ], + "required": ["server", "type"] + }, + "ServerSource": { + "ApiServer": { + "type": "object", + "title": "AthenaServer", + "properties": { + "location": { + "type": "string", + "format": "uri", + "description": "The url to the API.", + "examples": [ + "https://api.example.com/v1" + ] + } + }, + "required": [ + "location" + ] + }, + "AthenaServer": { + "type": "object", + "title": "AthenaServer", + "properties": { + "stagingDir": { + "type": "string", + "format": "uri", + "description": "Amazon Athena automatically stores query results and metadata information for each query that runs in a query result location that you can specify in Amazon S3.", + "examples": [ + "s3://my_storage_account_name/my_container/path" + ] + }, + "schema": { + "type": "string", + "description": "Identify the schema in the data source in which your tables exist." + }, + "catalog": { + "type": "string", + "description": "Identify the name of the Data Source, also referred to as a Catalog.", + "default": "awsdatacatalog" + }, + "regionName": { + "type": "string", + "description": "The region your AWS account uses.", + "examples": ["eu-west-1"] + } + }, + "required": [ + "staging_dir", + "schema" + ] + }, + "AzureServer": { + "type": "object", + "title": "AzureServer", + "properties": { + "location": { + "type": "string", + "format": "uri", + "description": "Fully qualified path to Azure Blob Storage or Azure Data Lake Storage (ADLS), supports globs.", + "examples": [ + "az://my_storage_account_name.blob.core.windows.net/my_container/path/*.parquet", + "abfss://my_storage_account_name.dfs.core.windows.net/my_container_name/path/*.parquet" + ] + }, + "format": { + "type": "string", + "enum": [ + "parquet", + "delta", + "json", + "csv" + ], + "description": "File format." + }, + "delimiter": { + "type": "string", + "enum": [ + "new_line", + "array" + ], + "description": "Only for format = json. How multiple json documents are delimited within one file" + } + }, + "required": [ + "location", + "format" + ] + }, + "BigQueryServer": { + "type": "object", + "title": "BigQueryServer", + "properties": { + "project": { + "type": "string", + "description": "The GCP project name." + }, + "dataset": { + "type": "string", + "description": "The GCP dataset name." + } + }, + "required": [ + "project", + "dataset" + ] + }, + "ClickHouseServer": { + "type": "object", + "title": "ClickHouseServer", + "properties": { + "host": { + "type": "string", + "description": "The host of the ClickHouse server." + }, + "port": { + "type": "integer", + "description": "The port to the ClickHouse server." + }, + "database": { + "type": "string", + "description": "The name of the database." + } + }, + "required": [ + "host", + "port", + "database" + ] + }, + "DatabricksServer": { + "type": "object", + "title": "DatabricksServer", + "properties": { + "host": { + "type": "string", + "description": "The Databricks host", + "examples": [ + "dbc-abcdefgh-1234.cloud.databricks.com" + ] + }, + "catalog": { + "type": "string", + "description": "The name of the Hive or Unity catalog" + }, + "schema": { + "type": "string", + "description": "The schema name in the catalog" + } + }, + "required": [ + "catalog", + "schema" + ] + }, + "DenodoServer": { + "type": "object", + "title": "DenodoServer", + "properties": { + "host": { + "type": "string", + "description": "The host of the Denodo server." + }, + "port": { + "type": "integer", + "description": "The port of the Denodo server." + }, + "database": { + "type": "string", + "description": "The name of the database." + } + }, + "required": [ + "host", + "port" + ] + }, + "DremioServer": { + "type": "object", + "title": "DremioServer", + "properties": { + "host": { + "type": "string", + "description": "The host of the Dremio server." + }, + "port": { + "type": "integer", + "description": "The port of the Dremio server." + }, + "schema": { + "type": "string", + "description": "The name of the schema." + } + }, + "required": [ + "host", + "port" + ] + }, + "DuckdbServer": { + "type": "object", + "title": "DuckdbServer", + "properties": { + "database": { + "type": "string", + "description": "Path to duckdb database file." + }, + "schema": { + "type": "integer", + "description": "The name of the schema." + } + }, + "required": [ + "database" + ] + }, + "GlueServer": { + "type": "object", + "title": "GlueServer", + "properties": { + "account": { + "type": "string", + "description": "The AWS Glue account", + "examples": [ + "1234-5678-9012" + ] + }, + "database": { + "type": "string", + "description": "The AWS Glue database name", + "examples": [ + "my_database" + ] + }, + "location": { + "type": "string", + "format": "uri", + "description": "The AWS S3 path. Must be in the form of a URL.", + "examples": [ + "s3://datacontract-example-orders-latest/data/{model}" + ] + }, + "format": { + "type": "string", + "description": "The format of the files", + "examples": [ + "parquet", + "csv", + "json", + "delta" + ] + } + }, + "required": [ + "account", + "database" + ] + }, + "GoogleCloudSqlServer": { + "type": "object", + "title": "GoogleCloudSqlServer", + "properties": { + "host": { + "type": "string", + "description": "The host of the Google Cloud Sql server." + }, + "port": { + "type": "integer", + "description": "The port of the Google Cloud Sql server." + }, + "database": { + "type": "string", + "description": "The name of the database." + }, + "schema": { + "type": "string", + "description": "The name of the schema." + } + }, + "required": [ + "host", + "port", + "database", + "schema" + ] + }, + "IBMDB2Server": { + "type": "object", + "title": "IBMDB2Server", + "properties": { + "host": { + "type": "string", + "description": "The host of the IBM DB2 server." + }, + "port": { + "type": "integer", + "description": "The port of the IBM DB2 server." + }, + "database": { + "type": "string", + "description": "The name of the database." + }, + "schema": { + "type": "string", + "description": "The name of the schema." + } + }, + "required": [ + "host", + "port", + "database" + ] + }, + "InformixServer": { + "type": "object", + "title": "InformixServer", + "properties": { + "host": { + "type": "string", + "description": "The host to the Informix server. " + }, + "port": { + "type": "integer", + "description": "The port to the Informix server. Defaults to 9088." + }, + "database": { + "type": "string", + "description": "The name of the database." + } + }, + "required": [ + "host", + "database" + ] + }, + "CustomServer": { + "type": "object", + "title": "CustomServer", + "properties": { + "account": { + "type": "string", + "description": "Account used by the server." + }, + "catalog": { + "type": "string", + "description": "Name of the catalog." + }, + "database": { + "type": "string", + "description": "Name of the database." + }, + "dataset": { + "type": "string", + "description": "Name of the dataset." + }, + "delimiter": { + "type": "string", + "description": "Delimiter." + }, + "endpointUrl": { + "type": "string", + "description": "Server endpoint.", + "format": "uri" + }, + "format": { + "type": "string", + "description": "File format." + }, + "host": { + "type": "string", + "description": "Host name or IP address." + }, + "location": { + "type": "string", + "description": "A URL to a location.", + "format": "uri" + }, + "path": { + "type": "string", + "description": "Relative or absolute path to the data file(s)." + }, + "port": { + "type": "integer", + "description": "Port to the server. No default value is assumed for custom servers." + }, + "project": { + "type": "string", + "description": "Project name." + }, + "region": { + "type": "string", + "description": "Cloud region." + }, + "regionName": { + "type": "string", + "description": "Region name." + }, + "schema": { + "type": "string", + "description": "Name of the schema." + }, + "serviceName": { + "type": "string", + "description": "Name of the service." + }, + "stagingDir": { + "type": "string", + "description": "Staging directory." + }, + "warehouse": { + "type": "string", + "description": "Name of the cluster or warehouse." + } + } + }, + "KafkaServer": { + "type": "object", + "title": "KafkaServer", + "description": "Kafka Server", + "properties": { + "host": { + "type": "string", + "description": "The bootstrap server of the kafka cluster." + }, + "format": { + "type": "string", + "description": "The format of the messages.", + "examples": ["json", "avro", "protobuf", "xml"], + "default": "json" + } + }, + "required": [ + "host" + ] + }, + "KinesisServer": { + "type": "object", + "title": "KinesisDataStreamsServer", + "description": "Kinesis Data Streams Server", + "properties": { + "region": { + "type": "string", + "description": "AWS region.", + "examples": [ + "eu-west-1" + ] + }, + "format": { + "type": "string", + "description": "The format of the record", + "examples": [ + "json", + "avro", + "protobuf" + ] + } + } + }, + "LocalServer": { + "type": "object", + "title": "LocalServer", + "properties": { + "path": { + "type": "string", + "description": "The relative or absolute path to the data file(s).", + "examples": [ + "./folder/data.parquet", + "./folder/*.parquet" + ] + }, + "format": { + "type": "string", + "description": "The format of the file(s)", + "examples": [ + "json", + "parquet", + "delta", + "csv" + ] + } + }, + "required": [ + "path", + "format" + ] + }, + "MySqlServer": { + "type": "object", + "title": "MySqlServer", + "properties": { + "host": { + "type": "string", + "description": "The host of the MySql server." + }, + "port": { + "type": "integer", + "description": "The port of the MySql server." + }, + "database": { + "type": "string", + "description": "The name of the database." + } + }, + "required": [ + "host", + "port", + "database" + ] + }, + "OracleServer": { + "type": "object", + "title": "OracleServer", + "properties": { + "host": { + "type": "string", + "description": "The host to the oracle server", + "examples": [ + "localhost" + ] + }, + "port": { + "type": "integer", + "description": "The port to the oracle server.", + "examples": [ + 1523 + ] + }, + "serviceName": { + "type": "string", + "description": "The name of the service.", + "examples": [ + "service" + ] + } + }, + "required": [ + "host", + "port", + "serviceName" + ] + }, + "PostgresServer": { + "type": "object", + "title": "PostgresServer", + "properties": { + "host": { + "type": "string", + "description": "The host to the Postgres server" + }, + "port": { + "type": "integer", + "description": "The port to the Postgres server." + }, + "database": { + "type": "string", + "description": "The name of the database." + }, + "schema": { + "type": "string", + "description": "The name of the schema in the database." + } + }, + "required": [ + "host", + "port", + "database", + "schema" + ] + }, + "PrestoServer": { + "type": "object", + "title": "PrestoServer", + "properties": { + "host": { + "type": "string", + "description": "The host to the Presto server", + "examples": [ + "localhost:8080" + ] + }, + "catalog": { + "type": "string", + "description": "The name of the catalog.", + "examples": [ + "postgres" + ] + }, + "schema": { + "type": "string", + "description": "The name of the schema.", + "examples": [ + "public" + ] + } + }, + "required": [ + "host" + ] + }, + "PubSubServer": { + "type": "object", + "title": "PubSubServer", + "properties": { + "project": { + "type": "string", + "description": "The GCP project name." + } + }, + "required": [ + "project" + ] + }, + "RedshiftServer": { + "type": "object", + "title": "RedshiftServer", + "properties": { + "host": { + "type": "string", + "description": "An optional string describing the server." + }, + "database": { + "type": "string", + "description": "The name of the database." + }, + "schema": { + "type": "string", + "description": "The name of the schema." + }, + "region": { + "type": "string", + "description": "AWS region of Redshift server.", + "examples": ["us-east-1"] + }, + "account": { + "type": "string", + "description": "The account used by the server." + } + }, + "required": [ + "database", + "schema" + ] + }, + "S3Server": { + "type": "object", + "title": "S3Server", + "properties": { + "location": { + "type": "string", + "format": "uri", + "description": "S3 URL, starting with `s3://`", + "examples": [ + "s3://datacontract-example-orders-latest/data/{model}/*.json" + ] + }, + "endpointUrl": { + "type": "string", + "format": "uri", + "description": "The server endpoint for S3-compatible servers.", + "examples": ["https://minio.example.com"] + }, + "format": { + "type": "string", + "enum": [ + "parquet", + "delta", + "json", + "csv" + ], + "description": "File format." + }, + "delimiter": { + "type": "string", + "enum": [ + "new_line", + "array" + ], + "description": "Only for format = json. How multiple json documents are delimited within one file" + } + }, + "required": [ + "location" + ] + }, + "SftpServer": { + "type": "object", + "title": "SftpServer", + "properties": { + "location": { + "type": "string", + "format": "uri", + "pattern": "^sftp://.*", + "description": "SFTP URL, starting with `sftp://`", + "examples": [ + "sftp://123.123.12.123/{model}/*.json" + ] + }, + "format": { + "type": "string", + "enum": [ + "parquet", + "delta", + "json", + "csv" + ], + "description": "File format." + }, + "delimiter": { + "type": "string", + "enum": [ + "new_line", + "array" + ], + "description": "Only for format = json. How multiple json documents are delimited within one file" + } + }, + "required": [ + "location" + ] + }, + "SnowflakeServer": { + "type": "object", + "title": "SnowflakeServer", + "properties": { + "host": { + "type": "string", + "description": "The host to the Snowflake server" + }, + "port": { + "type": "integer", + "description": "The port to the Snowflake server." + }, + "account": { + "type": "string", + "description": "The Snowflake account used by the server." + }, + "database": { + "type": "string", + "description": "The name of the database." + }, + "schema": { + "type": "string", + "description": "The name of the schema." + }, + "warehouse": { + "type": "string", + "description": "The name of the cluster of resources that is a Snowflake virtual warehouse." + } + }, + "required": [ + "account", + "database", + "schema" + ] + }, + "SqlserverServer": { + "type": "object", + "title": "SqlserverServer", + "properties": { + "host": { + "type": "string", + "description": "The host to the database server", + "examples": [ + "localhost" + ] + }, + "port": { + "type": "integer", + "description": "The port to the database server.", + "default": 1433, + "examples": [ + 1433 + ] + }, + "database": { + "type": "string", + "description": "The name of the database.", + "examples": [ + "database" + ] + }, + "schema": { + "type": "string", + "description": "The name of the schema in the database.", + "examples": [ + "dbo" + ] + } + }, + "required": [ + "host", + "database", + "schema" + ] + }, + "SynapseServer": { + "type": "object", + "title": "SynapseServer", + "properties": { + "host": { + "type": "string", + "description": "The host of the Synapse server." + }, + "port": { + "type": "integer", + "description": "The port of the Synapse server." + }, + "database": { + "type": "string", + "description": "The name of the database." + } + }, + "required": [ + "host", + "port", + "database" + ] + }, + "TrinoServer": { + "type": "object", + "title": "TrinoServer", + "properties": { + "host": { + "type": "string", + "description": "The Trino host URL.", + "examples": [ + "localhost" + ] + }, + "port": { + "type": "integer", + "description": "The Trino port." + }, + "catalog": { + "type": "string", + "description": "The name of the catalog.", + "examples": [ + "hive" + ] + }, + "schema": { + "type": "string", + "description": "The name of the schema in the database.", + "examples": [ + "my_schema" + ] + } + }, + "required": [ + "host", + "port", + "catalog", + "schema" + ] + }, + "VerticaServer": { + "type": "object", + "title": "VerticaServer", + "properties": { + "host": { + "type": "string", + "description": "The host of the Vertica server." + }, + "port": { + "type": "integer", + "description": "The port of the Vertica server." + }, + "database": { + "type": "string", + "description": "The name of the database." + }, + "schema": { + "type": "string", + "description": "The name of the schema." + } + }, + "required": [ + "host", + "port", + "database", + "schema" + ] + } + }, + "SchemaElement": { + "type": "object", + "properties": { + "name": { + "type": "string", + "description": "Name of the element." + }, + "physicalType": { + "type": "string", + "description": "The physical element data type in the data source.", + "examples": ["table", "view", "topic", "file"] + }, + "description": { + "type": "string", + "description": "Description of the element." + }, + "businessName": { + "type": "string", + "description": "The business name of the element." + }, + "authoritativeDefinitions": { + "$ref": "#/$defs/AuthoritativeDefinitions" + }, + "tags": { + "$ref": "#/$defs/Tags" + }, + "customProperties": { + "$ref": "#/$defs/CustomProperties" + } + } + }, + "SchemaObject": { + "type": "object", + "properties": { + "logicalType": { + "type": "string", + "description": "The logical element data type.", + "enum": ["object"] + }, + "physicalName": { + "type": "string", + "description": "Physical name.", + "examples": ["table_1_2_0"] + }, + "dataGranularityDescription": { + "type": "string", + "description": "Granular level of the data in the object.", + "examples": ["Aggregation by country"] + }, + "properties": { + "type": "array", + "description": "A list of properties for the object.", + "items": { + "$ref": "#/$defs/SchemaProperty" + } + }, + "quality": { + "$ref": "#/$defs/DataQualityChecks" + } + }, + "allOf": [ + { + "$ref": "#/$defs/SchemaElement" + } + ], + "required": ["name"], + "unevaluatedProperties": false + }, + "SchemaBaseProperty": { + "type": "object", + "properties": { + "primaryKey": { + "type": "boolean", + "description": "Boolean value specifying whether the element is primary or not. Default is false." + }, + "primaryKeyPosition": { + "type": "integer", + "default": -1, + "description": "If element is a primary key, the position of the primary key element. Starts from 1. Example of `account_id, name` being primary key columns, `account_id` has primaryKeyPosition 1 and `name` primaryKeyPosition 2. Default to -1." + }, + "logicalType": { + "type": "string", + "description": "The logical element data type.", + "enum": ["string", "date", "number", "integer", "object", "array", "boolean"] + }, + "logicalTypeOptions": { + "type": "object", + "description": "Additional optional metadata to describe the logical type." + }, + "physicalType": { + "type": "string", + "description": "The physical element data type in the data source. For example, VARCHAR(2), DOUBLE, INT." + }, + "required": { + "type": "boolean", + "default": false, + "description": "Indicates if the element may contain Null values; possible values are true and false. Default is false." + }, + "unique": { + "type": "boolean", + "default": false, + "description": "Indicates if the element contains unique values; possible values are true and false. Default is false." + }, + "partitioned": { + "type": "boolean", + "default": false, + "description": "Indicates if the element is partitioned; possible values are true and false." + }, + "partitionKeyPosition": { + "type": "integer", + "default": -1, + "description": "If element is used for partitioning, the position of the partition element. Starts from 1. Example of `country, year` being partition columns, `country` has partitionKeyPosition 1 and `year` partitionKeyPosition 2. Default to -1." + }, + "classification": { + "type": "string", + "description": "Can be anything, like confidential, restricted, and public to more advanced categorization. Some companies like PayPal, use data classification indicating the class of data in the element; expected values are 1, 2, 3, 4, or 5.", + "examples": ["confidential", "restricted", "public"] + }, + "encryptedName": { + "type": "string", + "description": "The element name within the dataset that contains the encrypted element value. For example, unencrypted element `email_address` might have an encryptedName of `email_address_encrypt`." + }, + "transformSourceObjects": { + "type": "array", + "description": "List of objects in the data source used in the transformation.", + "items": { + "type": "string" + } + }, + "transformLogic": { + "type": "string", + "description": "Logic used in the element transformation." + }, + "transformDescription": { + "type": "string", + "description": "Describes the transform logic in very simple terms." + }, + "examples": { + "type": "array", + "description": "List of sample element values.", + "items": { + "$ref": "#/$defs/AnyType" + } + }, + "criticalDataElement": { + "type": "boolean", + "default": false, + "description": "True or false indicator; If element is considered a critical data element (CDE) then true else false." + }, + "quality": { + "$ref": "#/$defs/DataQualityChecks" + } + }, + "allOf": [ + { + "$ref": "#/$defs/SchemaElement" + }, + { + "if": { + "properties": { + "logicalType": { + "const": "string" + } + } + }, + "then": { + "properties": { + "logicalTypeOptions": { + "type": "object", + "properties": { + "minLength": { + "type": "integer", + "minimum": 0, + "description": "Minimum length of the string." + }, + "maxLength": { + "type": "integer", + "minimum": 0, + "description": "Maximum length of the string." + }, + "pattern": { + "type": "string", + "description": "Regular expression pattern to define valid value. Follows regular expression syntax from ECMA-262 (https://262.ecma-international.org/5.1/#sec-15.10.1)." + }, + "format": { + "type": "string", + "examples": ["password", "byte", "binary", "email", "uuid", "uri", "hostname", "ipv4", "ipv6"], + "description": "Provides extra context about what format the string follows." + } + }, + "additionalProperties": false + } + } + } + }, + { + "if": { + "properties": { + "logicalType": { + "const": "date" + } + } + }, + "then": { + "properties": { + "logicalTypeOptions": { + "type": "object", + "properties": { + "format": { + "type": "string", + "examples": ["yyyy-MM-dd", "yyyy-MM-dd HH:mm:ss", "HH:mm:ss"], + "description": "Format of the date. Follows the format as prescribed by [JDK DateTimeFormatter](https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html). For example, format 'yyyy-MM-dd'." + }, + "exclusiveMaximum": { + "type": "boolean", + "default": false, + "description": "If set to true, all values are strictly less than the maximum value (values < maximum). Otherwise, less than or equal to the maximum value (values <= maximum)." + }, + "maximum": { + "type": "string", + "description": "All date values are less than or equal to this value (values <= maximum)." + }, + "exclusiveMinimum": { + "type": "boolean", + "default": false, + "description": "If set to true, all values are strictly greater than the minimum value (values > minimum). Otherwise, greater than or equal to the minimum value (values >= minimum)." + }, + "minimum": { + "type": "string", + "description": "All date values are greater than or equal to this value (values >= minimum)." + } + }, + "additionalProperties": false + } + } + } + }, + { + "if": { + "anyOf": [ + { + "properties": { + "logicalType": { + "const": "integer" + } + } + } + ] + }, + "then": { + "properties": { + "logicalTypeOptions": { + "type": "object", + "properties": { + "multipleOf": { + "type": "number", + "exclusiveMinimum": 0, + "description": "Values must be multiples of this number. For example, multiple of 5 has valid values 0, 5, 10, -5." + }, + "maximum": { + "type": "number", + "description": "All values are less than or equal to this value (values <= maximum)." + }, + "exclusiveMaximum": { + "type": "boolean", + "default": false, + "description": "If set to true, all values are strictly less than the maximum value (values < maximum). Otherwise, less than or equal to the maximum value (values <= maximum)." + }, + "minimum": { + "type": "number", + "description": "All values are greater than or equal to this value (values >= minimum)." + }, + "exclusiveMinimum": { + "type": "boolean", + "default": false, + "description": "If set to true, all values are strictly greater than the minimum value (values > minimum). Otherwise, greater than or equal to the minimum value (values >= minimum)." + }, + "format": { + "type": "string", + "default": "i32", + "description": "Format of the value in terms of how many bits of space it can use and whether it is signed or unsigned (follows the Rust integer types).", + "enum": ["i8", "i16", "i32", "i64", "i128", "u8", "u16", "u32", "u64", "u128"] + } + }, + "additionalProperties": false + } + } + } + }, + { + "if": { + "anyOf": [ + { + "properties": { + "logicalType": { + "const": "number" + } + } + } + ] + }, + "then": { + "properties": { + "logicalTypeOptions": { + "type": "object", + "properties": { + "multipleOf": { + "type": "number", + "exclusiveMinimum": 0, + "description": "Values must be multiples of this number. For example, multiple of 5 has valid values 0, 5, 10, -5." + }, + "maximum": { + "type": "number", + "description": "All values are less than or equal to this value (values <= maximum)." + }, + "exclusiveMaximum": { + "type": "boolean", + "default": false, + "description": "If set to true, all values are strictly less than the maximum value (values < maximum). Otherwise, less than or equal to the maximum value (values <= maximum)." + }, + "minimum": { + "type": "number", + "description": "All values are greater than or equal to this value (values >= minimum)." + }, + "exclusiveMinimum": { + "type": "boolean", + "default": false, + "description": "If set to true, all values are strictly greater than the minimum value (values > minimum). Otherwise, greater than or equal to the minimum value (values >= minimum)." + }, + "format": { + "type": "string", + "default": "i32", + "description": "Format of the value in terms of how many bits of space it can use (follows the Rust float types).", + "enum": ["f32", "f64"] + } + }, + "additionalProperties": false + } + } + } + }, + { + "if": { + "properties": { + "logicalType": { + "const": "object" + } + } + }, + "then": { + "properties": { + "logicalTypeOptions": { + "type": "object", + "properties": { + "maxProperties": { + "type": "integer", + "minimum": 0, + "description": "Maximum number of properties." + }, + "minProperties": { + "type": "integer", + "minimum": 0, + "default": 0, + "description": "Minimum number of properties." + }, + "required": { + "type": "array", + "items": { + "type": "string" + }, + "minItems": 1, + "uniqueItems": true, + "description": "Property names that are required to exist in the object." + } + }, + "additionalProperties": false + }, + "properties": { + "type": "array", + "description": "A list of properties for the object.", + "items": { + "$ref": "#/$defs/SchemaProperty" + } + } + } + } + }, + { + "if": { + "properties": { + "logicalType": { + "const": "array" + } + } + }, + "then": { + "properties": { + "logicalTypeOptions": { + "type": "object", + "properties": { + "maxItems": { + "type": "integer", + "minimum": 0, + "description": "Maximum number of items." + }, + "minItems": { + "type": "integer", + "minimum": 0, + "default": 0, + "description": "Minimum number of items" + }, + "uniqueItems": { + "type": "boolean", + "default": false, + "description": "If set to true, all items in the array are unique." + } + }, + "additionalProperties": false + }, + "items": { + "$ref": "#/$defs/SchemaItemProperty", + "description": "List of items in an array (only applicable when `logicalType: array`)." + } + } + } + } + ] + }, + "SchemaProperty": { + "type": "object", + "$ref": "#/$defs/SchemaBaseProperty", + "required": ["name"], + "unevaluatedProperties": false + }, + "SchemaItemProperty": { + "type": "object", + "$ref": "#/$defs/SchemaBaseProperty", + "properties": { + "properties": { + "type": "array", + "description": "A list of properties for the object.", + "items": { + "$ref": "#/$defs/SchemaProperty" + } + } + }, + "unevaluatedProperties": false + }, + "Tags": { + "type": "array", + "description": "A list of tags that may be assigned to the elements (object or property); the tags keyword may appear at any level.", + "items": { + "type": "string" + } + }, + "DataQuality": { + "type": "object", + "properties": { + "authoritativeDefinitions": { + "$ref": "#/$defs/AuthoritativeDefinitions" + }, + "businessImpact": { + "type": "string", + "description": "Consequences of the rule failure.", + "examples": ["operational", "regulatory"] + }, + "customProperties": { + "type": "array", + "description": "Additional properties required for rule execution.", + "items": { + "$ref": "#/$defs/CustomProperty" + } + }, + "description": { + "type": "string", + "description": "Describe the quality check to be completed." + }, + "dimension": { + "type": "string", + "description": "The key performance indicator (KPI) or dimension for data quality.", + "enum": ["accuracy", "completeness", "conformity", "consistency", "coverage", "timeliness", "uniqueness"] + }, + "method": { + "type": "string", + "examples": ["reconciliation"] + }, + "name": { + "type": "string", + "description": "Name of the data quality check." + }, + "schedule": { + "type": "string", + "description": "Rule execution schedule details.", + "examples": ["0 20 * * *"] + }, + "scheduler": { + "type": "string", + "description": "The name or type of scheduler used to start the data quality check.", + "examples": ["cron"] + }, + "severity": { + "type": "string", + "description": "The severance of the quality rule.", + "examples": ["info", "warning", "error"] + }, + "tags": { + "$ref": "#/$defs/Tags" + }, + "type": { + "type": "string", + "description": "The type of quality check. 'text' is human-readable text that describes the quality of the data. 'library' is a set of maintained predefined quality attributes such as row count or unique. 'sql' is an individual SQL query that returns a value that can be compared. 'custom' is quality attributes that are vendor-specific, such as Soda or Great Expectations.", + "enum": ["text", "library", "sql", "custom"], + "default": "library" + }, + "unit": { + "type": "string", + "description": "Unit the rule is using, popular values are `rows` or `percent`, but any value is allowed.", + "examples": ["rows", "percent"] + } + }, + "allOf": [ + { + "if": { + "properties": { + "type": { + "const": "library" + } + } + }, + "then": { + "$ref": "#/$defs/DataQualityLibrary" + } + }, + { + "if": { + "properties": { + "type": { + "const": "sql" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/DataQualitySql" + } + }, + { + "if": { + "properties": { + "type": { + "const": "custom" + } + }, + "required": ["type"] + }, + "then": { + "$ref": "#/$defs/DataQualityCustom" + } + } + ] + }, + "DataQualityChecks": { + "type": "array", + "description": "Data quality rules with all the relevant information for rule setup and execution.", + "items": { + "$ref": "#/$defs/DataQuality" + } + }, + "DataQualityLibrary": { + "type": "object", + "properties": { + "rule": { + "type": "string", + "description": "Define a data quality check based on the predefined rules as per ODCS.", + "examples": ["duplicateCount", "validValues", "rowCount"] + }, + "mustBe": { + "description": "Must be equal to the value to be valid. When using numbers, it is equivalent to '='." + }, + "mustNotBe": { + "description": "Must not be equal to the value to be valid. When using numbers, it is equivalent to '!='." + }, + "mustBeGreaterThan": { + "type": "number", + "description": "Must be greater than the value to be valid. It is equivalent to '>'." + }, + "mustBeGreaterOrEqualTo": { + "type": "number", + "description": "Must be greater than or equal to the value to be valid. It is equivalent to '>='." + }, + "mustBeLessThan": { + "type": "number", + "description": "Must be less than the value to be valid. It is equivalent to '<'." + }, + "mustBeLessOrEqualTo": { + "type": "number", + "description": "Must be less than or equal to the value to be valid. It is equivalent to '<='." + }, + "mustBeBetween": { + "type": "array", + "description": "Must be between the two numbers to be valid. Smallest number first in the array.", + "minItems": 2, + "maxItems": 2, + "uniqueItems": true, + "items": { + "type": "number" + } + }, + "mustNotBeBetween": { + "type": "array", + "description": "Must not be between the two numbers to be valid. Smallest number first in the array.", + "minItems": 2, + "maxItems": 2, + "uniqueItems": true, + "items": { + "type": "number" + } + } + }, + "required": ["rule"] + }, + "DataQualitySql": { + "type": "object", + "properties": { + "query": { + "type": "string", + "description": "Query string that adheres to the dialect of the provided server.", + "examples": ["SELECT COUNT(*) FROM ${table} WHERE ${column} IS NOT NULL"] + } + }, + "required": ["query"] + }, + "DataQualityCustom": { + "type": "object", + "properties": { + "engine": { + "type": "string", + "description": "Name of the engine which executes the data quality checks.", + "examples": ["soda", "great-expectations", "monte-carlo", "dbt"] + }, + "implementation": { + "oneOf": [ + { + "type": "string" + }, + { + "type": "object" + } + ] + } + }, + "required": ["engine", "implementation"] + }, + "AuthoritativeDefinitions": { + "type": "array", + "description": "List of links to sources that provide more details on the dataset; examples would be a link to an external definition, a training video, a GitHub repo, Collibra, or another tool. Authoritative definitions follow the same structure in the standard.", + "items": { + "type": "object", + "properties": { + "url": { + "type": "string", + "description": "URL to the authority." + }, + "type": { + "type": "string", + "description": "Type of definition for authority: v2.3 adds standard values: `businessDefinition`, `transformationImplementation`, `videoTutorial`, `tutorial`, and `implementation`.", + "examples": ["businessDefinition", "transformationImplementation", "videoTutorial", "tutorial", "implementation"] + } + }, + "required": ["url", "type"] + } + }, + "Support": { + "type": "array", + "description": "Top level for support channels.", + "items": { + "$ref": "#/$defs/SupportItem" + } + }, + "SupportItem": { + "type": "object", + "properties": { + "channel": { + "type": "string", + "description": "Channel name or identifier." + }, + "url": { + "type": "string", + "description": "Access URL using normal [URL scheme](https://en.wikipedia.org/wiki/URL#Syntax) (https, mailto, etc.)." + }, + "description": { + "type": "string", + "description": "Description of the channel, free text." + }, + "tool": { + "type": "string", + "description": "Name of the tool, value can be `email`, `slack`, `teams`, `discord`, `ticket`, or `other`.", + "examples": ["email", "slack", "teams", "discord", "ticket", "other"] + }, + "scope": { + "type": "string", + "description": "Scope can be: `interactive`, `announcements`, `issues`.", + "examples": ["interactive", "announcements", "issues"] + }, + "invitationUrl": { + "type": "string", + "description": "Some tools uses invitation URL for requesting or subscribing. Follows the [URL scheme](https://en.wikipedia.org/wiki/URL#Syntax)." + } + }, + "required": ["channel", "url"] + }, + "Pricing": { + "type": "object", + "properties": { + "priceAmount": { + "type": "number", + "description": "Subscription price per unit of measure in `priceUnit`." + }, + "priceCurrency": { + "type": "string", + "description": "Currency of the subscription price in `price.priceAmount`." + }, + "priceUnit": { + "type": "string", + "description": "The unit of measure for calculating cost. Examples megabyte, gigabyte." + } + } + }, + "Team": { + "type": "object", + "properties": { + "username": { + "type": "string", + "description": "The user's username or email." + }, + "role": { + "type": "string", + "description": "The user's job role; Examples might be owner, data steward. There is no limit on the role." + }, + "dateIn": { + "type": "string", + "format": "date", + "description": "The date when the user joined the team." + }, + "dateOut": { + "type": "string", + "format": "date", + "description": "The date when the user ceased to be part of the team." + }, + "replacedByUsername": { + "type": "string", + "description": "The username of the user who replaced the previous user." + } + } + }, + "Role": { + "type": "object", + "properties": { + "role": { + "type": "string", + "description": "Name of the IAM role that provides access to the dataset." + }, + "description": { + "type": "string", + "description": "Description of the IAM role and its permissions." + }, + "access": { + "type": "string", + "description": "The type of access provided by the IAM role." + }, + "firstLevelApprovers": { + "type": "string", + "description": "The name(s) of the first-level approver(s) of the role." + }, + "secondLevelApprovers": { + "type": "string", + "description": "The name(s) of the second-level approver(s) of the role." + } + }, + "required": ["role"] + }, + "ServiceLevelAgreementProperty": { + "type": "object", + "properties": { + "property": { + "type": "string", + "description": "Specific property in SLA, check the periodic table. May requires units (more details to come)." + }, + "value": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "number" + }, + { + "type": "integer" + }, + { + "type": "boolean" + }, + { + "type": "null" + } + ], + "description": "Agreement value. The label will change based on the property itself." + }, + "valueExt": { + "$ref": "#/$defs/AnyNonCollectionType", + "description": "Extended agreement value. The label will change based on the property itself." + }, + "unit": { + "type": "string", + "description": "**d**, day, days for days; **y**, yr, years for years, etc. Units use the ISO standard." + }, + "element": { + "type": "string", + "description": "Element(s) to check on. Multiple elements should be extremely rare and, if so, separated by commas." + }, + "driver": { + "type": "string", + "description": "Describes the importance of the SLA from the list of: `regulatory`, `analytics`, or `operational`.", + "examples": ["regulatory", "analytics", "operational"] + } + }, + "required": ["property", "value"] + }, + "CustomProperties": { + "type": "array", + "description": "A list of key/value pairs for custom properties.", + "items": { + "$ref": "#/$defs/CustomProperty" + } + }, + "CustomProperty": { + "type": "object", + "properties": { + "property": { + "type": "string", + "description": "The name of the key. Names should be in camel case–the same as if they were permanent properties in the contract." + }, + "value": { + "$ref": "#/$defs/AnyType", + "description": "The value of the key." + } + } + }, + "AnyType": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "number" + }, + { + "type": "integer" + }, + { + "type": "boolean" + }, + { + "type": "null" + }, + { + "type": "array" + }, + { + "type": "object" + } + ] + }, + "AnyNonCollectionType": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "number" + }, + { + "type": "integer" + }, + { + "type": "boolean" + }, + { + "type": "null" + } + ] + } + } +} diff --git a/script/README.md b/script/README.md deleted file mode 100644 index 1cd84e8..0000000 --- a/script/README.md +++ /dev/null @@ -1,9 +0,0 @@ -# Script - -## Building docs - -The script [`build_docs.sh`](build_docs.sh) is used to help move some top level markdown files into the -[`docs`](../docs) directory to allow for mkdocs to create a website based on the markdown files. - -This script gets called via the [GitHub Action](../.github/workflows/docs-site-deploy.yaml) that will build and deploy -the documentation website. diff --git a/script/build_docs.sh b/script/build_docs.sh deleted file mode 100644 index 1a5f56c..0000000 --- a/script/build_docs.sh +++ /dev/null @@ -1,7 +0,0 @@ -#!/usr/bin/env bash - -echo "Moving top level markdown files into 'docs' folder" -cat README.md | sed 's/(docs\//(/g' | sed 's/CONTRIBUTING.md/contributing.md/g' > docs/home.md -cat CHANGELOG.md | sed 's/(docs\//(/g' > docs/changelog.md -cp CONTRIBUTING.md docs/contributing.md -cp vendors.md docs/vendors.md diff --git a/src/script/README.md b/src/script/README.md new file mode 100644 index 0000000..7154fbb --- /dev/null +++ b/src/script/README.md @@ -0,0 +1,9 @@ +# Script + +## Building docs + +The script [`build_docs.sh`](build_docs.sh) is used to help move some top level markdown files into the +[`docs`](../../docs) directory to allow for mkdocs to create a website based on the markdown files. + +This script gets called via the [GitHub Action](../../.github/workflows/docs-site-deploy.yaml) that will build and deploy +the documentation website. diff --git a/src/script/build_docs.sh b/src/script/build_docs.sh new file mode 100644 index 0000000..bbfb0c5 --- /dev/null +++ b/src/script/build_docs.sh @@ -0,0 +1,29 @@ +#!/usr/bin/env bash + +echo "Moving top level markdown files into 'docs' folder" +cat README.md | sed 's/(docs\//(/g' | sed 's/CONTRIBUTING.md/contributing.md/g' | sed 's/resources.md/\.\.\/resources.md/g' > docs/home.md +cat CHANGELOG.md | sed 's/(docs\//(/g' > docs/changelog.md +cp CONTRIBUTING.md docs/contributing.md +cp vendors.md docs/vendors.md + +echo "Creating markdown file for each example" +for f in docs/examples/**/*.yaml; do + yaml_content=$(cat "$f") + markdown_file_path="${f//odcs.yaml/md}" + file_name=$(basename "$f" .odcs.yaml) + header=$(echo "${file_name//\-/ }" | sed -e "s/\b\(.\)/\u\1/g") + echo "Creating file: $markdown_file_path" + content=$(cat <<-END +# ${header} + +\`\`\`yaml +${yaml_content} +\`\`\` +END +) + echo "$content" > "$markdown_file_path" + base_path="${f//docs\/examples\//}" + escaped_header="${base_path//\//\\/}" + replacement_link="${escaped_header//odcs.yaml/md}" + sed -i -e "s/$escaped_header/$replacement_link/g" docs/examples/README.md +done diff --git a/src/script/generate-server-types.ruby b/src/script/generate-server-types.ruby new file mode 100644 index 0000000..1fe2a42 --- /dev/null +++ b/src/script/generate-server-types.ruby @@ -0,0 +1,60 @@ +require 'json' + +# Read the JSON schema file +file_path = 'schema/odcs-json-schema-v3.0.0.json' +schema = JSON.parse(File.read(file_path)) + +# Output file for Markdown documentation +output_file = 'server_types_tables.md' + +# Extract server types from the JSON Schema +server_definitions = schema['$defs']['Server']['allOf'] + +# Open the output file for writing +File.open(output_file, 'w') do |file| + # Iterate through each server type in the JSON schema + server_definitions.each do |definition| + puts definition + next unless definition['if'] && definition['then'] + + # Extract server type and details + server_type = definition['if']['properties']['type']['const'] + server_source_name = definition['then']['$ref'].split('/').last + puts server_type + server_details = schema['$defs']['ServerSource'][server_source_name] + required_fields = server_details['required'] || [] + puts required_fields + + # Write server type heading + file.puts "## #{server_type.capitalize} Server\n\n" + + # Write table header + file.puts "| Key | UX Label | Required | Description |\n" + file.puts "|--------------|-----------------|------------|------------------------------------------------------------|\n" + + puts "Before required fields" + + # First, print required fields + required_fields.each do |key| + if server_details['properties'].key?(key) + ux_label = key.split('_').map(&:capitalize).join(' ') # Generate UX label from the key + description = server_details['properties'][key]['description'] || "" + file.puts "| **#{key}** | #{ux_label} | Yes | #{description} |\n" + end + end + + puts "Required Fields" + + # Then, print the non-required fields + server_details['properties'].each do |key, value| + next if required_fields.include?(key) # Skip required fields as they are already printed + ux_label = key.split('_').map(&:capitalize).join(' ') # Generate UX label from the key + description = value['description'] || "" + file.puts "| **#{key}** | #{ux_label} | No | #{description} |\n" + end + + file.puts "\n" + end +end + +puts "Markdown tables generated in #{output_file}" diff --git a/src/script/validate-examples.sh b/src/script/validate-examples.sh new file mode 100644 index 0000000..b7ac1ac --- /dev/null +++ b/src/script/validate-examples.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash + +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +LIGHT_BLUE='\033[1;34m' +NC='\033[0m' + +script_dir=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) +json_schema_version=${JSON_SCHEMA_VERSION:-v3.0.0} +num_failed_validation=0 + +echo "Checking if $json_schema_version JSON schema is valid" +if ajv compile --spec=draft2019 -c ajv-formats -s "${script_dir}/../../schema/odcs-json-schema-${json_schema_version}.json"; then + echo -e "${GREEN}Valid JSON schema${NC}" +else + echo -e "${RED}Invalid JSON schema, exiting${NC}" + exit 1 +fi + +echo -e "Validating example ODCS files based on ${json_schema_version} JSON schema" +for file in docs/examples/*/*.yaml; do + if ajv validate --spec=draft2019 -c ajv-formats -s "${script_dir}/../../schema/odcs-json-schema-${json_schema_version}.json" -d "${script_dir}/../../${file}"; then + echo -e "${GREEN}Passed validation, file=${file}${NC}" + else + num_failed_validation=$((num_failed_validation+1)) + fi +done + +echo -e "${YELLOW}Total failed=${num_failed_validation}${NC}" +if [ "${num_failed_validation}" -gt 0 ]; then + exit 1 +else + exit 0 +fi diff --git a/vendors.md b/vendors.md index 305f9e3..073c181 100644 --- a/vendors.md +++ b/vendors.md @@ -10,6 +10,7 @@ catalogs, data quality platforms, security tools, and more. * [Data Caterer](https://data.catering/setup/guide/data-source/metadata/open-data-contract-standard/) - Test data management tool using data contracts as a metadata source * [Data Contract CLI](https://cli.datacontract.com) - Open Source tooling around data contracts +* [Data Contract Manager](https://datacontract-manager.com) - Professional data contract management tool with Data Marketplace, Access Management, and Data Governance AI. * [Data Contract Playground](https://data-catering.github.io/data-contract-playground/) - Playground site for creating, exporting and validating data contracts * [DQC.ai | DQ PLATFORM](https://www.dqc.ai/dqc-platform) - [Enhancing Data Quality with ODCS: A Standard Ensured by the DQ Platform ](https://www.dqc.ai/post/enhancing-data-quality-with-odcs-a-standard-ensured-by-the-dq-platform)