Releases: bitol-io/open-data-contract-standard
Releases · bitol-io/open-data-contract-standard
Version 3.0.0
Highlights
- New section: Support & communication channels.
- New section: Servers.
- Changes to fundamentals :
- Rename
uuid
toid
. - Add
name
. - Rename
quantumName
todataProduct
and make it optional. - Rename
datasetDomain
todomain
(we avoid the dataset prefix). - Drop
datasetKind
(example:virtualDataset
, was optional, have not seen any usage). - Drop
userConsumptionMode
(examples:analytical
, was optional, already deprecated in v2.). - Drop
sourceSystem
(example:bigQuery
, information will be encoded in servers). - Drop
sourcePlatform
(example:googleCloudPlatform
, information will be encoded in servers). - Drop
productSlackChannel
(will move to support channels). - Drop
productFeedbackUrl
(will move to support channels). - Drop
productDl
(will move to support channels). - Drop
username
(credentials should not be stored in the data contract). - Drop
password
(credentials should not be stored in the data contract). - Drop
driverVersion
(will move to servers if needed). - Drop
driver
(will move to servers if needed). - Drop
server
(will move to servers if needed). - Drop
project
(BigQuery-specific, will move to servers). - Drop
datasetName
(BigQuery-specific, will move to servers). - Drop
database
(BigQuery-specific, will move to servers). - Drop
schedulerAppName
(not part of the contract).
- Rename
- Changes to Schema:
- Major changes, check spec.
- Adds support for non table formats, hierarchies, and arrays.
name
is a new fielditems
is a new fieldpriorTableName
is not supported anymore, if needed, consider a custom property.table
is not supported anymore, if needed, consider usingname
.columns
is nowproperties
dataGranularity
is nowdataGranularityDescription
.encryptedColumnName
is nowencryptedName
.partitionStatus
is nowpartitioned
.clusterStatus
is not supported anymore, if needed, consider a custom property.clusterKeyPosition
is not supported anymore, if needed, consider a custom property.sampleValues
is nowexamples
.isNullable
is nowrequired
.isUnique
is nowunique
.isPrimaryKey
is nowprimaryKey
.criticalDataElementStatus
is nowcriticalDataElement
.clusterKeyPosition
is not supported anymore, if needed, consider a custom property.transformSourceTables
is nowtransformSourceObjects
- Restrict
schema.*.logicalType
to be one ofstring
,date
,number
,integer
,object
,array
,boolean
. - Add
schema.*.logicalTypeOptions
.
- Changes to Data Quality:
- Significant changes have been applied to support more tools and use cases. Please review the new section.
- If needed,
templateName
is a custom property. toolName
is obsolete, replaced bytype=custom; engine: <engine name>
.scheduleCronExpression
is replaced byschedule
andscheduler
.scheduleCronExpression: 0 20 * * *
becomesschedule: 0 20 * * *
andscheduler: cron
.
- Pricing:
- No changes.
- Changes to Team (fka stakeholders):
- Replaces
stakeholders
. Content stays the same.
- Replaces
- Changes to Role:
- Added
description
- Changed
access
is not required anymore
- Added
- Security:
- No changes.
- Changes to SLA:
- Starting with v3, the schema is not purely tables and columns, hence minor modifications: columns are now elements.
slaDefaultColumn
is nowslaDefaultElement
.column
is nowelement
.- Explicit reference to Data QoS.
- Changes to custom and other properties:
systemInstance
is not supported anymore, if needed, consider a custom property.
What's Changed
- Some examples included 'isPrimary', it should be 'isPrimaryKey' as pr… by @pflooky in #47
- Create vendors.md by @jgperrin in #51
- Update README.md by @jgperrin in #52
- Jgperrin patch 6 by @jgperrin in #53
- Update vendors.md by @simonharrer in #55
- Change schema file naming by @destouma in #56
- Add in data caterer as vendor, add vendor to docs site by @pflooky in #57
- Schemastore by @pflooky in #58
- Fixing a broken link for JSON Schema by @kidanekal in #60
- Rfc 0002 data types by @pflooky in #46
- RFC-5 Top Level Changes by @jochenchrist in #61
- Update markdown files in docs folder to use README file name, add in … by @pflooky in #62
- RFC 6 Support by @jgperrin in #63
- Explanation around rfc4 by @jgperrin in #64
- Update JSON Schema by @pflooky in #65
- Add properties key to logicalType object for nested fields. by @jochenchrist in #66
- Update database-table-sla.odcs.yaml by @jgperrin in #68
- More details in team member by @jgperrin in #67
- Add in data contract playground to vendors by @pflooky in #70
- RFC 7 by @jgperrin in #69
- Quality: Rename type implicit to library by @jochenchrist in #71
- Add Data Contract Manager to Vendor list by @jochenchrist in #72
- Rfc 7 update schema by @pflooky in #73
- Jgperrin's update on rfc1 by @jgperrin in #74
- Last changes for server by @simonharrer in #75
- Merge dev_v3 into dev by @jgperrin in #78
- ODCS 36 - Move script folder by @pflooky in #79
- Remove type from fundamentals by @pflooky in #81
- Update README.md by @dirkvandepoel in #83
- Update README.md by @dirkvandepoel in #84
- Update README.md by @jochenchrist in #85
- Last changes for server by @simonharrer in #88
- Add custom properties in schema by @simonharrer in #91
- Fix formatting issues in markdown that caused doc site to change text… by @pflooky in #89
- Dev by @pflooky in #92
New Contributors
- @simonharrer made their first contribution in #55
- @destouma made their first contribution in #56
- @kidanekal made their first contribution in #60
- @jochenchrist made their first contribution in #61
- @dirkvandepoel made their first contribution in #83
Full Changelog: v2.2.2...v3.0.0
Version 2.2.2
Highlights
- Change
dataset.description
data type from array to string - Change
dataset.column.isPrimaryKey
data type from string to boolean - Change
price.priceAmount
data type from string to number - Change
slaProperties.value
data type from string to oneOf[string, number] - Change
slaProperties.valueExt
data type from string to oneOf[string, number] - Update examples to adhere to JSON schema
- Full example from README directs to full-example.yaml
- Add in mkdocs for creating documentation website
What's Changed
- sync back w/ dev by @jgperrin in #27
- Dev examples fix schema by @pflooky in #31
- Documentation: Reorder definition tables to match examples by @chrfoyer in #32
- Update README.md by @jgperrin in #33
- Dev mkdocs by @pflooky in #35
- Update postgresql-adventureworks-contract.yaml by @jgperrin in #37
- Fixing the links to Issues and New Issue Page by @caladogan in #40
- Dev merge fix by @pflooky in #43
- RFC-0002 data types by @pflooky in #44
- Revert "RFC-0002 data types" by @pflooky in #45
- v2.2.2 release by @pflooky in #42
New Contributors
- @chrfoyer made their first contribution in #32
- @caladogan made their first contribution in #40
Full Changelog: v2.2.1...v2.2.2
Version 2.2.1
Highlights:
- Additional optional details for columns.
- Clarification of authoritative definitions.
- JSON Schema for validating contracts.
v2.2.1 is a non-breaking update of v2.2.0.
See CHANGELOG for a list of changes.
Version 2.2.0
Merge pull request #3 from AIDAUserGroup/dev v2.2 release