Skip to content

Commit

Permalink
Fix up data type in file guides, update title for data source guides
Browse files Browse the repository at this point in the history
  • Loading branch information
pflooky committed Jun 5, 2024
1 parent ed74f07 commit ebfe54b
Show file tree
Hide file tree
Showing 12 changed files with 43 additions and 25 deletions.
2 changes: 1 addition & 1 deletion docs/setup/guide/data-source/database/cassandra.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Cassandra data generation and validation tool"
title: "Cassandra Test Data Management"
description: "Example of Cassandra data generation and testing tool that can automatically discover, generate and validate."
image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
---
Expand Down
4 changes: 2 additions & 2 deletions docs/setup/guide/data-source/file/csv.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "CSV Test Data Management tool"
title: "CSV Test Data Management"
description: "Example of CSV test data management tool that can automatically discover, generate and validate."
image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
---
Expand Down Expand Up @@ -92,7 +92,7 @@ to the accounts generated.
var accountTask = csv("customer_accounts", "/opt/app/data/customer/account", Map.of("header", "true"))
.schema(
field().name("account_id"),
field().name("balance").type(DoubleType.instance()),
field().name("balance").type(new DecimalType(5, 2)),
field().name("created_by"),
field().name("name"),
field().name("open_time").type(TimestampType.instance()),
Expand Down
30 changes: 24 additions & 6 deletions docs/setup/guide/data-source/file/iceberg.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
---
title: "Iceberg Test Data Management tool"
title: "Iceberg Test Data Management"
description: "Example of Iceberg test data management tool that can automatically discover, generate and validate."
image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
---

# Iceberg

Creating a data generator for Iceberg. You will have the ability to generate and validate Iceberg tables via Docker.
Creating a data generator for Iceberg. You will have the ability to generate and validate Iceberg tables.

## Requirements

Expand All @@ -19,9 +19,27 @@ Creating a data generator for Iceberg. You will have the ability to generate and

First, we will clone the data-caterer-example repo which will already have the base project setup required.

```shell
git clone [email protected]:data-catering/data-caterer-example.git
```
=== "Java"

```shell
git clone [email protected]:data-catering/data-caterer-example.git
```

=== "Scala"

```shell
git clone [email protected]:data-catering/data-caterer-example.git
```

=== "YAML"

```shell
git clone [email protected]:data-catering/data-caterer-example.git
```

=== "UI"

[Run Data Caterer UI via the 'Quick Start' found here.](../../../../get-started/quick-start.md)

### Plan Setup

Expand Down Expand Up @@ -98,7 +116,7 @@ to the accounts generated.
var accountTask = iceberg("customer_accounts", "/opt/app/data/customer/iceberg", "account.accounts")
.schema(
field().name("account_id"),
field().name("balance").type(DoubleType.instance()),
field().name("balance").type(new DecimalType(5, 2)),
field().name("created_by"),
field().name("name"),
field().name("open_time").type(TimestampType.instance()),
Expand Down
4 changes: 2 additions & 2 deletions docs/setup/guide/data-source/file/json.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "JSON Test Data Management tool"
title: "JSON Test Data Management"
description: "Example of JSON test data management tool that can automatically discover, generate and validate."
image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
---
Expand Down Expand Up @@ -92,7 +92,7 @@ to the accounts generated.
var accountTask = json("customer_accounts", "/opt/app/data/customer/account_json")
.schema(
field().name("account_id"),
field().name("balance").type(DoubleType.instance()),
field().name("balance").type(new DecimalType(5, 2)),
field().name("created_by"),
field().name("open_time").type(TimestampType.instance()),
field().name("status"),
Expand Down
12 changes: 6 additions & 6 deletions docs/setup/guide/data-source/file/orc.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "ORC Test Data Management tool"
title: "ORC Test Data Management"
description: "Example of ORC test data management tool that can automatically discover, generate and validate."
image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
---
Expand Down Expand Up @@ -61,9 +61,9 @@ Within our class, we can start by defining the connection properties to read/wri

```java
var accountTask = orc(
"customer_accounts", //name
"customer_accounts", //name
"/opt/app/data/customer/account_orc", //path
Map.of() //additional options
Map.of() //additional options
);
```

Expand All @@ -73,9 +73,9 @@ Within our class, we can start by defining the connection properties to read/wri

```scala
val accountTask = orc(
"customer_accounts", //name
"customer_accounts", //name
"/opt/app/data/customer/account_orc", //path
Map() //additional options
Map() //additional options
)
```

Expand All @@ -92,7 +92,7 @@ to the accounts generated.
var accountTask = orc("customer_accounts", "/opt/app/data/customer/account_orc")
.schema(
field().name("account_id"),
field().name("balance").type(DoubleType.instance()),
field().name("balance").type(new DecimalType(5, 2)),
field().name("created_by"),
field().name("name"),
field().name("open_time").type(TimestampType.instance()),
Expand Down
4 changes: 2 additions & 2 deletions docs/setup/guide/data-source/file/parquet.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Parquet Test Data Management tool"
title: "Parquet Test Data Management"
description: "Example of Parquet test data management tool that can automatically discover, generate and validate."
image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
---
Expand Down Expand Up @@ -92,7 +92,7 @@ to the accounts generated.
var accountTask = parquet("customer_accounts", "/opt/app/data/customer/account_parquet")
.schema(
field().name("account_id"),
field().name("balance").type(DoubleType.instance()),
field().name("balance").type(new DecimalType(5, 2)),
field().name("created_by"),
field().name("name"),
field().name("open_time").type(TimestampType.instance()),
Expand Down
2 changes: 1 addition & 1 deletion docs/setup/guide/data-source/http/http.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "HTTP data generation and validation tool"
title: "HTTP Test Data Management"
description: "Example of automatically generating data for OpenAPI/Swagger docs to HTTP endpoints."
image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
---
Expand Down
2 changes: 1 addition & 1 deletion docs/setup/guide/data-source/messaging/kafka.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Kafka data generation and validation tool"
title: "Kafka Test Data Management"
description: "Example of Kafka data generation and testing tool that can automatically discover, generate and validate."
image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
---
Expand Down
2 changes: 1 addition & 1 deletion docs/setup/guide/data-source/messaging/solace.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Solace data generation and validation tool"
title: "Solace Test Data Management"
description: "Example of Solace for data generation and validation tool for queue/topic."
image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Using Great Expectations for data generation and validation"
title: "Using Great Expectations for Test Data Management"
description: "Example of using Great Expectations for data generation and testing in Data Caterer."
image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
---
Expand Down
2 changes: 1 addition & 1 deletion docs/setup/guide/data-source/metadata/marquez.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Using Marquez for data generation and validation"
title: "Using Marquez for Test Data Management"
description: "Example of using Marquez for data generation and testing tool."
image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
---
Expand Down
2 changes: 1 addition & 1 deletion docs/setup/guide/data-source/metadata/open-metadata.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Using OpenMetadata for data generation and validation"
title: "Using OpenMetadata for Test Data Management"
description: "Example of using OpenMetadata for data generation and testing tool."
image: "https://data.catering/diagrams/logo/data_catering_logo.svg"
---
Expand Down

0 comments on commit ebfe54b

Please sign in to comment.