Skip to content

Commit

Permalink
Update to 0.12.3, include round and regex docs to data generator
Browse files Browse the repository at this point in the history
  • Loading branch information
pflooky committed Dec 4, 2024
1 parent bcf5298 commit 3bfc1d6
Show file tree
Hide file tree
Showing 47 changed files with 51 additions and 49 deletions.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ Below is a list of all supported data types for generating data:
| long | LongType | `min, max, stddev, mean` | |
| short | ShortType | `min, max, stddev, mean` | |
| decimal(precision, scale) | DecimalType(precision, scale) | `min, max, stddev, mean` | |
| double | DoubleType | `min, max, stddev, mean` | |
| float | FloatType | `min, max, stddev, mean` | |
| double | DoubleType | `min, max, round, stddev, mean` | |
| float | FloatType | `min, max, round, stddev, mean` | |
| date | DateType | `min, max, enableNull` | |
| timestamp | TimestampType | `min, max, enableNull` | |
| boolean | BooleanType | | |
Expand All @@ -38,6 +38,7 @@ descriptions:
| `enableEdgeCase` | false | `enableEdgeCase: "true"` | Enable/disable generated data to contain edge cases based on the data type. For example, integer data type has edge cases of (Int.MaxValue, Int.MinValue and 0) |
| `edgeCaseProbability` | 0.0 | `edgeCaseProb: "0.1"` | Probability of generating a random edge case value if `enableEdgeCase` is true |
| `isUnique` | false | `isUnique: "true"` | Enable/disable generated data to be unique for that column. Errors will be thrown when it is unable to generate unique data |
| `regex` | <empty> | `regex: "ACC[0-9]{10}"` | Regular expression to define pattern generated data should follow |
| `seed` | <empty> | `seed: "1"` | Defines the random seed for generating data for that particular column. It will override any seed defined at a global level |
| `sql` | <empty> | `sql: "CASE WHEN amount < 10 THEN true ELSE false END"` | Define any SQL statement for generating that columns value. Computation occurs after all non-SQL fields are generated. This means any columns used in the SQL cannot be based on other SQL generated columns. Data type of generated value from SQL needs to match data type defined for the field |

Expand Down Expand Up @@ -233,6 +234,7 @@ as defined by the data source (i.e. max value as per database type).
|----------|-------------|-----------------|----------------------------------------------------------------------|
| `min` | 0.0 | `min: "2.1"` | Ensures that all generated values are greater than or equal to `min` |
| `max` | 1000.0 | `max: "25.9"` | Ensures that all generated values are less than or equal to `max` |
| `round` | N/A | `round: "2"` | Round to particular number of decimal places |
| `stddev` | 1.0 | `stddev: "2.0"` | Standard deviation for normal distributed data |
| `mean` | `max - min` | `mean: "5.0"` | Mean for normal distributed data |

Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion docs/get-started/quick-start.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ image: "https://data.catering/diagrams/logo/data_catering_logo.svg"

1. Docker
```shell
docker run -d -i -p 9898:9898 -e DEPLOY_MODE=standalone --name datacaterer datacatering/data-caterer:0.12.2
docker run -d -i -p 9898:9898 -e DEPLOY_MODE=standalone --name datacaterer datacatering/data-caterer:0.12.3
```
2. [Open localhost:9898](http://localhost:9898)

Expand Down
2 changes: 1 addition & 1 deletion docs/use-case/comparison.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ The companies/products not shown below either have:

| Tool | Description | Cost | Pros | Cons |
|-----------------------------------------------------------------------|---------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Data Catering](https://data.catering/) | Scala based data generation and validation tool via metadata | Free (Open Source)<br>[Sponsorship model for support or additional features](../sponsor.md)<br>Starts at $100 per month | :white_check_mark: Data generation and validation<br>:white_check_mark: Batch and event generation<br>:white_check_mark: Maintain referential integrity<br>:white_check_mark: Scala/Java SDK<br>:white_check_mark: Customisable scenarios and validations<br>:white_check_mark: Open source<br>:white_check_mark: Metadata driven<br>:white_check_mark: Report generation<br>:white_check_mark: Use validation rules from existing tools<br>:white_check_mark: Data clean up<br>:white_check_mark: UI<br>:white_check_mark: Alerting | :octicons-x-circle-fill-12:{ .red-cross } No load testing metrics<br>:octicons-x-circle-fill-12:{ .red-cross } No validation of real time data sources |
| [Data Catering](https://data.catering/) | Scala based data generation and validation tool via metadata | Free (Open Source)<br>[Sponsorship model for support or additional features](../sponsor.md) | :white_check_mark: Data generation and validation<br>:white_check_mark: Batch and event generation<br>:white_check_mark: Maintain referential integrity<br>:white_check_mark: Scala/Java SDK<br>:white_check_mark: Customisable scenarios and validations<br>:white_check_mark: Open source<br>:white_check_mark: Metadata driven<br>:white_check_mark: Report generation<br>:white_check_mark: Use validation rules from existing tools<br>:white_check_mark: Data clean up<br>:white_check_mark: UI<br>:white_check_mark: Alerting | :octicons-x-circle-fill-12:{ .red-cross } No load testing metrics<br>:octicons-x-circle-fill-12:{ .red-cross } No validation of real time data sources |
| [Clearbox AI](https://www.clearbox.ai/) | Python based data generation tool via ML | [Unclear](https://www.clearbox.ai/pricing) | :white_check_mark: Python SDK<br>:white_check_mark: UI interface<br>:white_check_mark: Detect private data<br>:white_check_mark: Report generation | :octicons-x-circle-fill-12:{ .red-cross } Batch data only<br>:octicons-x-circle-fill-12:{ .red-cross } No data clean up<br>:octicons-x-circle-fill-12:{ .red-cross } Limited/no documentation |
| [Curiosity Software](https://www.curiositysoftware.ie/) | Platform solution for test data management | [Unclear](https://www.curiositysoftware.ie/test-modeller-test-data-automation-pricing) | :white_check_mark: Extensive documentation<br>:white_check_mark: Generate data based off test cases<br>:white_check_mark: UI interface<br>:white_check_mark: Web/API/UI/mobile testing | :octicons-x-circle-fill-12:{ .red-cross } No quick start<br>:octicons-x-circle-fill-12:{ .red-cross } No SDK<br>:octicons-x-circle-fill-12:{ .red-cross } Many components that may not be required<br>:octicons-x-circle-fill-12:{ .red-cross } No event generation support |
| [DataCebo Synthetic Data Vault](https://docs.sdv.dev/sdv/) | Python based data generation tool via ML | Unclear | :white_check_mark: Python SDK<br>:white_check_mark: Report generation<br>:white_check_mark: Data quality checks<br>:white_check_mark: Business logic constraints | :octicons-x-circle-fill-12:{ .red-cross } No data connection support<br>:octicons-x-circle-fill-12:{ .red-cross } No data clean up<br>:octicons-x-circle-fill-12:{ .red-cross } No foreign key support |
Expand Down
2 changes: 1 addition & 1 deletion helm/data-caterer/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ image:
repository: "datacatering/data-caterer"
pullPolicy: "IfNotPresent"
# Overrides the image tag whose default is the chart appVersion.
tag: "0.12.2"
tag: "0.12.3"

imagePullSecrets: []
nameOverride: ""
Expand Down
Loading

0 comments on commit 3bfc1d6

Please sign in to comment.