Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extended functionalities #6

Open
wants to merge 50 commits into
base: main
Choose a base branch
from
Open

Conversation

ericchaves
Copy link

Hi guys, I`ve been working on a couple of features that extended quite a bit dg capabilities. Those features may require some polishment but they are being battle tested in a project of reasoble size (our database has over 50 tables with some and mixed design of traditional ER databases with some DW/OLAP tables).

In short we added the following:

Go Templates Integration: Integrated Go templates with the gen generator for enhanced templating capabilities.
Range Enhancements: Enhanced the range generator to pick up values from a previous table or from external command (like a query script), enabling the creation of "partitioned files" (multiple files for the same table design but generating dat for different test cenarios).
Brazilian Identifiers: Added functions for generating Brazilian CPF and CNPJ numbers.
File Output Improvements: Preserved the column order when writing files
Combining multiple configs: Support for mutiple config flags making easier to share/reuse configuration files. By informing multiple config we can use a base config to declare tables and use aditional configs to adjust counts or merge testing cenarios.
Fakeit integration: upgraded some libs and integrate fakeit into the new expr generator.

New Generators:

  • cuid2: Next-generation UUIDs, offering improved uniqueness.
  • expr: Enables dynamic expressions using the expr-lang lib for flexible data manipulation.
  • rand: Generates random values for integers, float64, and dates.
  • rel_date: Generates dates within a specified time span from a given range.
  • case: Generates values based on a set of conditional (when/then) statements.
  • fk: Supports generating relationships with variable cardinality, particularly useful for database seeding with 1:N cardinality.
  • map: map values based on the distinct values of another column.
  • lookup: generates column values based on cross-table lookups.
  • dist: generates distribution proportionally.

Would love to hear your thoughts and comments on this PR.

Regards,

Eric

ericchaves added 30 commits July 6, 2024 15:03
updated gofakeit to v7
added support to gofakeit templates

added generator for dates relative to now

added rand generator with support for dates, int and float64

added expr generator

minor adjustments plus added a section named expertimental  to readme explaining the new generators
replaced array search by samber/lo functions

added column lookup in relative_date
naming adjustments for clarity
readme corrections and improvements

added column lookup in relative_date
renamed relative_date generator for clarity

fixed type in rel_date test folder
added support for brazilian cpf\cnpj in gen generator
Fixed typo on range partioned tables examples
fixed case for rel_date units
replace govaluate with expr-lang
improved tests and clean-up coding
…by expr now.

added once generator
added count_values generator
preserving column order when writing csv files
fixed bugs on fk generator
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant