Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Generate tests that can check against an actual postgres schema #274

Open
JonathanLorimer opened this issue Sep 1, 2023 · 8 comments · May be fixed by #335
Open

[Feature Request] Generate tests that can check against an actual postgres schema #274

JonathanLorimer opened this issue Sep 1, 2023 · 8 comments · May be fixed by #335

Comments

@JonathanLorimer
Copy link
Contributor

It would be really nice to be able to be able to generate a test suite that will connect to a local postgres instance and verify that all TableSchema types match the postgres database's schema.

I am imagining that you would register each table schema in a list and maybe provide a pg connection string or something, and then the tests would be generated.

I am happy to help implement this, but I would definitely need lots of direction.

@KaneTW
Copy link

KaneTW commented Nov 22, 2023

This would be immensely helpful, as it's very easy to make a mistake when writing the Haskell types.

@KaneTW
Copy link

KaneTW commented Nov 24, 2023

Quick and dirty poc https://gist.github.com/KaneTW/0f321b6ccf0143611c43e2d087e939d3

Basically checks if the types match (i.e. bidirectional casts exists, so varchar and text match, but int8 and int2 don't), all non-nullables are present, and the hs ty has no extraneous types.

vst added a commit to vst/rel8 that referenced this issue May 1, 2024
This is a temporary attempt. See related issues:

- [Feature Request] Generate tests that can check against an actual postgres schema circuithub#274 [1]
- Provide access to rel8 internals circuithub#291 [2]

[1]: circuithub#274
[2]: circuithub#291
@vst
Copy link

vst commented May 1, 2024

Quick and dirty poc https://gist.github.com/KaneTW/0f321b6ccf0143611c43e2d087e939d3

Thanks a ton! This was super helpful.

If only #291 would become a reality. I had to fork Rel8 like you apparently did.

@KaneTW
Copy link

KaneTW commented May 1, 2024

Glad this helped.
Here's an updated version that I ended up using with more detailed error handling (and some other changes that I don't remember):
https://gist.github.com/KaneTW/c223d4da042f5d66e41ee27a46bcefc5

And a way to extract the schemas using the GHC API (at least as of writing that code half a year ago, this was not possible with any other method). Can also list them manually, but I found that too error-prone.
https://gist.github.com/KaneTW/d8c897a86a7e6049e607f5c4246d677e

Runs like this:
https://gist.github.com/KaneTW/586cd308736803efeca37fa6cb59f2d0

@vst
Copy link

vst commented May 2, 2024

That's getting better 😃 Thanks!

@abigailalice
Copy link
Contributor

@KaneTW is this correct for tables which contain HLists? I'm trying to clean it up to make a PR to close the issue, as well as #186 and #311, and noticed while testing it on a table which contains an HList f (SomeRecord f) that it seems to directly recurse into the names of SomeRecord, when I think it should wrap the types of each contained name in an array. Like, it reports types like int4 not null for the Haskell side and _int4 for the database.

I'm still working through figuring out the internals though, so I'm not sure if it's a bug in your gist, or a bug in the way rel8 reports the information, or a bug in my own code, though I don't think it's the latter, simply because I've been relying on this schema for a while, and everything has been working fine (as far as I can tell).

@KaneTW
Copy link

KaneTW commented Jul 29, 2024

Honestly not sure. It was correct when I used it, but it could have been a case I hadn't encountered.

It's been too long so I'd have to dig into the code as well to understand it again.

@abigailalice
Copy link
Contributor

I think it might be an issue of TypeInfo only representing the type by a QualifiedName, while possibly being better to use Rel8.Type.Name.TypeName, which includes more information. Provided I can figure out how the HTable class works I think I might be able to fix it, assuming I'm right that this case is a bug.

@abigailalice abigailalice linked a pull request Aug 2, 2024 that will close this issue
abigailalice added a commit to abigailalice/rel8 that referenced this issue Aug 3, 2024
This closes circuithub#274 and circuithub#186, allowing the generation of CREATE TABLE
statements from a TableSchema, as well as testing the table's schema in
the database against the schema implied by the TableSchema type.

Co-authored-by: David Kraeutmann <[email protected]>
abigailalice added a commit to abigailalice/rel8 that referenced this issue Aug 12, 2024
This closes circuithub#274 and circuithub#186, allowing the generation of CREATE TABLE
statements from a TableSchema, as well as testing the table's schema in
the database against the schema implied by the TableSchema type.

Co-authored-by: David Kraeutmann <[email protected]>
abigailalice added a commit to abigailalice/rel8 that referenced this issue Aug 12, 2024
This closes circuithub#274 and circuithub#186, allowing the generation of CREATE TABLE
statements from a TableSchema, as well as checking a TableSchema against
a database to determine if the tables are defined correctly in the
database to be read from/written to by rel8.

It also adds tests for the creation and type checking of tables, to
ensure they succeed/fail in appropriate cases.

Co-authored-by: David Kraeutmann <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants