Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(electric): Define __validate_table_column_types() at startup to use the up-to-date list of supported types #431

Merged

Conversation

alco
Copy link
Member

@alco alco commented Sep 15, 2023

Closes VAX-1033.

@linear
Copy link

linear bot commented Sep 15, 2023

VAX-1033 Future-proof the validation of column types by creating the SQL function at Electric startup time

See VAX-1016 for the larger context.

The goal of this issue is to unblock the path for adding more types on the server without modifying existing Extension migrations or duplicating the SQL function definition in new migrations.

@alco alco changed the title chore(electric): Move the definition of __validate_table_column_types() SQL function into a standalone module chore(electric): Define __validate_table_column_types() at startup to use the up-to-date list of supported types Sep 15, 2023
@alco alco force-pushed the alco/vax-1033-define-sql-type-checking-function-at-startup branch from 7638a06 to 329f40c Compare September 15, 2023 14:03
@@ -267,6 +278,8 @@ defmodule Electric.Postgres.Extension do
create_migration_table(txconn)

with_migration_lock(txconn, fn ->
define_functions(txconn)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better to move the function definitions after the schema migrations. This would allow for any run-once preliminary setup to happen before defining the functions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would mean taking the definition of the electrify function out of an internal migration and moving it to runtime as well. I wanted to avoid such drastic changes in this PR and only limit it to the type-checking function.

I can add a note explaining that this call needs to be moved elsewhere as part of VAX-1016.

@alco alco force-pushed the alco/vax-1033-define-sql-type-checking-function-at-startup branch 2 times, most recently from c9a6082 to d18bc15 Compare September 18, 2023 11:14
@alco alco marked this pull request as ready for review September 18, 2023 11:15
@icehaunter icehaunter force-pushed the alco/vax-1033-define-sql-type-checking-function-at-startup branch from d18bc15 to c4d6369 Compare September 19, 2023 18:55
Copy link
Contributor

@icehaunter icehaunter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you've taken a great approach with limiting the scope. I was worried we'd be redefining electrify on every run, but redefining just this function seems reasonable. Lovely work!

|> Enum.join(",")
%>

CREATE OR REPLACE FUNCTION electric.__validate_table_column_types(table_name text)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to merge this PR in front of launch so that we have a better upgrade story, but electric here should probably be taken from @schema in Extension module, as in all other places.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code used to have <%= @schema %> when it was part of the internal migration because schema is passed as an argument to the migration's up() function. But I also noticed that the migration that adds conflict resolution triggers has electric hard-coded, and so I made it hard-coded here too before I knew the context in which the higher-level define_functions() function would be called in.

@icehaunter icehaunter merged commit da9a718 into main Sep 19, 2023
4 checks passed
@icehaunter icehaunter deleted the alco/vax-1033-define-sql-type-checking-function-at-startup branch September 19, 2023 19:07
alco added a commit that referenced this pull request Oct 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants