feat: use kysely for handling migrations & add scripts #20
Conversation
| Example migration file: | ||
|
|
||
| ```typescript | ||
| import { Kysely } from "kysely"; | ||
|
|
||
| export async function up(db: Kysely<any>): Promise<void> { | ||
| // Your migration code here | ||
| } | ||
|
|
||
| export async function down(db: Kysely<any>): Promise<void> { | ||
| // Code to reverse the migration | ||
| } | ||
| ``` |
There was a problem hiding this comment.
Extremely nice to have but it'd be cool to have a third script to be able to run something like:
pnpm script:db:prepare <migration name>
And automagically generate the timestamped migration file with this sweet placeholder.
There was a problem hiding this comment.
aaa 💯 i think kysely has a cli-tool but i can easily home-craft it
There was a problem hiding this comment.
| ## Usage | ||
|
|
||
| First, install dependencies: | ||
|
|
||
| ```bash | ||
| pnpm install | ||
| ``` | ||
|
|
||
| ### Running Migrations | ||
|
|
||
| To apply all pending migrations: | ||
|
|
||
| ```bash | ||
| pnpm script:db:migrate | ||
| ``` | ||
|
|
||
| This will: | ||
|
|
||
| 1. Load environment variables | ||
| 2. Connect to the database | ||
| 3. Create the schema if it doesn't exist | ||
| 4. Run any pending migrations | ||
| 5. Log the results |
There was a problem hiding this comment.
Thinking about the "productive" stage of this app, how should migrations be applied? Should the operator stop the indexer, apply the migrations and then restart the indexer? Can we afford the indexer to be stopped to apply migrations or do you need a more sophisticated strategy?
There was a problem hiding this comment.
nice question ser, we yet have to discuss how this should work in prod but definitely the processor will need to be stopped
There was a problem hiding this comment.
will save this into Open questions to ask gitcoin team.
| migrationTableSchema: config.schema, | ||
| }); | ||
|
|
||
| const { error, results } = await migrator.migrateToLatest(); |
There was a problem hiding this comment.
Just to be absolutely sure, this migrateToLatest short-circuits if one migration fails, right? So migrations after the failing one are not applied
There was a problem hiding this comment.
it does short-circuit (and also skips migrations already applied), but i'm not sure if it applied 2 migrations and the 3rd failed, if it rollbacks all (like apply all migrations or none), will research more about this
There was a problem hiding this comment.
it doesn't rollback successful migrations, just short-circuit on failed migration
There was a problem hiding this comment.
Loved the update, sounds good though
|
|
||
| await db.destroy(); | ||
|
|
||
| process.exit(0); |
There was a problem hiding this comment.
This process.exit(0) might not be needed as 0 should be the default Node exit code (unless there's something I'm missing and you want to explicitly exit the process)
| * @returns A configured Kysely instance for the Database. | ||
| * | ||
| * This function sets up a PostgreSQL database connection using Kysely ORM. | ||
| * | ||
| * It uses the `CamelCasePlugin` to convert all table names to camel case. | ||
| * It uses the `WithSchemaPlugin` to automatically prefix all table names with the schema name on queries. |
| * ref: https://kysely.dev/docs/migrations#migration-files | ||
| */ | ||
| // eslint-disable-next-line @typescript-eslint/no-explicit-any | ||
| export async function up(db: Kysely<any>): Promise<void> { |
There was a problem hiding this comment.
Do we know if Kysely applies every of these SQL clauses inside a DB transaction or something similar? Thinking about what would happen if the n-th create table query fails: would the previous create table queries be reverted automatically?
There was a problem hiding this comment.
it's in a transaction, all or nothing because i've had some errors while writing the migration and that's how i found out xd
There was a problem hiding this comment.
+1 to Kysely for using sane defaults
| ## Usage | ||
|
|
||
| First, install dependencies: | ||
|
|
||
| ```bash | ||
| pnpm install | ||
| ``` | ||
|
|
||
| ### Running Migrations | ||
|
|
||
| To apply all pending migrations: | ||
|
|
||
| ```bash | ||
| pnpm script:db:migrate | ||
| ``` | ||
|
|
||
| This will: | ||
|
|
||
| 1. Load environment variables | ||
| 2. Connect to the database | ||
| 3. Create the schema if it doesn't exist | ||
| 4. Run any pending migrations | ||
| 5. Log the results |
There was a problem hiding this comment.
will save this into Open questions to ask gitcoin team.
| * Since WithSchemaPlugin doesn't work with `sql.table`, we need to get the schema name manually. | ||
| * ref: https://github.com/kysely-org/kysely/issues/761 | ||
| */ | ||
| export const getSchemaName = (schema: SchemaModule): string => { | ||
| let name = "public"; | ||
| schema.createTable("test").$call((b) => { | ||
| name = b.toOperationNode().table.table.schema?.name ?? "public"; | ||
| }); | ||
| return name; | ||
| }; |
There was a problem hiding this comment.
I didn't understand what is this function for
There was a problem hiding this comment.
so queries in migration have the 'schemaName.field/enumType' format. the plugin doesn't support all of kysely features and some don't get the schemaName appended automatically
| * ref: https://kysely.dev/docs/migrations#migration-files | ||
| */ | ||
| // eslint-disable-next-line @typescript-eslint/no-explicit-any | ||
| export async function up(db: Kysely<any>): Promise<void> { |
🤖 Linear
Closes GIT-113 GIT-114 GIT-115
Description
apps/scriptsfolderChecklist before requesting a review