diff --git a/docs/arch-1.svg b/docs/arch-1.svg deleted file mode 100644 index 47f51bf..0000000 --- a/docs/arch-1.svg +++ /dev/null @@ -1 +0,0 @@ -ServerManagerTemplate[]templateCollectionHashPool[]poolCollectionHashPoolTestDatabasedatabaseTemplateTemplateDatabasedatabaseTestDatabaseintIDDatabasedatabaseTemplateDatabaseDatabasedatabaseDatabasestringTemplateHashConfigDatabaseConfigownshashasmanagessetsisis \ No newline at end of file diff --git a/docs/arch-2.svg b/docs/arch-2.svg deleted file mode 100644 index ce33a3b..0000000 --- a/docs/arch-2.svg +++ /dev/null @@ -1 +0,0 @@ -
Task EXTEND
HashPool
TestDatabase
init
GetTestDatabase()
ReturnTestDatabase()
RecreateTestDatabase()
Task CLEAN_DIRTY
generation++
retry (still in use)
ready
dirty
recreating
\ No newline at end of file diff --git a/docs/arch.md b/docs/arch.md deleted file mode 100644 index e13f481..0000000 --- a/docs/arch.md +++ /dev/null @@ -1,27 +0,0 @@ - - -# IntegreSQL Architecture - -## Pool structure - -The following describes the relationship between the components of IntegreSQL. - -![diagram](./arch-1.svg) - -## TestDatabase states - -The following describes the state and transitions of a TestDatabase. - -![diagram](./arch-2.svg) \ No newline at end of file diff --git a/docs/arch.template.md b/docs/arch.template.md deleted file mode 100644 index 8d96ee7..0000000 --- a/docs/arch.template.md +++ /dev/null @@ -1,71 +0,0 @@ - - -# IntegreSQL Architecture - -## TestDatabase states - -The following describes the state and transitions of a TestDatabase. - -```mermaid -stateDiagram-v2 - - HashPool --> TestDatabase: Task EXTEND - - state TestDatabase { - [*] --> ready: init - ready --> dirty: GetTestDatabase() - dirty --> ready: ReturnTestDatabase() - dirty --> recreating: RecreateTestDatabase()\nTask CLEAN_DIRTY - recreating --> ready: generation++ - recreating --> recreating: retry (still in use) - } -``` - -## Pool structure - -The following describes the relationship between the components of IntegreSQL. - -```mermaid -erDiagram - Server ||--o| Manager : owns - Manager { - Template[] templateCollection - HashPool[] poolCollection - } - Manager ||--o{ HashPool : has - Manager ||--o{ Template : has - Template { - TemplateDatabase database - } - HashPool { - TestDatabase database - } - HashPool ||--o{ TestDatabase : "manages" - Template ||--|| TemplateDatabase : "sets" - TestDatabase { - int ID - Database database - } - TemplateDatabase { - Database database - } - Database { - string TemplateHash - Config DatabaseConfig - } - TestDatabase o|--|| Database : "is" - TemplateDatabase o|--|| Database : "is" -``` - diff --git a/docs/integration-1.svg b/docs/integration-1.svg deleted file mode 100644 index e4c622f..0000000 --- a/docs/integration-1.svg +++ /dev/null @@ -1 +0,0 @@ -PostgreSQLIntegreSQLTestrunnerYouPostgreSQLIntegreSQLTestrunnerYouCompute a hash over all related files that affect your database (migrations, fixtures, imports, etc.)Create a new PostgreSQL template database identified a the same unique hash payload: {"hash": "string"}Parse the received database connection payload and connect to the template database.Finalize the template so it can be used!You can now get isolated test databases for this hash from the pool!make testInitializeTemplate: POST /api/v1/templatesCREATE DATABASE template_<hash>StatusOK: 200Truncate, apply all migrations, seed all fixtures, ..., disconnect.FinalizeTemplate: PUT /api/v1/templates/:hashStatusOK: 200 \ No newline at end of file diff --git a/docs/integration-2.svg b/docs/integration-2.svg deleted file mode 100644 index ae8bf64..0000000 --- a/docs/integration-2.svg +++ /dev/null @@ -1 +0,0 @@ -PostgreSQLIntegreSQLTestrunnerYouPostgreSQLIntegreSQLTestrunnerYouSubsequent testrunners or multiple processes simply call with the same template hash again.Some other testrunner / process has already recreated this PostgreSQL template database identified by this hash (or is currently doing it), you can just consider the template ready at this point.You can now get isolated test databases for this hash from the pool!make testInitializeTemplate: POST /api/v1/templatesStatusLocked: 423 \ No newline at end of file diff --git a/docs/integration-3.svg b/docs/integration-3.svg deleted file mode 100644 index 20d2dba..0000000 --- a/docs/integration-3.svg +++ /dev/null @@ -1 +0,0 @@ -PostgreSQLIntegreSQLTestrunnerYouPostgreSQLIntegreSQLTestrunnerYouTypically happens if IntegreSQL cannot communicate withPostgreSQL, fail the test runner process in this case (e.g. exit 1).make testInitializeTemplate: POST /api/v1/templatesStatusServiceUnavailable: 503 \ No newline at end of file diff --git a/docs/integration-4.svg b/docs/integration-4.svg deleted file mode 100644 index a0bce99..0000000 --- a/docs/integration-4.svg +++ /dev/null @@ -1 +0,0 @@ -PostgreSQLIntegreSQLTestrunnerYouPostgreSQLIntegreSQLTestrunnerYou...Before each test, get a new isolated test database from the pool for the template hash.Blocks until the template is finalizedThe test databases for the template poolwere already created and are simply returned.Your runner now has a fully isolated PostgreSQL databasefrom our already migrated/seeded template database to use within your test.Run your test code!Your test is finished.GetTestDatabase: GET /api/v1/templates/:hash/testsStatusOK: 200Directly connect to the test database.Disconnect from the test database \ No newline at end of file diff --git a/docs/integration-5.svg b/docs/integration-5.svg deleted file mode 100644 index bfaa721..0000000 --- a/docs/integration-5.svg +++ /dev/null @@ -1 +0,0 @@ -PostgreSQLIntegreSQLTestrunnerYouPostgreSQLIntegreSQLTestrunnerYou...Run your **readonly** test code!Your **readonly** test is finished. As you did not modify the test database, you can unlock it again(immediately available in the pool again).GetTestDatabase: GET /api/v1/templates/:hash/testsStatusOK: 200Directly connect to the test database.Disconnect from the test databaseReturnTestDatabase: POST /api/v1/templates/:hash/tests/:id/unlock(previously and soft-deprecated DELETE /api/v1/templates/:hash/tests/:id)StatusOK: 200 \ No newline at end of file diff --git a/docs/integration-6.svg b/docs/integration-6.svg deleted file mode 100644 index f67a582..0000000 --- a/docs/integration-6.svg +++ /dev/null @@ -1 +0,0 @@ -PostgreSQLIntegreSQLTestrunnerYouPostgreSQLIntegreSQLTestrunnerYou...Run your test code!Your test is finished. As you don't want to wait for FIFO autocleaning, you can manually recreate the test database.GetTestDatabase: GET /api/v1/templates/:hash/testsStatusOK: 200Directly connect to the test database.Disconnect from the test databaseRecreateTestDatabase: POST /api/v1/templates/:hash/tests/:id/recreateStatusOK: 200 \ No newline at end of file diff --git a/docs/integration.md b/docs/integration.md deleted file mode 100644 index 4fd5ef0..0000000 --- a/docs/integration.md +++ /dev/null @@ -1,91 +0,0 @@ - - -# Integrate via REST API - -- [Integrate via REST API](#integrate-via-rest-api) - - [Once per test runner/process](#once-per-test-runnerprocess) - - [Testrunner creates a new template database](#testrunner-creates-a-new-template-database) - - [Testrunner reuses an existing template database](#testrunner-reuses-an-existing-template-database) - - [Failure modes while template database setup: 503](#failure-modes-while-template-database-setup-503) - - [Per each test](#per-each-test) - - [New test database per test](#new-test-database-per-test) - - [Optional: Manually unlocking a test database after a readonly test](#optional-manually-unlocking-a-test-database-after-a-readonly-test) - - [Optional: Manually recreating a test database](#optional-manually-recreating-a-test-database) - - [Failure modes while getting a new test database](#failure-modes-while-getting-a-new-test-database) - - [StatusNotFound 404](#statusnotfound-404) - - [StatusGone 410](#statusgone-410) - - [StatusServiceUnavailable 503](#statusserviceunavailable-503) - -First start IntegreSQL and leave it running in the background (your PostgreSQL template and test database pool will then always be warm). When you trigger your test command (e.g. `make test`), 1..n test runners/processes start in parallel. - -A really good starting point to write your own integresql-client for a specific language can be found [here (go code)](https://github.com/allaboutapps/integresql-client-go/blob/master/client.go) and [here (godoc)](https://pkg.go.dev/github.com/allaboutapps/integresql-client-go?tab=doc). It's just RESTful JSON after all. - -## Once per test runner/process - -Each test runner starts and need to communicate with IntegreSQL to setup 1..n template database pools. The following sections describe the flows/scenarios you need to implement. - -### Testrunner creates a new template database - -![diagram](./integration-1.svg) - -### Testrunner reuses an existing template database - -![diagram](./integration-2.svg) - -### Failure modes while template database setup: 503 - -![diagram](./integration-3.svg) - -## Per each test - -### New test database per test - -Well, this is the normal flow to get a new isolated test database (prepopulated as its created from the template) for your test. - -![diagram](./integration-4.svg) - -### Optional: Manually unlocking a test database after a readonly test - -* Returns the given test DB directly to the pool, without cleaning (recreating it). -* **This is optional!** If you don't call this endpoints, the test database will be recreated in a FIFO manner (first in, first out) as soon as possible, even though it actually had no changes. -* This is useful if you are sure, you did not do any changes to the database and thus want to skip the recreation process by returning it to the pool directly. - - -![diagram](./integration-5.svg) - -### Optional: Manually recreating a test database - -* Recreates the test DB according to the template and returns it back to the pool. -* **This is optional!** If you don't call this endpoint, the test database will be recreated in a FIFO manner (first in, first out) as soon as possible. -* This is useful if you have parallel testing with a mix of very long and super short tests. Our auto–FIFO recreation handling might block there. - -![diagram](./integration-6.svg) - - -### Failure modes while getting a new test database - -Some typical status codes you might encounter while getting a new test database. - -#### StatusNotFound 404 - -Well, seems like someone forgot to call InitializeTemplate or it errored out. - -#### StatusGone 410 - -There was an error during test setup with our fixtures, someone called `DiscardTemplate`, thus this template cannot be used. - -#### StatusServiceUnavailable 503 - -Well, typically a PostgreSQL connectivity problem \ No newline at end of file diff --git a/docs/integration.template.md b/docs/integration.template.md deleted file mode 100644 index 97d1f65..0000000 --- a/docs/integration.template.md +++ /dev/null @@ -1,201 +0,0 @@ - - -### Integrate via REST API - -- [Integrate via REST API](#integrate-via-rest-api) - - [Once per test runner/process](#once-per-test-runnerprocess) - - [Testrunner creates a new template database](#testrunner-creates-a-new-template-database) - - [Testrunner reuses an existing template database](#testrunner-reuses-an-existing-template-database) - - [Failure modes while template database setup: 503](#failure-modes-while-template-database-setup-503) - - [Per each test](#per-each-test) - - [New test database per test](#new-test-database-per-test) - - [Optional: Manually unlocking a test database after a readonly test](#optional-manually-unlocking-a-test-database-after-a-readonly-test) - - [Optional: Manually recreating a test database](#optional-manually-recreating-a-test-database) - - [Failure modes while getting a new test database](#failure-modes-while-getting-a-new-test-database) - - [StatusNotFound 404](#statusnotfound-404) - - [StatusGone 410](#statusgone-410) - - [StatusServiceUnavailable 503](#statusserviceunavailable-503) - -First start IntegreSQL and leave it running in the background (your PostgreSQL template and test database pool will then always be warm). When you trigger your test command (e.g. `make test`), 1..n test runners/processes start in parallel. - -A really good starting point to write your own integresql-client for a specific language can be found [here (go code)](https://github.com/allaboutapps/integresql-client-go/blob/master/client.go) and [here (godoc)](https://pkg.go.dev/github.com/allaboutapps/integresql-client-go?tab=doc). It's just RESTful JSON after all. - -#### Once per test runner/process - -Each test runner starts and need to communicate with IntegreSQL to setup 1..n template database pools. The following sections describe the flows/scenarios you need to implement. - -##### Testrunner creates a new template database - -```mermaid -sequenceDiagram - You->>Testrunner: make test - - Note right of Testrunner: Compute a hash over all related
files that affect your database
(migrations, fixtures, imports, etc.) - - Note over Testrunner,IntegreSQL: Create a new PostgreSQL template database
identified a the same unique hash
payload: {"hash": "string"} - - Testrunner->>IntegreSQL: InitializeTemplate: POST /api/v1/templates - - IntegreSQL->>PostgreSQL: CREATE DATABASE
template_ - PostgreSQL-->>IntegreSQL: - - IntegreSQL-->>Testrunner: StatusOK: 200 - - Note over Testrunner,PostgreSQL: Parse the received database connection payload and connect to the template database. - - Testrunner->>PostgreSQL: Truncate, apply all migrations, seed all fixtures, ..., disconnect. - PostgreSQL-->>Testrunner: - - Note over Testrunner,IntegreSQL: Finalize the template so it can be used! - - Testrunner->>IntegreSQL: FinalizeTemplate: PUT /api/v1/templates/:hash - IntegreSQL-->>Testrunner: StatusOK: 200 - - Note over Testrunner,PostgreSQL: You can now get isolated test databases for this hash from the pool! -``` - -##### Testrunner reuses an existing template database - -```mermaid -sequenceDiagram - - You->>Testrunner: make test - - Note over Testrunner,IntegreSQL: Subsequent testrunners or multiple processes
simply call with the same template hash again. - - Testrunner->>IntegreSQL: InitializeTemplate: POST /api/v1/templates - IntegreSQL-->>Testrunner: StatusLocked: 423 - - Note over Testrunner,IntegreSQL: Some other testrunner / process has already recreated
this PostgreSQL template database identified by this hash
(or is currently doing it), you can just consider
the template ready at this point. - - Note over Testrunner,PostgreSQL: You can now get isolated test databases for this hash from the pool! - -``` - -##### Failure modes while template database setup: 503 - -```mermaid -sequenceDiagram - - You->>Testrunner: make test - - Testrunner->>IntegreSQL: InitializeTemplate: POST /api/v1/templates - IntegreSQL-->>Testrunner: StatusServiceUnavailable: 503 - - Note over Testrunner,PostgreSQL: Typically happens if IntegreSQL cannot communicate with
PostgreSQL, fail the test runner process in this case (e.g. exit 1). - -``` - -#### Per each test - -##### New test database per test - -Well, this is the normal flow to get a new isolated test database (prepopulated as its created from the template) for your test. - -```mermaid -sequenceDiagram - - Note right of You: ... - - Note right of Testrunner: Before each test, get a new isolated test database
from the pool for the template hash. - - Testrunner->>IntegreSQL: GetTestDatabase: GET /api/v1/templates/:hash/tests - - Note over Testrunner,IntegreSQL: Blocks until the template is finalized - - Note right of IntegreSQL: The test databases for the template pool
were already created and are simply returned. - - IntegreSQL-->>Testrunner: StatusOK: 200 - - Note over Testrunner,PostgreSQL: Your runner now has a fully isolated PostgreSQL database
from our already migrated/seeded template database to use within your test. - - Testrunner->>PostgreSQL: Directly connect to the test database. - - Note over Testrunner,PostgreSQL: Run your test code! - - Testrunner-xPostgreSQL: Disconnect from the test database - - Note over Testrunner,PostgreSQL: Your test is finished. -``` - -##### Optional: Manually unlocking a test database after a readonly test - -* Returns the given test DB directly to the pool, without cleaning (recreating it). -* **This is optional!** If you don't call this endpoints, the test database will be recreated in a FIFO manner (first in, first out) as soon as possible, even though it actually had no changes. -* This is useful if you are sure, you did not do any changes to the database and thus want to skip the recreation process by returning it to the pool directly. - - -```mermaid -sequenceDiagram - - Note right of You: ... - - Testrunner->>IntegreSQL: GetTestDatabase: GET /api/v1/templates/:hash/tests - IntegreSQL-->>Testrunner: StatusOK: 200 - - Testrunner->>PostgreSQL: Directly connect to the test database. - - Note over Testrunner,PostgreSQL: Run your **readonly** test code! - - Testrunner-xPostgreSQL: Disconnect from the test database - - Note over Testrunner,PostgreSQL: Your **readonly** test is finished.
As you did not modify the test database, you can unlock it again
(immediately available in the pool again). - - Testrunner->>IntegreSQL: ReturnTestDatabase: POST /api/v1/templates/:hash/tests/:id/unlock
(previously and soft-deprecated DELETE /api/v1/templates/:hash/tests/:id) - IntegreSQL-->>Testrunner: StatusOK: 200 -``` - -##### Optional: Manually recreating a test database - -* Recreates the test DB according to the template and returns it back to the pool. -* **This is optional!** If you don't call this endpoint, the test database will be recreated in a FIFO manner (first in, first out) as soon as possible. -* This is useful if you have parallel testing with a mix of very long and super short tests. Our auto–FIFO recreation handling might block there. - -```mermaid -sequenceDiagram - - Note right of You: ... - - Testrunner->>IntegreSQL: GetTestDatabase: GET /api/v1/templates/:hash/tests - IntegreSQL-->>Testrunner: StatusOK: 200 - - Testrunner->>PostgreSQL: Directly connect to the test database. - - Note over Testrunner,PostgreSQL: Run your test code! - - Testrunner-xPostgreSQL: Disconnect from the test database - - Note over Testrunner,PostgreSQL: Your test is finished.
As you don't want to wait for FIFO autocleaning,
you can manually recreate the test database. - - Testrunner->>IntegreSQL: RecreateTestDatabase: POST /api/v1/templates/:hash/tests/:id/recreate - IntegreSQL-->>Testrunner: StatusOK: 200 -``` - - -##### Failure modes while getting a new test database - -Some typical status codes you might encounter while getting a new test database. - -###### StatusNotFound 404 - -Well, seems like someone forgot to call InitializeTemplate or it errored out. - -###### StatusGone 410 - -There was an error during test setup with our fixtures, someone called `DiscardTemplate`, thus this template cannot be used. - -###### StatusServiceUnavailable 503 - -Well, typically a PostgreSQL connectivity problem \ No newline at end of file