Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.1.0 #16

Merged
merged 170 commits into from
Jan 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
170 commits
Select commit Hold shift + click to select a range
34e6f45
add testclient
anjankow Jun 14, 2023
20f9e7c
update go version to 1.17.13
anjankow Jun 14, 2023
d0ee9e7
add BenchmarkGetDatabaseFromNewTemplate
anjankow Jun 14, 2023
724a832
add debug endpoints
anjankow Jun 15, 2023
d4fc2fb
add tracing tasks to manager
anjankow Jun 15, 2023
c04b8ff
add context to database functions and trace regions
anjankow Jun 15, 2023
373fa74
add benchmark for reusing existing template
anjankow Jun 15, 2023
d42c31c
fix manager tests
anjankow Jun 15, 2023
b573dff
refactor: remove wg reference from testing.go
anjankow Jun 15, 2023
359b2a8
add template config
anjankow Jun 16, 2023
00a152f
add pool GetReadyDB and AddTestDatabase
anjankow Jun 16, 2023
c4d0034
implement using pool per hash
anjankow Jun 16, 2023
5ad627e
improve ready/dirty system
anjankow Jun 16, 2023
fdd1084
keep pool, ready and dirty in dbHashPool
anjankow Jun 16, 2023
6524cbc
move pool to its own package
anjankow Jun 19, 2023
2d9fdac
add template collection
anjankow Jun 19, 2023
7dc9fde
add template collection
anjankow Jun 19, 2023
faab03f
use template collection for template init
anjankow Jun 19, 2023
e316b5e
use template collection for discard template
anjankow Jun 19, 2023
bb51cba
implement template state management
anjankow Jun 19, 2023
ba99810
adapt get and finalize template
anjankow Jun 20, 2023
173ae53
adapt reset all templates
anjankow Jun 20, 2023
8070312
remove old implementation remainings
anjankow Jun 20, 2023
f403872
update to go 1.20, update Dockerfile pkgs, use development user
anjankow Jun 20, 2023
cd34798
update devcontainer settings
anjankow Jun 20, 2023
937b493
fix unlocking DBPool
anjankow Jun 20, 2023
2c3da0d
add WaitWithTimeout to util
anjankow Jun 21, 2023
76f3fce
use WaitWithTimeout in template
anjankow Jun 21, 2023
971dee6
don't cleanup dirty db by manager
anjankow Jun 21, 2023
f89bafd
recreate db in pool in background using channels
anjankow Jun 21, 2023
c7cd87f
add pool tests
anjankow Jun 21, 2023
788004f
fix RemoveAll
anjankow Jun 21, 2023
ac4ea86
move down pool private functions
anjankow Jun 21, 2023
4d8f411
rename template state ready to finalized
anjankow Jun 21, 2023
54191a4
fix disconnecting manager
anjankow Jun 21, 2023
6ccb806
use context to cancel manager tasks
anjankow Jun 21, 2023
b4d8db7
move manager test code to _test package
anjankow Jun 22, 2023
ac341f3
fix template WaitUntilFinalized
anjankow Jun 22, 2023
a487bfc
init pool when finalizing a template
anjankow Jun 22, 2023
874eff9
fix mananger tests
anjankow Jun 22, 2023
efc70ab
stop pool worker with msg and assign 'inUse' state when extending pool
anjankow Jun 22, 2023
e7df274
use previous api when returning template
anjankow Jun 22, 2023
a2fe093
fix deadlock when init pool
anjankow Jun 22, 2023
06aeab9
vs debug config
anjankow Jun 22, 2023
fb28311
remove all from pool before discarding template
anjankow Jun 22, 2023
1ffcc14
use RLock for get
anjankow Jun 22, 2023
fa98c1c
move wg to dbHashPool
anjankow Jun 22, 2023
e5da171
check if manager ready before using db
anjankow Jun 22, 2023
d1b5986
discard template after test
anjankow Jun 22, 2023
f8e2fdc
allow to return db not existing in pool
anjankow Jun 22, 2023
74c26fa
set correct state inside extend func
anjankow Jun 23, 2023
2dabe15
fix test db name
anjankow Jun 27, 2023
a3ff1c3
lock template when finalizing
anjankow Jun 27, 2023
d7f185f
remove richgo from Makefile
anjankow Jun 27, 2023
9abacea
launch config: build before launching
anjankow Jun 27, 2023
864ea4a
return ErrTestDBInUse when deleting
anjankow Jun 27, 2023
558c1b2
add bench to Makefile
anjankow Jun 27, 2023
099e8d1
fix returning test database
anjankow Jun 27, 2023
c30a1f0
split template finalize and db get timeout
anjankow Jun 27, 2023
cf26115
remap postgres to 5434
anjankow Jun 27, 2023
ce85829
ignore error TemplateAlreadyInitialized
anjankow Jun 27, 2023
aa9b949
increase timeout for template finalization
anjankow Jun 27, 2023
6d07aa1
return http.StatusInsufficientStorage when pool is full
anjankow Jun 28, 2023
d1d45c2
add more trace info
anjankow Jun 28, 2023
3ac3fdd
add adjustable number of cleaning workers
anjankow Jun 28, 2023
0f7faa0
add worker_cleanup_task
anjankow Jun 28, 2023
f3aaf88
fix worker trace regions
anjankow Jun 28, 2023
a9b2fbf
add pool comments
anjankow Jun 28, 2023
a57ab09
add util comments
anjankow Jun 28, 2023
2f4eafb
add template tests and comments
anjankow Jun 28, 2023
5e8c4e7
use distroless/base-debian11:debug image
anjankow Jun 30, 2023
8764d04
add drone file
anjankow Jul 4, 2023
33d1ee7
fix indentation
anjankow Jul 4, 2023
448bf34
remove go-starter steps
anjankow Jul 4, 2023
c8d2f97
uncomment GCR_REGISTRY_SETTINGS
anjankow Jul 4, 2023
313fe7f
fix expired postgres certificate
anjankow Jul 4, 2023
3173a06
drone: fix target name
anjankow Jul 4, 2023
1211a8c
trivy: ignore vulnerabilities
anjankow Jul 4, 2023
81a439d
add info to Makefile
anjankow Jul 4, 2023
a754040
add tmp directory
anjankow Jul 4, 2023
b622abb
add lichen to tools
anjankow Jul 5, 2023
1e8d0c5
makefile: add get-embedded-modules-count and get-licenses
anjankow Jul 5, 2023
35ea8a6
tidy go.mod
anjankow Jul 5, 2023
0fa5fba
drone: use builder-integresql
anjankow Jul 5, 2023
a0d782d
drone: remove docker run step
anjankow Jul 5, 2023
8c768a7
drone: add test branch to publish branches
anjankow Jul 5, 2023
11f00ae
drone: remove test branch name
anjankow Jul 5, 2023
53f1a80
drone: add dev branch to publish
anjankow Jul 5, 2023
ec11caf
Pull request #3: add drone file
anjankow Jul 5, 2023
f300cca
Merge branch 'master' of github.com:allaboutapps/integresql into master
anjankow Jul 5, 2023
602e404
Merge branch 'master' of ssh://git-svc.allaboutapps.at:2222/aw/integr…
anjankow Jul 5, 2023
61f9f30
update Makefile
anjankow Jul 5, 2023
c79532d
trivy: remove ignored vulnerabilities
anjankow Jul 5, 2023
f4f30e5
update outdate packages
anjankow Jul 5, 2023
700b9e2
add TestDatabaseForceReturn config
anjankow Jul 5, 2023
6f4eb43
implement simple recycling of InUse db
anjankow Jul 5, 2023
b1b388d
track db creation time
anjankow Jul 5, 2023
b067fb3
mr/dev prepublish docker image aj/pooling-improvements
majodev Jul 5, 2023
7ddb508
recycle InUse dbs in order of creation time
anjankow Jul 5, 2023
61fdb22
move forceDBReturn param to pool
anjankow Jul 6, 2023
bfc9e7b
Merge branch 'aj/pooling-improvements' into mr/dev
majodev Jul 6, 2023
9e63829
use cfg struct to create new pool
anjankow Jul 6, 2023
a223855
drone: publish image
anjankow Jul 7, 2023
3e7ff61
manager: create new db in background on each get request
anjankow Jul 7, 2023
914671b
pool: reset not returned db
anjankow Jul 7, 2023
ea2e6e6
try recreate used test DB in a loop
anjankow Jul 12, 2023
ecde004
test manager without returning db
anjankow Jul 12, 2023
11f5278
fix create test db context
anjankow Jul 12, 2023
95bc264
increase template finalize timeout
anjankow Jul 12, 2023
17d327e
restore previous meaning of dirty
anjankow Jul 12, 2023
84fd49f
fix data race
anjankow Jul 13, 2023
46afe76
rename inUse to dirty and dirty to waitingForCleaning
anjankow Jul 13, 2023
8f75afa
add restore endpoint
anjankow Jul 13, 2023
373794c
don't return error when returning db that is ready
anjankow Jul 13, 2023
a763dec
add POST /unlock endpint
anjankow Jul 13, 2023
e9fd898
slit dbHashPool and PoolCollection file
anjankow Jul 14, 2023
758e63e
rename /restore to /reset
anjankow Jul 14, 2023
c34f768
add enableDBReset param to templates
anjankow Jul 14, 2023
f13f2ed
allow template config update
anjankow Jul 14, 2023
620bf6a
require pool init before AddTestDatabase
anjankow Jul 14, 2023
2c6d16f
remove all from pool if template config changed
anjankow Jul 14, 2023
87e0642
add ctx to RemoveFunc definition
anjankow Jul 14, 2023
395073f
merge aj/pooling-improvements
majodev Jul 14, 2023
7f172d4
refactor pool, rename forceReturn to enableDBReset
anjankow Jul 14, 2023
6e78810
refactor, rename dbHashPool to HashPool
anjankow Jul 14, 2023
90f35b7
rename /reset to /recreate
anjankow Jul 14, 2023
9f33c80
add missing defer statement
anjankow Jul 14, 2023
c052a24
rename enableReset to enableRecreate in api
anjankow Jul 14, 2023
9a06c94
write to dirty only if /recreate not enabled
anjankow Jul 19, 2023
5a06623
checkDatabaseConnected, integrate legacy handling with background wor…
majodev Jul 19, 2023
5d54a0a
merge up
majodev Jul 19, 2023
bcd9778
disable print debugging
majodev Jul 19, 2023
bf58e7a
Merge branch 'mr/dev' of ssh://git-svc.allaboutapps.at:2222/aw/integr…
anjankow Jul 21, 2023
041bd28
apply review comments
anjankow Jul 21, 2023
35844ec
extend and clean in pool background, remove waitingForCleaning channel
anjankow Jul 21, 2023
ee3e8e9
remove /recreate endpoint and all related config
anjankow Jul 22, 2023
e17d08c
clean up manager after moving pool size mgmt to pool pkg
anjankow Jul 22, 2023
12f7fcf
use PoolMaxParallelTasks
anjankow Jul 22, 2023
f6d48e8
added EXPOSE to dockerfile to expose default port
Jul 27, 2023
5878138
review only, reactivate print debugging
majodev Aug 4, 2023
752fad2
bring back simple /recreate endpoint
anjankow Aug 9, 2023
22f984e
remove unused utils
anjankow Aug 9, 2023
0fe17dc
Merge branch 'aj/pooling-improvements' into mr/aj-review
majodev Aug 24, 2023
c7342a6
close, range and refill dirty channel on unlock and recreate, debug s…
majodev Aug 25, 2023
8e64b27
filter via tmp channel without closing the initial, fix tests / stabi…
majodev Aug 28, 2023
1283351
redesign recreate handling
majodev Aug 29, 2023
a3b5be0
adds backoff handling for recreateDatabaseGracefully
majodev Aug 30, 2023
5bcf409
adds test database generations and TestDatabaseMinimalLifetime (block…
majodev Aug 30, 2023
96dd1c3
fix toolchain, reenable pipefails, fix linting, cleanup
majodev Sep 4, 2023
1e64a0c
private disableWorkerAutostart
majodev Sep 4, 2023
34fdd8e
intro zerolog and add proper log statements in pool and manager, into…
majodev Sep 5, 2023
7b99232
don't lock while extending, reuse recreateDatabaseGracefully
majodev Sep 5, 2023
7f8c1a9
Pull request #6: Mr/aj review
majodev Jan 8, 2024
a17f245
bump deps
majodev Jan 8, 2024
a2d84e7
add github actions multiarch build and publish
majodev Jan 24, 2024
1b012bb
Merge branch 'lucvankessel-master' into mr/v1.1.0
majodev Jan 24, 2024
f135ee0
upgrade golang and go.mod deps, prepare changelog (WIP), rearrange re…
majodev Jan 24, 2024
9e4ca7f
WIP docs and changelog, generic timeout handling
majodev Jan 29, 2024
5a5abd1
ref closes github issues #2 #15 #13
majodev Jan 29, 2024
989cb2f
doc env
majodev Jan 29, 2024
a74c36f
fix env vars doc
majodev Jan 29, 2024
da3ae8d
improve readme readablity
majodev Jan 29, 2024
f496328
docs
majodev Jan 30, 2024
8b1bba0
integration diagrams
majodev Jan 30, 2024
d454660
prepare move into readme
majodev Jan 30, 2024
e71d581
move diagrams into readme
majodev Jan 30, 2024
274e260
rm no longer needed extra md docs
majodev Jan 30, 2024
a39cb96
intro diagram
majodev Jan 30, 2024
7322909
readme typos
majodev Jan 30, 2024
be36a10
changelog mention diagrams
majodev Jan 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 75 additions & 81 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,94 +14,88 @@
// The optional 'workspaceFolder' property is the path VS Code should open by default when
// connected. This is typically a file mount in .devcontainer/docker-compose.yml
"workspaceFolder": "/app",
// Set *default* container specific settings.json values on container create.
"settings": {
"terminal.integrated.shell.linux": null,
// https://github.com/golang/tools/blob/master/gopls/doc/vscode.md#vscode
"go.useLanguageServer": true,
"[go]": {
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": true,
// All containers should stop if we close / reload the VSCode window.
"shutdownAction": "stopCompose",
"customizations": {
"vscode": {
// Set *default* container specific settings.json values on container create.
"settings": {
// https://github.com/golang/tools/blob/master/gopls/doc/vscode.md#vscode
"go.useLanguageServer": true,
"[go]": {
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": true,
},
// Optional: Disable snippets, as they conflict with completion ranking.
"editor.snippetSuggestions": "none",
},
"[go.mod]": {
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": true,
},
},
"[sql]": {
"editor.formatOnSave": true
},
"gopls": {
// Add parameter placeholders when completing a function.
"usePlaceholders": true,
// If true, enable additional analyses with staticcheck.
// Warning: This will significantly increase memory usage.
// DISABLED, done via
"staticcheck": false,
},
// https://code.visualstudio.com/docs/languages/go#_intellisense
"go.autocompleteUnimportedPackages": true,
// https://github.com/golangci/golangci-lint#editor-integration
"go.lintTool": "golangci-lint",
"go.lintFlags": [
"--fast",
"--timeout",
"5m"
],
// disable test caching, race and show coverage (in sync with makefile)
"go.testFlags": [
"-cover",
"-race",
"-count=1",
"-v"
],
"go.coverMode": "atomic", // atomic is required when utilizing -race
"go.delveConfig": {
"dlvLoadConfig": {
// increase max length of strings displayed in debugger
"maxStringLen": 2048,
},
"apiVersion": 2,
},
// ensure that the pgFormatter VSCode extension uses the pgFormatter that comes preinstalled in the Dockerfile
"pgFormatter.pgFormatterPath": "/usr/local/bin/pg_format"
},
// Optional: Disable snippets, as they conflict with completion ranking.
"editor.snippetSuggestions": "none",
},
"[go.mod]": {
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": true,
},
},
"[sql]": {
"editor.formatOnSave": true
},
"gopls": {
// Add parameter placeholders when completing a function.
"usePlaceholders": true,
// If true, enable additional analyses with staticcheck.
// Warning: This will significantly increase memory usage.
// DISABLED, done via
"staticcheck": false,
},
// https://code.visualstudio.com/docs/languages/go#_intellisense
"go.autocompleteUnimportedPackages": true,
// https://github.com/golangci/golangci-lint#editor-integration
"go.lintTool": "golangci-lint",
"go.lintFlags": [
"--fast"
],
// disable test caching, race and show coverage (in sync with makefile)
"go.testFlags": [
"-cover",
"-race",
"-count=1",
"-v"
],
// "go.lintOnSave": "workspace"
// general build settings in sync with our makefile
// "go.buildFlags": [
// "-o",
// "bin/app"
// ]
// "sqltools.connections": [
// {
// "database": "sample",
// "dialect": "PostgreSQL",
// "name": "postgres",
// "password": "9bed16f749d74a3c8bfbced18a7647f5",
// "port": 5432,
// "server": "postgres",
// "username": "dbuser"
// }
// ],
// "sqltools.autoConnectTo": [
// "postgres"
// ],
// // only use pg_format to actually format!
// "sqltools.formatLanguages": [],
// "sqltools.telemetry": false,
// "sqltools.autoOpenSessionFiles": false
// Add the IDs of extensions you want installed when the container is created.
"extensions": [
// required:
"golang.go",
"bradymholt.pgformatter",
// optional:
"42crunch.vscode-openapi",
"heaths.vscode-guid",
"bungcip.better-toml",
"eamodio.gitlens",
"casualjim.gotemplate",
"yzhang.markdown-all-in-one"
]
}
},
// Add the IDs of extensions you want installed when the container is created.
"extensions": [
// required:
"ms-vscode.go",
"bradymholt.pgformatter",
// optional:
// "766b.go-outliner",
"heaths.vscode-guid",
"bungcip.better-toml",
"eamodio.gitlens",
"casualjim.gotemplate"
// "mtxr.sqltools",
]
// Uncomment the next line if you want start specific services in your Docker Compose config.
// "runServices": [],
// Uncomment the next line if you want to keep your containers running after VS Code shuts down.
// "shutdownAction": "none",
// Uncomment the next line to run commands after the container is created - for example installing git.
"postCreateCommand": "go version",
// "postCreateCommand": "apt-get update && apt-get install -y git",
// Uncomment to connect as a non-root user. See https://aka.ms/vscode-remote/containers/non-root.
// "remoteUser": "vscode"
// "remoteUser": ""
}
3 changes: 2 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,5 @@
.tools-versions
Dockerfile
docker-compose.*
docker-helper.sh
docker-helper.sh
docs
Loading
Loading