Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add DB initialization module #556

Merged
merged 25 commits into from
Feb 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
b18f84a
feat(data-formats): move StoredItem from rendezvous
7flying Nov 14, 2023
0bdb995
feat: add DB initialization module
7flying Sep 29, 2023
299592f
ci: update to handle DB tests
7flying Oct 2, 2023
8f37826
feat: add the Manufacturer DB store
7flying Nov 20, 2023
ed65dfe
feat: don't allow db settings in per-device config
7flying Oct 24, 2023
55189de
feat: add the Owner DB store configuration
7flying Oct 25, 2023
e4f9de1
feat: add the Rendezvous DB store configuration
7flying Oct 27, 2023
202edc0
feat(store): change store initializations based on DB type
7flying Oct 30, 2023
4982d09
fix(data-formats): error when trying to parse an empty slice
7flying Nov 22, 2023
1eeea64
feat(owner-tool): add an option to export OVs from the Manufacturer DB
7flying Nov 2, 2023
4da0a49
feat(owner-tool): add an option to import OVs to the Owner DB
7flying Nov 6, 2023
35369a9
fix(db): do not bail on non-critical errors
7flying Nov 7, 2023
1ddbe3a
fix(db): remove dotenv handling
7flying Nov 7, 2023
1e430f5
feat(store): add query_ovs_db method to store trait
7flying Nov 9, 2023
6df89ca
feat(owner-server): add DB handling to report_to_rendezvous
7flying Nov 9, 2023
1a9d368
feat(store): add Database error type
7flying Nov 10, 2023
e219eea
fix(db): change select_ov_to2_performed_and_ov_to0_less_than query
7flying Nov 13, 2023
4c9598a
fix(store): expect StoredItem in rendezvous store
7flying Nov 14, 2023
8af3381
chore(containers): update DB dependencies
7flying Nov 15, 2023
55d85d3
feat(store): add postgresql store variant
7flying Nov 20, 2023
48d8abd
docs: add database usage and configuration
7flying Nov 22, 2023
3fcef6b
test: add postgres db test
henrywang Jan 3, 2024
442ac2c
chore: update spec to add sql files and unify migrations
7flying Nov 28, 2023
8b65567
fix: adds necessary crates and scripts to dev container
djach7 Jan 3, 2024
530fef0
chore: bump db to 0.4.13
7flying Jan 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
FROM fedora:latest

ENV PATH "$PATH:/home/vscode/.cargo/bin"

RUN bash -c "$(curl -fsSL "https://raw.githubusercontent.com/microsoft/vscode-dev-containers/main/script-library/common-redhat.sh")" -- "true" "vscode" "1000" "1000" "true"

RUN dnf install -y \
sudo git cargo rust rust-src git-core openssl openssl-devel clippy rustfmt golang tpm2-tss-devel clevis clevis-luks cryptsetup cryptsetup-devel clang-devel \
sudo git cargo rust rust-src git-core openssl openssl-devel clippy rustfmt golang tpm2-tss-devel clevis clevis-luks cryptsetup cryptsetup-devel clang-devel sqlite sqlite-devel libpq libpq-devel \
&& dnf clean all

USER vscode

RUN cargo install --force diesel_cli --no-default-features --features sqlite
12 changes: 12 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,17 @@
"serayuzgur.crates",
"rust-lang.rust-analyzer"
],
"remoteEnv": {
"PATH": "${containerEnv:PATH}:/home/vscode/.cargo/bin",
"SQLITE_MANUFACTURER_DATABASE_URL": "../ci-manufacturer-db.sqlite",
"SQLITE_OWNER_DATABASE_URL": "../ci-owner-db.sqlite",
"SQLITE_RENDEZVOUS_DATABASE_URL": "../ci-rendezvous-db.sqlite"
},
"containerEnv": {
"SQLITE_MANUFACTURER_DATABASE_URL": "../ci-manufacturer-db.sqlite",
"SQLITE_OWNER_DATABASE_URL": "../ci-owner-db.sqlite",
"SQLITE_RENDEZVOUS_DATABASE_URL": "../ci-rendezvous-db.sqlite"
},
"hostRequirements": {
"memory": "4gb"
},
Expand All @@ -35,5 +46,6 @@
"cargo",
"build"
],
"postCreateCommand": "cargo install --force diesel_cli --no-default-features --features sqlite && diesel migration run --migration-dir ./migrations/migrations_manufacturing_server_sqlite --database-url ./ci-manufacturer-db.sqlite && diesel migration run --migration-dir ./migrations/migrations_owner_onboarding_server_sqlite --database-url ./ci-owner-db.sqlite && diesel migration run --migration-dir ./migrations/migrations_rendezvous_server_sqlite --database-url ./ci-rendezvous-db.sqlite",
7flying marked this conversation as resolved.
Show resolved Hide resolved
"waitFor": "onCreateCommand"
}
35 changes: 32 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ jobs:
steps:
- name: Install deps
run: |
dnf install -y make gcc openssl openssl-devel findutils golang git tpm2-tss-devel swtpm swtpm-tools git clevis clevis-luks cryptsetup cryptsetup-devel clang-devel cracklib-dicts
dnf install -y make gcc openssl openssl-devel findutils golang git tpm2-tss-devel swtpm swtpm-tools git clevis clevis-luks cryptsetup cryptsetup-devel clang-devel cracklib-dicts sqlite sqlite-devel libpq libpq-devel
- uses: actions/checkout@v3
with:
persist-credentials: false
Expand All @@ -112,7 +112,19 @@ jobs:
env:
FDO_PRIVILEGED: true
PER_DEVICE_SERVICEINFO: false
run: cargo test --workspace
SQLITE_MANUFACTURER_DATABASE_URL: ../ci-manufacturer-db.sqlite
SQLITE_OWNER_DATABASE_URL: ../ci-owner-db.sqlite
SQLITE_RENDEZVOUS_DATABASE_URL: ../ci-rendezvous-db.sqlite
run: |
# prep for database tests
cargo install --force diesel_cli --no-default-features --features sqlite
diesel migration run --migration-dir ./migrations/migrations_manufacturing_server_sqlite --database-url ./ci-manufacturer-db.sqlite
diesel migration run --migration-dir ./migrations/migrations_owner_onboarding_server_sqlite --database-url ./ci-owner-db.sqlite
diesel migration run --migration-dir ./migrations/migrations_rendezvous_server_sqlite --database-url ./ci-rendezvous-db.sqlite
# run tests
cargo test --workspace
# delete sqlite databases
rm -f ./ci-manufacturer-db.sqlite ./ci-owner-db.sqlite ./ci-rendezvous-db.sqlite
- name: Check aio
run: |
mkdir aio-dir/
Expand All @@ -127,6 +139,15 @@ jobs:
run: |
git diff --exit-code

postgres_test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Run test
run: test/fdo-postgres.sh

commitlint:
runs-on: ubuntu-latest
steps:
Expand Down Expand Up @@ -169,4 +190,12 @@ jobs:
- name: Test building in devcontainer
run: docker run --rm -v `pwd`:/code:z --workdir /code --user root devcontainer-fdo-rs cargo build --workspace --verbose
- name: Test testing in devcontainer
run: docker run --rm -v `pwd`:/code:z --workdir /code --user root devcontainer-fdo-rs cargo test --lib --bins --workspace --verbose
run: |
docker run -d -v `pwd`:/code:z --workdir /code --user root -e SQLITE_MANUFACTURER_DATABASE_URL='../ci-manufacturer-db.sqlite' -e SQLITE_OWNER_DATABASE_URL='../ci-owner-db.sqlite' -e SQLITE_RENDEZVOUS_DATABASE_URL='../ci-rendezvous-db.sqlite' --name tests devcontainer-fdo-rs sleep infinity
docker exec --user root tests cargo build --lib --bins --workspace --verbose
docker exec --user root tests diesel migration run --migration-dir ./migrations/migrations_manufacturing_server_sqlite --database-url ./ci-manufacturer-db.sqlite
docker exec --user root tests diesel migration run --migration-dir ./migrations/migrations_owner_onboarding_server_sqlite --database-url ./ci-owner-db.sqlite
docker exec --user root tests diesel migration run --migration-dir ./migrations/migrations_rendezvous_server_sqlite --database-url ./ci-rendezvous-db.sqlite
docker exec --user root tests cargo test
docker stop tests
docker rm tests
102 changes: 102 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ members = [
"http-wrapper",
"store",
"util",
"db",

"client-linuxapp",
"owner-onboarding-server",
Expand All @@ -24,6 +25,7 @@ default-members = [
"http-wrapper",
"store",
"util",
"db",

"client-linuxapp",
"owner-onboarding-server",
Expand Down
Loading
Loading