diff --git a/SUMMARY.md b/SUMMARY.md index 6fa31c9..24b62f4 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -44,7 +44,7 @@ * [License](doc/license.md) * [Advanced doc](doc/README.md) * [MYDATA\_USAGE\_CONTROL](doc/MYDATA\_USAGE\_CONTROL.md) - * [PLATOON\_USAGE\_CONTROL](doc/PLATOON\_USAGE\_CONTROL.md) + * [Postgre Configuration](doc/postgreConfiguration.md) * [Test cases](doc/TEST\_API.md) * [REST API](doc/rest\_api/REST\_API.md) * [Testbed](doc/testbed/TESTBED.md) diff --git a/doc/PLATOON_USAGE_CONTROL.md b/doc/PLATOON_USAGE_CONTROL.md deleted file mode 100644 index 8c616ff..0000000 --- a/doc/PLATOON_USAGE_CONTROL.md +++ /dev/null @@ -1,109 +0,0 @@ -# Usage Control database - -Usage control application uses in-memory database with persisting db on file system (in uc-dataapp_resources_consumer and dataapp_resources_provider folders). This setup can be used for some small POC projects, to verify if integration is working and similar, but for real use case scenario, some more resilient database should be used, for example PostgreSQL (provided config) or some other database. - -In order to switch to PostgreSQL database, following steps are needed: - - - modify docker compose file, and add 2 PostgreSQL services, one for Provider and one for Consumer: - -``` - postgres_provider: - image: postgres - hostname: postgres_provider - expose: - - "5432" - networks: - - provider - env_file: - - ./postgres_provider.env - volumes: - - ./app_provider:/var/lib/postgresql/data - - - postgres_consumer: - image: postgres - hostname: postgres_consumer - expose: - - "5432" - networks: - - consumer - env_file: - - ./postgres_consumer.env - volumes: - - ./app_consumer:/var/lib/postgresql/data - -``` - - - add dependency for usage control applications to postgres - -``` - uc-dataapp-provider: - ... - depends_on: - - postgres_provider - - uc-dataapp-consumer: - ... - depends_on: - - postgres_consumer - -``` - - - modify usage control property files, both for *uc-dataapp_resources_provider* - -``` -## H2 DB with persisting on disk -#spring.datasource.url=jdbc:h2:file:/etc/platoon_db_provider -#spring.jpa.database-platform=org.hibernate.dialect.H2Dialect -#spring.datasource.driver-class-name=org.h2.Driver - -## PostgreSQL -spring.jpa.database=POSTGRESQL -spring.datasource.url = jdbc:postgresql://postgres_provider:5432/usagecontrol_provider -spring.datasource.driver-class-name = org.postgresql.Driver -spring.jpa.database-platform = org.hibernate.dialect.PostgreSQLDialect - -``` - -and *uc-dataapp_resources_consumer* - -``` -## H2 DB with persisting on disk -#spring.datasource.url=jdbc:h2:file:/etc/platoon_db_consumer -#spring.jpa.database-platform=org.hibernate.dialect.H2Dialect -#spring.datasource.driver-class-name=org.h2.Driver - -##PostgreSQL -spring.jpa.database=POSTGRESQL -spring.datasource.url = jdbc:postgresql://postgres_consumer:5432/usagecontrol_consumer -spring.datasource.driver-class-name = org.postgresql.Driver -spring.jpa.database-platform = org.hibernate.dialect.PostgreSQLDialect - -``` - - - postgres env file - -2 env files needed for PostgreSQL should be created, in root of TRUE Connector directory: - -*postgres_provider.env* with content - -``` -POSTGRES_USER=connector -POSTGRES_PASSWORD=12345 -POSTGRES_DB=usagecontrol_provider - -``` - -*postgres_consumer.env* with content - -``` -POSTGRES_USER=connector -POSTGRES_PASSWORD=12345 -POSTGRES_DB=usagecontrol_consumer - -``` - -# Usage control examples - -For more information and examples of policies compatible with Platoon UC app, please check [README](https://github.com/Engineering-Research-and-Development/true-connector-uc_data_app_platoon/blob/1.7.9/README.md) - diff --git a/doc/TRUEConnector/prerequisite.md b/doc/TRUEConnector/prerequisite.md index a07b378..767d526 100644 --- a/doc/TRUEConnector/prerequisite.md +++ b/doc/TRUEConnector/prerequisite.md @@ -139,7 +139,7 @@ By regularly updating SSH keys every three months, administrators will enhance t ## Secure DB -It's mandatory to set **AES256-SECRET-KEY** as OS variable which is valid password for column encryption with AES256 algorithm. +It's mandatory to set **AES-SECRET-KEY** as OS variable which is valid password for column encryption with AES/GCM/NoPadding algorithm. ## Post configuration steps diff --git a/doc/advancedConfiguration/auditlogs.md b/doc/advancedConfiguration/auditlogs.md index a01f3dd..dadbb99 100644 --- a/doc/advancedConfiguration/auditlogs.md +++ b/doc/advancedConfiguration/auditlogs.md @@ -1,7 +1,7 @@ ### Audit logs -Audit logs must be turned on for a proper and secure deployment of the connector. All -All audit events are stored in database (H2 with default configuration, possible to replace with PostgreSQL), this way tampering of the logs is prohibited. Entries in database are done only by the Execution Core Container. Column for storing auditLog entry is encrypted using *AES/GCM/NoPadding* algorithm which requires user to set valid password. It must be done using environment variable with following name: *AES256-SECRET-KEY*.
+Audit logs must be turned on for a proper and secure deployment of the connector. +All audit events are stored in database (H2 with default configuration, possible to replace with PostgreSQL, more information can be found [here](../postgreConfiguration.md)), this way tampering of the logs is prohibited. Entries in database are done only by the Execution Core Container. Column for storing auditLog entry is encrypted using *AES/GCM/NoPadding* algorithm which requires user to set valid password. It must be done using environment variable with following name: *AES256-SECRET-KEY*.
When ECC inserts audit entry into Database, AuditLog value will be encrypted using provided algorithm, and when data is requested, it will be decrypted.
If you wish to configure it or even turn off please follow this [document](https://github.com/Engineering-Research-and-Development/true-connector-execution_core_container/blob/1.14.8/doc/AUDIT.md) . diff --git a/doc/advancedConfiguration/usagecontrol.md b/doc/advancedConfiguration/usagecontrol.md index 9a224d2..9d118a9 100644 --- a/doc/advancedConfiguration/usagecontrol.md +++ b/doc/advancedConfiguration/usagecontrol.md @@ -1,6 +1,6 @@ ### Usage Control -Details about the PMP and PEP components and how to switch to PostgeSQL from the default H2 in-memory database you can find [here](../PLATOON\_USAGE\_CONTROL.md). +Details about the PMP and PEP components and examples of policies compatible with Platoon UC app, please check [README](https://github.com/Engineering-Research-and-Development/true-connector-uc_data_app_platoon/blob/1.7.9/README.md) Since Usage Control is disabled by default, in order to enable it, set following property to true: @@ -19,4 +19,6 @@ or ``` UC_DATAAPP_URI=http://uc-dataapp-consumer:8080/platoontec/PlatoonDataUsage/1.0/ -``` \ No newline at end of file +``` + +Usage Control by default uses H2 database with default configuration it is possible to replace with PostgreSQL, more information can be found [here](../postgreConfiguration.md)). \ No newline at end of file diff --git a/doc/postgreConfiguration.md b/doc/postgreConfiguration.md new file mode 100644 index 0000000..edffaf9 --- /dev/null +++ b/doc/postgreConfiguration.md @@ -0,0 +1,212 @@ +# Postgre configuration + +Both ECC and UCApp uses in-memory database (H2) with persisting db on file system. This setup can be used for some small POC projects, to verify if integration is working and similar, but for real use case scenario, some more resilient database should be used, for example PostgreSQL (provided config) or some other database. + +In order to switch to PostgreSQL database, following steps are needed: + + - Modify docker compose file, and add 2 PostgreSQL services, one for Provider and one for Consumer: + +``` + postgres_provider: + image: postgres:16.2-alpine3.19 + hostname: postgres_provider + ports: + - "5432:5432" + networks: + - provider + env_file: + - ./postgres_provider.env + volumes: + - ./app_provider:/var/lib/postgresql/data + - ./create-multiple-postgresql-databases.sh:/docker-entrypoint-initdb.d/create-multiple-postgresql-databases.sh + + postgres_consumer: + image: postgres:16.2-alpine3.19 + hostname: postgres_consumer + ports: + - "5433:5432" + networks: + - consumer + env_file: + - ./postgres_consumer.env + volumes: + - ./app_consumer:/var/lib/postgresql/data + - ./create-multiple-postgresql-databases.sh:/docker-entrypoint-initdb.d/create-multiple-postgresql-databases.sh + + +``` + + - Add dependency for ECC and UCApp to postgres + +``` + ecc-provider: + ... + depends_on: + - postgres_provider + + ecc-consumer: + ... + depends_on: + - postgres_consumer + + uc-dataapp-provider: + ... + depends_on: + - postgres_provider + + uc-dataapp-consumer: + ... + depends_on: + - postgres_consumer + +``` + + - Modify usage control property files, for `ecc_resources_provider` + +``` +#H2 properties +#spring.datasource.url=jdbc:h2:file:/home/nobody/data/audit_logs_provider;CIPHER=AES +#spring.jpa.database-platform=org.hibernate.dialect.H2Dialect +#spring.datasource.driver-class-name=org.h2.Driver +spring.h2.console.enabled=false + +##PostgreSQL +spring.jpa.database=POSTGRESQL +spring.datasource.url = jdbc:postgresql://postgres_provider:5432/ecc_provider +spring.datasource.driver-class-name = org.postgresql.Driver +spring.jpa.database-platform = org.hibernate.dialect.PostgreSQLDialect + +spring.datasource.username=connector +spring.datasource.password=12345 +spring.jpa.show-sql=false +spring.jpa.generate-ddl=true +spring.jpa.hibernate.ddl-auto=update +spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true +``` + +and `ecc_resources_consumer` + +``` +#H2 properties +#spring.datasource.url=jdbc:h2:file:/home/nobody/data/audit_logs_consumer;CIPHER=AES +#spring.jpa.database-platform=org.hibernate.dialect.H2Dialect +#spring.datasource.driver-class-name=org.h2.Driver +spring.h2.console.enabled=false + +##PostgreSQL +spring.jpa.database=POSTGRESQL +spring.datasource.url = jdbc:postgresql://postgres_consumer:5432/ecc_consumer +spring.datasource.driver-class-name = org.postgresql.Driver +spring.jpa.database-platform = org.hibernate.dialect.PostgreSQLDialect + +spring.datasource.username=connector +spring.datasource.password=12345 +spring.jpa.show-sql=false +spring.jpa.generate-ddl=true +spring.jpa.hibernate.ddl-auto=update +spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true + +``` + +and `uc-dataapp_resources_provider` + +``` +## H2 DB with persisting on disk +#spring.datasource.url=jdbc:h2:file:/home/nobody/data/platoon_db_provider;CIPHER=AES +#spring.jpa.database-platform=org.hibernate.dialect.H2Dialect +#spring.datasource.driver-class-name=org.h2.Driver +spring.h2.console.enabled=false + +## PostgreSQL +spring.jpa.database=POSTGRESQL +spring.datasource.url = jdbc:postgresql://postgres_provider:5432/usagecontrol_provider +spring.datasource.driver-class-name = org.postgresql.Driver +spring.jpa.database-platform = org.hibernate.dialect.PostgreSQLDialect + +spring.datasource.platform = usagecontrol +spring.datasource.username = connector +spring.datasource.password = 12345 +spring.jpa.show-sql=false +spring.jpa.generate-ddl=true +spring.jpa.hibernate.ddl-auto=update +spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true +``` + +and `uc-dataapp_resources_consumer` + +``` +## H2 DB with persisting on disk +#spring.datasource.url=jdbc:h2:file:/home/nobody/data/platoon_db_consumer;CIPHER=AES +#spring.jpa.database-platform=org.hibernate.dialect.H2Dialect +#spring.datasource.driver-class-name=org.h2.Driver +spring.h2.console.enabled=false + +# PostgreSQL +spring.jpa.database=POSTGRESQL +spring.datasource.url = jdbc:postgresql://postgres_consumer:5432/usagecontrol_consumer +spring.datasource.driver-class-name = org.postgresql.Driver +spring.jpa.database-platform = org.hibernate.dialect.PostgreSQLDialect + +spring.datasource.platform = usagecontrol +spring.datasource.username = connector +spring.datasource.password = 12345 +spring.jpa.show-sql=false +spring.jpa.generate-ddl=true +spring.jpa.hibernate.ddl-auto=update +spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true +``` + + - Postgres env file + +2 env files needed for PostgreSQL should be created, in root of TRUE Connector directory: + +`postgres_provider.env` with content + +``` +POSTGRES_USER=connector +POSTGRES_PASSWORD=12345 +POSTGRES_MULTIPLE_DATABASES=usagecontrol_provider, ecc_provider +``` + +`postgres_consumer.env` with content + +``` +POSTGRES_USER=connector +POSTGRES_PASSWORD=12345 +POSTGRES_MULTIPLE_DATABASES=usagecontrol_consumer, ecc_consumer +``` + + - Script for creating multiple databases + +Script `create-multiple-postgresql-databases.sh` should be created with content: + +``` +#!/bin/bash + +set -e +set -u + +function create_database() { + local database=$1 + echo " Creating database '$database'" + psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname="postgres" <<-EOSQL + CREATE DATABASE $database; + GRANT ALL PRIVILEGES ON DATABASE $database TO $POSTGRES_USER; +EOSQL +} + +if [ -n "$POSTGRES_MULTIPLE_DATABASES" ]; then + echo "Multiple database creation requested: $POSTGRES_MULTIPLE_DATABASES" + for db in $(echo $POSTGRES_MULTIPLE_DATABASES | tr ',' ' '); do + create_database $db + done + echo "Multiple databases created" +fi +``` + +After saving script, please ensure that script is executable. If it is not executable, you can make it executable using the following terminal command: + +```bash +chmod +x create-multiple-postgresql-databases.sh +``` +