Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use a unique set of definitions to initialize the database #834

Merged
27 commits merged into from Sep 15, 2021
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 7 additions & 9 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -112,17 +112,15 @@ before_install:
- ln -s ../../../../share ./lib/auto/share/dist/Zonemaster-Backend

before_script:
- if [[ "$TARGET" == "PostgreSQL" ]]; then psql -c "create user travis_zonemaster WITH PASSWORD 'travis_zonemaster';" -U postgres; fi
- if [[ "$TARGET" == "PostgreSQL" ]]; then psql -c 'create database travis_zonemaster OWNER travis_zonemaster;' -U postgres; fi
- if [[ "$TARGET" == "PostgreSQL" ]]; then psql -U postgres -c "CREATE USER travis_zonemaster WITH PASSWORD 'travis_zonemaster';"; fi
- if [[ "$TARGET" == "PostgreSQL" ]]; then psql -U postgres -c 'CREATE DATABASE travis_zonemaster OWNER travis_zonemaster;'; fi
- if [[ "$TARGET" == "PostgreSQL" ]]; then cpanm DBD::Pg; fi
- if [[ "$TARGET" == "PostgreSQL" ]]; then perl -I./lib ./script/create_db_postgresql_9.3.pl; fi

- if [[ "$TARGET" == "MySQL" ]]; then mysql -e "CREATE USER 'travis_zm'@'localhost' IDENTIFIED BY 'travis_zonemaster';" -u root; fi
- if [[ "$TARGET" == "MySQL" ]]; then mysql -e "CREATE DATABASE travis_zonemaster CHARACTER SET utf8 COLLATE utf8_bin;" -u root; fi
- if [[ "$TARGET" == "MySQL" ]]; then mysql -e "GRANT ALL ON travis_zonemaster.* TO 'travis_zm'@'localhost';" -u root; fi
- if [[ "$TARGET" == "MySQL" ]]; then mysql -e "FLUSH PRIVILEGES;" -u root; fi
- if [[ "$TARGET" == "MySQL" ]]; then cpanm --force DBD::mysql; fi
- if [[ "$TARGET" == "MySQL" ]]; then perl -I./lib ./script/create_db_mysql.pl; fi
- if [[ "$TARGET" == "MySQL" ]]; then mysql -u root -e "CREATE USER 'travis_zm'@'localhost' IDENTIFIED BY 'travis_zonemaster';"; fi
- if [[ "$TARGET" == "MySQL" ]]; then mysql -u root -e "CREATE DATABASE travis_zonemaster CHARACTER SET utf8 COLLATE utf8_bin;"; fi
- if [[ "$TARGET" == "MySQL" ]]; then mysql -u root -e "GRANT ALL ON travis_zonemaster.* TO 'travis_zm'@'localhost';"; fi
- if [[ "$TARGET" == "MySQL" ]]; then cpanm DBD::mysql; fi

script:
- perl -I./lib ./share/create_db.pl
- perl Makefile.PL && make test
6 changes: 1 addition & 5 deletions MANIFEST
Original file line number Diff line number Diff line change
Expand Up @@ -40,19 +40,15 @@ MANIFEST This list of files
META.yml
README.md
script/add-batch-job.pl
script/create_db_mysql.pl
script/create_db_postgresql_9.3.pl
script/zmb
script/zmtest
script/zonemaster_backend_rpcapi.psgi
script/zonemaster_backend_testagent
share/backend_config.ini
share/cleanup-mysql.sql
share/cleanup-postgres.sql
share/create_db_sqlite.pl
share/create_db.pl
share/freebsd-pwd.conf
share/initial-mysql.sql
share/initial-postgres.sql
share/patch_db_README.txt
share/patch_mysql_db_zonemaster_backend_ver_1.0.3.pl
share/patch_mysql_db_zonemaster_backend_ver_5.0.0.pl
Expand Down
179 changes: 117 additions & 62 deletions docs/Installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,22 @@
* [2. Prerequisites](#2-prerequisites)
* [3. Installation on CentOS](#3-installation-on-centos)
* [3.1 Install Zonemaster::Backend and related dependencies (CentOS)](#31-install-zonemasterbackend-and-related-dependencies-centos)
* [3.2 Database engine installation and configuration (CentOS)](#32-database-engine-installation-and-configuration-centos)
* [3.3 Service configuration and startup (CentOS)](#33-service-configuration-and-startup-centos)
* [3.4 Post-installation (CentOS)](#34-post-installation-centos)
* [3.2 Database engine installation (CentOS)](#32-database-engine-installation-centos)
* [3.3 Database configuration (CentOS)](#33-database-configuration-centos)
* [3.4 Service configuration and startup (CentOS)](#34-service-configuration-and-startup-centos)
* [3.5 Post-installation (CentOS)](#35-post-installation-centos)
* [4. Installation on Debian and Ubuntu](#4-installation-on-debian-and-ubuntu)
* [4.1 Install Zonemaster::Backend and related dependencies (Debian/Ubuntu)](#41-install-zonemasterbackend-and-related-dependencies-debianubuntu)
* [4.2 Database engine installation and configuration (Debian/Ubuntu)](#42-database-engine-installation-and-configuration-debianubuntu)
* [4.3 Service configuration and startup (Debian/Ubuntu)](#43-service-configuration-and-startup-debianubuntu)
* [4.4 Post-installation (Debian/Ubuntu)](#44-post-installation-debianubuntu)
* [4.2 Database engine installation (Debian/Ubuntu)](#42-database-engine-installation-debianubuntu)
* [4.3 Database configuration (Debian/Ubuntu)](#43-database-configuration-debianubuntu)
* [4.4 Service configuration and startup (Debian/Ubuntu)](#44-service-configuration-and-startup-debianubuntu)
* [4.5 Post-installation (Debian/Ubuntu)](#45-post-installation-debianubuntu)
* [5. Installation on FreeBSD](#5-installation-on-freebsd)
* [5.1 Install Zonemaster::Backend and related dependencies (FreeBSD)](#51-install-zonemasterbackend-and-related-dependencies-freebsd)
* [5.2 Database engine installation and configuration (FreeBSD)](#52-database-engine-installation-and-configuration-freebsd)
* [5.3 Service startup (FreeBSD)](#53-service-startup-freebsd)
* [5.4 Post-installation (FreeBSD)](#54-post-installation-freebsd)
* [5.2 Database engine installation (FreeBSD)](#52-database-engine-installation-freebsd)
* [5.3 Database configuration (FreeBSD)](#53-database-configuration-freebsd)
* [5.4 Service startup (FreeBSD)](#54-service-startup-freebsd)
* [5.5 Post-installation (FreeBSD)](#55-post-installation-freebsd)
* [6. Post-installation](#6-post-installation)
* [6.1 Smoke test](#61-smoke-test)
* [6.2 What to do next?](#62-what-to-do-next)
Expand Down Expand Up @@ -107,7 +110,7 @@ sudo install -v -m 755 ./tmpfiles.conf /usr/lib/tmpfiles.d/zonemaster.conf
> previous version of Zonemaster-Backend).


### 3.2 Database engine installation and configuration (CentOS)
### 3.2 Database engine installation (CentOS)

Check the [declaration of prerequisites] to make sure your preferred combination
of operating system version and database engine version is supported.
Expand All @@ -126,13 +129,10 @@ but if you have removed the old Zonemaster database, then do the initialization.
> **Note:** Zonemaster with SQLite is not meant for an installation with heavy
> load.

Create database directory, database and set correct ownership:
Create database directory:

```sh
cd `perl -MFile::ShareDir=dist_dir -E 'say dist_dir("Zonemaster-Backend")'`
sudo install -v -m 755 -o zonemaster -g zonemaster -d /var/lib/zonemaster
sudo perl create_db_sqlite.pl
sudo chown zonemaster:zonemaster /var/lib/zonemaster/db.sqlite
```

> Some parameters can be changed, see the [backend configuration] documentation
Expand All @@ -145,7 +145,17 @@ See appendices for [MariaDB][MariaDB instructions for CentOS] and
[PostgreSQL][PostgreSQL instructions for CentOS].


### 3.3 Service configuration and startup (CentOS)
### 3.3 Database configuration (CentOS)

Finally initialize the database:

```sh
cd `perl -MFile::ShareDir -le 'print File::ShareDir::dist_dir("Zonemaster-Backend")'`
sudo -u zonemaster perl create_db.pl
```
matsduf marked this conversation as resolved.
Show resolved Hide resolved


### 3.4 Service configuration and startup (CentOS)

Make sure our tmpfiles configuration takes effect:

Expand All @@ -163,7 +173,7 @@ sudo systemctl start zm-testagent
```


### 3.4 Post-installation (CentOS)
### 3.5 Post-installation (CentOS)

See the [post-installation] section for post-installation matters.

Expand Down Expand Up @@ -238,7 +248,7 @@ sudo install -v -m 755 ./tmpfiles.conf /usr/lib/tmpfiles.d/zonemaster.conf
> `/etc/init.d/zm-backend.sh` (script from previous version of Zonemaster-Backend).


### 4.2 Database engine installation and configuration (Debian/Ubuntu)
### 4.2 Database engine installation (Debian/Ubuntu)

Check the [declaration of prerequisites] to make sure your preferred combination
of operating system version and database engine version is supported.
Expand All @@ -257,13 +267,10 @@ but if you have removed the old Zonemaster database, then do the initialization.
> **Note:** Zonemaster with SQLite is not meant for an installation with heavy
> load.

Create database directory, database and set correct ownership:
Create database directory:

```sh
cd `perl -MFile::ShareDir=dist_dir -E 'say dist_dir("Zonemaster-Backend")'`
sudo install -v -m 755 -o zonemaster -g zonemaster -d /var/lib/zonemaster
sudo perl create_db_sqlite.pl
sudo chown zonemaster:zonemaster /var/lib/zonemaster/db.sqlite
```

> Some parameters can be changed, see the [backend configuration] documentation
Expand All @@ -276,7 +283,17 @@ See appendices for [MariaDB][MariaDB instructions for Debian] and
[PostgreSQL][PostgreSQL instructions for Debian].


### 4.3 Service configuration and startup (Debian/Ubuntu)
### 4.3 Database configuration (Debian/Ubuntu)

Finally initialize the database:

```sh
cd `perl -MFile::ShareDir -le 'print File::ShareDir::dist_dir("Zonemaster-Backend")'`
sudo -u zonemaster perl create_db.pl
```
matsduf marked this conversation as resolved.
Show resolved Hide resolved


### 4.4 Service configuration and startup (Debian/Ubuntu)

Make sure our tmpfiles configuration takes effect:

Expand All @@ -294,7 +311,7 @@ sudo systemctl start zm-testagent
```


### 4.4 Post-installation (Debian/Ubuntu)
### 4.5 Post-installation (Debian/Ubuntu)

See the [post-installation] section for post-installation matters.

Expand Down Expand Up @@ -349,7 +366,7 @@ install -v -m 755 ./zm_rpcapi-bsd /usr/local/etc/rc.d/zm_rpcapi
install -v -m 755 ./zm_testagent-bsd /usr/local/etc/rc.d/zm_testagent
```

### 5.2 Database engine installation and configuration (FreeBSD)
### 5.2 Database engine installation (FreeBSD)

Check the [declaration of prerequisites] to make sure your preferred combination
of operating system version and database engine version is supported.
Expand All @@ -373,12 +390,10 @@ Configure Zonemaster::Backend to use the correct database path:
sed -i '' '/[[:<:]]database_file[[:>:]]/ s:=.*:= /var/db/zonemaster/db.sqlite:' /usr/local/etc/zonemaster/backend_config.ini
```

Create database directory and database with correct ownership:
Create database directory:

```sh
cd `perl -MFile::ShareDir -le 'print File::ShareDir::dist_dir("Zonemaster-Backend")'`
install -v -m 755 -o zonemaster -g zonemaster -d /var/db/zonemaster
env ZONEMASTER_BACKEND_CONFIG_FILE=/usr/local/etc/zonemaster/backend_config.ini su -m zonemaster -c "perl create_db_sqlite.pl"
```

> Some parameters can be changed, see the [backend configuration] documentation
Expand All @@ -390,7 +405,17 @@ See appendices for [MariaDB][MariaDB instructions for FreeBSD] and
[PostgreSQL][PostgreSQL instructions for FreeBSD].


### 5.3 Service startup (FreeBSD)
### 5.3 Database configuration (FreeBSD)

Finally initialize the database:

```sh
cd `perl -MFile::ShareDir -le 'print File::ShareDir::dist_dir("Zonemaster-Backend")'`
su -m zonemaster -c "perl create_db.pl"
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion to make it easier to just copy and paste:

  1. Make create_db.pl executable
  2. Make the command a one-liner:
su -m zonemaster -c "$(perl -MFile::ShareDir -le 'print File::ShareDir::dist_dir("Zonemaster-Backend")')/create_db.pl"

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry, for FreeBSD it was a bad suggestion from me. Default shell for FreeBSD root is C shell (or tcsh), and that does not support "$( )". Bourne shell (or sh) is the default shell for other users, and that supports "$( )". Back ticks have to be used, and that affects <"> too.

su -m zonemaster -c "`perl -MFile::ShareDir -le 'print File::ShareDir::dist_dir(qw(Zonemaster-Backend))'`/create_db.pl"

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be fixed (and I realized that I forgot to remove the call to perl since the script is now executable).



### 5.4 Service startup (FreeBSD)

Enable services at startup:

Expand All @@ -406,7 +431,7 @@ service zm_rpcapi start
service zm_testagent start
```

### 5.4 Post-installation (FreeBSD)
### 5.5 Post-installation (FreeBSD)

To check the running daemons run:

Expand Down Expand Up @@ -477,15 +502,25 @@ sudo systemctl enable mariadb
sudo systemctl start mariadb
```

Initialize the database (unless you keep an old database):
To initialize the database (unless you keep an old database) connect to the
MariaDB server:

```sh
sudo mysql < $(perl -MFile::ShareDir=dist_dir -E 'say dist_dir("Zonemaster-Backend")')/initial-mysql.sql
sudo mysql
```

Create the database:
```sql
CREATE DATABASE zonemaster;
```

Create a new user and give it all permissions on the newly created database:
```sql
CREATE USER 'zonemaster'@'localhost' IDENTIFIED BY 'zonemaster';
GRANT ALL ON zonemaster.* TO 'zonemaster'@'localhost';
```

matsduf marked this conversation as resolved.
Show resolved Hide resolved
> **Note:** This creates a database called `zonemaster`, as well as a user
> called "zonemaster" with the password "zonemaster" (as stated in the config
> file). This user has just enough permissions to run the backend software.
Then update the `/etc/zonemaster/backend_config.ini` file accordingly.
matsduf marked this conversation as resolved.
Show resolved Hide resolved


#### A.2. MariaDB installation on Debian and Ubuntu
Expand All @@ -504,15 +539,25 @@ sudo sed -i '/\bengine\b/ s/=.*/= MySQL/' /etc/zonemaster/backend_config.ini

> **Note:** See the [backend configuration] documentation for details.

Initialize Zonemaster database (unless you keep an old database):
To initialize the database (unless you keep an old database) connect to the
MariaDB server:

```sh
sudo mysql < $(perl -MFile::ShareDir=dist_dir -E 'say dist_dir("Zonemaster-Backend")')/initial-mysql.sql
sudo mysql
```

Create the database:
```sql
CREATE DATABASE zonemaster;
```

Create a new user and give it all permissions on the newly created database:
```sql
CREATE USER 'zonemaster'@'localhost' IDENTIFIED BY 'zonemaster';
GRANT ALL ON zonemaster.* TO 'zonemaster'@'localhost';
```

matsduf marked this conversation as resolved.
Show resolved Hide resolved
> **Note:** This creates a database called `zonemaster`, as well as a user
> called "zonemaster" with the password "zonemaster" (as stated in the config
> file). This user has just enough permissions to run the backend software.
Then update the `/etc/zonemaster/backend_config.ini` file accordingly.
matsduf marked this conversation as resolved.
Show resolved Hide resolved


#### A.3. MySQL installation on FreeBSD
Expand Down Expand Up @@ -554,23 +599,24 @@ Reset root password in MySQL (required by MySQL). Replace
ALTER USER 'root'@'localhost' IDENTIFIED BY '<selected root password>';
```

Logout from database:
Unless you keep an old database, initialize the database:
```sql
CREATE DATABASE zonemaster;
```

Create a new user and give it all permissions on the newly created database:
```sql
exit;
CREATE USER 'zonemaster'@'localhost' IDENTIFIED BY 'zonemaster';
GRANT ALL ON zonemaster.* TO 'zonemaster'@'localhost';
```

Unless you keep an old database, initialize the database (and give the
root password when prompted):
Logout from database:

```sh
cd `perl -MFile::ShareDir -le 'print File::ShareDir::dist_dir("Zonemaster-Backend")'`
mysql -u root -p < ./initial-mysql.sql
```sql
exit;
```

> **Note:** This creates a database called `zonemaster`, as well as a user
> called "zonemaster" with the password "zonemaster" (as stated in the config
> file). This user has just enough permissions to run the backend software.
Then update the `/etc/zonemaster/backend_config.ini` file accordingly.
matsduf marked this conversation as resolved.
Show resolved Hide resolved


### B. Installation with PostgreSQL
Expand Down Expand Up @@ -613,15 +659,17 @@ Install, configure and start database engine:
sudo systemctl start postgresql
```

Initialize Zonemaster database (unless you keep an old database):
Initialize Zonemaster database and user (unless you keep an old database):

```sh
sudo -u postgres psql -f $(perl -MFile::ShareDir=dist_dir -E 'say dist_dir("Zonemaster-Backend")')/initial-postgres.sql
sudo -u postgres createuser -P zonemaster
```

> **Note:** This creates a database called `zonemaster`, as well as a user called
> "zonemaster" with the password "zonemaster" (as stated in the config file).
> This user has just enough permissions to run the backend software.
This will ask for a password that should be copied in `backend_config.ini`.

```sh
sudo -u postgres createdb -O zonemaster -E UTF8 zonemaster
```


#### B.2. PostgreSQL installation on Debian and Ubuntu
Expand All @@ -640,15 +688,17 @@ sudo apt install postgresql libdbd-pg-perl

> **Note:** See the [backend configuration] documentation for details.

Initialize Zonemaster database (unless you keep an old database):
Initialize Zonemaster database and user (unless you keep an old database):

```sh
sudo -u postgres psql -f $(perl -MFile::ShareDir=dist_dir -E 'say dist_dir("Zonemaster-Backend")')/initial-postgres.sql
sudo -u postgres createuser -P zonemaster
```

> **Note:** This creates a database called `zonemaster`, as well as a user called
> "zonemaster" with the password "zonemaster" (as stated in the config file).
> This user has just enough permissions to run the backend software.
This will ask for a password that should be copied in `backend_config.ini`.

```sh
sudo -u postgres createdb -O zonemaster -E UTF8 zonemaster
```


#### B.3. PostgreSQL installation on FreeBSD
Expand All @@ -669,11 +719,16 @@ service postgresql initdb
service postgresql start
```

Initialize Zonemaster database (unless you keep an old database):
Initialize Zonemaster database and user (unless you keep an old database):

```sh
cd `perl -MFile::ShareDir -le 'print File::ShareDir::dist_dir("Zonemaster-Backend")'`
psql -U postgres -f ./initial-postgres.sql
sudo -u postgres createuser -P zonemaster
```

This will ask for a password that should be copied in `backend_config.ini`.

```sh
sudo -u postgres createdb -O zonemaster -E UTF8 zonemaster
```


Expand Down
Loading