diff --git a/.gitattributes b/.gitattributes index 2757b057..f79c1a17 100644 --- a/.gitattributes +++ b/.gitattributes @@ -12,9 +12,13 @@ gradlew text eol=lf diff=bash *.patch text eol=lf diff=batch *.properties text eol=lf *.sh text eol=lf diff=bash +*.toml text eol=lf diff=bash +*.tmpl text eol=lf diff=bash APKBUILD text eol=lf diff=batch finish text eol=lf diff=bash run text eol=lf diff=bash +drush text eol=lf diff=bash +composer text eol=lf diff=bash # These files are binary and should be left untouched # (binary is a macro for -text -diff) diff --git a/.github/workflows/master.yml b/.github/workflows/master.yml index c3023f2c..59557b0c 100644 --- a/.github/workflows/master.yml +++ b/.github/workflows/master.yml @@ -27,7 +27,7 @@ jobs: - name: Enable buildkit shell: bash run: | - echo '{"experimental": "enabled"}' > ~/.docker/config.json + echo '{"experimental": "enabled", "dns": ["8.8.8.8", "8.8.4.4"]}' > ~/.docker/config.json - name: Build Docker images uses: eskatos/gradle-command-action@v1 with: diff --git a/.gitignore b/.gitignore index d9ed2266..cd8fe820 100644 --- a/.gitignore +++ b/.gitignore @@ -4,3 +4,4 @@ scratch scratch.md volumes +docker-compose.yml \ No newline at end of file diff --git a/.vscode/settings.json b/.vscode/settings.json index 33df578f..7d2d2398 100644 --- a/.vscode/settings.json +++ b/.vscode/settings.json @@ -9,6 +9,8 @@ "*.properties.tmpl": "properties", "*.sh.tmpl": "shellscript", "*.sql.tmpl": "sql", - "*.xml.tmpl": "xml" + "*.xml.tmpl": "xml", + "drush": "shellscript", + "composer": "shellscript" } } \ No newline at end of file diff --git a/README.md b/README.md index be5dfff7..7269a6a1 100644 --- a/README.md +++ b/README.md @@ -11,13 +11,7 @@ - [Build Specific Image](#build-specific-image) - [Building Continuously](#building-continuously) - [Running](#running) -- [Scripts](#scripts) - [Docker Images](#docker-images) -- [Docker Compose](#docker-compose) - - [Watchtower](#watchtower) - - [Traefik](#traefik) -- [ETCD](#etcd) -- [Customizing the Drupal Installation](#customizing-the-drupal-installation) - [Design Considerations](#design-considerations) - [Confd](#confd) - [S6 Overlay](#s6-overlay) @@ -26,13 +20,11 @@ - [Build System](#build-system) - [Design Constraints](#design-constraints) - [Issues / FAQ](#issues--faq) -- [To Do](#to-do) ## Introduction -This repository provides a number of docker images and an example -[docker-compose.yml](./docker-compose.yml) for running the demo version of -Islandora. It is not yet in a production ready state. +This repository provides a number of docker images which can be used to build an +Islandora 8 site. ## Requirements @@ -41,14 +33,8 @@ To build the Docker images using the provided Gradle build scripts with [BuildKi - [Docker 18.09+](https://docs.docker.com/get-docker/) - [OpenJDK or Oracle JDK 8+](https://www.java.com/en/download/) -To run the Docker images with Docker Compose requires: - -- [Docker 18.06+](https://docs.docker.com/get-docker/) -- [Docker Compose 1.22+](https://docs.docker.com/compose/install/) - -That being said the images themselves are compatible with older versions of -Docker, if you require running on an older version you'll need to write your own -docker-compose file. +That being said the images themselves are compatible of running with older +versions of Docker. ## Building @@ -158,55 +144,13 @@ of that image and any downstream dependencies. To do this add the ``` When this is combined with the use of ``watchtower`` and -``restart: unless-stopped`` in the [docker-compose.yml](./docker-compose.yml) -file. Images will be redeployed with the latest changes while you develop -automatically. See the [Docker Compose](#Docker-Compose) section of this -document for more details. +``restart: unless-stopped`` in a ``docker-compose.yml`` file. Images will be +redeployed with the latest changes while you develop automatically. ## Running -At the moment the example [docker-compose.yml] is the only orchestration -mechanism provided to launch all the containers, and have them work as a whole. - -To start the containers use the following command: - -```bash -docker-compose up -d -``` - -With [Docker Compose] there are many features such as displaying logs among -other things for which you can find detailed descriptions in the -[Docker Composer CLI Documentation](https://docs.docker.com/compose/reference/overview/) - -For more information on the structure and design of the example -[docker-compose.yml] file see the [Docker Compose](#Docker-Compose) section of -this document. - -## Scripts - -Some helper scripts are provided to make development and testing more pleasurable. - -- [./commands/drush.sh](./commands/drush.sh) - Wrapper around [drush] in the ``drupal service`` container. -- [./commands/etcdctrl.sh](./commands/etcdctrl.sh) - Wrapper around [etcdctrl] in the ``etcd service`` container. -- [./commands/mysql.sh](./commands/mysql.sh) - Wrapper around [mysql] client in the ``database service`` container. -- [./commands/open-in-browser.sh](./commands/shell.sh) - Attempts to open the given service in the users browser. -- [./commands/shell.sh](./commands/shell.sh) - Open ``ash`` shell in the given service container. - -All of the above commands include a usage statement, which can be accessed with ``-h`` flag like so: - -```bash -$ ./commands/shell.sh - usage: shell.sh SERVICE - - Opens an ash shell in the given SERVICE's container. - - OPTIONS: - -h --help Show this help. - -x --debug Debug this script. - - Examples: - shell.sh database -``` +There is no method for running the containers in `isle-buildkit`, instead please +refer to . ## Docker Images @@ -219,7 +163,6 @@ The following docker images are provided: - [blazegraph](./blazegraph/README.md) - [build](./build/README.md) - [cantaloupe](./cantaloupe/README.md) -- [composer](./composer/README.md) - [crayfish](./crayfish/README.md) - [crayfits](./crayfits/README.md) - [drupal](./drupal/README.md) @@ -237,7 +180,7 @@ The following docker images are provided: - [milliner](./milliner/README.md) - [nginx](./nginx/README.md) - [recast](./recast/README.md) -- [sandbox](./sandbox/README.md) +- [demo](./demo/README.md) - [solr](./solr/README.md) - [tomcat](./tomcat/README.md) @@ -245,135 +188,6 @@ Many are intermediate images used to build other images in the list, for example [java](./java/README.md). Please see the README of each image to find out what settings, and ports, are exposed and what functionality it provides. -## Docker Compose - -The example [docker-compose.yml] provided with this repository is a -template for those who wish to use [Docker Compose] for orchestration. The -images are not limited to running via [Docker Compose]. As time permits -additional tooling will be added to support [Kubernetes], and deploying to -[Amazon Elastic Container Service], and perhaps others. - -The example [docker-compose.yml] runs the [sandbox](./sandbox/README.md) version -of Islandora, for the purposes of testing the images, and Islandora. When -creating a ``docker-compose.yml`` for running a production Islandora 8 site. you -will use your own image. Please see -[Customizing the Drupal Installation](#customizing-the-drupal-installation) for -instructions on how to do so. - -In addition to the images provided as described in the section -[Docker Images](#docker-images). Several others are used by the -[docker-compose.yml] file. - -### Watchtower - -The [watchtower](https://hub.docker.com/r/v2tec/watchtower/) container monitors -the running Docker containers and watches for changes to the images that those -containers were originally started from. If watchtower detects that an image has -changed, it will automatically restart the container using the new image. This -allows for automatic deployment, and overall faster development time. - -Note however Watchtower will not restart stopped container or containers that -exited due to error. To ensure a container is always running, unless explicitly -stopped, add ``restart: unless-stopped`` property to the container in the -[docker-compose.yml] file. For example: - -```yaml -database: - image: islandora/mariadb:latest - restart: unless-stopped -``` - -### Traefik - -The [traefik](https://containo.us/traefik/) container acts as a reverse proxy, -and exposes some containers through port ``80`` on the localhost via the -[loopback](https://www.tldp.org/LDP/nag/node66.html). This allows access to the -following urls. - -- -- -- -- -- - -Note if you cannot map ``traefik`` to the hosts port 80, you will need to -manually modify your ``/etc/hosts`` file and add entries for each of the urls -above like so, assuming the IP of ``traefik`` container is ``x.x.x.x`` on its -virtual network, and you can access that address from your machine. - -```properties -x.x.x.x activemq.localhost -x.x.x.x blazegraph.localhost -x.x.x.x islandora.localhost -x.x.x.x fcrepo.localhost -x.x.x.x matomo.localhost -``` - -Since Drupal passes its ``Base URL`` along to other services in AS2 as a means -of allowing them to find their way back. As well as having services like Fedora -exposed at the same URL they are accessed by the micro-services to end users. We -need to allow containers within the network to be accessible via the same URL, -though not by routing through ``traefik`` since it is and edge router. - -So alias like the following are defined: - -```yaml -drupal: - image: islandora/sandbox:latest - # ... - networks: - default: - aliases: - - drupal.localhost -``` - -These are set on the ``default`` network as that is the internal network (no -access to the outside) on which all containers reside. - -## ETCD - -The [etcd](https://github.com/etcd-io/etcd) container is a distributed reliable -key-value store, which this project uses for configuration settings and secrets. -Chosen in particular for it's existing integration with -[Kubernetes](https://kubernetes.io/docs/concepts/overview/components/#etcd). - -Alternatively if removed from the [docker-compose.yml] file or explicitly not -started the containers will fall back to pulling configuration from -**environment variables**. - -A convenience script is provided that allows for users to put and get key/values -from the store after it has been started. For example changing the log level of -[houdini](./houdini/README.md) to ``DEBUG``. - -```bash -./commands/etcdctl.sh put /houdini/log/level DEBUG -``` - -Or checking what the current log level is set to (*if not set to the default, in -which case the key/value store is not used*): - -```bash -./commands/etcdctl.sh get /houdini/log/level -``` - -## Customizing the Drupal Installation - -This needs to be thought about more in-depth, and needs fleshing out. Ideally we -will provide the base image for building / deploying the Drupal image. End users -will consume these containers and provide their ``composer.json`` and -``composer.lock`` files, along with the name of the -[Drupal Installation Profile] (either the one we will provide or one they create -on their own). At the moment the [composer](./composer/README.md) project is -provided as an early example of this. Where it user the -``islandora/nginx as compose`` image to perform do the composer installation, -and ``islandora/drupal`` as the container to run the installation created in the -previous step. - -Additionally more documentation is needed to describe how developers can use the -existing images for local development allowing them to create their own -``composer.json`` and ``composer.lock`` files as well as any custom modules, -themes, etc. - ## Design Considerations All of the images build by this project are derived from the @@ -387,7 +201,8 @@ The [base](./base/README.md) image includes two tools essential to the functioning of all the images. - [Confd](https://github.com/kelseyhightower/confd) - Configuration Management -- [S6 Overlay](https://github.com/just-containers/s6-overlay) - Process Manager / Initialization system. +- [S6 Overlay](https://github.com/just-containers/s6-overlay) - Process Manager + / Initialization system. ### Confd @@ -498,7 +313,6 @@ are arranged in a hierarchy, that roughly follows below: │ └── fits ├── mariadb └── nginx - ├── composer ├── crayfish │ ├── gemini │ ├── homarus @@ -508,7 +322,7 @@ are arranged in a hierarchy, that roughly follows below: │ └── recast ├── crayfits ├── drupal - │ └── sandbox + │ └── demo └── matomo ``` @@ -522,8 +336,8 @@ To make reasoning about what files go where each image follows the same filesystem layout for copying files into the image. A folder called ``rootfs`` maps directly onto the linux filesystem of the final -image. So for example ``rootfs/opt/islandora/configs`` will be -``/opt/islandora/configs`` in the generated image. +image. So for example ``rootfs/etc/islandora/configs`` will be +``/etc/islandora/configs`` in the generated image. ### Build System @@ -551,14 +365,14 @@ wether it be ``etcd``, ``consul``, or ``environment variables``, containers can successfully start without any other container present. This does not completely remove dependencies between containers, for example, -when the [sandbox](../docker/sandbox/README.md) starts it requires a running +when the [demo](../docker/demo/README.md) starts it requires a running [fcrepo](../docker/fcrepo/README.md) to be able to ingest nodes created by ``islandora_default`` features. In these cases an initialization script can block until another container is available or a timeout has been reached. For example: ```bash -local fcrepo_host="{{ getv "/fcrepo/host" "fcrepo.localhost" }}" +local fcrepo_host="{{ getv "/fcrepo/host" "fcrepo.isle-dc.localhost" }}" local fcrepo_port="{{ getv "/fcrepo/host" "80" }}" local fcrepo_url= @@ -594,30 +408,4 @@ failed to solve with frontend dockerfile.v0: failed to solve with frontend gatew [Overlay2](https://docs.docker.com/storage/storagedriver/overlayfs-driver/#configure-docker-with-the-overlay-or-overlay2-storage-driver) filesystem with Docker. If that doesn't work trying building [without BuildKit](#building-without-buildkit). -## To Do - -- Blazegraph isn't working -- Check if Cantaloupe is working -- Add support for multiple backends to fedora (currently only file is being used) -- Confirm all derivative generation is working and check if additional tools - need to be build or custom builds of say `poppler utils` or some other tool, - etc are needed (need to test against a variety of inputs for each mimetype as - there may be some edge case). -- Change public/private key generation to not rely on shared volume use the configuration management -- Do we need to support tls for etcd? Probably not since it shouldn't be exposed outside of the network. Though would be nice. -- Change solr configuration to no rely on shared volume -- Ideally no shared volumes as then container can be more easily moved between nodes in a cluster -- Get working under ECS/EKS -- Get working with Kubernetes with auto scaling for micro-services -- Developer workflow documentation / examples - -[Amazon Elastic Container Service]: https://aws.amazon.com/ecs/ [Buildkit]: https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/experimental.md -[Docker Compose]: https://docs.docker.com/compose/ -[docker-compose.yml]: ./docker-compose.yml -[Drupal Installation Profile]: https://www.drupal.org/docs/8/distributions/creating-distributions/how-to-write-a-drupal-8-installation-profile -[drush]: https://drushcommands.com/ -[etcdctrl]: https://etcd.io/docs/v3.4.0/dev-guide/interacting_v3/ -[Kubernetes]: https://kubernetes.io/ -[mysql]: https://dev.mysql.com/doc/refman/8.0/en/mysql.html - diff --git a/alpaca/README.md b/alpaca/README.md index 6fb02361..bbe3964b 100644 --- a/alpaca/README.md +++ b/alpaca/README.md @@ -23,51 +23,51 @@ additional settings, volumes, ports, etc. ## Settings -| Environment Variable | Etcd Key | Default | Description | -| :----------------------------------------- | :----------------------------------------- | :-------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------- | -| ALPACA_ACTIVEMQ_PASSWORD | /alpaca/activemq/password | password | Password to authenticate with | -| ALPACA_ACTIVEMQ_URL | /alpaca/activemq/url | tcp://broker:61616 | The url for connecting to the ActiveMQ broker, shared by all components | -| ALPACA_ACTIVEMQ_USER | /alpaca/activemq/user | admin | User to authenticate as | -| ALPACA_FCREPO_AUTH_HOST | /alpaca/fcrepo/auth/host | | User to authenticate as | -| ALPACA_FCREPO_AUTH_PASSWORD | /alpaca/fcrepo/auth/password | | Password to authenticate with | -| ALPACA_FCREPO_AUTH_USER | /alpaca/fcrepo/auth/user | | URL to authenticate against | +| Environment Variable | Etcd Key | Default | Description | +| :----------------------------------------- | :----------------------------------------- | :--------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------- | +| ALPACA_ACTIVEMQ_PASSWORD | /alpaca/activemq/password | password | Password to authenticate with | +| ALPACA_ACTIVEMQ_URL | /alpaca/activemq/url | tcp://broker:61616 | The url for connecting to the ActiveMQ broker, shared by all components | +| ALPACA_ACTIVEMQ_USER | /alpaca/activemq/user | admin | User to authenticate as | +| ALPACA_FCREPO_AUTH_HOST | /alpaca/fcrepo/auth/host | | User to authenticate as | +| ALPACA_FCREPO_AUTH_PASSWORD | /alpaca/fcrepo/auth/password | | Password to authenticate with | +| ALPACA_FCREPO_AUTH_USER | /alpaca/fcrepo/auth/user | | URL to authenticate against | | ALPACA_FCREPO_URL | /alpaca/fcrepo/url | http://fcrepo/fcrepo/rest | The url of fcrepo rest API | -| ALPACA_FITS_QUEUE | /alpaca/fits/queue | broker:queue:islandora-connector-fits | ActiveMQ Queue to consume from | -| ALPACA_FITS_REDELIVERIES | /alpaca/fits/redeliveries | 10 | Number of attempts to redeliver if an exception occurs | -| ALPACA_FITS_SERVICE | /alpaca/fits/service | http://crayfits:8000 | Url of micro-service | -| ALPACA_HOMARUS_QUEUE | /alpaca/homarus/queue | broker:queue:islandora-connector-homarus | ActiveMQ Queue to consume from | -| ALPACA_HOMARUS_REDELIVERIES | /alpaca/homarus/redeliveries | 10 | Number of attempts to redeliver if an exception occurs | -| ALPACA_HOMARUS_SERVICE | /alpaca/homarus/service | http://homarus:8000/convert | Url of micro-service | -| ALPACA_HOUDINI_QUEUE | /alpaca/houdini/queue | broker:queue:islandora-connector-houdini | ActiveMQ Queue to consume from | -| ALPACA_HOUDINI_REDELIVERIES | /alpaca/houdini/redeliveries | 10 | Number of attempts to redeliver if an exception occurs | -| ALPACA_HOUDINI_SERVICE | /alpaca/houdini/service | http://houdini:8000/convert | Url of micro-service | -| ALPACA_HTTP_TOKEN | /alpaca/http/token | islandora | The static token value to be used for authentication by the HttpClient available as an OSGi service for other services to use against the Fedora repository | -| ALPACA_INDEXING_GEMINI_URL | /alpaca/indexing/gemini/url | http://gemini:8000 | Url of micro-service | -| ALPACA_INDEXING_MILLINER_URL | /alpaca/indexing/milliner/url | http://milliner:8000 | Url of micro-service | -| ALPACA_INDEXING_REDELIVERIES | /alpaca/indexing/redeliveries | 10 | Number of attempts to redeliver if an exception occurs | -| ALPACA_INDEXING_STREAM_FILE_DELETE | /alpaca/indexing/stream/file/delete | broker:queue:islandora-indexing-fcrepo-file-delete | ActiveMQ Queue to consume from | -| ALPACA_INDEXING_STREAM_FILE_INDEX | /alpaca/indexing/stream/file/index | broker:queue:islandora-indexing-fcrepo-file | ActiveMQ Queue to consume from | -| ALPACA_INDEXING_STREAM_INPUT | /alpaca/indexing/stream/input | broker:topic:fedora | ActiveMQ Topic to consume | -| ALPACA_INDEXING_STREAM_MEDIA_INDEX | /alpaca/indexing/stream/media/index | broker:queue:islandora-indexing-fcrepo-media | ActiveMQ Queue to consume from | -| ALPACA_INDEXING_STREAM_NODE_DELETE | /alpaca/indexing/stream/node/delete | broker:queue:islandora-indexing-fcrepo-delete | ActiveMQ Queue to consume from | -| ALPACA_INDEXING_STREAM_NODE_INDEX | /alpaca/indexing/stream/node/index | broker:queue:islandora-indexing-fcrepo-content | ActiveMQ Queue to consume from | -| ALPACA_INDEXING_STREAM_TRIPLESTORE_DELETE | /alpaca/indexing/stream/triplestore/delete | broker:queue:islandora-indexing-triplestore-delete | ActiveMQ Queue to consume from | -| ALPACA_INDEXING_STREAM_TRIPLESTORE_INDEX | /alpaca/indexing/stream/triplestore/index | broker:queue:islandora-indexing-triplestore-index | ActiveMQ Queue to consume from | -| ALPACA_INDEXING_STREAM_TRIPLESTORE_REINDEX | /alpaca/indexing/stream/reindex | broker:queue:triplestore.reindex | ActiveMQ Queue to consume from | +| ALPACA_FITS_QUEUE | /alpaca/fits/queue | broker:queue:islandora-connector-fits | ActiveMQ Queue to consume from | +| ALPACA_FITS_REDELIVERIES | /alpaca/fits/redeliveries | 10 | Number of attempts to redeliver if an exception occurs | +| ALPACA_FITS_SERVICE | /alpaca/fits/service | http://crayfits:8000 | Url of micro-service | +| ALPACA_HOMARUS_QUEUE | /alpaca/homarus/queue | broker:queue:islandora-connector-homarus | ActiveMQ Queue to consume from | +| ALPACA_HOMARUS_REDELIVERIES | /alpaca/homarus/redeliveries | 10 | Number of attempts to redeliver if an exception occurs | +| ALPACA_HOMARUS_SERVICE | /alpaca/homarus/service | http://homarus:8000/convert | Url of micro-service | +| ALPACA_HOUDINI_QUEUE | /alpaca/houdini/queue | broker:queue:islandora-connector-houdini | ActiveMQ Queue to consume from | +| ALPACA_HOUDINI_REDELIVERIES | /alpaca/houdini/redeliveries | 10 | Number of attempts to redeliver if an exception occurs | +| ALPACA_HOUDINI_SERVICE | /alpaca/houdini/service | http://houdini:8000/convert | Url of micro-service | +| ALPACA_HTTP_TOKEN | /alpaca/http/token | islandora | The static token value to be used for authentication by the HttpClient available as an OSGi service for other services to use against the Fedora repository | +| ALPACA_INDEXING_GEMINI_URL | /alpaca/indexing/gemini/url | http://gemini:8000 | Url of micro-service | +| ALPACA_INDEXING_MILLINER_URL | /alpaca/indexing/milliner/url | http://milliner:8000 | Url of micro-service | +| ALPACA_INDEXING_REDELIVERIES | /alpaca/indexing/redeliveries | 10 | Number of attempts to redeliver if an exception occurs | +| ALPACA_INDEXING_STREAM_FILE_DELETE | /alpaca/indexing/stream/file/delete | broker:queue:islandora-indexing-fcrepo-file-delete | ActiveMQ Queue to consume from | +| ALPACA_INDEXING_STREAM_FILE_INDEX | /alpaca/indexing/stream/file/index | broker:queue:islandora-indexing-fcrepo-file | ActiveMQ Queue to consume from | +| ALPACA_INDEXING_STREAM_INPUT | /alpaca/indexing/stream/input | broker:topic:fedora | ActiveMQ Topic to consume | +| ALPACA_INDEXING_STREAM_MEDIA_INDEX | /alpaca/indexing/stream/media/index | broker:queue:islandora-indexing-fcrepo-media | ActiveMQ Queue to consume from | +| ALPACA_INDEXING_STREAM_NODE_DELETE | /alpaca/indexing/stream/node/delete | broker:queue:islandora-indexing-fcrepo-delete | ActiveMQ Queue to consume from | +| ALPACA_INDEXING_STREAM_NODE_INDEX | /alpaca/indexing/stream/node/index | broker:queue:islandora-indexing-fcrepo-content | ActiveMQ Queue to consume from | +| ALPACA_INDEXING_STREAM_TRIPLESTORE_DELETE | /alpaca/indexing/stream/triplestore/delete | broker:queue:islandora-indexing-triplestore-delete | ActiveMQ Queue to consume from | +| ALPACA_INDEXING_STREAM_TRIPLESTORE_INDEX | /alpaca/indexing/stream/triplestore/index | broker:queue:islandora-indexing-triplestore-index | ActiveMQ Queue to consume from | +| ALPACA_INDEXING_STREAM_TRIPLESTORE_REINDEX | /alpaca/indexing/stream/reindex | broker:queue:triplestore.reindex | ActiveMQ Queue to consume from | | ALPACA_INDEXING_URL | /alpaca/indexing/url | http://blazegraph/bigdata/namespace/islandora/sparql | Url to triple store indexer | -| ALPACA_LOGGER_CAMEL_LEVEL | /alpaca/logger/camel/level | WARN | Camel [Log Level] | -| ALPACA_LOGGER_ISLANDORA_LEVEL | /alpaca/logger/islandora/level | WARN | Islandora [Log Level] | -| ALPACA_LOGGER_ROOT_LEVEL | /alpaca/logger/root/level | WARN | Root [Log Level] | -| ALPACA_OCR_QUEUE | /alpaca/ocr/queue | broker:queue:islandora-connector-ocr | ActiveMQ Queue to consume from | -| ALPACA_OCR_REDELIVERIES | /alpaca/ocr/redeliveries | 10 | Number of attempts to redeliver if an exception occurs | -| ALPACA_OCR_SERVICE | /alpaca/ocr/service | http://hypercube:8000 | Url of micro-service | +| ALPACA_LOGGER_CAMEL_LEVEL | /alpaca/logger/camel/level | WARN | Camel [Log Level] | +| ALPACA_LOGGER_ISLANDORA_LEVEL | /alpaca/logger/islandora/level | WARN | Islandora [Log Level] | +| ALPACA_LOGGER_ROOT_LEVEL | /alpaca/logger/root/level | WARN | Root [Log Level] | +| ALPACA_OCR_QUEUE | /alpaca/ocr/queue | broker:queue:islandora-connector-ocr | ActiveMQ Queue to consume from | +| ALPACA_OCR_REDELIVERIES | /alpaca/ocr/redeliveries | 10 | Number of attempts to redeliver if an exception occurs | +| ALPACA_OCR_SERVICE | /alpaca/ocr/service | http://hypercube:8000 | Url of micro-service | ## Logs -| Path | Description | -| :-------------------------------- | :--------------- | -| /opt/karaf/data/log/camel.log | Camel Log | -| /opt/karaf/data/log/islandora.log | Islandora Log | +| Path | Description | +| :-------------------------------- | :------------ | +| /opt/karaf/data/log/camel.log | Camel Log | +| /opt/karaf/data/log/islandora.log | Islandora Log | [Alpaca Documentation]: https://islandora.github.io/documentation/ [Alpaca]: https://github.com/Islandora/Alpaca diff --git a/base/rootfs/etc/profile b/base/rootfs/etc/profile new file mode 100644 index 00000000..ce272985 --- /dev/null +++ b/base/rootfs/etc/profile @@ -0,0 +1,16 @@ +export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin +export PAGER=less +export PS1='\h:\w\$ ' +umask 022 + +if [ "${BASH-}" ] && [ "$BASH" != "/bin/sh" ]; then + if [ -f /etc/bash.bashrc ]; then + . /etc/bash.bashrc + fi +fi + +for script in /etc/profile.d/*.sh ; do + if [ -r $script ] ; then + . $script + fi +done diff --git a/build.gradle.kts b/build.gradle.kts index 577b3f0a..e8af902c 100644 --- a/build.gradle.kts +++ b/build.gradle.kts @@ -6,4 +6,4 @@ buildscript { classpath("ca.islandora:isle-gradle-docker-plugin:0.0.1") } } -apply(plugin = "ca.islandora.gradle.docker") \ No newline at end of file +apply(plugin = "ca.islandora.gradle.docker") diff --git a/cantaloupe/Dockerfile b/cantaloupe/Dockerfile index 07181d47..038ccdc3 100644 --- a/cantaloupe/Dockerfile +++ b/cantaloupe/Dockerfile @@ -2,7 +2,6 @@ FROM local/tomcat:latest # Opted for OpenJPG over Kakadu but that could be changed. -# Check 00-cantaloupe-setup-environment for the defaults. # For reference see: https://cantaloupe-project.github.io/manual/3.3/processors.html RUN --mount=type=cache,target=/var/cache/apk \ --mount=type=cache,target=/etc/cache/apk \ @@ -21,11 +20,9 @@ RUN --mount=id=downloads,type=cache,target=/opt/downloads \ unzip "${DOWNLOAD_CACHE_DIRECTORY}/${CANTALOUPE_FILE}" -d /tmp && \ CANTALOUPE_UNPACKED="${CANTALOUPE_FILE%.zip}" && \ install-war-into-tomcat.sh --name "cantaloupe" --file "/tmp/${CANTALOUPE_UNPACKED}/${CANTALOUPE_UNPACKED}.war" && \ - rm -fr "/tmp/${CANTALOUPE_UNPACKED}" - - -COPY rootfs / - -RUN mkdir /data && \ + rm -fr "/tmp/${CANTALOUPE_UNPACKED}" && \ + mkdir /data && \ chown tomcat:tomcat /data && \ chown -R tomcat:tomcat /opt/tomcat + +COPY rootfs / diff --git a/commands/drush.sh b/commands/drush.sh deleted file mode 100755 index 09c9d9c7..00000000 --- a/commands/drush.sh +++ /dev/null @@ -1,71 +0,0 @@ -#!/usr/bin/env bash -set -e - -readonly PROGNAME=$(basename $0) -readonly PROGDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -readonly ROOT="$(realpath ${PROGDIR}/..)" - -readonly ARGS="$@" - -function usage { - cat <<- EOF - usage: $PROGNAME [OPTIONS] - - Executes the drush command inside of the drupal service container. - - OPTIONS: - -h --help Show this help. - -x --debug Debug this script. - - Additionally any options that are provided by the drush command. - - Examples: - Clear the Drupal cache: - $PROGNAME cr -EOF -} - -function cmdline { - local arg= - for arg - do - local delim="" - case "$arg" in - # Translate --gnu-long-options to -g (short options) - --help) args="${args}-h ";; - --debug) args="${args}-x ";; - # Pass through anything else - *) [[ "${arg:0:1}" == "-" ]] || delim="\"" - args="${args}${delim}${arg}${delim} ";; - esac - done - - # Reset the positional parameters to the short options - eval set -- $args - - # Ignore illegal options as they get passed to mysql. - while getopts "hx" OPTION &> /dev/null; - do - case $OPTION in - h) - usage - exit 0 - ;; - x) - readonly DEBUG='-x' - set -x - ;; - esac - done - - # Check if the service exists and is running. - [[ "$(docker-compose ps -q drupal)" == "" ]] && (echo "Drupal service is not running."; exit 1) - - return 0 -} - -function main { - cmdline ${ARGS} - docker-compose -f "${ROOT}/docker-compose.yml" exec drupal s6-setuidgid nginx php -d memory_limit=-1 /usr/local/bin/drush ${ARGS} -} -main diff --git a/commands/etcdctl.sh b/commands/etcdctl.sh deleted file mode 100755 index 86069c96..00000000 --- a/commands/etcdctl.sh +++ /dev/null @@ -1,80 +0,0 @@ -#!/usr/bin/env bash -set -e - -readonly PROGNAME=$(basename $0) -readonly PROGDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -readonly ROOT="$(realpath ${PROGDIR}/..)" - -readonly ARGS="$@" - -function usage { - cat <<- EOF - usage: $PROGNAME [OPTIONS] [PARAMS] - - Executes etcdctl command inside of the etcd service container. - - OPTIONS: - -h --help Show this help. - -x --debug Debug this script. - - Additionally any options that are provided by etcdctl. - - Examples: - Display etcdctl help: - $PROGNAME help - - Put a key/value into the store: - $PROGNAME put /houdini/log/level DEBUG - - Get a value for the given key: - $PROGNAME get /houdini/log/level - - Get help for sub-command: - $PROGNAME put -h -EOF -} - -function cmdline { - local arg= - for arg - do - local delim="" - case "$arg" in - # Translate --gnu-long-options to -g (short options) - --help) args="${args}-h ";; - --debug) args="${args}-x ";; - # Pass through anything else - *) [[ "${arg:0:1}" == "-" ]] || delim="\"" - args="${args}${delim}${arg}${delim} ";; - esac - done - - # Reset the positional parameters to the short options - eval set -- $args - - # Ignore illegal options as they get passed to etcdctl. - while getopts "hx" OPTION &> /dev/null; - do - case $OPTION in - h) - usage - exit 0 - ;; - x) - readonly DEBUG='-x' - set -x - ;; - esac - done - - # Check if the service exists and is running. - [[ "$(docker-compose ps -q etcd)" == "" ]] && (echo "Etcd service is not running."; exit 1) - - return 0 -} - -function main { - cmdline ${ARGS} - docker-compose -f "${ROOT}/docker-compose.yml" exec etcd etcdctl ${ARGS} -} -main diff --git a/commands/mysql.sh b/commands/mysql.sh deleted file mode 100755 index b436ffbd..00000000 --- a/commands/mysql.sh +++ /dev/null @@ -1,71 +0,0 @@ -#!/usr/bin/env bash -set -e - -readonly PROGNAME=$(basename $0) -readonly PROGDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -readonly ROOT="$(realpath ${PROGDIR}/..)" - -readonly ARGS="$@" - -function usage { - cat <<- EOF - usage: $PROGNAME [OPTIONS] [database] - - Executes the mysql client inside of the database service container. - - OPTIONS: - -h --help Show this help. - -x --debug Debug this script. - - Additionally any options that are provided by the mysql client. - - Examples: - $PROGNAME -u root -p drupal_default -EOF -} - -function cmdline { - local arg= - for arg - do - local delim="" - case "$arg" in - # Translate --gnu-long-options to -g (short options) - --help) args="${args}-h ";; - --debug) args="${args}-x ";; - # Pass through anything else - *) [[ "${arg:0:1}" == "-" ]] || delim="\"" - args="${args}${delim}${arg}${delim} ";; - esac - done - - # Reset the positional parameters to the short options - eval set -- $args - - # Ignore illegal options as they get passed to mysql. - while getopts "hx" OPTION &> /dev/null; - do - case $OPTION in - h) - usage - exit 0 - ;; - x) - readonly DEBUG='-x' - set -x - ;; - esac - done - - # Check if the service exists and is running. - [[ "$(docker-compose ps -q database)" == "" ]] && (echo "Database service is not running."; exit 1) - - return 0 -} - -function main { - cmdline ${ARGS} - docker-compose -f "${ROOT}/docker-compose.yml" exec database mysql ${ARGS} -} -main - diff --git a/commands/open-in-browser.sh b/commands/open-in-browser.sh deleted file mode 100755 index 072c18ad..00000000 --- a/commands/open-in-browser.sh +++ /dev/null @@ -1,122 +0,0 @@ -#!/usr/bin/env bash -set -e - -readonly PROGNAME=$(basename $0) -readonly PROGDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -readonly ARGS="$@" - -function usage { - cat <<- EOF - usage: $PROGNAME SERVICE - - Opens the given SERVICE in the users browser. - - OPTIONS: - -h --help Show this help. - -x --debug Debug this script. - - Examples: - Opens activemq web console: - $PROGNAME activemq -EOF -} - -function cmdline { - local arg= - for arg - do - local delim="" - case "$arg" in - # Translate --gnu-long-options to -g (short options) - --help) args="${args}-h ";; - --debug) args="${args}-x ";; - # Pass through anything else - *) [[ "${arg:0:1}" == "-" ]] || delim="\"" - args="${args}${delim}${arg}${delim} ";; - esac - done - - # Reset the positional parameters to the short options - eval set -- $args - - while getopts "hx" OPTION - do - case $OPTION in - h) - usage - exit 0 - ;; - x) - readonly DEBUG='-x' - set -x - ;; - esac - done - - shift $((OPTIND-1)) - readonly SERVICE="${1}" - - # Check if the service exists and is running. - docker-compose ps ${SERVICE} &> /dev/null || ( echo "Service ${SERVICE} does not exist."; exit 1 ) - [[ "$(docker-compose ps -q ${SERVICE})" == "" ]] && (echo "Service ${SERVICE} is not running."; exit 1) - - return 0 -} - -function open { - local url="${1}" - if [[ "$OSTYPE" == "linux-gnu"* ]]; then - xdg-open "${url}" - elif [[ "$OSTYPE" == "darwin"* ]]; then - osascript -e "open location \"${URL}\"" - elif [[ "$OSTYPE" == "cygwin" ]]; then - cygstart "${url}" - else - echo "Unknown OS ${OSTYPE}" - exit 1 - fi -} - -function image { - local service="${1}" - docker-compose images ${service} | tail -1 | awk '{print $1}' -} - -function ip { - local service="${1}" - local image=$(image ${service}) - local template="{{range .NetworkSettings.Networks}}{{println .IPAddress}}{{end}}" - # Assumes the default network is listed first. - docker inspect -f "${template}" "${image}" | head -n1 -} - -function url { - local service="${1}"; shift - local port="${1}"; shift - local path="${1}"; shift - echo "http://$(ip ${service}):${port}${path}" -} - -function main { - cmdline ${ARGS} - - case "${SERVICE}" in - activemq) open "http://activemq.localhost/admin" &> /dev/null;; - alpaca) open $(url alpaca 8181 /system/console) &> /dev/null;; - blazegraph) open "http://blazegraph.localhost/bigdata" &> /dev/null;; - cantaloupe) open "http://islandora.localhost/cantaloupe" &> /dev/null;; - crayfits) open $(url crayfits 8000 /) &> /dev/null;; - fcrepo) open "http://fcrepo.localhost/fcrepo/rest" &> /dev/null;; - gemini) open $(url gemini 8000 /) &> /dev/null;; - homarus) open $(url homarus 8000 /) &> /dev/null;; - houdini) open $(url houdini 8000 /) &> /dev/null;; - hypercube) open $(url hypercube 8000 /) &> /dev/null;; - drupal) open "http://islandora.localhost" &> /dev/null;; - milliner) open $(url milliner 8000 /) &> /dev/null;; - recast) open $(url recast 8000 /) &> /dev/null;; - solr) open "http://solr.localhost/solr" &> /dev/null;; - matomo) open "http://islandora.localhost/matomo" &> /dev/null;; - *) exit 1;; - esac -} -main diff --git a/commands/shell.sh b/commands/shell.sh deleted file mode 100755 index 59ee29cf..00000000 --- a/commands/shell.sh +++ /dev/null @@ -1,73 +0,0 @@ -#!/usr/bin/env bash -set -e - -readonly PROGNAME=$(basename $0) -readonly PROGDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -readonly ROOT="$(realpath ${PROGDIR}/..)" - -readonly ARGS="$@" - -function usage { - cat <<- EOF - usage: $PROGNAME SERVICE - - Opens an ash shell in the given SERVICE's container. - - OPTIONS: - -h --help Show this help. - -x --debug Debug this script. - - Examples: - $PROGNAME database -EOF -} - -function cmdline { - local arg= - for arg - do - local delim="" - case "$arg" in - # Translate --gnu-long-options to -g (short options) - --help) args="${args}-h ";; - --debug) args="${args}-x ";; - # Pass through anything else - *) [[ "${arg:0:1}" == "-" ]] || delim="\"" - args="${args}${delim}${arg}${delim} ";; - esac - done - - # Reset the positional parameters to the short options - eval set -- $args - - while getopts "hx" OPTION - do - case $OPTION in - h) - usage - exit 0 - ;; - x) - readonly DEBUG='-x' - set -x - ;; - esac - done - - shift $((OPTIND-1)) - readonly SERVICE="${1}" - - # Check if the service exists and is running. - [[ -z ${SERVICE} ]] && (echo "No SERVICE specified."; usage; exit 1) - docker-compose ps ${SERVICE} &> /dev/null || ( echo "Service ${SERVICE} does not exist."; exit 1 ) - [[ "$(docker-compose ps -q ${SERVICE})" == "" ]] && (echo "Service ${SERVICE} is not running."; exit 1) - - return 0 -} - -function main { - cmdline ${ARGS} - docker-compose -f "${ROOT}/docker-compose.yml" exec ${SERVICE} ash -} -main - diff --git a/composer/Dockerfile b/composer/Dockerfile deleted file mode 100644 index f1c8f72b..00000000 --- a/composer/Dockerfile +++ /dev/null @@ -1,30 +0,0 @@ -# syntax=docker/dockerfile:experimental -FROM local/nginx:latest as composer - -# Overwrite with your own arguments or explicitly copy in -# your own composer.json / composer.lock files instead. -# https://getcomposer.org/doc/03-cli.md#create-project -ARG composer_project="drupal/recommended-project" - -WORKDIR /build - -RUN --mount=type=cache,target=/root/.composer/cache \ - composer create-project --no-install ${composer_project} /build && \ - composer require -- drush/drush && \ - composer install - -FROM local/drupal:latest - -RUN --mount=type=bind,from=composer,source=/build,target=/build \ - --mount=id=downloads,type=cache,target=/opt/downloads \ - DOWNLOAD_CACHE_DIRECTORY="/opt/downloads" && \ - DRUSH_VERSION="0.6.0" && \ - DRUSH_FILE="drush.phar" && \ - DRUSH_URL="https://github.com/drush-ops/drush-launcher/releases/download/${DRUSH_VERSION}/${DRUSH_FILE}" && \ - DRUSH_SHA256="c3f32a800a2f18470b0010cd71c49e49ef5c087f8131eecfe9b686dc1f3f3d4e" && \ - download.sh --url "${DRUSH_URL}" --sha256 "${DRUSH_SHA256}" "${DOWNLOAD_CACHE_DIRECTORY}" && \ - cp "${DOWNLOAD_CACHE_DIRECTORY}/${DRUSH_FILE}" /usr/local/bin/drush && \ - chmod a+x /usr/local/bin/drush && \ - cp -r /build/* /var/www/drupal && \ - chown -R nginx:nginx /var/www/drupal && \ - cleanup.sh diff --git a/composer/README.md b/composer/README.md deleted file mode 100644 index 148b37ad..00000000 --- a/composer/README.md +++ /dev/null @@ -1,6 +0,0 @@ -# Composer - -Serves as a template for those that wish to build a production installation -using a composer.json/composer.lock file from another repository. - -Copying the resulting composer install site into the drupal image. diff --git a/crayfish/rootfs/etc/confd/conf.d/public.key.toml b/crayfish/rootfs/etc/confd/conf.d/public.key.toml index 7cad455e..e7765b34 100644 --- a/crayfish/rootfs/etc/confd/conf.d/public.key.toml +++ b/crayfish/rootfs/etc/confd/conf.d/public.key.toml @@ -4,4 +4,4 @@ dest = "/opt/keys/jwt/public.key" uid = 100 gid = 101 mode = "0600" -keys = [ "/jwt" ] \ No newline at end of file +keys = [ "/jwt" ] diff --git a/composer/.dockerignore b/demo/.dockerignore similarity index 100% rename from composer/.dockerignore rename to demo/.dockerignore diff --git a/sandbox/Dockerfile b/demo/Dockerfile similarity index 78% rename from sandbox/Dockerfile rename to demo/Dockerfile index 352ff94c..0fc62ab9 100644 --- a/sandbox/Dockerfile +++ b/demo/Dockerfile @@ -1,18 +1,11 @@ # syntax=docker/dockerfile:experimental -FROM local/drupal:latest +FROM local/nginx:latest as composer # Islandora based Drupal install. RUN --mount=type=cache,target=/root/.composer/cache \ --mount=id=downloads,type=cache,target=/opt/downloads \ DOWNLOAD_CACHE_DIRECTORY="/opt/downloads" && \ - DRUSH_VERSION="0.6.0" && \ - DRUSH_FILE="drush.phar" && \ - DRUSH_URL="https://github.com/drush-ops/drush-launcher/releases/download/${DRUSH_VERSION}/${DRUSH_FILE}" && \ - DRUSH_SHA256="c3f32a800a2f18470b0010cd71c49e49ef5c087f8131eecfe9b686dc1f3f3d4e" && \ - download.sh --url "${DRUSH_URL}" --sha256 "${DRUSH_SHA256}" "${DOWNLOAD_CACHE_DIRECTORY}" && \ - cp "${DOWNLOAD_CACHE_DIRECTORY}/${DRUSH_FILE}" /usr/local/bin/drush && \ - chmod a+x /usr/local/bin/drush && \ - php -d memory_limit=-1 /usr/bin/composer create-project islandora/drupal-project:8.8.1 \ + composer create-project islandora/drupal-project:8.8.1 \ --prefer-dist \ --no-interaction \ --stability stable \ @@ -20,28 +13,27 @@ RUN --mount=type=cache,target=/root/.composer/cache \ --no-install \ /var/www/drupal \ && \ - php -d memory_limit=-1 /usr/bin/composer require --update-no-dev -- drush/drush:^9.7.1 && \ - mkdir /var/www/drupal/config && \ - mkdir -p /var/www/drupal/web/libraries && \ - chown -R nginx:nginx /var/www && \ - php -d memory_limit=-1 /usr/bin/composer require --update-no-dev -- \ - zaporylie/composer-drupal-optimizations:^1.0 \ + cd /var/www/drupal && \ + composer require --update-no-dev -- \ + drupal/admin_toolbar:^2.0 \ drupal/console:~1.0 \ + drupal/content_browser:^1.0@alpha \ drupal/devel:^2.0 \ - drush/drush:^9.0 \ - drupal/rdfui:^1.0-beta1 \ - drupal/restui:^1.16 \ - drupal/search_api_solr:^3.8 \ drupal/facets:^1.3 \ - drupal/content_browser:^1.0@alpha \ drupal/matomo:^1.7 \ drupal/pdf:1.x-dev \ - drupal/admin_toolbar:^2.0 \ + drupal/rdfui:^1.0-beta1 \ drupal/rest_oai_pmh:^1.0 \ + drupal/restui:^1.16 \ + drupal/search_api_solr:^3.8 \ drupal/transliterate_filenames:^1.3 \ + drush/drush:^9.7.1 \ + islandora-rdm/islandora_fits:dev-master \ islandora/carapace:dev-8.x-3.x \ islandora/islandora_defaults:dev-8.x-1.x \ - islandora-rdm/islandora_fits:dev-master && \ + zaporylie/composer-drupal-optimizations:^1.0 \ + && \ + mkdir -p /var/www/drupal/web/libraries && \ MASONRY_VERSION="3.3.2" && \ MASONRY_FILE="v${MASONRY_VERSION}.zip" && \ MASONRY_URL="https://github.com/desandro/masonry/archive/${MASONRY_FILE}" && \ @@ -69,4 +61,8 @@ RUN --mount=type=cache,target=/root/.composer/cache \ chown -R nginx:nginx /var/www/drupal && \ cleanup.sh +FROM local/drupal:latest + +COPY --chown=nginx:nginx --from=composer /var/www/drupal /var/www/drupal + COPY rootfs / diff --git a/sandbox/README.md b/demo/README.md similarity index 77% rename from sandbox/README.md rename to demo/README.md index ae9e1142..5fa8541b 100644 --- a/sandbox/README.md +++ b/demo/README.md @@ -1,6 +1,6 @@ -# Sandbox +# Demo -Islandora Sandbox image meant for demoing a functioning system. If you want to +Islandora Demo image meant for demoing a functioning system. If you want to build your own Islandora site site the composer folder for a template on how to do that. diff --git a/demo/rootfs/etc/cont-init.d/04-demo-setup.sh b/demo/rootfs/etc/cont-init.d/04-demo-setup.sh new file mode 100644 index 00000000..2e7666d4 --- /dev/null +++ b/demo/rootfs/etc/cont-init.d/04-demo-setup.sh @@ -0,0 +1,89 @@ +#!/usr/bin/with-contenv bash +set -e + +# Does not run in the context of a login shell so we must explicitly include utilities.sh +source /etc/islandora/utilities.sh + +function main { + local modules=( + admin_toolbar + basic_auth + content_browser + controlled_access_terms_defaults + devel + facets + islandora_breadcrumbs + islandora_defaults + islandora_fits + islandora_iiif + islandora_oaipmh + islandora_search + matomo + pdf + rdf + responsive_image + rest + restui + search_api_solr + search_api_solr_defaults + serialization + simpletest + syslog + transliterate_filenames + ) + local features=( + islandora_core_feature + controlled_access_terms_defaults + islandora_defaults + islandora_search + ) + + # We do all sites in a stepwise fashion as it turns out to be faster than + # configuring one site completely at a time as they get blocked waiting on + # other services like Fedora to start. + for_all_sites create_database + for_all_sites install_site + for_all_sites update_settings_php + + # Uses the same public/private key for all sub-sites. + # Required if they share the same backend microservices, etc. + for_all_sites configure_jwt_module + + # Enable islandora before all other modules so services that require direct + # communication like ActiveMQ etc have the appropriate settings. + for_all_sites configure_islandora_module + + # The following commands require several services + # to be up and running before they can complete. + for_all_sites wait_for_required_services + + # Create namespace assumed one per site. + for_all_sites create_blazegraph_namespace_with_default_properties + + # Theme must be enabled before importing features. + for_all_sites set_carapace_default_theme + + # All subsites are assumed to use the same modules and features. + for_all_sites enable_modules ${modules[@]} + for_all_sites import_features ${features[@]} + + # Features overrides some settings for services, so we need to explicitly + # set them again. + for_all_sites configure_islandora_module + for_all_sites configure_matomo_module + for_all_sites configure_search_api_solr_module + for_all_sites configure_openseadragon + for_all_sites configure_islandora_default_module + + # Export configuration now that features have been imported. + for_all_sites create_solr_core_with_default_config + + # Run migrations. + for_all_sites import_islandora_migrations + + s6-setuidgid nginx mkdir -p /var/www/drupal/web/simpletest /var/www/drupal/web/sites/simpletest + s6-setuidgid nginx mkdir -p config/sync + + for_all_sites cache_rebuild +} +main diff --git a/docker-compose.yml b/docker-compose.yml deleted file mode 100644 index 8d8b6caa..00000000 --- a/docker-compose.yml +++ /dev/null @@ -1,217 +0,0 @@ -# file: docker-compose.yml -# DO NOT USE IN PRODUCTION -# -# Development docker-compose for testing locally: -# - All users are 'admin' -# - All passwords are 'password' -# -# With the exception of the database where the user is -# 'root' rather than admin. And the database users for -# each service are named after their respective service -# to avoid conflict. -# -# The Traefik image acts as Reverse Proxy to the services, -# assuming you can map to port 80 on your machine. The -# loop back for localhost should make the following urls -# accessible in your browser: -# -# - islandora.localhost -# - activemq.localhost/admin -# - blazegraph.localhost/bigdata -# - fcrepo.localhost/fcrepo/reset -# - matomo.localhost -# -# For other images you can use the command: -# -# ./commands/open-in-browser SERVICE -version: "3.7" -networks: - external: - internal: false - default: - internal: true -volumes: - activemq-data: - blazegraph-data: - drupal-config-data: - drupal-sites-data: - cantaloupe-data: - etcd-data: - fcrepo-data: - matomo-config-data: - mysql-data: - mysql-files: - solr-data: -services: - # Single node cluster. - # - # If disabled the system will fall back on Environment Variables. - # To set key/value pairs use the following: - # - # ./commands/etcdctl.sh put /drupal/site/default/name "default" - # - etcd: - image: gcr.io/etcd-development/etcd:v3.4.7 - environment: - ETCD_ADVERTISE_CLIENT_URLS: "http://0.0.0.0:2379" - ETCD_LISTEN_CLIENT_URLS: "http://0.0.0.0:2379" - volumes: - - etcd-data:/data - ports: - - 2379 - - 2380 - - 4001 - command: > - etcd - --data-dir=/data - watchtower: - image: v2tec/watchtower - volumes: - - /var/run/docker.sock:/var/run/docker.sock - command: --interval 1 --no-pull - activemq: - image: ${REPOSITORY:-local}/activemq:latest - volumes: - - activemq-data:/opt/activemq/data - labels: - # Do not expose in production. - - traefik.http.services.activemq.loadbalancer.server.port=8161 - - traefik.http.routers.activemq_http.service=activemq - - traefik.http.routers.activemq_http.entrypoints=http - alpaca: - image: ${REPOSITORY:-local}/alpaca:latest - blazegraph: - image: ${REPOSITORY:-local}/blazegraph:latest - volumes: - - blazegraph-data:/data - networks: - default: - aliases: - - blazegraph.localhost - labels: - - traefik.http.services.blazegraph.loadbalancer.server.port=80 - - traefik.http.routers.blazegraph_http.service=blazegraph - - traefik.http.routers.blazegraph_http.entrypoints=http - cantaloupe: - image: ${REPOSITORY:-local}/cantaloupe:latest - volumes: - - cantaloupe-data:/data - labels: - - traefik.http.services.cantaloupe.loadbalancer.server.port=80 - - traefik.http.routers.cantaloupe_http.service=cantaloupe - - traefik.http.routers.cantaloupe_http.entrypoints=http - - traefik.http.routers.cantaloupe_http.rule=Host(`islandora.localhost`) && PathPrefix(`/cantaloupe`) - crayfits: - image: ${REPOSITORY:-local}/crayfits:latest - depends_on: - - fits - # Database is chosen as the service name rather than mariadb, - # as institutions may want to swap out back-ends later and - # database is a more sensible default. - database: - image: ${REPOSITORY:-local}/mariadb:latest - volumes: - - mysql-data:/var/lib/mysql - - mysql-files:/var/lib/mysql-files - drupal: - image: ${REPOSITORY:-local}/sandbox:latest - restart: unless-stopped - volumes: - - drupal-config-data:/var/www/drupal/config - - drupal-sites-data:/var/www/drupal/web/sites - - solr-data:/opt/solr/server/solr - depends_on: - - solr - - fcrepo - - database - - activemq - labels: - - traefik.http.services.drupal.loadbalancer.server.port=80 - - traefik.http.routers.drupal_http.service=drupal - - traefik.http.routers.drupal_http.entrypoints=http - - traefik.http.routers.drupal_http.rule=Host(`islandora.localhost`) - fcrepo: - image: ${REPOSITORY:-local}/fcrepo:latest - volumes: - - fcrepo-data:/data - depends_on: - - activemq - - database - networks: - default: - aliases: - - fcrepo.localhost - labels: - # Do not expose in production. - - traefik.http.services.fcrepo.loadbalancer.server.port=80 - - traefik.http.routers.fcrepo_http.service=fcrepo - - traefik.http.routers.fcrepo_http.entrypoints=http - fits: - image: ${REPOSITORY:-local}/fits - gemini: - image: ${REPOSITORY:-local}/gemini:latest - depends_on: - - database - homarus: - image: ${REPOSITORY:-local}/homarus:latest - houdini: - image: ${REPOSITORY:-local}/houdini:latest - hypercube: - image: ${REPOSITORY:-local}/hypercube:latest - matomo: - image: ${REPOSITORY:-local}/matomo - volumes: - - matomo-config-data:/var/www/matomo - depends_on: - - database - networks: - default: - external: # Needs external access to request plugins, etc. - labels: - # Do not expose in production over http, setup https. - - traefik.http.services.matomo.loadbalancer.server.port=80 - - traefik.http.routers.matomo_http.service=matomo - - traefik.http.routers.matomo_http.entrypoints=http - - traefik.http.routers.matomo_http.rule=Host(`islandora.localhost`) && PathPrefix(`/matomo`) - - traefik.http.middlewares.matomo-stripprefix.stripprefix.prefixes=/matomo - - traefik.http.middlewares.matomo-customrequestheaders.headers.customrequestheaders.X-Forwarded-Uri=/matomo - - traefik.http.middlewares.matomo.chain.middlewares=matomo-stripprefix,matomo-customrequestheaders - - traefik.http.routers.matomo_http.middlewares=matomo - milliner: - image: ${REPOSITORY:-local}/milliner:latest - recast: - image: ${REPOSITORY:-local}/recast:latest - solr: - image: ${REPOSITORY:-local}/solr:latest - volumes: - - solr-data:/opt/solr/server/solr - labels: - # Do not expose in production. - - traefik.http.services.solr.loadbalancer.server.port=8983 - - traefik.http.routers.solr_http.service=solr - - traefik.http.routers.solr_http.entrypoints=http - traefik: - image: traefik:2.2.1 - command: > - --api.insecure=true - --api.dashboard=true - --api.debug=true - --entryPoints.http.address=:80 - --entryPoints.https.address=:443 - --providers.docker - --providers.docker.network=external - '--providers.docker.defaultRule=Host(`{{ index .Labels "com.docker.compose.service" }}.localhost`)' - ports: - - 80:80 - - 443:443 - - 8080 - volumes: - - /var/run/docker.sock:/var/run/docker.sock - labels: - # Do not expose in production. - - traefik.http.routers.api.service=api@internal - networks: - external: - default: - aliases: - - islandora.localhost diff --git a/drupal/Dockerfile b/drupal/Dockerfile index e31cee18..02f6d7ae 100644 --- a/drupal/Dockerfile +++ b/drupal/Dockerfile @@ -1,8 +1,18 @@ # syntax=docker/dockerfile:experimental FROM local/nginx:latest -RUN mkdir -p /var/www/drupal && \ - chown nginx:nginx /var/www/drupal && \ +RUN --mount=id=downloads,type=cache,target=/opt/downloads \ + DOWNLOAD_CACHE_DIRECTORY="/opt/downloads" && \ + DRUSH_VERSION="0.6.0" && \ + DRUSH_FILE="drush.phar" && \ + DRUSH_URL="https://github.com/drush-ops/drush-launcher/releases/download/${DRUSH_VERSION}/${DRUSH_FILE}" && \ + DRUSH_SHA256="c3f32a800a2f18470b0010cd71c49e49ef5c087f8131eecfe9b686dc1f3f3d4e" && \ + download.sh --url "${DRUSH_URL}" --sha256 "${DRUSH_SHA256}" "${DOWNLOAD_CACHE_DIRECTORY}" && \ + cp "${DOWNLOAD_CACHE_DIRECTORY}/${DRUSH_FILE}" /usr/bin/drush && \ + chmod a+x /usr/bin/drush && \ + mkdir -p /var/www/drupal/config && \ + mkdir -p /var/www/drupal/web/libraries && \ + chown -R nginx:nginx /var/www && \ mkdir -p /opt/keys/jwt && \ chown nginx:nginx /opt/keys/jwt diff --git a/drupal/rootfs/etc/bash.bashrc b/drupal/rootfs/etc/bash.bashrc new file mode 100644 index 00000000..cbe06a15 --- /dev/null +++ b/drupal/rootfs/etc/bash.bashrc @@ -0,0 +1,4 @@ +#!/bin/env bash + +# For bash login-shells include utilities so they can be called directly. +source /etc/islandora/utilities.sh diff --git a/drupal/rootfs/etc/confd/templates/drupal-setup-environment.sh.tmpl b/drupal/rootfs/etc/confd/templates/drupal-setup-environment.sh.tmpl index f370f496..3e486d80 100644 --- a/drupal/rootfs/etc/confd/templates/drupal-setup-environment.sh.tmpl +++ b/drupal/rootfs/etc/confd/templates/drupal-setup-environment.sh.tmpl @@ -2,6 +2,46 @@ # -*- mode: sh -*- # vi: set ft=sh: with-contenv +multisubstitute +{ + # Default settings to apply if none given. + define ACCOUNT_EMAIL "webmaster@localhost.com" + define ACCOUNT_NAME "admin" + define ACCOUNT_PASSWORD "password" + define CONFIGDIR "" + define DB_DRIVER "mysql" + define DB_HOST "database" + define DB_NAME "drupal_default" + define DB_PASSWORD "password" + define DB_PORT "3306" + define DB_ROOT_PASSWORD "password" + define DB_ROOT_USER "root" + define DB_USER "drupal_default" + define EMAIL "webmaster@localhost.com" + define INSTALL "true" + define INSTALL_EXISTING_CONFIG "false" + define LOCALE "en" + define NAME "Default" + define PROFILE "standard" + define SALT "9PPaL0CxZAIcq0l9wxgDGlCZrp7JdT_x7v9gVzpdbUjMt1PqDz3uD0Zy-i16DuJ1-Htuq5hqeg" + define SUBDIR "default" + # Islandora Specific Variables. + define BROKER_HOST "activemq" + define BROKER_PORT "61613" + define CANTALOUPE_URL "http://islandora.isle-dc.localhost/cantaloupe/iiif/2" + define FCREPO_HOST "fcrepo.isle-dc.localhost" + define FCREPO_PORT "80" + define GEMINI_HOST "gemini" + define GEMINI_PORT "8000" + define MATOMO_URL "http://islandora.isle-dc.localhost/matomo/" + define SITE_URL "http://islandora.isle-dc.localhost" + define SOLR_CORE "ISLANDORA" + define SOLR_HOST "solr" + define SOLR_PORT "8983" + define TRIPLESTORE_HOST "blazegraph" + define TRIPLESTORE_NAMESPACE "islandora" + define TRIPLESTORE_PORT "80" +} foreground { # We add / update the environment defined for the container, # this allows our other initialization and service scripts @@ -9,39 +49,126 @@ foreground { # environment as seen by linked containers. # Variables can only be seen when using '#!/usr/bin/with-contenv' s6-env -i - DRUPAL_DB_DRIVER="{{ getv "/db/driver" "mysql" }}" - DRUPAL_DB_HOST="{{ getv "/db/host" "database" }}" - DRUPAL_DB_PORT="{{ getv "/db/port" "3306" }}" - DRUPAL_DB_ROOT_PASSWORD="{{ getv "/db/root/password" "password" }}" - DRUPAL_DB_ROOT_USER="{{ getv "/db/root/user" "root" }}" - DRUPAL_DEFAULT_ACCOUNT_EMAIL="{{ getv "/default/account/email" "webmaster@localhost.com" }}" - DRUPAL_DEFAULT_ACCOUNT_NAME="{{ getv "/default/account/name" "admin" }}" - DRUPAL_DEFAULT_ACCOUNT_PASSWORD="{{ getv "/default/account/password" "password" }}" - DRUPAL_DEFAULT_DB_NAME="{{ getv "/default/db/name" "drupal_default" }}" - DRUPAL_DEFAULT_DB_PASSWORD="{{ getv "/default/db/password" "password" }}" - DRUPAL_DEFAULT_DB_USER="{{ getv "/default/db/user" "drupal_default" }}" - DRUPAL_DEFAULT_EMAIL="{{ getv "/default/email" "webmaster@localhost.com" }}" - DRUPAL_DEFAULT_LOCALE="{{ getv "/default/locale" "en" }}" - DRUPAL_DEFAULT_NAME="{{ getv "/default/name" "Default" }}" - DRUPAL_DEFAULT_PROFILE="{{ getv "/default/profile" "standard" }}" - DRUPAL_DEFAULT_SUBDIR="{{ getv "/default/subdir" "default" }}" - DRUPAL_DEFAULT_CONFIGDIR="{{ getv "/default/configdir" "" }}" - DRUPAL_DEFAULT_INSTALL="{{ getv "/default/install" "true" }}" - DRUPAL_SITES="{{ toUpper (join (lsdir "/site") " ") }}" - {{ range $site := lsdir "/site" }} - DRUPAL_SITE_{{ toUpper $site }}_ACCOUNT_EMAIL="{{ getv (printf "/site/%s/account/email" $site) "webmaster@localhost.com" }}" - DRUPAL_SITE_{{ toUpper $site }}_ACCOUNT_NAME="{{ getv (printf "/site/%s/account/name" $site) "admin" }}" - DRUPAL_SITE_{{ toUpper $site }}_ACCOUNT_PASSWORD="{{ getv (printf "/site/%s/account/password" $site) "password" }}" + DRUPAL_DEFAULT_ACCOUNT_EMAIL="{{ getv "/default/account/email" "${ACCOUNT_EMAIL}" }}" + DRUPAL_DEFAULT_ACCOUNT_NAME="{{ getv "/default/account/name" "${ACCOUNT_NAME}" }}" + DRUPAL_DEFAULT_ACCOUNT_PASSWORD="{{ getv "/default/account/password" "${ACCOUNT_PASSWORD}" }}" + DRUPAL_DEFAULT_CONFIGDIR="{{ getv "/default/configdir" "${CONFIGDIR}" }}" + DRUPAL_DEFAULT_DB_DRIVER="{{ getv "/default/db/driver" "${DB_DRIVER}" }}" + DRUPAL_DEFAULT_DB_HOST="{{ getv "/default/db/host" "${DB_HOST}" }}" + DRUPAL_DEFAULT_DB_NAME="{{ getv "/default/db/name" "${DB_NAME}" }}" + DRUPAL_DEFAULT_DB_PASSWORD="{{ getv "/default/db/password" "${DB_PASSWORD}" }}" + DRUPAL_DEFAULT_DB_PORT="{{ getv "/default/db/port" "${DB_PORT}" }}" + DRUPAL_DEFAULT_DB_ROOT_PASSWORD="{{ getv "/default/db/root/password" "${DB_ROOT_PASSWORD}" }}" + DRUPAL_DEFAULT_DB_ROOT_USER="{{ getv "/default/db/root/user" "${DB_ROOT_USER}" }}" + DRUPAL_DEFAULT_DB_USER="{{ getv "/default/db/user" "${DB_USER}" }}" + DRUPAL_DEFAULT_EMAIL="{{ getv "/default/email" "${EMAIL}" }}" + DRUPAL_DEFAULT_INSTALL="{{ getv "/default/install" "${INSTALL}" }}" + DRUPAL_DEFAULT_INSTALL_EXISTING_CONFIG="{{ getv "/default/install/existing/config" "${INSTALL_EXISTING_CONFIG}" }}" + DRUPAL_DEFAULT_LOCALE="{{ getv "/default/locale" "${LOCALE}" }}" + DRUPAL_DEFAULT_NAME="{{ getv "/default/name" "${NAME}" }}" + DRUPAL_DEFAULT_PROFILE="{{ getv "/default/profile" "${PROFILE}" }}" + DRUPAL_DEFAULT_SALT="{{ getv "/default/salt" "${SALT}" }}" + DRUPAL_DEFAULT_SUBDIR="{{ getv "/default/subdir" "${SUBDIR}" }}" + # Islandora Specific Variables. + DRUPAL_DEFAULT_BROKER_HOST="{{ getv "/default/broker/host" "${BROKER_HOST}" }}" + DRUPAL_DEFAULT_BROKER_PORT="{{ getv "/default/broker/port" "${BROKER_PORT}" }}" + DRUPAL_DEFAULT_CANTALOUPE_URL="{{ getv "/default/cantaloupe/url" "${CANTALOUPE_URL}" }}" + DRUPAL_DEFAULT_FCREPO_HOST="{{ getv "/default/fcrepo/host" "${FCREPO_HOST}" }}" + DRUPAL_DEFAULT_FCREPO_PORT="{{ getv "/default/fcrepo/port" "${FCREPO_PORT}" }}" + DRUPAL_DEFAULT_GEMINI_HOST="{{ getv "/default/gemini/host" "${GEMINI_HOST}" }}" + DRUPAL_DEFAULT_GEMINI_PORT="{{ getv "/default/gemini/port" "${GEMINI_PORT}" }}" + DRUPAL_DEFAULT_MATOMO_URL="{{ getv "/default/matomo/url" "${MATOMO_URL}" }}" + DRUPAL_DEFAULT_SITE_URL="{{ getv "/default/site/url" "${SITE_URL}" }}" + DRUPAL_DEFAULT_SOLR_CORE="{{ getv "/default/solr/core" "${SOLR_CORE}" }}" + DRUPAL_DEFAULT_SOLR_HOST="{{ getv "/default/solr/host" "${SOLR_HOST}" }}" + DRUPAL_DEFAULT_SOLR_PORT="{{ getv "/default/solr/port" "${SOLR_PORT}" }}" + DRUPAL_DEFAULT_TRIPLESTORE_HOST="{{ getv "/default/triplestore/host" "${TRIPLESTORE_HOST}" }}" + DRUPAL_DEFAULT_TRIPLESTORE_NAMESPACE="{{ getv "/default/triplestore/namespace" "${TRIPLESTORE_NAMESPACE}" }}" + DRUPAL_DEFAULT_TRIPLESTORE_PORT="{{ getv "/default/triplestore/port" "${TRIPLESTORE_PORT}" }}" + # Used by scripts to iterate over all/sub sites environment variables. + DRUPAL_SUBSITES="{{ toUpper (join (lsdir "/site") "") }}" + DRUPAL_SITES="DEFAULT {{ toUpper (join (lsdir "/site") "") }}" + s6-dumpenv -- /var/run/s6/container_environment +} +{{ range $site := lsdir "/site" }} +# Import default settings from the previous statement so the default value set +# by the user can propagate to subsites. +with-contenv +multisubstitute +{ + importas -i SUBSITE_ACCOUNT_EMAIL DRUPAL_DEFAULT_ACCOUNT_EMAIL + importas -i SUBSITE_ACCOUNT_NAME DRUPAL_DEFAULT_ACCOUNT_NAME + importas -i SUBSITE_ACCOUNT_PASSWORD DRUPAL_DEFAULT_ACCOUNT_PASSWORD + importas -i SUBSITE_DB_DRIVER DRUPAL_DEFAULT_DB_DRIVER + importas -i SUBSITE_DB_HOST DRUPAL_DEFAULT_DB_HOST + importas -i SUBSITE_DB_NAME DRUPAL_DEFAULT_DB_NAME + importas -i SUBSITE_DB_PASSWORD DRUPAL_DEFAULT_DB_PASSWORD + importas -i SUBSITE_DB_PORT DRUPAL_DEFAULT_DB_PORT + importas -i SUBSITE_DB_ROOT_PASSWORD DRUPAL_DEFAULT_DB_ROOT_PASSWORD + importas -i SUBSITE_DB_ROOT_USER DRUPAL_DEFAULT_DB_ROOT_USER + importas -i SUBSITE_DB_USER DRUPAL_DEFAULT_DB_USER + importas -i SUBSITE_EMAIL DRUPAL_DEFAULT_EMAIL + importas -i SUBSITE_INSTALL DRUPAL_DEFAULT_INSTALL + importas -i SUBSITE_INSTALL_EXISTING_CONFIG DRUPAL_DEFAULT_INSTALL_EXISTING_CONFIG + importas -i SUBSITE_LOCALE DRUPAL_DEFAULT_LOCALE + importas -i SUBSITE_NAME DRUPAL_DEFAULT_NAME + importas -i SUBSITE_PROFILE DRUPAL_DEFAULT_PROFILE + importas -i SUBSITE_SUBDIR DRUPAL_DEFAULT_SUBDIR + # Islandora Specific Variables. + importas -i SUBSITE_BROKER_HOST DRUPAL_DEFAULT_BROKER_HOST + importas -i SUBSITE_BROKER_PORT DRUPAL_DEFAULT_BROKER_PORT + importas -i SUBSITE_CANTALOUPE_URL DRUPAL_DEFAULT_CANTALOUPE_URL + importas -i SUBSITE_FCREPO_HOST DRUPAL_DEFAULT_FCREPO_HOST + importas -i SUBSITE_FCREPO_PORT DRUPAL_DEFAULT_FCREPO_PORT + importas -i SUBSITE_GEMINI_HOST DRUPAL_DEFAULT_GEMINI_HOST + importas -i SUBSITE_GEMINI_PORT DRUPAL_DEFAULT_GEMINI_PORT + importas -i SUBSITE_MATOMO_URL DRUPAL_DEFAULT_MATOMO_URL + importas -i SUBSITE_SITE_URL DRUPAL_DEFAULT_SITE_URL + importas -i SUBSITE_SOLR_HOST DRUPAL_DEFAULT_SOLR_HOST + importas -i SUBSITE_SOLR_PORT DRUPAL_DEFAULT_SOLR_PORT + importas -i SUBSITE_TRIPLESTORE_HOST DRUPAL_DEFAULT_TRIPLESTORE_HOST + importas -i SUBSITE_TRIPLESTORE_PORT DRUPAL_DEFAULT_TRIPLESTORE_PORT +} +foreground { + # In general subsites will use the same values specified for the default site + # except where it does not make sense like install directory or database name. + s6-env -i + DRUPAL_SITE_{{ toUpper $site }}_ACCOUNT_EMAIL="{{ getv (printf "/site/%s/account/email" $site) "${SUBSITE_ACCOUNT_EMAIL}" }}" + DRUPAL_SITE_{{ toUpper $site }}_ACCOUNT_NAME="{{ getv (printf "/site/%s/account/name" $site) "${SUBSITE_ACCOUNT_NAME}" }}" + DRUPAL_SITE_{{ toUpper $site }}_ACCOUNT_PASSWORD="{{ getv (printf "/site/%s/account/password" $site) }}" + DRUPAL_SITE_{{ toUpper $site }}_CONFIGDIR="{{ getv (printf "/site/%s/configdir" $site) "" }}" + DRUPAL_SITE_{{ toUpper $site }}_DB_DRIVER="{{ getv (printf "/site/%s/db/driver" $site) "${SUBSITE_DB_DRIVER}" }}" + DRUPAL_SITE_{{ toUpper $site }}_DB_HOST="{{ getv (printf "/site/%s/db/host" $site) "${SUBSITE_DB_HOST}" }}" DRUPAL_SITE_{{ toUpper $site }}_DB_NAME="{{ getv (printf "/site/%s/db/name" $site) (printf "drupal_%s" $site) }}" - DRUPAL_SITE_{{ toUpper $site }}_DB_PASSWORD="{{ getv (printf "/site/%s/db/password" $site) "password" }}" + DRUPAL_SITE_{{ toUpper $site }}_DB_PASSWORD="{{ getv (printf "/site/%s/db/password" $site) }}" + DRUPAL_SITE_{{ toUpper $site }}_DB_PORT="{{ getv (printf "/site/%s/db/port" $site) "${SUBSITE_DB_PORT}" }}" + DRUPAL_SITE_{{ toUpper $site }}_DB_ROOT_PASSWORD="{{ getv (printf "/site/%s/db/root/password" $site) "${SUBSITE_DB_ROOT_PASSWORD}" }}" + DRUPAL_SITE_{{ toUpper $site }}_DB_ROOT_USER="{{ getv (printf "/site/%s/db/root/password" $site) "${SUBSITE_DB_ROOT_USER}" }}" DRUPAL_SITE_{{ toUpper $site }}_DB_USER="{{ getv (printf "/site/%s/db/user" $site) (printf "drupal_%s" $site) }}" - DRUPAL_SITE_{{ toUpper $site }}_EMAIL="{{ getv (printf "/site/%s/email" $site) "webmaster@localhost.com" }}" - DRUPAL_SITE_{{ toUpper $site }}_LOCALE="{{ getv (printf "/site/%s/locale" $site) "en" }}" - DRUPAL_SITE_{{ toUpper $site }}_NAME="{{ getv (printf "/site/%s/name" $site) }}" - DRUPAL_SITE_{{ toUpper $site }}_PROFILE="{{ getv (printf "/site/%s/profile" $site) "standard" }}" + DRUPAL_SITE_{{ toUpper $site }}_EMAIL="{{ getv (printf "/site/%s/email" $site) "${SUBSITE_EMAIL}" }}" + DRUPAL_SITE_{{ toUpper $site }}_INSTALL="{{ getv (printf "/site/%s/install" $site) "${SUBSITE_INSTALL}" }}" + DRUPAL_SITE_{{ toUpper $site }}_INSTALL_EXISTING_CONFIG="{{ getv (printf "/site/%s/install/existing/config" $site) "${SUBSITE_INSTALL_EXISTING_CONFIG}" }}" + DRUPAL_SITE_{{ toUpper $site }}_LOCALE="{{ getv (printf "/site/%s/locale" $site) "${SUBSITE_LOCALE}" }}" + DRUPAL_SITE_{{ toUpper $site }}_NAME="{{ getv (printf "/site/%s/name" $site) $site }}" + DRUPAL_SITE_{{ toUpper $site }}_PROFILE="{{ getv (printf "/site/%s/profile" $site) "${SUBSITE_PROFILE}" }}" + DRUPAL_SITE_{{ toUpper $site }}_SALT="{{ getv (printf "/site/%s/salt" $site) }}" DRUPAL_SITE_{{ toUpper $site }}_SUBDIR="{{ getv (printf "/site/%s/subdir" $site) $site }}" - DRUPAL_SITE_{{ toUpper $site }}_CONFIGDIR="{{ getv (printf "/site/%s/configdir" $site) "" }}" - DRUPAL_SITE_{{ toUpper $site }}_INSTALL="{{ getv (printf "/site/%s/install" $site) "true" }}" - {{ end }} + # Islandora Specific Variables. + DRUPAL_SITE_{{ toUpper $site }}_BROKER_HOST="{{ getv (printf "/site/%s/broker/host" $site) "${SUBSITE_BROKER_HOST}" }}" + DRUPAL_SITE_{{ toUpper $site }}_BROKER_PORT="{{ getv (printf "/site/%s/broker/port" $site) "${SUBSITE_BROKER_PORT}" }}" + DRUPAL_SITE_{{ toUpper $site }}_CANTALOUPE_URL="{{ getv (printf "/site/%s/cantaloupe/url" $site) "${SUBSITE_CANTALOUPE_URL}" }}" + DRUPAL_SITE_{{ toUpper $site }}_FCREPO_HOST="{{ getv (printf "/site/%s/fcrepo/host" $site) "${SUBSITE_FCREPO_HOST}" }}" + DRUPAL_SITE_{{ toUpper $site }}_FCREPO_HOST="{{ getv (printf "/site/%s/fcrepo/host" $site) "${SUBSITE_FCREPO_HOST}" }}" + DRUPAL_SITE_{{ toUpper $site }}_GEMINI_HOST="{{ getv (printf "/site/%s/gemini/host" $site) "${SUBSITE_GEMINI_HOST}" }}" + DRUPAL_SITE_{{ toUpper $site }}_GEMINI_PORT="{{ getv (printf "/site/%s/gemini/port" $site) "${SUBSITE_GEMINI_PORT}" }}" + DRUPAL_SITE_{{ toUpper $site }}_MATOMO_URL="{{ getv (printf "/site/%s/matomo/url" $site) "${SUBSITE_MATOMO_URL}" }}" + DRUPAL_SITE_{{ toUpper $site }}_SITE_URL="{{ getv (printf "/site/%s/site/url" $site) "${SUBSITE_SITE_URL}" }}" + DRUPAL_SITE_{{ toUpper $site }}_SOLR_CORE="{{ getv (printf "/site/%s/solr/core" $site) $site }}" + DRUPAL_SITE_{{ toUpper $site }}_SOLR_HOST="{{ getv (printf "/site/%s/solr/host" $site) "${SUBSITE_SOLR_HOST}" }}" + DRUPAL_SITE_{{ toUpper $site }}_SOLR_PORT="{{ getv (printf "/site/%s/solr/port" $site) "${SUBSITE_SOLR_PORT}" }}" + DRUPAL_SITE_{{ toUpper $site }}_TRIPLESTORE_HOST="{{ getv (printf "/site/%s/triplestore/host" $site) "${SUBSITE_TRIPLESTORE_HOST}" }}" + DRUPAL_SITE_{{ toUpper $site }}_TRIPLESTORE_NAMESPACE="{{ getv (printf "/site/%s/triplestore/namespace" $site) $site }}" + DRUPAL_SITE_{{ toUpper $site }}_TRIPLESTORE_PORT="{{ getv (printf "/site/%s/triplestore/port" $site) "${SUBSITE_TRIPLESTORE_PORT}" }}" s6-dumpenv -- /var/run/s6/container_environment } +{{ end }} diff --git a/drupal/rootfs/etc/cont-init.d/04-drupal-setup.sh b/drupal/rootfs/etc/cont-init.d/04-drupal-setup.sh deleted file mode 100644 index 76a2123c..00000000 --- a/drupal/rootfs/etc/cont-init.d/04-drupal-setup.sh +++ /dev/null @@ -1,119 +0,0 @@ -#!/usr/bin/with-contenv bash -set -e - -function create_database { - local db_name="${1}"; shift - local db_user="${1}"; shift - local db_password="${1}"; shift - /usr/local/bin/create-drupal-database.sh \ - --driver "${DRUPAL_DB_DRIVER}" \ - --host "${DRUPAL_DB_HOST}" \ - --port "${DRUPAL_DB_PORT}" \ - --user "${DRUPAL_DB_ROOT_USER}" \ - --password "${DRUPAL_DB_ROOT_PASSWORD}" \ - --db-name "${db_name}" \ - --db-user "${db_user}" \ - --db-password "${db_password}" -} - -function create_default_site_database { - create_database \ - "${DRUPAL_DEFAULT_DB_NAME}" \ - "${DRUPAL_DEFAULT_DB_USER}" \ - "${DRUPAL_DEFAULT_DB_PASSWORD}" -} - -function create_site_database { - local site="${1}" - local db_name_var="DRUPAL_SITE_${site}_DB_NAME" - local db_user_var="DRUPAL_SITE_${site}_DB_USER" - local db_password_var="DRUPAL_SITE_${site}_DB_PASSWORD" - create_database \ - "${!db_name_var}" \ - "${!db_user_var}" \ - "${!db_password_var}" -} - -function create_subsite_databases { - for site in ${DRUPAL_SITES}; do - create_site_database "${site}" - done -} - -function install_site { - local db_name="${1}"; shift - local config_dir_arg="${1}"; shift - # Config directory is optional only works with non-standard profiles. - # https://www.drupal.org/project/drupal/issues/2982052 - if [[ ! -z "${config_dir_arg}" ]]; then - config_dir_arg="--config-dir=${config_dir_arg}" - fi - /usr/local/bin/install-drupal-site.sh \ - --driver "${DRUPAL_DB_DRIVER}" \ - --host "${DRUPAL_DB_HOST}" \ - --port "${DRUPAL_DB_PORT}" \ - --user "${DRUPAL_DB_ROOT_USER}" \ - --password "${DRUPAL_DB_ROOT_PASSWORD}" \ - --db-name "${db_name}" \ - ${@} ${config_dir_arg} -} - -function install_default_site { - if [ "${DRUPAL_DEFAULT_INSTALL}" = "true" ]; then - install_site \ - "${DRUPAL_DEFAULT_DB_NAME}" \ - "${DRUPAL_DEFAULT_CONFIGDIR}" \ - "${DRUPAL_DEFAULT_PROFILE}" \ - --sites-subdir="${DRUPAL_DEFAULT_SUBDIR}" \ - --site-name="${DRUPAL_DEFAULT_NAME}" \ - --site-mail="${DRUPAL_DEFAULT_EMAIL}" \ - --locale="${DRUPAL_DEFAULT_LOCALE}" \ - --account-name="${DRUPAL_DEFAULT_ACCOUNT_NAME}" \ - --account-pass="${DRUPAL_DEFAULT_ACCOUNT_PASSWORD}" \ - --account-mail="${DRUPAL_DEFAULT_ACCOUNT_EMAIL}" - fi -} - -function install_subsite { - local site="${1}" - local db_name_var="DRUPAL_SITE_${site}_DB_NAME" - local config_dir_var="DRUPAL_SITE_${site}_CONFIGDIR" - local profile_var="DRUPAL_SITE_${site}_PROFILE" - local subdir_var="DRUPAL_SITE_${site}_SUBDIR" - local site_name_var="DRUPAL_SITE_${site}_NAME" - local site_email_var="DRUPAL_SITE_${site}_EMAIL" - local site_locale_var="DRUPAL_SITE_${site}_LOCALE" - local account_name_var="DRUPAL_SITE_${site}_ACCOUNT_NAME" - local account_password_var="DRUPAL_SITE_${site}_ACCOUNT_PASSWORD" - local account_email_var="DRUPAL_SITE_${site}_ACCOUNT_EMAIL" - - install_site \ - "${!db_name_var}" \ - "${!config_dir_var}" \ - "${!profile_var}" \ - --sites-subdir="${!subdir_var}" \ - --site-name="${!site_name_var}" \ - --site-mail="${!site_email_var}" \ - --locale="${!site_locale_var}" \ - --account-name="${!account_name_var}" \ - --account-pass="${!account_password_var}" \ - --account-mail="${!account_email_var}" -} - -function install_subsites { - local install_var= - for site in ${DRUPAL_SITES}; do - install_var="DRUPAL_SITE_${site}_INSTALL" - if [ "${!install_var}" = "true" ]; then - install_subsite "${site}" - fi - done -} - -function main { - create_default_site_database - install_default_site - create_subsite_databases - install_subsites -} -main diff --git a/drupal/rootfs/etc/drush/drush.yml b/drupal/rootfs/etc/drush/drush.yml new file mode 100644 index 00000000..05bf5247 --- /dev/null +++ b/drupal/rootfs/etc/drush/drush.yml @@ -0,0 +1,4 @@ +drush: + paths: + include: + - '/usr/share/drush/' diff --git a/sandbox/rootfs/etc/confd/templates/inference.nt.tmpl b/drupal/rootfs/etc/islandora/configs/inference.nt similarity index 100% rename from sandbox/rootfs/etc/confd/templates/inference.nt.tmpl rename to drupal/rootfs/etc/islandora/configs/inference.nt diff --git a/sandbox/rootfs/opt/islandora/configs/jwt/jwt.config.yml b/drupal/rootfs/etc/islandora/configs/jwt/jwt.config.yml similarity index 100% rename from sandbox/rootfs/opt/islandora/configs/jwt/jwt.config.yml rename to drupal/rootfs/etc/islandora/configs/jwt/jwt.config.yml diff --git a/sandbox/rootfs/opt/islandora/configs/jwt/key.key.islandora_rsa_key.yml b/drupal/rootfs/etc/islandora/configs/jwt/key.key.islandora_rsa_key.yml similarity index 100% rename from sandbox/rootfs/opt/islandora/configs/jwt/key.key.islandora_rsa_key.yml rename to drupal/rootfs/etc/islandora/configs/jwt/key.key.islandora_rsa_key.yml diff --git a/drupal/rootfs/etc/islandora/utilities.sh b/drupal/rootfs/etc/islandora/utilities.sh new file mode 100644 index 00000000..af76cb2b --- /dev/null +++ b/drupal/rootfs/etc/islandora/utilities.sh @@ -0,0 +1,498 @@ +#!/usr/bin/with-contenv bash + +# Capitalize the given string. +function capitalize { + local string="${1}"; shift + echo $(tr '[:lower:]' '[:upper:]' <<< ${string:0:1})$(tr '[:upper:]' '[:lower:]' <<< ${string:1}) +} + +# Transform the given string to uppercase. +function uppercase { + local string="${1}"; shift + echo $(tr '[:lower:]' '[:upper:]' <<< ${string}) +} + +# Joins the given array into a string delimited by the first argument. +function join_by { + local IFS="${1}"; shift + echo "$*" +} + +# Get variable value for given site. +function drupal_site_env { + local site="$(uppercase ${1})"; shift + local suffix="$(uppercase ${1})"; shift + local var= + if [ "${site}" = "DEFAULT" ]; then + var="DRUPAL_DEFAULT_${suffix}" + echo "${!var}" + else + var="DRUPAL_SITE_${site}_${suffix}" + echo "${!var}" + fi +} + +# Get the index of the given site in the lists of site. +# Useful for generating distinct identifiers. +function site_index { + local site="${1}"; shift + local array=(${DRUPAL_SITES}) + for i in "${!array[@]}"; do + if [[ "${array[$i]}" = "${site}" ]]; then + echo "${i}"; + return 0 + fi + done + # Should be unreachable under normal use. + exit 1 +} + +# Wait for service to respond. +function wait_for_service { + local site="${1}"; shift + local service="${1}"; shift + local time="${1-300}"; + local host=$(drupal_site_env "${site}" "${service}_HOST") + local port=$(drupal_site_env "${site}" "${service}_PORT") + local service_name=$(capitalize "${service}") + + if timeout ${time} wait-for-open-port.sh "${host}" "${port}" ; then + echo "${service_name} Found at ${host}:${port}" + return 0 + else + echo "Could not connect to ${service_name} at ${host}:${port}" + exit 1 + fi +} + +# Waits for services that are required to be running to successfully ingest content. +function wait_for_required_services { + local site="${1}"; shift + wait_for_service "${site}" "SOLR" + wait_for_service "${site}" "FCREPO" + wait_for_service "${site}" "BROKER" + wait_for_service "${site}" "GEMINI" + wait_for_service "${site}" "TRIPLESTORE" +} + +# Apply given function for all sites in parallel, up to the number of cores available. +function for_all_sites { + local function="${1}"; shift + local n=$(nproc) + local pids=() + for site in ${DRUPAL_SITES}; do + $function "${site}" ${@} & + pids+=(${!}) + # Allow only to execute ${n} jobs in parallel + if [[ $(jobs -r -p | wc -l) -gt ${n} ]]; then + # Wait only for first job, exit code here will propigate + wait -n + fi + done + # To ensure the exit code propigates we must wait for each process individually + for pid in ${pids[@]}; do + wait "${pid}" + done +} + +# Create a database for the given site. +function create_database { + local site="${1}"; shift + local driver=$(drupal_site_env "${site}" "DB_DRIVER") + local host=$(drupal_site_env "${site}" "DB_HOST") + local port=$(drupal_site_env "${site}" "DB_PORT") + local user=$(drupal_site_env "${site}" "DB_ROOT_USER") + local password=$(drupal_site_env "${site}" "DB_ROOT_PASSWORD") + local db_name=$(drupal_site_env "${site}" "DB_NAME") + local db_user=$(drupal_site_env "${site}" "DB_USER") + local db_password=$(drupal_site_env "${site}" "DB_PASSWORD") + /usr/local/bin/create-drupal-database.sh \ + --driver "${driver}" \ + --host "${host}" \ + --port "${port}" \ + --user "${user}" \ + --password "${password}" \ + --db-name "${db_name}" \ + --db-user "${db_user}" \ + --db-password "${db_password}" +} + +# Install the given site. +function install_site { + local site="${1}"; shift + local drupal_root=$(drush drupal:directory) + local driver=$(drupal_site_env "${site}" "DB_DRIVER") + local host=$(drupal_site_env "${site}" "DB_HOST") + local port=$(drupal_site_env "${site}" "DB_PORT") + local user=$(drupal_site_env "${site}" "DB_USER") + local password=$(drupal_site_env "${site}" "DB_PASSWORD") + local db_name=$(drupal_site_env "${site}" "DB_NAME") + local account_email=$(drupal_site_env "${site}" "ACCOUNT_EMAIL") + local account_name=$(drupal_site_env "${site}" "ACCOUNT_NAME") + local account_password=$(drupal_site_env "${site}" "ACCOUNT_PASSWORD") + local profile=$(drupal_site_env "${site}" "PROFILE") + local site_email=$(drupal_site_env "${site}" "EMAIL") + local site_locale=$(drupal_site_env "${site}" "LOCALE") + local site_name=$(drupal_site_env "${site}" "NAME") + local subdir=$(drupal_site_env "${site}" "SUBDIR") + local site_directory=$(realpath "${drupal_root}/sites/${subdir}") + local files_directory=$(realpath "${site_directory}/files") + local install=$(drupal_site_env "${site}" "INSTALL") + local use_existing_config=$(drupal_site_env "${site}" "INSTALL_EXISTING_CONFIG") + local use_existing_config_arg= + + if [ "${install}" != "true" ]; then + echo "Skipping install of site: $(capitalize ${site})" + return 0 + fi + + # Installing from an existing config is optional only works with + # non-standard profiles. Ones that do not specify an install hook. + # + # https://www.drupal.org/node/2897299 + # https://www.drupal.org/project/drupal/issues/2982052 + if [[ "${use_existing_config}" == "true" ]]; then + use_existing_config_arg="--existing-config" + fi + + # Ensure the files directory is writable by nginx, as when it is a new volume it is owned by root. + chown -R 100:101 "${files_directory}" + chmod -R ug+rw "${files_directory}" + + # Allow changes to settings.php if it exists. + if [[ -f "${site_directory}/settings.php" ]]; then + chmod a=rwx "${site_directory}/settings.php" + fi + + /usr/local/bin/install-drupal-site.sh \ + --driver "${driver}" \ + --host "${host}" \ + --port "${port}" \ + --db-user "${user}" \ + --db-password "${password}" \ + --db-name "${db_name}" \ + "${profile}" \ + --account-mail="${account_email}" \ + --account-name="${account_name}" \ + --account-pass="${account_password}" \ + --site-mail="${site_email}" \ + --locale="${site_locale}" \ + --site-name="${site_name}" \ + --sites-subdir="${subdir}" \ + "${use_existing_config_arg}" \ + ${@} + + # Restrict changes to settings.php + if [[ -f "${site_directory}/settings.php" ]]; then + chmod a=,ug=r "${site_directory}/settings.php" + fi +} + +# Get the base url of fedora. +function fedora_url { + local site="${1}"; shift + local fcrepo_host=$(drupal_site_env "${site}" "FCREPO_HOST") + local fcrepo_port=$(drupal_site_env "${site}" "FCREPO_PORT") + + # Indexing fails if port 80 is given explicitly. + if [[ "${fcrepo_port}" == "80" ]]; then + echo "http://${fcrepo_host}/fcrepo/rest/" + else + echo "http://${fcrepo_host}:${fcrepo_port}/fcrepo/rest/" + fi +} + +# Regenerate / Update settings.php +function update_settings_php { + local site="${1}"; shift + local drupal_root=$(drush drupal:directory) + local site_url=$(drupal_site_env "${site}" "SITE_URL") + local driver=$(drupal_site_env "${site}" "DB_DRIVER") + local host=$(drupal_site_env "${site}" "DB_HOST") + local port=$(drupal_site_env "${site}" "DB_PORT") + local user=$(drupal_site_env "${site}" "DB_USER") + local password=$(drupal_site_env "${site}" "DB_PASSWORD") + local db_name=$(drupal_site_env "${site}" "DB_NAME") + local config_dir=$(drupal_site_env "${site}" "CONFIGDIR") + local fcrepo_host=$(drupal_site_env "${site}" "FCREPO_HOST") + local fcrepo_port=$(drupal_site_env "${site}" "FCREPO_PORT") + local salt=$(drupal_site_env "${site}" "SALT") + local subdir=$(drupal_site_env "${site}" "SUBDIR") + local site_directory=$(realpath "${drupal_root}/sites/${subdir}") + local install=$(drupal_site_env "${site}" "INSTALL") + local fedora_url=$(fedora_url "${site}") + local previous_owner_group= + + if [ "${install}" != "true" ]; then + echo "Skipping update of settings.php for site: $(capitalize "${site}")" + return 0 + fi + + # Allow modifications to settings.php + if [ -f "${site_directory}/settings.php" ]; then + previous_owner_group=$(stat -c "%u:%g" "${site_directory}/settings.php") + chown 100:101 "${site_directory}/settings.php" + chmod a=rwx "${site_directory}/settings.php" + fi + + drush -l "${site_url}" islandora:settings:create-settings-if-missing + drush -l "${site_url}" islandora:settings:set-hash-salt "${salt}" + drush -l "${site_url}" islandora:settings:set-flystem-fedora-url "${fedora_url}" + drush -l "${site_url}" islandora:settings:set-database-settings \ + "${db_name}" \ + "${user}" \ + "${password}" \ + "${host}" \ + "${port}" \ + "${driver}" + + # Specifiying the config_dir is optional, some users will hardcode it in + # their settings.php so it does not need updating. + if [ ! -z "${config_dir}" ]; then + drush -l "${site_url}" islandora:settings:set-config-sync-directory ${config_dir} + fi + + # Restore owner/group to previous value + if [ ! -z "${previous_owner_group}" ]; then + chown "${previous_owner_group}" "${site_directory}/settings.php" + fi + + # Restrict access to settings.php + chmod 444 "${site_directory}/settings.php" +} + +# Enable module and apply configuration. +function configure_jwt_module { + local site="${1}" + local site_url=$(drupal_site_env "${site}" "SITE_URL") + drush -l "${site_url}" -y pm:enable jwt + drush -l "${site_url}" -y config:import --partial --source=/etc/islandora/configs/jwt +} + +# Install and configure the islandora module. +function configure_islandora_module { + local site="${1}"; shift + local site_url=$(drupal_site_env "${site}" "SITE_URL") + local broker_host=$(drupal_site_env "${site}" "BROKER_HOST") + local broker_port=$(drupal_site_env "${site}" "BROKER_PORT") + local broker_url="tcp://${broker_host}:${broker_port}" + local gemini_host=$(drupal_site_env "${site}" "GEMINI_HOST") + local gemini_port=$(drupal_site_env "${site}" "GEMINI_PORT") + local gemini_url="http://${gemini_host}:${gemini_port}" + + drush -l "${site_url}" -y pm:enable islandora + drush -l "${site_url}" -y config:set --input-format=yaml jsonld.settings remove_jsonld_format true + drush -l "${site_url}" -y config:set --input-format=yaml islandora.settings broker_url "${broker_url}" + drush -l "${site_url}" -y config:set --input-format=yaml islandora.settings gemini_url "${gemini_url}" + drush -l "${site_url}" -y config:set --input-format=yaml islandora.settings gemini_pseudo_bundles.0 "islandora_object:node" + drush -l "${site_url}" -y config:set --input-format=yaml islandora.settings gemini_pseudo_bundles.1 "image:media" + drush -l "${site_url}" -y config:set --input-format=yaml islandora.settings gemini_pseudo_bundles.2 "file:media" + drush -l "${site_url}" -y config:set --input-format=yaml islandora.settings gemini_pseudo_bundles.3 "audio:media" + drush -l "${site_url}" -y config:set --input-format=yaml islandora.settings gemini_pseudo_bundles.4 "video:media" +} + +# After enabling and importing features a number of configurations need to be updated. +function configure_islandora_default_module { + local site="${1}"; shift + local site_url=$(drupal_site_env "${site}" "SITE_URL") + local host=$(drupal_site_env "${site}" "SOLR_HOST") + local port=$(drupal_site_env "${site}" "SOLR_PORT") + + drush -l "${site_url}" -y user:role:add fedoraadmin admin + drush -l "${site_url}" -y config:set search_api.server.default_solr_server backend_config.connector_config.host "${host}" + drush -l "${site_url}" -y config:set search_api.server.default_solr_server backend_config.connector_config.port "${port}" +} + +# Install search_api_solr and configure. Also uninstall the default search module. +function configure_search_api_solr_module { + local site="${1}"; shift + local site_url=$(drupal_site_env "${site}" "SITE_URL") + + drush -l "${site_url}" -y pm:enable search_api_solr + drush -l "${site_url}" -y pm:uninstall search +} + +# Enables and sets carapace as the default theme. +function set_carapace_default_theme { + local site="${1}"; shift + local site_url=$(drupal_site_env "${site}" "SITE_URL") + drush -l "${site_url}" -y theme:enable carapace + drush -l "${site_url}" -y config:set system.theme default carapace +} + +# Generate solr config using the search_api_solr module. +# +# Assumes the search_api_solr module has already been installed. +# Assumes that the destination will be a shared volume. +function generate_solr_config { + local site="${1}"; shift + local site_url=$(drupal_site_env "${site}" "SITE_URL") + local core=$(drupal_site_env "${site}" "SOLR_CORE") + local dest="${1-/opt/solr/server/solr/${core}}"; + + mkdir -p "/tmp/${core}" || true + chmod a+rwx "/tmp/${core}" + drush -l "${site_url}" -y search-api-solr:get-server-config default_solr_server "/tmp/${core}/solr_config.zip" 7.1 + mkdir -p "${dest}/conf" || true + mkdir -p "${dest}/data" || true + unzip -o "/tmp/${core}/solr_config.zip" -d "${dest}/conf" + + # The uid:gid "100:1000" is "solr:solr" inside of the solr container. + chown -R 100:1000 "${dest}" +} + +# Creates a SOLR core for the site using the Solr REST API. +function create_solr_core { + local site="${1}"; shift + local core=$(drupal_site_env "${site}" "SOLR_CORE") + local host=$(drupal_site_env "${site}" "SOLR_HOST") + local port=$(drupal_site_env "${site}" "SOLR_PORT") + + # Require a running Solr to create a core. + wait_for_service "${site}" "SOLR" + + curl -s "http://${host}:${port}/solr/admin/cores?action=CREATE&name=${core}&instanceDir=${core}&config=solrconfig.xml&dataDir=data" &> /dev/null +} + +# Generate solr config and create a core for it. +function create_solr_core_with_default_config { + local site="${1}"; shift + generate_solr_config "${site}" + create_solr_core "${site}" +} + +# Install matomo and configure. +function configure_matomo_module { + local site="${1}"; shift + local site_url=$(drupal_site_env "${site}" "SITE_URL") + local site_id=$(($(site_index "${site}")+1)) + local matamo_url=$(drupal_site_env "${site}" "MATOMO_URL") + + drush -l "${site_url}" -y pm:enable matomo + drush -l "${site_url}" -y config-set matomo.settings site_id "${site_id}" + drush -l "${site_url}" -y config-set matomo.settings url_http "${matamo_url}" +} + +# Configure Openseadragon to point use cantaloupe. +function configure_openseadragon { + local site="${1}"; shift + local site_url=$(drupal_site_env "${site}" "SITE_URL") + local cantaloupe_url=$(drupal_site_env "${site}" "CANTALOUPE_URL") + + drush -l "${site_url}" -y config-set --input-format=yaml media.settings standalone_url true + drush -l "${site_url}" -y config-set --input-format=yaml openseadragon.settings iiif_server "${cantaloupe_url}" + drush -l "${site_url}" -y config-set --input-format=yaml openseadragon.settings manifest_view iiif_manifest + drush -l "${site_url}" -y config-set --input-format=yaml islandora_iiif.settings iiif_server "${cantaloupe_url}" +} + +# Imports any migrations in the 'islandora' group. +function import_islandora_migrations { + local site="${1}"; shift + local site_url=$(drupal_site_env "${site}" "SITE_URL") + drush -l "${site_url}" -y --userid=1 migrate:import --group=islandora +} + +# Enable module and apply configuration. +function enable_modules { + local site="${1}"; shift + local site_url=$(drupal_site_env "${site}" "SITE_URL") + drush -l "${site_url}" -y pm:enable ${@} +} + +# Enable module and apply configuration. +function import_features { + local site="${1}"; shift + local site_url=$(drupal_site_env "${site}" "SITE_URL") + local features=$(join_by , ${@}); shift + drush -l "${site_url}" fim --no-interaction --yes "${features}" +} + +# Rebuild the cache for the given site. +function cache_rebuild { + local site="${1}"; shift + local site_url=$(drupal_site_env "${site}" "SITE_URL") + drush -l "${site_url}" -y cache:rebuild +} + +# Changes the site ID to match the configuration folder to allow it to be imported. +function set_site_uuid { + local site="${1}"; shift + local site_url=$(drupal_site_env "${site}" "SITE_URL") + local drupal_root=$(drush drupal:directory) + local config_dir=$(cd $drupal_root; realpath `drush php:eval "echo \Drupal\Core\Site\Settings::get('config_sync_directory');"`) # Handle the case if config_dir is a relative path. + local uuid=${1-$(cat ${config_dir}/system.site.yml | awk '/uuid/ { print $2 }')} + drush -l ${site_url} -y config:set --input-format=yaml system.site uuid ${uuid} +} + +# Replace references to standard profile in the config files with minimal. +# +# Often we build sites with the standard profile but it is not possible to install +# from a configuration that was generated on a standard profile site. +# +# https://www.drupal.org/project/drupal/issues/2982052 +function remove_standard_profile_references_from_config { + # Do not modify configuration in in the core module. + local config_files=$(find /var/www/drupal -name "core.extension.yml" ! -path '*/core/*') + for config_file in ${config_files}; do + # Remove standard profile references, and replace with minimal. + sed -i 's|\( *\)standard:\(.*\)|\1minimal:\2|' ${config_file} + sed -i 's|profile: *standard|profile: minimal|' ${config_file} + done +} + +# Import sites configuration. +function import_config { + local site="${1}" + local site_url=$(drupal_site_env "${site}" "SITE_URL") + drush -l ${site_url} -y config:import +} + +# Export sites configuration. +function export_config { + local site="${1}" + local site_url=$(drupal_site_env "${site}" "SITE_URL") + drush -l ${site_url} -y config:export +} + +# Generates blazegraph properties for the given site using its namespace. +function default_blazegraph_properties { + local site="${1}"; shift + local namespace=$(drupal_site_env "${site}" "TRIPLESTORE_NAMESPACE") + cat <<- EOF +com.bigdata.rdf.store.AbstractTripleStore.textIndex=false +com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.OwlAxioms +com.bigdata.rdf.sail.isolatableIndices=false +com.bigdata.rdf.store.AbstractTripleStore.justify=true +com.bigdata.rdf.sail.truthMaintenance=true +com.bigdata.rdf.sail.namespace=${namespace} +com.bigdata.rdf.store.AbstractTripleStore.quads=false +com.bigdata.namespace.${namespace}.lex.com.bigdata.btree.BTree.branchingFactor=400 +com.bigdata.journal.Journal.groupCommit=false +com.bigdata.namespace.${namespace}.spo.com.bigdata.btree.BTree.branchingFactor=1024 +com.bigdata.rdf.store.AbstractTripleStore.geoSpatial=false +com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false +EOF +} + +# Create a namespace with the given properties file. +function create_blazegraph_namespace { + local site="${1}"; shift + local properties_file="${1}"; shift + local host=$(drupal_site_env "${site}" "TRIPLESTORE_HOST") + local port=$(drupal_site_env "${site}" "TRIPLESTORE_PORT") + local namespace=$(drupal_site_env "${site}" "TRIPLESTORE_NAMESPACE") + local triplestore_url="http://${host}:${port}/bigdata" + + # Require a running blazegraph to update it. + wait_for_service "${site}" "TRIPLESTORE" + + # Setup namespace / inference for the given namespace. + curl -X POST -H "Content-type: text/plain" --data-binary "@${properties_file}" "${triplestore_url}/namespace" + curl -X POST -H "Content-type: text/plain" --data-binary @/etc/islandora/configs/inference.nt "${triplestore_url}/namespace/${namespace}/sparql" +} + +# Create a namespace with default properties for the given site. +function create_blazegraph_namespace_with_default_properties { + local site="${1}"; shift + create_blazegraph_namespace "${site}" <(default_blazegraph_properties "${site}") +} diff --git a/drupal/rootfs/usr/local/bin/create-drupal-database.sh b/drupal/rootfs/usr/local/bin/create-drupal-database.sh index 3baaf02a..74df00eb 100755 --- a/drupal/rootfs/usr/local/bin/create-drupal-database.sh +++ b/drupal/rootfs/usr/local/bin/create-drupal-database.sh @@ -101,7 +101,6 @@ function cmdline { esac done - if [[ -z $DRIVER || -z $HOST || -z $PORT || -z $ROOT_USER || -z $ROOT_PASSWORD || -z $DB_NAME || -z $DB_USER || -z $DB_PASS ]]; then echo "Missing one of required options: --host --port --user --password --db-name --db-user --db-password" exit 1 diff --git a/drupal/rootfs/usr/local/bin/drush b/drupal/rootfs/usr/local/bin/drush new file mode 100755 index 00000000..8598f6ae --- /dev/null +++ b/drupal/rootfs/usr/local/bin/drush @@ -0,0 +1,7 @@ +#!/usr/bin/env bash +set -e + +# Ensures drush runs as the correct user, and does not run out of memory. +# Takes precedence due to order of $PATH. Preferred to an alias as it will apply +# regardless of which shell is used or how it is started (login, interactive, etc) +s6-setuidgid nginx php -d memory_limit=-1 /usr/bin/drush "${@}" diff --git a/drupal/rootfs/usr/local/bin/install-drupal-site.sh b/drupal/rootfs/usr/local/bin/install-drupal-site.sh index 383f8334..ed8e0f45 100755 --- a/drupal/rootfs/usr/local/bin/install-drupal-site.sh +++ b/drupal/rootfs/usr/local/bin/install-drupal-site.sh @@ -14,8 +14,8 @@ function usage { --driver The database driver. --host The database host. --port The database port. - --user The user to connect as. - --password The password to use for the user. + --db-user The user to connect as. + --db-password The password to use for the user. --db-name The name of the database to install into. -h --help Show this help. @@ -27,8 +27,8 @@ function usage { --driver "mysql" \\ --host "database" \\ --port "3306" \\ - --user "root" \\ - --password "password" \\ + --db-user "root" \\ + --db-password "password" \\ --db-name "drupal_default" \\ standard --sites-subdir=default --site-name=Islandora EOF @@ -41,14 +41,14 @@ function cmdline { local delim="" case "$arg" in # Translate --gnu-long-options to -g (short options) - --driver) args="${args}-a ";; - --host) args="${args}-b ";; - --port) args="${args}-c ";; - --user) args="${args}-d ";; - --password) args="${args}-e ";; - --db-name) args="${args}-f ";; - --help) args="${args}-h ";; - --debug) args="${args}-x ";; + --driver) args="${args}-a ";; + --host) args="${args}-b ";; + --port) args="${args}-c ";; + --db-user) args="${args}-d ";; + --db-password) args="${args}-e ";; + --db-name) args="${args}-f ";; + --help) args="${args}-h ";; + --debug) args="${args}-x ";; # Pass through anything else *) [[ "${arg:0:1}" == "-" ]] || delim="\"" args="${args}${delim}${arg}${delim} ";; @@ -71,10 +71,10 @@ function cmdline { readonly PORT=${OPTARG} ;; d) - readonly ROOT_USER=${OPTARG} + readonly DB_USER=${OPTARG} ;; e) - readonly ROOT_PASSWORD=${OPTARG} + readonly DB_PASSWORD=${OPTARG} ;; f) readonly DB_NAME=${OPTARG} @@ -90,8 +90,8 @@ function cmdline { esac done - if [[ -z $DRIVER || -z $HOST || -z $PORT || -z $ROOT_USER || -x $ROOT_PASSWORD ]]; then - echo "Missing one of required options: --host --port --user --password" + if [[ -z $DRIVER || -z $HOST || -z $PORT || -z $DB_USER || -z $DB_PASSWORD || -z $DB_NAME ]]; then + echo "Missing one of required options: --host --port --db-user --db-password --db-name" exit 1 fi @@ -113,8 +113,8 @@ EOF function installed { # Check the number of tables to determine if it has already been installed. local count=$(mysql \ - --user="${ROOT_USER}" \ - --password="${ROOT_PASSWORD}" \ + --user="${DB_USER}" \ + --password="${DB_PASSWORD}" \ --host="${HOST}" \ --port="${PORT}" \ --protocol=tcp \ @@ -130,9 +130,9 @@ function main { return 0 fi echo "Installing site." - s6-setuidgid nginx drush \ + drush \ -n \ si ${DRUSH_ARGS} \ - --db-url="${DRIVER}://${ROOT_USER}:${ROOT_PASSWORD}@${HOST}:${PORT}/${DB_NAME}" + --db-url="${DRIVER}://${DB_USER}:${DB_PASSWORD}@${HOST}:${PORT}/${DB_NAME}" } main diff --git a/drupal/rootfs/usr/share/drush/Commands/UpdateSettingsCommands.php b/drupal/rootfs/usr/share/drush/Commands/UpdateSettingsCommands.php new file mode 100644 index 00000000..d4d814d9 --- /dev/null +++ b/drupal/rootfs/usr/share/drush/Commands/UpdateSettingsCommands.php @@ -0,0 +1,180 @@ +getSettingFilePath(); + $fs = new Filesystem(); + if (!$fs->exists($settings_file)) { + $fs->copy(DRUPAL_ROOT . '/sites/default/default.settings.php', $settings_file); + } + } + + /** + * Set `config_sync_directory` in settings.php + * + * @command islandora:settings:set-config-sync-directory + * @bootstrap site + * @param $path The path to use for the `config_sync_directory` setting + * @usage drush islandora:settings:set-config-sync-directory ../config/sync + * Sets `config_sync_directory` in settings.php. + */ + public function setConfigSyncDirectory($path) + { + $settings['settings']['config_sync_directory'] = (object) [ + 'value' => $path, + 'required' => TRUE, + ]; + $this->writeSettings($settings); + } + + /** + * Set `hash_salt` in settings.php + * + * @command islandora:settings:set-hash-salt + * @bootstrap site + * @param $salt The value of the salt + * @usage drush islandora:settings:set-hash-salt IpWSRBrkTDKAL_ykij_WJJrnqcDv + * Sets `hash_salt` in settings.php, use something like Crypt::randomBytesBase64(55). + */ + public function setHashSalt($salt) + { + $settings['settings']['hash_salt'] = (object) [ + 'value' => $salt, + 'required' => TRUE, + ]; + $this->writeSettings($settings); + } + + /** + * Set `flysystem` in settings.php + * + * @command islandora:settings:set-flystem-fedora-url + * @bootstrap site + * @param $url The root url to Fedora + * @usage drush islandora:settings:set-flystem-fedora-url http://fcrepo.isle-dc.localhost/fcrepo/rest/ + * Sets `flysystem` in settings.php. + */ + public function setFlystemFedoraUrl($url) + { + $settings['settings']['flysystem']['fedora']['driver'] = (object) [ + 'value' => 'fedora', + 'required' => TRUE, + ]; + $settings['settings']['flysystem']['fedora']['config']['root'] = (object) [ + 'value' => $url, + 'required' => TRUE, + ]; + $this->writeSettings($settings); + } + + /** + * Set `database` settings in settings.php + * + * @command islandora:settings:set-database-settings + * @bootstrap site + * @param $database The name of the database + * @param $username The user name to connect with + * @param $password The password of the user + * @param $host The database host + * @param $port The database port + * @param $driver Database driver defaults to 'mysql' + * @param $prefix Table prefix + * + * Sets `database` in settings.php. + */ + public function setDatabaseSettings( + $database, + $username, + $password, + $host, + $port, + $driver = 'mysql', + $prefix = '' + ) { + $default_database['database'] = (object) [ + 'value' => $database, + 'required' => TRUE, + ]; + $default_database['username'] = (object) [ + 'value' => $username, + 'required' => TRUE, + ]; + $default_database['password'] = (object) [ + 'value' => $password, + 'required' => TRUE, + ]; + $default_database['host'] = (object) [ + 'value' => $host, + 'required' => TRUE, + ]; + $default_database['port'] = (object) [ + 'value' => $port, + 'required' => TRUE, + ]; + $default_database['prefix'] = (object) [ + 'value' => $prefix, + 'required' => TRUE, + ]; + $default_database['driver'] = (object) [ + 'value' => $driver, + 'required' => TRUE, + ]; + $default_database['namespace'] = (object) [ + 'value' => 'Drupal\\Core\\Database\\Driver\\' . $driver, + 'required' => TRUE, + ]; + $settings['databases']['default']['default'] = $default_database; + $this->writeSettings($settings); + } + + /** + * Determine which settings file to update. + */ + private function getSettingFilePath() + { + return DRUPAL_ROOT . "/" . \Drush\Drush::bootstrap()->confPath() . "/settings.php"; + } + + /** + * Determine which settings file to update. + */ + private function writeSettings($settings) + { + require_once DRUPAL_ROOT . '/core/includes/install.inc'; + $settings_file = $this->getSettingFilePath(); + $fs = new Filesystem(); + $fs->chmod($settings_file, 0755); + new Settings([]); + drupal_rewrite_settings($settings, $settings_file); + $fs->chmod($settings_file, 0400); + } +} diff --git a/fcrepo/rootfs/etc/confd/conf.d/public.key.toml b/fcrepo/rootfs/etc/confd/conf.d/public.key.toml index 6306435d..ec9810ec 100644 --- a/fcrepo/rootfs/etc/confd/conf.d/public.key.toml +++ b/fcrepo/rootfs/etc/confd/conf.d/public.key.toml @@ -4,4 +4,4 @@ dest = "/opt/keys/jwt/public.key" uid = 100 gid = 1000 mode = "0600" -keys = [ "/jwt" ] \ No newline at end of file +keys = [ "/jwt" ] diff --git a/fcrepo/rootfs/etc/cont-init.d/03-fcrepo-setup.sh b/fcrepo/rootfs/etc/cont-init.d/03-fcrepo-setup.sh index 1d6a319f..9d5bf949 100644 --- a/fcrepo/rootfs/etc/cont-init.d/03-fcrepo-setup.sh +++ b/fcrepo/rootfs/etc/cont-init.d/03-fcrepo-setup.sh @@ -1,8 +1,5 @@ #!/usr/bin/with-contenv bash set -e -# Key needs to be present before startup otherwise, -# it will ignore subsequent requests even if the key has been generated. -timeout 300 bash -c 'until [[ -f /opt/keys/jwt/public.key ]]; do sleep 1; done' # Change log files to redirect to stdout/stderr ln -sf /dev/stdout /opt/tomcat/logs/velocity.log diff --git a/matomo/README.md b/matomo/README.md index f5f86e14..ebdc7899 100644 --- a/matomo/README.md +++ b/matomo/README.md @@ -30,7 +30,7 @@ Additionally it requires a database backend to run, and website to aggregate met | MATOMO_DB_ROOT_PASSWORD | /matomo/db/root/password | password | The root user password (used to create the database / user) | | MATOMO_DB_ROOT_USER | /matomo/db/root/user | root | The root user (used to create the database / user) | | MATOMO_DB_USER | /matomo/db/user | matomo | The user to create / use when interacting with the database | -| MATOMO_SITE_HOST | /matomo/site/host | islandora.localhost | The URL of the site for which to gather metrics for | +| MATOMO_SITE_HOST | /matomo/site/host | islandora.isle-dc.localhost | The URL of the site for which to gather metrics for | | MATOMO_SITE_NAME | /matomo/site/name | Islandora | The name of the site | | MATOMO_SITE_TIMEZONE | /matomo/site/timezone | America/Halifax | The timezone the site is hosted in | | MATOMO_USER_EMAIL | /matomo/user/email | admin@example.org | The site administrator email | diff --git a/matomo/rootfs/etc/confd/templates/config.ini.php.tmpl b/matomo/rootfs/etc/confd/templates/config.ini.php.tmpl index 40966584..00650b25 100644 --- a/matomo/rootfs/etc/confd/templates/config.ini.php.tmpl +++ b/matomo/rootfs/etc/confd/templates/config.ini.php.tmpl @@ -12,7 +12,7 @@ proxy_uri_header = 1 proxy_client_headers[] = "HTTP_X_FORWARDED_FOR" proxy_host_headers[] = "HTTP_X_FORWARDED_HOST" salt = "5a472390550bd59e4428a41aa472137b" -trusted_hosts[] = {{ getv "/site/host" "islandora.localhost" }} +trusted_hosts[] = {{ getv "/site/host" "islandora.isle-dc.localhost" }} [PluginsInstalled] PluginsInstalled[] = "Diagnostics" diff --git a/matomo/rootfs/etc/confd/templates/create-matomo-database.sql.tmpl b/matomo/rootfs/etc/confd/templates/create-matomo-database.sql.tmpl index ff5a95d0..922babc7 100644 --- a/matomo/rootfs/etc/confd/templates/create-matomo-database.sql.tmpl +++ b/matomo/rootfs/etc/confd/templates/create-matomo-database.sql.tmpl @@ -640,7 +640,7 @@ CREATE TABLE `matomo_option` ( LOCK TABLES `matomo_option` WRITE; /*!40000 ALTER TABLE `matomo_option` DISABLE KEYS */; -INSERT INTO `matomo_option` VALUES ('install_version','3.13.5',0),('lastTrackerCronRun','1591620993',0),('MobileMessaging_DelegatedManagement','false',0),('piwikUrl','http://{{ getv "/site/host" "islandora.localhost" }}/',1),('PrivacyManager.doNotTrackEnabled','1',0),('PrivacyManager.ipAnonymizerEnabled','1',0),('SitesManagerHadTrafficInPast_1','1',0),('SitesManager_DefaultTimezone','America/Halifax',0),('TaskScheduler.timetable','a:23:{s:60:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.invalidateOutdatedArchives\";i:1591660833;s:55:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.purgeOutdatedArchives\";i:1591660833;s:55:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.purgeOrphanedArchives\";i:1592179233;s:56:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.updateSpammerBlacklist\";i:1592179233;s:61:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.checkSiteHasTrackedVisits_1\";i:1591927200;s:49:\"Piwik\\Plugins\\Referrers\\Tasks.updateSearchEngines\";i:1592179233;s:43:\"Piwik\\Plugins\\Referrers\\Tasks.updateSocials\";i:1592179233;s:47:\"Piwik\\Plugins\\Login\\Tasks.cleanupBruteForceLogs\";i:1591660833;s:63:\"Piwik\\Plugins\\UsersManager\\Tasks.setUserDefaultReportPreference\";i:1591660833;s:47:\"Piwik\\Plugins\\CustomPiwikJs\\Tasks.updateTracker\";i:1591621233;s:58:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.purgeInvalidatedArchives\";i:1591660833;s:51:\"Piwik\\Plugins\\PrivacyManager\\Tasks.deleteReportData\";i:1591660833;s:48:\"Piwik\\Plugins\\PrivacyManager\\Tasks.deleteLogData\";i:1591621233;s:52:\"Piwik\\Plugins\\PrivacyManager\\Tasks.anonymizePastData\";i:1591621233;s:63:\"Piwik\\Plugins\\PrivacyManager\\Tasks.deleteLogDataForDeletedSites\";i:1592179233;s:54:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.optimizeArchiveTable\";i:1593561633;s:57:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.cleanupTrackingFailures\";i:1591660833;s:56:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.notifyTrackingFailures\";i:1592179233;s:49:\"Piwik\\Plugins\\UserCountry\\GeoIPAutoUpdater.update\";i:1593561633;s:45:\"Piwik\\Plugins\\GeoIp2\\GeoIP2AutoUpdater.update\";i:1593561633;s:65:\"Piwik\\Plugins\\CoreUpdater\\Tasks.sendNotificationIfUpdateAvailable\";i:1591660833;s:52:\"Piwik\\Plugins\\Marketplace\\Tasks.clearAllCacheEntries\";i:1591660833;s:66:\"Piwik\\Plugins\\Marketplace\\Tasks.sendNotificationIfUpdatesAvailable\";i:1591660833;}',0),('TransactionLevel.testOption','1',0),('UpdateCheck_LastTimeChecked','1591620890',1),('UpdateCheck_LatestVersion','',0),('useridsalt','IC0LXAntug7v6386PknYKbYVNuRxriff41lkUGgR',1),('UsersManager.lastSeen.admin','1591621332',1),('version_Actions','3.13.5',1),('version_Annotations','3.13.5',1),('version_API','3.13.5',1),('version_BulkTracking','3.13.5',1),('version_Contents','3.13.5',1),('version_core','3.13.5',1),('version_CoreAdminHome','3.13.5',1),('version_CoreConsole','3.13.5',1),('version_CoreHome','3.13.5',1),('version_CorePluginsAdmin','3.13.5',1),('version_CoreUpdater','3.13.5',1),('version_CoreVisualizations','3.13.5',1),('version_CustomPiwikJs','3.13.5',1),('version_CustomVariables','3.13.5',1),('version_Dashboard','3.13.5',1),('version_DevicePlugins','3.13.5',1),('version_DevicesDetection','3.13.5',1),('version_Diagnostics','3.13.5',1),('version_Ecommerce','3.13.5',1),('version_Events','3.13.5',1),('version_Feedback','3.13.5',1),('version_GeoIp2','3.13.5',1),('version_Goals','3.13.5',1),('version_Heartbeat','3.13.5',1),('version_ImageGraph','3.13.5',1),('version_Insights','3.13.5',1),('version_Installation','3.13.5',1),('version_Intl','3.13.5',1),('version_IntranetMeasurable','3.13.5',1),('version_LanguagesManager','3.13.5',1),('version_Live','3.13.5',1),('version_Login','3.13.5',1),('version_log_conversion.revenue','float default NULL',1),('version_log_conversion.revenue_discount','float default NULL',1),('version_log_conversion.revenue_shipping','float default NULL',1),('version_log_conversion.revenue_subtotal','float default NULL',1),('version_log_conversion.revenue_tax','float default NULL',1),('version_log_link_visit_action.idaction_content_interaction','INTEGER(10) UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.idaction_content_name','INTEGER(10) UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.idaction_content_piece','INTEGER(10) UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.idaction_content_target','INTEGER(10) UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.idaction_event_action','INTEGER(10) UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.idaction_event_category','INTEGER(10) UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.idaction_name','INTEGER(10) UNSIGNED',1),('version_log_link_visit_action.idaction_url','INTEGER(10) UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.idpageview','CHAR(6) NULL DEFAULT NULL',1),('version_log_link_visit_action.interaction_position','SMALLINT UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.server_time','DATETIME NOT NULL',1),('version_log_link_visit_action.time_spent_ref_action','INTEGER(10) UNSIGNED NULL',1),('version_log_visit.config_browser_engine','VARCHAR(10) NULL',1),('version_log_visit.config_browser_name','VARCHAR(10) NULL',1),('version_log_visit.config_browser_version','VARCHAR(20) NULL',1),('version_log_visit.config_cookie','TINYINT(1) NULL',1),('version_log_visit.config_device_brand','VARCHAR( 100 ) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL1',1),('version_log_visit.config_device_model','VARCHAR( 100 ) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL1',1),('version_log_visit.config_device_type','TINYINT( 100 ) NULL DEFAULT NULL1',1),('version_log_visit.config_director','TINYINT(1) NULL',1),('version_log_visit.config_flash','TINYINT(1) NULL',1),('version_log_visit.config_gears','TINYINT(1) NULL',1),('version_log_visit.config_java','TINYINT(1) NULL',1),('version_log_visit.config_os','CHAR(3) NULL',1),('version_log_visit.config_os_version','VARCHAR( 100 ) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL',1),('version_log_visit.config_pdf','TINYINT(1) NULL',1),('version_log_visit.config_quicktime','TINYINT(1) NULL',1),('version_log_visit.config_realplayer','TINYINT(1) NULL',1),('version_log_visit.config_resolution','VARCHAR(18) NULL',1),('version_log_visit.config_silverlight','TINYINT(1) NULL',1),('version_log_visit.config_windowsmedia','TINYINT(1) NULL',1),('version_log_visit.location_browser_lang','VARCHAR(20) NULL',1),('version_log_visit.location_city','varchar(255) DEFAULT NULL1',1),('version_log_visit.location_country','CHAR(3) NULL1',1),('version_log_visit.location_latitude','decimal(9, 6) DEFAULT NULL1',1),('version_log_visit.location_longitude','decimal(9, 6) DEFAULT NULL1',1),('version_log_visit.location_region','char(3) DEFAULT NULL1',1),('version_log_visit.referer_keyword','VARCHAR(255) NULL1',1),('version_log_visit.referer_name','VARCHAR(70) NULL1',1),('version_log_visit.referer_type','TINYINT(1) UNSIGNED NULL1',1),('version_log_visit.referer_url','TEXT NULL',1),('version_log_visit.user_id','VARCHAR(200) NULL',1),('version_log_visit.visitor_count_visits','INT(11) UNSIGNED NOT NULL1',1),('version_log_visit.visitor_days_since_first','SMALLINT(5) UNSIGNED NULL1',1),('version_log_visit.visitor_days_since_last','SMALLINT(5) UNSIGNED NULL',1),('version_log_visit.visitor_days_since_order','SMALLINT(5) UNSIGNED NULL1',1),('version_log_visit.visitor_localtime','TIME NULL',1),('version_log_visit.visitor_returning','TINYINT(1) NULL1',1),('version_log_visit.visit_entry_idaction_name','INTEGER(10) UNSIGNED NULL',1),('version_log_visit.visit_entry_idaction_url','INTEGER(11) UNSIGNED NULL DEFAULT NULL',1),('version_log_visit.visit_exit_idaction_name','INTEGER(10) UNSIGNED NULL',1),('version_log_visit.visit_exit_idaction_url','INTEGER(10) UNSIGNED NULL DEFAULT 0',1),('version_log_visit.visit_first_action_time','DATETIME NOT NULL',1),('version_log_visit.visit_goal_buyer','TINYINT(1) NULL',1),('version_log_visit.visit_goal_converted','TINYINT(1) NULL',1),('version_log_visit.visit_total_actions','INT(11) UNSIGNED NULL',1),('version_log_visit.visit_total_events','INT(11) UNSIGNED NULL',1),('version_log_visit.visit_total_interactions','SMALLINT UNSIGNED DEFAULT 0',1),('version_log_visit.visit_total_searches','SMALLINT(5) UNSIGNED NULL',1),('version_log_visit.visit_total_time','INT(11) UNSIGNED NOT NULL',1),('version_Marketplace','3.13.5',1),('version_MobileMessaging','3.13.5',1),('version_Monolog','3.13.5',1),('version_Morpheus','3.13.5',1),('version_MultiSites','3.13.5',1),('version_Overlay','3.13.5',1),('version_PrivacyManager','3.13.5',1),('version_ProfessionalServices','3.13.5',1),('version_Proxy','3.13.5',1),('version_Referrers','3.13.5',1),('version_Resolution','3.13.5',1),('version_RssWidget','1.0',1),('version_ScheduledReports','3.13.5',1),('version_SegmentEditor','3.13.5',1),('version_SEO','3.13.5',1),('version_SitesManager','3.13.5',1),('version_Tour','3.13.5',1),('version_Transitions','3.13.5',1),('version_TwoFactorAuth','3.13.5',1),('version_UserCountry','3.13.5',1),('version_UserCountryMap','3.13.5',1),('version_UserId','3.13.5',1),('version_UserLanguage','3.13.5',1),('version_UsersManager','3.13.5',1),('version_VisitFrequency','3.13.5',1),('version_VisitorInterest','3.13.5',1),('version_VisitsSummary','3.13.5',1),('version_VisitTime','3.13.5',1),('version_WebsiteMeasurable','3.13.5',1),('version_Widgetize','3.13.5',1); +INSERT INTO `matomo_option` VALUES ('install_version','3.13.5',0),('lastTrackerCronRun','1591620993',0),('MobileMessaging_DelegatedManagement','false',0),('piwikUrl','http://{{ getv "/site/host" "islandora.isle-dc.localhost" }}/',1),('PrivacyManager.doNotTrackEnabled','1',0),('PrivacyManager.ipAnonymizerEnabled','1',0),('SitesManagerHadTrafficInPast_1','1',0),('SitesManager_DefaultTimezone','America/Halifax',0),('TaskScheduler.timetable','a:23:{s:60:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.invalidateOutdatedArchives\";i:1591660833;s:55:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.purgeOutdatedArchives\";i:1591660833;s:55:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.purgeOrphanedArchives\";i:1592179233;s:56:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.updateSpammerBlacklist\";i:1592179233;s:61:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.checkSiteHasTrackedVisits_1\";i:1591927200;s:49:\"Piwik\\Plugins\\Referrers\\Tasks.updateSearchEngines\";i:1592179233;s:43:\"Piwik\\Plugins\\Referrers\\Tasks.updateSocials\";i:1592179233;s:47:\"Piwik\\Plugins\\Login\\Tasks.cleanupBruteForceLogs\";i:1591660833;s:63:\"Piwik\\Plugins\\UsersManager\\Tasks.setUserDefaultReportPreference\";i:1591660833;s:47:\"Piwik\\Plugins\\CustomPiwikJs\\Tasks.updateTracker\";i:1591621233;s:58:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.purgeInvalidatedArchives\";i:1591660833;s:51:\"Piwik\\Plugins\\PrivacyManager\\Tasks.deleteReportData\";i:1591660833;s:48:\"Piwik\\Plugins\\PrivacyManager\\Tasks.deleteLogData\";i:1591621233;s:52:\"Piwik\\Plugins\\PrivacyManager\\Tasks.anonymizePastData\";i:1591621233;s:63:\"Piwik\\Plugins\\PrivacyManager\\Tasks.deleteLogDataForDeletedSites\";i:1592179233;s:54:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.optimizeArchiveTable\";i:1593561633;s:57:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.cleanupTrackingFailures\";i:1591660833;s:56:\"Piwik\\Plugins\\CoreAdminHome\\Tasks.notifyTrackingFailures\";i:1592179233;s:49:\"Piwik\\Plugins\\UserCountry\\GeoIPAutoUpdater.update\";i:1593561633;s:45:\"Piwik\\Plugins\\GeoIp2\\GeoIP2AutoUpdater.update\";i:1593561633;s:65:\"Piwik\\Plugins\\CoreUpdater\\Tasks.sendNotificationIfUpdateAvailable\";i:1591660833;s:52:\"Piwik\\Plugins\\Marketplace\\Tasks.clearAllCacheEntries\";i:1591660833;s:66:\"Piwik\\Plugins\\Marketplace\\Tasks.sendNotificationIfUpdatesAvailable\";i:1591660833;}',0),('TransactionLevel.testOption','1',0),('UpdateCheck_LastTimeChecked','1591620890',1),('UpdateCheck_LatestVersion','',0),('useridsalt','IC0LXAntug7v6386PknYKbYVNuRxriff41lkUGgR',1),('UsersManager.lastSeen.admin','1591621332',1),('version_Actions','3.13.5',1),('version_Annotations','3.13.5',1),('version_API','3.13.5',1),('version_BulkTracking','3.13.5',1),('version_Contents','3.13.5',1),('version_core','3.13.5',1),('version_CoreAdminHome','3.13.5',1),('version_CoreConsole','3.13.5',1),('version_CoreHome','3.13.5',1),('version_CorePluginsAdmin','3.13.5',1),('version_CoreUpdater','3.13.5',1),('version_CoreVisualizations','3.13.5',1),('version_CustomPiwikJs','3.13.5',1),('version_CustomVariables','3.13.5',1),('version_Dashboard','3.13.5',1),('version_DevicePlugins','3.13.5',1),('version_DevicesDetection','3.13.5',1),('version_Diagnostics','3.13.5',1),('version_Ecommerce','3.13.5',1),('version_Events','3.13.5',1),('version_Feedback','3.13.5',1),('version_GeoIp2','3.13.5',1),('version_Goals','3.13.5',1),('version_Heartbeat','3.13.5',1),('version_ImageGraph','3.13.5',1),('version_Insights','3.13.5',1),('version_Installation','3.13.5',1),('version_Intl','3.13.5',1),('version_IntranetMeasurable','3.13.5',1),('version_LanguagesManager','3.13.5',1),('version_Live','3.13.5',1),('version_Login','3.13.5',1),('version_log_conversion.revenue','float default NULL',1),('version_log_conversion.revenue_discount','float default NULL',1),('version_log_conversion.revenue_shipping','float default NULL',1),('version_log_conversion.revenue_subtotal','float default NULL',1),('version_log_conversion.revenue_tax','float default NULL',1),('version_log_link_visit_action.idaction_content_interaction','INTEGER(10) UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.idaction_content_name','INTEGER(10) UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.idaction_content_piece','INTEGER(10) UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.idaction_content_target','INTEGER(10) UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.idaction_event_action','INTEGER(10) UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.idaction_event_category','INTEGER(10) UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.idaction_name','INTEGER(10) UNSIGNED',1),('version_log_link_visit_action.idaction_url','INTEGER(10) UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.idpageview','CHAR(6) NULL DEFAULT NULL',1),('version_log_link_visit_action.interaction_position','SMALLINT UNSIGNED DEFAULT NULL',1),('version_log_link_visit_action.server_time','DATETIME NOT NULL',1),('version_log_link_visit_action.time_spent_ref_action','INTEGER(10) UNSIGNED NULL',1),('version_log_visit.config_browser_engine','VARCHAR(10) NULL',1),('version_log_visit.config_browser_name','VARCHAR(10) NULL',1),('version_log_visit.config_browser_version','VARCHAR(20) NULL',1),('version_log_visit.config_cookie','TINYINT(1) NULL',1),('version_log_visit.config_device_brand','VARCHAR( 100 ) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL1',1),('version_log_visit.config_device_model','VARCHAR( 100 ) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL1',1),('version_log_visit.config_device_type','TINYINT( 100 ) NULL DEFAULT NULL1',1),('version_log_visit.config_director','TINYINT(1) NULL',1),('version_log_visit.config_flash','TINYINT(1) NULL',1),('version_log_visit.config_gears','TINYINT(1) NULL',1),('version_log_visit.config_java','TINYINT(1) NULL',1),('version_log_visit.config_os','CHAR(3) NULL',1),('version_log_visit.config_os_version','VARCHAR( 100 ) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL',1),('version_log_visit.config_pdf','TINYINT(1) NULL',1),('version_log_visit.config_quicktime','TINYINT(1) NULL',1),('version_log_visit.config_realplayer','TINYINT(1) NULL',1),('version_log_visit.config_resolution','VARCHAR(18) NULL',1),('version_log_visit.config_silverlight','TINYINT(1) NULL',1),('version_log_visit.config_windowsmedia','TINYINT(1) NULL',1),('version_log_visit.location_browser_lang','VARCHAR(20) NULL',1),('version_log_visit.location_city','varchar(255) DEFAULT NULL1',1),('version_log_visit.location_country','CHAR(3) NULL1',1),('version_log_visit.location_latitude','decimal(9, 6) DEFAULT NULL1',1),('version_log_visit.location_longitude','decimal(9, 6) DEFAULT NULL1',1),('version_log_visit.location_region','char(3) DEFAULT NULL1',1),('version_log_visit.referer_keyword','VARCHAR(255) NULL1',1),('version_log_visit.referer_name','VARCHAR(70) NULL1',1),('version_log_visit.referer_type','TINYINT(1) UNSIGNED NULL1',1),('version_log_visit.referer_url','TEXT NULL',1),('version_log_visit.user_id','VARCHAR(200) NULL',1),('version_log_visit.visitor_count_visits','INT(11) UNSIGNED NOT NULL1',1),('version_log_visit.visitor_days_since_first','SMALLINT(5) UNSIGNED NULL1',1),('version_log_visit.visitor_days_since_last','SMALLINT(5) UNSIGNED NULL',1),('version_log_visit.visitor_days_since_order','SMALLINT(5) UNSIGNED NULL1',1),('version_log_visit.visitor_localtime','TIME NULL',1),('version_log_visit.visitor_returning','TINYINT(1) NULL1',1),('version_log_visit.visit_entry_idaction_name','INTEGER(10) UNSIGNED NULL',1),('version_log_visit.visit_entry_idaction_url','INTEGER(11) UNSIGNED NULL DEFAULT NULL',1),('version_log_visit.visit_exit_idaction_name','INTEGER(10) UNSIGNED NULL',1),('version_log_visit.visit_exit_idaction_url','INTEGER(10) UNSIGNED NULL DEFAULT 0',1),('version_log_visit.visit_first_action_time','DATETIME NOT NULL',1),('version_log_visit.visit_goal_buyer','TINYINT(1) NULL',1),('version_log_visit.visit_goal_converted','TINYINT(1) NULL',1),('version_log_visit.visit_total_actions','INT(11) UNSIGNED NULL',1),('version_log_visit.visit_total_events','INT(11) UNSIGNED NULL',1),('version_log_visit.visit_total_interactions','SMALLINT UNSIGNED DEFAULT 0',1),('version_log_visit.visit_total_searches','SMALLINT(5) UNSIGNED NULL',1),('version_log_visit.visit_total_time','INT(11) UNSIGNED NOT NULL',1),('version_Marketplace','3.13.5',1),('version_MobileMessaging','3.13.5',1),('version_Monolog','3.13.5',1),('version_Morpheus','3.13.5',1),('version_MultiSites','3.13.5',1),('version_Overlay','3.13.5',1),('version_PrivacyManager','3.13.5',1),('version_ProfessionalServices','3.13.5',1),('version_Proxy','3.13.5',1),('version_Referrers','3.13.5',1),('version_Resolution','3.13.5',1),('version_RssWidget','1.0',1),('version_ScheduledReports','3.13.5',1),('version_SegmentEditor','3.13.5',1),('version_SEO','3.13.5',1),('version_SitesManager','3.13.5',1),('version_Tour','3.13.5',1),('version_Transitions','3.13.5',1),('version_TwoFactorAuth','3.13.5',1),('version_UserCountry','3.13.5',1),('version_UserCountryMap','3.13.5',1),('version_UserId','3.13.5',1),('version_UserLanguage','3.13.5',1),('version_UsersManager','3.13.5',1),('version_VisitFrequency','3.13.5',1),('version_VisitorInterest','3.13.5',1),('version_VisitsSummary','3.13.5',1),('version_VisitTime','3.13.5',1),('version_WebsiteMeasurable','3.13.5',1),('version_Widgetize','3.13.5',1); /*!40000 ALTER TABLE `matomo_option` ENABLE KEYS */; UNLOCK TABLES; @@ -890,7 +890,7 @@ CREATE TABLE `matomo_site` ( LOCK TABLES `matomo_site` WRITE; /*!40000 ALTER TABLE `matomo_site` DISABLE KEYS */; -INSERT INTO `matomo_site` VALUES (1,'{{ getv "/site/name" "Islandora" }}','http://{{ getv "/site/host" "islandora.localhost" }}','2020-06-07 00:00:00',0,1,'','','{{ getv "/site/timezone" "America/Halifax" }}','USD',0,'','','','','website',0,'anonymous'); +INSERT INTO `matomo_site` VALUES (1,'{{ getv "/site/name" "Islandora" }}','http://{{ getv "/site/host" "islandora.isle-dc.localhost" }}','2020-06-07 00:00:00',0,1,'','','{{ getv "/site/timezone" "America/Halifax" }}','USD',0,'','','','','website',0,'anonymous'); /*!40000 ALTER TABLE `matomo_site` ENABLE KEYS */; UNLOCK TABLES; diff --git a/matomo/rootfs/etc/confd/templates/update-matomo-database.sql.tmpl b/matomo/rootfs/etc/confd/templates/update-matomo-database.sql.tmpl index ebcf4c80..3916d351 100644 --- a/matomo/rootfs/etc/confd/templates/update-matomo-database.sql.tmpl +++ b/matomo/rootfs/etc/confd/templates/update-matomo-database.sql.tmpl @@ -4,7 +4,7 @@ SET PASSWORD FOR '{{ getv "/db/user" "matomo" }}'@'%' = PASSWORD('{{ getv "/db/p -- Update site name, host, timezone. UPDATE matomo_site set name = '{{ getv "/site/name" "Islandora" }}', - main_url = '{{ getv "/site/host" "islandora.localhost" }}', + main_url = '{{ getv "/site/host" "islandora.isle-dc.localhost" }}', timezone = '{{ getv "/site/timezone" "America/Halifax" }}' WHERE idsite = 1; diff --git a/nginx/rootfs/usr/local/bin/composer b/nginx/rootfs/usr/local/bin/composer new file mode 100755 index 00000000..8cf03669 --- /dev/null +++ b/nginx/rootfs/usr/local/bin/composer @@ -0,0 +1,9 @@ +#!/usr/bin/env bash +set -e + +# Ensures composer does not run out of memory. Not run as nginx as there is no +# actual user with that account, and we want to make use of the cache in the +# root users $HOME directory. +# Takes precedence due to order of $PATH. Preferred to an alias as it will apply +# regardless of which shell is used or how it is started (login, interactive, etc) +php -d memory_limit=-1 /usr/bin/composer "${@}" diff --git a/sample.env b/sample.env deleted file mode 100644 index 8726422f..00000000 --- a/sample.env +++ /dev/null @@ -1,109 +0,0 @@ -# -# These are, at a bare minimum, the settings you need to configure to deploy -# between environments. -# - -DRUPAL_SITE_HOST="islandora.localhost" -DRUPAL_FCREPO_HOST="fcrepo.localhost" -MATOMO_SITE_TIMEZONE="America/Halifax" - -DRUPAL_DEFAULT_ACCOUNT_EMAIL="admin@example.org" -MATOMO_USER_EMAIL="admin@example.org" - -# -# These variables are dependent on those above -# - -DRUPAL_SITE_URL="http://${DRUPAL_SITE_HOST}" -DRUPAL_MATOMO_URL="${DRUPAL_SITE_URL}/matomo/" -DRUPAL_CANTALOUPE_URL="${DRUPAL_SITE_URL}/cantaloupe/iiif/2" -MATOMO_SITE_HOST="${DRUPAL_SITE_HOST}" - -# -# Passwords / Secrets -# - -ACTIVEMQ_WEB_ADMIN_PASSWORD="password" - -# Maybe one day we'll uses auth on activemq -#ACTIVEMQ_PASSWORD=password -#FCREPO_ACTIVEMQ_PASSWORD= - -MYSQL_ROOT_PASSWORD="password" -DRUPAL_DB_ROOT_PASSWORD="${MYSQL_ROOT_PASSWORD}" -FCREPO_DB_ROOT_PASSWORD="${MYSQL_ROOT_PASSWORD}" -GEMINI_DB_ROOT_PASSWORD="${MYSQL_ROOT_PASSWORD}" -MATOMO_DB_ROOT_PASSWORD="${MYSQL_ROOT_PASSWORD}" - -DRUPAL_DEFAULT_ACCOUNT_PASSWORD="password" -DRUPAL_DEFAULT_DB_PASSWORD="password" -FCREPO_DB_PASSWORD="password" -GEMINI_DB_PASSWORD="password" -KARAF_ADMIN_PASSWORD="password" -MATOMO_DB_PASSWORD="password" - -FCREPO_JWT_ADMIN_TOKEN="islandora" -GEMINI_JWT_ADMIN_TOKEN="${FCREPO_JWT_ADMIN_TOKEN}" -HOUDINI_JWT_ADMIN_TOKEN="${FCREPO_JWT_ADMIN_TOKEN}" -HOMARUS_JWT_ADMIN_TOKEN="${FCREPO_JWT_ADMIN_TOKEN}" -HYPERCUBE_JWT_ADMIN_TOKEN="${FCREPO_JWT_ADMIN_TOKEN}" -RECAST_JWT_ADMIN_TOKEN="${FCREPO_JWT_ADMIN_TOKEN}" -CRAYFITS_JWT_ADMIN_TOKEN="${FCREPO_JWT_ADMIN_TOKEN}" - -# This is an encoded form of "password". I don't know how best to handle this. -MATOMO_USER_PASSWORD="$2y$10$S38e7HPM9LI3aOIvcnRsfuMCm4ipNP572QsvbCK60upoHVJ61hMrS" - -# -# JWT Key Pair -# -DRUPAL_JWT_PRIVATE_KEY=$(cat << EOD ------BEGIN RSA PRIVATE KEY----- -MIIEpAIBAAKCAQEA6uK3nozywVaRCAB3FHdRZNHunSZvN/c31QimZAqQMGxj7JrG -h1LF8JRX+XAQ+CJcPD9r6xXjKSS1Gqa2Os2wARr/9abIwG5QeNsrJ8GMt3Z/WICn -NeaFAkUVviwKWcA61iFJWvTDAuI0hCaxArRKsk0BfFSMh+4u3JAdD9tUxUx6AAUX -UCdtPyluaBd53wuB0r9xRlPnDw6I9QHfKK80Xrrsu1PYATgrsy69stzCln3KlO5O -xc6O8OjMdjC2D2c3HmsO4CKPvvaVuaow/a9Pa3SNje4UXN+/1xUfQskxafP8CKVS -r8xxtwzSureiskb5/98moAiutpUtp15yyAm0rwIDAQABAoIBAA0PZh5OwAC4C4Bi -ZjyhFcmBUr8yL+Twvg3+WSIe5D2NCVFSmc9UbuUdmnaoIIlrf61p6Vo88VCMVfWR -Z3iFj0/AbJMAHxF0EM1nglLHlEdvM018ec+pbaPeq4LTeA/dfGgDmcyQ53b1lO30 -KMt5st2PIpIDMX0tZTWmXbdP/rqplqiQmdwH0gv8PzEG6Y2ZVLBf5viH5IvVRpg7 -9nDqLfe2W2ylFib+CtleX626xTUzGcJ0aqTRP1UkY4Jj5PI2/yVqttYPkvJtxZco -5/14AEMcu9FMBhADCSk/0y1TkKCGsi6/VNd78AB/RrZK32HfHCwTwxoXHQoaVKq6 -hNQfokECgYEA+keGdVJrXMDylamARQdDe/nNgljqZZhfkKKYCGeckNqjp3iqfmld -/tqCPVxAO2mIo8dNNfM2MMv6loEPx9F2REe8k9NbWFrUZ486fMHPeO7WnHt92JyU -DtfJJSZ1GdCki1PthmpmP8WdoF6VpLvr5AgwuYKAzkMNth+OV/dvgqkCgYEA8EEe -E3mKvePHV/PVsLt6TaqJcZEKKu6L9EgeDyzv3zz4+2zG8MVctyUFyfSl4EIC/oJy -X0T5Tj1l4A3mPwZOJfQOkXnr9TaPNff1zjNx12RhUZjFtJU5V+Wn1ldtzs+XwCFc -x5O/B8LWYgV4bOixNlc6tTRq/m8Txvtde9vPa5cCgYEA3OOdnxRD31QHheFYbRPx -Eo0xPNae4VWvGmb2SYywmQPuplMQHot+Qvy1L9SoeAc3alzvHytta3nLy2NS+yc5 -+x9ZJxrGJt/bUR8PHqarJu+ch/VR54ih/8uhImGjvknvv2wuWZC0d5pA+RYheofE -tLgp0MCGUATMKC4HokmmqCkCgYBEjoBTlFIn33CJw3WNyeGbefdgZb/eAlYDbfTN -5cfJDvAJZr/aAqdzR2hAecQ/mvaZw4V5dAgj8Fc6uRyjjVwNbngdwQm43km9X7VP -ktSAXw96Jjr8TbygPVNIUYhvBEPMOnjsJlfTkiB0thToFvpChF+nR37kfbPKCv5h -Epc8nwKBgQCdKyLi54Fm24nqEuZYbAxxGI9TVT7wJjoKGn64JWrXtX7xRltmJC3t -nLwNCojcbyG4kVB+Myzr2OEtFkO45j83GjrZ4O+jCuSj+AmCxEcc7xNA9cgu9usG -sQXdGmIIB0Cbk54OyHNdsZgZCXi9GTRF9uvYZKL9qktS+UZMJ1Xz/g== ------END RSA PRIVATE KEY----- -EOD -) - -DRUPAL_JWT_PUBLIC_KEY=$(cat << EOD ------BEGIN PUBLIC KEY----- -MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA6uK3nozywVaRCAB3FHdR -ZNHunSZvN/c31QimZAqQMGxj7JrGh1LF8JRX+XAQ+CJcPD9r6xXjKSS1Gqa2Os2w -ARr/9abIwG5QeNsrJ8GMt3Z/WICnNeaFAkUVviwKWcA61iFJWvTDAuI0hCaxArRK -sk0BfFSMh+4u3JAdD9tUxUx6AAUXUCdtPyluaBd53wuB0r9xRlPnDw6I9QHfKK80 -Xrrsu1PYATgrsy69stzCln3KlO5Oxc6O8OjMdjC2D2c3HmsO4CKPvvaVuaow/a9P -a3SNje4UXN+/1xUfQskxafP8CKVSr8xxtwzSureiskb5/98moAiutpUtp15yyAm0 -rwIDAQAB ------END PUBLIC KEY----- -EOD -) - -FCREPO_JWT_PUBLIC_KEY="$(JWT_PUBLIC_KEY)" -GEMINI_JWT_PUBLIC_KEY="${JWT_PUBLIC_KEY}" -HOUDINI_JWT_PUBLIC_KEY="${JWT_PUBLIC_KEY}" -HOMARUS_JWT_PUBLIC_KEY="${JWT_PUBLIC_KEY}" -HYPERCUBE_JWT_PUBLIC_KEY="${JWT_PUBLIC_KEY}" -RECAST_JWT_PUBLIC_KEY="${JWT_PUBLIC_KEY}" -CRAYFITS_JWT_PUBLIC_KEY="${JWT_PUBLIC_KEY}" \ No newline at end of file diff --git a/sandbox/.dockerignore b/sandbox/.dockerignore deleted file mode 100644 index b43bf86b..00000000 --- a/sandbox/.dockerignore +++ /dev/null @@ -1 +0,0 @@ -README.md diff --git a/sandbox/rootfs/etc/confd/conf.d/blazegraph.properties.toml b/sandbox/rootfs/etc/confd/conf.d/blazegraph.properties.toml deleted file mode 100644 index fcebdff1..00000000 --- a/sandbox/rootfs/etc/confd/conf.d/blazegraph.properties.toml +++ /dev/null @@ -1,7 +0,0 @@ -[template] -src = "blazegraph.properties.tmpl" -dest = "/var/run/islandora/blazegraph.properties" -uid = 0 -gid = 0 -mode = "0700" -keys = [ "/" ] diff --git a/sandbox/rootfs/etc/confd/conf.d/inference.nt.toml b/sandbox/rootfs/etc/confd/conf.d/inference.nt.toml deleted file mode 100644 index 966edce8..00000000 --- a/sandbox/rootfs/etc/confd/conf.d/inference.nt.toml +++ /dev/null @@ -1,7 +0,0 @@ -[template] -src = "inference.nt.tmpl" -dest = "/var/run/islandora/inference.nt" -uid = 0 -gid = 0 -mode = "0700" -keys = [ "/" ] diff --git a/sandbox/rootfs/etc/confd/conf.d/sandbox-setup.sh.toml b/sandbox/rootfs/etc/confd/conf.d/sandbox-setup.sh.toml deleted file mode 100644 index f677eb46..00000000 --- a/sandbox/rootfs/etc/confd/conf.d/sandbox-setup.sh.toml +++ /dev/null @@ -1,7 +0,0 @@ -[template] -src = "sandbox-setup.sh.tmpl" -dest = "/var/run/islandora/sandbox-setup.sh" -uid = 0 -gid = 0 -mode = "0700" -keys = [ "/" ] diff --git a/sandbox/rootfs/etc/confd/confd.toml b/sandbox/rootfs/etc/confd/confd.toml deleted file mode 100644 index 925d1a48..00000000 --- a/sandbox/rootfs/etc/confd/confd.toml +++ /dev/null @@ -1,6 +0,0 @@ -backend = "env" -confdir = "/etc/confd" -log-level = "debug" -interval = 600 -noop = false -prefix = "/drupal" diff --git a/sandbox/rootfs/etc/confd/templates/blazegraph.properties.tmpl b/sandbox/rootfs/etc/confd/templates/blazegraph.properties.tmpl deleted file mode 100644 index c8a6bba3..00000000 --- a/sandbox/rootfs/etc/confd/templates/blazegraph.properties.tmpl +++ /dev/null @@ -1,12 +0,0 @@ -com.bigdata.rdf.store.AbstractTripleStore.textIndex=false -com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.OwlAxioms -com.bigdata.rdf.sail.isolatableIndices=false -com.bigdata.rdf.store.AbstractTripleStore.justify=true -com.bigdata.rdf.sail.truthMaintenance=true -com.bigdata.rdf.sail.namespace=islandora -com.bigdata.rdf.store.AbstractTripleStore.quads=false -com.bigdata.namespace.islandora.lex.com.bigdata.btree.BTree.branchingFactor=400 -com.bigdata.journal.Journal.groupCommit=false -com.bigdata.namespace.islandora.spo.com.bigdata.btree.BTree.branchingFactor=1024 -com.bigdata.rdf.store.AbstractTripleStore.geoSpatial=false -com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false diff --git a/sandbox/rootfs/etc/confd/templates/sandbox-setup.sh.tmpl b/sandbox/rootfs/etc/confd/templates/sandbox-setup.sh.tmpl deleted file mode 100644 index 50c96033..00000000 --- a/sandbox/rootfs/etc/confd/templates/sandbox-setup.sh.tmpl +++ /dev/null @@ -1,176 +0,0 @@ -#!/usr/bin/with-contenv bash -set -ex - -function drush { - s6-setuidgid nginx php -d memory_limit=-1 /usr/local/bin/drush ${@} -} - -function main { - local drupal_url="{{ getv "/site/url" "http://islandora.localhost" }}" - local broker_host="{{ getv "/broker/host" "activemq" }}" - local broker_port="{{ getv "/broker/port" "61613" }}" - local broker_url="tcp://${broker_host}:${broker_port}" - local gemini_host="{{ getv "/gemini/host" "gemini" }}" - local gemini_port="{{ getv "/gemini/port" "8000" }}" - local gemini_url="http://${gemini_host}:${gemini_port}" - local solr_host="{{ getv "/solr/host" "solr" }}" - local solr_port="{{ getv "/solr/port" "8983" }}" - local matamo_url="{{ getv "/matomo/url" "http://islandora.localhost/matomo/" }}" - local cantaloupe_url="{{ getv "/cantaloupe/url" "http://islandora.localhost/cantaloupe/iiif/2" }}" - local triplestore_host="{{ getv "/triplestore/host" "blazegraph" }}" - local triplestore_port="{{ getv "/triplestore/port" "80" }}" - local triplestore_url="http://${triplestore_host}:${triplestore_port}/bigdata" - local fcrepo_host="{{ getv "/fcrepo/host" "fcrepo.localhost" }}" - local fcrepo_port="{{ getv "/fcrepo/host" "80" }}" - local fcrepo_url= - - # Indexing fails if port 80 is given explicitly. - if [[ "${fcrepo_port}" == "80" ]]; then - fcrepo_url="http://${fcrepo_host}/fcrepo/rest/" - else - fcrepo_url="http://${fcrepo_host}:${fcrepo_port}/fcrepo/rest/" - fi - - - # Hackity hack sack. - if ! grep flysystem /var/www/drupal/web/sites/default/settings.php &> /dev/null; then - cat <<-EOT >> /var/www/drupal/web/sites/default/settings.php - -\$settings['flysystem'] = [ - 'fedora' => [ - 'driver' => 'fedora', - 'config' => [ - 'root' => '${fcrepo_url}', - ], - ], -]; -EOT - fi - - # Ensure JWT is setup before any module installations run. - drush en --uri "${drupal_url}" --no-interaction --yes jwt - drush config-import -y --partial --source=/opt/islandora/configs/jwt - - # Ensure Islandora settings are correct, before install hooks run. - drush en --uri "${drupal_url}" --no-interaction --yes islandora - drush -y cset --input-format=yaml islandora.settings broker_url "${broker_url}" - drush -y cset --input-format=yaml islandora.settings gemini_url "${gemini_url}" - - # Need access to Solr before we can actually import the right config. - if timeout 300 wait-for-open-port.sh "${solr_host}" "${solr_port}" ; then - echo "Solr Found" - else - echo "Could not connect to Solr" - exit 1 - fi - - # Need access to Solr before we can actually import the right config. - if timeout 300 wait-for-open-port.sh ${fcrepo_host} ${fcrepo_port}; then - echo "Fcrepo Found" - else - echo "Could not connect to Fcrepo" - exit 1 - fi - - # Need access to activemq so we can publish the creation of before we can actually import the right config. - if timeout 300 wait-for-open-port.sh ${broker_host} ${broker_port}; then - echo "Broker Found" - else - echo "Could not connect to Broker" - exit 1 - fi - - # Need access to gemini before importing features. - if timeout 300 wait-for-open-port.sh "${gemini_host}" "${gemini_port}"; then - echo "Gemini Found" - else - echo "Could not connect to Gemini" - exit 1 - fi - - if timeout 300 wait-for-open-port.sh "${triplestore_host}" "${triplestore_port}"; then - echo "Triplestore Found" - else - echo "Could not connect to Triplestore" - exit 1 - fi - - # Setup namespace / inference. - curl -X POST -H "Content-type: text/plain" --data-binary @/var/run/islandora/blazegraph.properties "${triplestore_url}/namespace" - curl -X POST -H "Content-type: text/plain" --data-binary @/var/run/islandora/inference.nt "${triplestore_url}/namespace/islandora/sparql" - - # From: islandora-playbook/inventory/vagrant/group_vars/webserver/drupal.yml - drush en --uri "${drupal_url}" --no-interaction --yes \ - admin_toolbar \ - basic_auth \ - content_browser \ - controlled_access_terms_defaults \ - devel \ - facets \ - islandora_breadcrumbs \ - islandora_defaults \ - islandora_fits \ - islandora_iiif \ - islandora_oaipmh \ - islandora_search \ - matomo \ - pdf \ - rdf \ - responsive_image \ - rest \ - restui \ - search_api_solr \ - search_api_solr_defaults \ - serialization \ - simpletest \ - syslog \ - transliterate_filenames - - # search_api.index.default_solr_index? - drush -y cset search_api.server.default_solr_server backend_config.connector_config.host "${solr_host}" - drush -y cset search_api.server.default_solr_server backend_config.connector_config.port "${solr_port}" - - # From: islandora-playbook/roles/external/Islandora-Devops.drupal-openseadragon/tasks/config.yml - drush -y cset --input-format=yaml openseadragon.settings iiif_server "${cantaloupe_url}" - - # From: islandora-playbook/roles/internal/webserver-app/tasks/drupal.yml - drush -y pm-uninstall search - drush -y "then" -y carapace - drush -y config-set system.theme default carapace - drush -y config-set matomo.settings site_id 1 - drush -y config-set matomo.settings url_http "${matamo_url}" - drush fim --uri "${drupal_url}" --no-interaction --yes islandora_core_feature,controlled_access_terms_defaults,islandora_defaults,islandora_search - - # Fix what the features overides. - drush -y cset search_api.server.default_solr_server backend_config.connector_config.host "${solr_host}" - drush -y cset search_api.server.default_solr_server backend_config.connector_config.port "${solr_port}" - - # Export the configuration into shared volume. - drush -y solr-gsc default_solr_server /tmp/solr_config.zip 7.1 - mkdir -p /opt/solr/server/solr/ISLANDORA/conf || true - mkdir -p /opt/solr/server/solr/ISLANDORA/data || true - unzip -o /tmp/solr_config.zip -d /opt/solr/server/solr/ISLANDORA/conf - # The uid:gid "100:1000" is "solr:solr" inside of the solr container. - chown -R 100:1000 /opt/solr/server/solr/ISLANDORA - curl -s "http://${solr_host}:${solr_port}/solr/admin/cores?action=CREATE&name=ISLANDORA&instanceDir=ISLANDORA&config=solrconfig.xml&dataDir=data" &> /dev/null - - # From: islandora-playbook/post-install.yml - drush -y urol fedoraadmin admin - drush -y cset --input-format=yaml jsonld.settings remove_jsonld_format true - drush -y cset --input-format=yaml islandora.settings broker_url "${broker_url}" - drush -y cset --input-format=yaml islandora.settings gemini_url "${gemini_url}" - drush -y cset --input-format=yaml islandora.settings gemini_pseudo_bundles.0 "islandora_object:node" - drush -y cset --input-format=yaml islandora.settings gemini_pseudo_bundles.1 "image:media" - drush -y cset --input-format=yaml islandora.settings gemini_pseudo_bundles.2 "file:media" - drush -y cset --input-format=yaml islandora.settings gemini_pseudo_bundles.3 "audio:media" - drush -y cset --input-format=yaml islandora.settings gemini_pseudo_bundles.4 "video:media" - drush -y cset --input-format=yaml media.settings standalone_url true - drush -y cset --input-format=yaml islandora_iiif.settings iiif_server "${cantaloupe_url}" - drush -y cset --input-format=yaml openseadragon.settings manifest_view iiif_manifest - drush -y --uri "${drupal_url}" --userid=1 mim --group=islandora - s6-setuidgid nginx mkdir -p /var/www/drupal/web/simpletest /var/www/drupal/web/sites/simpletest - - # Cache clear. - drush -y cr -} -main diff --git a/sandbox/rootfs/etc/cont-init.d/04-sandbox-setup.sh b/sandbox/rootfs/etc/cont-init.d/04-sandbox-setup.sh deleted file mode 100644 index f220d927..00000000 --- a/sandbox/rootfs/etc/cont-init.d/04-sandbox-setup.sh +++ /dev/null @@ -1,3 +0,0 @@ -#!/usr/bin/with-contenv bash -set -e -/var/run/islandora/sandbox-setup.sh