Skip to content

Commit

Permalink
Merge branch 'main' into st-backups
Browse files Browse the repository at this point in the history
  • Loading branch information
joshdentremont authored Jun 12, 2024
2 parents ed7e306 + e21fa44 commit 818d31e
Show file tree
Hide file tree
Showing 8 changed files with 265 additions and 28 deletions.
2 changes: 1 addition & 1 deletion docs/contributing/testing-a-pull-request.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
If you are testing a pull request, or for other reasons need to replace the
"official" code with code that's in a PR, or a different branch or fork, this
page offers three methods: using Composer Patches, using Composer to
require the branch and/or fork or installing source repositories with Composer.
require the branch and/or fork, or installing source repositories with Composer.

This documentation applies to Drupal modules, themes, and recipes, or any
other project that is managed by Composer.
Expand Down
25 changes: 4 additions & 21 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,13 @@

## This is Islandora

[Islandora](https://islandora.ca) is an open-source framework that provides the necessary tools to use a [Drupal](https://www.drupal.org) website as a fully-functional Digital Assets Management System.
[Islandora](https://islandora.ca) is an open-source framework that provides the necessary tools to use a [Drupal](https://www.drupal.org) website as a fully-functional Digital Assets Management System. See [Quickstart](installation/quickstart.md) to get started.

Islandora:

[//]: # (We should probably replace as many of the links in this section as possible with links to within this documentation, and make it clear which are internal links and which link out.)
- **Is native Drupal** - With Islandora, you can create preservation-ready digital resources using Drupal nodes, media, files, and taxonomy terms. A site with a ready-made suite of customized Drupal resource types and configurations that illustrate Islandora's capabilities is available as the [Islandora Starter Site module](https://github.com/Islandora/islandora-starter-site/).
- **Is native Drupal** - With Islandora, you can create preservation-ready digital resources using Drupal nodes, media, files, and taxonomy terms.
- **Provides a sensible starting place** - No two Islandora sites are the same, but we provide a starting point for Drupal - the [Islandora Starter Site](https://github.com/Islandora/islandora-starter-site/) - as well as several methods of [installation](installation/quickstart.md) that set up a working suite of services external to Drupal and show off Islandora's capabilities.
- **Integrates with Fedora** - Drupal resources can be stored in Lyrasis's [Fedora Repository](https://wiki.duraspace.org/display/FF/Fedora+Repository+Home) (version 5.0 or greater) as binaries (files) and RDF metadata.
- **Uses microservices** - Islandora provides an architecture for messaging and integration with any number of microservices, that provide services outside of the Drupal framework. Islandora's [Crayfish](https://github.com/Islandora/crayfish) suite of microservices provides functionality for synchronizing resources into Fedora and for automatically generating helper files, called derivatives.
- **Can handle messages at scale** - Islandora created [Alpaca](https://github.com/Islandora/Alpaca), an integration middleware based on Apache Camel, to handle messaging and queueing at an enterprise scale. To the user, this means large batch uploads can be processed gracefully.
Expand All @@ -20,28 +21,10 @@ Islandora:
- **Offers flexibility** - As Islandora content is Drupal content, migrations and batch editing can be done through Drupal's built-in migrate framework and vocabularies can be created using Drupal taxonomies. Contributed Drupal modules such as [Solr Search API](https://www.drupal.org/project/search_api_solr) enable in-site search, and [Matomo Analytics](https://www.drupal.org/project/matomo) provides usage metrics for site analytics.
- **Is a community** - A [dedicated, active community of users and developers](https://groups.google.com/forum/#!forum/islandora) is working to push new features, collaborate on improvements, design custom solutions, and create extended functionality. Some of these for Islandora 8 take the form of [Recipes](https://github.com/Islandora-Labs/Islandora-Cookbook).

## Try Islandora

### Online Sandbox

Try Islandora without installing anything at [sandbox.islandora.ca](https://sandbox.islandora.ca/).

Login credentials for the sandbox can be found [here](https://github.com/Islandora/documentation/wiki/Sandbox.Islandora.ca-online-credentials).

Anyone can log in to this sandbox as an administrator and explore the interface! However, this site is refreshed every night so your changes will not be permanent. This site uses the Islandora Starter Site. This is not the only way that Islandora can be made to work! This sandbox includes some sample content and configuration (such as views and blocks) to increase its usefulness as a sandbox. [Learn more about the Islandora Starter Site](https://github.com/Islandora-Devops/islandora-starter-site).

### Ansible Playbook

Islandora can be installed via an an [Ansible Playbook](https://github.com/Islandora-Devops/islandora-playbook), a machine-operable set of instructions which provisions the full Islandora stack. It can be used to create a locally installed Islandora (requiring your computer to have Vagrant and Virtualbox) or can be used to provision a remote Linux server. The provisioning process involves many steps where software is downloaded and installed, so it can take a while. There is an option to get a basic ("standard") site, or to install a suite of demo configurations known as the Demo Install Profile. See [Installation - Ansible Playbook](installation/playbook) for more details.

### Docker

Islandora sites can also be created using Docker. This can be done using the [ISLE-DC project](https://github.com/Islandora-Devops/isle-dc), which launches the Islandora Docker images created by [Isle Buildkit](https://github.com/Islandora-Devops/isle-buildkit). Like in the Ansible Playbook, there is an option to use a pre-built demo site, or build it completely from scratch. (There is also an option to build the demo site on command, which takes a bit more time, or to use ISLE-DC to build up the environment to support a Drupal site that you have already exported.) See [Installation - Docker ISLE](installation/docker-introduction) for more details

## Join the Community

The [Islandora community](https://www.islandora.ca/community) is an active group of users, managers, librarians, documenters, and developers from galleries, libraries, archives, museums, and other institutions worldwide. We welcome discussion and contribution through various mailing lists, channels, interest groups, and calls. The Islandora community operates under the [Islandora Code Of Conduct](https://www.islandora.ca/code-of-conduct). See our Contributing Guidelines for more information.


!!! note "Documentation for previous versions"
Documentation for Islandora 6 and 7 is on the [Lyrasis documentation wiki](https://wiki.lyrasis.org/display/ISLANDORA/Start).
Documentation for Islandora Legacy (6 and 7) is on the [Lyrasis documentation wiki](https://wiki.lyrasis.org/display/ISLANDORA/Start).
48 changes: 48 additions & 0 deletions docs/installation/docker/site-template/docker-modifications.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Docker Modifications

ISLE Site Template provides you with a `docker-compose.yml` file that allows you to get an Islandora site running quickly, but it makes some assumptions about how the site will run, and which containers you will use. Once you have your site running you may want to make some modifications to the default setup that the Site Template uses.

## Adding / Editing Environment Variables

Islandora Buildkit provides several environment variables that can be modified when creating containers.

Please see the README for the different buildkit images to see what is available:

- [ActiveMQ](https://github.com/Islandora-Devops/isle-buildkit/tree/main/activemq)
- [Alpaca](https://github.com/Islandora-Devops/isle-buildkit/tree/main/alpaca)
- [Blazegraph](https://github.com/Islandora-Devops/isle-buildkit/tree/main/blazegraph)
- [Cantaloupe](https://github.com/Islandora-Devops/isle-buildkit/tree/main/cantaloupe)
- [Code Server](https://github.com/Islandora-Devops/isle-buildkit/tree/main/code-server)
- [Crayfits](https://github.com/Islandora-Devops/isle-buildkit/tree/main/crayfits)
- [Drupal](https://github.com/Islandora-Devops/isle-buildkit/tree/main/drupal)
- [Fedora](https://github.com/Islandora-Devops/isle-buildkit/tree/main/fcrepo6)
- [Fits](https://github.com/Islandora-Devops/isle-buildkit/tree/main/fits)
- [Homarus](https://github.com/Islandora-Devops/isle-buildkit/tree/main/homarus)
- [Houdini](https://github.com/Islandora-Devops/isle-buildkit/tree/main/houdini)
- [Hypercube](https://github.com/Islandora-Devops/isle-buildkit/tree/main/hypercube)
- [MariaDB](https://github.com/Islandora-Devops/isle-buildkit/tree/main/mariadb)
- [Milliner](https://github.com/Islandora-Devops/isle-buildkit/tree/main/milliner)
- [Solr](https://github.com/Islandora-Devops/isle-buildkit/tree/main/solr)

You can add these environment variables to your docker-compose.yml in order to change their values. For example, if you want to increase the PHP memory limit in your production Drupal container, you can do so like this:

```
drupal-prod:
<<: [*prod, *drupal]
Environment:
PHP_MEMORY_LIMIT: 1G
```


## Removing Services

You may not want to use all the images that are included in the Site Template’s `docker-compose.yml`. You can remove containers by deleting their sections in the docker-compose.yml file.

For example, to remove Fedora, you would delete the services called fcrepo-dev and fcrepo-prod.

Depending on the container you are removing, you may need to delete references to it as well. For example, some containers are referenced by others in the `depends_on` field. You will need to also delete these references, so if you delete the `fedora-dev` service, you will need to remove the rule that `traefik-dev` depends on it.

If you are removing a container which is referenced by Drupal, ensure that you update Drupal as well (e.g. if removing Fedora, ensure your Media's files are not writing to the Fedora filesystem).

After doing `docker compose down`, run `docker compose up -d --remove-orphans` to remove the containers you removed from the docker-compose.yml file.

65 changes: 65 additions & 0 deletions docs/installation/docker/site-template/setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Initial Setup

Instructions for creating your site can be found in the [project's README file](https://github.com/Islandora-Devops/isle-site-template)

This page lists a few things to note about the process, but is not meant to be a replacement for the instructions in the README file.

## Custom Drupal Image

ISLE Site Template uses a custom Drupal image that you build on top of the provided Islandora Drupal image. This means you will not be running the islandora/drupal image directly, but the provided Dockerfile will use it to build your image.

!!! Note "Note for those coming from ISLE-DC"

In ISLE-DC, we only use a custom image in production, but in the ISLE Site Template, we use it for both.

Building your custom Drupal image is done by running

`docker compose --profile dev build` for your development image

or

`docker compose --profile prod build` for your production image

This builds the docker image based on the Dockerfile in the `drupal` directory, which uses your composer files to pull the Drupal modules it needs into the image. Because the Dockerfile and composer files are part of the git repository, you can build your Drupal image locally, or on your production server.

This documentation assumes you will be building your production image on the production server. If you do it this way, it is not necessary to push your image to a container registry. Instead you just pull your git repository anytime you make changes to your composer files, and run `docker compose build` again.

!!! Note "Using a Container Registry"
If you want to build your production images somewhere other than on your production server, you can do so. The .env file allows you to set your image repository URL, which will allow you to push / pull your Drupal image to / from your container registry. If you do this, you can then run `docker compose pull` instead of `docker compose build` on your production server, to pull the already built image to that server.

For more information please see the documentation on [docker compose pull](https://docs.docker.com/reference/cli/docker/compose/pull/) and [docker compose build](https://docs.docker.com/reference/cli/docker/compose/build/)

## Adding a Staging Site

The process for setting up a staging site is the same as production, but you will need to change the `DOMAIN` variable in your `.env` file on your staging server, to contain the URL for your staging site.

!!! Note
By default, the `.env` file is stored in the git repository for your site, which means there is only support for one URL. If you are adding a staging site, you may wish to modify this so that you do not accidentally push your staging URL to your git repository.

!!! note "Restricting Access to Staging Servers"

Using letsencrypt to generate your certs requires port 80 to be accessible on your server. If you would like to keep your site private by limiting access to certain IP addresses, you can still firewall port 443, but you will have to leave port 80 open. If you need to firewall port 80 as well, you will have to either use your own certs or look into another method of generating certs.

## Adding Demo Content

If you are spinning up a new site for testing, you can add some demo content to your site by running
```
[ -d "islandora_workbench" ] || (git clone https://github.com/mjordan/islandora_workbench)
cd islandora_workbench ; cd islandora_demo_objects || git clone https://github.com/Islandora-Devops/islandora_demo_objects.git
$(SED_DASH_I) 's#^host.*#host: $(SITE)/#g' islandora_workbench/islandora_demo_objects/create_islandora_objects.yml
$(SED_DASH_I) 's/^password.*/password: "$(shell cat secrets/DRUPAL_DEFAULT_ACCOUNT_PASSWORD | sed s#/#\\\\\\\\/#g)"/g' islandora_workbench/islandora_demo_objects/create_islandora_objects.yml
cd islandora_workbench && docker build -t workbench-docker .
cd islandora_workbench && docker run -it --rm --network="host" -v $(QUOTED_CURDIR)/islandora_workbench:/workbench --name my-running-workbench workbench-docker bash -lc "./workbench --config /workbench/islandora_demo_objects/create_islandora_objects.yml"
`docker compose exec -T drupal-dev with-contenv bash -lc 'drush --root /var/www/drupal/web -l ${DRUPAL_DEFAULT_SITE_URL} search-api-reindex'
docker compose exec -T drupal-dev with-contenv bash -lc 'drush --root /var/www/drupal/web -l ${DRUPAL_DEFAULT_SITE_URL} search-api-index'
```

## Custom Themes & Modules

You may wish to copy themes and modules into your project directly, instead of using Composer to manage them. For example, if you are creating your own theme instead of using a contributed one.

Isle Site Template provides directories at `drupal/rootfs/var/www/drupal/web/modules/custom` and `drupal/rootfs/var/www/drupal/web/themes/custom` for you to add your custom themes and modules.

These directories are mounted in development, so any changes to them will be shared between your host machine and your Drupal container.

In production, these themes and modules will be included when the Drupal image is built.
105 changes: 105 additions & 0 deletions docs/installation/docker/site-template/updating.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Updating

The following sections describe how to keep your Islandora install up to date with current versions of Drupal modules and Docker images.

## Updating Isle Buildkit (Islandora Docker Images)

Updating to a new version of Isle Buildkit is done by setting the ISLANDORA_TAG variable in your .env file. Once you have updated the .env file, you need to pull the new Islandora images, and rebuild your custom Drupal image from the specified Drupal image.

For example, to upgrade from Buildkit 2.0.0 to 3.0.0 you would do the following steps:

1. Change your .env file to say ISLANDORA_TAG=3.0.0

2. Stop your Docker containers

`docker compose --profile dev down`

3. Pull the new Docker images (except the Drupal image)

`docker compose --profile dev pull --ignore-buildable --ignore-pull-failures`

4. Build your custom Drupal image

`docker compose --profile dev build`

5. Start your containers from the new images

`docker compose --profile dev up -d`

Once you have upgraded your images, you may need to perform extra steps for Solr and Mariadb, depending on whether these have new versions in the new images.

!!! note "Production Sites"

Upgrading a production site works the same way, just replace `dev` with `prod` in the above instructions.

### Solr

You may need to regenerate your Solr configs if Solr has been updated to a new version, or when the Search API Solr Drupal module has been updated. If you visit /admin/config/search/search-api/server/default_solr_server on your Islandora site it will tell you if the configs need to be updated.

To generate new configs perform the following steps:

1. Remove existing solr configs

`docker compose exec -T solr-dev with-contenv bash -lc 'rm -r server/solr/default/*'`

2. Recreate solr configs for new solr versions

`docker compose exec -T drupal-dev with-contenv bash -lc "for_all_sites create_solr_core_with_default_config"`

3. Reindex Solr through the admin page or via Drush

### MariaDB

After updating MariaDB, you may need to run [mariadb-upgrade](https://mariadb.com/kb/en/mariadb-upgrade/) inside your MariaDB container, to update your system tables. This should be safe to run any time, but it is a good idea to back up your database first, just in case.

You can run this with

`docker compose exec mariadb-dev mariadb-upgrade`

## Updating Traefik

Traefik is not updated with the other buildkit images. It is recommended that you periodically update Traefik by changing the image version in your docker-compose.yml file to the [current version in use by Isle Site Template](https://github.com/Islandora-Devops/isle-site-template/blob/main/docker-compose.yml)

## Updating Drupal Modules

Drupal updates are performed through composer on your development site. Once the modules have been added/removed/updated, your `composer.json` and `composer.lock` files can be checked into your git repository and you can rebuild your production Drupal container with the new files.

### Development

Composer commands need to [run in your Drupal container](/documentation/installation/docker/site-template/containers.md). For example:

​​`docker compose exec drupal-dev composer update -W`

Or

`docker compose exec drupal-dev composer require 'drupal/islandora:^2.11'`

Running database updates is also done in the container like this:

`docker compose exec drupal-dev drush updb`

!!! note

You should backup your database before running database updates

If you are enabling or uninstalling modules, you will also need to export your Drupal configuration.

Once you have finished your composer changes you can commit and push your repository with the new `composer.json` and `composer.lock` changes.

### Production

First you will `git pull` to get the `composer.json` and `composer.lock` changes you made in development.

Next you build your Drupal image again, which will install the modules specified in those files:

`docker compose --profile prod build`

Then you stop and start your containers to get the new image:

`docker compose --profile prod down`

`docker compose --profile prod up -d`

And if necessary, run database updates:

`docker compose exec drupal-prod drush updb`
Loading

0 comments on commit 818d31e

Please sign in to comment.