Skip to content
/ dock_aem Public
forked from ciechanowiec/dock_aem

Program for setting up and managing local Adobe Experience Manager (AEM) development environment via Docker

License

View license
0 stars 1 fork Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

bmorris84/dock_aem

BranchesTags
 
 

Repository files navigation

Dock AEM

Table of Contents

1. Overview

Dock AEM is a program for setting up and managing local Adobe Experience Manager (AEM) development environment via Docker. It provides ready-to-go tools for swift bootstrapping and controlling interconnected AEM Author, AEM Publish, AEM Dispatcher, Varnish Cache, Apache Solr, Apache ZooKeeper and SMTP instances. The program supports both on-premise (AEM 6.5) and cloud (AEMaaCS) versions of Adobe Experience Manager.

2. Usage

  1. Dock AEM provides a docker-compose.yml file and related artifacts, enabling developers to:

    1. build various AEM and AEM-related images

    2. run, start, and stop AEM and AEM-related containers

  2. Supported images include:

    1. AEM Author 6.5

    2. AEM Author as a Cloud Service

    3. AEM Publish 6.5

    4. AEM Publish as a Cloud Service

    5. AEM Dispatcher for AMD CPU architecture

    6. AEM Dispatcher for ARM CPU architecture

    7. Varnish Cache

    8. Apache Solr

    9. Apache ZooKeeper

    10. Fake SMTP Server

  3. Apache Solr and Apache ZooKeeper are configured as a cluster of containers: 3x for Apache Solr and 3x for Apache ZooKeeper. However, all Apache Solr containers are based on the same solr image and all Apache ZooKeeper are based on the same zookeeper image.

  4. It is recommended that a machine, either virtual or real, on which Dock AEM images are built and containers are used, has at least 12 GB RAM allocated. Otherwise, the respective Java processes started by the artifacts might be unexpectedly terminated.

2.1. Images Build

In order to build the Dock AEM images follow the steps below.

  1. Clone the Dock AEM source code repository:

    git clone https://github.com/ciechanowiec/dock_aem

  2. Remove all preceding Dock AEM containers, images, volumes and networks:

    artifactsToRemove="aem-base aem-author-65 aem-author-cloud aem-publish-65 aem-publish-cloud dispatcher-amd dispatcher-arm varnish solr1 solr2 solr3 zookeeper1 zookeeper2 zookeeper3 fake-smtp-server"

docker container stop --time 30 "$artifactsToRemove"

docker container remove --force "$artifactsToRemove"

docker image remove --force "$artifactsToRemove"

docker volume remove --force aem-author-65-data aem-author-cloud-data aem-publish-65-data aem-publish-cloud-data

docker network rm aem-network

  3. license.properties

    Put a license.properties file into src/commons directory. It will be used during the build for both AEM 6.5 and AEMaaCS.

  4. src/quickstart

    Put respective AEM QuickStart Jars into src/quickstart/65 (AEM 6.5) and src/quickstart/cloud (AEMaaCS) directories in the Dock AEM source code repository (files inside those directories are git-ignored, so there is no relevant risk of exposing them in the source control). Give both those Jars the following name: aem-quickstart.jar.

  5. src/packages

    If there are any packages that should be preinstalled on AEM Author and AEM Publish, put them respectively into src/packages/65 (AEM 6.5) or src/packages/cloud (AEMaaCS) directories in the Dock AEM source code repository (files inside those directories, just like the files inside src/quickstart subdirectories, are git-ignored, so there is no relevant risk of exposing them in the source control).

    For instance, into src/packages/65 the following packages can be put:

    1. aem-service-pkg-6.5.16-NPR-40551-B002.zip (AEM Service Pack 6.5.16)

    2. adobe-aemfd-linux-pkg-6.0.914.zip (AEM Forms Addon related to AEM 6.5.16)

      Note
      		 Regardless of the host operating system on which the images are built (Windows/macOS/Linux), in case of AEM Forms addon a Linux version of the addon should be chosen, since the Dock AEM images are based on Linux.

    3. content.zip (preinstalled content)

    The src/packages/cloud directory, in turn, can be populated with these packages:

    1. aem-forms-addon-2023.10.25.00-231100.far (AEM Forms Addon related to AEMaaCS)

    2. content.zip (preinstalled content)

  6. src/commons/crypto

    An src/commons/crypto directory contains a data directory with default hmac and master keys. Those keys are used by com.adobe.granite.crypto.file bundle for secrets encoding and default versions of the keys are provided for deterministic and predictable behavior. If it is necessary to use different keys, replace the existing ones with the new ones. However, note that those keys aren’t git-ignored, so be careful and don’t git commit sensitive secrets.

  7. During the images build, AEM Author and AEM Publish instances are started. Among others, this is done in order to initialize persistence layer and reduce the amount of time required for the first clean start of containers based on the built images. The start of AEM Author and AEM Publish is controlled by src/2_aem/aem-starter.sh script. That script also shutdowns the instances once the initialization is finished.

    The initialization is assumed to be finished when all required bundles have started and are active. The amount of required bundles might differ depending on the exact set of preinstalled packages from src/packages, on run mode choice (author/publish, samplecontent/nosamplecontent) and the subsequent number of an instance start.

    Table 1. Expected bundles status for selected AEM setups

    First Start

    Second and Subsequent Starts

    AEM 6.5
    - author/publish
    - nosamplecontent
    - aem-service-pkg-6.5.15.0.zip
    - adobe-aemfd-linux-pkg-6.0.856.zip

    717

    723

    AEM 6.5
    - author/publish
    - nosamplecontent
    - aem-service-pkg-6.5.16-NPR-40551-B002.zip
    - adobe-aemfd-linux-pkg-6.0.914.zip

    719

    725

    AEMaaCS
    - author
    - nosamplecontent
    - aem-sdk-quickstart-2024.5.16461.20240524T172309Z-240500.jar
    - aem-forms-addon-2024.07.05.00-240600.far

    683

    683

    AEMaaCS
    - publish
    - nosamplecontent
    - aem-sdk-quickstart-2024.5.16461.20240524T172309Z-240500.jar
    - aem-forms-addon-2024.07.05.00-240600.far

    677

    677
    Warning
    		 Due to the bundles readiness check described above, the exact expected amount of active bundles for the specified AEM setups must be configured in the docker-compose.yml file.

  8. Go to src directory in the Dock AEM source code repository. Inside that directory, run the build defined in the docker-compose.yml file. As a result, ten images will be built: aem-base, aem-author-65, aem-author-cloud, aem-publish-65, aem-publish-cloud, dispatcher-amd, dispatcher-arm, varnish, solr and zookeeper. For fully fledged AEM instances the build might take ~40 minutes in total, depending on hardware capabilities:

    cd "$pathToDockAEMSourceCodeRepository/src"

docker compose build
    Note

    1. In case of certain system setups, the command above should be hyphenated: docker-compose.

    2. Add --progress=plain to the above command to see unfolded build output:

      docker compose --progress=plain build

    3. It is possible to build independently only some specific image defined in the docker-compose.yml file by providing that image name to the build command in the following way:

      docker compose build aem-author-cloud

  9. Verify the build and make sure that all Dock AEM images are available on host:

    ❯ docker images
REPOSITORY          TAG       IMAGE ID       CREATED      SIZE
aem-base            latest    bb188da069d9   1 hour ago   1.31GB
aem-author-65       latest    fc205742190d   1 hour ago   10.1GB
aem-author-cloud    latest    25eb57cdf070   1 hour ago   4.3GB
aem-publish-65      latest    3a48c564925e   1 hour ago   10.1GB
aem-publish-cloud   latest    aeb9052d3096   1 hour ago   4.2GB
dispatcher-amd      latest    a6724be27ef7   1 hour ago   1.28GB
dispatcher-arm      latest    ff8962803d23   1 hour ago   1.28GB
varnish             latest    01f2cda11b61   1 hour ago   1.23GB
solr                latest    e3d19f01025e   1 hour ago   1.63GB
zookeeper           latest    f8713ce13cc2   1 hour ago   1.39GB

2.2. Container Run, Start & Stop

  1. Containers with AEM Author and AEM Publish instances can be run, started and stopped independently. Containers with other applications for their correct running might, in turn, require some other containers to be active. Such dependencies are either specified in the docker-compose.yml file in the depends_on directive (e.g. dependency of Apache Solr on Apache ZooKeeper) or is implied by the nature of the application (e.g. dependency of AEM Dispatcher on AEM Publish).

  2. In order to run a given container for the first time, use the docker-compose.yml file located inside src directory in the Dock AEM source code repository. As a parameter of the command provide the name of the respective service defined in the docker-compose.yml file:

    docker compose up [service name] --detach

    For instance, the command for running AEM Author as a Cloud Service would look the following way:

    docker compose up aem-author-cloud --detach

    It is also possible to pass more than one service name as parameters of the run command:

    docker compose up aem-author-cloud aem-publish-cloud --detach

    In case of containers with dependencies it is sufficient just to run the container that has dependencies, because the dependency-containers will be activated automatically. For example, Apache Solr instances depend on Apache ZooKeeper instances. Therefore, running Apache Solr instances will automatically activate all related Apache ZooKeeper containers:

    docker compose up solr1 solr2 solr3 --detach
    Note
    		 In case of certain system setups, the commands above should be hyphenated: docker-compose.

  3. After the first run a given container can be started and stopped via regular Docker commands by providing respective container names, like in these examples:

    # AEM Author as a Cloud Service:
docker start aem-author-cloud
docker stop aem-author-cloud

# AEM Publish as a Cloud Service:
docker start aem-publish-cloud
docker stop aem-publish-cloud

# Apache Solr:
docker start solr1 solr2 solr3
docker stop solr1 solr2 solr3

2.3. AEM Dispatcher & Varnish Cache Specifics

  1. AEM Dispatcher and Varnish Cache containers might not run and start if no AEM Publish instance is active. The reason for such behavior is that AEM Dispatcher and Varnish Cache reference an AEM Publish domain name, hence when AEM Publish instance isn’t available, the reference is void, which is not supported by AEM Dispatcher or Varnish Cache.

  2. AEM Dispatcher and Varnish Cache configuration files used during images build are the default ones, but adjusted as little as possible according to the official instructions. The original versions of the configuration files for the sake of comparison are kept in respective directories besides the changed ones.

  3. In order to transfer files (primarily new configuration files) from the host into a container in which AEM Dispatcher or Varnish Cache are run, use commands constructed in the following way:

    # AEM Dispatcher - AMD:
docker cp "$HOME/dispatcher.any" dispatcher-amd:/etc/apache2/conf/dispatcher.any

# AEM Dispatcher - ARM:
docker cp "$HOME/dispatcher.any" dispatcher-arm:/etc/apache2/conf/dispatcher.any

# Varnish Cache:
docker cp "$HOME/default.vcl" varnish:/etc/varnish/default.vcl

  4. In order to activate new configuration of AEM Dispatcher or Varnish Cache, there is no need to restart containers. New configuration can be applied via reloading:

    # AEM Dispatcher - AMD:
docker exec dispatcher-amd /etc/init.d/apache2 reload

# AEM Dispatcher - ARM:
docker exec dispatcher-arm /etc/init.d/apache2 reload

# Varnish Cache:
docker exec varnish varnishreload

2.4. Solr Configuration Sets Updates

Apache Solr image embeds an update-configset.sh script that can be used to update configuration sets of the instance. Usage manual is provided in the script itself. There is also a download-configsets.sh script embedded for exporting existing configuration sets.

2.5. Mobile Volumes

  1. Persistence layers of AEM Author and AEM Publish instances are linked to /opt/aem/author/crx-quickstart and /opt/aem/publish/crx-quickstart paths inside respective containers. Those paths are mount points for aem-author-data-65, aem-author-data-cloud, aem-publish-data-65 and aem-publish-data-cloud volumes respectively, physically stored on a host at /var/lib/docker/volumes and managed by Docker. It means that persistence layers of AEM Author and AEM Publish instances are separated from the application.

  2. If aem-author-data-65, aem-author-data-cloud, aem-publish-data-65 or aem-publish-data-cloud volume doesn’t exist when a container with AEM Author or AEM Publish respectively is run for the first time, then a respective volume will be created and mounted to the container. However, if a respective volume does already exist, then no new volume will be created and the existing one will be reused, so that even to a new container the old volume with old persistence layer will be mounted. In order to avoid such reuse, before a new container is run for the first time, the respective volume should be priorly removed:

    # AEM Author 6.5:
docker volume remove --force aem-author-data-65

# AEM Author as a Cloud Service:
docker volume remove --force aem-author-data-cloud

# AEM Publish 6.5:
docker volume remove --force aem-publish-data-65

# AEM Publish as a Cloud Service:
docker volume remove --force aem-publish-data-cloud

  3. The described volumes mechanism makes AEM Author and AEM Publish persistence layers mobile, transferable and backupable. That mechanism can be rolled out to remote environments in order to make those environments fully reproducible locally.

3. Start Command Decomposition

This section explains every part of commands used to start AEM instances. The explanation employs an example based on the command for the AEM Author, but nevertheless for AEM Publish the command is analogous.

  1. Set max heap size:

    -Xmx4096M

    Docs:
    https://experienceleague.adobe.com/docs/experience-manager-65/deploying/deploying/deploy.html?lang=en (-Xmx1024M is given as recommended, but it is too little for parallel garbage collection)

  2. Fix Java 11 bug related to ZIP validation:

    -Djdk.util.zip.disableZip64ExtraFieldValidation=true

    Docs:

  3. Run AEM in a headless mode because it is run inside a Docker container:

    -Djava.awt.headless=true

  4. Set JVM specific parameters for Java 11:

    -XX:+UseParallelGC --add-opens=java.desktop/com.sun.imageio.plugins.jpeg=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.jrt=ALL-UNNAMED --add-opens=java.naming/javax.naming.spi=ALL-UNNAMED --add-opens=java.xml/com.sun.org.apache.xerces.internal.dom=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/jdk.internal.loader=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED -Dnashorn.args=--no-deprecation-warning

  5. Run AEM in debug mode on the given port, additionally to the basic port:

    -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:8888

  6. Set initial admin password in non-interactive mode. The admin password from a passwordfile.properties file is assessed only during the first AEM start. If after the first AEM start the admin password is changed, the new password is effective regardless of the content of a passwordfile.properties file:

    -Dadmin.password.file=/opt/aem/author/passwordfile.properties
    (…​)
    -nointeractive

  7. Set run modes. The hyphenated author-local run mode is required for cases where dot-driven run mode like author.local isn’t supported:

    -Dsling.run.modes=author,nosamplecontent,local,author-local

  8. Set port:

    -port 4502

  9. Exclude forks (among others, it mitigates warnings in the console):

    -nofork

  10. Don’t open AEM in a browser, since AEM is run inside a Docker container:

    -nobrowser

4. License

The program is subject to MIT No Attribution License

Copyright © 2023-2024 Herman Ciechanowiec

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so.

The Software is provided 'as is', without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the authors or copyright holders be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the Software or the use or other dealings in the Software.

About

Program for setting up and managing local Adobe Experience Manager (AEM) development environment via Docker

Resources

Readme

License

View license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • Shell 72.7%
  • Dockerfile 12.2%
  • VCL 8.0%
  • HTML 7.1%