Add ability to launch custom scripts before the start of the app #4030

edevosc2c · 2023-09-07T12:23:51Z

Why

It is becoming a major thing that a lot of clients will ask us to have small modifications over the existing georchestra components.

For instance, Datagrandest mostly have only customized images for their needs.

The downside of maintaining custom images is that it adds maintenance cost, (sometimes confusion) and a big lag with the upstream/official image.

Creating a whole new docker image is not always needed, especially for small modifications.

Proposed solution

At the geonetwork-ui project, they introduced the ability to execute custom scripts: geonetwork/geonetwork-ui#507

So I propose to have a similar system in place for every docker image that we manage. I'm not entirely sure how to do it, but the ability to specify the directory where all the custom scripts will be executed is a good idea.
This would allow specifying these scripts inside the datadir for example.

Having this functionality will avoid doing "hacky" stuff, especially in the helm chart of georchestra for customizing the container, example:

Additional comment

geoserver docker image seems to already support this "script" thing: https://github.com/georchestra/georchestra/tree/master/geoserver/webapp/src/docker/docker-entrypoint.d

On top of that this would fit the need explained by @jeanpommier here: georchestra/helm-georchestra#46 (comment)

Ping @jeanmi151 @jeanpommier @pmauduit @fvanderbiest

jeanpommier · 2023-09-07T12:45:05Z

Yup, sounds like a good idea.
+1

jeanmi151 · 2023-09-07T13:09:18Z

Okay after discussion with @edevosc2c , We think something like it is done for geonetwork-ui (datahub) would be a great thing :
https://github.com/geonetwork/geonetwork-ui/pull/507/files#diff-964be1ec80820060f908f3aec54f46def8145c723a5b8a5bb44b9ec69c76d0c9

Whats we are thinking is :

have a script with a env var (link to a path) that would activate (copy or sylmbolic link) customize webapp before running the service
have a script with a env var (link to a path) that would execute all insider script contained in it
we could store this customization script and webressources in the /etc/georchestra/[application_name]/webappp_ressources/ and /etc/georchestra/[application_name]/init_scripts/

Would it be okay with you @jeanpommier ?

jeanpommier · 2023-09-07T13:47:15Z

Can't it all be covered by the init scripts ? An init script can very well copy the webapp_resources to the right place, can't it ?

edevosc2c · 2023-09-07T15:28:46Z

The point here is to have built-in support for two major functionalities.

The first one being to support the ability to easily override the current files in the container from the files located for example in the datadir or an external volume.

So for example, datadir/geonetwork/ressources/webapps would overwrite the files located in /var/lib/jetty/webapps of the geonetwork container. We can specify the source directory using an environment variable.

And the second one is the ability to execute any custom scripts (mounted as a directory from a volume).

We could only support the latter one, but we think customizing the actual files of the app is such a common thing that it is a good idea to have a built-in support. We could have the same script that offer the first functionality in every datadir of our clients, but it would mean more maintenance involved in maintaining them.

Take a look a bit at what is being done inside datahub here: https://github.com/geonetwork/geonetwork-ui/blob/main/tools/docker/docker-entrypoint.sh

We can still discuss on how to implement them but the general ideas are said. Let me know what you think about them.

edevosc2c · 2023-09-08T11:59:27Z

I have been thinking of backporting these changes to previous georchestra docker images.

As it won't affect the usability/functionalities of the actual components of georchestra.

jeanpommier · 2023-09-08T12:25:34Z

Yup, seems reasonable.

jeanpommier · 2023-09-08T12:30:43Z

About the entrypoint implementation, it might be too soon for that but for the records, I'm pointing to a discussion I had some time ago about the pros and cons of runparts vs sourcing the sh scripts, which now might have my preference: dalibo/ldap2pg#381

jeanmi151 · 2023-10-13T07:33:34Z

notes of functionnality wanted, in kubernetes deployment the following geoserver files : https://github.com/georchestra/geoserver_minimal_datadir/blob/master/security/usergroup/my_ldap/config.xml#L5
https://github.com/georchestra/geoserver_minimal_datadir/blob/master/security/role/my_ldap/config.xml#L5

They are in bad configuration and we need to modify it by hand, I would be great to have an init script that overwrite this values.

fvanderbiest · 2023-10-15T10:04:18Z

They are in bad configuration and we need to modify it by hand, I would be great to have an init script that overwrite this values.

Please note the image's entrypoint already tries to do some magic wrt these file:

georchestra/geoserver/webapp/src/docker/docker-entrypoint.d/00-geoserver-bootstrap

Lines 13 to 15 in b480219

    
           echo 'Change LDAP url according to docker setup' 
        
           sed -i 's:<serverURL>ldap\://localhost\:389</serverURL>:<serverURL>ldap\://ldap\:389</serverURL>:g' /mnt/geoserver_datadir/security/role/my_ldap/config.xml 
        
           sed -i 's:<serverURL>ldap\://localhost\:389</serverURL>:<serverURL>ldap\://ldap\:389</serverURL>:g' /mnt/geoserver_datadir/security/usergroup/my_ldap/config.xml

jeanmi151 · 2023-11-08T09:59:08Z

We discussed with @edevosc2c, in priority we will create autoscript run for personalization for the following apps:

security proxy for need for overwrite 404.jsp
geoserver (for web.xml, and maybe plugings)
geonetwork
cas (add pluging)
console (customization of error messages translations)
header (customization of the style for the header)
ldap (sasl customization)

if you see others customizations for app I didn't put here please let me know

f-necas · 2023-11-08T15:11:00Z

edevosc2c · 2023-11-08T15:52:01Z

georchestra/georchestra#4030

* feat: allow using startup scripts with cas datadir georchestra/georchestra#4030 * feat: improve custom scripts to match other repos * feat: taking account of edevos recommandations

edevosc2c · 2023-12-01T18:09:25Z

merged in master, backport PRs:

Implement custom scripts (#4083) - backport to 23 #4106
backport custom scripts enhancement to 23 georchestra-cas-server#19
datafeeder ui-frontend, no idea?? seems to be making a tag with release
backport header and custom scripts changes to 2022.02.xx mapstore2-georchestra#676
backport header and custom scripts changes to 2023.02.xx mapstore2-georchestra#675
docker-exim - smtp, no idea??? master
Enhance custom-scripts (#7) geonetwork-microservices#8

edevosc2c · 2023-12-07T13:22:45Z

geOrchestra 23.0.4 was released with the custom scripts support:

georchestra/analytics:23.0.4
georchestra/cas:23.0.4-georchestra
georchestra/console:23.0.4
georchestra/datafeeder:23.0.4
georchestra/datafeeder-frontend:23.0.4
georchestra/gn-cloud-ogc-api-records-service:23.0.4
georchestra/geoserver:23.0.4
georchestra/header:23.0.4
georchestra/mapstore:23.0.4-georchestra
georchestra/ldap:23.0.4
georchestra/security-proxy:23.0.4

georchestra gateway does not have yet custom scripts.

The usage is the following:
With the environment variable CUSTOM_SCRIPTS_DIRECTORY, this specify the directory where all the scripts should be executed before the start of the application.

edevosc2c · 2023-12-11T10:16:23Z

Still missing custom scripts on georchestra gateway

edevosc2c · 2024-06-06T11:26:18Z

I'm closing this issue in favor of a separate one in georchestra/georchestra-gateway#127

edevosc2c added the enhancement label Sep 7, 2023

edevosc2c transferred this issue from georchestra/docker Sep 7, 2023

jeanmi151 mentioned this issue Sep 7, 2023

add webapp prepare initcontainer for security-proxy georchestra/helm-georchestra#53

Merged

jeanmi151 added the gardening label Sep 7, 2023

f-necas added a commit to georchestra/georchestra-cas-server that referenced this issue Nov 9, 2023

feat: allow using startup scripts with cas datadir

256ae15

georchestra/georchestra#4030

This was referenced Nov 12, 2023

Implemention of new WebComponent header #4065

Merged

Implement custom scripts #4083

Merged

Implement custom-scripts to geonetwork georchestra/geonetwork#262

Merged

This was referenced Nov 20, 2023

Implement custom scripts for mapstore georchestra/mapstore2-georchestra#669

Merged

Implement custom scripts for ogc-api-records-services georchestra/geonetwork-microservices#5

Merged

Implement custom script in smtp camptocamp/docker-exim#8

Merged

edevosc2c modified the milestones: 23.1.0, 23.0.4 Nov 24, 2023

This was referenced Dec 1, 2023

Implement custom scripts georchestra/georchestra-cas-server#18

Merged

Implement custom scripts georchestra/geonetwork-ui#6

Closed

Implement custom scripts georchestra/geonetwork-ui#7

Merged

edevosc2c mentioned this issue Dec 1, 2023

Remove all the "prepare override" georchestra/helm-georchestra#77

Closed

fvanderbiest closed this as completed Dec 10, 2023

edevosc2c reopened this Dec 11, 2023

edevosc2c modified the milestones: 23.0.4, 23.0.5 Dec 11, 2023

f-necas modified the milestones: 23.0.5, 24.0.0 Dec 20, 2023

edevosc2c mentioned this issue Jan 8, 2024

remove all prepare emptydir workarounds georchestra/helm-georchestra#79

Merged

edevosc2c mentioned this issue Jun 6, 2024

Add ability to launch custom scripts before the start of the app georchestra/georchestra-gateway#127

Open

edevosc2c closed this as completed Jun 6, 2024

edevosc2c mentioned this issue Jul 31, 2024

add ability to launch custom scripts geowebcache #4291

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ability to launch custom scripts before the start of the app #4030

Add ability to launch custom scripts before the start of the app #4030

edevosc2c commented Sep 7, 2023 •

edited

Loading

jeanpommier commented Sep 7, 2023

jeanmi151 commented Sep 7, 2023

jeanpommier commented Sep 7, 2023

edevosc2c commented Sep 7, 2023 •

edited

Loading

edevosc2c commented Sep 8, 2023

jeanpommier commented Sep 8, 2023

jeanpommier commented Sep 8, 2023

jeanmi151 commented Oct 13, 2023

fvanderbiest commented Oct 15, 2023

jeanmi151 commented Nov 8, 2023 •

edited by edevosc2c

Loading

f-necas commented Nov 8, 2023 •

edited by edevosc2c

Loading

edevosc2c commented Nov 8, 2023 •

edited

Loading

edevosc2c commented Dec 1, 2023 •

edited by f-necas

Loading

edevosc2c commented Dec 7, 2023 •

edited

Loading

edevosc2c commented Dec 11, 2023

edevosc2c commented Jun 6, 2024

Add ability to launch custom scripts before the start of the app #4030

Add ability to launch custom scripts before the start of the app #4030

Comments

edevosc2c commented Sep 7, 2023 • edited Loading

Why

Proposed solution

Additional comment

jeanpommier commented Sep 7, 2023

jeanmi151 commented Sep 7, 2023

jeanpommier commented Sep 7, 2023

edevosc2c commented Sep 7, 2023 • edited Loading

edevosc2c commented Sep 8, 2023

jeanpommier commented Sep 8, 2023

jeanpommier commented Sep 8, 2023

jeanmi151 commented Oct 13, 2023

fvanderbiest commented Oct 15, 2023

jeanmi151 commented Nov 8, 2023 • edited by edevosc2c Loading

f-necas commented Nov 8, 2023 • edited by edevosc2c Loading

edevosc2c commented Nov 8, 2023 • edited Loading

edevosc2c commented Dec 1, 2023 • edited by f-necas Loading

edevosc2c commented Dec 7, 2023 • edited Loading

edevosc2c commented Dec 11, 2023

edevosc2c commented Jun 6, 2024

edevosc2c commented Sep 7, 2023 •

edited

Loading

edevosc2c commented Sep 7, 2023 •

edited

Loading

jeanmi151 commented Nov 8, 2023 •

edited by edevosc2c

Loading

f-necas commented Nov 8, 2023 •

edited by edevosc2c

Loading

edevosc2c commented Nov 8, 2023 •

edited

Loading

edevosc2c commented Dec 1, 2023 •

edited by f-necas

Loading

edevosc2c commented Dec 7, 2023 •

edited

Loading