Loosing Semaphore Arrays on HTTPD Reload #522

ThorstenGuentherDSA · 2021-01-23T05:21:07Z

ThorstenGuentherDSA
Jan 23, 2021

Hi!

I'm using the mod_auth_openidc-2.4.5-1.el7.x86_64.rpm package on a RedHat 7.9 with Apache HTTPD 2.4.6 Release 97.el7_9.

When the HTTPD is reloaded, the number of used semaphore arrays is incremented by one. On another system the number increses by two on reload. On the system that is loosing only one semaphore, the module is loaded but not configured.
When mod_auth_openidc is disabled in the HTTPD config, there is no increase in the used semaphore arrays on reload.

Before reload:
------ Semaphore Status --------
used arrays = 9
allocated semaphores = 22

------ Semaphore Arrays --------
key semid owner perms nsems
0x00000000 2 zabbix 600 14
0x00000000 5 apache 600 1
0x00000000 6 apache 600 1
0x00000000 7 apache 600 1
0x00000000 8 apache 600 1
0x00000000 9 apache 600 1
0x00000000 10 apache 600 1
0x00000000 11 apache 600 1
0x00000000 12 apache 600 1

After reload:
------ Semaphore Status --------
used arrays = 11
allocated semaphores = 24

------ Semaphore Arrays --------
key semid owner perms nsems
0x00000000 2 zabbix 600 14
0x00000000 5 apache 600 1
0x00000000 10 apache 600 1
0x00000000 11 apache 600 1
0x00000000 13 apache 600 1
0x00000000 14 apache 600 1
0x00000000 15 apache 600 1
0x00000000 16 apache 600 1
0x00000000 17 apache 600 1
0x00000000 18 apache 600 1
0x00000000 19 apache 600 1

HTTPD error log during reload / log rotate:
[Fri Jan 22 03:23:01.630051 2021] [mpm_prefork:notice] [pid 1757] AH00171: Graceful restart requested, doing restart
(log rotate)
[Fri Jan 22 03:23:01.668199 2021] [lbmethod_heartbeat:notice] [pid 1757] AH02282: No slotmem from mod_heartmonitor
[Fri Jan 22 03:23:01.673051 2021] [mpm_prefork:notice] [pid 1757] AH00163: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.2k-fips configured -- resuming normal operations
[Fri Jan 22 03:23:01.673062 2021] [core:notice] [pid 1757] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'

Is there a more approriate way to report this problem? As I understood the issue tracker should not be used.
Also, is there any log or further info I can provide to pinpoint this issue? Unfortunately I'm am not a c-developer, so I will require some guidiance.

Btw, otherwise the module works like a charm! Thanks for providing it!

Best regards,
Thorsten

Answered by zandbelt

Jan 23, 2021

I reproduced it on Ubuntu; it appears child init is called twice for each child process (not sure why); I have fixed it here: 03548e2

would you be able to test this from master?

View full answer

zandbelt · 2021-01-23T07:19:18Z

zandbelt
Jan 23, 2021
Maintainer

thanks for reporting, I will try to reproduce, file it as a bug and try to fix it; there have been a number of problems related to graceful restarts before, it is a complex topic

1 reply

ThorstenGuentherDSA Jan 23, 2021
Author

Great, thank you! Again, if there is anything I can contribute I will be more than happy to help.

zandbelt · 2021-01-23T09:56:46Z

zandbelt
Jan 23, 2021
Maintainer

I reproduced it on Ubuntu; it appears child init is called twice for each child process (not sure why); I have fixed it here: 03548e2

would you be able to test this from master?

7 replies

zandbelt Jan 23, 2021
Maintainer

I have this:

root@7062d539db19:~# apache2ctl start
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
root@7062d539db19:~# ipcs -s -u

------ Semaphore Status --------
used arrays = 2
allocated semaphores = 2

root@7062d539db19:~# apache2ctl graceful
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
root@7062d539db19:~# ipcs -s -u

------ Semaphore Status --------
used arrays = 2
allocated semaphores = 2

root@7062d539db19:~# apache2ctl graceful
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
root@7062d539db19:~# ipcs -s -u

------ Semaphore Status --------
used arrays = 2
allocated semaphores = 2

root@7062d539db19:~# apache2ctl graceful
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
root@7062d539db19:~# ipcs -s -u

------ Semaphore Status --------
used arrays = 2
allocated semaphores = 2

leading to the logs as in:

[Sat Jan 23 13:59:15.746241 2021] [ssl:warn] [pid 33:tid 140286009715648] AH01909: 172.17.0.2:443:0 server certificate does NOT include an ID which matches the server name
[Sat Jan 23 13:59:15.762194 2021] [ssl:warn] [pid 34:tid 140286009715648] AH01909: 172.17.0.2:443:0 server certificate does NOT include an ID which matches the server name
[Sat Jan 23 13:59:15.762326 2021] [auth_openidc:info] [pid 34:tid 140286009715648] mod_auth_openidc-2.4.6-dev - init - cjose 0.6.1.5, OpenSSL 1.1.1  11 Sep 2018, EC=yes, GCM=yes, Memcache=yes, Redis=no, JQ=no
[Sat Jan 23 13:59:15.763660 2021] [auth_openidc:debug] [pid 34:tid 140286009715648] src/cache/common.c(97): oidc_cache_mutex_post_config: enter: -1 (m=0,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:15.763972 2021] [auth_openidc:debug] [pid 34:tid 140286009715648] src/cache/shm.c(126): oidc_cache_shm_post_config: initialized shared memory with a cache size (# entries) of: 500, and a max (single) entry size of: 16913
[Sat Jan 23 13:59:15.763995 2021] [auth_openidc:debug] [pid 34:tid 140286009715648] src/config.c(2330): oidc_config_check_vhost_config: enter
[Sat Jan 23 13:59:15.764013 2021] [auth_openidc:warn] [pid 34:tid 140286009715648] oidc_check_config_oauth: The OAuth 2.0 Resource Server functionality is deprecated and superseded by a new module, see: https://github.com/zmartzone/mod_oauth2!
[Sat Jan 23 13:59:15.765082 2021] [mpm_event:notice] [pid 34:tid 140286009715648] AH00489: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.1 configured -- resuming normal operations
[Sat Jan 23 13:59:15.765121 2021] [core:notice] [pid 34:tid 140286009715648] AH00094: Command line: '/usr/sbin/apache2'
[Sat Jan 23 13:59:15.765134 2021] [auth_openidc:debug] [pid 35:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 1 (m=7f96e1c459d0,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:15.765183 2021] [auth_openidc:debug] [pid 35:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 2 (m=7f96e1c459d0,s=7f96e1baf388, p=0)
[Sat Jan 23 13:59:15.765204 2021] [auth_openidc:debug] [pid 35:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 2 (m=7f96e1c459d0,s=7f96e1bae418, p=0)
[Sat Jan 23 13:59:15.765536 2021] [auth_openidc:debug] [pid 36:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 2 (m=7f96e1c459d0,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:15.765579 2021] [auth_openidc:debug] [pid 36:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 3 (m=7f96e1c459d0,s=7f96e1baf388, p=0)
[Sat Jan 23 13:59:15.765592 2021] [auth_openidc:debug] [pid 36:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 3 (m=7f96e1c459d0,s=7f96e1bae418, p=0)
[Sat Jan 23 13:59:25.055233 2021] [mpm_event:notice] [pid 34:tid 140286009715648] AH00493: SIGUSR1 received.  Doing graceful restart
[Sat Jan 23 13:59:25.055344 2021] [auth_openidc:debug] [pid 34] src/cache/common.c(213): oidc_cache_mutex_destroy: enter: 3 (m=7f96e1c459d0,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:25.055362 2021] [auth_openidc:debug] [pid 34] src/cache/common.c(221): oidc_cache_mutex_destroy: processing: 2 (m=7f96e1c459d0,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:25.055429 2021] [auth_openidc:info] [pid 34] mod_auth_openidc-2.4.6-dev - shutdown
[Sat Jan 23 13:59:25.056625 2021] [auth_openidc:debug] [pid 35:tid 140286009715648] src/cache/common.c(213): oidc_cache_mutex_destroy: enter: 2 (m=7f96e1c459d0,s=7f96e1c414a0, p=0)
[Sat Jan 23 13:59:25.056695 2021] [auth_openidc:debug] [pid 35:tid 140286009715648] src/cache/common.c(221): oidc_cache_mutex_destroy: processing: 1 (m=7f96e1c459d0,s=7f96e1c414a0, p=0)
[Sat Jan 23 13:59:25.058481 2021] [auth_openidc:debug] [pid 36:tid 140286009715648] src/cache/common.c(213): oidc_cache_mutex_destroy: enter: 1 (m=7f96e1c459d0,s=7f96e1c414a0, p=0)
[Sat Jan 23 13:59:25.058507 2021] [auth_openidc:debug] [pid 36:tid 140286009715648] src/cache/common.c(221): oidc_cache_mutex_destroy: processing: 0 (m=7f96e1c459d0,s=7f96e1c414a0, p=0)
[Sat Jan 23 13:59:25.058539 2021] [auth_openidc:debug] [pid 36:tid 140286009715648] src/cache/common.c(226): oidc_cache_mutex_destroy: apr_shm_destroy for semaphore returned: 0
[Sat Jan 23 13:59:25.058559 2021] [auth_openidc:debug] [pid 36:tid 140286009715648] src/cache/common.c(232): oidc_cache_mutex_destroy: apr_global_mutex_destroy returned :0
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
[Sat Jan 23 13:59:25.067407 2021] [ssl:warn] [pid 34:tid 140286009715648] AH01909: 172.17.0.2:443:0 server certificate does NOT include an ID which matches the server name
[Sat Jan 23 13:59:25.067467 2021] [auth_openidc:info] [pid 34:tid 140286009715648] mod_auth_openidc-2.4.6-dev - init - cjose 0.6.1.5, OpenSSL 1.1.1  11 Sep 2018, EC=yes, GCM=yes, Memcache=yes, Redis=no, JQ=no
[Sat Jan 23 13:59:25.068413 2021] [auth_openidc:debug] [pid 34:tid 140286009715648] src/cache/common.c(97): oidc_cache_mutex_post_config: enter: -1 (m=0,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:25.068534 2021] [auth_openidc:debug] [pid 34:tid 140286009715648] src/cache/shm.c(126): oidc_cache_shm_post_config: initialized shared memory with a cache size (# entries) of: 500, and a max (single) entry size of: 16913
[Sat Jan 23 13:59:25.068549 2021] [auth_openidc:debug] [pid 34:tid 140286009715648] src/config.c(2330): oidc_config_check_vhost_config: enter
[Sat Jan 23 13:59:25.068562 2021] [auth_openidc:warn] [pid 34:tid 140286009715648] oidc_check_config_oauth: The OAuth 2.0 Resource Server functionality is deprecated and superseded by a new module, see: https://github.com/zmartzone/mod_oauth2!
[Sat Jan 23 13:59:25.068704 2021] [mpm_event:notice] [pid 34:tid 140286009715648] AH00489: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.1 configured -- resuming normal operations
[Sat Jan 23 13:59:25.068718 2021] [core:notice] [pid 34:tid 140286009715648] AH00094: Command line: '/usr/sbin/apache2'
[Sat Jan 23 13:59:25.069419 2021] [auth_openidc:debug] [pid 96:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 1 (m=7f96e1c45e88,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:25.069454 2021] [auth_openidc:debug] [pid 96:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 2 (m=7f96e1c45e88,s=7f96e1ba9388, p=0)
[Sat Jan 23 13:59:25.069465 2021] [auth_openidc:debug] [pid 96:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 2 (m=7f96e1c45e88,s=7f96e1ba8418, p=0)
[Sat Jan 23 13:59:25.069770 2021] [auth_openidc:debug] [pid 97:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 2 (m=7f96e1c45e88,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:25.069804 2021] [auth_openidc:debug] [pid 97:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 3 (m=7f96e1c45e88,s=7f96e1ba9388, p=0)
[Sat Jan 23 13:59:25.069816 2021] [auth_openidc:debug] [pid 97:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 3 (m=7f96e1c45e88,s=7f96e1ba8418, p=0)
[Sat Jan 23 13:59:28.663856 2021] [mpm_event:notice] [pid 34:tid 140286009715648] AH00493: SIGUSR1 received.  Doing graceful restart
[Sat Jan 23 13:59:28.663945 2021] [auth_openidc:debug] [pid 34] src/cache/common.c(213): oidc_cache_mutex_destroy: enter: 3 (m=7f96e1c45e88,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:28.663962 2021] [auth_openidc:debug] [pid 34] src/cache/common.c(221): oidc_cache_mutex_destroy: processing: 2 (m=7f96e1c45e88,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:28.664019 2021] [auth_openidc:info] [pid 34] mod_auth_openidc-2.4.6-dev - shutdown
[Sat Jan 23 13:59:28.665288 2021] [auth_openidc:debug] [pid 96:tid 140286009715648] src/cache/common.c(213): oidc_cache_mutex_destroy: enter: 2 (m=7f96e1c45e88,s=7f96e1c414a0, p=0)
[Sat Jan 23 13:59:28.665288 2021] [auth_openidc:debug] [pid 97:tid 140286009715648] src/cache/common.c(213): oidc_cache_mutex_destroy: enter: 2 (m=7f96e1c45e88,s=7f96e1c414a0, p=0)
[Sat Jan 23 13:59:28.665320 2021] [auth_openidc:debug] [pid 96:tid 140286009715648] src/cache/common.c(221): oidc_cache_mutex_destroy: processing: 1 (m=7f96e1c45e88,s=7f96e1c414a0, p=0)
[Sat Jan 23 13:59:28.665357 2021] [auth_openidc:debug] [pid 97:tid 140286009715648] src/cache/common.c(221): oidc_cache_mutex_destroy: processing: 0 (m=7f96e1c45e88,s=7f96e1c414a0, p=0)
[Sat Jan 23 13:59:28.665378 2021] [auth_openidc:debug] [pid 97:tid 140286009715648] src/cache/common.c(226): oidc_cache_mutex_destroy: apr_shm_destroy for semaphore returned: 0
[Sat Jan 23 13:59:28.665397 2021] [auth_openidc:debug] [pid 97:tid 140286009715648] src/cache/common.c(232): oidc_cache_mutex_destroy: apr_global_mutex_destroy returned :0
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
[Sat Jan 23 13:59:28.675810 2021] [ssl:warn] [pid 34:tid 140286009715648] AH01909: 172.17.0.2:443:0 server certificate does NOT include an ID which matches the server name
[Sat Jan 23 13:59:28.675874 2021] [auth_openidc:info] [pid 34:tid 140286009715648] mod_auth_openidc-2.4.6-dev - init - cjose 0.6.1.5, OpenSSL 1.1.1  11 Sep 2018, EC=yes, GCM=yes, Memcache=yes, Redis=no, JQ=no
[Sat Jan 23 13:59:28.676835 2021] [auth_openidc:debug] [pid 34:tid 140286009715648] src/cache/common.c(97): oidc_cache_mutex_post_config: enter: -1 (m=0,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:28.676971 2021] [auth_openidc:debug] [pid 34:tid 140286009715648] src/cache/shm.c(126): oidc_cache_shm_post_config: initialized shared memory with a cache size (# entries) of: 500, and a max (single) entry size of: 16913
[Sat Jan 23 13:59:28.676988 2021] [auth_openidc:debug] [pid 34:tid 140286009715648] src/config.c(2330): oidc_config_check_vhost_config: enter
[Sat Jan 23 13:59:28.677004 2021] [auth_openidc:warn] [pid 34:tid 140286009715648] oidc_check_config_oauth: The OAuth 2.0 Resource Server functionality is deprecated and superseded by a new module, see: https://github.com/zmartzone/mod_oauth2!
[Sat Jan 23 13:59:28.677116 2021] [mpm_event:notice] [pid 34:tid 140286009715648] AH00489: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.1 configured -- resuming normal operations
[Sat Jan 23 13:59:28.677131 2021] [core:notice] [pid 34:tid 140286009715648] AH00094: Command line: '/usr/sbin/apache2'
[Sat Jan 23 13:59:28.677827 2021] [auth_openidc:debug] [pid 157:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 1 (m=7f96e1c49b70,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:28.677858 2021] [auth_openidc:debug] [pid 157:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 2 (m=7f96e1c49b70,s=7f96e1c13388, p=0)
[Sat Jan 23 13:59:28.677872 2021] [auth_openidc:debug] [pid 157:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 2 (m=7f96e1c49b70,s=7f96e1c12418, p=0)
[Sat Jan 23 13:59:28.678174 2021] [auth_openidc:debug] [pid 158:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 2 (m=7f96e1c49b70,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:28.678200 2021] [auth_openidc:debug] [pid 158:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 3 (m=7f96e1c49b70,s=7f96e1c13388, p=0)
[Sat Jan 23 13:59:28.678211 2021] [auth_openidc:debug] [pid 158:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 3 (m=7f96e1c49b70,s=7f96e1c12418, p=0)
[Sat Jan 23 13:59:32.402278 2021] [mpm_event:notice] [pid 34:tid 140286009715648] AH00493: SIGUSR1 received.  Doing graceful restart
[Sat Jan 23 13:59:32.402373 2021] [auth_openidc:debug] [pid 34] src/cache/common.c(213): oidc_cache_mutex_destroy: enter: 3 (m=7f96e1c49b70,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:32.402388 2021] [auth_openidc:debug] [pid 34] src/cache/common.c(221): oidc_cache_mutex_destroy: processing: 2 (m=7f96e1c49b70,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:32.402452 2021] [auth_openidc:info] [pid 34] mod_auth_openidc-2.4.6-dev - shutdown
[Sat Jan 23 13:59:32.404117 2021] [auth_openidc:debug] [pid 157:tid 140286009715648] src/cache/common.c(213): oidc_cache_mutex_destroy: enter: 2 (m=7f96e1c49b70,s=7f96e1c414a0, p=0)
[Sat Jan 23 13:59:32.404132 2021] [auth_openidc:debug] [pid 158:tid 140286009715648] src/cache/common.c(213): oidc_cache_mutex_destroy: enter: 2 (m=7f96e1c49b70,s=7f96e1c414a0, p=0)
[Sat Jan 23 13:59:32.404156 2021] [auth_openidc:debug] [pid 157:tid 140286009715648] src/cache/common.c(221): oidc_cache_mutex_destroy: processing: 1 (m=7f96e1c49b70,s=7f96e1c414a0, p=0)
[Sat Jan 23 13:59:32.404183 2021] [auth_openidc:debug] [pid 158:tid 140286009715648] src/cache/common.c(221): oidc_cache_mutex_destroy: processing: 0 (m=7f96e1c49b70,s=7f96e1c414a0, p=0)
[Sat Jan 23 13:59:32.404204 2021] [auth_openidc:debug] [pid 158:tid 140286009715648] src/cache/common.c(226): oidc_cache_mutex_destroy: apr_shm_destroy for semaphore returned: 0
[Sat Jan 23 13:59:32.404222 2021] [auth_openidc:debug] [pid 158:tid 140286009715648] src/cache/common.c(232): oidc_cache_mutex_destroy: apr_global_mutex_destroy returned :0
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
[Sat Jan 23 13:59:32.414071 2021] [ssl:warn] [pid 34:tid 140286009715648] AH01909: 172.17.0.2:443:0 server certificate does NOT include an ID which matches the server name
[Sat Jan 23 13:59:32.414134 2021] [auth_openidc:info] [pid 34:tid 140286009715648] mod_auth_openidc-2.4.6-dev - init - cjose 0.6.1.5, OpenSSL 1.1.1  11 Sep 2018, EC=yes, GCM=yes, Memcache=yes, Redis=no, JQ=no
[Sat Jan 23 13:59:32.415054 2021] [auth_openidc:debug] [pid 34:tid 140286009715648] src/cache/common.c(97): oidc_cache_mutex_post_config: enter: -1 (m=0,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:32.415166 2021] [auth_openidc:debug] [pid 34:tid 140286009715648] src/cache/shm.c(126): oidc_cache_shm_post_config: initialized shared memory with a cache size (# entries) of: 500, and a max (single) entry size of: 16913
[Sat Jan 23 13:59:32.415181 2021] [auth_openidc:debug] [pid 34:tid 140286009715648] src/config.c(2330): oidc_config_check_vhost_config: enter
[Sat Jan 23 13:59:32.415193 2021] [auth_openidc:warn] [pid 34:tid 140286009715648] oidc_check_config_oauth: The OAuth 2.0 Resource Server functionality is deprecated and superseded by a new module, see: https://github.com/zmartzone/mod_oauth2!
[Sat Jan 23 13:59:32.415291 2021] [mpm_event:notice] [pid 34:tid 140286009715648] AH00489: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.1 configured -- resuming normal operations
[Sat Jan 23 13:59:32.415305 2021] [core:notice] [pid 34:tid 140286009715648] AH00094: Command line: '/usr/sbin/apache2'
[Sat Jan 23 13:59:32.416018 2021] [auth_openidc:debug] [pid 218:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 1 (m=7f96e1c49fc8,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:32.416049 2021] [auth_openidc:debug] [pid 218:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 2 (m=7f96e1c49fc8,s=7f96e1c3f388, p=0)
[Sat Jan 23 13:59:32.416061 2021] [auth_openidc:debug] [pid 218:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 2 (m=7f96e1c49fc8,s=7f96e1c3e418, p=0)
[Sat Jan 23 13:59:32.416322 2021] [auth_openidc:debug] [pid 219:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 2 (m=7f96e1c49fc8,s=7f96e1c414a0, p=1)
[Sat Jan 23 13:59:32.416354 2021] [auth_openidc:debug] [pid 219:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 3 (m=7f96e1c49fc8,s=7f96e1c3f388, p=0)
[Sat Jan 23 13:59:32.416367 2021] [auth_openidc:debug] [pid 219:tid 140286009715648] src/cache/common.c(152): oidc_cache_mutex_child_init: enter: 3 (m=7f96e1c49fc8,s=7f96e1c3e418, p=0)

which was different before that last commit, one semaphore added on each graceful restart, as you reported

zandbelt Jan 23, 2021
Maintainer

I reproduced on both Ubuntu Bionic and CentOS 7.9 and on both platforms the latest commit fixes the semaphone cleanup, I hope you can confirm.

ThorstenGuentherDSA Jan 23, 2021
Author

Yes, I can confirm it. It just takes a few seconds for them to be removed. Sorry for the confusion. :-/

zandbelt Jan 23, 2021
Maintainer

ok, great; good to know that it takes a few seconds, I did not notice that myself yet; thanks for your help

ThorstenGuentherDSA Jan 23, 2021
Author

Maybe the VM I'm using is a bit slow / loaded:

ipcs -su && systemctl reload httpd && for i in {1..10}; do uptime; ipcs -su ; sleep 1; done

------ Semaphore Status --------
used arrays = 10
allocated semaphores = 508

16:58:28 up 152 days, 9:25, 4 users, load average: 1.30, 1.50, 4.10

------ Semaphore Status --------
used arrays = 9
allocated semaphores = 507

16:58:29 up 152 days, 9:25, 4 users, load average: 1.30, 1.50, 4.10