Merge branch 'master' into php8.1-upgrade

# Conflicts:
#	setup/nextcloud.sh

.github/workflows/commit-tests.yml

@@ -15,7 +15,7 @@ jobs:
         run: sudo -E tests/system-setup/remote-nextcloud-docker.sh
       - name: test-runner
         run: sudo -E tests/runner.sh -dumpoutput -no-smtp-remote remote-nextcloud ehdd default
-      - uses: actions/upload-artifact@v3
+      - uses: actions/upload-artifact@v4
         if: failure()
         with:
           name: tests-output

CHANGELOG.md

@@ -1,6 +1,23 @@
 CHANGELOG
 =========
 
+Version 69 (July 20, 2024)
+--------------------------
+
+Package updates:
+
+* Nextcloud is updated to 26.0.13.
+* Z-Push is updated to 2.7.3.
+
+Other updates:
+
+* Fixed an error generating the weekly statistics.
+* Fixed file permissions when setting up Nextcloud.
+* Added an undocumented option to proxy websockets.
+* Internal improvements to the code to make it more reliable and readable.
+
+Version 69a (July 21, 2024) and 69b (July 23, 2024) correct setup failures.
+
 Version 68 (April 1, 2024)
 --------------------------
 
@@ -25,7 +42,6 @@ Other:
 * fail2ban is updated to see "HTTP/2.0" requests to munin also.
 * Internal improvements to the code to make it more reliable and readable.
 
-
 Version 67 (December 22, 2023)
 ------------------------------
 
@@ -188,7 +204,7 @@ Other:
 
 * Set systemd journald log retention to 10 days (from no limit) to reduce disk usage.
 * Fixed log processing for submission lines that have a sasl_sender or other extra information.
-* Fix DNS secondary nameserver refesh failure retry period.
+* Fix DNS secondary nameserver refresh failure retry period.
 
 Version 55 (October 18, 2021)
 -----------------------------
@@ -213,7 +229,7 @@ Control panel:
 Other:
 
 * Fail2ban's IPv6 support is enabled.
-* The mail log tool now doesn't crash if there are email addresess in log messages with invalid UTF-8 characters.
+* The mail log tool now doesn't crash if there are email addresses in log messages with invalid UTF-8 characters.
 * Additional nsd.conf files can be placed in /etc/nsd.conf.d.
 
 v0.54 (June 20, 2021)
@@ -246,7 +262,7 @@ Setup:
 v0.53a (May 8, 2021)
 --------------------
 
-The download URL for Z-Push has been revised becaue the old URL stopped working.
+The download URL for Z-Push has been revised because the old URL stopped working.
 
 v0.53 (April 12, 2021)
 ----------------------
@@ -465,7 +481,7 @@ Changes:
 * Added support for S3-compatible backup services besides Amazon S3.
 * Fixed the control panel login page to let LastPass save passwords.
 * Fixed an error in the user privileges API.
-* Silenced some spurrious messages.
+* Silenced some spurious messages.
 
 Software updates:
 
@@ -529,7 +545,7 @@ Setup:
 
 Control Panel:
 
-* The users page now documents that passwords should only have ASCII characters to prevent character encoding mismaches between clients and the server.
+* The users page now documents that passwords should only have ASCII characters to prevent character encoding mismatches between clients and the server.
 * The users page no longer shows user mailbox sizes because this was extremely slow for very large mailboxes.
 * The Mail-in-a-Box version is now shown in the system status checks even when the new-version check is disabled.
 * The alises page now warns that alises should not be used to forward mail off of the box. Mail filters within Roundcube are better for that.
@@ -857,7 +873,7 @@ v0.17c (April 1, 2016)
 
 This update addresses some minor security concerns and some installation issues.
 
-ownCoud:
+ownCloud:
 
 * Block web access to the configuration parameters (config.php). There is no immediate impact (see [#776](https://github.com/mail-in-a-box/mailinabox/pull/776)), although advanced users may want to take note.
 
@@ -873,7 +889,7 @@ Control panel:
 Setup:
 
 * Setup dialogs did not appear correctly when connecting to SSH using Putty on Windows.
-* We now install Roundcube from our own mirror because Sourceforge's downloads experience frequent intermittant unavailability.
+* We now install Roundcube from our own mirror because Sourceforge's downloads experience frequent intermittent unavailability.
 
 v0.17b (March 1, 2016)
 ----------------------
@@ -916,7 +932,7 @@ This update primarily adds automatic SSL (now "TLS") certificate provisioning fr
 
 Control Panel:
 
-* The SSL certificates (now referred to as "TLS ccertificates") page now supports provisioning free certificates from Let's Encrypt.
+* The SSL certificates (now referred to as "TLS certificates") page now supports provisioning free certificates from Let's Encrypt.
 * Report free memory usage.
 * Fix a crash when the git directory is not checked out to a tag.
 * When IPv6 is enabled, check that all domains (besides the system hostname) resolve over IPv6.
@@ -1009,7 +1025,7 @@ Control panel:
 System:
 
 * Tweaks to fail2ban settings.
-* Fixed a spurrious warning while installing munin.
+* Fixed a spurious warning while installing munin.
 
 v0.13b (August 30, 2015)
 ------------------------
@@ -1023,7 +1039,7 @@ Note: v0.13 (no 'a', August 19, 2015) was pulled immediately due to an ownCloud
 
 Mail:
 
-* Outbound mail headers (the Recieved: header) are tweaked to possibly improve deliverability.
+* Outbound mail headers (the Received: header) are tweaked to possibly improve deliverability.
 * Some MIME messages would hang Roundcube due to a missing package.
 * The users permitted to send as an alias can now be different from where an alias forwards to.
 
@@ -1055,7 +1071,7 @@ v0.12c was posted to work around the current Sourceforge.net outage: pyzor's rem
 v0.12b (July 4, 2015)
 ---------------------
 
-This version corrects a minor regression in v0.12 related to creating aliases targetting multiple addresses.
+This version corrects a minor regression in v0.12 related to creating aliases targeting multiple addresses.
 
 v0.12 (July 3, 2015)
 --------------------

CODE_OF_CONDUCT.md

@@ -2,13 +2,13 @@
 
 Mail-in-a-Box is an open source community project about working, as a group, to empower ourselves and others to have control over our own digital communications. Just as we hope to increase technological diversity on the Internet through decentralization, we also believe that diverse viewpoints and voices among our community members foster innovation and creative solutions to the challenges we face.
 
-We are committed to providing a safe, welcoming, and harrassment-free space for collaboration, for everyone, without regard to age, disability, economic situation, ethnicity, gender identity and expression, language fluency, level of knowledge or experience, nationality, personal appearance, race, religion, sexual identity and orientation, or any other attribute. Community comes first. This policy supersedes all other project goals.
+We are committed to providing a safe, welcoming, and harassment-free space for collaboration, for everyone, without regard to age, disability, economic situation, ethnicity, gender identity and expression, language fluency, level of knowledge or experience, nationality, personal appearance, race, religion, sexual identity and orientation, or any other attribute. Community comes first. This policy supersedes all other project goals.
 
 The maintainers of Mail-in-a-Box share the dual responsibility of leading by example and enforcing these policies as necessary to maintain an open and welcoming environment. All community members should be excellent to each other.
 
 ## Scope
 
-This Code of Conduct applies to all places where Mail-in-a-Box community activity is ocurring, including on GitHub, in discussion forums, on Slack, on social media, and in real life. The Code of Conduct applies not only on websites/at events run by the Mail-in-a-Box community (e.g. our GitHub organization, our Slack team) but also at any other location where the Mail-in-a-Box community is present (e.g. in issues of other GitHub organizations where Mail-in-a-Box community members are discussing problems related to Mail-in-a-Box, or real-life professional conferences), or whenever a Mail-in-a-Box community member is representing Mail-in-a-Box to the public at large or acting on behalf of Mail-in-a-Box.
+This Code of Conduct applies to all places where Mail-in-a-Box community activity is occurring, including on GitHub, in discussion forums, on Slack, on social media, and in real life. The Code of Conduct applies not only on websites/at events run by the Mail-in-a-Box community (e.g. our GitHub organization, our Slack team) but also at any other location where the Mail-in-a-Box community is present (e.g. in issues of other GitHub organizations where Mail-in-a-Box community members are discussing problems related to Mail-in-a-Box, or real-life professional conferences), or whenever a Mail-in-a-Box community member is representing Mail-in-a-Box to the public at large or acting on behalf of Mail-in-a-Box.
 
 This code does not apply to activity on a server running Mail-in-a-Box software, unless your server is hosting a service for the Mail-in-a-Box community at large.
 

changelog/v69a.md

@@ -0,0 +1,37 @@
+## Commits for v69a
+| COMMIT | DATE | AUTHOR | TITLE |
+| ------ | ---- | ------ | ----- |
+| [2ae8cd57](https://github.com/downtownallday/mailinabox-ldap/commit/2ae8cd57130df86722e2c6896f78bd52dbe8fb89) | 2024-07-21 | _Joshua Tauberer_ | v69a |
+| [bc14e80b](https://github.com/downtownallday/mailinabox-ldap/commit/bc14e80b12aa612db950cb9b7412216e5569c00d) | 2024-07-21 | _Teal Dulcet_ | Fix no password prompt. Fixes #2408 (#2409) |
+| [cd959bc5](https://github.com/downtownallday/mailinabox-ldap/commit/cd959bc52263935930f5175b7c5a13215ffae205) | 2024-07-21 | _Viktor Szépe_ | Fix typos (#2406) |
+| [2803d888](https://github.com/downtownallday/mailinabox-ldap/commit/2803d8889454fa3a8295c0cfd382cdbe996284d9) | 2024-07-20 | _KiekerJan_ | increase timeout for the nginx proxy that provides access to the Mailinabox management daemon (#2407) |
+| [1b3e5e81](https://github.com/downtownallday/mailinabox-ldap/commit/1b3e5e818c2272fb3b71afa6087ea3d4601125b0) | 2024-07-20 | _Joshua Tauberer_ | v69 |
+| [2f5e736f](https://github.com/downtownallday/mailinabox-ldap/commit/2f5e736fa0714cc9870dcc284bc26811e4a0148f) | 2024-07-20 | _Joshua Tauberer_ | Clean up Nextcloud email settings for calendar invitations |
+| [f118a6c0](https://github.com/downtownallday/mailinabox-ldap/commit/f118a6c0bfc40c602118bfc923e226f31acd2117) | 2024-07-08 | _Michael Heuberger_ | Apply small Nextcloud upgrade to 26.0.13 (#2401) |
+| [25439e2b](https://github.com/downtownallday/mailinabox-ldap/commit/25439e2bf7d22a7f1888eff167c0c22fcec5dfcb) | 2024-06-18 | _downtownallday_ | QA: update to Actions upload-artifact v4 |
+| [a685955f](https://github.com/downtownallday/mailinabox-ldap/commit/a685955f2963481fcb674d1e42aa557c35b218de) | 2024-06-18 | _downtownallday_ | QA: attempt fixing logout click intercepted exception on github actions #2 |
+| [626e2290](https://github.com/downtownallday/mailinabox-ldap/commit/626e2290e74719715e53db9bb20fb04bd3ca6262) | 2024-06-18 | _downtownallday_ | QA: attempt fixing logout click intercepted exception on github actions |
+| [de0fc796](https://github.com/downtownallday/mailinabox-ldap/commit/de0fc796d43f2fea0655036c16b9aa0f594f340e) | 2024-06-18 | _jvolkenant_ | Fix chown during Nexcloud upgrades (#2377) |
+| [4dd1e75e](https://github.com/downtownallday/mailinabox-ldap/commit/4dd1e75ee7100c3eb5c8937595cb019f78f52566) | 2024-06-18 | _Matt_ | Allow for `Union[None, List[datetime.datetime]]` values when printing user table in weekly mail logs (#2378) |
+| [8b9f0489](https://github.com/downtownallday/mailinabox-ldap/commit/8b9f0489c8cb412458376089a0b59aa30309325b) | 2024-06-18 | _John James Jacoby_ | Add custom.yaml support for WebSockets (#2385) |
+| [6321ce6e](https://github.com/downtownallday/mailinabox-ldap/commit/6321ce6ef0876bef6124d13fd068268b1856517c) | 2024-06-18 | _matidau_ | Add php8.0-intl package to z-push setup (#2389) |
+| [30d78cd3](https://github.com/downtownallday/mailinabox-ldap/commit/30d78cd35a33bb68324e25c7e72ee3048ec46330) | 2024-06-18 | _matidau_ | Update zpush.sh to version 2.7.3 (#2390) |
+| [53a6cc15](https://github.com/downtownallday/mailinabox-ldap/commit/53a6cc1501bea67219f8ccb5becf2b9f09c98cf9) | 2024-04-15 | _Matt_ | Fix - Allow for `Union[None, List[datetime.datetime]]` when printing user tables for the weekly mail logs. |
+| [c9c0a51b](https://github.com/downtownallday/mailinabox-ldap/commit/c9c0a51be2649d2a2ef253b77e34cde06648fe4c) | 2024-04-03 | _downtownallday_ | Create temporary file in TMP not the current working directory |
+| [95f23961](https://github.com/downtownallday/mailinabox-ldap/commit/95f23961dc9b59d304aedecc1e9c5f34c0eb7d80) | 2024-04-03 | _downtownallday_ | QA: wait for first run wizard to display |
+| [dbc7d87f](https://github.com/downtownallday/mailinabox-ldap/commit/dbc7d87f4b2827d598c1ea3ba1ff08cedee35249) | 2024-04-03 | _downtownallday_ | Actions: update checkout to v4 and remove duplicate test suite |
+| [03e70941](https://github.com/downtownallday/mailinabox-ldap/commit/03e70941f53aa1fb6979f28ecd01e5be007b3022) | 2024-04-03 | _downtownallday_ | QA: redirect command output to test output file |
+| [c7faccf1](https://github.com/downtownallday/mailinabox-ldap/commit/c7faccf1fa9fc9deb6051e96acb13bb120b50c49) | 2024-04-03 | _Teal Dulcet_ | Fixed SC2244: Prefer explicit -n to check non-empty string. |
+| [ec497efa](https://github.com/downtownallday/mailinabox-ldap/commit/ec497efa69e9cbad466519c21e5cdaa4ea6fd57f) | 2024-04-03 | _Teal Dulcet_ | Quote echo commands to preserve whitespace. |
+| [55a8be4a](https://github.com/downtownallday/mailinabox-ldap/commit/55a8be4aa97508c3e7c453da06d8dcbe6b0e7993) | 2024-04-03 | _Teal Dulcet_ | Removed unnecessary bc commands. |
+| [3399b250](https://github.com/downtownallday/mailinabox-ldap/commit/3399b250845bc1f4303a3a9715bdd7c2e449a2b6) | 2024-04-03 | _Teal Dulcet_ | Replaced the pwd command with Bash's $PWD variable. |
+| [2afd0451](https://github.com/downtownallday/mailinabox-ldap/commit/2afd0451c13a03bb84fc947cacf8f1929c1d6006) | 2024-04-03 | _Teal Dulcet_ | Fixed SC2007: Use $((..)) instead of deprecated $[..]. |
+| [27cf11d8](https://github.com/downtownallday/mailinabox-ldap/commit/27cf11d8ec6f554749da60ae781c2668612efe35) | 2024-04-03 | _Teal Dulcet_ | Fixed SC2005: Useless echo. |
+| [44d9f6ee](https://github.com/downtownallday/mailinabox-ldap/commit/44d9f6eebd14e150b6f17a4cdb7f25470b092485) | 2024-04-03 | _Teal Dulcet_ | Fixed SC2236: Use -n instead of ! -z. |
+| [4b7d4ba0](https://github.com/downtownallday/mailinabox-ldap/commit/4b7d4ba0a6a8b96e7ee38077c53636adf514de93) | 2024-04-03 | _Teal Dulcet_ | Fixed SC2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined. |
+| [67bcaea7](https://github.com/downtownallday/mailinabox-ldap/commit/67bcaea71e8d59e34787f5fd708ab4f664967e9d) | 2024-04-03 | _Teal Dulcet_ | Fixed SC2091: Remove surrounding $() to avoid executing output. |
+| [bdf4155b](https://github.com/downtownallday/mailinabox-ldap/commit/bdf4155bedccb36117a41bd7d0952c1559e4da67) | 2024-04-03 | _Teal Dulcet_ | Fixed SC2046: Quote to prevent word splitting. |
+| [f1888f20](https://github.com/downtownallday/mailinabox-ldap/commit/f1888f2043b0ff2a555ac885d2d81aa0ec5f86a9) | 2024-04-03 | _Teal Dulcet_ | Fixed SC2148: Add a shebang. |
+| [33559bb8](https://github.com/downtownallday/mailinabox-ldap/commit/33559bb84433b4ca78959c40a5de46444f5f6dc3) | 2024-04-03 | _Teal Dulcet_ | Fixed SC2164: Use 'cd ... || exit' in case cd fails. |
+| [30c4681e](https://github.com/downtownallday/mailinabox-ldap/commit/30c4681e802f32c4d0f95c22895abb329ba9d85e) | 2024-04-03 | _Teal Dulcet_ | Fixed SC2086: Double quote to prevent globbing and word splitting. |
+| [133bae13](https://github.com/downtownallday/mailinabox-ldap/commit/133bae13001bc2ba5f5279f9286e5d118bc5c4aa) | 2024-04-03 | _Teal Dulcet_ | Fixed SC2006: Use $(...) notation instead of legacy backticks `...`. |

changelog/v69b.md

@@ -0,0 +1,5 @@
+## Commits for v69b
+| COMMIT | DATE | AUTHOR | TITLE |
+| ------ | ---- | ------ | ----- |
+| [162e509b](https://github.com/downtownallday/mailinabox-ldap/commit/162e509b8bf0683a70cf92fb8b7623c0dba5d051) | 2024-07-23 | _Joshua Tauberer_ | v69b |
+| [60a2b58e](https://github.com/downtownallday/mailinabox-ldap/commit/60a2b58e5735308c1f6f3292271948f8730b9559) | 2024-07-23 | _Joshua Tauberer_ | Revert "Fixed SC2091: Remove surrounding $() to avoid executing output." and fix it another way |

conf/dovecot-mailboxes.conf

@@ -52,7 +52,7 @@ namespace inbox {
 
   # dovevot's standard mailboxes configuration file marks two sent folders
   # with the \Sent attribute, just in case clients don't agree about which
-  # they're using. We'll keep that, plus add Junk as an alterative for Spam.
+  # they're using. We'll keep that, plus add Junk as an alternative for Spam.
   # These are not auto-created.
   mailbox "Sent Messages" {
     special_use = \Sent

conf/fail2ban/jails.conf

@@ -74,7 +74,7 @@ action   = iptables-allports[name=recidive]
 # The last line on the action will sent an email to the configured address. This mail will
 # notify the administrator that someone has been repeatedly triggering one of the other jails.
 # By default we don't configure this address and no action is required from the admin anyway.
-# So the notification is ommited. This will prevent message appearing in the mail.log that mail
+# So the notification is omitted. This will prevent message appearing in the mail.log that mail
 # can't be delivered to fail2ban@$HOSTNAME.
 
 [postfix-sasl]

conf/nginx-alldomains.conf

@@ -37,7 +37,7 @@
 		return 403;
 	}
 	location ~ /mail/.*\.php {
-		# note: ~ has precendence over a regular location block
+		# note: ~ has precedence over a regular location block
 		include fastcgi_params;
 		fastcgi_split_path_info ^/mail(/.*)()$;
 		fastcgi_index index.php;

conf/nginx-primaryonly.conf

@@ -8,6 +8,7 @@
 	rewrite ^/admin/munin$ /admin/munin/ redirect;
 	location /admin/ {
 		proxy_pass http://127.0.0.1:10222/;
+		proxy_read_timeout 600s;
 		proxy_set_header X-Forwarded-For $remote_addr;
 		add_header X-Frame-Options "DENY";
 		add_header X-Content-Type-Options nosniff;
@@ -38,7 +39,7 @@
 		}
 	}
 	location ~ ^(/cloud)((?:/ocs)?/[^/]+\.php)(/.*)?$ {
-		# note: ~ has precendence over a regular location block
+		# note: ~ has precedence over a regular location block
 		# Accept URLs like:
 		# /cloud/index.php/apps/files/
 		# /cloud/index.php/apps/files/ajax/scan.php (it's really index.php; see 6fdef379adfdeac86cc2220209bdf4eb9562268d)

management/backup.py

@@ -25,7 +25,7 @@
 from utils import load_environment, shell, wait_for_service
 
 def backup_status(env):
-	# If backups are dissbled, return no status.
+	# If backups are disabled, return no status.
 	config = get_backup_config(env)
 	if config["target"] == "off":
 		return { }

management/dns_update.py

@@ -24,7 +24,7 @@
 
 # From https://stackoverflow.com/questions/3026957/how-to-validate-a-domain-name-using-regex-php/16491074#16491074
 # This regular expression matches domain names according to RFCs, it also accepts fqdn with an leading dot,
-# underscores, as well as asteriks which are allowed in domain names but not hostnames (i.e. allowed in
+# underscores, as well as asterisks which are allowed in domain names but not hostnames (i.e. allowed in
 # DNS but not in URLs), which are common in certain record types like for DKIM.
 DOMAIN_RE = r"^(?!\-)(?:[*][.])?(?:[a-zA-Z\d\-_]{0,62}[a-zA-Z\d_]\.){1,126}(?!\d+)[a-zA-Z\d_]{1,63}(\.?)$"
 
@@ -458,7 +458,7 @@ def build_sshfp_records():
 
 	# Get our local fingerprints by running ssh-keyscan. The output looks
 	# like the known_hosts file: hostname, keytype, fingerprint. The order
-	# of the output is arbitrary, so sort it to prevent spurrious updates
+	# of the output is arbitrary, so sort it to prevent spurious updates
 	# to the zone file (that trigger bumping the serial number). However,
 	# if SSH has been configured to listen on a nonstandard port, we must
 	# specify that port to sshkeyscan.