chore: add mongo 7 to testing matrix

[DanielVZ96]

Description

MongoDB will enter EOL in February 2024. Because of this, depending on the vendor, managed mongo atlas instances will be forced to upgrade to newer releases. We've been testing mongo 7.0 and it works fine in a local and atlas environment.

Impacted user roles: developers

Supporting information

• https://www.mongodb.com/legal/support-policy/lifecycles

Testing instructions

Open a PR and verify that CI runs well.

Deadline

None.

Other notes

Private-ref: BB-8450

[openedx-webhooks]

Thanks for the pull request, @DanielVZ96! Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket as you can:

• supporting documentation
• Open edX discussion forum threads
• timeline information ("this must be merged by XX date", and why that is)
• partner information ("this is a course on edx.org")
• any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

[Agrendalath]

How can we verify which version is running during the test? For the python-3.8,django-pinned,mongo-7.0.5,lms-1 job, I see the following logs for the "start mongod server for tests" step:

{"t":{"$date":"2024-02-28T04:48:15.954+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2024-02-28T04:48:15.955+00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2024-02-28T04:48:15.955+00:00"},"s":"I",  "c":"STORAGE",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1065,"port":27017,"dbPath":"/data/db","architecture":"64-bit","host":"edx-platform-openedx-ci-runner-deployment-7xdl7-5lvs7"}}
{"t":{"$date":"2024-02-28T04:48:15.955+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"4.4.18","gitVersion":"8ed32b5c2c68ebe7f8ae2ebe8d23f36037a17dea","openSSLVersion":"OpenSSL 1.1.1f  31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2024-02-28T04:48:15.955+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"20.04"}}}
{"t":{"$date":"2024-02-28T04:48:15.955+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{}}}
{"t":{"$date":"2024-02-28T04:48:15.956+00:00"},"s":"I",  "c":"STORAGE",  "id":22315,   "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=31124M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],"}}
{"t":{"$date":"2024-02-28T04:48:16.561+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1709095696:561957][1065:0x7fce91670cc0], txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Set global recovery timestamp: (0, 0)"}}
{"t":{"$date":"2024-02-28T04:48:16.562+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1709095696:562004][1065:0x7fce91670cc0], txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Set global oldest timestamp: (0, 0)"}}
{"t":{"$date":"2024-02-28T04:48:16.570+00:00"},"s":"I",  "c":"STORAGE",  "id":4795906, "ctx":"initandlisten","msg":"WiredTiger opened","attr":{"durationMillis":614}}
{"t":{"$date":"2024-02-28T04:48:16.570+00:00"},"s":"I",  "c":"RECOVERY", "id":23987,   "ctx":"initandlisten","msg":"WiredTiger recoveryTimestamp","attr":{"recoveryTimestamp":{"$timestamp":{"t":0,"i":0}}}}
{"t":{"$date":"2024-02-28T04:48:16.581+00:00"},"s":"I",  "c":"STORAGE",  "id":22262,   "ctx":"initandlisten","msg":"Timestamp monitor starting"}
{"t":{"$date":"2024-02-28T04:48:16.590+00:00"},"s":"W",  "c":"CONTROL",  "id":22120,   "ctx":"initandlisten","msg":"Access control is not enabled for the database. Read and write access to data and configuration is unrestricted","tags":["startupWarnings"]}
{"t":{"$date":"2024-02-28T04:48:16.590+00:00"},"s":"W",  "c":"CONTROL",  "id":22140,   "ctx":"initandlisten","msg":"This server is bound to localhost. Remote systems will be unable to connect to this server. Start the server with --bind_ip <address> to specify which IP addresses it should serve responses from, or with --bind_ip_all to bind to all interfaces. If this behavior is desired, start the server with --bind_ip 127.0.0.1 to disable this warning","tags":["startupWarnings"]}
{"t":{"$date":"2024-02-28T04:48:16.591+00:00"},"s":"I",  "c":"STORAGE",  "id":20320,   "ctx":"initandlisten","msg":"createCollection","attr":{"namespace":"admin.system.version","uuidDisposition":"provided","uuid":{"uuid":{"$uuid":"c9f0fb9a-7861-453c-bc1d-d99e261af668"}},"options":{"uuid":{"$uuid":"c9f0fb9a-7861-453c-bc1d-d99e261af668"}}}}
{"t":{"$date":"2024-02-28T04:48:16.601+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"initandlisten","msg":"Index build: done building","attr":{"buildUUID":null,"namespace":"admin.system.version","index":"_id_","commitTimestamp":{"$timestamp":{"t":0,"i":0}}}}
{"t":{"$date":"2024-02-28T04:48:16.601+00:00"},"s":"I",  "c":"COMMAND",  "id":20459,   "ctx":"initandlisten","msg":"Setting featureCompatibilityVersion","attr":{"newVersion":"4.4"}}
{"t":{"$date":"2024-02-28T04:48:16.601+00:00"},"s":"I",  "c":"STORAGE",  "id":20536,   "ctx":"initandlisten","msg":"Flow Control is enabled on this deployment"}
{"t":{"$date":"2024-02-28T04:48:16.601+00:00"},"s":"I",  "c":"STORAGE",  "id":20320,   "ctx":"initandlisten","msg":"createCollection","attr":{"namespace":"local.startup_log","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"ffb7d724-ce57-4302-803c-07535cc87b23"}},"options":{"capped":true,"size":10485760}}}
{"t":{"$date":"2024-02-28T04:48:16.614+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"initandlisten","msg":"Index build: done building","attr":{"buildUUID":null,"namespace":"local.startup_log","index":"_id_","commitTimestamp":{"$timestamp":{"t":0,"i":0}}}}
{"t":{"$date":"2024-02-28T04:48:16.614+00:00"},"s":"I",  "c":"FTDC",     "id":20625,   "ctx":"initandlisten","msg":"Initializing full-time diagnostic data capture","attr":{"dataDirectory":"/data/db/diagnostic.data"}}
{"t":{"$date":"2024-02-28T04:48:16.614+00:00"},"s":"I",  "c":"REPL",     "id":6015317, "ctx":"initandlisten","msg":"Setting new configuration state","attr":{"newState":"ConfigReplicationDisabled","oldState":"ConfigPreStart"}}
{"t":{"$date":"2024-02-28T04:48:16.615+00:00"},"s":"I",  "c":"NETWORK",  "id":23015,   "ctx":"listener","msg":"Listening on","attr":{"address":"/tmp/mongodb-27017.sock"}}
{"t":{"$date":"2024-02-28T04:48:16.615+00:00"},"s":"I",  "c":"NETWORK",  "id":23015,   "ctx":"listener","msg":"Listening on","attr":{"address":"127.0.0.1"}}
{"t":{"$date":"2024-02-28T04:48:16.615+00:00"},"s":"I",  "c":"NETWORK",  "id":23016,   "ctx":"listener","msg":"Waiting for connections","attr":{"port":27017,"ssl":"off"}}
{"t":{"$date":"2024-02-28T04:48:16.615+00:00"},"s":"I",  "c":"CONTROL",  "id":20712,   "ctx":"LogicalSessionCacheReap","msg":"Sessions collection is not set up; waiting until next sessions reap interval","attr":{"error":"NamespaceNotFound: config.system.sessions does not exist"}}
{"t":{"$date":"2024-02-28T04:48:16.615+00:00"},"s":"I",  "c":"STORAGE",  "id":20320,   "ctx":"LogicalSessionCacheRefresh","msg":"createCollection","attr":{"namespace":"config.system.sessions","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"435f4dd4-27c0-4d87-b52b-b3f9add33bc8"}},"options":{}}}
{"t":{"$date":"2024-02-28T04:48:16.638+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"LogicalSessionCacheRefresh","msg":"Index build: done building","attr":{"buildUUID":null,"namespace":"config.system.sessions","index":"_id_","commitTimestamp":{"$timestamp":{"t":0,"i":0}}}}
{"t":{"$date":"2024-02-28T04:48:16.638+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"LogicalSessionCacheRefresh","msg":"Index build: done building","attr":{"buildUUID":null,"namespace":"config.system.sessions","index":"lsidTTLIndex","commitTimestamp":{"$timestamp":{"t":0,"i":0}}}}

[DanielVZ96]

good catch, turns out I had to run apt purge before installing the new version.

{"version":"7.0.6","gitVersion":"66cdc1f28172cb33ff68263050d73d4ade73b9a4","openSSLVersion":"OpenSSL 1.1.1f  31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}}

[Agrendalath]

Could you please open a temporary PR against our fork so that we can verify this workflow?

[DanielVZ96]

done. here's the PR open-craft#639

[Agrendalath]

👍

• I tested this: checked that the CI uses MongoDB 7, and the tests are passing
• I read through the code
• I checked for accessibility issues: n/a
• Includes documentation: n/a

[Agrendalath]

@feanil, this is the PR we briefly mentioned during the first Maintenance WG meeting. Would you like to take a look before we merge it?

[feanil]

@Agrendalath thanks for the ping, this is a nice way to work around the hosted runners not having the correct mongo version installed and it looks like it did not significantly effect the test run time.

[feanil]

@Agrendalath @DanielVZ96 for existing databases, were there any complexities in upgrading the database that we should document as operational notes in https://openedx.atlassian.net/wiki/spaces/COMM/pages/3890380898/Next+Release+Redwood ?

[Agrendalath]

@feanil, we upgraded our clusters step-by-step (i.e., 4.4 -> 5.0 -> 6.0 -> 7.0) because it was the limitation of MongoDB Atlas. I only noticed that the failover sometimes failed for the forum MongoDB driver during the 6.0 upgrade. We've been running 7.0 on production instances for over a week (with no changes other than restarting the forum services) and did not observe any regressions.

[feanil]

Thanks for that info, FYI, I think this is good to merge whenever you're ready to do that. Not sure if you were waiting for anything else before this lands.

[Agrendalath]

@feanil, I just didn't want to merge PRs before taking the last two days off. I'll merge this one tomorrow.

[edx-pipeline-bot]

2U Release Notice: This PR has been deployed to the edX staging environment in preparation for a release to production.

[openedx-webhooks]

@DanielVZ96 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.

[edx-pipeline-bot]

2U Release Notice: This PR has been deployed to the edX production environment.

[edx-pipeline-bot]

2U Release Notice: This PR has been deployed to the edX staging environment in preparation for a release to production.

[edx-pipeline-bot]

2U Release Notice: This PR has been deployed to the edX production environment.

[edx-pipeline-bot]

2U Release Notice: This PR has been deployed to the edX staging environment in preparation for a release to production.

[edx-pipeline-bot]

2U Release Notice: This PR has been deployed to the edX production environment.

[edx-pipeline-bot]

2U Release Notice: This PR has been deployed to the edX production environment.