Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add mongo 7 to testing matrix #34213

Merged
merged 2 commits into from
Mar 8, 2024
Merged

chore: add mongo 7 to testing matrix #34213

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
216e0d3
chore: add mongo 7 to testing matrix
DanielVZ96 Feb 10, 2024
dcdb598
chore: add testing matrix to other workflows
DanielVZ96 Feb 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/migrations-check.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,17 @@ jobs:
# within the global constraint i.e. Django==4.2.8 in current case
# because we have global constraint of Django<4.2
django-version: ["pinned"]
mongo-version: ["4"]
mongo-version: ["4", "7"]
mysql-version: ["8"]
services:
mongo:
image: mongo:${{ matrix.mongo-version }}
ports:
- 27017:27017
# Note: Calling mongo here only works with mongo 4, in newer versions of mongo
# we'll have to use `mongosh`
# we'll have to use `mongosh`, hence the 'which mongosh mongo'.
options: >-
--health-cmd "mongo --quiet --eval 'db.runCommand(\"ping\")'"
--health-cmd "$(which mongosh mongo) --quiet --eval 'db.runCommand(\"ping\")'"
--health-interval 10s
--health-timeout 5s
--health-retries 3
Expand Down
14 changes: 14 additions & 0 deletions .github/workflows/static-assets-check.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,20 @@ jobs:
python-version: [ 3.8 ]
node-version: [ 16 ]
npm-version: [ 8.5.x ]
mongo-version: ["4.4", "7.0"]

services:
mongo:
image: mongo:${{ matrix.mongo-version }}
ports:
- 27017:27017
# Note: Calling mongo here only works with mongo 4, in newer versions of mongo
# we'll have to use `mongosh`, hence the 'which mongosh mongo'.
options: >-
--health-cmd "$(which mongosh mongo) --quiet --eval 'db.runCommand(\"ping\")'"
--health-interval 10s
--health-timeout 5s
--health-retries 3

steps:
- name: Checkout repo
Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/unit-tests-gh-hosted.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,10 @@ jobs:
- "common-with-cms"
- "xmodule-with-lms"
- "xmodule-with-cms"
name: gh-hosted-python-${{ matrix.python-version }},django-${{ matrix.django-version }},${{ matrix.shard_name }}
mongo-version:
- "4.4"
- "7.0"
name: gh-hosted-python-${{ matrix.python-version }},django-${{ matrix.django-version }},mongo-${{ matrix.mongo-version }}${{ matrix.shard_name }}
steps:
- uses: actions/checkout@v2

Expand All @@ -46,7 +49,7 @@ jobs:
- name: Start MongoDB
uses: supercharge/[email protected]
with:
mongodb-version: 4.4
mongodb-version: ${{ matrix.mongo-version}}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please open a temporary PR against our fork so that we can verify this workflow?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done. here's the PR open-craft#639


- name: Setup Python
uses: actions/setup-python@v4
Expand Down
15 changes: 14 additions & 1 deletion .github/workflows/unit-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:

jobs:
run-tests:
name: python-${{ matrix.python-version }},django-${{ matrix.django-version }},${{ matrix.shard_name }}
name: python-${{ matrix.python-version }},django-${{ matrix.django-version }},mongo-${{ matrix.mongo-version }},${{ matrix.shard_name }}
if: (github.repository == 'edx/edx-platform-private') || (github.repository == 'openedx/edx-platform' && (startsWith(github.base_ref, 'open-release') == false))
runs-on: [ edx-platform-runner ]
strategy:
Expand Down Expand Up @@ -36,13 +36,26 @@ jobs:
- "common-with-cms"
- "xmodule-with-lms"
- "xmodule-with-cms"
mongo-version:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can we verify which version is running during the test? For the python-3.8,django-pinned,mongo-7.0.5,lms-1 job, I see the following logs for the "start mongod server for tests" step:

{"t":{"$date":"2024-02-28T04:48:15.954+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2024-02-28T04:48:15.955+00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2024-02-28T04:48:15.955+00:00"},"s":"I",  "c":"STORAGE",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1065,"port":27017,"dbPath":"/data/db","architecture":"64-bit","host":"edx-platform-openedx-ci-runner-deployment-7xdl7-5lvs7"}}
{"t":{"$date":"2024-02-28T04:48:15.955+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"4.4.18","gitVersion":"8ed32b5c2c68ebe7f8ae2ebe8d23f36037a17dea","openSSLVersion":"OpenSSL 1.1.1f  31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2024-02-28T04:48:15.955+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"20.04"}}}
{"t":{"$date":"2024-02-28T04:48:15.955+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{}}}
{"t":{"$date":"2024-02-28T04:48:15.956+00:00"},"s":"I",  "c":"STORAGE",  "id":22315,   "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=31124M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],"}}
{"t":{"$date":"2024-02-28T04:48:16.561+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1709095696:561957][1065:0x7fce91670cc0], txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Set global recovery timestamp: (0, 0)"}}
{"t":{"$date":"2024-02-28T04:48:16.562+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1709095696:562004][1065:0x7fce91670cc0], txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Set global oldest timestamp: (0, 0)"}}
{"t":{"$date":"2024-02-28T04:48:16.570+00:00"},"s":"I",  "c":"STORAGE",  "id":4795906, "ctx":"initandlisten","msg":"WiredTiger opened","attr":{"durationMillis":614}}
{"t":{"$date":"2024-02-28T04:48:16.570+00:00"},"s":"I",  "c":"RECOVERY", "id":23987,   "ctx":"initandlisten","msg":"WiredTiger recoveryTimestamp","attr":{"recoveryTimestamp":{"$timestamp":{"t":0,"i":0}}}}
{"t":{"$date":"2024-02-28T04:48:16.581+00:00"},"s":"I",  "c":"STORAGE",  "id":22262,   "ctx":"initandlisten","msg":"Timestamp monitor starting"}
{"t":{"$date":"2024-02-28T04:48:16.590+00:00"},"s":"W",  "c":"CONTROL",  "id":22120,   "ctx":"initandlisten","msg":"Access control is not enabled for the database. Read and write access to data and configuration is unrestricted","tags":["startupWarnings"]}
{"t":{"$date":"2024-02-28T04:48:16.590+00:00"},"s":"W",  "c":"CONTROL",  "id":22140,   "ctx":"initandlisten","msg":"This server is bound to localhost. Remote systems will be unable to connect to this server. Start the server with --bind_ip <address> to specify which IP addresses it should serve responses from, or with --bind_ip_all to bind to all interfaces. If this behavior is desired, start the server with --bind_ip 127.0.0.1 to disable this warning","tags":["startupWarnings"]}
{"t":{"$date":"2024-02-28T04:48:16.591+00:00"},"s":"I",  "c":"STORAGE",  "id":20320,   "ctx":"initandlisten","msg":"createCollection","attr":{"namespace":"admin.system.version","uuidDisposition":"provided","uuid":{"uuid":{"$uuid":"c9f0fb9a-7861-453c-bc1d-d99e261af668"}},"options":{"uuid":{"$uuid":"c9f0fb9a-7861-453c-bc1d-d99e261af668"}}}}
{"t":{"$date":"2024-02-28T04:48:16.601+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"initandlisten","msg":"Index build: done building","attr":{"buildUUID":null,"namespace":"admin.system.version","index":"_id_","commitTimestamp":{"$timestamp":{"t":0,"i":0}}}}
{"t":{"$date":"2024-02-28T04:48:16.601+00:00"},"s":"I",  "c":"COMMAND",  "id":20459,   "ctx":"initandlisten","msg":"Setting featureCompatibilityVersion","attr":{"newVersion":"4.4"}}
{"t":{"$date":"2024-02-28T04:48:16.601+00:00"},"s":"I",  "c":"STORAGE",  "id":20536,   "ctx":"initandlisten","msg":"Flow Control is enabled on this deployment"}
{"t":{"$date":"2024-02-28T04:48:16.601+00:00"},"s":"I",  "c":"STORAGE",  "id":20320,   "ctx":"initandlisten","msg":"createCollection","attr":{"namespace":"local.startup_log","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"ffb7d724-ce57-4302-803c-07535cc87b23"}},"options":{"capped":true,"size":10485760}}}
{"t":{"$date":"2024-02-28T04:48:16.614+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"initandlisten","msg":"Index build: done building","attr":{"buildUUID":null,"namespace":"local.startup_log","index":"_id_","commitTimestamp":{"$timestamp":{"t":0,"i":0}}}}
{"t":{"$date":"2024-02-28T04:48:16.614+00:00"},"s":"I",  "c":"FTDC",     "id":20625,   "ctx":"initandlisten","msg":"Initializing full-time diagnostic data capture","attr":{"dataDirectory":"/data/db/diagnostic.data"}}
{"t":{"$date":"2024-02-28T04:48:16.614+00:00"},"s":"I",  "c":"REPL",     "id":6015317, "ctx":"initandlisten","msg":"Setting new configuration state","attr":{"newState":"ConfigReplicationDisabled","oldState":"ConfigPreStart"}}
{"t":{"$date":"2024-02-28T04:48:16.615+00:00"},"s":"I",  "c":"NETWORK",  "id":23015,   "ctx":"listener","msg":"Listening on","attr":{"address":"/tmp/mongodb-27017.sock"}}
{"t":{"$date":"2024-02-28T04:48:16.615+00:00"},"s":"I",  "c":"NETWORK",  "id":23015,   "ctx":"listener","msg":"Listening on","attr":{"address":"127.0.0.1"}}
{"t":{"$date":"2024-02-28T04:48:16.615+00:00"},"s":"I",  "c":"NETWORK",  "id":23016,   "ctx":"listener","msg":"Waiting for connections","attr":{"port":27017,"ssl":"off"}}
{"t":{"$date":"2024-02-28T04:48:16.615+00:00"},"s":"I",  "c":"CONTROL",  "id":20712,   "ctx":"LogicalSessionCacheReap","msg":"Sessions collection is not set up; waiting until next sessions reap interval","attr":{"error":"NamespaceNotFound: config.system.sessions does not exist"}}
{"t":{"$date":"2024-02-28T04:48:16.615+00:00"},"s":"I",  "c":"STORAGE",  "id":20320,   "ctx":"LogicalSessionCacheRefresh","msg":"createCollection","attr":{"namespace":"config.system.sessions","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"435f4dd4-27c0-4d87-b52b-b3f9add33bc8"}},"options":{}}}
{"t":{"$date":"2024-02-28T04:48:16.638+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"LogicalSessionCacheRefresh","msg":"Index build: done building","attr":{"buildUUID":null,"namespace":"config.system.sessions","index":"_id_","commitTimestamp":{"$timestamp":{"t":0,"i":0}}}}
{"t":{"$date":"2024-02-28T04:48:16.638+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"LogicalSessionCacheRefresh","msg":"Index build: done building","attr":{"buildUUID":null,"namespace":"config.system.sessions","index":"lsidTTLIndex","commitTimestamp":{"$timestamp":{"t":0,"i":0}}}}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, turns out I had to run apt purge before installing the new version.

{"version":"7.0.6","gitVersion":"66cdc1f28172cb33ff68263050d73d4ade73b9a4","openSSLVersion":"OpenSSL 1.1.1f  31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}}

- "4.4"
- "7.0"
# We expect Django 4.0 to fail, so don't stop when it fails.
continue-on-error: ${{ matrix.django-version == '4.0' }}

steps:
- name: sync directory owner
run: sudo chown runner:runner -R .*

- name: install mongo version
run: |
if [[ "${{ matrix.mongo-version }}" != "4.4" ]]; then
sudo apt-get purge -y "mongodb-org*"
sudo apt-get remove -y mongodb-org
wget -qO - https://www.mongodb.org/static/pgp/server-${{ matrix.mongo-version }}.asc | sudo apt-key add -
echo "deb https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/${{ matrix.mongo-version }} multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-${{ matrix.mongo-version }}.list
sudo apt-get update && sudo apt-get install -y mongodb-org="${{ matrix.mongo-version }}.*"
fi

- name: checkout repo
uses: actions/checkout@v3

Expand Down
Loading