mmtests: add mmtests plugins

[chaws]

The MMtests plugin will aid projects into processing performance benchmark tests. For now, it will rely solely on downloading attachments from backends and associating them to test runs for later post-processing.

Here is a sample of it working with my local squad instance. It download attachments from this tuxsuite test.

In [1]: from squad.ci.models import TestJob, Backend
   ...: from squad.ci.tasks import fetch
   ...: from squad.core.models import Group
   ...: import datetime
   ...: 
   ...: tuxsuite = Backend.objects.get(name="tuxsuite.com")
   ...: g = Group.objects.get(slug="~chaws")
   ...: p = g.projects.get(slug="test-mmtest-download")
   ...: b = p.builds.create(version=f"mmtest-{datetime.datetime.now()}")
   ...: job = TestJob.objects.create(target=p, target_build=b, backend=tuxsuite, job_id="TEST:tuxsuite@vishal#2YRWmR5DPWVXojWJWBfoWXqfiMV")
   ...: 
   ...: fetch(job.id)
[2023-11-21 20:20:25 +0000] [INFO] fetching 1173638
[2023-11-21 20:20:25 +0000] [DEBUG] Starting new HTTPS connection (1): tuxapi.tuxsuite.com:443
[2023-11-21 20:20:25 +0000] [DEBUG] https://tuxapi.tuxsuite.com:443 "GET /v1/groups/tuxsuite/projects/vishal/tests/2YRWmR5DPWVXojWJWBfoWXqfiMV HTTP/1.1" 200 1297
[2023-11-21 20:20:26 +0000] [DEBUG] https://tuxapi.tuxsuite.com:443 "GET /v1/groups/tuxsuite/projects/vishal/tests/2YRWmR5DPWVXojWJWBfoWXqfiMV/logs?format=txt HTTP/1.1" 307 0
[2023-11-21 20:20:26 +0000] [DEBUG] Starting new HTTPS connection (1): storage.tuxsuite.com:443
[2023-11-21 20:20:27 +0000] [DEBUG] https://storage.tuxsuite.com:443 "GET /public/tuxsuite/vishal/tests/2YRWmR5DPWVXojWJWBfoWXqfiMV/logs.txt HTTP/1.1" 307 0
[2023-11-21 20:20:27 +0000] [DEBUG] Starting new HTTPS connection (1): tuxapi-prod-storage-public-tuxsuite.s3.amazonaws.com:443
[2023-11-21 20:20:28 +0000] [DEBUG] https://tuxapi-prod-storage-public-tuxsuite.s3.amazonaws.com:443 "GET /vishal/tests/2YRWmR5DPWVXojWJWBfoWXqfiMV/logs.txt?AWSAccessKeyId=ASIA4PEBGJPLOICIPMEK&Signature=pJK14SHn5a6dICypi7jmyQGvejY%3D&x-amz-security-token=IQoJb3JpZ2luX2VjEMP%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJIMEYCIQDSizJ%2BIBC1hnzC6QufcA0cEtuYj%2FmOmz726UosAvZiAQIhAJq2r1huu2dT3jOIX%2F2IUsDaNrAObM7JORAFd%2Fr4sLuxKoQDCBwQAhoMODU3MTE2OTIwNzkwIgwTm7iBwhGV1dY1b3sq4QKpWLyP%2ByVhVYn%2BEvca96zldXngCppEaubmWqq4TxPWqgRx%2BSpNDTzsTIGI0PU25XKGP3zhSaIYPrrM3FWEcjXmEYyDg4hjcY4%2BCI4MXdDXfsnIzum3uaomkxjkL4QjroCRrLMothx6vZqPwElLoZrJ3LNlOTimvZN82bsBJ6PuvCHD6xIGTCLa%2FVrmpwcI2ermC9L0qe2kwi3ifvtrkUaT8agK3%2BFroTjepTk4H95RwgCxpA%2F8QTOCHlEdkpG83F9ooQ8dM0Mb1RR4DDRd7K1x2I713T8ZaJn%2FXk79f%2BLEsC7PFKRdEM6Tcd8iHl5moL6bdxRsHtVlAUr%2Fda99FWX9ZMzBq%2F%2B8ThYWNunEsn7mhRr7p6SVc%2BvYcV8SvP0HvfJFfpQkUKAmvyrxXm9CpnhcfQyLVxGNUxkZOH1Q4kxAI%2FdH59sv2zsxZ0xevIrMHtpkj85UT4NAlFsG0dhzALdCwzCC9vOqBjqdAREY32mA3A7GBDkEUGAlHdKKALBB%2Ftg1iqSb36Lu9X%2F9JGHs6T1z%2FA3U2abl3whRnuA4c2wwEoAlv4rJ%2F5%2B9bklRnNM3FwZYVur7uY6SDo0ogfVFtjlUjNRv6hbtniGQXuHLGhi8WVYkIY84zkDUjO2PFJCdNDn0O%2FqAW4IOqIIjKmNE540RVQAzPG9EgwNZHtx9UkVJKaH74aCVLWE%3D&Expires=1700601626 HTTP/1.1" 200 23168
[2023-11-21 20:20:28 +0000] [DEBUG] https://tuxapi.tuxsuite.com:443 "GET /v1/groups/tuxsuite/projects/vishal/tests/2YRWmR5DPWVXojWJWBfoWXqfiMV/results HTTP/1.1" 307 0
[2023-11-21 20:20:28 +0000] [DEBUG] https://storage.tuxsuite.com:443 "GET /public/tuxsuite/vishal/tests/2YRWmR5DPWVXojWJWBfoWXqfiMV/results.json HTTP/1.1" 307 0
[2023-11-21 20:20:28 +0000] [DEBUG] https://tuxapi-prod-storage-public-tuxsuite.s3.amazonaws.com:443 "GET /vishal/tests/2YRWmR5DPWVXojWJWBfoWXqfiMV/results.json?AWSAccessKeyId=ASIA4PEBGJPLOICIPMEK&Signature=beD6Lp33fi6BLkpw8bnf37P88wk%3D&x-amz-security-token=IQoJb3JpZ2luX2VjEMP%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJIMEYCIQDSizJ%2BIBC1hnzC6QufcA0cEtuYj%2FmOmz726UosAvZiAQIhAJq2r1huu2dT3jOIX%2F2IUsDaNrAObM7JORAFd%2Fr4sLuxKoQDCBwQAhoMODU3MTE2OTIwNzkwIgwTm7iBwhGV1dY1b3sq4QKpWLyP%2ByVhVYn%2BEvca96zldXngCppEaubmWqq4TxPWqgRx%2BSpNDTzsTIGI0PU25XKGP3zhSaIYPrrM3FWEcjXmEYyDg4hjcY4%2BCI4MXdDXfsnIzum3uaomkxjkL4QjroCRrLMothx6vZqPwElLoZrJ3LNlOTimvZN82bsBJ6PuvCHD6xIGTCLa%2FVrmpwcI2ermC9L0qe2kwi3ifvtrkUaT8agK3%2BFroTjepTk4H95RwgCxpA%2F8QTOCHlEdkpG83F9ooQ8dM0Mb1RR4DDRd7K1x2I713T8ZaJn%2FXk79f%2BLEsC7PFKRdEM6Tcd8iHl5moL6bdxRsHtVlAUr%2Fda99FWX9ZMzBq%2F%2B8ThYWNunEsn7mhRr7p6SVc%2BvYcV8SvP0HvfJFfpQkUKAmvyrxXm9CpnhcfQyLVxGNUxkZOH1Q4kxAI%2FdH59sv2zsxZ0xevIrMHtpkj85UT4NAlFsG0dhzALdCwzCC9vOqBjqdAREY32mA3A7GBDkEUGAlHdKKALBB%2Ftg1iqSb36Lu9X%2F9JGHs6T1z%2FA3U2abl3whRnuA4c2wwEoAlv4rJ%2F5%2B9bklRnNM3FwZYVur7uY6SDo0ogfVFtjlUjNRv6hbtniGQXuHLGhi8WVYkIY84zkDUjO2PFJCdNDn0O%2FqAW4IOqIIjKmNE540RVQAzPG9EgwNZHtx9UkVJKaH74aCVLWE%3D&Expires=1700601628 HTTP/1.1" 200 2550
job_url >>>>>>>> https://tuxapi.tuxsuite.com/v1/groups/tuxsuite/projects/vishal/tests/2YRWmR5DPWVXojWJWBfoWXqfiMV/
[2023-11-21 20:20:29 +0000] [DEBUG] https://storage.tuxsuite.com:443 "GET /public/tuxsuite/vishal/tests/2YRWmR5DPWVXojWJWBfoWXqfiMV/attachments/ HTTP/1.1" 200 251
[2023-11-21 20:20:29 +0000] [DEBUG] https://storage.tuxsuite.com:443 "GET /public/tuxsuite/vishal/tests/2YRWmR5DPWVXojWJWBfoWXqfiMV/attachments/issue.txt HTTP/1.1" 307 0
[2023-11-21 20:20:29 +0000] [DEBUG] https://tuxapi-prod-storage-public-tuxsuite.s3.amazonaws.com:443 "GET /vishal/tests/2YRWmR5DPWVXojWJWBfoWXqfiMV/attachments/issue.txt?AWSAccessKeyId=ASIA4PEBGJPLOICIPMEK&Signature=%2Fu4W7ksanCybKHNmWH3q4fOcM3I%3D&x-amz-security-token=IQoJb3JpZ2luX2VjEMP%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJIMEYCIQDSizJ%2BIBC1hnzC6QufcA0cEtuYj%2FmOmz726UosAvZiAQIhAJq2r1huu2dT3jOIX%2F2IUsDaNrAObM7JORAFd%2Fr4sLuxKoQDCBwQAhoMODU3MTE2OTIwNzkwIgwTm7iBwhGV1dY1b3sq4QKpWLyP%2ByVhVYn%2BEvca96zldXngCppEaubmWqq4TxPWqgRx%2BSpNDTzsTIGI0PU25XKGP3zhSaIYPrrM3FWEcjXmEYyDg4hjcY4%2BCI4MXdDXfsnIzum3uaomkxjkL4QjroCRrLMothx6vZqPwElLoZrJ3LNlOTimvZN82bsBJ6PuvCHD6xIGTCLa%2FVrmpwcI2ermC9L0qe2kwi3ifvtrkUaT8agK3%2BFroTjepTk4H95RwgCxpA%2F8QTOCHlEdkpG83F9ooQ8dM0Mb1RR4DDRd7K1x2I713T8ZaJn%2FXk79f%2BLEsC7PFKRdEM6Tcd8iHl5moL6bdxRsHtVlAUr%2Fda99FWX9ZMzBq%2F%2B8ThYWNunEsn7mhRr7p6SVc%2BvYcV8SvP0HvfJFfpQkUKAmvyrxXm9CpnhcfQyLVxGNUxkZOH1Q4kxAI%2FdH59sv2zsxZ0xevIrMHtpkj85UT4NAlFsG0dhzALdCwzCC9vOqBjqdAREY32mA3A7GBDkEUGAlHdKKALBB%2Ftg1iqSb36Lu9X%2F9JGHs6T1z%2FA3U2abl3whRnuA4c2wwEoAlv4rJ%2F5%2B9bklRnNM3FwZYVur7uY6SDo0ogfVFtjlUjNRv6hbtniGQXuHLGhi8WVYkIY84zkDUjO2PFJCdNDn0O%2FqAW4IOqIIjKmNE540RVQAzPG9EgwNZHtx9UkVJKaH74aCVLWE%3D&Expires=1700601629 HTTP/1.1" 200 19
[2023-11-21 20:20:29 +0000] [INFO] Task squad.core.tasks.notification.notify_patch_build_finished[236aa056-505d-4d3c-a122-78f50b7875d2] succeeded in 0.0005813179996039253s: None
[2023-11-21 20:20:29 +0000] [INFO] Task squad.core.tasks.notification.maybe_notify_project_status[743239c9-fb9b-4c9d-b9af-afe3490b3286] succeeded in 0.01042162999874563s: None
[2023-11-21 20:20:29 +0000] [WARNING] Task squad.ci.tasks.fetch(1173638,) consumed 4MB of memory