From 8f57f70bfbcc9b6a5cfb56f3b73c59146c1660b6 Mon Sep 17 00:00:00 2001 From: mpage Date: Fri, 18 Oct 2024 16:56:08 +0000 Subject: [PATCH] Benchmarking results for @mpage --- README.md | 1 + RESULTS.md | 1 + longitudinal.json | 3 +- longitudinal.svg | 199 +- memory_long.json | 3 +- memory_long.svg | 93 +- .../bm-20241018-3.14.0a1+-2e950e3/README.md | 33 + ...python-main-3.14.0a1+-2e950e3-vs-3.12.6.md | 199 ++ ...ython-main-3.14.0a1+-2e950e3-vs-3.12.6.svg | 3102 ++++++++++++++++ ...hon-main-3.14.0a1+-2e950e3-vs-3.13.0rc2.md | 193 + ...on-main-3.14.0a1+-2e950e3-vs-3.13.0rc2.svg | 3153 +++++++++++++++++ 11 files changed, 6833 insertions(+), 147 deletions(-) create mode 100644 results/bm-20241018-3.14.0a1+-2e950e3/README.md create mode 100644 results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.12.6.md create mode 100644 results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.12.6.svg create mode 100644 results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.13.0rc2.md create mode 100644 results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.13.0rc2.svg diff --git a/README.md b/README.md index 154a106f..4d09be54 100644 --- a/README.md +++ b/README.md @@ -17,6 +17,7 @@ Here are some recent and important revisions. πŸ‘‰ [Complete list of results](RE ## linux x86_64 (vultr) | date | fork/ref | hash/flags | vs. 3.12.6: | vs. 3.13.0rc2: | vs. base: | | --- | --- | --- | ---: | ---: | ---: | +| [2024-10-18](results/bm-20241018-3.14.0a1%2B-2e950e3) | python/main | 2e950e3 | 1.01x ↑
[πŸ“„](results/bm-20241018-3.14.0a1%2B-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1%2B-2e950e3-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241018-3.14.0a1%2B-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1%2B-2e950e3-vs-3.12.6.svg) | 1.01x ↓
[πŸ“„](results/bm-20241018-3.14.0a1%2B-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1%2B-2e950e3-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241018-3.14.0a1%2B-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1%2B-2e950e3-vs-3.13.0rc2.svg) | | | [2024-10-17](results/bm-20241017-3.14.0a1%2B-a0f5c8e) | python/main | a0f5c8e | 1.00x ↑
[πŸ“„](results/bm-20241017-3.14.0a1%2B-a0f5c8e/bm-20241017-vultr-x86_64-python-main-3.14.0a1%2B-a0f5c8e-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241017-3.14.0a1%2B-a0f5c8e/bm-20241017-vultr-x86_64-python-main-3.14.0a1%2B-a0f5c8e-vs-3.12.6.svg) | 1.01x ↓
[πŸ“„](results/bm-20241017-3.14.0a1%2B-a0f5c8e/bm-20241017-vultr-x86_64-python-main-3.14.0a1%2B-a0f5c8e-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241017-3.14.0a1%2B-a0f5c8e/bm-20241017-vultr-x86_64-python-main-3.14.0a1%2B-a0f5c8e-vs-3.13.0rc2.svg) | | | [2024-10-16](results/bm-20241016-3.14.0a1%2B-28871ed) | colesbury/gh_125608_dict_watch | 28871ed | 1.00x ↑
[πŸ“„](results/bm-20241016-3.14.0a1%2B-28871ed/bm-20241016-vultr-x86_64-colesbury-gh_125608_dict_watch-3.14.0a1%2B-28871ed-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241016-3.14.0a1%2B-28871ed/bm-20241016-vultr-x86_64-colesbury-gh_125608_dict_watch-3.14.0a1%2B-28871ed-vs-3.12.6.svg) | 1.01x ↓
[πŸ“„](results/bm-20241016-3.14.0a1%2B-28871ed/bm-20241016-vultr-x86_64-colesbury-gh_125608_dict_watch-3.14.0a1%2B-28871ed-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241016-3.14.0a1%2B-28871ed/bm-20241016-vultr-x86_64-colesbury-gh_125608_dict_watch-3.14.0a1%2B-28871ed-vs-3.13.0rc2.svg) | 1.00x ↓
[πŸ“„](results/bm-20241016-3.14.0a1%2B-28871ed/bm-20241016-vultr-x86_64-colesbury-gh_125608_dict_watch-3.14.0a1%2B-28871ed-vs-base.md)[πŸ“ˆ](results/bm-20241016-3.14.0a1%2B-28871ed/bm-20241016-vultr-x86_64-colesbury-gh_125608_dict_watch-3.14.0a1%2B-28871ed-vs-base.svg)[🧠](results/bm-20241016-3.14.0a1%2B-28871ed/bm-20241016-vultr-x86_64-colesbury-gh_125608_dict_watch-3.14.0a1%2B-28871ed-vs-base-mem.svg) | | [2024-10-16](results/bm-20241016-3.14.0a1%2B-3a126a8) | colesbury/gh_125610_STORE_ATTR | 3a126a8 | 1.01x ↓
[πŸ“„](results/bm-20241016-3.14.0a1%2B-3a126a8/bm-20241016-vultr-x86_64-colesbury-gh_125610_STORE_ATTR-3.14.0a1%2B-3a126a8-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241016-3.14.0a1%2B-3a126a8/bm-20241016-vultr-x86_64-colesbury-gh_125610_STORE_ATTR-3.14.0a1%2B-3a126a8-vs-3.12.6.svg) | 1.02x ↓
[πŸ“„](results/bm-20241016-3.14.0a1%2B-3a126a8/bm-20241016-vultr-x86_64-colesbury-gh_125610_STORE_ATTR-3.14.0a1%2B-3a126a8-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241016-3.14.0a1%2B-3a126a8/bm-20241016-vultr-x86_64-colesbury-gh_125610_STORE_ATTR-3.14.0a1%2B-3a126a8-vs-3.13.0rc2.svg) | 1.01x ↓
[πŸ“„](results/bm-20241016-3.14.0a1%2B-3a126a8/bm-20241016-vultr-x86_64-colesbury-gh_125610_STORE_ATTR-3.14.0a1%2B-3a126a8-vs-base.md)[πŸ“ˆ](results/bm-20241016-3.14.0a1%2B-3a126a8/bm-20241016-vultr-x86_64-colesbury-gh_125610_STORE_ATTR-3.14.0a1%2B-3a126a8-vs-base.svg)[🧠](results/bm-20241016-3.14.0a1%2B-3a126a8/bm-20241016-vultr-x86_64-colesbury-gh_125610_STORE_ATTR-3.14.0a1%2B-3a126a8-vs-base-mem.svg) | diff --git a/RESULTS.md b/RESULTS.md index 4fb81289..c4f061c5 100644 --- a/RESULTS.md +++ b/RESULTS.md @@ -101,6 +101,7 @@ ## linux x86_64 (vultr) | date | fork/ref | hash/flags | vs. 3.12.6: | vs. 3.13.0rc2: | vs. base: | | --- | --- | --- | ---: | ---: | ---: | +| [2024-10-18](results/bm-20241018-3.14.0a1%2B-2e950e3) | python/main | 2e950e3 | 1.01x ↑
[πŸ“„](results/bm-20241018-3.14.0a1%2B-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1%2B-2e950e3-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241018-3.14.0a1%2B-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1%2B-2e950e3-vs-3.12.6.svg) | 1.01x ↓
[πŸ“„](results/bm-20241018-3.14.0a1%2B-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1%2B-2e950e3-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241018-3.14.0a1%2B-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1%2B-2e950e3-vs-3.13.0rc2.svg) | | | [2024-10-17](results/bm-20241017-3.14.0a1%2B-a0f5c8e) | python/main | a0f5c8e | 1.00x ↑
[πŸ“„](results/bm-20241017-3.14.0a1%2B-a0f5c8e/bm-20241017-vultr-x86_64-python-main-3.14.0a1%2B-a0f5c8e-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241017-3.14.0a1%2B-a0f5c8e/bm-20241017-vultr-x86_64-python-main-3.14.0a1%2B-a0f5c8e-vs-3.12.6.svg) | 1.01x ↓
[πŸ“„](results/bm-20241017-3.14.0a1%2B-a0f5c8e/bm-20241017-vultr-x86_64-python-main-3.14.0a1%2B-a0f5c8e-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241017-3.14.0a1%2B-a0f5c8e/bm-20241017-vultr-x86_64-python-main-3.14.0a1%2B-a0f5c8e-vs-3.13.0rc2.svg) | | | [2024-10-16](results/bm-20241016-3.14.0a1%2B-28871ed) | colesbury/gh_125608_dict_watch | 28871ed | 1.00x ↑
[πŸ“„](results/bm-20241016-3.14.0a1%2B-28871ed/bm-20241016-vultr-x86_64-colesbury-gh_125608_dict_watch-3.14.0a1%2B-28871ed-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241016-3.14.0a1%2B-28871ed/bm-20241016-vultr-x86_64-colesbury-gh_125608_dict_watch-3.14.0a1%2B-28871ed-vs-3.12.6.svg) | 1.01x ↓
[πŸ“„](results/bm-20241016-3.14.0a1%2B-28871ed/bm-20241016-vultr-x86_64-colesbury-gh_125608_dict_watch-3.14.0a1%2B-28871ed-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241016-3.14.0a1%2B-28871ed/bm-20241016-vultr-x86_64-colesbury-gh_125608_dict_watch-3.14.0a1%2B-28871ed-vs-3.13.0rc2.svg) | 1.00x ↓
[πŸ“„](results/bm-20241016-3.14.0a1%2B-28871ed/bm-20241016-vultr-x86_64-colesbury-gh_125608_dict_watch-3.14.0a1%2B-28871ed-vs-base.md)[πŸ“ˆ](results/bm-20241016-3.14.0a1%2B-28871ed/bm-20241016-vultr-x86_64-colesbury-gh_125608_dict_watch-3.14.0a1%2B-28871ed-vs-base.svg)[🧠](results/bm-20241016-3.14.0a1%2B-28871ed/bm-20241016-vultr-x86_64-colesbury-gh_125608_dict_watch-3.14.0a1%2B-28871ed-vs-base-mem.svg) | | [2024-10-16](results/bm-20241016-3.14.0a1%2B-3a126a8) | colesbury/gh_125610_STORE_ATTR | 3a126a8 | 1.01x ↓
[πŸ“„](results/bm-20241016-3.14.0a1%2B-3a126a8/bm-20241016-vultr-x86_64-colesbury-gh_125610_STORE_ATTR-3.14.0a1%2B-3a126a8-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241016-3.14.0a1%2B-3a126a8/bm-20241016-vultr-x86_64-colesbury-gh_125610_STORE_ATTR-3.14.0a1%2B-3a126a8-vs-3.12.6.svg) | 1.02x ↓
[πŸ“„](results/bm-20241016-3.14.0a1%2B-3a126a8/bm-20241016-vultr-x86_64-colesbury-gh_125610_STORE_ATTR-3.14.0a1%2B-3a126a8-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241016-3.14.0a1%2B-3a126a8/bm-20241016-vultr-x86_64-colesbury-gh_125610_STORE_ATTR-3.14.0a1%2B-3a126a8-vs-3.13.0rc2.svg) | 1.01x ↓
[πŸ“„](results/bm-20241016-3.14.0a1%2B-3a126a8/bm-20241016-vultr-x86_64-colesbury-gh_125610_STORE_ATTR-3.14.0a1%2B-3a126a8-vs-base.md)[πŸ“ˆ](results/bm-20241016-3.14.0a1%2B-3a126a8/bm-20241016-vultr-x86_64-colesbury-gh_125610_STORE_ATTR-3.14.0a1%2B-3a126a8-vs-base.svg)[🧠](results/bm-20241016-3.14.0a1%2B-3a126a8/bm-20241016-vultr-x86_64-colesbury-gh_125610_STORE_ATTR-3.14.0a1%2B-3a126a8-vs-base-mem.svg) | diff --git a/longitudinal.json b/longitudinal.json index 3c771b86..b4125d9d 100644 --- a/longitudinal.json +++ b/longitudinal.json @@ -57,5 +57,6 @@ "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241005-3.14.0a0-16cd6cc/bm-20241005-vultr-x86_64-python-16cd6cc86b3ba20074ae-3.14.0a0-16cd6cc.json,3.13.0rc2": 1.02, "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241012-3.14.0a0-330c527/bm-20241012-vultr-x86_64-python-330c527299a5380f39c6-3.14.0a0-330c527.json,3.13.0rc2": 1.0, "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241016-3.14.0a1+-760872e/bm-20241016-vultr-x86_64-python-760872efecb95017db8e-3.14.0a1+-760872e.json,3.13.0rc2": 1.0, - "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241017-3.14.0a1+-a0f5c8e/bm-20241017-vultr-x86_64-python-main-3.14.0a1+-a0f5c8e.json,3.13.0rc2": 1.0 + "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241017-3.14.0a1+-a0f5c8e/bm-20241017-vultr-x86_64-python-main-3.14.0a1+-a0f5c8e.json,3.13.0rc2": 1.0, + "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3.json,3.13.0rc2": 1.0 } \ No newline at end of file diff --git a/longitudinal.svg b/longitudinal.svg index 5ab7938c..1eec6875 100644 --- a/longitudinal.svg +++ b/longitudinal.svg @@ -18,7 +18,7 @@ - + @@ -28,7 +28,7 @@ - + @@ -46,7 +46,7 @@ - + @@ -59,7 +59,7 @@ - + @@ -77,7 +77,7 @@ - + @@ -90,7 +90,7 @@ - + @@ -108,7 +108,7 @@ - + @@ -121,7 +121,7 @@ - + @@ -139,7 +139,7 @@ - + @@ -152,7 +152,7 @@ - + @@ -183,7 +183,7 @@ - + @@ -192,7 +192,7 @@ - + @@ -201,25 +201,25 @@ - + - + - + - + - + @@ -228,7 +228,7 @@ - + @@ -240,7 +240,7 @@ - + @@ -248,18 +248,18 @@ - + - - - + + + - - + + @@ -272,7 +272,7 @@ - + @@ -340,7 +340,7 @@ - + @@ -351,14 +351,14 @@ - + - + @@ -371,7 +371,7 @@ - + @@ -387,7 +387,7 @@ - + @@ -398,11 +398,11 @@ - + - - - + + + @@ -411,12 +411,12 @@ - + - - - + + + @@ -426,11 +426,11 @@ - + - - - + + + @@ -439,35 +439,35 @@ - + - - - + + + - + - + - - - + + + - + - + @@ -491,7 +491,7 @@ - + @@ -500,7 +500,7 @@ - + @@ -509,25 +509,25 @@ - + - + - + - + - + @@ -536,7 +536,7 @@ - + @@ -545,50 +545,51 @@ - + - + - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + - + - - - - - - - + + + + + + + + @@ -634,7 +635,7 @@ - + @@ -645,7 +646,7 @@ - + @@ -655,7 +656,7 @@ - + @@ -664,7 +665,7 @@ - + diff --git a/memory_long.json b/memory_long.json index de7786ec..ab255eff 100644 --- a/memory_long.json +++ b/memory_long.json @@ -57,5 +57,6 @@ "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241005-3.14.0a0-16cd6cc/bm-20241005-vultr-x86_64-python-16cd6cc86b3ba20074ae-3.14.0a0-16cd6cc.json,3.13.0rc2": 1.0, "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241012-3.14.0a0-330c527/bm-20241012-vultr-x86_64-python-330c527299a5380f39c6-3.14.0a0-330c527.json,3.13.0rc2": 1.0, "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241016-3.14.0a1+-760872e/bm-20241016-vultr-x86_64-python-760872efecb95017db8e-3.14.0a1+-760872e.json,3.13.0rc2": 1.0, - "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241017-3.14.0a1+-a0f5c8e/bm-20241017-vultr-x86_64-python-main-3.14.0a1+-a0f5c8e.json,3.13.0rc2": 0.97 + "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241017-3.14.0a1+-a0f5c8e/bm-20241017-vultr-x86_64-python-main-3.14.0a1+-a0f5c8e.json,3.13.0rc2": 0.97, + "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3.json,3.13.0rc2": 1.0 } \ No newline at end of file diff --git a/memory_long.svg b/memory_long.svg index ae3aec4a..f745cbd6 100644 --- a/memory_long.svg +++ b/memory_long.svg @@ -381,7 +381,7 @@ - + @@ -394,9 +394,9 @@ - - - + + + @@ -408,9 +408,9 @@ - - - + + + @@ -422,9 +422,9 @@ - - - + + + @@ -436,9 +436,9 @@ - - - + + + @@ -450,9 +450,9 @@ - - - + + + @@ -546,42 +546,43 @@ - + - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + - + - - - - - - - + + + + + + + + diff --git a/results/bm-20241018-3.14.0a1+-2e950e3/README.md b/results/bm-20241018-3.14.0a1+-2e950e3/README.md new file mode 100644 index 00000000..bfd2d3cd --- /dev/null +++ b/results/bm-20241018-3.14.0a1+-2e950e3/README.md @@ -0,0 +1,33 @@ +# Results + +- fork: python +- version: 3.14.0a1+ +- config: +- commit hash: [2e950e3](https://github.com/python/cpython/commit/2e950e3) +- commit date: 2024-10-18T16:51:29+03:00 +- commit merge base: [cda0ec8e7c4e9a010e5f73c5afaf18f86cb27b97](https://github.com/python/cpython/commit/cda0ec8e7c4e9a010e5f73c5afaf18f86cb27b97) +- ref: main + +## linux x86_64 (vultr) + +- [GitHub Action run](https://github.com/facebookexperimental/free-threading-benchmarking/actions/runs/11405745068) +- cpu model: Intel(R) Xeon(R) E-2286G CPU @ 4.00GHz +- platform: Linux-6.8.0-45-generic-x86_64-with-glibc2.39 +- [raw results](bm-20241018-vultr-x86_64-python-main-3.14.0a1%2B-2e950e3.json) + +### vs. 3.12.6 + +- Geometric mean: 1.01x faster (HPT: reliability of 99.97%, 1.00x faster at 99th %ile) +- Memory usage: 1.02x +- missing benchmarks: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn, mypy2, sqlalchemy_declarative, sqlalchemy_imperative +- [πŸ“„table](bm-20241018-vultr-x86_64-python-main-3.14.0a1%2B-2e950e3-vs-3.12.6.md) +- [πŸ“ˆtime plot](bm-20241018-vultr-x86_64-python-main-3.14.0a1%2B-2e950e3-vs-3.12.6.svg) + +### vs. 3.13.0rc2 + +- Geometric mean: 1.01x slower (HPT: reliability of 93.31%, 1.00x faster at 99th %ile) +- Memory usage: 1.00x +- missing benchmarks: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn +- [πŸ“„table](bm-20241018-vultr-x86_64-python-main-3.14.0a1%2B-2e950e3-vs-3.13.0rc2.md) +- [πŸ“ˆtime plot](bm-20241018-vultr-x86_64-python-main-3.14.0a1%2B-2e950e3-vs-3.13.0rc2.svg) + diff --git a/results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.12.6.md b/results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.12.6.md new file mode 100644 index 00000000..633d01dc --- /dev/null +++ b/results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.12.6.md @@ -0,0 +1,199 @@ +# Results vs. 3.12.6 + +- fork: python +- ref: main +- machine: linux-x86_64 +- commit hash: 2e950e3 +- commit date: 2024-10-18 +- overall geometric mean: 1.01x faster +- HPT reliability: 99.97% +- HPT 99th percentile: 1.00x faster +- Memory change: 1.02x + +Benchmarks with tag 'apps': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|----------------|:------------------------------------------------------:|:------------------------------------------------------:| +| 2to3 | 264 ms | 253 ms: 1.04x faster | +| docutils | 2.64 sec | 2.59 sec: 1.02x faster | +| tornado_http | 119 ms | 113 ms: 1.05x faster | +| Geometric mean | (ref) | 1.03x faster | + +Benchmark hidden because not significant (1): html5lib + +Benchmarks with tag 'asyncio': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|--------------------|:------------------------------------------------------:|:------------------------------------------------------:| +| coroutines | 23.9 ms | 22.3 ms: 1.08x faster | +| async_generators | 384 ms | 369 ms: 1.04x faster | +| asyncio_tcp | 519 ms | 503 ms: 1.03x faster | +| asyncio_websockets | 517 ms | 521 ms: 1.01x slower | +| asyncio_tcp_ssl | 1.51 sec | 1.52 sec: 1.01x slower | +| Geometric mean | (ref) | 1.03x faster | + +Benchmarks with tag 'math': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|----------------|:------------------------------------------------------:|:------------------------------------------------------:| +| float | 80.8 ms | 78.1 ms: 1.03x faster | +| pidigits | 184 ms | 185 ms: 1.00x slower | +| nbody | 89.3 ms | 92.4 ms: 1.04x slower | +| Geometric mean | (ref) | 1.00x slower | + +Benchmarks with tag 'regex': +============================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|----------------|:------------------------------------------------------:|:------------------------------------------------------:| +| regex_compile | 142 ms | 129 ms: 1.10x faster | +| regex_effbot | 3.17 ms | 3.28 ms: 1.04x slower | +| regex_dna | 168 ms | 179 ms: 1.07x slower | +| regex_v8 | 20.6 ms | 23.1 ms: 1.12x slower | +| Geometric mean | (ref) | 1.03x slower | + +Benchmarks with tag 'serialize': +================================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|----------------------|:------------------------------------------------------:|:------------------------------------------------------:| +| unpickle_pure_python | 221 us | 215 us: 1.03x faster | +| unpickle_list | 4.67 us | 4.58 us: 1.02x faster | +| json_loads | 26.5 us | 26.1 us: 1.02x faster | +| xml_etree_parse | 139 ms | 136 ms: 1.02x faster | +| pickle_pure_python | 308 us | 305 us: 1.01x faster | +| xml_etree_iterparse | 96.7 ms | 96.1 ms: 1.01x faster | +| tomli_loads | 2.11 sec | 2.09 sec: 1.01x faster | +| xml_etree_generate | 85.2 ms | 84.8 ms: 1.00x faster | +| pickle_list | 4.77 us | 4.84 us: 1.02x slower | +| pickle_dict | 31.8 us | 32.7 us: 1.03x slower | +| pickle | 10.9 us | 11.3 us: 1.03x slower | +| json_dumps | 10.4 ms | 11.6 ms: 1.12x slower | +| Geometric mean | (ref) | 1.00x slower | + +Benchmark hidden because not significant (2): unpickle, xml_etree_process + +Benchmarks with tag 'startup': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|------------------------|:------------------------------------------------------:|:------------------------------------------------------:| +| python_startup_no_site | 7.16 ms | 7.41 ms: 1.03x slower | +| python_startup | 9.93 ms | 11.0 ms: 1.11x slower | +| Geometric mean | (ref) | 1.07x slower | + +Benchmarks with tag 'template': +=============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|-----------------|:------------------------------------------------------:|:------------------------------------------------------:| +| genshi_text | 22.8 ms | 22.4 ms: 1.02x faster | +| genshi_xml | 50.2 ms | 49.7 ms: 1.01x faster | +| django_template | 34.7 ms | 35.8 ms: 1.03x slower | +| mako | 11.0 ms | 11.7 ms: 1.06x slower | +| Geometric mean | (ref) | 1.02x slower | + +All benchmarks: +=============== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|--------------------------|:------------------------------------------------------:|:------------------------------------------------------:| +| deepcopy | 352 us | 262 us: 1.34x faster | +| deepcopy_memo | 40.3 us | 30.0 us: 1.34x faster | +| go | 139 ms | 119 ms: 1.17x faster | +| comprehensions | 19.8 us | 16.9 us: 1.17x faster | +| pathlib | 21.5 ms | 18.5 ms: 1.17x faster | +| deepcopy_reduce | 3.08 us | 2.65 us: 1.16x faster | +| raytrace | 299 ms | 259 ms: 1.15x faster | +| crypto_pyaes | 76.6 ms | 66.4 ms: 1.15x faster | +| unpack_sequence | 52.1 ns | 45.6 ns: 1.14x faster | +| generators | 32.2 ms | 28.9 ms: 1.12x faster | +| regex_compile | 142 ms | 129 ms: 1.10x faster | +| logging_simple | 6.63 us | 6.02 us: 1.10x faster | +| logging_format | 7.35 us | 6.72 us: 1.09x faster | +| bpe_tokeniser | 4.74 sec | 4.34 sec: 1.09x faster | +| sympy_sum | 166 ms | 154 ms: 1.08x faster | +| coroutines | 23.9 ms | 22.3 ms: 1.08x faster | +| deltablue | 3.45 ms | 3.22 ms: 1.07x faster | +| sympy_str | 292 ms | 273 ms: 1.07x faster | +| chaos | 62.8 ms | 58.8 ms: 1.07x faster | +| dulwich_log | 78.9 ms | 74.3 ms: 1.06x faster | +| scimark_monte_carlo | 68.4 ms | 64.8 ms: 1.06x faster | +| gc_traversal | 3.46 ms | 3.28 ms: 1.05x faster | +| sqlglot_parse | 1.36 ms | 1.29 ms: 1.05x faster | +| sqlglot_transpile | 1.67 ms | 1.59 ms: 1.05x faster | +| tornado_http | 119 ms | 113 ms: 1.05x faster | +| typing_runtime_protocols | 163 us | 155 us: 1.05x faster | +| hexiom | 6.17 ms | 5.88 ms: 1.05x faster | +| pprint_safe_repr | 743 ms | 709 ms: 1.05x faster | +| async_generators | 384 ms | 369 ms: 1.04x faster | +| thrift | 791 us | 759 us: 1.04x faster | +| scimark_lu | 114 ms | 109 ms: 1.04x faster | +| pycparser | 1.17 sec | 1.12 sec: 1.04x faster | +| 2to3 | 264 ms | 253 ms: 1.04x faster | +| json | 5.02 ms | 4.83 ms: 1.04x faster | +| logging_silent | 109 ns | 105 ns: 1.04x faster | +| mdp | 2.42 sec | 2.34 sec: 1.03x faster | +| pprint_pformat | 1.52 sec | 1.47 sec: 1.03x faster | +| float | 80.8 ms | 78.1 ms: 1.03x faster | +| asyncio_tcp | 519 ms | 503 ms: 1.03x faster | +| sympy_integrate | 20.5 ms | 20.0 ms: 1.03x faster | +| unpickle_pure_python | 221 us | 215 us: 1.03x faster | +| scimark_fft | 342 ms | 333 ms: 1.03x faster | +| unpickle_list | 4.67 us | 4.58 us: 1.02x faster | +| genshi_text | 22.8 ms | 22.4 ms: 1.02x faster | +| sympy_expand | 468 ms | 458 ms: 1.02x faster | +| richards_super | 51.9 ms | 50.9 ms: 1.02x faster | +| nqueens | 80.1 ms | 78.6 ms: 1.02x faster | +| docutils | 2.64 sec | 2.59 sec: 1.02x faster | +| json_loads | 26.5 us | 26.1 us: 1.02x faster | +| xml_etree_parse | 139 ms | 136 ms: 1.02x faster | +| richards | 45.9 ms | 45.3 ms: 1.02x faster | +| meteor_contest | 104 ms | 102 ms: 1.01x faster | +| spectral_norm | 110 ms | 109 ms: 1.01x faster | +| genshi_xml | 50.2 ms | 49.7 ms: 1.01x faster | +| pickle_pure_python | 308 us | 305 us: 1.01x faster | +| xml_etree_iterparse | 96.7 ms | 96.1 ms: 1.01x faster | +| tomli_loads | 2.11 sec | 2.09 sec: 1.01x faster | +| xml_etree_generate | 85.2 ms | 84.8 ms: 1.00x faster | +| pidigits | 184 ms | 185 ms: 1.00x slower | +| pyflate | 448 ms | 450 ms: 1.00x slower | +| sqlglot_optimize | 53.3 ms | 53.6 ms: 1.01x slower | +| asyncio_websockets | 517 ms | 521 ms: 1.01x slower | +| asyncio_tcp_ssl | 1.51 sec | 1.52 sec: 1.01x slower | +| pickle_list | 4.77 us | 4.84 us: 1.02x slower | +| fannkuch | 372 ms | 379 ms: 1.02x slower | +| pickle_dict | 31.8 us | 32.7 us: 1.03x slower | +| scimark_sor | 130 ms | 133 ms: 1.03x slower | +| django_template | 34.7 ms | 35.8 ms: 1.03x slower | +| pickle | 10.9 us | 11.3 us: 1.03x slower | +| python_startup_no_site | 7.16 ms | 7.41 ms: 1.03x slower | +| nbody | 89.3 ms | 92.4 ms: 1.04x slower | +| regex_effbot | 3.17 ms | 3.28 ms: 1.04x slower | +| mako | 11.0 ms | 11.7 ms: 1.06x slower | +| regex_dna | 168 ms | 179 ms: 1.07x slower | +| bench_thread_pool | 941 us | 1.01 ms: 1.07x slower | +| python_startup | 9.93 ms | 11.0 ms: 1.11x slower | +| json_dumps | 10.4 ms | 11.6 ms: 1.12x slower | +| telco | 6.53 ms | 7.32 ms: 1.12x slower | +| regex_v8 | 20.6 ms | 23.1 ms: 1.12x slower | +| coverage | 71.4 ms | 82.2 ms: 1.15x slower | +| create_gc_cycles | 1.09 ms | 1.38 ms: 1.26x slower | +| bench_mp_pool | 10.8 ms | 63.3 ms: 5.87x slower | +| Geometric mean | (ref) | 1.01x faster | + +Benchmark hidden because not significant (7): unpickle, html5lib, scimark_sparse_mat_mult, sqlglot_normalize, pylint, xml_etree_process, sqlite_synth +Ignored benchmarks (16) of results/bm-20240906-3.12.6-a4a2d2b/bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b.json: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn, mypy2, sqlalchemy_declarative, sqlalchemy_imperative + +# HPT report + +- Reliability score: 99.97% likely to be faster +- 90% likely to have a speedup of 1.01x +- 95% likely to have a speedup of 1.01x +- 99% likely to have a speedup of 1.00x + +# Memory +- memory change: 1.02x \ No newline at end of file diff --git a/results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.12.6.svg b/results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.12.6.svg new file mode 100644 index 00000000..005f76b2 --- /dev/null +++ b/results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.12.6.svg @@ -0,0 +1,3102 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.13.0rc2.md b/results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.13.0rc2.md new file mode 100644 index 00000000..e3014050 --- /dev/null +++ b/results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.13.0rc2.md @@ -0,0 +1,193 @@ +# Results vs. 3.13.0rc2 + +- fork: python +- ref: main +- machine: linux-x86_64 +- commit hash: 2e950e3 +- commit date: 2024-10-18 +- overall geometric mean: 1.01x slower +- HPT reliability: 93.31% +- HPT 99th percentile: 1.00x faster +- Memory change: 1.00x + +Benchmarks with tag 'apps': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|----------------|:------------------------------------------------------------:|:------------------------------------------------------:| +| 2to3 | 260 ms | 253 ms: 1.03x faster | +| docutils | 2.62 sec | 2.59 sec: 1.01x faster | +| html5lib | 67.0 ms | 63.0 ms: 1.06x faster | +| Geometric mean | (ref) | 1.03x faster | + +Benchmark hidden because not significant (1): tornado_http + +Benchmarks with tag 'asyncio': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|------------------|:------------------------------------------------------------:|:------------------------------------------------------:| +| coroutines | 23.6 ms | 22.3 ms: 1.06x faster | +| async_generators | 377 ms | 369 ms: 1.02x faster | +| asyncio_tcp_ssl | 1.51 sec | 1.52 sec: 1.01x slower | +| Geometric mean | (ref) | 1.02x faster | + +Benchmark hidden because not significant (2): asyncio_tcp, asyncio_websockets + +Benchmarks with tag 'math': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|----------------|:------------------------------------------------------------:|:------------------------------------------------------:| +| pidigits | 217 ms | 185 ms: 1.17x faster | +| float | 77.5 ms | 78.1 ms: 1.01x slower | +| nbody | 85.1 ms | 92.4 ms: 1.09x slower | +| Geometric mean | (ref) | 1.02x faster | + +Benchmarks with tag 'regex': +============================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|----------------|:------------------------------------------------------------:|:------------------------------------------------------:| +| regex_compile | 132 ms | 129 ms: 1.02x faster | +| regex_dna | 180 ms | 179 ms: 1.01x faster | +| regex_v8 | 22.7 ms | 23.1 ms: 1.02x slower | +| regex_effbot | 3.08 ms | 3.28 ms: 1.06x slower | +| Geometric mean | (ref) | 1.01x slower | + +Benchmarks with tag 'serialize': +================================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|----------------------|:------------------------------------------------------------:|:------------------------------------------------------:| +| json_loads | 27.0 us | 26.1 us: 1.03x faster | +| unpickle | 14.3 us | 13.9 us: 1.03x faster | +| unpickle_list | 4.71 us | 4.58 us: 1.03x faster | +| pickle_list | 4.93 us | 4.84 us: 1.02x faster | +| xml_etree_generate | 85.4 ms | 84.8 ms: 1.01x faster | +| xml_etree_process | 59.3 ms | 59.1 ms: 1.00x faster | +| pickle_dict | 32.5 us | 32.7 us: 1.00x slower | +| xml_etree_iterparse | 94.9 ms | 96.1 ms: 1.01x slower | +| unpickle_pure_python | 210 us | 215 us: 1.02x slower | +| pickle_pure_python | 294 us | 305 us: 1.04x slower | +| tomli_loads | 2.01 sec | 2.09 sec: 1.04x slower | +| json_dumps | 10.5 ms | 11.6 ms: 1.10x slower | +| Geometric mean | (ref) | 1.01x slower | + +Benchmark hidden because not significant (2): pickle, xml_etree_parse + +Benchmarks with tag 'startup': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|------------------------|:------------------------------------------------------------:|:------------------------------------------------------:| +| python_startup_no_site | 7.39 ms | 7.41 ms: 1.00x slower | +| python_startup | 11.0 ms | 11.0 ms: 1.00x slower | +| Geometric mean | (ref) | 1.00x slower | + +Benchmarks with tag 'template': +=============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|-----------------|:------------------------------------------------------------:|:------------------------------------------------------:| +| genshi_xml | 48.8 ms | 49.7 ms: 1.02x slower | +| mako | 11.3 ms | 11.7 ms: 1.03x slower | +| genshi_text | 21.5 ms | 22.4 ms: 1.04x slower | +| django_template | 34.1 ms | 35.8 ms: 1.05x slower | +| Geometric mean | (ref) | 1.03x slower | + +All benchmarks: +=============== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3 | +|-------------------------|:------------------------------------------------------------:|:------------------------------------------------------:| +| deepcopy | 355 us | 262 us: 1.36x faster | +| deepcopy_memo | 39.1 us | 30.0 us: 1.30x faster | +| go | 141 ms | 119 ms: 1.19x faster | +| deepcopy_reduce | 3.11 us | 2.65 us: 1.17x faster | +| pidigits | 217 ms | 185 ms: 1.17x faster | +| scimark_sparse_mat_mult | 4.71 ms | 4.38 ms: 1.08x faster | +| telco | 7.82 ms | 7.32 ms: 1.07x faster | +| html5lib | 67.0 ms | 63.0 ms: 1.06x faster | +| coroutines | 23.6 ms | 22.3 ms: 1.06x faster | +| scimark_fft | 349 ms | 333 ms: 1.05x faster | +| pprint_safe_repr | 738 ms | 709 ms: 1.04x faster | +| pathlib | 19.2 ms | 18.5 ms: 1.04x faster | +| json_loads | 27.0 us | 26.1 us: 1.03x faster | +| unpickle | 14.3 us | 13.9 us: 1.03x faster | +| unpickle_list | 4.71 us | 4.58 us: 1.03x faster | +| scimark_lu | 113 ms | 109 ms: 1.03x faster | +| 2to3 | 260 ms | 253 ms: 1.03x faster | +| thrift | 778 us | 759 us: 1.03x faster | +| bpe_tokeniser | 4.45 sec | 4.34 sec: 1.02x faster | +| regex_compile | 132 ms | 129 ms: 1.02x faster | +| async_generators | 377 ms | 369 ms: 1.02x faster | +| logging_simple | 6.16 us | 6.02 us: 1.02x faster | +| crypto_pyaes | 67.9 ms | 66.4 ms: 1.02x faster | +| spectral_norm | 111 ms | 109 ms: 1.02x faster | +| json | 4.93 ms | 4.83 ms: 1.02x faster | +| pprint_pformat | 1.50 sec | 1.47 sec: 1.02x faster | +| hexiom | 5.99 ms | 5.88 ms: 1.02x faster | +| logging_format | 6.84 us | 6.72 us: 1.02x faster | +| pickle_list | 4.93 us | 4.84 us: 1.02x faster | +| richards_super | 51.6 ms | 50.9 ms: 1.01x faster | +| sympy_sum | 156 ms | 154 ms: 1.01x faster | +| coverage | 83.0 ms | 82.2 ms: 1.01x faster | +| docutils | 2.62 sec | 2.59 sec: 1.01x faster | +| scimark_monte_carlo | 65.4 ms | 64.8 ms: 1.01x faster | +| scimark_sor | 134 ms | 133 ms: 1.01x faster | +| mdp | 2.36 sec | 2.34 sec: 1.01x faster | +| xml_etree_generate | 85.4 ms | 84.8 ms: 1.01x faster | +| dulwich_log | 74.8 ms | 74.3 ms: 1.01x faster | +| sympy_str | 275 ms | 273 ms: 1.01x faster | +| regex_dna | 180 ms | 179 ms: 1.01x faster | +| xml_etree_process | 59.3 ms | 59.1 ms: 1.00x faster | +| python_startup_no_site | 7.39 ms | 7.41 ms: 1.00x slower | +| python_startup | 11.0 ms | 11.0 ms: 1.00x slower | +| pycparser | 1.12 sec | 1.12 sec: 1.00x slower | +| pickle_dict | 32.5 us | 32.7 us: 1.00x slower | +| meteor_contest | 102 ms | 102 ms: 1.01x slower | +| asyncio_tcp_ssl | 1.51 sec | 1.52 sec: 1.01x slower | +| sqlglot_normalize | 106 ms | 106 ms: 1.01x slower | +| sympy_integrate | 19.8 ms | 20.0 ms: 1.01x slower | +| float | 77.5 ms | 78.1 ms: 1.01x slower | +| xml_etree_iterparse | 94.9 ms | 96.1 ms: 1.01x slower | +| sqlglot_optimize | 52.7 ms | 53.6 ms: 1.02x slower | +| unpack_sequence | 44.8 ns | 45.6 ns: 1.02x slower | +| regex_v8 | 22.7 ms | 23.1 ms: 1.02x slower | +| sqlglot_transpile | 1.56 ms | 1.59 ms: 1.02x slower | +| genshi_xml | 48.8 ms | 49.7 ms: 1.02x slower | +| unpickle_pure_python | 210 us | 215 us: 1.02x slower | +| logging_silent | 103 ns | 105 ns: 1.02x slower | +| fannkuch | 370 ms | 379 ms: 1.03x slower | +| raytrace | 253 ms | 259 ms: 1.03x slower | +| chaos | 57.3 ms | 58.8 ms: 1.03x slower | +| mako | 11.3 ms | 11.7 ms: 1.03x slower | +| comprehensions | 16.5 us | 16.9 us: 1.03x slower | +| deltablue | 3.12 ms | 3.22 ms: 1.03x slower | +| create_gc_cycles | 1.34 ms | 1.38 ms: 1.03x slower | +| sqlglot_parse | 1.25 ms | 1.29 ms: 1.03x slower | +| pickle_pure_python | 294 us | 305 us: 1.04x slower | +| genshi_text | 21.5 ms | 22.4 ms: 1.04x slower | +| tomli_loads | 2.01 sec | 2.09 sec: 1.04x slower | +| gc_traversal | 3.14 ms | 3.28 ms: 1.04x slower | +| django_template | 34.1 ms | 35.8 ms: 1.05x slower | +| regex_effbot | 3.08 ms | 3.28 ms: 1.06x slower | +| nbody | 85.1 ms | 92.4 ms: 1.09x slower | +| bench_thread_pool | 919 us | 1.01 ms: 1.10x slower | +| json_dumps | 10.5 ms | 11.6 ms: 1.10x slower | +| bench_mp_pool | 11.0 ms | 63.3 ms: 5.76x slower | +| Geometric mean | (ref) | 1.01x slower | + +Benchmark hidden because not significant (13): asyncio_tcp, tornado_http, pickle, nqueens, richards, asyncio_websockets, xml_etree_parse, generators, sympy_expand, pylint, pyflate, typing_runtime_protocols, sqlite_synth +Ignored benchmarks (13) of results/bm-20240906-3.13.0rc2-ec61006/bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006.json: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn + +# HPT report + +- Reliability score: 93.31% likely to be faster +- 90% likely to have a speedup of 1.00x +- 95% likely to have a speedup of 1.00x +- 99% likely to have a speedup of 1.00x + +# Memory +- memory change: 1.00x \ No newline at end of file diff --git a/results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.13.0rc2.svg b/results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.13.0rc2.svg new file mode 100644 index 00000000..2a1275a2 --- /dev/null +++ b/results/bm-20241018-3.14.0a1+-2e950e3/bm-20241018-vultr-x86_64-python-main-3.14.0a1+-2e950e3-vs-3.13.0rc2.svg @@ -0,0 +1,3153 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +