diff --git a/README.md b/README.md index 774b30ad..f58e06bf 100644 --- a/README.md +++ b/README.md @@ -19,14 +19,14 @@ Here are some recent and important revisions. 👉 [Complete list of results](RE ## linux x86_64 (vultr) | date | fork/ref | hash/flags | vs. 3.12.6: | vs. 3.13.0rc2: | vs. base: | | --- | --- | --- | ---: | ---: | ---: | +| [2024-10-31](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL) | python/ac3a2c8abceeec448486 | ac3a2c8 (NOGIL) | 1.55x ↓
[📄](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.12.6.md)[📈](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.12.6.svg) | 1.57x ↓
[📄](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.13.0rc2.md)[📈](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.13.0rc2.svg) | 1.55x ↓
[📄](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-base.md)[📈](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-base.svg)[🧠](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-base-mem.svg) | +| [2024-10-31](results/bm-20241031-3.14.0a1%2B-ac3a2c8) | python/ac3a2c8abceeec448486 | ac3a2c8 | 1.00x ↓
[📄](results/bm-20241031-3.14.0a1%2B-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.12.6.md)[📈](results/bm-20241031-3.14.0a1%2B-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.12.6.svg) | 1.02x ↓
[📄](results/bm-20241031-3.14.0a1%2B-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.13.0rc2.md)[📈](results/bm-20241031-3.14.0a1%2B-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.13.0rc2.svg) | | | [2024-10-29](results/bm-20241029-3.14.0a1%2B-d4b6d84) | python/d4b6d84cc84029b598fc | d4b6d84 | 1.00x ↓
[📄](results/bm-20241029-3.14.0a1%2B-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.12.6.md)[📈](results/bm-20241029-3.14.0a1%2B-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.12.6.svg) | 1.02x ↓
[📄](results/bm-20241029-3.14.0a1%2B-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.13.0rc2.md)[📈](results/bm-20241029-3.14.0a1%2B-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.13.0rc2.svg) | | | [2024-10-29](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL) | python/d4b6d84cc84029b598fc | d4b6d84 (NOGIL) | 1.55x ↓
[📄](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.12.6.md)[📈](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.12.6.svg) | 1.58x ↓
[📄](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.13.0rc2.md)[📈](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.13.0rc2.svg) | 1.55x ↓
[📄](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-base.md)[📈](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-base.svg)[🧠](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-base-mem.svg) | | [2024-10-28](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL) | python/85799f1ffd5f285ef93a | 85799f1 (NOGIL) | 1.55x ↓
[📄](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-3.12.6.md)[📈](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-3.12.6.svg) | 1.57x ↓
[📄](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-3.13.0rc2.md)[📈](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-3.13.0rc2.svg) | 1.55x ↓
[📄](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-base.md)[📈](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-base.svg)[🧠](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-base-mem.svg) | | [2024-10-28](results/bm-20241028-3.14.0a1%2B-85799f1) | python/85799f1ffd5f285ef93a | 85799f1 | 1.00x ↑
[📄](results/bm-20241028-3.14.0a1%2B-85799f1/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-3.12.6.md)[📈](results/bm-20241028-3.14.0a1%2B-85799f1/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-3.12.6.svg) | 1.01x ↓
[📄](results/bm-20241028-3.14.0a1%2B-85799f1/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-3.13.0rc2.md)[📈](results/bm-20241028-3.14.0a1%2B-85799f1/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-3.13.0rc2.svg) | | | [2024-10-27](results/bm-20241027-3.14.0a1%2B-19e93e2-NOGIL) | python/19e93e2e269889ecb3c4 | 19e93e2 (NOGIL) | 1.54x ↓
[📄](results/bm-20241027-3.14.0a1%2B-19e93e2-NOGIL/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1%2B-19e93e2-vs-3.12.6.md)[📈](results/bm-20241027-3.14.0a1%2B-19e93e2-NOGIL/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1%2B-19e93e2-vs-3.12.6.svg) | 1.56x ↓
[📄](results/bm-20241027-3.14.0a1%2B-19e93e2-NOGIL/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1%2B-19e93e2-vs-3.13.0rc2.md)[📈](results/bm-20241027-3.14.0a1%2B-19e93e2-NOGIL/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1%2B-19e93e2-vs-3.13.0rc2.svg) | 1.53x ↓
[📄](results/bm-20241027-3.14.0a1%2B-19e93e2-NOGIL/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1%2B-19e93e2-vs-base.md)[📈](results/bm-20241027-3.14.0a1%2B-19e93e2-NOGIL/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1%2B-19e93e2-vs-base.svg)[🧠](results/bm-20241027-3.14.0a1%2B-19e93e2-NOGIL/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1%2B-19e93e2-vs-base-mem.svg) | | [2024-10-27](results/bm-20241027-3.14.0a1%2B-19e93e2) | python/19e93e2e269889ecb3c4 | 19e93e2 | 1.00x ↓
[📄](results/bm-20241027-3.14.0a1%2B-19e93e2/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1%2B-19e93e2-vs-3.12.6.md)[📈](results/bm-20241027-3.14.0a1%2B-19e93e2/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1%2B-19e93e2-vs-3.12.6.svg) | 1.02x ↓
[📄](results/bm-20241027-3.14.0a1%2B-19e93e2/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1%2B-19e93e2-vs-3.13.0rc2.md)[📈](results/bm-20241027-3.14.0a1%2B-19e93e2/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1%2B-19e93e2-vs-3.13.0rc2.svg) | | -| [2024-10-26](results/bm-20241026-3.14.0a1%2B-f6cc7c8) | python/f6cc7c8bd01d8468af70 | f6cc7c8 | 1.00x ↑
[📄](results/bm-20241026-3.14.0a1%2B-f6cc7c8/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1%2B-f6cc7c8-vs-3.12.6.md)[📈](results/bm-20241026-3.14.0a1%2B-f6cc7c8/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1%2B-f6cc7c8-vs-3.12.6.svg) | 1.01x ↓
[📄](results/bm-20241026-3.14.0a1%2B-f6cc7c8/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1%2B-f6cc7c8-vs-3.13.0rc2.md)[📈](results/bm-20241026-3.14.0a1%2B-f6cc7c8/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1%2B-f6cc7c8-vs-3.13.0rc2.svg) | | -| [2024-10-26](results/bm-20241026-3.14.0a1%2B-f6cc7c8-NOGIL) | python/f6cc7c8bd01d8468af70 | f6cc7c8 (NOGIL) | 1.55x ↓
[📄](results/bm-20241026-3.14.0a1%2B-f6cc7c8-NOGIL/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1%2B-f6cc7c8-vs-3.12.6.md)[📈](results/bm-20241026-3.14.0a1%2B-f6cc7c8-NOGIL/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1%2B-f6cc7c8-vs-3.12.6.svg) | 1.57x ↓
[📄](results/bm-20241026-3.14.0a1%2B-f6cc7c8-NOGIL/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1%2B-f6cc7c8-vs-3.13.0rc2.md)[📈](results/bm-20241026-3.14.0a1%2B-f6cc7c8-NOGIL/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1%2B-f6cc7c8-vs-3.13.0rc2.svg) | 1.55x ↓
[📄](results/bm-20241026-3.14.0a1%2B-f6cc7c8-NOGIL/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1%2B-f6cc7c8-vs-base.md)[📈](results/bm-20241026-3.14.0a1%2B-f6cc7c8-NOGIL/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1%2B-f6cc7c8-vs-base.svg)[🧠](results/bm-20241026-3.14.0a1%2B-f6cc7c8-NOGIL/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1%2B-f6cc7c8-vs-base-mem.svg) | diff --git a/RESULTS.md b/RESULTS.md index 1882df4e..868e0721 100644 --- a/RESULTS.md +++ b/RESULTS.md @@ -107,6 +107,8 @@ ## linux x86_64 (vultr) | date | fork/ref | hash/flags | vs. 3.12.6: | vs. 3.13.0rc2: | vs. base: | | --- | --- | --- | ---: | ---: | ---: | +| [2024-10-31](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL) | python/ac3a2c8abceeec448486 | ac3a2c8 (NOGIL) | 1.55x ↓
[📄](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.12.6.md)[📈](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.12.6.svg) | 1.57x ↓
[📄](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.13.0rc2.md)[📈](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.13.0rc2.svg) | 1.55x ↓
[📄](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-base.md)[📈](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-base.svg)[🧠](results/bm-20241031-3.14.0a1%2B-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-base-mem.svg) | +| [2024-10-31](results/bm-20241031-3.14.0a1%2B-ac3a2c8) | python/ac3a2c8abceeec448486 | ac3a2c8 | 1.00x ↓
[📄](results/bm-20241031-3.14.0a1%2B-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.12.6.md)[📈](results/bm-20241031-3.14.0a1%2B-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.12.6.svg) | 1.02x ↓
[📄](results/bm-20241031-3.14.0a1%2B-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.13.0rc2.md)[📈](results/bm-20241031-3.14.0a1%2B-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.13.0rc2.svg) | | | [2024-10-29](results/bm-20241029-3.14.0a1%2B-d4b6d84) | python/d4b6d84cc84029b598fc | d4b6d84 | 1.00x ↓
[📄](results/bm-20241029-3.14.0a1%2B-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.12.6.md)[📈](results/bm-20241029-3.14.0a1%2B-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.12.6.svg) | 1.02x ↓
[📄](results/bm-20241029-3.14.0a1%2B-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.13.0rc2.md)[📈](results/bm-20241029-3.14.0a1%2B-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.13.0rc2.svg) | | | [2024-10-29](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL) | python/d4b6d84cc84029b598fc | d4b6d84 (NOGIL) | 1.55x ↓
[📄](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.12.6.md)[📈](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.12.6.svg) | 1.58x ↓
[📄](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.13.0rc2.md)[📈](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-3.13.0rc2.svg) | 1.55x ↓
[📄](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-base.md)[📈](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-base.svg)[🧠](results/bm-20241029-3.14.0a1%2B-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1%2B-d4b6d84-vs-base-mem.svg) | | [2024-10-28](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL) | python/85799f1ffd5f285ef93a | 85799f1 (NOGIL) | 1.55x ↓
[📄](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-3.12.6.md)[📈](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-3.12.6.svg) | 1.57x ↓
[📄](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-3.13.0rc2.md)[📈](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-3.13.0rc2.svg) | 1.55x ↓
[📄](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-base.md)[📈](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-base.svg)[🧠](results/bm-20241028-3.14.0a1%2B-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1%2B-85799f1-vs-base-mem.svg) | diff --git a/configs.json b/configs.json index faa28dbf..70cbcc7a 100644 --- a/configs.json +++ b/configs.json @@ -36,5 +36,6 @@ "bm-20241026-3.14.0a1+-f6cc7c8/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1+-f6cc7c8.json,bm-20241026-3.14.0a1+-f6cc7c8-NOGIL/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1+-f6cc7c8.json": 0.5800000000000001, "bm-20241027-3.14.0a1+-19e93e2/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1+-19e93e2.json,bm-20241027-3.14.0a1+-19e93e2-NOGIL/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1+-19e93e2.json": 0.5800000000000001, "bm-20241028-3.14.0a1+-85799f1/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1+-85799f1.json,bm-20241028-3.14.0a1+-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1+-85799f1.json": 0.5800000000000001, - "bm-20241029-3.14.0a1+-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1+-d4b6d84.json,bm-20241029-3.14.0a1+-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1+-d4b6d84.json": 0.5800000000000001 + "bm-20241029-3.14.0a1+-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1+-d4b6d84.json,bm-20241029-3.14.0a1+-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1+-d4b6d84.json": 0.5800000000000001, + "bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json,bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json": 0.5900000000000001 } \ No newline at end of file diff --git a/configs.svg b/configs.svg index a305c880..9de74896 100644 --- a/configs.svg +++ b/configs.svg @@ -6,12 +6,12 @@ - + - - + + @@ -31,9 +31,9 @@ - - - + + + @@ -48,9 +48,9 @@ - - - + + + @@ -65,9 +65,9 @@ - - - + + + @@ -79,9 +79,9 @@ - - - + + + @@ -93,9 +93,9 @@ - - - + + + @@ -107,9 +107,9 @@ - - - + + + @@ -177,57 +177,58 @@ - + - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + - + @@ -372,36 +373,36 @@ - + - + - - - + + + - + - - - + + + @@ -461,7 +462,7 @@ - + @@ -488,7 +489,7 @@ - + diff --git a/longitudinal.json b/longitudinal.json index b9444a70..3b05c504 100644 --- a/longitudinal.json +++ b/longitudinal.json @@ -72,5 +72,6 @@ "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241026-3.14.0a1+-f6cc7c8/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1+-f6cc7c8.json,3.13.0rc2": 1.0, "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241027-3.14.0a1+-19e93e2/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1+-19e93e2.json,3.13.0rc2": 1.0, "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241028-3.14.0a1+-85799f1/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1+-85799f1.json,3.13.0rc2": 1.0, - "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241029-3.14.0a1+-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1+-d4b6d84.json,3.13.0rc2": 1.0 + "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241029-3.14.0a1+-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1+-d4b6d84.json,3.13.0rc2": 1.0, + "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json,3.13.0rc2": 1.0 } \ No newline at end of file diff --git a/longitudinal.svg b/longitudinal.svg index 837db6c9..fc8716b0 100644 --- a/longitudinal.svg +++ b/longitudinal.svg @@ -5,13 +5,13 @@ - - + + - - + + @@ -31,9 +31,9 @@ - - - + + + @@ -48,9 +48,9 @@ - - - + + + @@ -62,9 +62,9 @@ - - - + + + @@ -79,9 +79,9 @@ - - - + + + @@ -93,9 +93,9 @@ - - - + + + @@ -110,9 +110,9 @@ - - - + + + @@ -124,9 +124,9 @@ - - - + + + @@ -141,9 +141,9 @@ - - - + + + @@ -155,7 +155,7 @@ - + @@ -171,7 +171,7 @@ - + @@ -180,7 +180,7 @@ - + @@ -189,7 +189,7 @@ - + @@ -198,7 +198,7 @@ - + @@ -207,7 +207,7 @@ - + @@ -216,7 +216,7 @@ - + @@ -225,7 +225,7 @@ - + @@ -234,7 +234,7 @@ - + @@ -246,36 +246,36 @@ - + - - - + + + - + - - + + - - - - - + + + + + - - + + @@ -322,7 +322,7 @@ - + @@ -385,10 +385,10 @@ - + - - + + @@ -401,7 +401,7 @@ - + @@ -415,7 +415,7 @@ - + @@ -429,7 +429,7 @@ - + @@ -443,7 +443,7 @@ - + @@ -457,7 +457,7 @@ - + @@ -471,7 +471,7 @@ - + @@ -484,7 +484,7 @@ - + @@ -493,7 +493,7 @@ - + @@ -502,7 +502,7 @@ - + @@ -511,7 +511,7 @@ - + @@ -520,7 +520,7 @@ - + @@ -529,7 +529,7 @@ - + @@ -538,7 +538,7 @@ - + @@ -547,7 +547,7 @@ - + @@ -556,7 +556,7 @@ - + @@ -566,66 +566,67 @@ - + - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + - - - - - + + + + + @@ -653,7 +654,7 @@ - + @@ -756,10 +757,10 @@ - + - + diff --git a/memory_configs.json b/memory_configs.json index 5b9de620..2cd516bd 100644 --- a/memory_configs.json +++ b/memory_configs.json @@ -36,5 +36,6 @@ "bm-20241026-3.14.0a1+-f6cc7c8/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1+-f6cc7c8.json,bm-20241026-3.14.0a1+-f6cc7c8-NOGIL/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1+-f6cc7c8.json": 1.18, "bm-20241027-3.14.0a1+-19e93e2/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1+-19e93e2.json,bm-20241027-3.14.0a1+-19e93e2-NOGIL/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1+-19e93e2.json": 1.18, "bm-20241028-3.14.0a1+-85799f1/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1+-85799f1.json,bm-20241028-3.14.0a1+-85799f1-NOGIL/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1+-85799f1.json": 1.18, - "bm-20241029-3.14.0a1+-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1+-d4b6d84.json,bm-20241029-3.14.0a1+-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1+-d4b6d84.json": 1.19 + "bm-20241029-3.14.0a1+-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1+-d4b6d84.json,bm-20241029-3.14.0a1+-d4b6d84-NOGIL/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1+-d4b6d84.json": 1.19, + "bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json,bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json": 1.19 } \ No newline at end of file diff --git a/memory_configs.svg b/memory_configs.svg index dcfe0685..41310b4a 100644 --- a/memory_configs.svg +++ b/memory_configs.svg @@ -6,12 +6,12 @@ - + - - + + @@ -31,9 +31,9 @@ - - - + + + @@ -48,9 +48,9 @@ - - - + + + @@ -65,9 +65,9 @@ - - - + + + @@ -79,9 +79,9 @@ - - - + + + @@ -93,9 +93,9 @@ - - - + + + @@ -107,9 +107,9 @@ - - - + + + @@ -212,57 +212,58 @@ - + - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + - + @@ -402,22 +403,22 @@ - + - + - - - + + + @@ -426,15 +427,15 @@ - + - - - + + + @@ -494,7 +495,7 @@ - + @@ -517,7 +518,7 @@ - + diff --git a/memory_long.json b/memory_long.json index 0b9e07ce..112fb29f 100644 --- a/memory_long.json +++ b/memory_long.json @@ -72,5 +72,6 @@ "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241026-3.14.0a1+-f6cc7c8/bm-20241026-vultr-x86_64-python-f6cc7c8bd01d8468af70-3.14.0a1+-f6cc7c8.json,3.13.0rc2": 1.0, "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241027-3.14.0a1+-19e93e2/bm-20241027-vultr-x86_64-python-19e93e2e269889ecb3c4-3.14.0a1+-19e93e2.json,3.13.0rc2": 1.0, "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241028-3.14.0a1+-85799f1/bm-20241028-vultr-x86_64-python-85799f1ffd5f285ef93a-3.14.0a1+-85799f1.json,3.13.0rc2": 1.0, - "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241029-3.14.0a1+-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1+-d4b6d84.json,3.13.0rc2": 1.0 + "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241029-3.14.0a1+-d4b6d84/bm-20241029-vultr-x86_64-python-d4b6d84cc84029b598fc-3.14.0a1+-d4b6d84.json,3.13.0rc2": 1.0, + "bm-20240920-3.13.0rc2-4981ec5/bm-20240920-vultr-x86_64-python-4981ec59ded050919eb2-3.13.0rc2-4981ec5.json,bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json,3.13.0rc2": 1.0 } \ No newline at end of file diff --git a/memory_long.svg b/memory_long.svg index 8da4706c..9bca7101 100644 --- a/memory_long.svg +++ b/memory_long.svg @@ -5,13 +5,13 @@ - - + + - - + + @@ -31,9 +31,9 @@ - - - + + + @@ -48,9 +48,9 @@ - - - + + + @@ -62,9 +62,9 @@ - - - + + + @@ -79,9 +79,9 @@ - - - + + + @@ -93,9 +93,9 @@ - - - + + + @@ -110,9 +110,9 @@ - - - + + + @@ -124,9 +124,9 @@ - - - + + + @@ -141,9 +141,9 @@ - - - + + + @@ -155,7 +155,7 @@ - + @@ -171,7 +171,7 @@ - + @@ -180,7 +180,7 @@ - + @@ -189,7 +189,7 @@ - + @@ -198,7 +198,7 @@ - + @@ -207,7 +207,7 @@ - + @@ -216,7 +216,7 @@ - + @@ -225,7 +225,7 @@ - + @@ -234,7 +234,7 @@ - + @@ -246,36 +246,36 @@ - + - - - + + + - + - - + + - - - - - + + + + + - - + + @@ -315,7 +315,7 @@ - + @@ -379,10 +379,10 @@ - + - - + + @@ -395,7 +395,7 @@ - + @@ -409,7 +409,7 @@ - + @@ -423,7 +423,7 @@ - + @@ -437,7 +437,7 @@ - + @@ -451,7 +451,7 @@ - + @@ -465,7 +465,7 @@ - + @@ -478,7 +478,7 @@ - + @@ -487,7 +487,7 @@ - + @@ -496,7 +496,7 @@ - + @@ -505,7 +505,7 @@ - + @@ -514,7 +514,7 @@ - + @@ -523,7 +523,7 @@ - + @@ -532,7 +532,7 @@ - + @@ -541,7 +541,7 @@ - + @@ -550,7 +550,7 @@ - + @@ -560,65 +560,66 @@ - + - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + - - - - - + + + + + @@ -645,7 +646,7 @@ - + @@ -744,10 +745,10 @@ - + - + diff --git a/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/README.md b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/README.md new file mode 100644 index 00000000..f7b3d0f1 --- /dev/null +++ b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/README.md @@ -0,0 +1,41 @@ +# Results + +- fork: python +- version: 3.14.0a1+ +- config: NOGIL +- commit hash: [ac3a2c8](https://github.com/python/cpython/commit/ac3a2c8) +- commit date: 2024-10-31T00:12:37+00:00 +- commit merge base: [951cb2c369e8b8fb9f93662658391a53fd727787](https://github.com/python/cpython/commit/951cb2c369e8b8fb9f93662658391a53fd727787) +- ref: ac3a2c8abceeec448486 + +## linux x86_64 (vultr) + +- [GitHub Action run](https://github.com/facebookexperimental/free-threading-benchmarking/actions/runs/11603802920) +- cpu model: Intel(R) Xeon(R) E-2286G CPU @ 4.00GHz +- platform: Linux-6.8.0-45-generic-x86_64-with-glibc2.39 +- [raw results](bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8.json) + +### vs. 3.12.6 + +- Geometric mean: 1.55x slower (HPT: reliability of 100.00%, 1.39x slower at 99th %ile) +- Memory usage: 1.21x +- missing benchmarks: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn, mypy2, sqlalchemy_declarative, sqlalchemy_imperative +- [📄table](bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.12.6.md) +- [📈time plot](bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.12.6.svg) + +### vs. 3.13.0rc2 + +- Geometric mean: 1.57x slower (HPT: reliability of 100.00%, 1.42x slower at 99th %ile) +- Memory usage: 1.20x +- missing benchmarks: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn +- [📄table](bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.13.0rc2.md) +- [📈time plot](bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.13.0rc2.svg) + +### vs. base + +- Geometric mean: 1.55x slower (HPT: reliability of 100.00%, 1.41x slower at 99th %ile) +- Memory usage: 1.19x +- [🧠memory plot](bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-base-mem.svg) +- [📄table](bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-base.md) +- [📈time plot](bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-base.svg) + diff --git a/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.12.6.md b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.12.6.md new file mode 100644 index 00000000..ebc27214 --- /dev/null +++ b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.12.6.md @@ -0,0 +1,205 @@ +# Results vs. 3.12.6 + +- fork: python +- ref: ac3a2c8abceeec448486 +- machine: linux-x86_64 +- commit hash: ac3a2c8 +- commit date: 2024-10-31 +- overall geometric mean: 1.55x slower +- HPT reliability: 100.00% +- HPT 99th percentile: 1.39x slower +- Memory change: 1.21x + +Benchmarks with tag 'apps': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| 2to3 | 264 ms | 413 ms: 1.57x slower | +| docutils | 2.64 sec | 3.37 sec: 1.28x slower | +| html5lib | 63.6 ms | 106 ms: 1.67x slower | +| tornado_http | 119 ms | 162 ms: 1.36x slower | +| Geometric mean | (ref) | 1.46x slower | + +Benchmarks with tag 'asyncio': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|------------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| asyncio_tcp | 519 ms | 580 ms: 1.12x slower | +| asyncio_tcp_ssl | 1.51 sec | 1.80 sec: 1.19x slower | +| async_generators | 384 ms | 500 ms: 1.30x slower | +| coroutines | 23.9 ms | 32.3 ms: 1.35x slower | +| Geometric mean | (ref) | 1.19x slower | + +Benchmark hidden because not significant (1): asyncio_websockets + +Benchmarks with tag 'math': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| pidigits | 184 ms | 186 ms: 1.01x slower | +| float | 80.8 ms | 154 ms: 1.90x slower | +| nbody | 89.3 ms | 219 ms: 2.46x slower | +| Geometric mean | (ref) | 1.68x slower | + +Benchmarks with tag 'regex': +============================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| regex_dna | 168 ms | 186 ms: 1.11x slower | +| regex_v8 | 20.6 ms | 25.3 ms: 1.23x slower | +| regex_compile | 142 ms | 228 ms: 1.60x slower | +| Geometric mean | (ref) | 1.22x slower | + +Benchmark hidden because not significant (1): regex_effbot + +Benchmarks with tag 'serialize': +================================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| xml_etree_parse | 139 ms | 133 ms: 1.04x faster | +| pickle_dict | 31.8 us | 31.2 us: 1.02x faster | +| pickle | 10.9 us | 10.8 us: 1.02x faster | +| unpickle | 14.1 us | 14.8 us: 1.05x slower | +| unpickle_list | 4.67 us | 5.04 us: 1.08x slower | +| json_loads | 26.5 us | 29.8 us: 1.12x slower | +| xml_etree_iterparse | 96.7 ms | 111 ms: 1.14x slower | +| xml_etree_generate | 85.2 ms | 113 ms: 1.33x slower | +| json_dumps | 10.4 ms | 15.1 ms: 1.46x slower | +| tomli_loads | 2.11 sec | 3.30 sec: 1.56x slower | +| xml_etree_process | 59.0 ms | 93.5 ms: 1.59x slower | +| unpickle_pure_python | 221 us | 427 us: 1.94x slower | +| pickle_pure_python | 308 us | 620 us: 2.01x slower | +| Geometric mean | (ref) | 1.26x slower | + +Benchmark hidden because not significant (1): pickle_list + +Benchmarks with tag 'startup': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|------------------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| python_startup_no_site | 7.16 ms | 10.1 ms: 1.41x slower | +| python_startup | 9.93 ms | 15.4 ms: 1.55x slower | +| Geometric mean | (ref) | 1.48x slower | + +Benchmarks with tag 'template': +=============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|-----------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| genshi_xml | 50.2 ms | 82.4 ms: 1.64x slower | +| genshi_text | 22.8 ms | 40.4 ms: 1.77x slower | +| django_template | 34.7 ms | 64.9 ms: 1.87x slower | +| mako | 11.0 ms | 21.1 ms: 1.92x slower | +| Geometric mean | (ref) | 1.80x slower | + +All benchmarks: +=============== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|--------------------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| gc_traversal | 3.46 ms | 2.44 ms: 1.42x faster | +| xml_etree_parse | 139 ms | 133 ms: 1.04x faster | +| pickle_dict | 31.8 us | 31.2 us: 1.02x faster | +| pickle | 10.9 us | 10.8 us: 1.02x faster | +| pidigits | 184 ms | 186 ms: 1.01x slower | +| create_gc_cycles | 1.09 ms | 1.11 ms: 1.01x slower | +| pathlib | 21.5 ms | 22.0 ms: 1.02x slower | +| unpickle | 14.1 us | 14.8 us: 1.05x slower | +| unpickle_list | 4.67 us | 5.04 us: 1.08x slower | +| json | 5.02 ms | 5.51 ms: 1.10x slower | +| regex_dna | 168 ms | 186 ms: 1.11x slower | +| asyncio_tcp | 519 ms | 580 ms: 1.12x slower | +| sqlite_synth | 2.20 us | 2.47 us: 1.12x slower | +| json_loads | 26.5 us | 29.8 us: 1.12x slower | +| xml_etree_iterparse | 96.7 ms | 111 ms: 1.14x slower | +| asyncio_tcp_ssl | 1.51 sec | 1.80 sec: 1.19x slower | +| regex_v8 | 20.6 ms | 25.3 ms: 1.23x slower | +| deepcopy | 352 us | 438 us: 1.24x slower | +| mdp | 2.42 sec | 3.08 sec: 1.27x slower | +| docutils | 2.64 sec | 3.37 sec: 1.28x slower | +| async_generators | 384 ms | 500 ms: 1.30x slower | +| dulwich_log | 78.9 ms | 104 ms: 1.31x slower | +| pylint | 319 ms | 419 ms: 1.32x slower | +| scimark_fft | 342 ms | 452 ms: 1.32x slower | +| xml_etree_generate | 85.2 ms | 113 ms: 1.33x slower | +| generators | 32.2 ms | 42.8 ms: 1.33x slower | +| bpe_tokeniser | 4.74 sec | 6.32 sec: 1.33x slower | +| meteor_contest | 104 ms | 138 ms: 1.34x slower | +| deepcopy_memo | 40.3 us | 54.3 us: 1.35x slower | +| coroutines | 23.9 ms | 32.3 ms: 1.35x slower | +| tornado_http | 119 ms | 162 ms: 1.36x slower | +| python_startup_no_site | 7.16 ms | 10.1 ms: 1.41x slower | +| crypto_pyaes | 76.6 ms | 109 ms: 1.43x slower | +| coverage | 71.4 ms | 102 ms: 1.43x slower | +| telco | 6.53 ms | 9.50 ms: 1.46x slower | +| pycparser | 1.17 sec | 1.70 sec: 1.46x slower | +| json_dumps | 10.4 ms | 15.1 ms: 1.46x slower | +| scimark_sparse_mat_mult | 4.39 ms | 6.45 ms: 1.47x slower | +| deepcopy_reduce | 3.08 us | 4.59 us: 1.49x slower | +| nqueens | 80.1 ms | 120 ms: 1.49x slower | +| fannkuch | 372 ms | 576 ms: 1.55x slower | +| python_startup | 9.93 ms | 15.4 ms: 1.55x slower | +| typing_runtime_protocols | 163 us | 254 us: 1.56x slower | +| comprehensions | 19.8 us | 31.0 us: 1.56x slower | +| tomli_loads | 2.11 sec | 3.30 sec: 1.56x slower | +| 2to3 | 264 ms | 413 ms: 1.57x slower | +| xml_etree_process | 59.0 ms | 93.5 ms: 1.59x slower | +| thrift | 791 us | 1.26 ms: 1.60x slower | +| spectral_norm | 110 ms | 176 ms: 1.60x slower | +| regex_compile | 142 ms | 228 ms: 1.60x slower | +| sympy_integrate | 20.5 ms | 33.0 ms: 1.61x slower | +| genshi_xml | 50.2 ms | 82.4 ms: 1.64x slower | +| html5lib | 63.6 ms | 106 ms: 1.67x slower | +| sqlglot_normalize | 107 ms | 181 ms: 1.70x slower | +| sqlglot_optimize | 53.3 ms | 91.6 ms: 1.72x slower | +| pyflate | 448 ms | 790 ms: 1.76x slower | +| genshi_text | 22.8 ms | 40.4 ms: 1.77x slower | +| pprint_safe_repr | 743 ms | 1.35 sec: 1.82x slower | +| pprint_pformat | 1.52 sec | 2.81 sec: 1.85x slower | +| sympy_str | 292 ms | 540 ms: 1.85x slower | +| logging_format | 7.35 us | 13.7 us: 1.86x slower | +| logging_simple | 6.63 us | 12.4 us: 1.87x slower | +| django_template | 34.7 ms | 64.9 ms: 1.87x slower | +| float | 80.8 ms | 154 ms: 1.90x slower | +| scimark_monte_carlo | 68.4 ms | 131 ms: 1.92x slower | +| mako | 11.0 ms | 21.1 ms: 1.92x slower | +| unpickle_pure_python | 221 us | 427 us: 1.94x slower | +| richards | 45.9 ms | 89.7 ms: 1.95x slower | +| sqlglot_transpile | 1.67 ms | 3.31 ms: 1.98x slower | +| logging_silent | 109 ns | 217 ns: 1.99x slower | +| pickle_pure_python | 308 us | 620 us: 2.01x slower | +| hexiom | 6.17 ms | 12.6 ms: 2.04x slower | +| chaos | 62.8 ms | 129 ms: 2.05x slower | +| richards_super | 51.9 ms | 110 ms: 2.12x slower | +| sqlglot_parse | 1.36 ms | 2.90 ms: 2.14x slower | +| go | 139 ms | 299 ms: 2.15x slower | +| raytrace | 299 ms | 650 ms: 2.17x slower | +| scimark_sor | 130 ms | 283 ms: 2.18x slower | +| scimark_lu | 114 ms | 252 ms: 2.21x slower | +| sympy_expand | 468 ms | 1.06 sec: 2.26x slower | +| sympy_sum | 166 ms | 382 ms: 2.30x slower | +| nbody | 89.3 ms | 219 ms: 2.46x slower | +| deltablue | 3.45 ms | 9.25 ms: 2.68x slower | +| unpack_sequence | 52.1 ns | 144 ns: 2.77x slower | +| bench_thread_pool | 941 us | 3.19 ms: 3.38x slower | +| bench_mp_pool | 10.8 ms | 70.8 ms: 6.55x slower | +| Geometric mean | (ref) | 1.55x slower | + +Benchmark hidden because not significant (3): pickle_list, regex_effbot, asyncio_websockets +Ignored benchmarks (16) of results/bm-20240906-3.12.6-a4a2d2b/bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b.json: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn, mypy2, sqlalchemy_declarative, sqlalchemy_imperative + +# HPT report + +- Reliability score: 100.00% likely to be slow +- 90% likely to have a slowdown of 1.46x +- 95% likely to have a slowdown of 1.44x +- 99% likely to have a slowdown of 1.39x + +# Memory +- memory change: 1.21x \ No newline at end of file diff --git a/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.12.6.svg b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.12.6.svg new file mode 100644 index 00000000..6f2d70a9 --- /dev/null +++ b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.12.6.svg @@ -0,0 +1,3092 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.13.0rc2.md b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.13.0rc2.md new file mode 100644 index 00000000..675880b1 --- /dev/null +++ b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.13.0rc2.md @@ -0,0 +1,203 @@ +# Results vs. 3.13.0rc2 + +- fork: python +- ref: ac3a2c8abceeec448486 +- machine: linux-x86_64 +- commit hash: ac3a2c8 +- commit date: 2024-10-31 +- overall geometric mean: 1.57x slower +- HPT reliability: 100.00% +- HPT 99th percentile: 1.42x slower +- Memory change: 1.20x + +Benchmarks with tag 'apps': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| 2to3 | 260 ms | 413 ms: 1.59x slower | +| docutils | 2.62 sec | 3.37 sec: 1.29x slower | +| html5lib | 67.0 ms | 106 ms: 1.58x slower | +| tornado_http | 114 ms | 162 ms: 1.42x slower | +| Geometric mean | (ref) | 1.46x slower | + +Benchmarks with tag 'asyncio': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|--------------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| asyncio_websockets | 520 ms | 518 ms: 1.01x faster | +| asyncio_tcp | 505 ms | 580 ms: 1.15x slower | +| asyncio_tcp_ssl | 1.51 sec | 1.80 sec: 1.19x slower | +| async_generators | 377 ms | 500 ms: 1.33x slower | +| coroutines | 23.6 ms | 32.3 ms: 1.37x slower | +| Geometric mean | (ref) | 1.20x slower | + +Benchmarks with tag 'math': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| pidigits | 217 ms | 186 ms: 1.17x faster | +| float | 77.5 ms | 154 ms: 1.98x slower | +| nbody | 85.1 ms | 219 ms: 2.58x slower | +| Geometric mean | (ref) | 1.64x slower | + +Benchmarks with tag 'regex': +============================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| regex_effbot | 3.08 ms | 3.17 ms: 1.03x slower | +| regex_dna | 180 ms | 186 ms: 1.03x slower | +| regex_v8 | 22.7 ms | 25.3 ms: 1.12x slower | +| regex_compile | 132 ms | 228 ms: 1.72x slower | +| Geometric mean | (ref) | 1.19x slower | + +Benchmarks with tag 'serialize': +================================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| pickle | 11.3 us | 10.8 us: 1.05x faster | +| pickle_dict | 32.5 us | 31.2 us: 1.04x faster | +| pickle_list | 4.93 us | 4.76 us: 1.04x faster | +| xml_etree_parse | 136 ms | 133 ms: 1.03x faster | +| unpickle | 14.3 us | 14.8 us: 1.03x slower | +| unpickle_list | 4.71 us | 5.04 us: 1.07x slower | +| json_loads | 27.0 us | 29.8 us: 1.10x slower | +| xml_etree_iterparse | 94.9 ms | 111 ms: 1.17x slower | +| xml_etree_generate | 85.4 ms | 113 ms: 1.32x slower | +| json_dumps | 10.5 ms | 15.1 ms: 1.43x slower | +| xml_etree_process | 59.3 ms | 93.5 ms: 1.58x slower | +| tomli_loads | 2.01 sec | 3.30 sec: 1.64x slower | +| unpickle_pure_python | 210 us | 427 us: 2.03x slower | +| pickle_pure_python | 294 us | 620 us: 2.10x slower | +| Geometric mean | (ref) | 1.26x slower | + +Benchmarks with tag 'startup': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|------------------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| python_startup_no_site | 7.39 ms | 10.1 ms: 1.36x slower | +| python_startup | 11.0 ms | 15.4 ms: 1.40x slower | +| Geometric mean | (ref) | 1.38x slower | + +Benchmarks with tag 'template': +=============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|-----------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| genshi_xml | 48.8 ms | 82.4 ms: 1.69x slower | +| mako | 11.3 ms | 21.1 ms: 1.86x slower | +| genshi_text | 21.5 ms | 40.4 ms: 1.87x slower | +| django_template | 34.1 ms | 64.9 ms: 1.90x slower | +| Geometric mean | (ref) | 1.83x slower | + +All benchmarks: +=============== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|--------------------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| gc_traversal | 3.14 ms | 2.44 ms: 1.29x faster | +| create_gc_cycles | 1.34 ms | 1.11 ms: 1.21x faster | +| pidigits | 217 ms | 186 ms: 1.17x faster | +| pickle | 11.3 us | 10.8 us: 1.05x faster | +| pickle_dict | 32.5 us | 31.2 us: 1.04x faster | +| pickle_list | 4.93 us | 4.76 us: 1.04x faster | +| xml_etree_parse | 136 ms | 133 ms: 1.03x faster | +| asyncio_websockets | 520 ms | 518 ms: 1.01x faster | +| regex_effbot | 3.08 ms | 3.17 ms: 1.03x slower | +| regex_dna | 180 ms | 186 ms: 1.03x slower | +| unpickle | 14.3 us | 14.8 us: 1.03x slower | +| unpickle_list | 4.71 us | 5.04 us: 1.07x slower | +| json_loads | 27.0 us | 29.8 us: 1.10x slower | +| regex_v8 | 22.7 ms | 25.3 ms: 1.12x slower | +| json | 4.93 ms | 5.51 ms: 1.12x slower | +| sqlite_synth | 2.21 us | 2.47 us: 1.12x slower | +| pathlib | 19.2 ms | 22.0 ms: 1.14x slower | +| asyncio_tcp | 505 ms | 580 ms: 1.15x slower | +| xml_etree_iterparse | 94.9 ms | 111 ms: 1.17x slower | +| asyncio_tcp_ssl | 1.51 sec | 1.80 sec: 1.19x slower | +| telco | 7.82 ms | 9.50 ms: 1.21x slower | +| deepcopy | 355 us | 438 us: 1.23x slower | +| coverage | 83.0 ms | 102 ms: 1.23x slower | +| docutils | 2.62 sec | 3.37 sec: 1.29x slower | +| scimark_fft | 349 ms | 452 ms: 1.29x slower | +| mdp | 2.36 sec | 3.08 sec: 1.31x slower | +| pylint | 317 ms | 419 ms: 1.32x slower | +| xml_etree_generate | 85.4 ms | 113 ms: 1.32x slower | +| async_generators | 377 ms | 500 ms: 1.33x slower | +| meteor_contest | 102 ms | 138 ms: 1.36x slower | +| python_startup_no_site | 7.39 ms | 10.1 ms: 1.36x slower | +| coroutines | 23.6 ms | 32.3 ms: 1.37x slower | +| scimark_sparse_mat_mult | 4.71 ms | 6.45 ms: 1.37x slower | +| dulwich_log | 74.8 ms | 104 ms: 1.39x slower | +| deepcopy_memo | 39.1 us | 54.3 us: 1.39x slower | +| python_startup | 11.0 ms | 15.4 ms: 1.40x slower | +| bpe_tokeniser | 4.45 sec | 6.32 sec: 1.42x slower | +| tornado_http | 114 ms | 162 ms: 1.42x slower | +| json_dumps | 10.5 ms | 15.1 ms: 1.43x slower | +| deepcopy_reduce | 3.11 us | 4.59 us: 1.47x slower | +| generators | 28.8 ms | 42.8 ms: 1.48x slower | +| nqueens | 78.6 ms | 120 ms: 1.52x slower | +| pycparser | 1.12 sec | 1.70 sec: 1.52x slower | +| fannkuch | 370 ms | 576 ms: 1.56x slower | +| xml_etree_process | 59.3 ms | 93.5 ms: 1.58x slower | +| html5lib | 67.0 ms | 106 ms: 1.58x slower | +| spectral_norm | 111 ms | 176 ms: 1.58x slower | +| 2to3 | 260 ms | 413 ms: 1.59x slower | +| crypto_pyaes | 67.9 ms | 109 ms: 1.61x slower | +| thrift | 778 us | 1.26 ms: 1.62x slower | +| tomli_loads | 2.01 sec | 3.30 sec: 1.64x slower | +| typing_runtime_protocols | 155 us | 254 us: 1.64x slower | +| sympy_integrate | 19.8 ms | 33.0 ms: 1.66x slower | +| genshi_xml | 48.8 ms | 82.4 ms: 1.69x slower | +| sqlglot_normalize | 106 ms | 181 ms: 1.71x slower | +| regex_compile | 132 ms | 228 ms: 1.72x slower | +| sqlglot_optimize | 52.7 ms | 91.6 ms: 1.74x slower | +| pyflate | 449 ms | 790 ms: 1.76x slower | +| pprint_safe_repr | 738 ms | 1.35 sec: 1.83x slower | +| mako | 11.3 ms | 21.1 ms: 1.86x slower | +| genshi_text | 21.5 ms | 40.4 ms: 1.87x slower | +| pprint_pformat | 1.50 sec | 2.81 sec: 1.87x slower | +| comprehensions | 16.5 us | 31.0 us: 1.88x slower | +| django_template | 34.1 ms | 64.9 ms: 1.90x slower | +| sympy_str | 275 ms | 540 ms: 1.97x slower | +| richards | 45.2 ms | 89.7 ms: 1.98x slower | +| float | 77.5 ms | 154 ms: 1.98x slower | +| logging_format | 6.84 us | 13.7 us: 2.00x slower | +| scimark_monte_carlo | 65.4 ms | 131 ms: 2.01x slower | +| logging_simple | 6.16 us | 12.4 us: 2.01x slower | +| unpickle_pure_python | 210 us | 427 us: 2.03x slower | +| hexiom | 5.99 ms | 12.6 ms: 2.10x slower | +| scimark_sor | 134 ms | 283 ms: 2.10x slower | +| pickle_pure_python | 294 us | 620 us: 2.10x slower | +| logging_silent | 103 ns | 217 ns: 2.11x slower | +| go | 141 ms | 299 ms: 2.12x slower | +| sqlglot_transpile | 1.56 ms | 3.31 ms: 2.12x slower | +| richards_super | 51.6 ms | 110 ms: 2.13x slower | +| scimark_lu | 113 ms | 252 ms: 2.24x slower | +| chaos | 57.3 ms | 129 ms: 2.24x slower | +| sympy_expand | 457 ms | 1.06 sec: 2.31x slower | +| sqlglot_parse | 1.25 ms | 2.90 ms: 2.32x slower | +| sympy_sum | 156 ms | 382 ms: 2.46x slower | +| raytrace | 253 ms | 650 ms: 2.57x slower | +| nbody | 85.1 ms | 219 ms: 2.58x slower | +| deltablue | 3.12 ms | 9.25 ms: 2.96x slower | +| unpack_sequence | 44.8 ns | 144 ns: 3.22x slower | +| bench_thread_pool | 919 us | 3.19 ms: 3.47x slower | +| bench_mp_pool | 11.0 ms | 70.8 ms: 6.44x slower | +| Geometric mean | (ref) | 1.57x slower | +Ignored benchmarks (13) of results/bm-20240906-3.13.0rc2-ec61006/bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006.json: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn + +# HPT report + +- Reliability score: 100.00% likely to be slow +- 90% likely to have a slowdown of 1.49x +- 95% likely to have a slowdown of 1.47x +- 99% likely to have a slowdown of 1.42x + +# Memory +- memory change: 1.20x \ No newline at end of file diff --git a/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.13.0rc2.svg b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.13.0rc2.svg new file mode 100644 index 00000000..4787a8c9 --- /dev/null +++ b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.13.0rc2.svg @@ -0,0 +1,3063 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-base-mem.svg b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-base-mem.svg new file mode 100644 index 00000000..7904b5cd --- /dev/null +++ b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-base-mem.svg @@ -0,0 +1,2250 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-base.md b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-base.md new file mode 100644 index 00000000..883c5b39 --- /dev/null +++ b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-base.md @@ -0,0 +1,202 @@ +# Results vs. base + +- fork: python +- ref: ac3a2c8abceeec448486 +- machine: linux-x86_64 +- commit hash: ac3a2c8 +- commit date: 2024-10-31 +- overall geometric mean: 1.55x slower +- HPT reliability: 100.00% +- HPT 99th percentile: 1.41x slower +- Memory change: 1.19x + +Benchmarks with tag 'apps': +=========================== + +| Benchmark | results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | +|----------------|:-----------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------:| +| 2to3 | 254 ms | 413 ms: 1.63x slower | +| docutils | 2.60 sec | 3.37 sec: 1.29x slower | +| html5lib | 65.3 ms | 106 ms: 1.62x slower | +| tornado_http | 114 ms | 162 ms: 1.42x slower | +| Geometric mean | (ref) | 1.48x slower | + +Benchmarks with tag 'asyncio': +============================== + +| Benchmark | results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | +|--------------------|:-----------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------:| +| asyncio_websockets | 521 ms | 518 ms: 1.01x faster | +| asyncio_tcp | 506 ms | 580 ms: 1.14x slower | +| asyncio_tcp_ssl | 1.52 sec | 1.80 sec: 1.18x slower | +| async_generators | 372 ms | 500 ms: 1.35x slower | +| coroutines | 23.7 ms | 32.3 ms: 1.36x slower | +| Geometric mean | (ref) | 1.20x slower | + +Benchmarks with tag 'math': +=========================== + +| Benchmark | results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | +|----------------|:-----------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------:| +| pidigits | 218 ms | 186 ms: 1.17x faster | +| float | 80.1 ms | 154 ms: 1.92x slower | +| nbody | 94.7 ms | 219 ms: 2.32x slower | +| Geometric mean | (ref) | 1.56x slower | + +Benchmarks with tag 'regex': +============================ + +| Benchmark | results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | +|----------------|:-----------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------:| +| regex_effbot | 3.27 ms | 3.17 ms: 1.03x faster | +| regex_dna | 180 ms | 186 ms: 1.03x slower | +| regex_v8 | 23.1 ms | 25.3 ms: 1.10x slower | +| regex_compile | 131 ms | 228 ms: 1.73x slower | +| Geometric mean | (ref) | 1.18x slower | + +Benchmarks with tag 'serialize': +================================ + +| Benchmark | results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | +|----------------------|:-----------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------:| +| pickle_list | 4.90 us | 4.76 us: 1.03x faster | +| xml_etree_parse | 136 ms | 133 ms: 1.03x faster | +| pickle | 10.9 us | 10.8 us: 1.01x faster | +| pickle_dict | 30.5 us | 31.2 us: 1.02x slower | +| unpickle_list | 4.66 us | 5.04 us: 1.08x slower | +| unpickle | 13.4 us | 14.8 us: 1.10x slower | +| json_loads | 26.0 us | 29.8 us: 1.15x slower | +| xml_etree_iterparse | 95.7 ms | 111 ms: 1.16x slower | +| json_dumps | 11.5 ms | 15.1 ms: 1.31x slower | +| xml_etree_generate | 84.6 ms | 113 ms: 1.34x slower | +| tomli_loads | 2.09 sec | 3.30 sec: 1.57x slower | +| xml_etree_process | 59.0 ms | 93.5 ms: 1.59x slower | +| unpickle_pure_python | 217 us | 427 us: 1.97x slower | +| pickle_pure_python | 307 us | 620 us: 2.02x slower | +| Geometric mean | (ref) | 1.26x slower | + +Benchmarks with tag 'startup': +============================== + +| Benchmark | results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | +|------------------------|:-----------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------:| +| python_startup_no_site | 7.45 ms | 10.1 ms: 1.35x slower | +| python_startup | 11.1 ms | 15.4 ms: 1.39x slower | +| Geometric mean | (ref) | 1.37x slower | + +Benchmarks with tag 'template': +=============================== + +| Benchmark | results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | +|-----------------|:-----------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------:| +| genshi_xml | 50.2 ms | 82.4 ms: 1.64x slower | +| mako | 11.7 ms | 21.1 ms: 1.80x slower | +| genshi_text | 22.3 ms | 40.4 ms: 1.81x slower | +| django_template | 35.2 ms | 64.9 ms: 1.84x slower | +| Geometric mean | (ref) | 1.77x slower | + +All benchmarks: +=============== + +| Benchmark | results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8.json | +|--------------------------|:-----------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------:| +| gc_traversal | 3.20 ms | 2.44 ms: 1.31x faster | +| create_gc_cycles | 1.38 ms | 1.11 ms: 1.25x faster | +| pidigits | 218 ms | 186 ms: 1.17x faster | +| pickle_list | 4.90 us | 4.76 us: 1.03x faster | +| regex_effbot | 3.27 ms | 3.17 ms: 1.03x faster | +| xml_etree_parse | 136 ms | 133 ms: 1.03x faster | +| pickle | 10.9 us | 10.8 us: 1.01x faster | +| asyncio_websockets | 521 ms | 518 ms: 1.01x faster | +| pickle_dict | 30.5 us | 31.2 us: 1.02x slower | +| regex_dna | 180 ms | 186 ms: 1.03x slower | +| unpickle_list | 4.66 us | 5.04 us: 1.08x slower | +| regex_v8 | 23.1 ms | 25.3 ms: 1.10x slower | +| unpickle | 13.4 us | 14.8 us: 1.10x slower | +| bench_mp_pool | 63.7 ms | 70.8 ms: 1.11x slower | +| json | 4.94 ms | 5.51 ms: 1.11x slower | +| sqlite_synth | 2.19 us | 2.47 us: 1.13x slower | +| asyncio_tcp | 506 ms | 580 ms: 1.14x slower | +| json_loads | 26.0 us | 29.8 us: 1.15x slower | +| xml_etree_iterparse | 95.7 ms | 111 ms: 1.16x slower | +| asyncio_tcp_ssl | 1.52 sec | 1.80 sec: 1.18x slower | +| pathlib | 18.4 ms | 22.0 ms: 1.19x slower | +| mdp | 2.50 sec | 3.08 sec: 1.23x slower | +| coverage | 81.3 ms | 102 ms: 1.26x slower | +| telco | 7.34 ms | 9.50 ms: 1.29x slower | +| docutils | 2.60 sec | 3.37 sec: 1.29x slower | +| json_dumps | 11.5 ms | 15.1 ms: 1.31x slower | +| pylint | 319 ms | 419 ms: 1.32x slower | +| scimark_fft | 341 ms | 452 ms: 1.32x slower | +| xml_etree_generate | 84.6 ms | 113 ms: 1.34x slower | +| async_generators | 372 ms | 500 ms: 1.35x slower | +| python_startup_no_site | 7.45 ms | 10.1 ms: 1.35x slower | +| coroutines | 23.7 ms | 32.3 ms: 1.36x slower | +| dulwich_log | 75.7 ms | 104 ms: 1.37x slower | +| meteor_contest | 101 ms | 138 ms: 1.37x slower | +| python_startup | 11.1 ms | 15.4 ms: 1.39x slower | +| scimark_sparse_mat_mult | 4.63 ms | 6.45 ms: 1.39x slower | +| tornado_http | 114 ms | 162 ms: 1.42x slower | +| bpe_tokeniser | 4.39 sec | 6.32 sec: 1.44x slower | +| generators | 29.3 ms | 42.8 ms: 1.46x slower | +| pycparser | 1.13 sec | 1.70 sec: 1.50x slower | +| spectral_norm | 116 ms | 176 ms: 1.52x slower | +| nqueens | 78.5 ms | 120 ms: 1.52x slower | +| fannkuch | 373 ms | 576 ms: 1.55x slower | +| tomli_loads | 2.09 sec | 3.30 sec: 1.57x slower | +| xml_etree_process | 59.0 ms | 93.5 ms: 1.59x slower | +| typing_runtime_protocols | 159 us | 254 us: 1.60x slower | +| html5lib | 65.3 ms | 106 ms: 1.62x slower | +| crypto_pyaes | 67.2 ms | 109 ms: 1.62x slower | +| 2to3 | 254 ms | 413 ms: 1.63x slower | +| genshi_xml | 50.2 ms | 82.4 ms: 1.64x slower | +| deepcopy | 266 us | 438 us: 1.65x slower | +| sympy_integrate | 19.9 ms | 33.0 ms: 1.65x slower | +| thrift | 757 us | 1.26 ms: 1.67x slower | +| sqlglot_normalize | 107 ms | 181 ms: 1.69x slower | +| sqlglot_optimize | 53.7 ms | 91.6 ms: 1.71x slower | +| regex_compile | 131 ms | 228 ms: 1.73x slower | +| deepcopy_reduce | 2.64 us | 4.59 us: 1.74x slower | +| pyflate | 447 ms | 790 ms: 1.77x slower | +| mako | 11.7 ms | 21.1 ms: 1.80x slower | +| genshi_text | 22.3 ms | 40.4 ms: 1.81x slower | +| deepcopy_memo | 30.1 us | 54.3 us: 1.81x slower | +| comprehensions | 17.1 us | 31.0 us: 1.82x slower | +| django_template | 35.2 ms | 64.9 ms: 1.84x slower | +| pprint_safe_repr | 715 ms | 1.35 sec: 1.89x slower | +| pprint_pformat | 1.47 sec | 2.81 sec: 1.90x slower | +| float | 80.1 ms | 154 ms: 1.92x slower | +| richards | 45.9 ms | 89.7 ms: 1.95x slower | +| unpickle_pure_python | 217 us | 427 us: 1.97x slower | +| sympy_str | 272 ms | 540 ms: 1.99x slower | +| scimark_monte_carlo | 65.5 ms | 131 ms: 2.01x slower | +| logging_format | 6.80 us | 13.7 us: 2.01x slower | +| pickle_pure_python | 307 us | 620 us: 2.02x slower | +| logging_simple | 6.01 us | 12.4 us: 2.06x slower | +| sqlglot_transpile | 1.61 ms | 3.31 ms: 2.06x slower | +| logging_silent | 105 ns | 217 ns: 2.07x slower | +| scimark_sor | 136 ms | 283 ms: 2.08x slower | +| hexiom | 5.92 ms | 12.6 ms: 2.12x slower | +| richards_super | 51.9 ms | 110 ms: 2.12x slower | +| chaos | 58.5 ms | 129 ms: 2.20x slower | +| sqlglot_parse | 1.30 ms | 2.90 ms: 2.22x slower | +| scimark_lu | 111 ms | 252 ms: 2.28x slower | +| sympy_expand | 461 ms | 1.06 sec: 2.29x slower | +| nbody | 94.7 ms | 219 ms: 2.32x slower | +| sympy_sum | 155 ms | 382 ms: 2.46x slower | +| raytrace | 264 ms | 650 ms: 2.47x slower | +| go | 120 ms | 299 ms: 2.48x slower | +| deltablue | 3.24 ms | 9.25 ms: 2.86x slower | +| unpack_sequence | 50.1 ns | 144 ns: 2.88x slower | +| bench_thread_pool | 1.01 ms | 3.19 ms: 3.15x slower | +| Geometric mean | (ref) | 1.55x slower | + +# HPT report + +- Reliability score: 100.00% likely to be slow +- 90% likely to have a slowdown of 1.48x +- 95% likely to have a slowdown of 1.46x +- 99% likely to have a slowdown of 1.41x + +# Memory +- memory change: 1.19x \ No newline at end of file diff --git a/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-base.svg b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-base.svg new file mode 100644 index 00000000..8a935147 --- /dev/null +++ b/results/bm-20241031-3.14.0a1+-ac3a2c8-NOGIL/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-base.svg @@ -0,0 +1,3064 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/results/bm-20241031-3.14.0a1+-ac3a2c8/README.md b/results/bm-20241031-3.14.0a1+-ac3a2c8/README.md new file mode 100644 index 00000000..cd77d1dd --- /dev/null +++ b/results/bm-20241031-3.14.0a1+-ac3a2c8/README.md @@ -0,0 +1,33 @@ +# Results + +- fork: python +- version: 3.14.0a1+ +- config: +- commit hash: [ac3a2c8](https://github.com/python/cpython/commit/ac3a2c8) +- commit date: 2024-10-31T00:12:37+00:00 +- commit merge base: [951cb2c369e8b8fb9f93662658391a53fd727787](https://github.com/python/cpython/commit/951cb2c369e8b8fb9f93662658391a53fd727787) +- ref: ac3a2c8abceeec448486 + +## linux x86_64 (vultr) + +- [GitHub Action run](https://github.com/facebookexperimental/free-threading-benchmarking/actions/runs/11603802920) +- cpu model: Intel(R) Xeon(R) E-2286G CPU @ 4.00GHz +- platform: Linux-6.8.0-45-generic-x86_64-with-glibc2.39 +- [raw results](bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8.json) + +### vs. 3.12.6 + +- Geometric mean: 1.00x slower (HPT: reliability of 99.33%, 1.00x faster at 99th %ile) +- Memory usage: 1.02x +- missing benchmarks: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn, mypy2, sqlalchemy_declarative, sqlalchemy_imperative +- [📄table](bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.12.6.md) +- [📈time plot](bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.12.6.svg) + +### vs. 3.13.0rc2 + +- Geometric mean: 1.02x slower (HPT: reliability of 87.78%, 1.00x slower at 99th %ile) +- Memory usage: 1.00x +- missing benchmarks: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn +- [📄table](bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.13.0rc2.md) +- [📈time plot](bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1%2B-ac3a2c8-vs-3.13.0rc2.svg) + diff --git a/results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.12.6.md b/results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.12.6.md new file mode 100644 index 00000000..a17494d4 --- /dev/null +++ b/results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.12.6.md @@ -0,0 +1,195 @@ +# Results vs. 3.12.6 + +- fork: python +- ref: ac3a2c8abceeec448486 +- machine: linux-x86_64 +- commit hash: ac3a2c8 +- commit date: 2024-10-31 +- overall geometric mean: 1.00x slower +- HPT reliability: 99.33% +- HPT 99th percentile: 1.00x faster +- Memory change: 1.02x + +Benchmarks with tag 'apps': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| 2to3 | 264 ms | 254 ms: 1.04x faster | +| docutils | 2.64 sec | 2.60 sec: 1.01x faster | +| html5lib | 63.6 ms | 65.3 ms: 1.03x slower | +| tornado_http | 119 ms | 114 ms: 1.04x faster | +| Geometric mean | (ref) | 1.02x faster | + +Benchmarks with tag 'asyncio': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|--------------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| async_generators | 384 ms | 372 ms: 1.03x faster | +| asyncio_tcp | 519 ms | 506 ms: 1.02x faster | +| coroutines | 23.9 ms | 23.7 ms: 1.01x faster | +| asyncio_websockets | 517 ms | 521 ms: 1.01x slower | +| asyncio_tcp_ssl | 1.51 sec | 1.52 sec: 1.01x slower | +| Geometric mean | (ref) | 1.01x faster | + +Benchmarks with tag 'math': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| float | 80.8 ms | 80.1 ms: 1.01x faster | +| nbody | 89.3 ms | 94.7 ms: 1.06x slower | +| pidigits | 184 ms | 218 ms: 1.18x slower | +| Geometric mean | (ref) | 1.08x slower | + +Benchmarks with tag 'regex': +============================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| regex_compile | 142 ms | 131 ms: 1.08x faster | +| regex_effbot | 3.17 ms | 3.27 ms: 1.03x slower | +| regex_dna | 168 ms | 180 ms: 1.07x slower | +| regex_v8 | 20.6 ms | 23.1 ms: 1.12x slower | +| Geometric mean | (ref) | 1.03x slower | + +Benchmarks with tag 'serialize': +================================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| unpickle | 14.1 us | 13.4 us: 1.05x faster | +| pickle_dict | 31.8 us | 30.5 us: 1.04x faster | +| json_loads | 26.5 us | 26.0 us: 1.02x faster | +| xml_etree_parse | 139 ms | 136 ms: 1.02x faster | +| unpickle_pure_python | 221 us | 217 us: 1.02x faster | +| xml_etree_iterparse | 96.7 ms | 95.7 ms: 1.01x faster | +| xml_etree_generate | 85.2 ms | 84.6 ms: 1.01x faster | +| tomli_loads | 2.11 sec | 2.09 sec: 1.01x faster | +| unpickle_list | 4.67 us | 4.66 us: 1.00x faster | +| pickle_list | 4.77 us | 4.90 us: 1.03x slower | +| json_dumps | 10.4 ms | 11.5 ms: 1.11x slower | +| Geometric mean | (ref) | 1.00x faster | + +Benchmark hidden because not significant (3): pickle, pickle_pure_python, xml_etree_process + +Benchmarks with tag 'startup': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|------------------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| python_startup_no_site | 7.16 ms | 7.45 ms: 1.04x slower | +| python_startup | 9.93 ms | 11.1 ms: 1.12x slower | +| Geometric mean | (ref) | 1.08x slower | + +Benchmarks with tag 'template': +=============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|-----------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| genshi_text | 22.8 ms | 22.3 ms: 1.02x faster | +| django_template | 34.7 ms | 35.2 ms: 1.01x slower | +| mako | 11.0 ms | 11.7 ms: 1.07x slower | +| Geometric mean | (ref) | 1.01x slower | + +Benchmark hidden because not significant (1): genshi_xml + +All benchmarks: +=============== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|--------------------------|:------------------------------------------------------:|:----------------------------------------------------------------------:| +| deepcopy_memo | 40.3 us | 30.1 us: 1.34x faster | +| deepcopy | 352 us | 266 us: 1.32x faster | +| pathlib | 21.5 ms | 18.4 ms: 1.17x faster | +| deepcopy_reduce | 3.08 us | 2.64 us: 1.16x faster | +| comprehensions | 19.8 us | 17.1 us: 1.16x faster | +| go | 139 ms | 120 ms: 1.16x faster | +| crypto_pyaes | 76.6 ms | 67.2 ms: 1.14x faster | +| raytrace | 299 ms | 264 ms: 1.13x faster | +| logging_simple | 6.63 us | 6.01 us: 1.10x faster | +| generators | 32.2 ms | 29.3 ms: 1.10x faster | +| regex_compile | 142 ms | 131 ms: 1.08x faster | +| logging_format | 7.35 us | 6.80 us: 1.08x faster | +| gc_traversal | 3.46 ms | 3.20 ms: 1.08x faster | +| bpe_tokeniser | 4.74 sec | 4.39 sec: 1.08x faster | +| chaos | 62.8 ms | 58.5 ms: 1.07x faster | +| sympy_str | 292 ms | 272 ms: 1.07x faster | +| sympy_sum | 166 ms | 155 ms: 1.07x faster | +| deltablue | 3.45 ms | 3.24 ms: 1.06x faster | +| unpickle | 14.1 us | 13.4 us: 1.05x faster | +| thrift | 791 us | 757 us: 1.05x faster | +| scimark_monte_carlo | 68.4 ms | 65.5 ms: 1.04x faster | +| tornado_http | 119 ms | 114 ms: 1.04x faster | +| pickle_dict | 31.8 us | 30.5 us: 1.04x faster | +| logging_silent | 109 ns | 105 ns: 1.04x faster | +| hexiom | 6.17 ms | 5.92 ms: 1.04x faster | +| dulwich_log | 78.9 ms | 75.7 ms: 1.04x faster | +| sqlglot_transpile | 1.67 ms | 1.61 ms: 1.04x faster | +| pprint_safe_repr | 743 ms | 715 ms: 1.04x faster | +| 2to3 | 264 ms | 254 ms: 1.04x faster | +| sqlglot_parse | 1.36 ms | 1.30 ms: 1.04x faster | +| unpack_sequence | 52.1 ns | 50.1 ns: 1.04x faster | +| async_generators | 384 ms | 372 ms: 1.03x faster | +| pycparser | 1.17 sec | 1.13 sec: 1.03x faster | +| scimark_lu | 114 ms | 111 ms: 1.03x faster | +| pprint_pformat | 1.52 sec | 1.47 sec: 1.03x faster | +| sympy_integrate | 20.5 ms | 19.9 ms: 1.03x faster | +| typing_runtime_protocols | 163 us | 159 us: 1.03x faster | +| meteor_contest | 104 ms | 101 ms: 1.03x faster | +| asyncio_tcp | 519 ms | 506 ms: 1.02x faster | +| json_loads | 26.5 us | 26.0 us: 1.02x faster | +| genshi_text | 22.8 ms | 22.3 ms: 1.02x faster | +| nqueens | 80.1 ms | 78.5 ms: 1.02x faster | +| xml_etree_parse | 139 ms | 136 ms: 1.02x faster | +| unpickle_pure_python | 221 us | 217 us: 1.02x faster | +| json | 5.02 ms | 4.94 ms: 1.02x faster | +| sympy_expand | 468 ms | 461 ms: 1.02x faster | +| docutils | 2.64 sec | 2.60 sec: 1.01x faster | +| coroutines | 23.9 ms | 23.7 ms: 1.01x faster | +| xml_etree_iterparse | 96.7 ms | 95.7 ms: 1.01x faster | +| float | 80.8 ms | 80.1 ms: 1.01x faster | +| xml_etree_generate | 85.2 ms | 84.6 ms: 1.01x faster | +| tomli_loads | 2.11 sec | 2.09 sec: 1.01x faster | +| sqlite_synth | 2.20 us | 2.19 us: 1.01x faster | +| unpickle_list | 4.67 us | 4.66 us: 1.00x faster | +| sqlglot_normalize | 107 ms | 107 ms: 1.01x slower | +| asyncio_websockets | 517 ms | 521 ms: 1.01x slower | +| sqlglot_optimize | 53.3 ms | 53.7 ms: 1.01x slower | +| asyncio_tcp_ssl | 1.51 sec | 1.52 sec: 1.01x slower | +| django_template | 34.7 ms | 35.2 ms: 1.01x slower | +| html5lib | 63.6 ms | 65.3 ms: 1.03x slower | +| pickle_list | 4.77 us | 4.90 us: 1.03x slower | +| regex_effbot | 3.17 ms | 3.27 ms: 1.03x slower | +| mdp | 2.42 sec | 2.50 sec: 1.03x slower | +| python_startup_no_site | 7.16 ms | 7.45 ms: 1.04x slower | +| scimark_sor | 130 ms | 136 ms: 1.05x slower | +| spectral_norm | 110 ms | 116 ms: 1.05x slower | +| scimark_sparse_mat_mult | 4.39 ms | 4.63 ms: 1.05x slower | +| nbody | 89.3 ms | 94.7 ms: 1.06x slower | +| mako | 11.0 ms | 11.7 ms: 1.07x slower | +| regex_dna | 168 ms | 180 ms: 1.07x slower | +| bench_thread_pool | 941 us | 1.01 ms: 1.07x slower | +| json_dumps | 10.4 ms | 11.5 ms: 1.11x slower | +| python_startup | 9.93 ms | 11.1 ms: 1.12x slower | +| regex_v8 | 20.6 ms | 23.1 ms: 1.12x slower | +| telco | 6.53 ms | 7.34 ms: 1.13x slower | +| coverage | 71.4 ms | 81.3 ms: 1.14x slower | +| pidigits | 184 ms | 218 ms: 1.18x slower | +| create_gc_cycles | 1.09 ms | 1.38 ms: 1.26x slower | +| bench_mp_pool | 10.8 ms | 63.7 ms: 5.90x slower | +| Geometric mean | (ref) | 1.00x slower | + +Benchmark hidden because not significant (10): pickle, scimark_fft, pyflate, pickle_pure_python, richards, genshi_xml, richards_super, fannkuch, pylint, xml_etree_process +Ignored benchmarks (16) of results/bm-20240906-3.12.6-a4a2d2b/bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b.json: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn, mypy2, sqlalchemy_declarative, sqlalchemy_imperative + +# HPT report + +- Reliability score: 99.33% likely to be faster +- 90% likely to have a speedup of 1.00x +- 95% likely to have a speedup of 1.00x +- 99% likely to have a speedup of 1.00x + +# Memory +- memory change: 1.02x \ No newline at end of file diff --git a/results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.12.6.svg b/results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.12.6.svg new file mode 100644 index 00000000..5e273972 --- /dev/null +++ b/results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.12.6.svg @@ -0,0 +1,3143 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.13.0rc2.md b/results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.13.0rc2.md new file mode 100644 index 00000000..ba528137 --- /dev/null +++ b/results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.13.0rc2.md @@ -0,0 +1,192 @@ +# Results vs. 3.13.0rc2 + +- fork: python +- ref: ac3a2c8abceeec448486 +- machine: linux-x86_64 +- commit hash: ac3a2c8 +- commit date: 2024-10-31 +- overall geometric mean: 1.02x slower +- HPT reliability: 87.78% +- HPT 99th percentile: 1.00x slower +- Memory change: 1.00x + +Benchmarks with tag 'apps': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| 2to3 | 260 ms | 254 ms: 1.02x faster | +| docutils | 2.62 sec | 2.60 sec: 1.01x faster | +| html5lib | 67.0 ms | 65.3 ms: 1.03x faster | +| Geometric mean | (ref) | 1.01x faster | + +Benchmark hidden because not significant (1): tornado_http + +Benchmarks with tag 'asyncio': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|------------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| async_generators | 377 ms | 372 ms: 1.02x faster | +| asyncio_tcp_ssl | 1.51 sec | 1.52 sec: 1.01x slower | +| Geometric mean | (ref) | 1.00x faster | + +Benchmark hidden because not significant (3): asyncio_websockets, asyncio_tcp, coroutines + +Benchmarks with tag 'math': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| pidigits | 217 ms | 218 ms: 1.01x slower | +| float | 77.5 ms | 80.1 ms: 1.03x slower | +| nbody | 85.1 ms | 94.7 ms: 1.11x slower | +| Geometric mean | (ref) | 1.05x slower | + +Benchmarks with tag 'regex': +============================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| regex_v8 | 22.7 ms | 23.1 ms: 1.02x slower | +| regex_effbot | 3.08 ms | 3.27 ms: 1.06x slower | +| Geometric mean | (ref) | 1.02x slower | + +Benchmark hidden because not significant (2): regex_compile, regex_dna + +Benchmarks with tag 'serialize': +================================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|----------------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| pickle_dict | 32.5 us | 30.5 us: 1.07x faster | +| unpickle | 14.3 us | 13.4 us: 1.07x faster | +| json_loads | 27.0 us | 26.0 us: 1.04x faster | +| pickle | 11.3 us | 10.9 us: 1.04x faster | +| unpickle_list | 4.71 us | 4.66 us: 1.01x faster | +| xml_etree_generate | 85.4 ms | 84.6 ms: 1.01x faster | +| xml_etree_process | 59.3 ms | 59.0 ms: 1.01x faster | +| pickle_list | 4.93 us | 4.90 us: 1.00x faster | +| xml_etree_iterparse | 94.9 ms | 95.7 ms: 1.01x slower | +| unpickle_pure_python | 210 us | 217 us: 1.03x slower | +| tomli_loads | 2.01 sec | 2.09 sec: 1.04x slower | +| pickle_pure_python | 294 us | 307 us: 1.04x slower | +| json_dumps | 10.5 ms | 11.5 ms: 1.09x slower | +| Geometric mean | (ref) | 1.00x faster | + +Benchmark hidden because not significant (1): xml_etree_parse + +Benchmarks with tag 'startup': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|------------------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| python_startup_no_site | 7.39 ms | 7.45 ms: 1.01x slower | +| python_startup | 11.0 ms | 11.1 ms: 1.01x slower | +| Geometric mean | (ref) | 1.01x slower | + +Benchmarks with tag 'template': +=============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|-----------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| genshi_xml | 48.8 ms | 50.2 ms: 1.03x slower | +| django_template | 34.1 ms | 35.2 ms: 1.03x slower | +| mako | 11.3 ms | 11.7 ms: 1.03x slower | +| genshi_text | 21.5 ms | 22.3 ms: 1.04x slower | +| Geometric mean | (ref) | 1.03x slower | + +All benchmarks: +=============== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8 | +|--------------------------|:------------------------------------------------------------:|:----------------------------------------------------------------------:| +| deepcopy | 355 us | 266 us: 1.34x faster | +| deepcopy_memo | 39.1 us | 30.1 us: 1.30x faster | +| deepcopy_reduce | 3.11 us | 2.64 us: 1.18x faster | +| go | 141 ms | 120 ms: 1.17x faster | +| pickle_dict | 32.5 us | 30.5 us: 1.07x faster | +| unpickle | 14.3 us | 13.4 us: 1.07x faster | +| telco | 7.82 ms | 7.34 ms: 1.07x faster | +| pathlib | 19.2 ms | 18.4 ms: 1.04x faster | +| json_loads | 27.0 us | 26.0 us: 1.04x faster | +| pickle | 11.3 us | 10.9 us: 1.04x faster | +| pprint_safe_repr | 738 ms | 715 ms: 1.03x faster | +| thrift | 778 us | 757 us: 1.03x faster | +| html5lib | 67.0 ms | 65.3 ms: 1.03x faster | +| scimark_fft | 349 ms | 341 ms: 1.02x faster | +| logging_simple | 6.16 us | 6.01 us: 1.02x faster | +| 2to3 | 260 ms | 254 ms: 1.02x faster | +| coverage | 83.0 ms | 81.3 ms: 1.02x faster | +| scimark_sparse_mat_mult | 4.71 ms | 4.63 ms: 1.02x faster | +| scimark_lu | 113 ms | 111 ms: 1.02x faster | +| async_generators | 377 ms | 372 ms: 1.02x faster | +| pprint_pformat | 1.50 sec | 1.47 sec: 1.01x faster | +| bpe_tokeniser | 4.45 sec | 4.39 sec: 1.01x faster | +| unpickle_list | 4.71 us | 4.66 us: 1.01x faster | +| hexiom | 5.99 ms | 5.92 ms: 1.01x faster | +| sympy_str | 275 ms | 272 ms: 1.01x faster | +| xml_etree_generate | 85.4 ms | 84.6 ms: 1.01x faster | +| crypto_pyaes | 67.9 ms | 67.2 ms: 1.01x faster | +| sqlite_synth | 2.21 us | 2.19 us: 1.01x faster | +| meteor_contest | 102 ms | 101 ms: 1.01x faster | +| docutils | 2.62 sec | 2.60 sec: 1.01x faster | +| xml_etree_process | 59.3 ms | 59.0 ms: 1.01x faster | +| pickle_list | 4.93 us | 4.90 us: 1.00x faster | +| richards_super | 51.6 ms | 51.9 ms: 1.00x slower | +| pidigits | 217 ms | 218 ms: 1.01x slower | +| sympy_integrate | 19.8 ms | 19.9 ms: 1.01x slower | +| asyncio_tcp_ssl | 1.51 sec | 1.52 sec: 1.01x slower | +| fannkuch | 370 ms | 373 ms: 1.01x slower | +| sympy_expand | 457 ms | 461 ms: 1.01x slower | +| python_startup_no_site | 7.39 ms | 7.45 ms: 1.01x slower | +| python_startup | 11.0 ms | 11.1 ms: 1.01x slower | +| xml_etree_iterparse | 94.9 ms | 95.7 ms: 1.01x slower | +| scimark_sor | 134 ms | 136 ms: 1.01x slower | +| dulwich_log | 74.8 ms | 75.7 ms: 1.01x slower | +| sqlglot_normalize | 106 ms | 107 ms: 1.01x slower | +| pycparser | 1.12 sec | 1.13 sec: 1.01x slower | +| richards | 45.2 ms | 45.9 ms: 1.02x slower | +| regex_v8 | 22.7 ms | 23.1 ms: 1.02x slower | +| generators | 28.8 ms | 29.3 ms: 1.02x slower | +| sqlglot_optimize | 52.7 ms | 53.7 ms: 1.02x slower | +| logging_silent | 103 ns | 105 ns: 1.02x slower | +| gc_traversal | 3.14 ms | 3.20 ms: 1.02x slower | +| chaos | 57.3 ms | 58.5 ms: 1.02x slower | +| typing_runtime_protocols | 155 us | 159 us: 1.03x slower | +| genshi_xml | 48.8 ms | 50.2 ms: 1.03x slower | +| sqlglot_transpile | 1.56 ms | 1.61 ms: 1.03x slower | +| create_gc_cycles | 1.34 ms | 1.38 ms: 1.03x slower | +| django_template | 34.1 ms | 35.2 ms: 1.03x slower | +| unpickle_pure_python | 210 us | 217 us: 1.03x slower | +| float | 77.5 ms | 80.1 ms: 1.03x slower | +| mako | 11.3 ms | 11.7 ms: 1.03x slower | +| deltablue | 3.12 ms | 3.24 ms: 1.04x slower | +| comprehensions | 16.5 us | 17.1 us: 1.04x slower | +| genshi_text | 21.5 ms | 22.3 ms: 1.04x slower | +| spectral_norm | 111 ms | 116 ms: 1.04x slower | +| tomli_loads | 2.01 sec | 2.09 sec: 1.04x slower | +| pickle_pure_python | 294 us | 307 us: 1.04x slower | +| raytrace | 253 ms | 264 ms: 1.04x slower | +| sqlglot_parse | 1.25 ms | 1.30 ms: 1.04x slower | +| regex_effbot | 3.08 ms | 3.27 ms: 1.06x slower | +| mdp | 2.36 sec | 2.50 sec: 1.06x slower | +| json_dumps | 10.5 ms | 11.5 ms: 1.09x slower | +| bench_thread_pool | 919 us | 1.01 ms: 1.10x slower | +| nbody | 85.1 ms | 94.7 ms: 1.11x slower | +| unpack_sequence | 44.8 ns | 50.1 ns: 1.12x slower | +| bench_mp_pool | 11.0 ms | 63.7 ms: 5.79x slower | +| Geometric mean | (ref) | 1.02x slower | + +Benchmark hidden because not significant (14): regex_compile, logging_format, pyflate, nqueens, regex_dna, sympy_sum, xml_etree_parse, asyncio_websockets, tornado_http, asyncio_tcp, scimark_monte_carlo, json, coroutines, pylint +Ignored benchmarks (13) of results/bm-20240906-3.13.0rc2-ec61006/bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006.json: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn + +# HPT report + +- Reliability score: 87.78% likely to be slow +- 90% likely to have a slowdown of 1.00x +- 95% likely to have a slowdown of 1.00x +- 99% likely to have a slowdown of 1.00x + +# Memory +- memory change: 1.00x \ No newline at end of file diff --git a/results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.13.0rc2.svg b/results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.13.0rc2.svg new file mode 100644 index 00000000..55060b8d --- /dev/null +++ b/results/bm-20241031-3.14.0a1+-ac3a2c8/bm-20241031-vultr-x86_64-python-ac3a2c8abceeec448486-3.14.0a1+-ac3a2c8-vs-3.13.0rc2.svg @@ -0,0 +1,3176 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +