diff --git a/README.md b/README.md index d1c08631..744f1d8e 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,7 @@ Here are some recent and important revisions. πŸ‘‰ [Complete list of results](RE ## linux x86_64 (vultr) | date | fork/ref | hash/flags | vs. 3.12.6: | vs. 3.13.0rc2: | vs. base: | | --- | --- | --- | ---: | ---: | ---: | +| [2024-11-20](results/bm-20241120-3.14.0a2%2B-8ebd73d) | mpage/gh_115999_tlbc_call | 8ebd73d | 1.00x ↓
[πŸ“„](results/bm-20241120-3.14.0a2%2B-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-3.12.6.svg) | 1.02x ↓
[πŸ“„](results/bm-20241120-3.14.0a2%2B-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-3.13.0rc2.svg) | 1.00x ↑
[πŸ“„](results/bm-20241120-3.14.0a2%2B-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-base.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-base.svg)[🧠](results/bm-20241120-3.14.0a2%2B-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-base-mem.svg) | | [2024-11-20](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL) | python/32428cf9ea03bce6d64c | 32428cf (NOGIL) | 1.54x ↓
[πŸ“„](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.12.6.svg) | 1.56x ↓
[πŸ“„](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.13.0rc2.svg) | 1.53x ↓
[πŸ“„](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-base.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-base.svg)[🧠](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-base-mem.svg) | | [2024-11-20](results/bm-20241120-3.14.0a2%2B-32428cf) | python/32428cf9ea03bce6d64c | 32428cf | 1.00x ↓
[πŸ“„](results/bm-20241120-3.14.0a2%2B-32428cf/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-32428cf/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.12.6.svg) | 1.02x ↓
[πŸ“„](results/bm-20241120-3.14.0a2%2B-32428cf/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-32428cf/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.13.0rc2.svg) | | | [2024-11-19](results/bm-20241119-3.14.0a2%2B-c9b399f) | python/c9b399fbdb01584dcfff | c9b399f | 1.01x ↓
[πŸ“„](results/bm-20241119-3.14.0a2%2B-c9b399f/bm-20241119-vultr-x86_64-python-c9b399fbdb01584dcfff-3.14.0a2%2B-c9b399f-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241119-3.14.0a2%2B-c9b399f/bm-20241119-vultr-x86_64-python-c9b399fbdb01584dcfff-3.14.0a2%2B-c9b399f-vs-3.12.6.svg) | 1.02x ↓
[πŸ“„](results/bm-20241119-3.14.0a2%2B-c9b399f/bm-20241119-vultr-x86_64-python-c9b399fbdb01584dcfff-3.14.0a2%2B-c9b399f-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241119-3.14.0a2%2B-c9b399f/bm-20241119-vultr-x86_64-python-c9b399fbdb01584dcfff-3.14.0a2%2B-c9b399f-vs-3.13.0rc2.svg) | | diff --git a/RESULTS.md b/RESULTS.md index bda6c8e5..97e68cc8 100644 --- a/RESULTS.md +++ b/RESULTS.md @@ -144,6 +144,7 @@ ## linux x86_64 (vultr) | date | fork/ref | hash/flags | vs. 3.12.6: | vs. 3.13.0rc2: | vs. base: | | --- | --- | --- | ---: | ---: | ---: | +| [2024-11-20](results/bm-20241120-3.14.0a2%2B-8ebd73d) | mpage/gh_115999_tlbc_call | 8ebd73d | 1.00x ↓
[πŸ“„](results/bm-20241120-3.14.0a2%2B-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-3.12.6.svg) | 1.02x ↓
[πŸ“„](results/bm-20241120-3.14.0a2%2B-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-3.13.0rc2.svg) | 1.00x ↑
[πŸ“„](results/bm-20241120-3.14.0a2%2B-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-base.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-base.svg)[🧠](results/bm-20241120-3.14.0a2%2B-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-base-mem.svg) | | [2024-11-20](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL) | python/32428cf9ea03bce6d64c | 32428cf (NOGIL) | 1.54x ↓
[πŸ“„](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.12.6.svg) | 1.56x ↓
[πŸ“„](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.13.0rc2.svg) | 1.53x ↓
[πŸ“„](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-base.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-base.svg)[🧠](results/bm-20241120-3.14.0a2%2B-32428cf-NOGIL/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-base-mem.svg) | | [2024-11-20](results/bm-20241120-3.14.0a2%2B-32428cf) | python/32428cf9ea03bce6d64c | 32428cf | 1.00x ↓
[πŸ“„](results/bm-20241120-3.14.0a2%2B-32428cf/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-32428cf/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.12.6.svg) | 1.02x ↓
[πŸ“„](results/bm-20241120-3.14.0a2%2B-32428cf/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241120-3.14.0a2%2B-32428cf/bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2%2B-32428cf-vs-3.13.0rc2.svg) | | | [2024-11-19](results/bm-20241119-3.14.0a2%2B-c9b399f) | python/c9b399fbdb01584dcfff | c9b399f | 1.01x ↓
[πŸ“„](results/bm-20241119-3.14.0a2%2B-c9b399f/bm-20241119-vultr-x86_64-python-c9b399fbdb01584dcfff-3.14.0a2%2B-c9b399f-vs-3.12.6.md)[πŸ“ˆ](results/bm-20241119-3.14.0a2%2B-c9b399f/bm-20241119-vultr-x86_64-python-c9b399fbdb01584dcfff-3.14.0a2%2B-c9b399f-vs-3.12.6.svg) | 1.02x ↓
[πŸ“„](results/bm-20241119-3.14.0a2%2B-c9b399f/bm-20241119-vultr-x86_64-python-c9b399fbdb01584dcfff-3.14.0a2%2B-c9b399f-vs-3.13.0rc2.md)[πŸ“ˆ](results/bm-20241119-3.14.0a2%2B-c9b399f/bm-20241119-vultr-x86_64-python-c9b399fbdb01584dcfff-3.14.0a2%2B-c9b399f-vs-3.13.0rc2.svg) | | diff --git a/results/bm-20241120-3.14.0a2+-8ebd73d/README.md b/results/bm-20241120-3.14.0a2+-8ebd73d/README.md new file mode 100644 index 00000000..cb7bf1d0 --- /dev/null +++ b/results/bm-20241120-3.14.0a2+-8ebd73d/README.md @@ -0,0 +1,41 @@ +# Results + +- fork: mpage +- version: 3.14.0a2+ +- config: +- commit hash: [8ebd73d](https://github.com/mpage/cpython/commit/8ebd73d) +- commit date: 2024-11-20T22:27:45-08:00 +- commit merge base: [32428cf9ea03bce6d64c7acd28e0b7d92774eb53](https://github.com/mpage/cpython/commit/32428cf9ea03bce6d64c7acd28e0b7d92774eb53) +- ref: gh_115999_tlbc_call + +## linux x86_64 (vultr) + +- [GitHub Action run](https://github.com/facebookexperimental/free-threading-benchmarking/actions/runs/11959013491) +- cpu model: Intel(R) Xeon(R) E-2286G CPU @ 4.00GHz +- platform: Linux-6.8.0-45-generic-x86_64-with-glibc2.39 +- [raw results](bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d.json) + +### vs. 3.12.6 + +- Geometric mean: 1.00x slower (HPT: reliability of 90.85%, 1.00x faster at 99th %ile) +- Memory usage: 1.02x +- missing benchmarks: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn, mypy2, sqlalchemy_declarative, sqlalchemy_imperative, tornado_http +- [πŸ“„table](bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-3.12.6.md) +- [πŸ“ˆtime plot](bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-3.12.6.svg) + +### vs. 3.13.0rc2 + +- Geometric mean: 1.02x slower (HPT: reliability of 90.85%, 1.00x slower at 99th %ile) +- Memory usage: 1.01x +- missing benchmarks: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn, tornado_http +- [πŸ“„table](bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-3.13.0rc2.md) +- [πŸ“ˆtime plot](bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-3.13.0rc2.svg) + +### vs. base + +- Geometric mean: 1.00x faster (HPT: reliability of 59.05%, 1.00x faster at 99th %ile) +- Memory usage: 1.00x +- [🧠memory plot](bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-base-mem.svg) +- [πŸ“„table](bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-base.md) +- [πŸ“ˆtime plot](bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2%2B-8ebd73d-vs-base.svg) + diff --git a/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-3.12.6.md b/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-3.12.6.md new file mode 100644 index 00000000..21f2ee9a --- /dev/null +++ b/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-3.12.6.md @@ -0,0 +1,194 @@ +# Results vs. 3.12.6 + +- fork: mpage +- ref: gh_115999_tlbc_call +- machine: linux-x86_64 +- commit hash: 8ebd73d +- commit date: 2024-11-20 +- overall geometric mean: 1.00x slower +- HPT reliability: 90.85% +- HPT 99th percentile: 1.00x faster +- Memory change: 1.02x + +Benchmarks with tag 'apps': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|----------------|:------------------------------------------------------:|:--------------------------------------------------------------------:| +| 2to3 | 264 ms | 255 ms: 1.03x faster | +| docutils | 2.64 sec | 2.59 sec: 1.02x faster | +| html5lib | 63.6 ms | 64.9 ms: 1.02x slower | +| Geometric mean | (ref) | 1.01x faster | + +Benchmarks with tag 'asyncio': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|--------------------|:------------------------------------------------------:|:--------------------------------------------------------------------:| +| coroutines | 23.9 ms | 23.1 ms: 1.03x faster | +| asyncio_tcp | 519 ms | 504 ms: 1.03x faster | +| asyncio_websockets | 517 ms | 521 ms: 1.01x slower | +| asyncio_tcp_ssl | 1.51 sec | 1.53 sec: 1.01x slower | +| Geometric mean | (ref) | 1.01x faster | + +Benchmark hidden because not significant (1): async_generators + +Benchmarks with tag 'math': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|----------------|:------------------------------------------------------:|:--------------------------------------------------------------------:| +| float | 80.8 ms | 79.8 ms: 1.01x faster | +| pidigits | 184 ms | 186 ms: 1.01x slower | +| nbody | 89.3 ms | 95.2 ms: 1.07x slower | +| Geometric mean | (ref) | 1.02x slower | + +Benchmarks with tag 'regex': +============================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|----------------|:------------------------------------------------------:|:--------------------------------------------------------------------:| +| regex_effbot | 3.17 ms | 2.70 ms: 1.17x faster | +| regex_compile | 142 ms | 130 ms: 1.09x faster | +| regex_dna | 168 ms | 177 ms: 1.06x slower | +| regex_v8 | 20.6 ms | 22.7 ms: 1.10x slower | +| Geometric mean | (ref) | 1.02x faster | + +Benchmarks with tag 'serialize': +================================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|----------------------|:------------------------------------------------------:|:--------------------------------------------------------------------:| +| json_loads | 26.5 us | 25.0 us: 1.06x faster | +| pickle_dict | 31.8 us | 31.0 us: 1.02x faster | +| unpickle_pure_python | 221 us | 216 us: 1.02x faster | +| xml_etree_parse | 139 ms | 137 ms: 1.02x faster | +| xml_etree_generate | 85.2 ms | 84.8 ms: 1.01x faster | +| xml_etree_process | 59.0 ms | 59.4 ms: 1.01x slower | +| tomli_loads | 2.11 sec | 2.13 sec: 1.01x slower | +| pickle_pure_python | 308 us | 314 us: 1.02x slower | +| pickle_list | 4.77 us | 5.02 us: 1.05x slower | +| json_dumps | 10.4 ms | 11.4 ms: 1.10x slower | +| pickle | 10.9 us | 12.4 us: 1.13x slower | +| Geometric mean | (ref) | 1.01x slower | + +Benchmark hidden because not significant (3): unpickle, xml_etree_iterparse, unpickle_list + +Benchmarks with tag 'startup': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|------------------------|:------------------------------------------------------:|:--------------------------------------------------------------------:| +| python_startup_no_site | 7.16 ms | 7.42 ms: 1.04x slower | +| python_startup | 9.93 ms | 11.1 ms: 1.12x slower | +| Geometric mean | (ref) | 1.08x slower | + +Benchmarks with tag 'template': +=============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|-----------------|:------------------------------------------------------:|:--------------------------------------------------------------------:| +| genshi_text | 22.8 ms | 22.4 ms: 1.02x faster | +| genshi_xml | 50.2 ms | 50.6 ms: 1.01x slower | +| django_template | 34.7 ms | 35.3 ms: 1.02x slower | +| mako | 11.0 ms | 11.8 ms: 1.07x slower | +| Geometric mean | (ref) | 1.02x slower | + +All benchmarks: +=============== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|--------------------------|:------------------------------------------------------:|:--------------------------------------------------------------------:| +| deepcopy | 352 us | 264 us: 1.33x faster | +| deepcopy_memo | 40.3 us | 30.8 us: 1.31x faster | +| pylint | 319 ms | 270 ms: 1.18x faster | +| pathlib | 21.5 ms | 18.3 ms: 1.17x faster | +| regex_effbot | 3.17 ms | 2.70 ms: 1.17x faster | +| go | 139 ms | 121 ms: 1.16x faster | +| deepcopy_reduce | 3.08 us | 2.67 us: 1.15x faster | +| comprehensions | 19.8 us | 17.2 us: 1.15x faster | +| raytrace | 299 ms | 264 ms: 1.13x faster | +| crypto_pyaes | 76.6 ms | 68.0 ms: 1.13x faster | +| unpack_sequence | 52.1 ns | 46.8 ns: 1.11x faster | +| regex_compile | 142 ms | 130 ms: 1.09x faster | +| generators | 32.2 ms | 29.6 ms: 1.09x faster | +| sympy_sum | 166 ms | 153 ms: 1.09x faster | +| logging_format | 7.35 us | 6.79 us: 1.08x faster | +| json | 5.02 ms | 4.66 ms: 1.08x faster | +| bpe_tokeniser | 4.74 sec | 4.40 sec: 1.08x faster | +| chaos | 62.8 ms | 58.3 ms: 1.08x faster | +| logging_simple | 6.63 us | 6.16 us: 1.08x faster | +| sympy_str | 292 ms | 273 ms: 1.07x faster | +| json_loads | 26.5 us | 25.0 us: 1.06x faster | +| thrift | 791 us | 747 us: 1.06x faster | +| scimark_monte_carlo | 68.4 ms | 65.4 ms: 1.05x faster | +| sqlglot_parse | 1.36 ms | 1.30 ms: 1.04x faster | +| sqlglot_transpile | 1.67 ms | 1.60 ms: 1.04x faster | +| dulwich_log | 78.9 ms | 75.8 ms: 1.04x faster | +| coroutines | 23.9 ms | 23.1 ms: 1.03x faster | +| 2to3 | 264 ms | 255 ms: 1.03x faster | +| typing_runtime_protocols | 163 us | 158 us: 1.03x faster | +| sympy_integrate | 20.5 ms | 19.9 ms: 1.03x faster | +| asyncio_tcp | 519 ms | 504 ms: 1.03x faster | +| pycparser | 1.17 sec | 1.14 sec: 1.03x faster | +| meteor_contest | 104 ms | 101 ms: 1.03x faster | +| pprint_safe_repr | 743 ms | 723 ms: 1.03x faster | +| hexiom | 6.17 ms | 6.02 ms: 1.03x faster | +| pickle_dict | 31.8 us | 31.0 us: 1.02x faster | +| genshi_text | 22.8 ms | 22.4 ms: 1.02x faster | +| gc_traversal | 3.46 ms | 3.39 ms: 1.02x faster | +| unpickle_pure_python | 221 us | 216 us: 1.02x faster | +| pprint_pformat | 1.52 sec | 1.49 sec: 1.02x faster | +| scimark_lu | 114 ms | 112 ms: 1.02x faster | +| docutils | 2.64 sec | 2.59 sec: 1.02x faster | +| nqueens | 80.1 ms | 78.8 ms: 1.02x faster | +| xml_etree_parse | 139 ms | 137 ms: 1.02x faster | +| sympy_expand | 468 ms | 461 ms: 1.01x faster | +| float | 80.8 ms | 79.8 ms: 1.01x faster | +| scimark_fft | 342 ms | 337 ms: 1.01x faster | +| xml_etree_generate | 85.2 ms | 84.8 ms: 1.01x faster | +| sqlglot_normalize | 107 ms | 107 ms: 1.01x slower | +| xml_etree_process | 59.0 ms | 59.4 ms: 1.01x slower | +| asyncio_websockets | 517 ms | 521 ms: 1.01x slower | +| genshi_xml | 50.2 ms | 50.6 ms: 1.01x slower | +| pidigits | 184 ms | 186 ms: 1.01x slower | +| fannkuch | 372 ms | 376 ms: 1.01x slower | +| pyflate | 448 ms | 452 ms: 1.01x slower | +| sqlglot_optimize | 53.3 ms | 53.9 ms: 1.01x slower | +| asyncio_tcp_ssl | 1.51 sec | 1.53 sec: 1.01x slower | +| tomli_loads | 2.11 sec | 2.13 sec: 1.01x slower | +| django_template | 34.7 ms | 35.3 ms: 1.02x slower | +| pickle_pure_python | 308 us | 314 us: 1.02x slower | +| html5lib | 63.6 ms | 64.9 ms: 1.02x slower | +| mdp | 2.42 sec | 2.51 sec: 1.04x slower | +| python_startup_no_site | 7.16 ms | 7.42 ms: 1.04x slower | +| spectral_norm | 110 ms | 115 ms: 1.05x slower | +| scimark_sparse_mat_mult | 4.39 ms | 4.62 ms: 1.05x slower | +| pickle_list | 4.77 us | 5.02 us: 1.05x slower | +| scimark_sor | 130 ms | 137 ms: 1.06x slower | +| regex_dna | 168 ms | 177 ms: 1.06x slower | +| nbody | 89.3 ms | 95.2 ms: 1.07x slower | +| mako | 11.0 ms | 11.8 ms: 1.07x slower | +| bench_thread_pool | 941 us | 1.02 ms: 1.08x slower | +| json_dumps | 10.4 ms | 11.4 ms: 1.10x slower | +| regex_v8 | 20.6 ms | 22.7 ms: 1.10x slower | +| python_startup | 9.93 ms | 11.1 ms: 1.12x slower | +| telco | 6.53 ms | 7.29 ms: 1.12x slower | +| pickle | 10.9 us | 12.4 us: 1.13x slower | +| coverage | 71.4 ms | 81.7 ms: 1.14x slower | +| create_gc_cycles | 1.09 ms | 1.36 ms: 1.24x slower | +| bench_mp_pool | 10.8 ms | 63.5 ms: 5.88x slower | +| Geometric mean | (ref) | 1.00x slower | + +Benchmark hidden because not significant (9): unpickle, xml_etree_iterparse, richards, deltablue, unpickle_list, async_generators, richards_super, sqlite_synth, logging_silent +Ignored benchmarks (17) of results/bm-20240906-3.12.6-a4a2d2b/bm-20240906-vultr-x86_64-python-v3.12.6-3.12.6-a4a2d2b.json: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn, mypy2, sqlalchemy_declarative, sqlalchemy_imperative, tornado_http + +# HPT report + +- Reliability score: 90.85% likely to be faster +- 90% likely to have a speedup of 1.00x +- 95% likely to have a speedup of 1.00x +- 99% likely to have a speedup of 1.00x + +# Memory +- memory change: 1.02x \ No newline at end of file diff --git a/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-3.12.6.svg b/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-3.12.6.svg new file mode 100644 index 00000000..3a3dab9a --- /dev/null +++ b/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-3.12.6.svg @@ -0,0 +1,3109 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-3.13.0rc2.md b/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-3.13.0rc2.md new file mode 100644 index 00000000..689ed123 --- /dev/null +++ b/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-3.13.0rc2.md @@ -0,0 +1,193 @@ +# Results vs. 3.13.0rc2 + +- fork: mpage +- ref: gh_115999_tlbc_call +- machine: linux-x86_64 +- commit hash: 8ebd73d +- commit date: 2024-11-20 +- overall geometric mean: 1.02x slower +- HPT reliability: 90.85% +- HPT 99th percentile: 1.00x slower +- Memory change: 1.01x + +Benchmarks with tag 'apps': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|----------------|:------------------------------------------------------------:|:--------------------------------------------------------------------:| +| 2to3 | 260 ms | 255 ms: 1.02x faster | +| docutils | 2.62 sec | 2.59 sec: 1.01x faster | +| html5lib | 67.0 ms | 64.9 ms: 1.03x faster | +| Geometric mean | (ref) | 1.02x faster | + +Benchmarks with tag 'asyncio': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|------------------|:------------------------------------------------------------:|:--------------------------------------------------------------------:| +| coroutines | 23.6 ms | 23.1 ms: 1.02x faster | +| asyncio_tcp_ssl | 1.51 sec | 1.53 sec: 1.01x slower | +| async_generators | 377 ms | 385 ms: 1.02x slower | +| Geometric mean | (ref) | 1.00x slower | + +Benchmark hidden because not significant (2): asyncio_tcp, asyncio_websockets + +Benchmarks with tag 'math': +=========================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|----------------|:------------------------------------------------------------:|:--------------------------------------------------------------------:| +| pidigits | 217 ms | 186 ms: 1.17x faster | +| float | 77.5 ms | 79.8 ms: 1.03x slower | +| nbody | 85.1 ms | 95.2 ms: 1.12x slower | +| Geometric mean | (ref) | 1.00x faster | + +Benchmarks with tag 'regex': +============================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|----------------|:------------------------------------------------------------:|:--------------------------------------------------------------------:| +| regex_effbot | 3.08 ms | 2.70 ms: 1.14x faster | +| regex_dna | 180 ms | 177 ms: 1.02x faster | +| regex_compile | 132 ms | 130 ms: 1.01x faster | +| Geometric mean | (ref) | 1.04x faster | + +Benchmark hidden because not significant (1): regex_v8 + +Benchmarks with tag 'serialize': +================================ + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|----------------------|:------------------------------------------------------------:|:--------------------------------------------------------------------:| +| json_loads | 27.0 us | 25.0 us: 1.08x faster | +| pickle_dict | 32.5 us | 31.0 us: 1.05x faster | +| unpickle | 14.3 us | 13.9 us: 1.03x faster | +| xml_etree_generate | 85.4 ms | 84.8 ms: 1.01x faster | +| unpickle_list | 4.71 us | 4.68 us: 1.01x faster | +| xml_etree_iterparse | 94.9 ms | 96.4 ms: 1.02x slower | +| pickle_list | 4.93 us | 5.02 us: 1.02x slower | +| unpickle_pure_python | 210 us | 216 us: 1.03x slower | +| tomli_loads | 2.01 sec | 2.13 sec: 1.06x slower | +| pickle_pure_python | 294 us | 314 us: 1.07x slower | +| json_dumps | 10.5 ms | 11.4 ms: 1.08x slower | +| pickle | 11.3 us | 12.4 us: 1.09x slower | +| Geometric mean | (ref) | 1.01x slower | + +Benchmark hidden because not significant (2): xml_etree_process, xml_etree_parse + +Benchmarks with tag 'startup': +============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|------------------------|:------------------------------------------------------------:|:--------------------------------------------------------------------:| +| python_startup_no_site | 7.39 ms | 7.42 ms: 1.00x slower | +| python_startup | 11.0 ms | 11.1 ms: 1.01x slower | +| Geometric mean | (ref) | 1.01x slower | + +Benchmarks with tag 'template': +=============================== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|-----------------|:------------------------------------------------------------:|:--------------------------------------------------------------------:| +| django_template | 34.1 ms | 35.3 ms: 1.03x slower | +| genshi_xml | 48.8 ms | 50.6 ms: 1.04x slower | +| genshi_text | 21.5 ms | 22.4 ms: 1.04x slower | +| mako | 11.3 ms | 11.8 ms: 1.04x slower | +| Geometric mean | (ref) | 1.04x slower | + +All benchmarks: +=============== + +| Benchmark | bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006 | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|--------------------------|:------------------------------------------------------------:|:--------------------------------------------------------------------:| +| deepcopy | 355 us | 264 us: 1.35x faster | +| deepcopy_memo | 39.1 us | 30.8 us: 1.27x faster | +| pylint | 317 ms | 270 ms: 1.17x faster | +| go | 141 ms | 121 ms: 1.17x faster | +| deepcopy_reduce | 3.11 us | 2.67 us: 1.17x faster | +| pidigits | 217 ms | 186 ms: 1.17x faster | +| regex_effbot | 3.08 ms | 2.70 ms: 1.14x faster | +| json_loads | 27.0 us | 25.0 us: 1.08x faster | +| telco | 7.82 ms | 7.29 ms: 1.07x faster | +| json | 4.93 ms | 4.66 ms: 1.06x faster | +| pickle_dict | 32.5 us | 31.0 us: 1.05x faster | +| pathlib | 19.2 ms | 18.3 ms: 1.05x faster | +| thrift | 778 us | 747 us: 1.04x faster | +| scimark_fft | 349 ms | 337 ms: 1.04x faster | +| html5lib | 67.0 ms | 64.9 ms: 1.03x faster | +| unpickle | 14.3 us | 13.9 us: 1.03x faster | +| pprint_safe_repr | 738 ms | 723 ms: 1.02x faster | +| sympy_sum | 156 ms | 153 ms: 1.02x faster | +| scimark_sparse_mat_mult | 4.71 ms | 4.62 ms: 1.02x faster | +| 2to3 | 260 ms | 255 ms: 1.02x faster | +| coroutines | 23.6 ms | 23.1 ms: 1.02x faster | +| regex_dna | 180 ms | 177 ms: 1.02x faster | +| coverage | 83.0 ms | 81.7 ms: 1.02x faster | +| regex_compile | 132 ms | 130 ms: 1.01x faster | +| bpe_tokeniser | 4.45 sec | 4.40 sec: 1.01x faster | +| meteor_contest | 102 ms | 101 ms: 1.01x faster | +| docutils | 2.62 sec | 2.59 sec: 1.01x faster | +| logging_format | 6.84 us | 6.79 us: 1.01x faster | +| xml_etree_generate | 85.4 ms | 84.8 ms: 1.01x faster | +| unpickle_list | 4.71 us | 4.68 us: 1.01x faster | +| sympy_str | 275 ms | 273 ms: 1.01x faster | +| scimark_lu | 113 ms | 112 ms: 1.00x faster | +| python_startup_no_site | 7.39 ms | 7.42 ms: 1.00x slower | +| hexiom | 5.99 ms | 6.02 ms: 1.00x slower | +| sympy_integrate | 19.8 ms | 19.9 ms: 1.01x slower | +| richards_super | 51.6 ms | 52.0 ms: 1.01x slower | +| sympy_expand | 457 ms | 461 ms: 1.01x slower | +| python_startup | 11.0 ms | 11.1 ms: 1.01x slower | +| pyflate | 449 ms | 452 ms: 1.01x slower | +| asyncio_tcp_ssl | 1.51 sec | 1.53 sec: 1.01x slower | +| dulwich_log | 74.8 ms | 75.8 ms: 1.01x slower | +| sqlglot_normalize | 106 ms | 107 ms: 1.01x slower | +| create_gc_cycles | 1.34 ms | 1.36 ms: 1.01x slower | +| xml_etree_iterparse | 94.9 ms | 96.4 ms: 1.02x slower | +| richards | 45.2 ms | 46.0 ms: 1.02x slower | +| fannkuch | 370 ms | 376 ms: 1.02x slower | +| pycparser | 1.12 sec | 1.14 sec: 1.02x slower | +| chaos | 57.3 ms | 58.3 ms: 1.02x slower | +| scimark_sor | 134 ms | 137 ms: 1.02x slower | +| pickle_list | 4.93 us | 5.02 us: 1.02x slower | +| async_generators | 377 ms | 385 ms: 1.02x slower | +| sqlglot_optimize | 52.7 ms | 53.9 ms: 1.02x slower | +| typing_runtime_protocols | 155 us | 158 us: 1.02x slower | +| generators | 28.8 ms | 29.6 ms: 1.03x slower | +| sqlglot_transpile | 1.56 ms | 1.60 ms: 1.03x slower | +| float | 77.5 ms | 79.8 ms: 1.03x slower | +| unpickle_pure_python | 210 us | 216 us: 1.03x slower | +| django_template | 34.1 ms | 35.3 ms: 1.03x slower | +| genshi_xml | 48.8 ms | 50.6 ms: 1.04x slower | +| spectral_norm | 111 ms | 115 ms: 1.04x slower | +| genshi_text | 21.5 ms | 22.4 ms: 1.04x slower | +| mako | 11.3 ms | 11.8 ms: 1.04x slower | +| sqlglot_parse | 1.25 ms | 1.30 ms: 1.04x slower | +| raytrace | 253 ms | 264 ms: 1.04x slower | +| unpack_sequence | 44.8 ns | 46.8 ns: 1.04x slower | +| comprehensions | 16.5 us | 17.2 us: 1.05x slower | +| tomli_loads | 2.01 sec | 2.13 sec: 1.06x slower | +| mdp | 2.36 sec | 2.51 sec: 1.06x slower | +| pickle_pure_python | 294 us | 314 us: 1.07x slower | +| logging_silent | 103 ns | 110 ns: 1.07x slower | +| gc_traversal | 3.14 ms | 3.39 ms: 1.08x slower | +| json_dumps | 10.5 ms | 11.4 ms: 1.08x slower | +| pickle | 11.3 us | 12.4 us: 1.09x slower | +| deltablue | 3.12 ms | 3.45 ms: 1.10x slower | +| bench_thread_pool | 919 us | 1.02 ms: 1.11x slower | +| nbody | 85.1 ms | 95.2 ms: 1.12x slower | +| bench_mp_pool | 11.0 ms | 63.5 ms: 5.78x slower | +| Geometric mean | (ref) | 1.02x slower | + +Benchmark hidden because not significant (11): pprint_pformat, asyncio_tcp, scimark_monte_carlo, xml_etree_process, asyncio_websockets, logging_simple, regex_v8, sqlite_synth, xml_etree_parse, crypto_pyaes, nqueens +Ignored benchmarks (14) of results/bm-20240906-3.13.0rc2-ec61006/bm-20240906-vultr-x86_64-python-v3.13.0rc2-3.13.0rc2-ec61006.json: aiohttp, async_tree_cpu_io_mixed, async_tree_cpu_io_mixed_tg, async_tree_io, async_tree_io_tg, async_tree_memoization, async_tree_memoization_tg, async_tree_none, async_tree_none_tg, chameleon, dask, flaskblogging, gunicorn, tornado_http + +# HPT report + +- Reliability score: 90.85% likely to be slow +- 90% likely to have a slowdown of 1.00x +- 95% likely to have a slowdown of 1.00x +- 99% likely to have a slowdown of 1.00x + +# Memory +- memory change: 1.01x \ No newline at end of file diff --git a/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-3.13.0rc2.svg b/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-3.13.0rc2.svg new file mode 100644 index 00000000..a5a5be52 --- /dev/null +++ b/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-3.13.0rc2.svg @@ -0,0 +1,3139 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-base-mem.svg b/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-base-mem.svg new file mode 100644 index 00000000..cbb5849b --- /dev/null +++ b/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-base-mem.svg @@ -0,0 +1,3195 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-base.md b/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-base.md new file mode 100644 index 00000000..096f1cae --- /dev/null +++ b/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-base.md @@ -0,0 +1,178 @@ +# Results vs. base + +- fork: mpage +- ref: gh_115999_tlbc_call +- machine: linux-x86_64 +- commit hash: 8ebd73d +- commit date: 2024-11-20 +- overall geometric mean: 1.00x faster +- HPT reliability: 59.05% +- HPT 99th percentile: 1.00x faster +- Memory change: 1.00x + +Benchmarks with tag 'apps': +=========================== + +| Benchmark | bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2+-32428cf | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|----------------|:----------------------------------------------------------------------:|:--------------------------------------------------------------------:| +| 2to3 | 256 ms | 255 ms: 1.00x faster | +| docutils | 2.63 sec | 2.59 sec: 1.01x faster | +| html5lib | 66.9 ms | 64.9 ms: 1.03x faster | +| Geometric mean | (ref) | 1.01x faster | + +Benchmarks with tag 'asyncio': +============================== + +| Benchmark | bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2+-32428cf | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|------------------|:----------------------------------------------------------------------:|:--------------------------------------------------------------------:| +| async_generators | 378 ms | 385 ms: 1.02x slower | +| coroutines | 22.6 ms | 23.1 ms: 1.03x slower | +| Geometric mean | (ref) | 1.01x slower | + +Benchmark hidden because not significant (3): asyncio_tcp, asyncio_websockets, asyncio_tcp_ssl + +Benchmarks with tag 'math': +=========================== + +| Benchmark | bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2+-32428cf | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|----------------|:----------------------------------------------------------------------:|:--------------------------------------------------------------------:| +| pidigits | 217 ms | 186 ms: 1.17x faster | +| float | 78.8 ms | 79.8 ms: 1.01x slower | +| Geometric mean | (ref) | 1.05x faster | + +Benchmark hidden because not significant (1): nbody + +Benchmarks with tag 'regex': +============================ + +| Benchmark | bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2+-32428cf | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|----------------|:----------------------------------------------------------------------:|:--------------------------------------------------------------------:| +| regex_effbot | 2.86 ms | 2.70 ms: 1.06x faster | +| regex_v8 | 23.6 ms | 22.7 ms: 1.04x faster | +| regex_compile | 135 ms | 130 ms: 1.03x faster | +| regex_dna | 177 ms | 177 ms: 1.00x slower | +| Geometric mean | (ref) | 1.03x faster | + +Benchmarks with tag 'serialize': +================================ + +| Benchmark | bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2+-32428cf | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|--------------------|:----------------------------------------------------------------------:|:--------------------------------------------------------------------:| +| pickle_list | 5.29 us | 5.02 us: 1.05x faster | +| pickle_dict | 32.3 us | 31.0 us: 1.04x faster | +| unpickle_list | 4.81 us | 4.68 us: 1.03x faster | +| pickle_pure_python | 320 us | 314 us: 1.02x faster | +| xml_etree_generate | 86.2 ms | 84.8 ms: 1.02x faster | +| xml_etree_process | 60.2 ms | 59.4 ms: 1.01x faster | +| tomli_loads | 2.15 sec | 2.13 sec: 1.01x faster | +| json_dumps | 11.3 ms | 11.4 ms: 1.01x slower | +| pickle | 12.3 us | 12.4 us: 1.01x slower | +| unpickle | 13.5 us | 13.9 us: 1.03x slower | +| Geometric mean | (ref) | 1.01x faster | + +Benchmark hidden because not significant (4): xml_etree_parse, json_loads, unpickle_pure_python, xml_etree_iterparse + +Benchmarks with tag 'startup': +============================== + +| Benchmark | bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2+-32428cf | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|------------------------|:----------------------------------------------------------------------:|:--------------------------------------------------------------------:| +| python_startup | 11.1 ms | 11.1 ms: 1.00x slower | +| python_startup_no_site | 7.41 ms | 7.42 ms: 1.00x slower | +| Geometric mean | (ref) | 1.00x slower | + +Benchmarks with tag 'template': +=============================== + +| Benchmark | bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2+-32428cf | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|-----------------|:----------------------------------------------------------------------:|:--------------------------------------------------------------------:| +| mako | 12.0 ms | 11.8 ms: 1.01x faster | +| django_template | 35.7 ms | 35.3 ms: 1.01x faster | +| genshi_text | 21.9 ms | 22.4 ms: 1.02x slower | +| Geometric mean | (ref) | 1.00x faster | + +Benchmark hidden because not significant (1): genshi_xml + +All benchmarks: +=============== + +| Benchmark | bm-20241120-vultr-x86_64-python-32428cf9ea03bce6d64c-3.14.0a2+-32428cf | bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d | +|--------------------------|:----------------------------------------------------------------------:|:--------------------------------------------------------------------:| +| pidigits | 217 ms | 186 ms: 1.17x faster | +| gc_traversal | 3.63 ms | 3.39 ms: 1.07x faster | +| regex_effbot | 2.86 ms | 2.70 ms: 1.06x faster | +| pickle_list | 5.29 us | 5.02 us: 1.05x faster | +| pickle_dict | 32.3 us | 31.0 us: 1.04x faster | +| regex_v8 | 23.6 ms | 22.7 ms: 1.04x faster | +| regex_compile | 135 ms | 130 ms: 1.03x faster | +| logging_format | 6.99 us | 6.79 us: 1.03x faster | +| html5lib | 66.9 ms | 64.9 ms: 1.03x faster | +| unpickle_list | 4.81 us | 4.68 us: 1.03x faster | +| logging_simple | 6.30 us | 6.16 us: 1.02x faster | +| pickle_pure_python | 320 us | 314 us: 1.02x faster | +| xml_etree_generate | 86.2 ms | 84.8 ms: 1.02x faster | +| mako | 12.0 ms | 11.8 ms: 1.01x faster | +| sqlite_synth | 2.24 us | 2.21 us: 1.01x faster | +| pathlib | 18.6 ms | 18.3 ms: 1.01x faster | +| xml_etree_process | 60.2 ms | 59.4 ms: 1.01x faster | +| docutils | 2.63 sec | 2.59 sec: 1.01x faster | +| chaos | 59.0 ms | 58.3 ms: 1.01x faster | +| django_template | 35.7 ms | 35.3 ms: 1.01x faster | +| typing_runtime_protocols | 160 us | 158 us: 1.01x faster | +| go | 122 ms | 121 ms: 1.01x faster | +| deepcopy | 266 us | 264 us: 1.01x faster | +| pylint | 272 ms | 270 ms: 1.01x faster | +| thrift | 752 us | 747 us: 1.01x faster | +| sympy_str | 275 ms | 273 ms: 1.01x faster | +| scimark_lu | 113 ms | 112 ms: 1.01x faster | +| tomli_loads | 2.15 sec | 2.13 sec: 1.01x faster | +| sympy_sum | 153 ms | 153 ms: 1.01x faster | +| nqueens | 79.2 ms | 78.8 ms: 1.01x faster | +| comprehensions | 17.3 us | 17.2 us: 1.01x faster | +| sqlglot_normalize | 108 ms | 107 ms: 1.00x faster | +| unpack_sequence | 47.0 ns | 46.8 ns: 1.00x faster | +| sympy_integrate | 20.0 ms | 19.9 ms: 1.00x faster | +| 2to3 | 256 ms | 255 ms: 1.00x faster | +| python_startup | 11.1 ms | 11.1 ms: 1.00x slower | +| crypto_pyaes | 67.9 ms | 68.0 ms: 1.00x slower | +| python_startup_no_site | 7.41 ms | 7.42 ms: 1.00x slower | +| regex_dna | 177 ms | 177 ms: 1.00x slower | +| deepcopy_reduce | 2.65 us | 2.67 us: 1.01x slower | +| sqlglot_transpile | 1.59 ms | 1.60 ms: 1.01x slower | +| pprint_pformat | 1.48 sec | 1.49 sec: 1.01x slower | +| json_dumps | 11.3 ms | 11.4 ms: 1.01x slower | +| scimark_sor | 136 ms | 137 ms: 1.01x slower | +| hexiom | 5.96 ms | 6.02 ms: 1.01x slower | +| pyflate | 448 ms | 452 ms: 1.01x slower | +| telco | 7.22 ms | 7.29 ms: 1.01x slower | +| pprint_safe_repr | 716 ms | 723 ms: 1.01x slower | +| pickle | 12.3 us | 12.4 us: 1.01x slower | +| json | 4.61 ms | 4.66 ms: 1.01x slower | +| float | 78.8 ms | 79.8 ms: 1.01x slower | +| generators | 29.2 ms | 29.6 ms: 1.01x slower | +| fannkuch | 371 ms | 376 ms: 1.01x slower | +| bpe_tokeniser | 4.34 sec | 4.40 sec: 1.01x slower | +| create_gc_cycles | 1.34 ms | 1.36 ms: 1.01x slower | +| deepcopy_memo | 30.3 us | 30.8 us: 1.02x slower | +| spectral_norm | 113 ms | 115 ms: 1.02x slower | +| coverage | 80.2 ms | 81.7 ms: 1.02x slower | +| async_generators | 378 ms | 385 ms: 1.02x slower | +| genshi_text | 21.9 ms | 22.4 ms: 1.02x slower | +| coroutines | 22.6 ms | 23.1 ms: 1.03x slower | +| scimark_sparse_mat_mult | 4.48 ms | 4.62 ms: 1.03x slower | +| unpickle | 13.5 us | 13.9 us: 1.03x slower | +| deltablue | 3.25 ms | 3.45 ms: 1.06x slower | +| mdp | 2.36 sec | 2.51 sec: 1.06x slower | +| Geometric mean | (ref) | 1.00x faster | + +Benchmark hidden because not significant (23): nbody, asyncio_tcp, sqlglot_parse, xml_etree_parse, raytrace, json_loads, scimark_monte_carlo, bench_thread_pool, bench_mp_pool, dulwich_log, unpickle_pure_python, asyncio_websockets, asyncio_tcp_ssl, sqlglot_optimize, pycparser, richards, richards_super, scimark_fft, sympy_expand, xml_etree_iterparse, meteor_contest, genshi_xml, logging_silent + +# HPT report + +- Reliability score: 59.05% likely to be faster +- 90% likely to have a speedup of 1.00x +- 95% likely to have a speedup of 1.00x +- 99% likely to have a speedup of 1.00x + +# Memory +- memory change: 1.00x \ No newline at end of file diff --git a/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-base.svg b/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-base.svg new file mode 100644 index 00000000..754f197b --- /dev/null +++ b/results/bm-20241120-3.14.0a2+-8ebd73d/bm-20241120-vultr-x86_64-mpage-gh_115999_tlbc_call-3.14.0a2+-8ebd73d-vs-base.svg @@ -0,0 +1,3171 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +