[Perf] Windows/x64: 8 Regressions on 12/13/2024 8:12:44 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Mac OS X 10.18
Queue	IPhone
Baseline	f9c86dc7b449bbdfd2fcafa58e513357372509a5
Compare	43b89b4a40f7642891e48801d56bd706ab717d6b
Diff	Diff
Configs	CompilationMode:tiered, HybridGlobalization:true, iOSStripSymbols:true, RunKind:ios_scenarios, RuntimeType:nativeaot

Regressions in SOD - iOS HelloWorld .app Size nollvm nosymbols

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
app Size nollvm nosymbols - SOD - iOS HelloWorld .app Size nollvm nosymbols ADX - Test Multi Config Graph	1.53 MB	1.55 MB	1.01	0.00	True
app Size nollvm nosymbols - Aggregate - No Extension ADX - Test Multi Config Graph	1.53 MB	1.55 MB	1.01	0.00	True
app Size nollvm nosymbols - pub/iosHelloWorld/HelloiOS.app/HelloiOS ADX - Test Multi Config Graph	1.53 MB	1.55 MB	1.01	0.00	True
app Size nollvm nosymbols - pub ADX - Test Multi Config Graph	1.53 MB	1.55 MB	1.01	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SOD - iOS HelloWorld  .app Size nollvm nosymbols*'

SOD - iOS HelloWorld .app Size nollvm nosymbols

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Mac OS X 10.18
Queue	IPhone
Baseline	f9c86dc7b449bbdfd2fcafa58e513357372509a5
Compare	43b89b4a40f7642891e48801d56bd706ab717d6b
Diff	Diff
Configs	CompilationMode:tiered, HybridGlobalization:true, iOSStripSymbols:true, RunKind:ios_scenarios, RuntimeType:nativeaot

Regressions in SOD - iOS HelloWorld Zip Size nollvm nosymbols

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SOD - iOS HelloWorld Zip Size nollvm nosymbols - pub/iOSSampleApp.zip ADX - Test Multi Config Graph	678.26 KB	679.91 KB	1.00	0.00	True
SOD - iOS HelloWorld Zip Size nollvm nosymbols - pub ADX - Test Multi Config Graph	678.26 KB	679.91 KB	1.00	0.00	True
SOD - iOS HelloWorld Zip Size nollvm nosymbols - Aggregate - .zip ADX - Test Multi Config Graph	678.26 KB	679.91 KB	1.00	0.00	True
SOD - iOS HelloWorld Zip Size nollvm nosymbols - SOD - iOS HelloWorld Zip Size nollvm nosymbols ADX - Test Multi Config Graph	678.26 KB	679.91 KB	1.00	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SOD - iOS HelloWorld  Zip Size nollvm nosymbols*'

SOD - iOS HelloWorld Zip Size nollvm nosymbols

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[matouskozak]

Possibly dotnet/runtime#110469, regression of 1%, fyi: @MichalStrehovsky

[MichalStrehovsky]

Possibly dotnet/runtime#110469, regression of 1%, fyi: @MichalStrehovsky

I find that very unlikely. This is a bit more code because there's more functionality, but not that much. rt-sz measured 8 kB tops: MichalStrehovsky/rt-sz#91

[MichalStrehovsky]

Submitted MichalStrehovsky/rt-sz#92 for a different PR that I find suspicious in that range. Will see tomorrow.

[matouskozak]

Submitted MichalStrehovsky/rt-sz#92 for a different PR that I find suspicious in that range. Will see tomorrow.

Looking at the graph, the shortest range containing the regressions has just these 2 commits dotnet/runtime@eab352d...6362f24.

Maybe it could be another case of different configuration that we use to be "comparable" with MonoAOT like dotnet/runtime#104074 (comment). fyi: @ivanpovazan

That said I don't think it's worth to spend too much time on this as it's only ~20 kB of regression.