Improve benchmarks #1034
Improve benchmarks #1034
Conversation
Co-authored-by: Guillaume Petiot <[email protected]>
doc/driver.mld
Outdated
| "base"; | ||
| "core_kernel"; | ||
| "bin_prot"; | ||
| "sexplib"; | ||
| "sexplib0"; | ||
| "base_quickcheck"; | ||
| "ppx_sexp_conv"; | ||
| "ppx_hash"; | ||
| "core"; |
There was a problem hiding this comment.
How were the previous "extra deps" chosen? If the reason was valid, why not keeping them, adding "core"?
There was a problem hiding this comment.
I think the intent of the previous list was to have a world that is as large as core but that is not core, because it failed to build at some point. I might be wrong.
There was a problem hiding this comment.
No, it was that core had some dependencies and some were vaguely interesting, so might as well build them too. Particularly anything that core/core_kernel might have linked to (so sexplib, bin_prot, etc).
There was a problem hiding this comment.
Alright, I added them back. Though I did not add more as that would make the list very large without adding interesting test cases.
There was a problem hiding this comment.
My thinking was mainly that certain modules take vastly longer than others, so including a new library either means it's very quick (and not interesting) or that some modules now take ages, which is very interesting, and when you fix it, becomes a useful test case :-)
The previous 'longest' metric was not useful as it was a duplicate of the other running time metrics that output a 'max' value. Make this metric more useful by averaging the 5 longest runs. This correspond roughly to the top 1%.
Julow
force-pushed
the
driver-bench-longest
branch
from
37628e0
to
211442f
Compare
Currently, the benchmarks are running the driver with no extra work.
As the driver reports errors with promotion and not with an exit status, the driver could fail to generate `driver-benchmarks.json`, in which case Dune would complain about the missing target and would not show the diff containing the error.
Julow
force-pushed
the
driver-bench-longest
branch
from
211442f
to
2f59a5d
Compare
|
OK, green tick. In it goes. |
corein the benchmarks, which contains modules on which Odoc is very slow.