key perf metrics

[BluBloos]

I was curious if there was a dedicated test under ./tests for measuring some key perf metrics of Tasks, but I couldn't find anything like this?

Would such a test be valuable for Tasks development?

Obviously there couldn't be any requires but still a simple report for the perf metrics could help to validate any future perf-related PRs.

[claremacrae]

I am presuming you are referring to speed of execution?

FWIW In my experience when tests are run on anything other than very tightly controlled physical hardware they have little value, as there is no control over the performance of the underlying 'machine' - like, even if it sort-of-gets-swapped-out-altogether temporarily.

Are you aware of some other way of getting reliable performance measurements from GitHub Actions?

[BluBloos]

I am presuming you are referring to speed of execution?

Yep!

And, sorry! I should have clarified that I wasn't talking about adding any tests to run automagically via Github actions. I was more interested in tests that a dev would run locally on their machine.

As for what you mention about tightly controlled hardware, etc, this is where your testing experience over mine takes the cake...

To really narrow the scope of this question, I guess what I'm asking is for some reliable way to run the "test", write some performance-optimizing changes, then run it again and observe a change in runtime performance.

To narrow the scope even further, I was looking at the repo docs and found a resources/samples_vaults/Tasks-Demo/Manual Testing/Stress Test/

The Python script generated some files, but I thought that after doing so it would also run some of the algo's in Tasks on those files to test runtime perf

[claremacrae]

I am presuming you are referring to speed of execution?

Yep!

And, sorry! I should have clarified that I wasn't talking about adding any tests to run automagically via Github actions. I was more interested in tests that a dev would run locally on their machine.

Ok, that sounds really interesting.

As for what you mention about tightly controlled hardware, etc, this is where your testing experience over mine takes the cake...

:-)

To really narrow the scope of this question, I guess what I'm asking is for some reliable way to run the "test", write some performance-optimizing changes, then run it again and observe a change in runtime performance.

OK.

To narrow the scope even further, I was looking at the repo docs and found a resources/samples_vaults/Tasks-Demo/Manual Testing/Stress Test/

The Python script generated some files, but I thought that after doing so it would also run some of the algo's in Tasks on those files to test runtime perf

It could do, certainly.

By the way, all the reported performance issues here are interactive things, such as the plugin becoming unresponsive to user edits if it takes close to 2 seconds or more to refresh all the open tasks blocks.

https://github.com/obsidian-tasks-group/obsidian-tasks/issues?q=is%3Aopen+is%3Aissue+label%3A%22scope%3A+performance+issues%22

So I have been tending to assume that the kind of things that are amenable to timing of automated tests would not be too useful for performance monitoring... But (a) that may be untrue (b) if it were true now, it could change in future...

[BluBloos]

It's true! Those may be the pressing perf concerns and therefore what to focus on.

Maybe should those ever be "fixed", this sort of work comes next :)

[claremacrae]

It's true! Those may be the pressing perf concerns and therefore what to focus on.

Definitely...

Maybe should those ever be "fixed", this sort of work comes next :)

Indeed. So I think this isn't a very high priority for the foreseeable future.