optimize computation and memory

[felicepantaleo]

This PR should improve the overhead of looping over tiles by avoiding multiple copies

[jmcarcell]

Looks good. Maybe @ebrondol wants to comment on this.
Did you measure if there is any improvement? I would guess that not much since the copies were in the outer parts of the for loops, unless there is a huge number of points (?)

[ebrondol]

Looks good. Maybe @ebrondol wants to comment on this. Did you measure if there is any improvement? I would guess that not much since the copies were in the outer parts of the for loops, unless there is a huge number of points (?)

Hi @jmcarcell , I am testing the PR currently. The preliminary performance looks much better than before computing time wise, so I am confident it can be marged soon.

[felicepantaleo]

Looks good. Maybe @ebrondol wants to comment on this. Did you measure if there is any improvement? I would guess that not much since the copies were in the outer parts of the for loops, unless there is a huge number of points (?)

well, if every time you copy all the points in a layer, the overhead of creating tiles becomes bigger than the clustering itself and you enter the paradox in which coarser tiling gives better performance.

[ebrondol]

Indeed the timing is doing much better now.
After that all discussions are solved, I will merge the PR.

[ebrondol]

The changes align with the expectation of about a factor of 50-100 improvements in computing time.

[ebrondol]

The tests I run locally worked out fine.
I would rather wait until Issue #45 is closed and the CI is running again before merging it.

[felicepantaleo]

Thank you, Erica. Sure, I'm not in a hurry to have it merged.

[ebrondol]

Just for reference, I have produced the timing plots again with the latest commits from @felicepantaleo :

(disregard the label "before PR#43" because it is outside the x-axis region)

[ebrondol]

I have tested this PR extensively with other detectors and I think it can be merged also before the CI is fixed (still under progress and being monitored in #44 ).