RIX Haupt : Effort needed for the Pipeline work not clear

[astronomyk]

https://jira.eso.org/browse/MET-2114

The primary reason for the delay of the METIS pipeline FDR was the lack of available full-time equivalents to complete the FDR work for Nov 2022. It appears that the METIS pipeline team is now adequately staffed. Perhaps I missed it, but I could not find a staffing plan for the MAIT phase in the documentation. Please provide the staffing information for the MAIT phase and show what effort is required and how and by whom the work is done.
The most recently published WBS we have is E-LIS-NOVA-MET-0013 v6, which is dated December 2021 and is out of date.
Please provide the requested information and the current WBS.

@hugobuddel @wkausch @gotten @oczoske @ivh @sesquideus - could you provide some gut feeling estimates on how long it would take to implement each of the recipes - including algorithmic functions - in the last tab of the RIX tracker spreadsheet - (at least for the recipes you feel comfortable assessing)

Any estimate is better than me trying to guess.

Thank you!

[wkausch]

I have absolutely no idea how to estimate that..... And I don't deem it is useful to give such an estimate, which is most probably wrong... In the worst case we promise too much.

Or we'll do like Scotty did: Giving a ridiculously long estimate and let us celebrate as genies if we do in in half the time :-)

[hugobuddel]

I can't make such an estimate. Well I can, but it will be worthless, so I won't.

[wkausch]

I agree. We should make clear to Christoph that such an estimate is not reliable at all!

[astronomyk]

I also have no idea how to estimate things like this ... I think it would make more sense to think of this the other way round - we have deadlines when certain recipes need to be finished. We will aim to have a minimum "working" version finished as soon as possible, and then iterate on that working version as the more functionality (or accuracy) is needed.

At the end of the day, we are not producing hardware, and so it is difficult to estimate coding times. Especially as none of the team has ever written an ESO pipeline before

[astronomyk]

@ivh @oczoske @hugobuddel @wkausch - my apologies for dumping this thought on a Friday night. Please don't feel pressured to answer until the beginning of next week.

TL;DR
I had a different idea for how to guesstimate how many FTEs we need purely for the recipe development. Spoiler alert, I came up with 12 FTEs in total --> 6 FTEs for the recipe coding, and 6 FTEs for the dev-ops. I would very much appreciate your feedback on this before I respond to Christoph on Tuesday. Thanks!!

Long version:
You can find it in the same last tab of the RIX tracker spreadsheet. I would like your opinion on it, please, as this is one of the biggest outstanding RIDs that I (we) have.

Basically my logic is that the time taken to write a recipe (including functionality) is proportional to the number of "items" it has to consider. So, I counted up all the inputs, algorithm steps, outputs, and QC parameters for each recipe and assigned a weighting factor to each of these "items" (1 for inputs and outputs, 3 for algorithm steps, and 2 for QC parameter).

I took the weighted sum as the number of days needed to implement each recipe, and it gives me something that my gut is happy with - namely 6 FTEs for the implementation of the pipeline recipes. Obviously some recipes will in practice take longer and some may be shorter, but on average I would like to think that the time needed to write a recipe is indeed proportional to the sum of its parts.

Now, my questions to you

• do you think this is a reasonable way to do a bottom-up daumen-mal-pi calculation to the effort needed for the recipes alone?
• do you think the weighting factors (3 days per algorithm step, 1 day per in/output, 2 days per QC param) reasonable estimates? What would you think is appropriate?

Now again, I am fully aware that this is by no means a perfect (or even reliable) approach to this. But I need to somehow apply a method to quantify how much time we'll need for the implementation. As you all pointed out, gut feeling estimates on a per-recipe basis are meaningless, and ESO won't buy the "Scotty" approach, unfortunately ;)

Furthermore, I will use the assumption that 50% of our effort will be in writing the recipes, and 50% will be used in testing+debgging+learning EDPS+ etc. I think this assumption is relatively valid. The ratio may be 40-60 or 70-30, but that is on the same order of magnitude. The ratio between recipes and other, will not be 90-10, therefore the +/-20% either way will not completely kill my total effort estimates. Happy to hear your thoughts on this too.

Thanks in advance for all your constructive feedback!