Add Pinball metric

[peterdudfield]

Detailed Description

Would be good to see how well national_xg model is doing, and Pvnet. Perhaps just do this on National first of all

[peterdudfield]

https://www.lokad.com/pinball-loss-function-definition/#:~:text=The%20pinball%20loss%20function%2C%20also,forecast%20is%20a%20subtle%20problem.

[peterdudfield]

We could expand this out to Pvnet_2 too now

[peterdudfield]

We probably want to do a few probablistic metrics

• pinball
• exceedence, i.e how often the truth value exceeds the p_{XX} value
  And would be good to do this in the sql query

[peterdudfield]

For getting the values out of the json column. We could use json_extract_path(properties, '10')

[abhijelly]

@peterdudfield can I take this up if no one is working on this?

[abhijelly]

hey @peterdudfield I'm currently giving this a go and in order to test my approach, I believe I need the database - how do I connect to an actual or a dummy database if possible?

[peterdudfield]

Hey @abhijelly , ive atcually started this in #41 so this might not be the best one to do.

Would you like to do openclimatefix/pv-site-production#59 (I will add some more detail to this task)

[peterdudfield]

hey @peterdudfield I'm currently giving this a go and in order to test my approach, I believe I need the database - how do I connect to an actual or a dummy database if possible?

Yea connecting to the main database is going to be hard. In the past we've gone along

• write code
• added tests, which has a dummy database
• tested it on development database (OCF member)
  Sorry I know its not ideal

[peterdudfield]

@abhijelly you may want to consider doing this one too - openclimatefix/pv-site-api#124