Strange PV data

[peterdudfield]

Some PV systems have odd data
It may be better to remove some of these,
or we could still collect the data but not add them to the forecast

For example system 37543

Full list
ODD

• 20471: odd results - also in pvoutput.org - now looks ok
• 30248: lots of zeros - Manchester - error - should remove - correct_data=False
• 37543: not zero in the night - Scotland - correct_data=False
• 38254: constant non zero value - Near Luton. Issue only started on 06-06-2022 - correct_data=False
• 40283: lots of zeros - Reading - Looks fine, our installed capacity might be too big. It should be 4 kW - already at 3.6 - 40283 zeros PVConsumer#29

Capacity issue

• 2490: close to zero - Manchester - Looks fine, our installed capacity might be too big. It should be 3.43 kW - Updated
• 5257: close to zero - Hastings - Looks fine, our installed capacity might be too big. It should be 3.92 kW - Updated
• 27003: close to zeros - London - Looks fine, our installed capacity might be too big. It should be 2.1 kW - Updated
• 42586: lots of zeros - Ireland - Looks fine, our installed capacity might be too big. It should be 4 kW - updated
• 44135: lots of zeros - Ireland - Looks fine, our installed capacity might be too big. It should be 2 kW, already at 2.152
• 57569: always zero - Ipswich - Looks fine, our installed capacity might be too big. It should be 2.95 kW, already set at 2.26
• Also updated 5729

Alternating zeros

• 6964: alternating zeros - no zeros in pvoutput.org. Might be a bug in PVConsumer - also see these in the database
• 45118: alternating zeros - Eastbourne - no zeros in pvoutput.org. Might be a bug in PVConsumer
• 51657: alternating zeros - Seven oaks - no zeros in pvoutput.org. Might be a bug in PVConsumer
• 61764: alternating zeros - Wales - no zeros in pvoutput.org. Might be a bug in PVConsumer
  Moved this to Aternative zeros PVConsumer#26

Think it would be good to have two modifications in in the datamodel

1. add column, defect: boolean. We can then only load when defect=True
2. Add column that describes what post processing the PV data should have. This can be done when loading the data. Its good to save the raw data. We obviously dont want to do to much post processing, but we should use domain knowledge to adjust obvious errors

[peterdudfield]

61764 - seems to have lots of zeros in it

[peterdudfield]

20471

[peterdudfield]

57569 - always zero

[peterdudfield]

5257 - close to zero

[peterdudfield]

44135 - lots of zeros

[JackKelly]

A few quick thoughts:

• I wonder if the "lots of zeros" problem is that we're filling 15-minutely PV data with zeros?
• I'm surprised how often the PV power gets up to 100% in these plots. I'm pretty sure it's actually quite rare for PV to get to 100% (requiring lots of sunlight and a cool breeze to keep the panels cool 🙂 ) I'd guess this is related to Solar forecast consistly over estimates #114 (comment)

[peterdudfield]

A lot of zeros was solved by a incorrect installed capacity, so it looked like zero but was not.

Yea, we set the installed capacity by the maximum power (from training data), we could update these with the installed capacity from pv outputorg, but I thought this number was less reliable.

[peterdudfield]

Alos set 13764 to inccoect data as not zero in the night

[JackKelly]

Yea, we set the installed capacity by the maximum power (from training data), we could update these with the installed capacity from pv outputorg, but I thought this number was less reliable.

Yeah, exactly. I'd ignore the capacity from pvoutput.org, and just use the max over the entire timeseries (assuming we have at least, say, 6 months of data).

[peterdudfield]

can close this for the moment