Unexpected results from colour.colour_correction definition.

[bhive01]

Been doing basic linear regression in RGB space to do color correction for a long time and looking for something better. Thought I'd give this a try, but I"m not really getting great results. I have an imaging setup where we take photos of objects with a blue background. The lights are not high CRI, so I'll get a measurement next week.
Our standard protocol is to white balance the SLR (M6 mkII) and set the exposure to where there is no clipping on the xrite/calibrite passport (White ~ 240/255). My code finds the card, orientates it, and then reads the mean value from the square. I've used these values for years to correct the images via regression with RGB and more recently Lab* and it mostly works, but it's not always satisfying.
When I tried colour.colour_correction() I was getting very strange outputs in initially and eventually tried all combinations of polynomial expansion and degree.
0 Degree False

1 Degree False

1 Degree True

2 Degree False

2 Degree True

3 Degree False

3 Degree True

4 Degree False

4 Degree True

0 False is the card before correction at all. And then we go up in each degree True means polynomial expansion is applied and False is not. By eye, degree 2 and True appears to be the best, but I don't understand why when it gets to degree 4 it goes all the rails. You can see some weird clipping in degree 3 around the cyan square. I admit I'm a bit of a newbie with color theory even though I've been using it for years.

[bhive01]

A bit more information is probably needed.
The RGB, BGR and Lab values for the card are from the same source that you get them from.
xrite_target_colors = { 0: {'rgb': (105.0, 79.0, 65.0), 'bgr': (65.0, 79.0, 105.0), 'lab': (37.54, 14.37, 14.92), 'name': 'dark skin', 'munsell': '3 YR 3.7 / 3.2'}, 1: {'rgb': (193.0, 144.0, 127.0),'bgr': (127.0, 144.0, 193.0), 'lab': (64.66, 19.27, 17.5), 'name': 'light skin', 'munsell': '2.2 YR 6.47 / 4.1'}, 2: {'rgb': (90.0, 120.0, 155.0), 'bgr': (155.0, 120.0, 90.0), 'lab': (49.32, -3.82, -22.54), 'name': 'blue sky', 'munsell': '4.3 PB 4.95 / 5.5'}, 3: {'rgb': (74.0, 108.0, 64.0), 'bgr': (64.0, 108.0, 74.0), 'lab': (43.46, -12.74, 22.72), 'name': 'foliage', 'munsell': '6.7 GY 4.2 / 4.1'}, 4: {'rgb': (132.0, 127.0, 175.0),'bgr': (175.0, 127.0, 132.0), 'lab': (54.94, 9.61, -24.79), 'name': 'blue flower', 'munsell': '9.7 PB 5.47 / 6.7'}, 5: {'rgb': (115.0, 198.0, 172.0),'bgr': (172.0, 198.0, 115.0), 'lab': (70.48, -32.26, -0.37), 'name': 'bluish green', 'munsell': '2.5 BG 7 / 6'}, 6: {'rgb': (204.0, 124.0, 48.0), 'bgr': (48.0, 124.0, 204.0), 'lab': (62.73, 35.83, 56.5), 'name': 'orange', 'munsell': '5 YR 6 / 11'}, 7: {'rgb': (67.0, 90.0, 167.0), 'bgr': (167.0, 90.0, 67.0), 'lab': (39.43, 10.75, -45.17), 'name': 'purplish blue', 'munsell': '7.5 PB 4 / 10.7'}, 8: {'rgb': (199.0, 80.0, 95.0), 'bgr': (95.0, 80.0, 199.0), 'lab': (50.57, 48.64, 16.67), 'name': 'moderate red', 'munsell': '2.5 R 5 / 10'}, 9: {'rgb': (77.0, 58.0, 104.0), 'bgr': (104.0, 58.0, 77.0), 'lab': (30.1, 22.54, -20.87), 'name': 'purple', 'munsell': '5 P 3 / 7'}, 10: {'rgb': (144.0, 187.0, 58.0), 'bgr': (58.0, 187.0, 144.0), 'lab': (71.77, -24.13, 58.19), 'name': 'yellow green', 'munsell': '5 GY 7 / 10'}, 11: {'rgb': (201.0, 161.0, 39.0), 'bgr': (39.0, 161.0, 201.0), 'lab': (71.51, 18.24, 67.37), 'name': 'orange yellow', 'munsell': '10 YR 7 / 11.2'}, 12: {'rgb': (33.0, 62.0, 145.0), 'bgr': (145.0, 62.0, 33.0), 'lab': (28.37, 15.42, -49.8), 'name': 'blue', 'munsell': '7.5 PB 3 / 11'}, 13: {'rgb': (60.0, 147.0, 70.0), 'bgr': (70.0, 147.0, 60.0), 'lab': (54.38, -39.72, 32.27), 'name': 'green', 'munsell': '2.5 G 5.5 / 8'}, 14: {'rgb': (176.0, 54.0, 57.0), 'bgr': (57.0, 54.0, 176.0), 'lab': (42.43, 51.05, 28.62), 'name': 'red', 'munsell': '5 R 4.2 / 12.1'}, 15: {'rgb': (205.0, 199.0, 15.0), 'bgr': (15.0, 199.0, 205.0), 'lab': (81.8, 2.67, 80.41), 'name': 'yellow', 'munsell': '5 Y 8.5 / 11.9'}, 16: {'rgb': (183.0, 79.0, 146.0), 'bgr': (146.0, 79.0, 183.0), 'lab': (50.63, 51.28, -14.12), 'name': 'magenta', 'munsell': '2.5 RP 5 / 10.5'}, 17: {'rgb': (47.0, 133.0, 165.0), 'bgr': (165.0, 133.0, 47.0), 'lab': (49.57, -29.71, -28.32),'name': 'cyan', 'munsell': '5 B 5 / 8'}, 18: {'rgb': (235.0, 242.0, 235.0),'bgr': (235.0, 242.0, 235.0), 'lab': (95.19, -1.03, 2.93), 'name': 'white', 'munsell': 'N 9.5'}, 19: {'rgb': (206.0, 202.0, 201.0),'bgr': (201.0, 202.0, 206.0), 'lab': (81.29, -0.57, 0.44), 'name': 'neutral 8', 'munsell': 'N 8'}, 20: {'rgb': (167.0, 163.0, 163.0),'bgr': (163.0, 163.0, 167.0), 'lab': (66.89, -0.75, -0.06), 'name': 'neutral 6.5', 'munsell': 'N 6.5'}, 21: {'rgb': (120.0, 121.0, 121.0),'bgr': (121.0, 121.0, 120.0), 'lab': (50.76, -0.13, 0.14), 'name': 'neutral 5', 'munsell': 'N 5'}, 22: {'rgb': (75.0, 84.0, 85.0), 'bgr': (85.0, 84.0, 75.0), 'lab': (35.63, -0.46, -0.48), 'name': 'neutral 3.5', 'munsell': 'N 3.5'}, 23: {'rgb': (34.0, 50.0, 50.0), 'bgr': (50.0, 50.0, 34.0), 'lab': (20.64, 0.07, -0.46), 'name': 'black', 'munsell': 'N 2'} }
Reading the squares I get the following:
[[ 0.4748204 0.29742503 0.23292245] [ 0.8483145 0.668922 0.58630687] [ 0.43473458 0.57581115 0.74997324] [ 0.37822497 0.4667806 0.26845536] [ 0.6279988 0.58831877 0.79597074] [ 0.5127445 0.85497457 0.81820154] [ 0.9437081 0.5405256 0.16040738] [ 0.35413054 0.41023508 0.80760664] [ 0.8807828 0.36977154 0.39951536] [ 0.37621227 0.19235978 0.4424536 ] [ 0.7148696 0.83629775 0.37487543] [ 0.9354186 0.7101161 0.22871213] [ 0.19652951 0.263705 0.7072465 ] [ 0.29718855 0.72554016 0.3779713 ] [ 0.80051196 0.21381173 0.18222846] [ 0.9467603 0.8534184 0.26864812] [ 0.8847683 0.38623977 0.67271096] [ 0.2227685 0.6913714 0.83057106] [ 0.9378465 0.93397796 0.9265984 ] [ 0.8484042 0.85486525 0.8588813 ] [ 0.7260755 0.74119675 0.7477296 ] [ 0.53686935 0.54215515 0.55001384] [ 0.3165974 0.32535362 0.33982947] [ 0.14361759 0.14522603 0.14950034]]

Then the references:
[[ 0.4117647 0.30980393 0.25490198] [ 0.75686276 0.5647059 0.49803922] [ 0.3529412 0.47058824 0.60784316] [ 0.2901961 0.42352942 0.2509804 ] [ 0.5176471 0.49803922 0.6862745 ] [ 0.4509804 0.7764706 0.6745098 ] [ 0.8 0.4862745 0.1882353 ] [ 0.2627451 0.3529412 0.654902 ] [ 0.78039217 0.3137255 0.37254903] [ 0.3019608 0.22745098 0.40784314] [ 0.5647059 0.73333335 0.22745098] [ 0.7882353 0.6313726 0.15294118] [ 0.12941177 0.24313726 0.5686275 ] [ 0.23529412 0.5764706 0.27450982] [ 0.6901961 0.21176471 0.22352941] [ 0.8039216 0.78039217 0.05882353] [ 0.7176471 0.30980393 0.57254905] [ 0.18431373 0.52156866 0.64705884] [ 0.92156863 0.9490196 0.92156863] [ 0.80784315 0.7921569 0.7882353 ] [ 0.654902 0.6392157 0.6392157 ] [ 0.47058824 0.4745098 0.4745098 ] [ 0.29411766 0.32941177 0.33333334] [ 0.13333334 0.19607843 0.19607843]]

[KelSolaar]

Hello @bhive01,

The polynomial correction instability will increase as a function of the sparsity of your reference/training samples and the degree you use. This image is a good example of what is going on:

As you see here, the higher the degree and the higher are the chances there is overfitting if you don't have enough reference samples. This get even worse outside the reference domain where the polynomial will rapidly change direction when extrapolating beyond the boundaries of the reference samples.

Practically speaking, it will be hard to use anything beyond a degree 2 with only the 24 patches of a ColorChecker Classic, it is simply not enough reference samples to produce a stable result. That being said, I commonly use degree 2 to 4 for spectrally-based camera characterisation, but I have ~200 samples for that.

I hope this helps!

Cheers,

Thomas

[bhive01]

Thanks Thomas. Your example does indeed explain why the chromatic aberrations appear when I go beyond my skis with 3rd and 4th degree polynomials. I have one suggestion and a followup question on color spaces.
The example in the collab document uses the same 24 square color checker and fits a 4th degree polynomial to it. I propose that the example be edited to a more moderate 2nd with polynomial expansion. I was following this example and was wondering if I had messed it up for many days.
My follow up question is about what sort of color inputs the correction algorithm can take. Internally it looks like you specifically reference RGB, but as mentioned in the example you call out that Lab* is more linear colorspace. I've been using Lab color correction for the last two years instead of RGB. But, just for clarity, I could feed BGR, or Lab* into the algorithm and it wouldn't know the difference and it would likely work?

[KelSolaar]

Good point for the example, I have amended it! CIE Lab is a more perceptually uniform than CIE xyY. Anything would work fed through the correction algorithm, it is totally agnostic. What we tend to do in production is actually a bit more involved and I will try to create an example to highlight that.

[bhive01]

Thanks, I would be super interested to see a more professional workflow. I assume with 200 samples you're using a much more complex checker, or a spectrophotometer of some sort?

[bhive01]

Here is the spectrum of the LED lights we're using as source lighting.