P050_TCS34725 - RGB raw data and/or RGB normalization

[heinemannj]

At the end of the day I need at the minimum:

• normalized RGB / rgb values
• Color Temperature (CCT)

For optimum implementation plus:

• Lux value
• C, R, G, B raw data

To be calculated via:

/*!
 *  @brief  Read the RGB color detected by the sensor.
 *  @param  *r
 *          Red value normalized to 0-255
 *  @param  *g
 *          Green value normalized to 0-255
 *  @param  *b
 *          Blue value normalized to 0-255
 */
void Adafruit_TCS34725::getRGB(float *r, float *g, float *b) {
  uint16_t red, green, blue, clear;
  getRawData(&red, &green, &blue, &clear);
  uint32_t sum = clear;

  // Avoid divide by zero errors ... if clear = 0 return black
  if (clear == 0) {
    *r = *g = *b = 0;
    return;
  }

  *r = (float)red / sum * 255.0;
  *g = (float)green / sum * 255.0;
  *b = (float)blue / sum * 255.0;
}

/*!
 *  @brief  Converts the raw R/G/B values to color temperature in degrees
 *          Kelvin using the algorithm described in DN40 from Taos (now AMS).
 *  @param  r
 *          Red value
 *  @param  g
 *          Green value
 *  @param  b
 *          Blue value
 *  @param  c
 *          Clear channel value
 *  @return Color temperature in degrees Kelvin
 */
uint16_t Adafruit_TCS34725::calculateColorTemperature_dn40(uint16_t r,
                                                           uint16_t g,
                                                           uint16_t b,
                                                           uint16_t c) {
  uint16_t r2, b2; /* RGB values minus IR component */
  uint16_t sat;    /* Digital saturation level */
  uint16_t ir;     /* Inferred IR content */

  if (c == 0) {
    return 0;
  }

  /* Analog/Digital saturation:
   *
   * (a) As light becomes brighter, the clear channel will tend to
   *     saturate first since R+G+B is approximately equal to C.
   * (b) The TCS34725 accumulates 1024 counts per 2.4ms of integration
   *     time, up to a maximum values of 65535. This means analog
   *     saturation can occur up to an integration time of 153.6ms
   *     (64*2.4ms=153.6ms).
   * (c) If the integration time is > 153.6ms, digital saturation will
   *     occur before analog saturation. Digital saturation occurs when
   *     the count reaches 65535.
   */
  if ((256 - _tcs34725IntegrationTime) > 63) {
    /* Track digital saturation */
    sat = 65535;
  } else {
    /* Track analog saturation */
    sat = 1024 * (256 - _tcs34725IntegrationTime);
  }

  /* Ripple rejection:
   *
   * (a) An integration time of 50ms or multiples of 50ms are required to
   *     reject both 50Hz and 60Hz ripple.
   * (b) If an integration time faster than 50ms is required, you may need
   *     to average a number of samples over a 50ms period to reject ripple
   *     from fluorescent and incandescent light sources.
   *
   * Ripple saturation notes:
   *
   * (a) If there is ripple in the received signal, the value read from C
   *     will be less than the max, but still have some effects of being
   *     saturated. This means that you can be below the 'sat' value, but
   *     still be saturating. At integration times >150ms this can be
   *     ignored, but <= 150ms you should calculate the 75% saturation
   *     level to avoid this problem.
   */
  if ((256 - _tcs34725IntegrationTime) <= 63) {
    /* Adjust sat to 75% to avoid analog saturation if atime < 153.6ms */
    sat -= sat / 4;
  }

  /* Check for saturation and mark the sample as invalid if true */
  if (c >= sat) {
    return 0;
  }

  /* AMS RGB sensors have no IR channel, so the IR content must be */
  /* calculated indirectly. */
  ir = (r + g + b > c) ? (r + g + b - c) / 2 : 0;

  /* Remove the IR component from the raw RGB values */
  r2 = r - ir;
  b2 = b - ir;

  if (r2 == 0) {
    return 0;
  }

  /* A simple method of measuring color temp is to use the ratio of blue */
  /* to red light, taking IR cancellation into account. */
  uint16_t cct = (3810 * (uint32_t)b2) / /** Color temp coefficient. */
                     (uint32_t)r2 +
                 1391; /** Color temp offset. */

  return cct;
}

/*!
 *  @brief  Converts the raw R/G/B values to lux
 *  @param  r
 *          Red value
 *  @param  g
 *          Green value
 *  @param  b
 *          Blue value
 *  @return Lux value
 */
uint16_t Adafruit_TCS34725::calculateLux(uint16_t r, uint16_t g, uint16_t b) {
  float illuminance;

  /* This only uses RGB ... how can we integrate clear or calculate lux */
  /* based exclusively on clear since this might be more reliable?      */
  illuminance = (-0.32466F * r) + (1.57837F * g) + (-0.73191F * b);

  return (uint16_t)illuminance;
}

In addition the actual Adafruit_TCS34725 lib 1.3.3 was continiously improved during the last years.

[tonhuisman]

The number of values in ESPEasy is limited to 4, how would you want to select what to use? Currently the raw R/G/B value and Color temperature CCT are made available.
Because of the construction of this plugin it seems possible to have multiple instances, working on the same physical device, possible (the address is fixed anyway, but I can't validate that working as I don't have that sensor), the tasks just need to be set at an interval that avoids trying to read it at the same time (by setting the interval to different prime numbers, 11, 13, 17, etc., that would mostly be avoided), so a selection of Raw/Calibrated RGB, and CCT/Lux would be enough?

What interesting improvements have been made to the Adafruit library that we should upgrade it? In general, newer libraries are larger than previous versions, and we sure don't need more bloated/unused code... Bug-fixes are, of course, nice to have 😄

[tonhuisman]

I added the needed stuff to P050, applied the needed patches (minimally) to the library and the device screen now looks like this with cleaned up existing options to be a bit more readable:

(Rest of the options: see below)

[heinemannj]

The number of values in ESPEasy is limited to 4

Uppss ...
One of my next projects will be an 18 channel Spectroscopy Sensor
https://cdn.sparkfun.com/assets/8/5/f/0/3/AS7265x_Design_Considerations.pdf).
https://www.sparkfun.com/products/15050

In this case more than 5 instances on the same physical device will be required ...
All attempts to avoid parallel access/polling will end in a nightmare ...

[heinemannj]

Raw/Calibrated RGB ...

Big misunderstanding:

The AMS sensor and the Adafruit library dealing mainly with RGB raw data, which is dependant on different device settings like Gain and Integration Time.

This raw RGB data must be normalized to:

• RGB (0-255) (only 8 Bit infomation depth)
• rgb (0.00000-1.00000) -16 Bit infomation depth for further transformation into different CIE color spaces is highly recommended

Normalized data should be independant on device settings (e.g. Gain and Integration Time, ...)

Calibration for normalized data is potentially required due to the following:

• Measurement under specific light conditions (Wavelength, Illuminance (lux), Color Temperature (CCT), Daylight, LED light, ...)
• Distance between light source - Object - Sensor

Please let us discuss the calibration topic seperately to avoid the mixture of different topics.
But for sure ESPEasy Wiki content needs to be reviewed and corrected:

Calibration is done by experimenting with the time and gain and also the formula for each individual RGB color.

https://www.letscontrolit.com/wiki/index.php/TCS34725_RGB_Color_Sensor

There are interesting issues on Adafruit Git:
adafruit/Adafruit_TCS34725#21

adafruit/Adafruit_TCS34725#15
adafruit/Adafruit_TCS34725#29
adafruit/Adafruit_TCS34725#32

My conclusion:

• The older Adafruit lib versions are buggy and should be replaced by the newest one
• In addition the following study is very important: https://github.com/ha5dzs/colour_sensor_primary_calculator

Note:

• There is an old but open Pull request: enhancement of the Tcs34725 Plugin with Interrupts and Multiplexer Support #255

[tonhuisman]

One of my next projects will be an [18 channel Spectroscopy Sensor]

An alternative could be to generate events for all measurements. The names of these events can be stored in the settings (but have to be specifically programmed).
The four most important, or user-selectable, measurements could then me put into the default available values.

[tonhuisman]

Raw/Calibrated RGB ...

Big misunderstanding:

I took off on the initial post, but mistakenly used Calibrated where I should have used Normalized. I'll fix that.

The older Adafruit lib versions are buggy and should be replaced by the newest one

That's why I merged the (minimal) changes from their master branch into the current library source. I haven't taken the time yet to investigate if there are bigger fixes still pending in PR's. I'll be able to do that the next couple of days, I expect.

Please let us discuss the calibration topic seperately to avoid the mixture of different topics.

Please describe this as accurate as possible, as I don't own this sensor (yet) so have no way to confirm, nor the knowledge and equipment to perform such calibration.

[TD-er]

About the need for getting lots of units of measure from a single sensor...
The Eastron plugin does use a shared object to communicate with a single device and is used by all instances of the same plugin to get various values and thus you are no longer limited to the max 4 task values.

Another way could be to implement new commands for that plugin.
Currently all commands implemented in a plugin (via the PLUGIN_WRITE call) are "actions" to interact with the device.
But I guess we could also use a parameter of such a command to indicate the variable number to store the result of such a command.
This way you can have specific calculation functions which result in a value, like color space conversions.
A color space conversion does yield multiple values, so you can then use multiple variables (e.g. use 3 to address %v3% or [var#3])

[tonhuisman]

I've added the Normalized sRGB calculation, and other code improvements (as suggested in the PR comments).
Updated screenshots of the Device page:

The options for the RGB Values:

And for the 4th value:

(Not pushed yet, will be patching the (partly untested?) library code, as indicated in the issues there, after this)

[heinemannj]

Please wait with the final merge.

I'm actually working on the transformation of RGB data into CIE Color standards.

The final outcome can be used to simplify the plugin by providing only the basic data because other values can be easily calculated from the normalized RGB data (4 digit precision for RGB seems to be strong enough).

R: 156.0909
G: 100.4546
B: 73.4091

[tonhuisman]

I now also applied the patches suggested in the PR's for the library. They are mostly related to the timings of reading the values. Had to do some minor modifications because the project compiler setting are more strict than that author was using.
No testing was done, as I have no TCS34725 available here.

I can provide a .bin if you tell what configuration you'd like, I'm testing with a test_ESP8266_4M1M_VCC where I added P050.

[heinemannj]

Note about the proper usage of gain and integration time:

• In general, you want to use the minimum gain possible, since increasing the gain amplifies the noise as well as the signal.
• You also want to choose an integeration time long enough to produce a near full-scale value, since this will give you the most effective use of the sensor's resolution.

[TD-er]

I do understand the relation between noise and integration time very well, and also the importance of it.
Would it be an idea to add an iterative process to tune gain and integration time and maybe also noise filtering by averaging?

Factors to include:

• Net frequency (if it is an issue), forcing the integration time to be a multiple of 1/4 of the net period.
• Speed vs low noise priority

For example, if the net frequency is 50 Hz, then the integration time should be a multiple of 5 msec. For dimmed light it may perhaps be better to use 1/2 of the net period => 10 msec steps.

When speed is a priority, the iteration steps are:

• Set to max allowed integration time and average gain
• Tune using gain to get the brightest color between 50 and 95% of range.
• If still too bright, lower integration time

When low noise is a priority:

• Set to lowest allowed gain and average integration time
• Tune using integration time (per channel?) to get the brightest color between 50 and 95% of range.
• If still too dim, increase gain (per channel?)
• Take N samples and average.

N.B. this only can work if the light to measure is constant over some time.

[tonhuisman]

@heinemannj Are you working/starting on the documentation part? In that case I won't try to meddle with that, as I'm kind of 'the piano-player', not the 'composer' of this plugin, and am not anywhere nearly as informed about the functional stuff of this sensor as you are.
If so, I'll declare the PR 'ready for integration', unless you have suggestions or the code needs further improvements.

[heinemannj]

Please wait - I beleave I've found additional bugs in RGB caculations ...
Maybe I need to sleep a night for sorting out ...

The sensor without any kind of calibration is completely useless!!!
The documentation part will be very tricky ...
I will make some proposals.

[heinemannj]

The RGB values measured by the sensor are tristimulus values - NOT sRGB values!

https://medium.com/hipster-color-science/a-beginners-guide-to-colorimetry-401f1830b65a

• For that reason we should remove the "s" before sRGB in all descriptions
• Renaming Normalized sRGB into "rgb"
• Calculation of RGB values according to above => R=rawR/(rawR+rawG+rawB), ...
• Add a clear note on mandatory sensor calibration with the link to ESPEasy Doc or Wiki

Mandatory: Transform all RGB values with your evaluated calibration factors (below "Formulars" can be used)

[tonhuisman]

• For that reason we should remove the "s" before sRGB in all descriptions
• Renaming Normalized sRGB into "rgb"
• Calculation of RGB values according to above => R=rawR/(rawR+rawG+rawB), ...
• Add a clear note on mandatory sensor calibration with the link to ESPEasy Doc or Wiki

I'll adjust that according to your code above, this means the c value is only used for the ColorTemperature (Kelvin) calculation.

Transform all RGB values with your evaluated calibration factors (below "Formulars" can be used)

It might be less error-prone to provide the calibration values in the settings and apply that in the plugin, I'll add those.

The description on how to calibrate I'll leave to you 😄

[heinemannj]

Settings of Gain and Integration Time

• In general, you want to use the minimum Gain possible, since increasing the Gain amplifies the noise as well as the signal.
• You also want to choose an Integeration Time long enough to produce a near full-scale value, since this will give you the most effective use of the sensor's resolution.

To adjust Gain and Integration Time settings:

• Switch to RGB raw data plus the Clear channel
• Set Gain to 1
• Increase Integration Time until the Clear channel value reach 65535 (maximum value)=> Sensor is saturated
• Decrease Integration Time by one level => All channels MUST below 65535
• Control that Color Temperature value of daylight is not 0

Adjust sensor's values (precision)

• For transformations into CIE Color standards normalized RGB data with 4 digit precision seems to be strong enough

Calibrate your sensor

The sensor without any kind of calibration is completely useless!

General information

• https://learn.adafruit.com/calibrating-sensors
• https://ams.com/documents/20143/80162/ColorSensors_AN000518_1-00.pdf

Reference Measurements

Color charts/references such as the ColorChecker can be perfectly used for reference measurements.

• https://www.babelcolor.com/colorchecker-2.htm
• https://upload.wikimedia.org/wikipedia/commons/b/b4/CIE1931xy_ColorChecker_SMIL.svg

Because of its wide availability and use, its careful design, its consistency and because comprehensive spectrophotometric measurements are available, the ColorChecker has also been used in academic research into topics such as spectral imaging.

https://en.wikipedia.org/wiki/ColorChecker
https://xritephoto.com/documents/literature/en/ColorData-1p_EN.pdf

https://www.simpelfilter.de/en/colorman/monitorcalibration.html

Conditions

• Measurement under specific Light (Wavelength, Illuminance (lux), Color Temperature (CCT), Daylight, LED light, ...)
• Constant Distance between Light source <-> Object <-> Sensor
• Constant Light Reflectance - Angel
• Proper Gain and Integration Time settings

After applying calibration factors do NOT change your settings of Gain and Integration Time!

"White point" calibration - the most simple one

Good wikipedia article about color calibration ("Color balance")

• https://en.wikipedia.org/wiki/Color_balance

Illuminate a white paper with white/day light (please see above conditions) and measure sensor's RGB values:

R'w = 77.4623
G'w = 90.4528
B'w = 87.0849

Calculate calibration factors for "White Point" [RGB(255 255 255)]:

| R |		| 255/R’w		0		0	|		| R‘ |
| G |	= 	| 0			255/G’w		0	|	*	| G‘ |
| B |		| 0			0		255/B’w	|		| B‘ |

cR' = 255/R'w = 3.291924
cG' = 255/G'w = 2.81915
cB' = 255/B'w = 2.928177

but these factors obviously are NOT fitting for below RGB primaries ...

Mapping of sensor outputs for the three RGB primary colors

Measurement of Red ColorChecker Patch [Index No 15, sRGB(175 54 60)]:

R'r = 168.75
G'r = 41.25
B'r = 45.00

For Green ColorChecker Patch [Index No 14, sRGB(70 148 73)]:

R'g = 58.85
G'g = 130.77
B'g = 65.39

For Blue ColorChecker Patch [Index No 13, RGB(56 61 150)]:

R'b = 35.70
G'b = 66.30
B'b = 153.00

Transformation into sRGB color space

| R |				| R‘ |
| G |	=	| M |	*	| G‘ |
| B |				| B‘ |

Correlation constant matrix

		| R*r	R*g	R*b |			| R'r	R'g	R'b |	-1
| M |	=	| G*r	G*g	G*b |		*	| G'r	G'g	G'b |
		| B*r	B*g	B*b |			| B'r	B'g	B'b |
Where:
R*r, R*g, R*blue, G*r, G*g, G*b, and B*, B*g, B*b are the expected RGB values for red, green, and blue ColorChecker Patches.
R'r, R'g, R'b, G'r, G'g, G'b, and B'r, B'g, B'b are the responses from the sensor's RGB output channels.

		| 175	70	56	|		| 168.75	58.85	35.70	| -1
| M |	=	| 54	148	61	|	*	| 41.25		130.77	66.30	|
		| 60	73	150	|		| 45.00		65.39	153.00	|

Find the inverse Matrix: https://comnuan.com/cmnn0100f/cmnn0100f.php

		| 175	70	56	|		|  0.0067	-0.0029	-0.0003	| 
| M |	=	| 54	148	61	|	*	| -0.0014	 0.0104	-0.0042	|
		| 60	73	150	|		| -0.0014	-0.0036	 0.0084	|

		| 0.9961	0.0189	 0.1239	|
| M |	=	| 0.0692	1.163	-0.1254	|
		| 0.0898	0.0452	 0.9354	|

Additional Useful Color Equations

• http://www.brucelindbloom.com/index.html?Math.html

[TD-er]

"White point" calibration - the most simple one

Illuminate a white paper with white/day light and measure sensor's RGB values:

And there's the first wrong turn in the calibration procedure, as it highly depends on the weather and I believe even where you live, what color temperature is considered "daylight".
For example, cloudy weather does appear to have a higher color temperature (thus more blueish) compared to a sunny day.

I guess you could perhaps use the same trick with a white paper, but then point a photo camera at it to see what the camera tells you is the white balance color temperature.
Also you must make sure to read the sensor at the same angle compared to the paper as the photo camera.