Home

General Notes

This is a firmware modification based on the great RGBWWLed firmware from Patrick Jahns (https://github.com/patrickjahns/esp_rgbww_firmware). Big thanks to Patrick for his work.

There is also an updated FHEM module 32_LedController.pm to utilize the new functions: https://github.com/verybadsoldier/esp_rgbww_fhemmodule

Modifications

This is a (probably incomplete) list of changes, bugfixes and new features.

• Updated to Sming 3.5
• Designed for ESP8266 ESP-12F but other chips are also supported by customizable pin configuration
• MQTT support
• Device sychronization between multiple devices (includes accurate clock synchronization)
• TCP-Server for JSON realtime events (e.g. color status message and animation finished events)
• Relative commands (+/- xxx)
• Command requeuing (enabling animation loops)
• Pausing and continuing of animations
• Independent color channels (e.g. send command to hue channel without affecting other channels)
• Multiple commands in a single request
• Different queue policies for animation commands
• Instant blink commands
• Ramp speed - ramp timing can be specified as ramp speed instead of just ramp time
• Fixed inaccurate ramp timings
• Improved saving of current color
• more...

Installation

There are different flavors of this firmware available. All flavors can be updated using an OTA update URL.

Available flavors:

Stable

http://rgbww.dronezone.de/release/version.json

Testing

http://rgbww.dronezone.de/testing/version.json

Be aware that this is currently not a secure HTTPS connection and MITM attacks are possible!

Description of Modifications

---

MQTT

The device can connect to an MQTT broker to receive and publish messages. MQTT can be enabled using the config setting network/mqtt-enabled. The MQTT broker can be configured using the settings:

network/mqtt-server
network/mqtt-port
network/mqtt-username
network/mqtt-password

MQTT is currently used to synchronize multiple devices (see next chapter for details) and to publish events indicating finished animations.

When the controller publishes messages to the broker then the used topic always begins with a configurable prefix followed by the device name. It will have this format:

<prefix>/<device_name>/<message_specific_topic>

The prefix can be configured using the configuration setting network/mqtt-topic_base. device_name is configured in general/device_name.

Example:

If the topic_prefix is set to myHome/kitchen and device name is led1 then the topic used to publish color commands would be: myHome/kitchen/led1/command

Device Synchronization / Slave Mode

One or more devices can be synchronized in different ways. You should not enable more than one color synchronization mode at once as it might lead to undesired results. These mode are available:

• Command Synchronization: all color commands the master receives will be published to the MQTT broker. Any other device can subscribe to the command topic of the master and execute the same command. This will keep the slaves in sync.
• Direct Color Synchronization: the master publishes its current color values to the MQTT broker every time the color changes. Any other device can subscribe to that color topic and set itself to the same color.
• In case you are using a server-based solution to control the devices then you can just send all color commands to all devices. This is similar to the MQTT-based command synchronization but has lower command latency and does not require an MQTT broker. For FHEM look here: https://github.com/verybadsoldier/esp_rgbww_fhemmodule/wiki#slave-sync

So what is the recommended way to use?

The best results can currently be achieved by using the command synchronization with MQTT in combination with clock synchroniazion. Direct color commands should be disabled.

Command Synchronization

For this operation mode one of the devices has to be configured as the command master. To do this the config setting sync/cmd_master_enabled has to be set to true. The device will then publish all received color commands to the topic <topic_prefix>/<device_name>/command.

All devices meant to be slaves have to be configured to be command slaves. To do this the config setting sync/cmd_slave_enabled has to be set to true. Also the topic to listen for commands has to be configured for the slaves. This can be done by setting sync/cmd_slave_topic.

You will need to restart both master and slaves devices after changing those settings!

For long-running fades you might experience color differences since master and slave devices are executing color fades on their own and might get out-of-sync due to drifting internal clocks. To fix this please follow the section Clock Synchronization.

Example:

Let's say we have three devices myLed1, myLed2 and myLed3. We want myLed2 and myLed3 to be slaves of the master myLed1. Our example topic_base is home/.

So on myLed1 we set sync/cmd_master_enabled to true. The master will now publish commands to the topic home/myLed1/command. For both myLed2 and myLed3 we set sync/cmd_slave_enabled to true. And we set sync/cmd_slave_topic to home/myLed1/command.

After rebooting all devices should be in sync if send to commands to master.

Clock Synchronization

All devices have their own internal clocks that do not run all exactly on the same speed. When using command-based device synchronization this might lead to slightly differently executed fades caused by different timings. Especially for long-running fades this can be a problem. To compensate for this it is possible to synchronize the clocks of multiple devices. This will make sure that all devices will execute the (nearly) exact same fade timings. To use this one device has to be "clock master". The device will publish its current clock in regular intervals. Other devices then can be "clock slaves" by receiving the clock signal of the master and modify their clocks accordingly to match the clock of the master.

To make one device the "clock master" the setting sync/color_master_enabled has to be set to true. Clock message will then be published regularly to the topic <topic_base>/<device_name>/clock. The publishing interval can be configured by the setting sync/clock_master_interval. The default value 30 should be fine in normal use though.

For all devices that should be clock slaves the setting sync/color_slave_enabled has to be set to true. In addition the MQTT topic for receiving clock messages has to be configured using the setting sync/clock_slave_topic.

To check if clock synchronization is working correctly on each received clock message on the slave a clock_slave_status message will be published. This message will contain the current step offset to the master and the current clock interval in microseconds.

A clock_slave_status message from the event server looks like this:

{"jsonrpc":"2.0","method":"clock_slave_status","params":{"offset":1,"current_interval":19941},"id":948}

Example:

Let's say we have three devices myLed1, myLed2 and myLed3. We want myLed2 and myLed3 to be clock slaves of the master myLed1. Our example topic_base is home/.

So on myLed1 we set sync/clock_master_enabled to true. The master will now publish clock messages to the topic home/myLed1/clock. For both myLed2 and myLed3 we set sync/clock_slave_enabled to true. And we set sync/clock_slave_topic to home/myLed1/clock.

Direct Color Sychronization

Another method to synchronize devices is to use direct color synchronization. With this option the master will publish regularly (every 20 ms if you want) its current color state. Slave devices receiving those message set their own color to the same value.

The settings for using this works quite exactly in the same way as the settings for clock synchronization. So just read that chaper for reference. The config settings involved for direct color synchronization are:

sync/color_master_enabled
sync/color_master_interval_ms
sync/color_slave_enabled
sync/color_slave_topic

While testing this I personally had not much success in getting smooth and reliable color fades. So I cannot recommend this method. But feel free to test it and play around with it! Feedback is appreciated.

Independent Animation Channels

Each channel (H,S,V,CT, R,G,V,WW,CW) is split up into completely independent entities having their own animation queues. So while an animation is running on one channel the queues of the other channels can be altered without affecting the running animation. So for example you can issue a hue fade which is running for some time. While it is running you can decide to turn down the brightness (also with a fade if you wish) without affecting the hue animation.

The downside is that the controller now is always in a color mode which is either HSW or RAW. Only the queues of the active color mode will be processed at a time. So in HSV mode only H, S, V and CT is processed, when in RAW then only R, G, B, CW and WW is processed. So the currently visible color will either be calculated from the HSV channels or from the RAW channels. But is not possible to mix animations based on different modes. When issuing a color command the controller will immediately switch to the respective color mode. The queues of the inactive mode will not be affected.

Extended Color Commands

In all color commands (HSV and RGB) it is allowed to omit channels that should not be affected by the command. Those channels will be ignored and they will just continue processing their current animation queue.

Relative Color Values

Instead of using absolute values for color targets also relative values can be used. Relative values start with the characters + or -. Hue values will be mapped to a circle. When sending a hue value of +40 when the current hue 340will result in a target hue of 20. Saturation and brightness values will be capped to values of 0to 100.

Example:

{ "hsv" : { "h" : "+45", "v" : 70 } }

This would increase the current hue value by +45 and set the brightness to 70. The saturation value won't be affected.

JSON-RPC Event Server

A TCP server running on port 9090 publishes event messages to all connected clients (max. 4 clients). All messages have an ID which is an integer incremented with every message.

The server can be enabled/disabled by using the configuration setting events/server_enabled.

Color Status Message

Every 500 ms a color status message will be sent (only if the color status has changed since the last message). The interval can be configured by config setting events/color_interval_ms. A value of -1 disables color status messages completely. The color status message will also be sent if an animation on any channel finished (independently of events/color_interval_ms). This is done to get final channel states as soon as possible. The config setting events/color_mininterval_ms can be used to limit the rate of color status messages globablly (otherwise very-short animations could trigger messages faster than events/color_interval_ms).

The field mode will show the mode the controller is currently in. It can be raw or hsv. When in mode hsv then there will be a field hsv in the message that contains the HSV values.

Example:

{"jsonrpc":"2.0","method":"color_event","params":{"mode":"hsv","raw":{"r":0,"g":638,"b":442,"ww":379,"cw":6},"hsv":{"h":161.58,"s":62.37,"v":100.00,"ct":2746}},"id":63}

Animation Finished Message

Every time a named animation (see Named Animations) finishes a notification event will be sent. The name will be included in the message. The parameter requeued states if the animation has been requeued to the queue. Usually this is false and means that the animation finished and got removed from the queue. For animations with requeue flag this value can also be true which means that the animation has been finished once but has been requeued to the queue. The rate of this message can be limited using the config setting events/transfin_interval_ms. A value of -1 will disable this limit.

Example:

{"jsonrpc":"2.0","method":"transition_finished","params":{"name":"myFade", "requeued":true},"id":1356}

Keep-Alive Messages

Every 60 seconds the controller will send a keep-alive notification indicating that it is up and running.

Example:

{"jsonrpc":"2.0","method":"keep_alive","params":{},"id":1163}

Requeue Flag

There is a special flag called requeue which can be assigned to animations. When an animation flagged by requeue finishes then it will immediately re-added to the back of the queue resulting in a looped animation queue. There can be several queue entries flagged with requeue which will result in different animations being looped. In the JSON message a parameter named r is used to indicate a color command meant for requeueing.

Example:

{
  "hsv":{"ct":2700,"h":"+40"},
  "t":2000,
  "cmd":"fade",
  "r":"true"
}

Named Animations

When using the parameter name in a JSON message a user-defined name can be assigned to an animation. This named will be included in Animation Finished Message. This is useful for detecting if particular animations are finished.

{
  "hsv":{"ct":2700,"v":"100"},
  "d":1,
  "t":2000,
  "cmd":"fade",
  "name":"makebright"
}

Multiple Commands in a Request

Usually there is one command in a HTTP-POST request which looks for example like this (single JSON object):

{ "cmds" : [
{
  "hsv":{"ct":2700,"h":"150"},
  "d":1,
  "t":10000,
  "cmd":"fade"
},
{
  "hsv":{"ct":2700,"h":"20"},
  "d":1,
  "t":10000,
  "cmd":"fade",
  "q":"back"
}
]
}

Instead of this also a JSON array can be sent to issue multiple commands in a single HTTP-Post request:

[{
  "raw":{"g":"550", "r" : 0, "cw" : 0},
  "t":20000,
  "cmd":"fade",
  "q" : "back"
},
{
  "raw":{"g":"850", "r" : 1000, "cw" : 0},
  "t":20000,
  "cmd":"fade",
  "q" : "back"
}]

Channel Animation Control

There are some new commands available to control processing of animation channels. These commands are available:

• pause: Freezes the current animation immediately by stopping the processing of the current animation. The content of the animation queue is not affected and will continue to run on the next color command which uses a queue policy different from back. Also command continue can be used to continue processing.
• continue: This will continue the processing of a paused animation
• stop: Stops the current animation immediately and also wipes the queue completely
• skip: The currently running animation will be skipped by removing it from the queue. The next animation in the queue will immediately start to process.

All these commands can be parameterized by a parameter channel which is a JSON array of one or more channel abbreviations (h, s, v, r, g, b, cw and ww). If this parameter is passed then the command is only executed for individual channels. If no channel parameters are passed (or the array is empty) then the command is executed for all channels:

Examples

{
  "channels":["s", "v"]
}

The channel command will only be executed on the hue and the brightness channel. Saturation channel will not be affected.

{channels:[]}

Despite there is a channel parameter this command will affect all channels.

Ramp Speed

Usually ramps are issued using a fixed ramp time which specifies a timespan in which the ramp will be executed:

"t":1000

When using parameter s instead of t it is also possible to specify a fixed ramp speed when issuing a fade command:

"s":536.7

When using a ramp speed then the time the fade will need to execute will differ depending on the starting value. The speed is defined as percentage points (or hue degrees) per minute.

Example:

{  
   "q":"single",
   "hsv":{  
      "v":"100"
   },
   "s":"7",
   "cmd":"fade"
}

This will issue a fade from the current brightness to a value of 100 with a speed of 7 percentage points per minute. So for example if the current brightness is 65 then the fade will take 5 minutes to reach the target value of 100.

Blink Command

This command can be used to let one or more channels blink. Blink is a regular animation as a fade or as a set command. Therefore the same flags are supported as for the color commands (queue, requeue etc.). Additionally with the parameter channels the command can be parameterized to blink on one or more channels. Without this parameter in HSV mode the brightness channel will be used and in RAW mode both warm white and cold white channels will be used. The parameter tcan be used to specify the blink time in ms.

Example:

{
  "channels":["s", "v"],
  "t":1500
}

Blinks for 1,5 s on channel saturation and brightness.

Queue Policies

There are now 4 queue policies available. The queue policy decides about how the issued animation will be inserted into the queue of the affected channel.

• single: the current queue will be cleared and replaced with the provided animation
• back: the animation will be appended to the queue and therefore executed after all other queue entries are finished
• front: the animation will be inserted at the front of the queue. This means it will be executed immediately. The current animation will continue to execute right after the new animation finished at the exact same position it had previously.
• front_reset: Same as front but the current animation will be reset which means that it will start from the beginning taking the new start position into account.

Example:

{
  "hsv":{"h":20},
  "d":1,
  "t":20000,
  "cmd":"fade",
  "q" : "front"
}

Output Pin Configuration

The firmware was initially designed to run on the PCB designed by mrpj (project founder) on an ESP-12F. The default output pin configuration is designed accordingly. However it is possible to use the firmware for other ESP-based boards as well (for example Wemos D1 mini based).

Using another board might need a different output pin configuration. This can be done by modifying the config parameter pin_config.

The value is a comma-seperated string of five numbers. Each number is indicating a pin number used for a specific output channel. The order is red, green, blue, warmwhite, coldwhite. A custom string suitable for any custom pin configuration can be entered as needed.

Example pin configurations:

Chip Type	Pin Configuration String
mrpj PCB (ESP-12 E/F)	13,12,14,5,4
Shojo PCB (WemosD1 mini)	14,4,5,15,12

Startup Color (>= vbs33)

The initial color shown after boot can be configured using the config parameter color/startup_color. When the value is last then the the controller will regularly save the current color to the flash and will restore this last seen color upon boot.

Otherwise the value has to be a color string matching the format <hue>,<sat>,<val> or <hue>,<sat>,<val>,<ct>. This will result in a fixed color which is shown on every boot. In this mode no color states will be written to flash at runtime.

WARNING for firmwares vbs33b to vbs35b When supplying a malformed color string then it will crash the controller resulting in an infinite boot loop which needs to be fixed by a serial flash. So make sure to only supply a color string in the format described above!

Secured API (>= 4.1.0)

The JSON API can be secured by HTTP basic authentication (https://tools.ietf.org/html/rfc2617). When enabled then only HTTP requests providing a valid authentication header will be accepted. The username provided in requests is ignored, only the password gets validated.

The security option can be enabled/disabled using config-security-api_secured. The password can be set using config-security-api_password. NOTE: When enabling the option then in the same request also config-security-api_password has to be included. config-security-api_password can never be set separately. I can only be set within a request that also enables the security option.

Change History

4.1.0-rc1

• Feature: Enable secured API option
• Feature: Sming updated to 3.8.0

4.0.0-rc4

• Bugfix: actually use config-color-colortemp-cw and config-color-colortemp-ww as range for color temperature (only hardcore values 2700 and 6000 were used)

4.0.0-rc2

• Bugifx: switch back to regular PCM implementation due to subtle flickering when fading

...

-vbs30:

• Feature: webapp updated to shojo6 (thanks!)
• Feature: Added "uptime" in info message
• Improvement: Better handling of clock syncs when clock master reboots
• Improvement: Removed custom version string. Solely use Git version string now for consistent versioning
• Improvement: improved stability
• Improvement: compiled with SMING Release settings (no serial debug output)
• Bugfix: Fixed webclient crash when SSIDs with non-ASCII characters appear (SSID will be ignored!)
• Bugfix: Fixed MQTT clock sync drift

-vbs17 (withdrawn):

• Bugfix: Fix crashes on parallel HTTP requests -vbs16:
• Feature: Updated to Sming 3.5
• Bugfix: TCP connection handling. Added some elements to JSON info structure
• Bugfix: Fixed HTTP redirection to /webapp
• Feature: version tag to settings file

-vbs12:

• Bugfix: Config parameter mqtt/topic-base not being loaded from config

-vbs11:

• Bugfix: ap-password was missing in configuration API

-vbs10:

• Added output pin configuration

-vbs9

• MQTT: Fixed username/password

-vbs7

• EventServer: replaced event hsv_event and raw_event by color_event. So raw values will always be available

-vbs6

• fixed inverted CW <-> WW channels

-vbs5b

• added option to pass a ramp speed instead of a ramp time
• again improved synchronization accuracy
• fixed crash in configuration handling after upgrade from the original firmware
• avoid unneccesary color savings to file system

---

-vbs4b

• improved clock synchronization
• added publish message to display slave clock sync status
• fixed bug that showed transition finished messages multiple times

---

-vbs2b

• device synchronization by commands (MQTT)
• device synchronization by direct color (MQTT)
• clock synchronization (MQTT)

---

-vbs1b

• seperated channels featuring independently controllable and animated queues
• JSON-RPC event server notfiying about state changes via TCP (like current color and fade events)
• flag "requeue" for animations to allow for custom designed animation loops
• fades can be assigned a name which is used in fade events
• relative values are allowed in color commands (e.g. +10 for hue)
• color commands can modify only selected channels (e.g. modifying saturation animation without touching hue animation)
• possibility to send multiple commands in a single HTTP request (for efficiency)
• new commands to "pause", "continue" and "stop" fades (also for individual channels)
• command "skip" to skip current animation and jump to next animation (also for individual channels)
• command "blink" to issue configurable instant blink actions (also for individual channels)
• new queue modes "front" and "front_reset" to insert animations at the front of the queue
• updated to Sming 3.1.2
• probably some more...