Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MQTT] Controller improvements #4770

Open
wants to merge 46 commits into
base: mega
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
7f4fa7b
[MQTT] Controller improvements
tonhuisman Aug 18, 2023
7fa8319
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman Aug 26, 2023
70a99eb
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman Aug 31, 2023
1f2b98f
Merge branch 'mega' into feature/MQTT-controller-cmd-improvements
tonhuisman Sep 6, 2023
7528480
Merge branch 'feature/MQTT-controller-cmd-improvements' of https://gi…
tonhuisman Sep 6, 2023
8765c20
Merge branch 'mega' into feature/MQTT-controller-cmd-improvements
tonhuisman Sep 9, 2023
b5161be
Merge branch 'feature/MQTT-controller-cmd-improvements' of https://gi…
tonhuisman Sep 10, 2023
1839456
[P086] Format source using uncrustify
tonhuisman Sep 10, 2023
df0fb7a
[MQTT] Add support for P086 events to all MQTT Controllers
tonhuisman Sep 16, 2023
95236d3
[MQTT] Only handle plugin if plugin included
tonhuisman Sep 16, 2023
8a9dfa6
Merge branch 'mega' into feature/MQTT-controller-cmd-improvements
tonhuisman Sep 19, 2023
5a377a6
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman Sep 29, 2023
c46a8f4
Merge branch 'mega' into feature/MQTT-controller-cmd-improvements
tonhuisman Oct 4, 2023
05d99fa
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman Oct 11, 2023
7b85651
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman Oct 30, 2023
92a50d1
[C014] Use getHostname() for %sysname%, some code optimization
tonhuisman Oct 30, 2023
2de6997
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman Feb 16, 2024
16c31cf
Merge branch 'mega' into feature/MQTT-controller-cmd-improvements
tonhuisman Mar 2, 2024
4dea3a1
[C014] Process %sysname% via parseSystemVariables(), logging improvem…
tonhuisman Mar 2, 2024
175ee1e
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman Mar 24, 2024
8991cf1
Merge branch 'mega' into feature/MQTT-controller-cmd-improvements
TD-er Mar 30, 2024
d950e53
Merge branch 'feature/MQTT-controller-cmd-improvements' of https://gi…
tonhuisman Mar 30, 2024
4dea9b9
Merge branch 'mega' into feature/MQTT-controller-cmd-improvements
TD-er May 1, 2024
dc07fd9
Merge branch 'mega' into feature/MQTT-controller-cmd-improvements
tonhuisman May 8, 2024
d4c6828
Merge branch 'mega' into feature/MQTT-controller-cmd-improvements
tonhuisman May 24, 2024
05c2de5
Merge branch 'feature/MQTT-controller-cmd-improvements' of https://gi…
tonhuisman May 24, 2024
d944777
[Controllers] Minor improvements and corrections
tonhuisman May 24, 2024
4bb1d38
[C004] Some string optimization
tonhuisman May 29, 2024
4886924
[C004] Add documentation and examples
tonhuisman May 29, 2024
8dd13be
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman May 29, 2024
ec36a96
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman May 31, 2024
e3fac16
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman Jun 22, 2024
671916e
[C007] Emoncms: add configurable url (Controller Publish)
tonhuisman Jun 22, 2024
307bea9
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman Jul 5, 2024
60af4a6
[Coo4] Fix merge conflict issue
tonhuisman Jul 5, 2024
ecba48b
Merge branch 'mega' into feature/MQTT-controller-cmd-improvements
tonhuisman Jul 28, 2024
d57db07
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman Sep 25, 2024
bc7d41d
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman Sep 27, 2024
e6b2df3
Merge branch 'mega' into feature/MQTT-controller-cmd-improvements
TD-er Oct 2, 2024
7047020
Merge branch 'mega' into feature/MQTT-controller-cmd-improvements
TD-er Oct 3, 2024
1050b7f
Merge branch 'mega' into feature/MQTT-controller-cmd-improvements
TD-er Oct 7, 2024
85c1d62
Merge branch 'mega' into feature/MQTT-controller-cmd-improvements
tonhuisman Nov 2, 2024
c1013dd
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman Nov 10, 2024
3a8e46d
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman Nov 13, 2024
8ff6c04
Merge branch 'mega' of https://github.com/letscontrolit/ESPEasy into …
tonhuisman Nov 13, 2024
c6bc9f9
Merge mega and fix merge conflict
tonhuisman Dec 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 113 additions & 0 deletions docs/source/Controller/C004.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,125 @@ GitHub: |C004_github|_

Maintainer: |C004_maintainer|

Description
-----------

The ThingSpeak controller allows to send data to the ThingSpeak web service so it can be presented using the tools available there.

Both free and paid accounts are supported, where a free account has some restrictions like the number of 'Channels' that can be created, and the maximum update-frequency.

Configuration
-------------

.. image:: C004_ControllerConfiguration.png

* **Protocol**: The selected Controller Protocol.

* **Locate Controller**: This should be set to ``Use Hostname``, this will be forced when data is sent via the controller, but not saved.

* **Controller Hostname**: This should be set to ``api.thingspeak.com``, the only available API entrypoint. No url suffix should be added (at the time of writing this documentation).

* **Controller Port**: The default port number ``80`` should be used.

Controller Queue
^^^^^^^^^^^^^^^^

* **Minimum Send Interval**: The minimum time to wait before the next set of data can be sent to the controller. This can be set quite high, as the update frequency is not very high, and when sending data in too quick succession, the new data will be ignored. Especially on a free account, only a limited number of updates per day can be sent, so setting this to 600000 msec. (10 minutes) is quite realistic.

* **Max Queue Depth**: Determines the number of elements that can be stored in the queue before new samples are dismissed. When the receiving server is available, the data elements will be sent, emptying the queue, and making room for new samples again. To avoid sending too many messages at once, causing them to be discarded by ThingSpeak, the queue depth should best be set to 1. At the same time, to avoid samples to be discarded by the controller, the Interval for devices using this controller should also be set rather high, in the 20 to 30 minutes range.

* **Max Retries**: Setting is not used for this controller.

* **Full Queue Action**: The controller always uses the ``Ignore New`` strategy when the queue is filled.

* **Allow Expire**: Not applicable for this controller.

* **De-duplicate**: When enabled, avoids sending duplicate data, comparing to what's currently in the send-queue.

* **Check Reply**: The controller always uses the ``Ignore Acknowledgement`` setting.

* **Client Timeout**: The timeout to allow before the connection is failing. As ThingSpeak is an external internet service, a somewhat longer than the default timeout should be used. 1000 msec. should work in most cases, but on very high latency connections, this value can be increased.

Credentials
^^^^^^^^^^^

* **ThingHTTP Name**: This setting is currently not used for this controller.

* **API Key**: Enter the Write API Key for the channel the data should be sent to. To use multiple channels, an extra ThingSpeak controller should be configured, so a different API Key can be configured.

* **Enabled**: To enable the controller this box has to be checked.

Practical use-cases
-------------------

As ThingSpeak is a low-frequency external service, only a limited number of updates per day can be sent. For a Free account that's limited to 3 million messages per year, with a minimum interval of 15 seconds, effectively ca. 8000 updates per day. For a paid account it depends on the type of the account used, details and current price information can be found via the `ThingSpeak licensing FAQ <https://thingspeak.com/pages/license_faq>`_

Multiple sensors to a single channel
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

To send out data from multiple ESPEasy tasks (max. 4 values) to a single ThingSpeak Channel (max. 8 fields), the data has to be sent at least 15 seconds apart. To avoid losing data that is sent too soon, it's best to control the task 'activation' from rules. A simple round-robin system will do:

.. code-block:: none

On Clock#Time=All,**:** Do // Once every minute
If %v1% = 0
TaskRun,BME_280_1 // First BME, Idx 1 (field 1..3)
Elseif %v1% = 1
TaskRun,BME_280_2 // Second BME, Idx 4 (field 4..6)
Elseif %v1% = 2
TaskRun,DallasAB // 2 Dallas temperature sensors, Idx 7 (field 7..8)
Endif
Let 1,%v1%+1 // Next task
If %v1% > 2 // All done?
Let,1,0 // Reset
Endif
Endon

All above tasks, ``BME_280_1``, ``BME_280_2`` and ``DallasAB`` should be set to high Interval values, like 600 (every 10 minutes) to avoid them being run unexpectedly.

For adding more than 8 values, a new ThingSpeak Channel should be created, and because each Channel is using a different Write API Key, an extra ThingSpeak Controller should be configured, and extra tasks set up similarly but for the extra controller. ESPEasy allows up to 3 controllers to be configured, that *can* all 3 be ThingSpeak controllers, if needed.

Selected values from multiple sensors
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When sending data to a controller via the default task configuration options, all values will be sent to the controller.

If you want to send only a selection of the values to ThingSpeak, an intermediate Dummy Device should be used to store the values to be sent out, and configured to send the data to the ThingSpeak controller. Multiple sensor values can be collected in the Dummy Device, before it is triggered to send out the data.

.. code-block:: none

On BME_280_1#All Do // Single event with all values can best be enabled for the sensor
TaskValueSet,Dummy,1,%eventvalue1%
TaskValueSet,Dummy,2,%eventvalue2%
Endon

On BME_280_2#All Do
TaskValueSet,Dummy,3,%eventvalue1%
TaskValueSet,Dummy,4,%eventvalue2%
Endon

On Clock#Time=All,**:** Do // Once every minute
If %v1% = 0
TaskRun,Dummy // Dummy with BME values, Idx 1 (field 1..4)
Elseif %v1% = 1
TaskRun,DallasABCD // 4 Dallas temperature sensors, Idx 5 (field 5..8)
Endif
Let 1,%v1%+1 // Next task
If %v1% > 1 // All done?
Let,1,0 // Reset
Endif
Endon

The Interval for the BMEs can be set to 10..30 seconds, and the Interval for the Dummy task to 0, as it will be triggered from the minute timer.

Change log
----------

.. versionchanged:: 2.0
...

|added| 2024-05-29 Update documentation and add implementation examples.

|added|
Major overhaul for 2.0 release.

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
21 changes: 9 additions & 12 deletions src/_C002.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,9 @@ bool CPlugin_002(CPlugin::Function function, struct EventStruct *event, String&
proto.usesExtCreds = true;
proto.defaultPort = 1883;
proto.usesID = true;
#if FEATURE_MQTT_TLS
proto.usesTLS = true;
#endif
# if FEATURE_MQTT_TLS
proto.usesTLS = true;
# endif // if FEATURE_MQTT_TLS
break;
}

Expand Down Expand Up @@ -155,21 +155,18 @@ bool CPlugin_002(CPlugin::Function function, struct EventStruct *event, String&
}
break;
}
# if defined(USES_P088) // || defined(USES_P115)
# if defined(USES_P088)
case 88: // Send heatpump IR (P088) if IDX matches
// case 115: // Send heatpump IR (P115) if IDX matches
{
action = concat(F("heatpumpir,"), svalue1); // svalue1 is like 'gree,1,1,0,22,0,0'
break;
}
# endif // USES_P088 || USES_P115
# endif // if defined(USES_P088)
default:
break;
}

const bool validCommand = action.length() > 0;

if (validCommand) {
if (action.length() != 0) {
mustSendEvent = true;

// Try plugin and internal
Expand Down Expand Up @@ -207,19 +204,19 @@ bool CPlugin_002(CPlugin::Function function, struct EventStruct *event, String&
if (event->idx != 0)
{
String json = serializeDomoticzJson(event);
# ifndef BUILD_NO_DEBUG
# ifndef BUILD_NO_DEBUG

if (loglevelActiveFor(LOG_LEVEL_DEBUG)) {
addLogMove(LOG_LEVEL_DEBUG, concat(F("MQTT : "), json));
}
# endif // ifndef BUILD_NO_DEBUG
# endif // ifndef BUILD_NO_DEBUG

String pubname = CPlugin_002_pubname;
parseControllerVariables(pubname, event, false);

// Publish using move operator, thus pubname and json are empty after this call
success = MQTTpublish(event->ControllerIndex, event->TaskIndex, std::move(pubname), std::move(json), CPlugin_002_mqtt_retainFlag);
} // if ixd !=0
} // if idx !=0
else
{
addLog(LOG_LEVEL_ERROR, F("MQTT : IDX cannot be zero!"));
Expand Down
Loading