yasdi2mqtt as interface between SB4000TL-20 and Home-Assistant

[aufwindflieger]

Hi Paul,
is it possible to use yasdi2mqtt as an interface between my SB4000TL-20 (provides RS485 interface but not the newer modbus protocoll) and Home-Assistant tool?
I use Home-Assistant with the Home Assistant Operating System on a Raspberry Pi 3b.
I am not familiar with Linux, therefor maybe some stupid questions:
How would the data transfer between yasdi2mqtt and Home-Assistant work (via Ethernet)?
Would it be possible to run yasdi2mqtt and Home-Assistant on the same Raspberry Pi, if I use the Home Assistant container-based installation (e.g. Docker)?
If yasdi2mqtt runs on its own Raspberry Pi, what would be the smallest possible Hardware?
Thank you for sharing your SW and for an answer!
Eberhard

[dalklein]

Yes, that would probably work, Go for it!

You'll need an rs485 piggyback board for the inverter if it does not already have the actual rs485 output, then need an rs485-usb converter to go to the RPi.

For the PV inverters there maybe be a home assistant integration already, look around or ask in home-assistant forums.

I would start by making sure you can communicate over rs485 with the inverter, to the RPi. Install yasdi and see if yasdi shell works to see the data. Then set up yasdi2mqtt, and from there HA can get the data over mqtt (all within the same RPi if you want, or mqtt over your network.)

I am using yasdi / yasdi2mqtt to monitor a pair of SI6048US battery inverters, on the same RPI as I have IOTstack running with nodered / grafana / influx. The RPI controls a number loads in the house to manage available solar and charge & discharge separate battery banks, to mostly keep the main FLA SI battery floating full. It all works! Very Fun!

Here are my notes from setting mine up. Not a complete guide, but might be helpful.

11/9 installed rs485-usb adapter to SMA master inverter. seems to be detected, following FTDI driver
install guide for linux,
$ dmesg | grep FTDI
[ 6.035055] usb 1-1.4: Manufacturer: FTDI
[ 10.911385] usbserial: USB Serial support registered for FTDI USB Serial Device
[ 10.911713] ftdi_sio 1-1.4:1.0: FTDI USB Serial Device converter detected
[ 10.916067] usb 1-1.4: FTDI USB Serial Device converter now attached to ttyUSB0
hmm, what next...

11/13 tested yasdi, and mqtt2yasdi on the second RPi, it works and publishes to mqtt!
see if we can do the same on the SMA_RPi

 yasdi source was already downloaded sometime in the past, to /home/pi/yasdi    v. 1.8.1 maybe
 compile and install it: (~following yasdi setup  https://github.com/ardexa/sma-rs485-inverters)
$  sudo apt-get update
$  sudo apt-get install -y build-essential git cmake
$ cd yasdi
$ cd projects/generic-cmake
$ mkdir build-gcc
$ cd build-gcc
$ cmake ..
$ make
$  sudo make install     
$  sudo ldconfig
$ cd
$ nano yasdi.conf     (note, no leading whitespace, else yasdi fails)

[DriverModules]
Driver0=yasdi_drv_serial

[COM1]
Device=/dev/ttyUSB0 (ttyS0 should be the CAN-usb adapter, ttyUSB0 should be the 485-usb)
Media=RS485
Baudrate=1200
Protocol=SMANet

  plug in the rs485 usb to the RPi.   yasdishell runs, and detects SI, but freezes after that.

 install yasdi2mqtt
 $ git clone https://github.com/pkwagner/yasdi2mqtt.git
 $ cd yasdi2mqtt
 $ mkdir devices
 $ cd ..
 $ nano docker-compose.yml
        MQTT_TOPIC_PREFIX: "SI6048"
        MQTT_SERVER: 192.168.15.15
        MQTT_PORT: 1883
        MQTT_QOS_LEVEL: 0
        MQTT_USER: "mqtt_user"
        MQTT_PASSWORD: "mqtt_user_pw"
        LOG_LEVEL: 1
 $ docker-compose up          should download docker image (first time) and start running, with
       debug output in the command window.
 
- in portainer (192.168.15.15:9000), set restart policy to 'unless stopped', then start manually.  this should make it
run automatically on reboot.
- edited yasdi2mqtt/docker-compose.yml to set LOG_LEVEL: 0      (don't possibly fill something up
  with log messages.  I'll know if it stops working.)

reboot RPi and see if it starts and works... ... ...     yes, wow
changed update time to 10sec        seems to work, watching mqtt messages at least.  
RPi load (top) looks ok.

11/17 working fine, more nodered and dashboard changes. variable ET micros discharge control works
$ sudo rpi-clone sdb -s SMA-clone_111721 clone to USB card reader SD card, for a backup of the whole RPi

11/30 pi@hassbian:~/yasdi2mqtt $ yasdishell
(shell apparently works while yasdi2mqtt is also running)

[aufwindflieger]

OK, sounds good!

Regarding home assistant integrations for SMA inverters, my understaning after long search in the forum is, that direct SMA integration is only possible for devices with web interface (see https://www.home-assistant.io/integrations/sma/). The second possibility is integration via modus by serial (rs-485), TCP, or UDP connection. Since my 10 year old inverter doesn't provide modbus, this will not work. Third possibility might be Bluetooth, but with a highly indirect way via SBFspot (https://github.com/SBFspot/SBFspot).

Hardware is already there: a 485 Datenmodul (485I-MOD-G1 BGCB) and a RS485 to USB converter, which is detected as /dev/ttyUSB0.

When you say " Install yasdi", do you mean this software here: https://github.com/ardexa/sma-rs485-inverters ?
If yes, together with the Ardexa software?
If not, what else?
Thanks!

[dalklein]

I think you want yasdi from one of these links:
https://github.com/konstantinblaesi/yasdi
https://www.sma.de/en/products/apps-software/yasdi.html

And following the instructions at the ardexa projects for installing yasdi may be useful. Or that whole project, if you want to use that ardexa service or database or whatever it is. For me, (and the reason that yasdi2mqtt exists!), I want to get the data from the SMA inverters (which the yasdi library handles decoding) and publish to mqtt, then various things (including HA) can receive the data from MQTT messages anywhere on your network to do with as you like.

[aufwindflieger]

Worked like a charm!!
I mostly followed the instructions at the ardexa project including the added "_" in file packet.h, whatever this will do.
Here the output of the "Show spot channel values" command:

Device handle: 1
Device 'WRTL14CD SN:2100521179' has 23 Spot channels:
Reading channel values, please wait...
------------------------------------------------------------
Channel handle |    Channel name    | Channel value (Unit) |
------------------------------------------------------------
     358       | '        A.Ms.Amp' | '1.638' (A) 
     359       | '        A.Ms.Vol' | '341.430' (V) 
     360       | '       A.Ms.Watt' | '559.000' (W) 
     362       | '        B.Ms.Amp' | '1.234' (A) 
     363       | '        B.Ms.Vol' | '179.360' (V) 
     364       | '       B.Ms.Watt' | '220.000' (W) 
     365       | '             Pac' | '774.000' (W) 
     366       | '       GridMs.Hz' | '49.960' (Hz) 
     372       | ' GridMs.PhV.phsA' | '235.630' (V) 
     373       | ' GridMs.PhV.phsB' | '0.000' (V) 
     374       | ' GridMs.PhV.phsC' | '0.000' (V) 
     385       | '         E-Total' | '41762.925' (kWh) 
     389       | '     Mt.TotOpTmh' | '42430.746' (h) 
     390       | '       Mt.TotTmh' | '43595.608' (h) 
     393       | '       Op.TmsRmg' | '0.000' (s) 
     394       | '   Serial Number' | '2100521179.000'    
     395       | '    Op.EvtCntUsr' | '3981.000'    
     399       | '        Op.EvtNo' | '0.000'    
     506       | '            Mode' | 'MPP'    
     507       | '           Error' | '-------'    
     529       | '     Op.GriSwStt' | 'Cls'    
     530       | '   Inv.TmpLimStt' | 'None'    
     546       | '       MainModel' | 'Solar-WR'

As next step I would install a Mosquitto MQTT broker on my Home Assistant (HA) Raspberry Pi (For the time beeing I will run yasdi2mqtt and Home Assistant on seperate Raspberry Pi's).
Does the above mentioned content of docker-compose.yml fit to this approach?
I would assume the following content of the labels:

MQTT_TOPIC_PREFIX: "myNameOfInverter"  
MQTT_SERVER: https://mySubdomain.duckdns.org  
MQTT_PORT: 8123 (this is the forwarded port for HA)  
MQTT_USER: "userOfMosquittoBrokerInHA"  
MQTT_PASSWORD: "passwordOfMosquittoBrokerInHA"

Is my understaning correct?

[dalklein]

Congrats, pretty cool to see your data, eh?

If mosquitto is installed on your RPi, then it is the mqtt broker. It receives and relays all the mqtt messages. Then it would be the local ip address of your RPi, and maybe port 1883.

If you want to use an mqtt broker in the cloud (some people do), then it might be similar what you listed (I'm not sure, read HA setup info for it). Or, actually, if you've set up the RPi HA for remote access, then the RPi's ip address might be as you listed (getting it from duckdns). So yes, that might be right for that configuration.

[aufwindflieger]

Yes, really cool!!
You're right the port of the mqtt broker will be 1883 and this should work w/o port forwarding, since the messages from one Pi to the other will never leave the local network even with the duckdns address (yes, remote access is established for the sake of the mobile app of HA and external and internal address is the same.
Keep you updated - (I'm a little afraid of the docker stuff).

[aufwindflieger]

Command docker-compose up in path /home/pi/yasdi2mqtt where docker-compose.yml is, fires error:
bash: docker-compose: command not found

[aufwindflieger]

OK OK, after some googling, I followed this guide https://dev.to/elalemanyo/how-to-install-docker-and-docker-compose-on-raspberry-pi-1mo and installed docker and docker-compose plus the "Docker system service to start your containers on boot" thing.
After reboot and succesfull docker-compose, I had to change the https://mySubdomain.duckdns.org to the IP address of the RPi. I think, although the Mosquitto MQTT broker is integrated into HA, its data entry is not fed through HA. What should I say - success:

Message 8 received on solar/SB4000TL20/2100521179 at 09:09:

{
    "sn": 2100521179,
    "time": 1644912539,
    "values": {
        "A.Ms.Amp": 0.4930000234162435,
        "A.Ms.Vol": 363.0699918847531,
        "A.Ms.Watt": 178,
        "B.Ms.Amp": 0.41200001956894994,
        "B.Ms.Vol": 181.9899959322065,
        "B.Ms.Watt": 74,
        "Pac": 244,
        "GridMs.Hz": 49.99999888241291,
        "GridMs.PhV.phsA": 233.4999947808683,
        "GridMs.PhV.phsB": 0,
        "GridMs.PhV.phsC": 0,
        "E-Total": 41766.055983781116,
        "Mt.TotOpTmh": 42438.93208324094,
        "Mt.TotTmh": 43603.99742670983,
        "Op.TmsRmg": 0,
        "Serial Number": 2100521179,
        "Op.EvtCntUsr": 3982,
        "Op.EvtNo": 0,
        "Mode": "MPP",
        "Error": "-------",
        "Op.GriSwStt": "Cls",
        "Inv.TmpLimStt": "None",
        "MainModel": "Solar-WR"
    }
}

QoS: 0 - Retain: false 

Interpreting of the data will be some diligence work and then I can start optimizing the use of self generated electricity.

Viele Grüße und besten Dank für die Unterstützung von Vaihingen/Enz nach Zürich
Eberhard

[aufwindflieger]

One more question. Could you please explain, how to do this:

• in portainer (192.168.15.15:9000), set restart policy to 'unless stopped', then start manually. this should make it
  run automatically on reboot.

Do I have to stop the container (?) and then?

[dalklein]

If you have portainer installed on the RPi, go to it's gui (your RPi ip :9000) , select the yasdi2mqtt container and look for restart policy setting. If you are not using portainer, there's probably a docker setting or option somehow to accomplish the same thing. I'm not sure really, if you need it set that way.

I set it that way, in the hope that it will always be running, ie: it will restart if it gets crashed or when I reboot the Pi. My RPi may have too many things going on. In addition to the containers listed, there are several python scripts running to gather data from at least 3 other solar related devices and publish to mqtt, and nodered is doing a lot also. Sometimes the Pi will hang, seems to be if I attempt too look at too long a time period of data with grafana from influxdb. But it works well enough that I haven't debugged it further.

List from portainer:

Name	State Filter	Quick actions	Stack	Image	Created	Published Ports	Ownership
nodered	healthy		iotstack	iotstack_nodered	2021-09-23 12:23:27	1880:1880	administrators
yasdi2mqtt_yasdi2mqtt_1	running		yasdi2mqtt	pkwagner/yasdi2mqtt:latest	2021-11-13 15:12:06	-	administrators
python	running		iotstack	iotstack_python	2021-09-23 12:23:27	-	administrators
classic_mqtt	running		iotstack	classicdiy/classicmqtt	2021-09-23 12:23:27	-	administrators
influxdb	running		iotstack	influxdb:1.8	2021-09-23 12:23:27	2003:2003  8083:8083  8086:8086	administrators
grafana	running		iotstack	grafana/grafana	2021-09-23 12:23:27	3000:3000	administrators
mosquitto	running		iotstack	eclipse-mosquitto	2021-09-23 12:23:27	1883:1883	administrators
portainer-ce	running		iotstack	portainer/portainer-ce	2021-09-23 12:23:26	8000:8000  9000:9000	administrators

[aufwindflieger]

restart: always (https://serverfault.com/questions/884759/how-does-restart-always-policy-work-in-docker-compose) for every service did the job for me.

version: "3"
services:
    yasdi2mqtt:
        # You may also check out :stable release or `build: "."`, if :latest fails
        image: "pkwagner/yasdi2mqtt:latest"
        restart: always
        volumes:
            - "./devices:/etc/yasdi2mqtt/devices"
            - "./yasdi.ini:/etc/yasdi2mqtt/yasdi.ini:ro"
        devices: 
            - "/dev/ttyUSB0:/dev/ttyUSB0"
        environment:
            YASDI_CONFIG: "/etc/yasdi2mqtt/yasdi.ini"
            YASDI_DRIVER_ID: 0
            YASDI_MAX_DEVICE_COUNT: 1
            YASDI_UPDATE_INTERVAL: 10
            MQTT_TOPIC_PREFIX: "solar/SB4000TL20"
            MQTT_SERVER: "192.168.2.100"
            MQTT_PORT: 1883
            MQTT_QOS_LEVEL: 0
            MQTT_USER: "my-user"
            MQTT_PASSWORD: "whatsoever"
            LOG_LEVEL: 0

[Maxmat99]

hi there! iam reading this and wondering if it would help me! what id like to do is this ........
sma sunny boy 4000tl20 to pi then pi modbus to enable venus pi to pic up and display on victron device.....
i have a webbox connected to inverter , however for some reason the victron cannot see the webbox..
i receives pings etc on telnet i cant get past the loggin password mallarchy!
i am nott a programmer however i can follow instructions!
any help would be appreciated!
matt

[pkwagner]

Thanks @dalklein for handling the request! I'm glad everything works out as expected :)

PS: I don't know how applicable it is for your use case (your parameters look quite different), but I've already posted a parameter legend in a similar thread.