Improperly formatted map data (again)

[kollaesch]

Hi,

I finally wanted to get the map-integration get running in my home-assistant installation.
For now I used the valetudo from Hypfer and switched to this fork 2 days ago.
I also have the same trouble like here: #150

My setup:
Gen1 / Valetudo RE v0.10.4 / NodeJS 12.20.1

current log (mosquitto_sub -h "127.0.0.1" -t "valetudo/miepmiep/#" -p 1883 -v)

valetudo/miepmiep/map_data ^_<8B>^H^@^@^@^@^@^@^C<ED><9D>       x^U<D5>^U<80><E7>%<AC><A2>%hd3<AC>^B<D1>^@Z@E<B6>^@
<F5><E3><CD><CC>{y<B3><DE>{Ή,<82>F<B6>^@Q^A)<88><82>^T<D9>D!<80>`<D8>!^E\(T<A1> K^U<AA>`<B5>^PA<A9>(<8B>^K<95>MZ\<D2>   !<81><BC><BC>^P*͛<A1><BD>^?<BE>I<F2>f<BE>|<EF><FF><CE>=<F7>ιwf^^F^N<8C><97>b<97>K<92>O
<92><A4><B1>^U%<E9>T<8C>$<C5>H5%<FD>^UIҜ<9D>

Used Mapper:
docker-image: rand256/valetudo-mapper:latest (pulled yesterday)

Since I use both ran256-implementations I assume they should definitely work together.

2 days ago I got the original Valetudo working by disabling the base64encoding (?). (mqttInputHomeassistantMapHack?)
Then I got the map through to home-assistant. However it didn't get updates that was the reason to switch to Valetudo Re.

How can this get resolved?
Do you need other information?

Thanks in advance!

kollaesch

[rand256]

RE version sends to MQTT mapdata packets in the same untouched format they are received by valetudo from the device.

Hypfer's version unpacks mapdata packets, parses them and re-packs them again into a different format, and only then they are sent to MQTT.

Because of that you can't use mapdata from RE where it is supposed to be taken from Hypfer's and vice versa, and this can't be resolved.

[kollaesch]

Thanks a lot for the quick response!

Hypfer's version unpacks mapdata packets, parses them and re-packs them again into a different format, and only then they are sent to MQTT.

As I understood - that's the part where the/your valetudo-mapper comes into play.
And what was missing in my post was the actual error message from the valetudo-mapper.
Should I then move this to the valetudo-mapper-repo?

errorlog from the container:

> [email protected] start /app
> node app.js
Loading configuration file: /app/config.json
Connecting to MQTT Broker
Webserver running on port 3000
Connected to MQTT Broker
SyntaxError: Unexpected token � in JSON at position 0
    at JSON.parse (<anonymous>)
    at MqttClient.<anonymous> (/app/lib/MqttClient.js:91:46)
    at MqttClient.emit (events.js:315:20)
    at MqttClient._handlePublish (/app/node_modules/mqtt/lib/client.js:1277:12)
    at MqttClient._handlePacket (/app/node_modules/mqtt/lib/client.js:410:12)
    at work (/app/node_modules/mqtt/lib/client.js:321:12)
    at processTicksAndRejections (internal/process/task_queues.js:75:11)

[rand256]

The error you see is caused by the mapper tries to parse mapdata as a plain JSON. As seen here, the proper binary mapdata from RE must begin with bytes [0x1f, 0x8b, ...] - otherwise it's assumed to be JSON. So the question is whether you are completely sure that the current mapdata in MQTT is up to date value generated by RE release and not some older cached value?

[kollaesch]

I can't be 100% sure (to be honest), but I am confident that it is new data, because I let the vacuum run and this resulted in a fresh stream of data.
a) How can I make sure it's not cached from mosquitto?
b) How can I get more details about the running process on the vaccum by ssh? (valetudo is precompiled)
I had to install ntp by apt. Are there more requirements I should be aware of that can cause the problem?

Thanks.

[rand256]

Are there more requirements I should be aware of that can cause the problem?

There're none actually. Even ntp is needed only for gen3 devices to have scheduled cleaning start at proper time.
I have no idea why you get wrong mapdata from your broker. Which mosquitto version do you use?

[kollaesch]

Which mosquitto version do you use?

mosquitto 2.0.7-1 (archlinux)
I could check with an alternative in the meantime. Could this somehow be an encoding issue?

LC_ALL=de_DE.UTF-8
LC_COLLATE=C

[Syntaxrabbit]

Same issue here, brand new Roborock S6 with Valetudo RE 0.10.5 and valetudo-mapper 0.2.0.
MQTT cache is empty. Where do we start? :-)
Mosquitto version is a bit older... 1.6.4

[rand256]

Same issue here, brand new Roborock S6 with Valetudo RE 0.10.5 and valetudo-mapper 0.2.0.
MQTT cache is empty. Where do we start? :-)

You start by using valetudo-mapper current master - not years outdated 0.2.0 release.

[Syntaxrabbit]

Thanks a lot. I did not recognize that the released zip file was that old.

OK, so now I get stuck at this:

# npm run start

> [email protected] start /opt/openhab/tools/valetudo-mapper
> node app.js

Loading configuration file: /opt/openhab/tools/valetudo-mapper/config.json
Connecting to MQTT Broker
Webserver running on port 5444
Connected to MQTT Broker
/opt/openhab/tools/valetudo-mapper/lib/Tools.js:195
                drawLineByPoints(image,options.parsedMapData.path.points.map(point => {
                                                                  ^

TypeError: Cannot read property 'points' of undefined
    at Jimp.<anonymous> (/opt/openhab/tools/valetudo-mapper/lib/Tools.js:195:67)
    at Timeout._onTimeout (/opt/openhab/tools/valetudo-mapper/node_modules/@jimp/core/dist/index.js:264:25)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: `node app.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2021-03-18T15_43_02_115Z-debug.log

The log file does not seem to offer additional information.
(I run as root to be sure that permission problems are not the cause.)