MF-28 - enabled messages paho persistence #27
Conversation
pricelessrabbit
force-pushed
the
paho-file-storage
branch
3 times, most recently
from
cf42858 to
72c98c0
Compare
* enabled persistent session to avoid paho to discard pending message if service restarted * updated readme Signed-off-by: PricelessRabbit <[email protected]>
pricelessrabbit
force-pushed
the
paho-file-storage
branch
from
72c98c0 to
917a81b
Compare
mteodor
changed the title
pkg/export/service.go
Outdated
| @@ -162,11 +162,16 @@ func (e *exporter) mqttConnect(conf config.Config, logger logger.Logger) (mqtt.C | |||
| opts := mqtt.NewClientOptions(). | |||
| AddBroker(conf.MQTT.Host). | |||
| SetClientID(e.id). | |||
| SetCleanSession(true). | |||
| SetCleanSession(false). | |||
There was a problem hiding this comment.
I think this should be optional, so that when started for the first time it can start with the new session. Should be added to ENV vars.
There was a problem hiding this comment.
added variable to the env in new commit
- automatically put clean session to false if file message persistence enabled - updated doc Signed-off-by: PricelessRabbit <[email protected]>
pkg/export/service.go
Outdated
| if conf.MQTT.Persist { | ||
| store := mqtt.NewFileStore(conf.MQTT.PersistDir) | ||
| opts.SetStore(store) | ||
| //disable clean session because paho deletes stored messages when restarts |
There was a problem hiding this comment.
Comments start with the capital letter. Also, add one space after //.
| store := mqtt.NewFileStore(conf.MQTT.PersistDir) | ||
| opts.SetStore(store) | ||
| //disable clean session because paho deletes stored messages when restarts | ||
| opts.SetCleanSession(false) |
There was a problem hiding this comment.
Don't we want to make this configurable?
There was a problem hiding this comment.
if user set persistent = true but clean session = true, when service restarts messages are lost.
so i make it configurable (clean_session) in the config, but if user sets persist = true, the clean session is set accordingly.
I can change implementation and make the settings completely independent, but in that case user have to manually set clean_session = false when he wants to persist messages in fs
Signed-off-by: PricelessRabbit <[email protected]>
This PR enables export service to persist pending messages in the filesystem instead of in-memory.
In this way there is no data loss if export service fails or it is restarted when there are buffered messages.
However, there is still a flaw when the service is started when MQTT broker is unreachable (this is a known limitation of paho lib). See the issue i opened for details
Signed-off-by: PricelessRabbit [email protected]