[4] subscribe on initiative, region and tags in order to motivate to come back to the make and be updated about last changes on the map

[rizomaa]

in order to motivate to come back to the make and be updated about last changes

Make it possibel to subycribe:

• by tag
• by initiative
• by region

This issue at kvm-reposit: kartevonmorgen/kartevonmorgen#325

[fr0zen]

• Develop mailing system
• Create table that stored link between user mail and subscription area.
• Develop functionality to save user mail in openfairdb
• Develop functionality to get user mail from openfairdb
• Launch process, that checks changes, happening on map. Then triggering sending email to mailboxes based on subscription areas.

[rizomaa]

@fr0zen could you open a bit what function will be in "mailing system"?

[fr0zen]

@rizomaa
We have three ways here:

• use our mail server
• some third-party service like Mailchimp
• setup mail server specifically for MAPA based on something like Nodemailer

[rizomaa]

Trigers:

• delete initiatives/events
• edit initiatives (tags)
• rate initiative
• add initiative

Will the triggers be aggregated on hour or day or week basis?

[wellemut]

Why do we need a new mailserver? The one we have works finde for me.

Here the issue on kvm
kartevonmorgen/kartevonmorgen#325

[rizomaa]

@wellemut where is the discussion about mailing system?

Here is a little research done by @ksandrsis #19

[fr0zen]

@wellemut it's perfect to use already configured mail-server for testing and fastest launch our instance of map. May you share info, who is administrator of mailserver, that kvm uses for mailing?

[wellemut]

@fr0zen I have never discussed about the mailing system. It is all done by @flosse . Hopefully he can share insights about this.

[uklotzde]

There is no dedicated mail server for openfairdb. Mails are currently sent out directly via sendmail by the server.

Integration of external e-mail service providers would be a separate task.

[wellemut]

Now, after a call with @uklotzde I understand that we need a new, extended mailservice. @flosse will share his ideas and experiences about mail-services we can use, but that can be a nice task, which is quite independend form openfairDB to develope a comprehensive mail- and subscription service.

[rizomaa]

@fr0zen what do you think do we need to spend time for launching some third application?

[rizomaa]

@wellemut I've added "in order to" because developpers often ask for why we need the feature. It helps find better solution

[wellemut]

Yes of cours, the reason for a function is very important, but not for the headline, in my opinion...

[rizomaa]

Sub Issue "How to subscribe"

1. There is a logic for geo area subscribtion page
   

2. Initiative subcription
   

3. Tag subscription
   

[rizomaa]

How to deliver. Draft of Ideas

[uklotzde]

This is a big task that may require some preliminary work and could be split into multiple smaller tasks. We should create a separate openfairdb issue for each of those tasks.

Decouple notification processing

The check for subscriptions and sending of notifications is currently done during the request processing. Instead this functionality should be moved into a separate component that runs in its own thread. This component will receive messages about database updates asynchronously through a channel. There are already some TODO comments in the code, i.e. in the update entry flow.

Customize subscription triggers

Currently all users are implicitly subscribed to all triggers. If we want to support selective notifications for a limited subset of triggers the database needs to be updated. Users may define a default set of triggers that apply to all their subscriptions that apply if no specific triggers are defined for a subscription. I would recommend to start with implementing this default set of triggers. Later users may define an individual set of triggers for specific subscriptions that overwrite/shadow their default triggers.

[uklotzde]

Instead of implementing everything directly in openfairdb I can also think of distributing those update events from openfairdb via WebHooks or to provide an endpoint with an Atom feed where external service can pick up those events whenever they are ready. The latter would be my preferred solution, because it reduces coupling between services. Even a proprietary REST endpoint for querying the latest updates would work for such a polling solution. If we use polling then all update events have to be stored in openfairdb, at least temporarily for some (retention) time.

What do you think?

[uklotzde]

The more I think about this completely new notification architecture the more appealing it appears to me.

As a drawback notifications will not be sent out instantly, but only after polling for updates. If you want near real time updates you could also decide to poll say once a minute.

[uklotzde]

By decoupling the systems into independent microservices we could easily replace parts with a custom solution that fits our use case.

[uklotzde]

Update events should include:

• time stamp
• entry id
• entry version
• entry location (lat/lon) (do we need both old and new location if entries could be relocated?)
• trigger (create, update, delete)
• ...some additional basic information that are of common interest to avoid roundtrips to openfairdb?

[wellemut]

For kvm with thought of only having just one sharing button on the map to

• Share the link (of item, area and hashtag)
• Share the iframe
• Offe field and button to subscribe

Loock our issue kartevonmorgen/kartevonmorgen#325
and compare with osm.org https://www.openstreetmap.org/

[wellemut]

Why not summarize subscription Mail in a digest

If the subscription function would be for endusers who are just intrested in news about their city, the, the weekly or monthly digest would be reasonable.

But the subscription function is mainly for regional pilots. (500 subscribers now are by 90 % pilots) For them it is much easier, to have all changes in single mails, because

• You have the names of each changed entry in the subject line, so you can delete most of the mails allready without opening them, because you know, what changes where made (maybe by yourself)

• You can easily share changes with others by forwarding an email. If you have a subscription, you would have to copy the change manually.

• You can build filtrs in your mailprogramm and move changes to certain folders, depending on topic or region. If you get alle Mail in a digest, you can not seprate them.
  

• At the end I fear that pilots just delete the digest and dont take their control function serious. Deleting one big mail ist much easier than 20 small ones, where you can see all names in the subject at one glance.

Thats just the mainpoints I see, but lets discuss it.

The subscription and real digest for endusers should be step 2 because we first need to define, what is relevant for endusers. If an organisations just changes a telephonenumber, its not worth to send to endusers...

[rizomaa]

@uklotzde it is interesting idea around the microservice. Do you think it is the case for thinking around data storage as well (for example for Subscription) or just to implement mailing and logic around?

[uklotzde]

Each microservice should manage their own data. The service should be as independent as possible.

The subscription service could periodically fetch the latest updates from the mapping service. An update log is currently missing in openfairdb, this would be a new feature.

Open question: How to deal with user accounts? Currently those accounts are managed by openfairdb. The subscription service could replicate those user database and check the credentials locally. Only if the verification fails it might need to ask openfairdb for updated login credentials for this user. Credentials are email + password hash.

We are already using Nginx as a reverse proxy for routing. When splitting the functionality into different microservices we need an API gateway anyway. The routes for internal communication between the subscription service with openfairdb must not be published!

[uklotzde]

The replication of user credentials could be done lazily, i.e. only for yet unknown users or if the login fails the subscription service needs to ask openfairdb. This approach nicely decouples both services and we only need to add as single request to openfairdb to read the user credentials. Of course, this route must be hidden and should only be available to the subscription service internally!

[wellemut]

@rizomaa How is the process for subscription?
It just found out that wechange is using mailjet as Mailing-Services: https://www.mailjet.com/
Does that service help us too? wechange is very happy with it.

[rizomaa]

@wellemut thanks for the link we will drill the service.
This feature was not completed, so, we hide it and open after test on a stage server.

[rizomaa]

One more service https://www.mailgun.com/

[wellemut]

Beautiful new subscription popup.
Here is my feedback:

1. even for me I needed to think a while till I knew, what I have to select to subscribe the right thing. The Radio-Button with three options seems to me just confusing. Also: What happens, if the user chooses "Initiative" but no inititiative is selected? What if he chooses TagsGeo but no tag is used? Put away theses buttons and let just subscrieb to what ever is selected.

2. Cancel-Button can be removed as well.
   

3. I would prefer the sharing button ... not to be replaced by the two options, but rather that they appear open above the button
   

[rizomaa]

@wellemut thanks for you feedback. Something has been already done.

[rizomaa]

@VitalyMikulich праверыць працу рассыльшчыка.
Усталёщка аписана тут: https://github.com/diglabby/mapa-subscription

1. Усталяваць
2. Падписацца
   3 Разаслаць.
3. Аісаць складанасці ўсталёўкі і рассылкі

[wellemut]

kartevonmorgen/kartevonmorgen.ts#151