Sending notifications does not trigger mail check

[st3fan]

I'm not sure what is going on but sending notifications does not trigger mail check anymore.

Here is my debug log:

I've renewed by certificates by installing OS X Server on top of OS X 10.11. I enabled the mail server and push notifications. Exported the p12 and extracted the cert and key from it,

My client devices are a mix of iOS 10.0.1 and 9.3.2. Settings look good, notifications enabled. Visible on lock screen, etc.

The xaps-daemon connects to Apple's APNS service without complaints. I was worried that maybe the Entrust certificate was missing but that does not seem to make a difference. I even added a call to conn.ValidateHostname() to the code to make sure the Go TLS library is happy about the connection. It is.

I added a bunch more debug output and I see the xapsd server send notifications. They have the right format, they go to the APNS service. Does not trigger mail check though.

I added a feedback channel and check more explicitly for errors returned by Apple's APNS service. Nothing.

As another experiment I put together a minimal push client that uses https://github.com/sideshow/apns2 and just notifies my three iOS devices (2 on 10.0.1 and 1 on 9.3.2). Nothing happens.

... more later ...

[st3fan]

Some more things to look at:

• Generate a new certificate and see if that helps
• Check in Apple's certificate portal if things look good
• Does client registration work correctly?
• We parse a topic from the certificate and pass that to the client. Is it correct?
• Our notification just contains deviceToken and payload fields. Is that enough?
• Did anything change in the JSON payload?
• Can we look in the source code of APple's Dovecot patches?
• Are my devices really setup correctly to use Push, check mail in the background and display notifications? (Some of this can be ruled out by looking at mail.log on the server - even with a bad notification visibility setting, the iPhone should fetch)

[st3fan]

No code changes.

So here is an interesting one: I am only getting notifications from my Dovecot server if I add my (non-working) OS X Server (5.1.7) to my accounts on my iOS devices. The server is not configured with any sensible domain or anything.

Did Apple lock down email push notifications to OS X Server?

[st3fan]

Server capabilities. Not sure if this matters though.

OS X Server 5.1.4:

a1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE XAPPLEPUSHSERVICE SEARCH=FUZZY URLAUTH URLAUTH=BINARY COMPRESS=DEFLATE QUOTA ACL RIGHTS=texk] Logged in

Dovecot 2.0.19 with dovecot-xaps-plugin:

a1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS XAPPLEPUSHSERVICE] Logged in

[matthewpowell]

Looking at this with interest as I run Dovecot on Linux with iOS push, but using a different implementation. That's still working for me for what it's worth.

Did you renew the cert and then it stopped working? Or did it stop working and you tried to renew as a troubleshooting step?

Are you sure you're using the mail cert as opposed to one of the various other push certs that macOS Server grabs for itself? Using the wrong cert might cause the behaviour you've described.

[zero-one-devteam]

Have you already made some progress in this case? I'm running your code with dovecot 2.2, new mails are pushed, but if I've read them on another device, this status won't be pushed but only updated after 15 minutes pull interval. But maybe this behaviour is by design. Fastmail did archive a push of all states with cyrus, but as far as I know they've been supported by Apple.

[matthewpowell]

@zero-one-devteam – that sounds like a different issue, and the behaviour is more or less by design. Apple's IMAP push implementation is best thought of as a commanded fetch. Whenever the device receives a push notification for the account, it updates its view of the INBOX and alerts on new mail as required.

If you're using a mail delivery plugin to generate push events, you'll only get them when new mail arrives. Other events that might affect the badge count, like reading mail on another device, don't result in new mail delivery and so don't generate a push event.

If you generate push events on all relevant INBOX changes, you'll get a more immediate view of the mailbox across multiple devices. There's potentially a battery life tradeoff there, as reading activity on one device will then cause fetches on others. That's probably what Fastmail are doing.

[zero-one-devteam]

Thanks for pointing that out. We would like to got for Fastmail's approach, because waiting 15 minutes to sync makes some confusion. Maybe we can use the notify plugin as a wrapper around it, what do you think?

[matthewpowell]

That's more or less this issue if you want to discuss it further: st3fan/dovecot-xaps-plugin#5

[occcc]

Hi guys, any progress with this issue? Can I help somehow with the debugging?

[rlaager]

@matthewpowell What other implementation are you using?

[matthewpowell]

@rlaager It's a private implementation (at least until now), based fairly closely on Apple's macOS Server work. As a result, documentation is sparse to nonexistent, but you're welcome to take a look: https://github.com/matthewpowell/pushnotify

[zoc]

Interesting thread. I've been using @st3fan implementation of email push notifications for Dovecot since at least one year and a half and never had the issue reported here. And as of today it is still working pretty well :)

Any update @st3fan, were you able to understand what was wrong on your side ?

[freswa]

I ran into this a couple of weeks ago. I got it working by disabling push on the client and enable it a couple of minutes later.
Though we do not send any information that depends on the certificate, apple may do. By reenabling push I probably triggered the registration process which fixed this for me(temporarily until I install a new certificate).

• I would like to know what the vanilla implementation sends to the client especially when the client subscribes to push notifications of a subfolder (does the client get the information which subfolder it should fetch?)
• I would also like to know what the client receives from apple with this implementation or - even better - with the vanilla implementation to know if apple adds any information on their side.

[freswa]

Let's think about this case: I have so much time and money, that I have more than 1 OS X Server with dovecot installed and I want to use my device with both of them. Since iOS supports multiple mail accounts, this is a valid scenario.

How does iOS decide which INBOX to check when an APN comes in? It's just guessing, but the only thing we send back to the iOS device is the topic which identifies the OSX Server.
iOS probably maps this topic to a mail account. We do not send this topic with our notify request, but since the request has to be signed, it's easy for apple to add the topic to the request when it's forwarded to the client.