RFC5322 validation prevents importing of previously-received emails

[alexforencich]

I have been running in to issues with importing old emails into proton mail via the proton mail bridge due to the RFC5322 validation. TBH, I think this validation should probably be removed completely - if emails can be received that are missing the date field and have matching from and sender fields, then it should be possible to import such a message via imap.

[LBeernaertProton]

We have tracked the date issue internally as GODT-2847.

[yu-tendo]

Any ETA? Thank you!

[LBeernaertProton]

We currently have no ETA for this ticket. For anyone who is blocked by this, we recommend adding a Date header field to the message in question.

[alexforencich]

Do you have a guide for how to edit received emails like this in thunderbird? Better yet, is there a way to automate fixing broken messages, as there several messages that trigger this problem (and cause the bulk copy operation to abort) out of thousands of messages?

[yu-tendo]

Thanks for the reply! @LBeernaertProton

I think this issue goes behind fixing the "Date" field. I have several different parsing issues, all related to the RFC5322 validation. I received all of these e-Mails at some point so there should be a possibility to move them (or fix them easily/quickly). Here are some example error messages:

• invalid rfc5322 message: failed to parse From headers: [Error offset=84]: expected FWS or comment for ...
• invalid rfc5322 message: failed to parse From headers: [Error offset=62]: expected " for quoted string ...
• invalid rfc5322 message: failed to parse From headers: [Error offset=25]: expected : for quoted start ...
• invalid rfc5322 message: Required header field 'From' not found or empty

Other parsing issues

• failed to retrieve addresses for, literal: failed to parse addresses for 'Cc': [Error offset=98]: expected ...
• failed to retrieve addresses for, literal: failed to parse addresses for 'To' [Error offset=128]: expected ...
• illegal base64 data at input byte 306

@alexforencich
There is an add-on for Thunderbird named "Header Tools Improved". This tool can fix simple errors (e.g., you can easily edit the "date" or "from" field). I fixed some e-Mails successfully with this tool (but I have more than a hundred to fix...)

[LBeernaertProton]

@timo-sztyler Please open a different ticket for all the issues other than invalid rfc5322 message: Required header field 'From' not found or empty as they are parsing issues, rather than validation errors.

It would be great if you could attach the relevant values of the headers for those messages. If you don't feel comfortable with the information on Github, you could always send them to my email (see commit history).

invalid rfc5322 message: Required header field 'From' not found or empty

Finally regarding this particular error, the From header field is always required to be present.

[yu-tendo]

Hi @LBeernaertProton
Sorry that this took a while. I followed your advice and created new issues with examples:

Header parsing error (RFC5322) prevents importing of emails
Message parsing error prevents importing of emails
Failed to parse addresses prevents importing of emails
Illegal base64 data prevents importing of emails
Message size prevents importing of emails

Not directly related, but I also provided examples for the following issue:
Various issues while importing emails via bridge