"Incorrect string value" when saving non-UTF-8 emails to MySQL database

[rasmuscnielsen]

Hey there

This issue is mainly as a reference to people coming across the same hair-pulling issue as I did.

When receiving an email in ISO-8859-1 encoding with special characters (such as the danish "æøå") I found that a QueryException will be thrown such as

Illuminate\Database\QueryException: SQLSTATE[HY000]: General error: 1366 Incorrect string value: '\xE6ftet ...' for column 'message' at row 1 (SQL: insert into `mailbox_inbound_emails` (`message`, `message_id`, `subject`, `text`, `from`, `updated_at`, `cr...

This is due to attempting to insert ISO-8859-1 characters into a utf8mb4 MySQL column.

It would be cool if the package had a way to handle this internally - however due to this (hopefully) being a generally rare issue with most clients using UTF-8 I can see why this might not be a priority.

For those interested I solved it by introducing a new method on a custom App\Email model:

<?php

namespace App;

use BeyondCode\Mailbox\InboundEmail;

class Email extends InboundEmail
{
    public function ensureUtf8(): Email
    {
        $encoding = mb_detect_encoding($this->message, ['UTF-8', 'ISO-8859-1']); // Add more encodings to support here

        if ($encoding !== 'UTF-8') {
            $this->message = mb_convert_encoding($this->message, 'UTF-8', $encoding);
            $this->mimeMessage = null;
        }

        return $this;
    }
}

And in my AppServiceProvider@boot

Mailbox::catchAll(function (Email $email) {
   $email->ensureUtf8()->save();
});

Hope this can help somebody else out there :-)

[Labidev]

I have the same problem.
Your solution is great.
I just had to redo it a bit.
You need to override the InboundEmail model as App\Models\InboundEmail

<?php

namespace App\Models;

use BeyondCode\Mailbox\InboundEmail as BaseInboundEmail;

class InboundEmail extends BaseInboundEmail
{
    public function setMessageAttribute($value)
    {
        $encoding = mb_detect_encoding($value, ['UTF-8', 'ISO-8859-1']); // Add more encodings to support here
        if ($encoding !== 'UTF-8') {
            $this->attributes['message'] = mb_convert_encoding($value, 'UTF-8', $encoding);
        } else {
            $this->attributes['message'] = $value;
        }
    }
}

and set it in the config\mailbox.php file

'model' => App\Models\InboundEmail::class,