CoreShop Notifications Stack Due to Unhandled Exception

[giorgosliatsos]

Q	A
Bug report?	yes
Feature request?	no
BC Break report?	no
RFC?	yes

In my Pimcore-CoreShop project, I accidentally erased my SMTP configuration in the YAML file, which resulted in the following configuration:
mailer:
transports:
main: smtp://

I am using a supervisor configuration to handle coreshop_notification messages:
[program:coreshop-notification-maintenance]
command=php bin/console messenger:consume coreshop_notification -vv --memory-limit=250M --time-limit=3600 --sleep=3
autostart=true
autorestart=true
user=www-data
process_name=%(program_name)s%(process_num)02d
stdout_logfile=/var/log/supervisor/coreshop-notification-maintenance.log
redirect_stderr=true_

Additionally, I use Redis to store messages:
CORESHOP_NOTIFICATION_TRANSPORT_DSN=redis://localhost:6379/coreshop_notification_messages

The Problem
Erasing the SMTP configuration resulted in the following error:
In Dsn.php line 41:

[Symfony\Component\Mailer\Exception\InvalidArgumentException]
The mailer DSN is invalid.

Exception trace:
at /var/www/restock-backend/vendor/symfony/mailer/Transport/Dsn.php:41
Symfony\Component\Mailer\Transport\Dsn::fromString() at /var/www/restock-backend/vendor/symfony/mailer/Transport.php:151
Symfony\Component\Mailer\Transport->parseDsn() at /var/www/restock-backend/vendor/symfony/mailer/Transport.php:103
Symfony\Component\Mailer\Transport->fromString() at /var/www/restock-backend/vendor/symfony/mailer/Transport.php:95
Symfony\Component\Mailer\Transport->fromStrings() at /var/www/restock-backend/var/cache/dev/ContainerLbmfPsU/getMailer_TransportsService.php:30
ContainerLbmfPsU\getMailer_TransportsService::do() at /var/www/restock-backend/var/cache/dev/ContainerLbmfPsU/App_KernelDevDebugContainer.php:3220
ContainerLbmfPsU\App_KernelDevDebugContainer->load() at /var/www/restock-backend/var/cache/dev/ContainerLbmfPsU/getMailer_MailerService.php:24
ContainerLbmfPsU\getMailer_MailerService::do() at /var/www/restock-backend/var/cache/dev/ContainerLbmfPsU/App_KernelDevDebugContainer.php:3220
ContainerLbmfPsU\App_KernelDevDebugContainer->load() at /var/www/restock-backend/var/cache/dev/ContainerLbmfPsU/getOnMessageFailedListenerService.php:23
ContainerLbmfPsU\getOnMessageFailedListenerService::do() at /var/www/restock-backend/var/cache/dev/ContainerLbmfPsU/App_KernelDevDebugContainer.php:3231
ContainerLbmfPsU\App_KernelDevDebugContainer->load() at /var/www/restock-backend/var/cache/dev/ContainerLbmfPsU/App_KernelDevDebugContainer.php:9315
ContainerLbmfPsU\App_KernelDevDebugContainer::ContainerLbmfPsU{closure}() at /var/www/restock-backend/vendor/symfony/event-dispatcher/EventDispatcher.php:235
Symfony\Component\EventDispatcher\EventDispatcher->sortListeners() at /var/www/restock-backend/vendor/symfony/event-dispatcher/EventDispatcher.php:70
Symfony\Component\EventDispatcher\EventDispatcher->getListeners() at /var/www/restock-backend/vendor/symfony/event-dispatcher/Debug/TraceableEventDispatcher.php:276
Symfony\Component\EventDispatcher\Debug\TraceableEventDispatcher->preProcess() at /var/www/restock-backend/vendor/symfony/event-dispatcher/Debug/TraceableEventDispatcher.php:133
Symfony\Component\EventDispatcher\Debug\TraceableEventDispatcher->dispatch() at /var/www/restock-backend/vendor/symfony/messenger/Worker.php:197
Symfony\Component\Messenger\Worker->ack() at /var/www/restock-backend/vendor/symfony/messenger/Worker.php:173
Symfony\Component\Messenger\Worker->handleMessage() at /var/www/restock-backend/vendor/symfony/messenger/Worker.php:108
Symfony\Component\Messenger\Worker->run() at /var/www/restock-backend/vendor/symfony/messenger/Command/ConsumeMessagesCommand.php:238
Symfony\Component\Messenger\Command\ConsumeMessagesCommand->execute() at /var/www/restock-backend/vendor/symfony/console/Command/Command.php:326
Symfony\Component\Console\Command\Command->run() at /var/www/restock-backend/vendor/symfony/console/Application.php:1096
Symfony\Component\Console\Application->doRunCommand() at /var/www/restock-backend/vendor/symfony/framework-bundle/Console/Application.php:126
Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand() at /var/www/restock-backend/vendor/symfony/console/Application.php:324
Symfony\Component\Console\Application->doRun() at /var/www/restock-backend/vendor/symfony/framework-bundle/Console/Application.php:80
Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at /var/www/restock-backend/vendor/symfony/console/Application.php:175
Symfony\Component\Console\Application->run() at /var/www/restock-backend/vendor/symfony/runtime/Runner/Symfony/ConsoleApplicationRunner.php:49
Symfony\Component\Runtime\Runner\Symfony\ConsoleApplicationRunner->run() at /var/www/restock-backend/vendor/autoload_runtime.php:29
require_once() at /var/www/restock-backend/bin/console:22

This exception is unhandled, causing the supervisor process to shut down and restart in a loop. This blocks the queue and fills the Pimcore Admin panel’s "Sent Emails" tab with errors containing the message:
The mailer DSN is invalid.

At this point, I checked my coreshop_notification_messages queue, which contained only the stuck message once.

Steps Taken
I performed the following steps to resolve the issue:

1)Stopped the supervisor process.
2)Restored the correct SMTP configuration.
3)Cleared the coreshop_notification_messages queue from Redis.
4)Restarted the supervisor to run messenger:consume.

However, after restarting the process, the queue was inexplicably refilled with the same blocked message multiple times, seemingly as many times as the number of retries logged in the "Sent Emails" tab. This resulted in sending the same email repeatedly to the same recipient.

Any insights on that issue? I think this is 2 issues actually:
1) handling the 'The mailer DSN is invalid.' exception and
2) how and why is the coreshop_notification_messages queue being refilled after clearing it, and how can this be prevented

Thank you in advance!

[dpfaffenbauer]

use null://null as dsn

[giorgosliatsos]

Thank you for your prompt reply. However, my issue is not with the SMTP configuration itself—I fully understand that emails could not be sent without the correct setup. The real problem is how a single stuck email resulted in 30,000 messages being queued, all addressed to the same recipient. I am unable to understand which mechanism causes this behavior.

While reproducing the error for debugging, I observed that even if I stop the supervisor processes and clear the coreshop_messenger queue in Redis, then fix the SMTP configuration and restart the supervisor, the queue is somehow refilled with the same message repeatedly. Clearing the queue alone does not seem sufficient to resolve the issue.

Any insight into the root cause of this behavior and how to handle it would be extremely helpful. While I understand that having the correct configuration should prevent this from happening, I am concerned about the potential for similar issues in the future—whether caused by a new configuration mistake or another unforeseen exception. It would be much safer if resolving the issue ensured that only the blocked messages are sent, without duplicating them excessively.

Thank you in advance for your assistance!

[dpfaffenbauer]

I have no idea honestly. We have 2 queues that do this. 1 queue called "coreshop_notification" that stores the messages to process for sending notifications, and one for failed messages "coreshop_notification_failed" that is only there to see the failed messages. why it re-sends emails that are not within a queue is a mystery to me.

What we also do, we do retry messages. so if it fails, it gets resend with a delay. but if you clear the whole table (or redis queue) they are gone.