Write down that redis driver can't handle parallel workers

[wapmorgan]

I ask to Write it down that library can't handle parallel workers when redis driver used.

What steps will reproduce the problem?

Run few workers with redis driver

What's expected?

Few workers working in parallel

What do you get instead?

Few workers working as 1 worker - spreading payloads in one-thread-work

Additional info

Q	A
Yii version	latest
PHP version	doesn't matter
Operating system	RedHat

[samdark]

That is expected from queue but we may, indeed, state it more clearly.

[rob006]

@samdark I'm pretty sure that Redis driver handles multiple concurrent workers (or at least it did in the past).

[samdark]

@rob006 https://github.com/yiisoft/yii2-queue/blob/master/src/drivers/redis/Queue.php#L134

[rob006]

@samdark this is about moving delayed and interrupted jobs to waiting queue - one worker can handle this, while the rest will skip it and process actual jobs in queue.

[samdark]

Right. That needs to be verified. I've heard the same info about single worker being active while all others waiting for it to finish.

[wapmorgan]

That is expected from queue but we may, indeed, state it more clearly.

Sorry, but rabbit-driven queue do it in parallel.
Inconsistent

After all, In docs there is example with supervisor, where few workers used. Few workers for single-thread work?) doubt

[wapmorgan]

It would be good to clarify which drivers can handle few consumers

[rob006]

It would be good to clarify which drivers can handle few consumers

AFAIK all (except sync) drivers should support this. How do your run these multiple workers?

[wapmorgan]

It would be good to clarify which drivers can handle few consumers

AFAIK all (except sync) drivers should support this. How do your run these multiple workers?

Few docker containers with entrypoint: php yii queue/listen 3

[silverslice]

I tried to test redis queue in docker, everything works correctly, the workers process jobs in parallel.
@wapmorgan, please try my example.

The problem in redis driver is that the message can be lost between pop new message from "waiting" list and place it to the "reserved" set if worker crashes unexpectedly. To be more precise, the message will remain in the "messages" hash, but will not be sent.