If you’ve ever used kue before, you’ll
notice its callback based. What happens if you loose that callback? Well,
that job will stall your worker and remain “active” until the next time
you restart your worker.
Have you also noticed that failed and completed jobs take up redis’
memory space? That’s not good either especially if you have a small
Redis memory instance. You also want Redis to be the least CPU intensive
part of your application too.
Why do we need the other queue states too? I find those completely
redundant to have. The only ones I care about is how many jobs are
waiting to be executed and which ones are being executed. Everything
else can be stored in my text logs for me to take a look at later.
Here’s a little snippet of how I’ve been able to resolve these problems.
I do assume that you have some sort of external logging service such
as Papertrail and that you do not really care
about the output of the job that you are running.
If you’ve also noticed, async errors are not caught well within
promises and will cause Kue or your worker to crash. When that happens
any other job that is processing with it now goes into limbo or
an inactive state. Kue finally documented
several ways on how to keep this from happening. In my method, I do a hybrid
of both and wrap it with a Promise and a domain.