Getting jobs from beanstalkd - timed out exception

I am using Python 2.7, beanstalkd server with beanstalkc as the client library.

It takes about 500 to 1500 ms to process each job, depending on the size of the job.

I have a cron job that will keep adding jobs to the beanstalkd queue and a "worker" that will run in an infinite loop getting jobs and processing them.


def get_job(self):
    while True:
        job = self.beanstalk.reserve(timeout=0)
        if job is None:
            timeout = 10 #seconds
            timeout = 0 #seconds

This results in "timed out" exception.

Is this the best practice to pull a job from the queue?

Could someone please help me out here?

Asked by: Adelaide978 | Posted: 27-01-2022

Answer 1

Calling beanstalk.reserve(timeout=0) means to wait 0 seconds for a job to become available, so it'll time out immediately unless a job is already in the queue when it's called. If you want it never to time out, use timeout=None (or omit the timeout parameter, since None is the default).

Answered by: David214 | Posted: 28-02-2022

Similar questions

BeanStalkd on Solaris doesnt return anything when called from the python library

i am using Solaris 10 OS(x86). i installed beanstalkd and it starts fine by using command "beanstalkd -d -l hostip -p 11300". i have Python 2.4.4 on my system i installed YAML and beanstalkc python libraries to connect beanstalkd with python my problem is when i try to write some code: import beanstalkc beanstalk = beanstalkc.Connection(host='hostip', port=11300) no error so far but when i try to ...

python - Celery or beanstalkd or both?

I know (but I do not understand) that Celery can use Beanstalk as delivery mechanism. From the Beanstalkd Homepage "Beanstalk is a simple, fast work queue." From the Celery Homepage "It’s a task queue with focus on real-time processing, while also supporting task scheduling." That seems pretty simi...

python - Django server stuck on Beanstalkd

I'm trying to setup a django server with existing code. But I seemed to be stuck on this while trying to start server using python runserver this the traceback: (camdyvirtualenv) Mac-mini-3:big-picture-api Sqooge_Ahmed$ python runserver Traceback (most recent call last): File "", line 10, in <module> execute_from_command_line(sys.argv) File "/U...

python - how to use callback in beanstalkd ?

There is 3 machines and one publisher, two consumers. I am using golang to publish order to some machine. And the machine is used python to be consumer. I want to know how can I get the result that the order is finished or failed in publisher. And if the order is not belong to machine one, what should i do? release or bury it? python: consumer: import beanstalkc def get_beanstalk_data(c...

Still can't find your answer? Check out these communities...

PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python