Getting jobs from beanstalkd - timed out exception
I am using Python 2.7, beanstalkd server with beanstalkc as the client library.
It takes about 500 to 1500 ms to process each job, depending on the size of the job.
I have a cron job that will keep adding jobs to the beanstalkd queue and a "worker" that will run in an infinite loop getting jobs and processing them.
eg:
def get_job(self):
while True:
job = self.beanstalk.reserve(timeout=0)
if job is None:
timeout = 10 #seconds
continue
else:
timeout = 0 #seconds
self.process_job(job)
This results in "timed out" exception.
Is this the best practice to pull a job from the queue?
Could someone please help me out here?
Asked by: Adelaide978 | Posted: 27-01-2022
Answer 1
Calling beanstalk.reserve(timeout=0) means to wait 0 seconds for a job to become available,
so it'll time out immediately unless a job is already
in the queue when it's called. If you want it never to time out,
use timeout=None (or omit the timeout parameter, since None is the default).
Similar questions
BeanStalkd on Solaris doesnt return anything when called from the python library
i am using Solaris 10 OS(x86). i installed beanstalkd and it starts fine by using command "beanstalkd -d -l hostip -p 11300".
i have Python 2.4.4 on my system i installed YAML and beanstalkc python libraries to connect beanstalkd with python my problem is when i try to write some code:
import beanstalkc
beanstalk = beanstalkc.Connection(host='hostip', port=11300)
no error so far but when i try to ...
python - Celery or beanstalkd or both?
I know (but I do not understand) that Celery can use Beanstalk as delivery mechanism.
From the Beanstalkd Homepage
"Beanstalk is a simple, fast work queue."
From the Celery Homepage
"It’s a task queue with focus on real-time processing, while also supporting task scheduling."
That seems pretty simi...
python - Django server stuck on Beanstalkd
I'm trying to setup a django server with existing code. But I seemed to be stuck on this while trying to start server using python manage.py runserver
this the traceback:
(camdyvirtualenv) Mac-mini-3:big-picture-api Sqooge_Ahmed$ python manage.py runserver
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/U...
python - how to use callback in beanstalkd ？
There is 3 machines and one publisher, two consumers.
I am using golang to publish order to some machine.
And the machine is used python to be consumer.
I want to know how can I get the result that the order is finished or failed in publisher.
And if the order is not belong to machine one, what should i do?
release or bury it?
python:
consumer:
import beanstalkc
def get_beanstalk_data(c...
Still can't find your answer? Check out these communities...
PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python