Dynamic Visibility Timeout

[pehrlich]

Hello,

With SQS it is possible to continually bump the visibility timeout whilst working on a task. This allows a short retry time, but the ability to have jobs of potentially long length, as long as the worker keeps updating the timeout. Will PyQS support this?

[andrewgross]

Hmm good question. Not something I had original thought about adding. I think it could be done as long as we have a way to ensure that the worker is still alive and processing, then the manager could bump the timeout continuously. We could also try threads on the worker but I am hesitant to mix that in.

[jhorman]

This would be awesome. Sometimes you have tasks of variable lengths and so it is hard to decide on a visibility timeout. If tasks could dynamically extend would solve a lot of issues.

A simple approach might just be a method extend_visibility(timedelta(seconds=10)) that internally uses a thread local to get the current ReceiptHandle. Another option would be an optional keyword argument to functions of _reciept_handle or something, with the corresponding extend_visibility(receipt_handle, timedelta(seconds=10)).

I think for a v1 it would be ok to say pyqs doesn't do anything too fancy, just exposes a method.

[jhorman]

I opened a PR. The tests don't seem to run for me locally, and I didn't add more since I wanted to also make sure we agreed on this direction.