How to set up queue_job in odoo sh?

[elliotxin]

I followed the instruction on the ubuntu server the queue_job working very well, after that, I try to set up the queue_job in odoo sh, unfortunately, its does's work, when i go throw the odoo log, one error message show like this:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=8069): Max retries exceeded with url: /queue_job/session (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f25a8441550>: Failed to establish a new connection: [Errno 111] Connection refused',))

Anyone can tell me how to fix this in odoo sh, thanks a lot.

[CasVissers-360ERP]

@Cedric-Pigeon @guewen
Any idea on this? Or is Odoo.sh not yet supported?

[CasVissers-360ERP]

@elliotxin
I got it running, add this to the Odoo config file:

[queue_job]
channels = root:4
host="https://yourenv.odoo.com"
port=443
http_auth_user="youruser"
http_auth_password="yourpassword"

Good luck!

[CasVissers-360ERP]

@Cedric-Pigeon @guewen
Think this issue can be closed. This works.

[guewen]

Thanks for the feedback!

[acysos]

Hello @CasVissers ,

How do you locate the odoo config file in odoo.sh?

Greetings

[CasVissers-360ERP]

@acysos
nano .config/odoo/odoo.conf

[acysos]

@CasVissers thank you

[NMisko]

@CasVissers Did you have to do anything else except modifying the config file? Does the amount of database workers make a difference? I can't get it working on my odoo.sh (see #244 )

[amigrave]

Hi there, I'm Fabien Meghazi, core developer and responsible of the Odoo.sh platform.
I'm sorry to hijack this thread but I'd like to point out that queue_job (in it's current design) is not compatible with Odoo.sh which uses a hybrid model of multiple threaded workers.

queue_job can somewhat work on Odoo.sh when using only one worker but the problem is that for each job it will trigger as many http calls as there are running workers. When an instance is creating a lot of jobs, this translates to a significative drop of performance due to concurrent updates.

We recently listed the projects hosted on Odoo.sh using this module and we are about to contact the problematic ones.

Greets.

[pedrobaeza]

Thanks for informing that and I don't consider it as a hijacking, but a good appreciation that closes the discussion.

[guewen]

Hi @amigrave,

Thanks for the notice. Do you know any way we could make it work on odoo.sh?

queue_job can somewhat work on Odoo.sh when using only one worker but the problem is that for each job it will trigger as many http calls as there are running workers. When an instance is creating a lot of jobs, this translates to a significative drop of performance due to concurrent updates.

This looks like a misconfiguration issue. The number of parallel jobs (so number of workers busy for jobs at the same time) can be configured using ODOO_QUEUE_JOB_CHANNELS=root:1 or equivalent in the configuration file, the default being 1. Or am I misunderstanding what you mean by "for each job it will trigger as many http calls as there are running workers"?

EDIT: I got what you say.

Is it a peculiarity due to the way the hybrid model works, like, it starts several "jobrunner" workers, because you start several odoo main processes which all fork their own "jobrunner" worker? (if that's the case, it probably can be prevented in queue_job's code).

[amigrave]

Is it a peculiarity due to the way the hybrid model works, like, it starts several "jobrunner" workers, because you start several odoo main processes which all fork their own "jobrunner" worker? (if that's the case, it probably can be prevented in queue_job's code).

@guewen Indeed, you nailed it. Unfortunately there's no easy way to make it work. One could think that you just need to let one worker spawn the jobrunner and somehow prevent the other workers to spawn the jobrunner, but this would not work because Odoo.sh recycles workers on some events and there's no "master" process, hence you could end up in a situation where you don't have the jobrunner alive anymore.

In order to make queue_job work on Odoo.sh I guess it would need to support this hybrid mode of multiple threaded workers and setup a communication between the workers so they can decide who is the master (a bit like a derivative of the registry signaling) and in case the master dies another worker becomes the master. I understand that such a thing can't be foreseen in the development of queue_job because:

1. it seems overly complex
2. I can't guarantee the current hybrid model won't change in the future
3. this Odoo.sh hybrid mode is not "standard"

We already discussed internally about having this hybrid mode standardized in Odoo (I love the idea because it would be less code to maintain in Odoo.sh) but nothing has been started yet in that direction so far and some challenged should be tackled first, such as the longpolling routes which are specially handled by Odoo.sh.

Note: This is just a hot reaction, maybe there's an easier way I haven't tough of to make queue_job work on Odoo.sh but if later I think about an easier way I'll post it here.

[guewen]

@amigrave Thanks for the very informative and detailed answer!

Actually, the fact that we have to spawn only one jobrunner when we have multiple nodes is kind of annoying outside odoo.sh too, so we could tackle this problem for other uses cases. (At camptocamp, we deploy several nodes on production and the unique jobrunner defeats the improved availability of such setup, regarding jobs).

An idea which doesn't seem that difficult for me, would be that the jobrunner could acquire an advisory lock at loading, and the other nodes would only wait until they can acquire it, the first in the race gets to be the new master. Hot reaction too, I may be overlooking details :)

Anyway, I'll think about this and thanks again for sharing your much helping thoughts.