Only one job is running after Odoo 13 restart

[wamdam]

Nov 13 14:52:35 hostname odoo13-prod[190659]: 2020-11-13 13:52:35,760 190659 INFO ? odoo.addons.queue_job.jobrunner.runner: starting
Nov 13 14:52:35 hostname odoo13-prod[190659]: 2020-11-13 13:52:35,760 190659 INFO ? odoo.addons.queue_job.jobrunner.runner: initializing database connections
Nov 13 14:52:35 hostname odoo13-prod[190659]: 2020-11-13 13:52:35,792 190659 INFO ? odoo.addons.queue_job.jobrunner.runner: queue job runner ready for db odoo13-prod
Nov 13 14:52:35 hostname odoo13-prod[190659]: 2020-11-13 13:52:35,792 190659 INFO ? odoo.addons.queue_job.jobrunner.runner: database connections ready
Nov 13 14:52:35 hostname odoo13-prod[190659]: 2020-11-13 13:52:35,793 190659 INFO ? odoo.addons.queue_job.jobrunner.runner: asking Odoo to run job fbb79f8b-f912-4b92-9c65-a21874a697f4 on db odoo13-envola-prod
Nov 13 14:52:35 hostname odoo13-prod[190659]: 2020-11-13 13:52:35,875 190659 DEBUG ? odoo.addons.queue_job.jobrunner.runner: select() timeout: 60.00 sec
Nov 13 14:53:35 hostname odoo13-prod[190659]: 2020-11-13 13:53:35,936 190659 DEBUG ? odoo.addons.queue_job.jobrunner.runner: select() timeout: 60.00 sec
Nov 13 14:54:35 hostname odoo13-prod[190659]: 2020-11-13 13:54:35,996 190659 DEBUG ? odoo.addons.queue_job.jobrunner.runner: select() timeout: 60.00 sec
Nov 13 14:55:36 hostname odoo13-prod[190659]: 2020-11-13 13:55:36,058 190659 DEBUG ? odoo.addons.queue_job.jobrunner.runner: select() timeout: 60.00 sec

However there are about 30 jobs in pending state. There are 0 jobs in enqueued or started state.
This is an upgraded enterprise database from 12->13. I have reinstalled ("upgrade") queue_job on the new database.

--load=web,queue_job is also added to the command line, however this was added 2 days after the migration.

Part of .conf:

xmlrpc_port=8369
[queue_job]
channels = root:1,root.product_vaults:1

May port 8369 be a problem?

Any hints?

[guewen]

What commit are you using?

[guewen]

do you have a job stuck in "started" or "enqueued" which may block the queue?

[wamdam]

I'm using a089cb1. As I said, no started or enqueued jobs.

[wamdam]

let me try the lastest version in branch 13.0.

[wamdam]

New log output on the latest commit:
queue_job_notify trigger is missing in db odoo13-prod
Interesting. Will investigate.

[guewen]

queue_job_notify trigger is missing in db odoo13-prod

Good catch, yes the logger has been added in #264
Possible story: trigger lost during migration, then module if the module is upgraded (-u not -i), the post_init_hook is not run and no trigger is created.

[wamdam]

Correct. I manually ran the trigger creation from hooks/post_init_hook.py and now everything works.

psql::

        DROP TRIGGER IF EXISTS queue_job_notify ON queue_job;    
        CREATE OR REPLACE    
            FUNCTION queue_job_notify() RETURNS trigger AS $$    
        BEGIN    
            IF TG_OP = 'DELETE' THEN    
                IF OLD.state != 'done' THEN    
                    PERFORM pg_notify('queue_job', OLD.uuid);    
                END IF;    
            ELSE    
                PERFORM pg_notify('queue_job', NEW.uuid);    
            END IF;    
            RETURN NULL;    
        END;    
        $$ LANGUAGE plpgsql;    
        CREATE TRIGGER queue_job_notify    
            AFTER INSERT OR UPDATE OR DELETE    
            ON queue_job    
            FOR EACH ROW EXECUTE PROCEDURE queue_job_notify();    

Thanks for asking the right questions ;)