Support Ruby 3 by oeoeaio · Pull Request #319 · que-rb/que

oeoeaio · 2021-12-16T06:13:21Z

NOTE: still in draft

Should resolve #311
Depends on: #334

~~There is still a bit of cleanup to do yet (commit history is a bit of a mess), but here is a first cut at supporting ruby 3~~. Builds on the work of @ollym and @siegy22.

Notable changes

Database schema has changed to split job arguments into args and kwargs.
Passing a hash literal as the last job argument to be splatted into job keyword arguments is no longer supported
Dropped support for ruby < 2.7
Dropped support for rails < 6.0

TODO

ZimbiX

This is looking great. Nice work =)

lib/que/job.enqueue_spec.rb

lib/que/sequel/model.spec.rb

lib/que/job.rb

lib/que/job.spec.rb

spec/spec_helper.rb

Providing a Docker Compose service for running PostgreSQL, with a script for running the tests, makes it a lot easier to get started for developing Que. You no longer need to have installed & started PostgreSQL, set `DATABASE_URL` appropriately, or created a database for Que. Different versions of PostgreSQL can also be tested easily, by setting `POSTGRES_VERSION` - e.g. `auto/test-postgres-14`. This setup follows GreenSync conventions, although `auto/test` is not used on GitHub Actions. I've used the latest Ruby Docker image for Ruby 2.7. We can update this to Ruby 3 in #319.

Gemfile

keithlayne · 2022-01-06T17:28:06Z

Sorry if I've missed this elsewhere, but what's the target for this? When it's good to go, target v2?

maddymarkovitz · 2022-01-07T00:33:52Z

@keithlayne wrote:

Sorry if I've missed this elsewhere, but what's the target for this? When it's good to go, target v2?

I think that's the plan. We'd like to release 1.0 after we're satisfied that the current beta release doesn't have any issues, then these changes would come in as 2.0 (after another beta release in between to test them out).

Is Ruby 3 support something you need urgently or are able to test? We want to be guided by the community in terms of developing and documenting an upgrade process.

keithlayne · 2022-01-07T00:56:14Z

Thanks for the info. It's not urgent for us, but we're pre-release so I'd be willing to roll with the beta and help with testing and upgrade process. Just hook me up with that early-bird discount.

I think that this is our only dep really holding us back from trying out ruby 3 right now.

brunoprietog · 2022-01-09T06:18:37Z

It would be quite nice if it worked with ruby 3. A few months ago I started a project with ruby 3 and it was time to decide on the job library. Que looks very interesting, but it doesn't make me feel good to have to go back to ruby 2.7.
Thanks a lot for the work you guys are doing!

compactcode · 2022-02-01T01:00:29Z

We have switched to this branch for an upcoming ruby 3.1 project at Zepto. Still early days for us but I'm happy to help out with the effort/testing if needed.

ZimbiX · 2022-02-01T02:20:34Z

Hi @compactcode, thanks for the offer. Certainly some real-world testing would be great; let us know how you go.

In terms of what's left to do development-wise, it's as @maddymarkovitz said above: we just need to work out a suitable upgrade process from Que 1 for those who can't afford to clear their jobs table. I'm thinking we need to provide the ability to run both Que 1 and Que 2 simultaneously, so that existing jobs (with keyword arguments in the args column rather than kwargs) can be treated appropriately during the transition. Then cut over to Que 2 once all the Que 1 jobs are processed. We'd need a way to determine which jobs need to be handled by which worker - maybe by adding a que version column?

Providing a Docker Compose service for running PostgreSQL, with a script for running the tests, makes it a lot easier to get started for developing Que. You no longer need to have installed & started PostgreSQL, set `DATABASE_URL` appropriately, or created a database for Que. Different versions of PostgreSQL can also be tested easily, by setting `POSTGRES_VERSION` - e.g. `auto/test-postgres-14`. This setup follows GreenSync conventions, although `auto/test` is not used on GitHub Actions. I've used the latest Ruby Docker image for Ruby 2.7. We can update this to Ruby 3 in #319.

ZimbiX · 2022-02-21T05:54:25Z

Rebased to avoid conflicts

This one was used prior to v1.3.0, before polling needed to take into account `job_schema_version`

ZimbiX · 2022-03-23T12:19:28Z

Rebased to avoid conflicts in changelog and version file

…ecify the latest 1.x

Rubinius hasn't had any activity in a couple of years, and it isn't supported by the GitHub Actions ruby/setup-ruby@v1 action anyway.

…t everywhere we manually insert into the table

… requirement, and clarifying what job options are

ZimbiX · 2022-03-23T16:13:46Z

Damn, I thought I'd finally finished it, and was about to merge and release, but noticed we hadn't done this so added it to the todo to tackle later:

Add a test for passing job options via ActiveJob

oeoeaio · 2022-03-23T22:09:25Z

Damn, I thought I'd finally finished it, and was about to merge and release, but noticed we hadn't done this so added it to the todo to tackle later

Add a test for passing job options via ActiveJob

@ZimbiX I think we covered this in 7bcda94

ZimbiX · 2022-03-24T01:49:37Z

@ZimbiX I think we covered this in 7bcda94

Ah, excellent; thanks. Looks like we're good to go then! 🚀

Que and Rails (Active Job QueAdapter) have applied changes each other to support Ruby 3 keyword arguments and options provided as top level keywords, etc via these commits: rails#44734 rails#44248 que-rb/que@d9244b9 via que-rb/que#319 Finally, we've found it is quite difficult to maintain them. Going forward Active Job Que adapter can be included in the future version of que gem itself. Refer to rails#45899 the background for this change. Active Job Que adapter can be included in the future version of que gem itself.

ZimbiX mentioned this pull request Dec 16, 2021

Fixes for Ruby v3. #302

Closed

ZimbiX reviewed Dec 16, 2021

View reviewed changes

lib/que/job.enqueue_spec.rb Outdated Show resolved Hide resolved

lib/que/job.enqueue_spec.rb Outdated Show resolved Hide resolved

lib/que/sequel/model.spec.rb Show resolved Hide resolved

lib/que/job.rb Show resolved Hide resolved

oeoeaio commented Dec 16, 2021

View reviewed changes

lib/que/job.spec.rb Outdated Show resolved Hide resolved

oeoeaio commented Dec 20, 2021

View reviewed changes

spec/spec_helper.rb Outdated Show resolved Hide resolved

ZimbiX force-pushed the ruby3 branch from 5c5d3bc to f5eeb6c Compare December 23, 2021 04:15

ZimbiX mentioned this pull request Dec 23, 2021

Add Docker for easier setup of Que development environment #324

Merged

ZimbiX force-pushed the ruby3 branch from b4577fa to 55c14f1 Compare December 23, 2021 10:36

ZimbiX reviewed Jan 6, 2022

View reviewed changes

Gemfile Outdated Show resolved Hide resolved

ZimbiX mentioned this pull request Jan 17, 2022

Que::Locker.new(options) raises wrong number of arguments (given 1, expected 0) in Ruby 3.0.3 #326

Closed

maddymarkovitz force-pushed the ruby3 branch 3 times, most recently from 6267f6b to fad4c12 Compare January 24, 2022 04:49

maddymarkovitz force-pushed the ruby3 branch 2 times, most recently from d6a2403 to 78ff282 Compare February 14, 2022 05:44

ZimbiX force-pushed the ruby3 branch from 78ff282 to d0d32e8 Compare February 21, 2022 05:53

maddymarkovitz force-pushed the ruby3 branch from 6598450 to c61de71 Compare February 21, 2022 23:37

maddymarkovitz marked this pull request as ready for review February 21, 2022 23:37

maddymarkovitz force-pushed the ruby3 branch 2 times, most recently from ccbfdb5 to 2a859b4 Compare February 21, 2022 23:57

maddymarkovitz mentioned this pull request Feb 22, 2022

Add job_schema_version column #334

Merged

hlascelles mentioned this pull request Feb 23, 2022

Que scheduler does not enqueue jobs with the ruby3 branch of que hlascelles/que-scheduler#313

Closed

ZimbiX and others added 6 commits March 23, 2022 23:13

Drop outdated index: que_poll_idx

f0a290e

This one was used prior to v1.3.0, before polling needed to take into account `job_schema_version`

Bump version to 2.0.0.beta1

52180e5

Update active_job extensions spec to test handling of kwargs

10fd709

Add spec for using job options with ActiveJob

7bcda94

Update recommended upgrade process and docs to use migration version 6

dad9384

Changelog: Add tip for deploying Que 1.x and 2.x workers simultaneously

664ac26

ZimbiX force-pushed the ruby3 branch from 6939583 to 664ac26 Compare March 23, 2022 12:19

ZimbiX added 4 commits March 23, 2022 23:20

Changelog: Fix indentation of list sub-items

e8e55bf

Changelog: Change Que 2 recommended upgrade process to generically sp…

ed55bd5

…ecify the latest 1.x

Drop gems used to run the test suite against Rubinius

75cdf82

Rubinius hasn't had any activity in a couple of years, and it isn't supported by the GitHub Actions ruby/setup-ruby@v1 action anyway.

Rework GitHub Actions matrix to reduce permutations

3312f5c

ZimbiX force-pushed the ruby3 branch from fefc5a5 to 3312f5c Compare March 23, 2022 14:36

ZimbiX added 4 commits March 24, 2022 02:01

Remove line wrapping in latest migration

7bc3cf8

Add NOT NULL constraint to que_jobs.job_schema_version, and specify i…

6cc40bd

…t everywhere we manually insert into the table

Flesh out changelog a bit more, including noting job_schema_version…

9ba0134

… requirement, and clarifying what job options are

Rename index que_poll_idx_with_job_schema_version to que_poll_idx

ce0a2d6

Customise name of GitHub actions job for matrix clarity

4eaa8a7

ZimbiX merged commit 528f171 into que-rb:master Mar 24, 2022

ZimbiX deleted the ruby3 branch March 24, 2022 01:49

yahonda mentioned this pull request Sep 12, 2022

Remove QueAdapter from Active Job rails/rails#46005

Merged

9 tasks

kevinrobayna mentioned this pull request Apr 12, 2023

Que fails to calculate args and kwargs on Ruby 3.2 with ActiveJob #391

Closed

Comments

Conversation

oeoeaio commented Dec 16, 2021 • edited by ZimbiX Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

NOTE: still in draft

Notable changes

TODO

Uh oh!

ZimbiX left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

keithlayne commented Jan 6, 2022

Uh oh!

maddymarkovitz commented Jan 7, 2022

Uh oh!

keithlayne commented Jan 7, 2022

Uh oh!

brunoprietog commented Jan 9, 2022

Uh oh!

compactcode commented Feb 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ZimbiX commented Feb 1, 2022

Uh oh!

ZimbiX commented Feb 21, 2022

Uh oh!

ZimbiX commented Mar 23, 2022

Uh oh!

ZimbiX commented Mar 23, 2022

Uh oh!

oeoeaio commented Mar 23, 2022

Uh oh!

ZimbiX commented Mar 24, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

oeoeaio commented Dec 16, 2021 •

edited by ZimbiX

Loading

compactcode commented Feb 1, 2022 •

edited

Loading