Auto compaction errors after attempting 0.22.0 upgrade and downgrading to 0.20.0

[callumj]

Hello,

We recently performed an upgrade to 0.22.0 but had to perform a downgrade to 0.20.0 after we found some incompatibilities with some of our custom extensions. We now have lost auto compaction due what I am guessing is an invalid compaction spec across all our data sources. Looking at our logs, we see this:

59:59,936 WARN [qtp1922859107-120] org.apache.druid.metadata.SQLMetadataStorageActionHandler - Encountered exception[Could not resolve type id 'compaction' as a subtype of `org.apache.druid.indexing.common.task.batch.parallel.ParallelIndexTuningConfig`: known type ids = [index_parallel, realtime] (for POJO property 'tuningConfig')

What would be the remediation task for get compaction back online?

Thank you (and Happy Holidays!)

[callumj]

Turns out from the upgrade we had a stuck task that could not be killed (API operations would fail with same error) - we had to go into the backing store (MySQL DB) and delete these tasks. Might be something worth documenting.

[FrankChen021]

There is a refactoring on compaction configuration in #10676.
I think you need to delete the compaction configuration and re-create it after downgrading to the lower version.

[callumj]

Thanks @FrankChen021, that makes sense - we tried nuking the compaction config and that didn’t work. Deleting (manually via MySQL cli) the WAITING compaction task that was created in 0.22 but never finished was the only way to restore auto compaction in our (now) 0.20 cluster.

Under release notes this might be noted as a breaking change those that need to downgrade.

[FrankChen021]

You're right, I think we don't realize the compatibility of downgrading.

[github-actions]

This issue has been marked as stale due to 280 days of inactivity.
It will be closed in 4 weeks if no further activity occurs. If this issue is still
relevant, please simply write any comment. Even if closed, you can still revive the
issue at any time or discuss it on the dev@druid.apache.org list.
Thank you for your contributions.

[github-actions]

This issue has been closed due to lack of activity. If you think that
is incorrect, or the issue requires additional review, you can revive the issue at
any time.