Skip to content
This repository was archived by the owner on Mar 28, 2025. It is now read-only.

fix(recurring-task): reduce update load on task_processor_recurringtask table #23

Merged
khvn26 merged 3 commits intomainfrom
fix/centralise-update_or_create-logic-for-recurring-tasks
Feb 25, 2025
Merged

fix(recurring-task): reduce update load on task_processor_recurringtask table #23
khvn26 merged 3 commits intomainfrom
fix/centralise-update_or_create-logic-for-recurring-tasks

Conversation

@matthewelwell
Copy link
Copy Markdown
Contributor

@matthewelwell matthewelwell commented Feb 21, 2025

This is a WIP / PoC for a hypothesis that we are seeing a high number of updates on the task_processor_recurringtask table due to the fact that an update_or_create is called every time that a module is reloaded that includes a function decorated with @register_recurring_task().

The general approach is to only update the recurring tasks when the processor starts instead. There's a chance that we could optimise this further and only do it on a deployment, but this should at least reduce the load significantly in environments that are running a lot of instances of the Flagsmith application which may also be getting terminated / instantiated regularly.

Copy link
Copy Markdown
Contributor Author

@matthewelwell matthewelwell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor comment, but overall I'm happy with the additions from @khvn26 .

Comment thread task_processor/task_registry.py Outdated
)
registered_tasks[task_identifier] = registered_task

logger.debug(
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can probably get rid of these log statements. They're just noise, and the statement in initialise should log out all of the registered tasks on processor startup which is the desired behaviour, right?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@khvn26 khvn26 changed the title fix(recurring-task): reduce update load on task_processor_recurringtask table (WIP) fix(recurring-task): reduce update load on task_processor_recurringtask table Feb 24, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants