Revamp recurring tasks configuration and management #338
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`config/recurring.yml`. Run them using the least busy dispatcher (largest polling interval) or a new default dispatcher if none. This also introduces some new options for the CLI, to allow for `--dispatch_only` and `--work_only`, and to skip recurring tasks.
And also setting a description that we can use from Mission Control in the future.
A Scheduler, instead of Dispatcher, that will be created only when there are recurring tasks to run.
Calling `set` on the job class returns a `ConfiguredJob`, and calling `perform_later` on a `ConfiguredJob` doesn't yield the job to a block passed as parameter, so we can't check if the enqueuing was successful or not. To work around this, the only way is to instantiate the job ourselves with the arguments, then call `enqueue` on that, and rely on the result, which would be what `job_class.perform_later` would yield to the block.
To match worker's `queues` configuration.
No need to have a class, can specify this using just a command for which we'd provide a default RecurringtJob class that will just eval the command.
rosa
force-pushed
the
improve-recurring-tasks-config
branch
from
75b76ce to
8df600d
Compare
rosa
added a commit
to rails/mission_control-jobs
that referenced
this pull request
Sep 9, 2024
rosa
added a commit
to rails/mission_control-jobs
that referenced
this pull request
Sep 9, 2024
rosa
added a commit
to rails/mission_control-jobs
that referenced
this pull request
Sep 9, 2024
rosa
added a commit
to rails/mission_control-jobs
that referenced
this pull request
Sep 9, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR changes how recurring tasks are defined and how they're enqueued. They're no longer part of the
dispatchersconfiguration or enqueued by the dispatchers. Instead, they have their own configuration file (config/recurring.yml, but it can be changed) and their own process, a new scheduler that's just in charge of enqueuing jobs for recurring tasks. Besides, the configuration for each task has been extended, and now it allows optionally specifying aqueueandpriority, and it also supports providing just a "command" (any Ruby code) instead of requiring a job class and arguments. The command will be eval'ed within a newSolidQueue::RecurringJobprovided as well. This is how the new configuration looks like: