Replies: 1 comment 2 replies
-
No more configuration. Sidekiq was not designed to retry jobs 60-80 times. Why are you using linear backoff? |
Beta Was this translation helpful? Give feedback.
2 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
The app that I'm working on has a few jobs that retry after a constant number of minutes. In some cases, jobs decrease their retry frequency after a certain number of retries. This is done by configuring the
sidekiq_retry_in
block.Taking a look at a job that retries roughly every 5 minutes, I found out that some retries happened only after 10 minutes. After some investigation, I've tracked this variability down to these lines
sidekiq/lib/sidekiq/job_retry.rb
Lines 187 to 192 in 312672d
I understand the reasoning behind spreading the retries but when I've plotted the variability into a chart, I've noticed that the jitter value after some retries surpasses the original retry delay:
From the chart, after about 30 retries the jitter value matches the retry delay. After about 60 retries, the jitter value is more than double the retry delay.
This job halves its retry frequency after 80 retries. Even in that case, the chart shows a lot of retries where the jitter value is greater than the retry delay.
Taking inspiration from the
sidekiq_retry_in
block, I've created a draft:JoinRaylo/sidekiq#1
where the retry jitter is configurable to go hand in hand with the retry delay and allows to overcome the increasing deviation shown in the chart. The block can use the retry count and the delay to calculate the jitter and falls back to the original implementation if the block is not provided.
What do you think of this approach?
Beta Was this translation helpful? Give feedback.
All reactions