JvB
Photo

Jeroen van Baarsen

December 28, 2017
Clear your sidekiq queue when seeding your database

Every morning before I start working on a project I make sure my database is in a clean predictable state. I’m using Ruby on Rails, so the day starts with bin/setup. This script performs the following steps:

  1. Install missing gems.
  2. Delete database.
  3. Create the database and seeds it.

The good thing about this is that I actually get to see empty states (Which tend to be forgotten about in many projects if the database is never cleared), and also the app looks like it has actual usage instead of me typing sdfsdf in every form field.

A lot of projects I work on have Sidekiq for background jobs, and a thing I keep noticing is that a lot of those projects now also include scheduled jobs. With scheduled jobs I mean jobs that will run every X minutes for example.

I use Sidekiq-cron for this, which works really well! But there is a small problem I keep running into. When I seed my database again, all old data is being destroyed, expect in Sidekiq. So the retries will pile up because they all return ActiveRecord::RecordNotFound errors.

In most of my projects I know include a small snippet that will clean the sidekiq retry queue and also all the normal queued work.

The snippet itself is actually pretty simple, just add this to the top of your db/seeds.rb file:

unless ENV["CI"]
  require "sidekiq/api"

  Sidekiq::Queue.all do |queue|
    queue.clear
  end

  Sidekiq::RetrySet.new.clear
end

I have a check around the code to see if we are running in a CI environment since I run Sidekiq in memory mode on there.

What this little script does is go over all the queues and give it a clear command. Since Sidekiq has a separate list of jobs to retry we have to manually clear that also.

If you have any question about this, ping me on Twitter or drop me an Email