Validate Your YAML Files

By Nebojsa Stricevic

A few months ago, a client project we’re working on started using sidekiq-scheduler for handling scheduled jobs. The tool works well and we started migrating more and more Cron jobs to Sidekiq.

A few days ago, after deploying a new set of jobs, we scheduled a few of them to test if everything works in production. But, we realized that no jobs were being logged in the Sidekiq log file. Since Sidekiq is managed by systemd, we inspected the status of the service with systemctl and journalctl and found out that sidekiq.yml file is not a valid YAML file. That prevented Sidekiq from booting.

This raised 2 important questions:

As the first line of defence, we decided to implement a simple script to check if sidekiq.yml is a valid YAML file and to run the script as part of our Continuous Integration pipeline.

After a few minutes, we had a working version of the script:

#!/usr/bin/env ruby

require "yaml"

sidekiq_yml = File.open("config/sidekiq.yml")

  puts "sidekiq.yml is valid"
rescue => ex
  puts "sidekiq.yml is not valid"
  puts "Error: #{ex.message}"
  exit 1

The script works fine, but the name of the YAML file is hard-coded and is a bit verbose. The second version is a Ruby one-liner that’s executed from a console:

ruby -e "require 'yaml'; YAML.load(\$stdin.read);" < sidekiq.yml

This simple one-liner doesn’t solve all problems. It’s still possible to deploy a file that contains a configuration that’s not valid for a library or a service. In some cases, the library might validate the configuration. In others you might want to do that yourself. We’re still working on a good solution for that problem. For now, the smoke test one-liner within a CI pipeline will do the trick.

In following blog posts we’re going to explore ways to monitor a Sidekiq service and notify developers if anything goes wrong.

What’s your solution for validating YAML files and other configuration?

Thanks to Vladimir Saric for reviewing the blog post.

Feel free to get in touch any time by sending us an email.