Adding Code Quality Checks To Your Ruby Test Suite

Over time I've integrated a few static analysis tools into my workflow. These have been extremely helpful in maintaining high quality code, but running them manually is tedious. For many projects, this would be automated in some kind of CI pipeline. For my current project, however, there is no automated CI pipeline. This app is deployed to Heroku straight from the command line. So, I decided to see if I could integrate these tools into my test suite, reducing manual quality checks to a single step.

Tool #1: Brakeman

Since I started learning Rails development, I have been using Brakeman to scan my code for security vulnerabilities. This tool uses static analysis to check for vulnerabilities to attacks like SQL injection and cross-site scripting. Brakeman was designed to be run from the command line, but a little digging into the docs revealed that it can be run as a Ruby library within an app. Following these instructions, I opened up a Pry console and ran:

require 'brakeman' => true Brakeman.run Rails.root.to_s => #<Brakeman::Tracker:0x00007fde2d546da8 ... tons of output in this object 

That object had way too much output to read. Next, I ran the same command, but stored the result as a variable...

result = Brakeman.run Rails.root.to_s => # same result as before 

... so I could query its methods and attributes:

result.methods.sort => [:warnings] # There were tons of other methods in this array result.warnings => [] result.instance_variables.sort => [:@errors] # Lots more here too result.errors => [{:error=>"/Users/briankephart/Sites/some_app/app/jobs/some_job.rb:14 :: parse error on value \"**\" (error)", :backtrace=>["Could not parse /Users/briankephart/Sites/some_app/app/jobs/some_job.rb"]}] 

As you can see, I had no security warnings, but one error due to Brakeman failing to parse a ** operator. This was helpful in showing me that the errors are readable in this form. Note that both the errors and warnings are returned as Arrays.

Using what I learned in my Pry console, I wrote the following test:

class BrakemanTest < ActiveSupport::TestCase require 'brakeman' test 'no brakeman errors or warnings' do result = Brakeman.run Rails.root.to_s assert_equal [], result.errors assert_equal [], result.warnings end end 

This worked perfectly. I could see the Brakeman output during my test run, and the test responded correctly to the error seen above.

Tool #2: RuboCop

I've recently been using RuboCop in a lot more of my work to check for syntax errors, good styling, and general readability. Like Brakeman, Rubocop is designed as a command line tool. Unlike Brakeman, the docs were not helpful to me in using it any other way. Instead, I had to dive into the source code. The most important line I found was here, in the file that defines the command line executable. The important points:

require 'rubocop' cli = RuboCop::CLI.new result = cli.run 

What I needed to do was instantiate a RuboCop::CLI object and call its #run method. Looking at those files, I found that the #initialize method takes no arguments, while the #run method takes all arguments from the command line as an array (I usually run rubocop on the command line without arguments, though). Back to the Pry console:

require 'rubocop' => true RuboCop::CLI.new.run # Inspecting 397 files # ... # # 397 files inspected, no offenses detected => 0 

Notice that the return value is 0. Knowing this, I wrote the following test:

# Do not use this code class RubocopTest < ActiveSupport::TestCase require 'rubocop' test 'no rubocop offenses' do assert_equal 0, RuboCop::CLI.new.run end end 

I noticed something strange, though. When I ran my full test suite, RuboCop checked all my files, but when I ran the test above on its own, RuboCop only checked that test file. The context in which I called the test became the context for RuboCop. The fix for this was to add a path argument to the run method, just as RuboCop allows a path argument on the command line.

# Use this instead of the code above class RubocopTest < ActiveSupport::TestCase require 'rubocop' test 'no rubocop offenses' do cop = RuboCop::CLI.new args = [Rails.root.to_s] assert_equal 0, cop.run(args) end end 

Tool #3: Bundler Audit

Bundler Audit checks your Gemfile.lock for known-vulnerable versions of gems. This tool is pretty wedded to the command line, to the point where it proved easiest to just run system commands.

class RubocopTest < ActiveSupport::TestCase require 'bundler/audit/cli' test 'no vulns in bundle' do `bundle audit update -q` # Update vulnerability database result = `bundle audit` # Run the audit code = `echo $?`.squish # Returns '0' if successful, otherwise '1' # Print the scan result as the error message if it fails. assert_equal '0', code, result # If successful, output the success message puts "\nMessage from bundler-audit: #{result}" end end 

That's all! With these three small tests, I can check my code style and security whenever I run my test suite, with no extra steps.