thickpaddy.com

Edit: this script has now evolved into a gem. See https://github.com/thickpaddy/gittest.

Continuous testing with autotest is great, especially when working on an existing application. Autotest runs in the background, and when you save a file, runs any matching tests and tells you if you've broken something, fantastic! Thing is though, there are lots of reasons why you don't want tests to run every, single time you make some little, tiny change to your code.

For me, the most important feature of autotest is its ability to find and run relevant tests for the modified code files, rather than run the entire test suite. This is crucial when you have an enormous application with a horrendously large test suite. It means you can have a fairly good idea whether you've broken the build or not, without having to run the entire test suite, which saves a lot of time.

But what if you forget to run autotest? Then you realise that you want to run the relevant tests and you can't, unless you re-save each file you have open. Or what if you've made some minor changes that you know are going to break the tests, but you want to save the file because you need to make a cup of tea or grab a sandwich? Wouldn't it be nice if you could just run the relevant tests when you're ready, rather than every single time you make a change?

Gittest is a scrappy little script that abuses autotest. You run it, it looks for new or modified files in git, then it uses the autotest mappings to determine which tests (or specs) need to run and tells autotest to run them (err, sort of). It's really, really ugly because I found it hard to figure out what the hell autotest was doing under the hood, but it works. I'll clean it up when I get a chance, but here's a working version of the script...

#!/usr/bin/env ruby

require 'rubygems'
require 'autotest'
require 'optparse'

# exit if we're not in a git repository
exit $?.exitstatus unless system('git diff --name-only > /dev/null')

# make sure we're in the root path for the repository
loop do
  begin
    Dir.entries('.git')
    break
  rescue SystemCallError
    Dir.chdir('..')
    next   
  end
end

options = {:fast => false, :diff => 'HEAD', :trace => false}
OptionParser.new do |opts|
  opts.banner = "Usage: ./script/gittest [options]"
  opts.on("-f", "--fast", "Fast mode - skips preparation of test db") do |o|
    options[:fast] = o
  end
  opts.on("-d MANDATORY", "--diff MANDATORY", "Commit argument for git diff command used to check for new or modified files (defaults to HEAD)") do |o|
    options[:diff] = o
  end
  opts.on("-t", "--trace", "Enable trace option when calling rake tasks to prepare test db") do |o|
    options[:trace] = o
  end  
end.parse!

# prepare db if fast start not switched on
unless options[:fast]

  puts "Preparing test database..."
  puts "(You can use the -f switch to skip this in future)"
  rake_options = options[:trace] ? '--trace' : ''
  system "rake db:migrate RAILS_ENV=test #{$rake_options} > /dev/null"
  system "rake db:test:prepare #{$rake_options} > /dev/null"
 
end
      
# autotest options
$f = true # never run the entire test/spec suite on startup
$v = false
$h = false
$q = false
$DEBUG = false
$help = false

# use ansi colors to highlight test/spec passes, failures and errors
COLORS = { :red => 31, :green => 32, :yellow => 33 }

# get a list of new or modified files according to git (using terminal commands is faster than using a ruby git library)
new_or_modified_files = `git diff --name-only #{options[:diff]}`.split("\n").uniq

if new_or_modified_files.size == 0
  puts "No modified files, exiting"
  exit
end
  
msg = "#{new_or_modified_files.size} new or modified file" 
msg << "s" unless new_or_modified_files.size == 1
puts msg + ":"

new_or_modified_files.each {|f| puts "\t#{f}"}

at = Autotest.new
# Note: the initialize hook is normally called within Autotest#run
at.hook :initialize
at.reset
at.find_files # must populate the known files for autotest, otherwise Autotest#files_matching will always return nil

# this isn't pretty, but it will probably be reliable enough (can't see any good reason for renaming that particular instance variable, IMO it should be accessible anyway)
test_mappings = at.instance_eval { @test_mappings }

# find files to test
files_to_test = at.new_hash_of_arrays
new_or_modified_files.each do |f|
  next if f =~ at.exceptions # skip exceptions
  result = test_mappings.find { |file_re, ignored| f =~ file_re }
  unless result.nil?
    [result.last.call(f, $~)].flatten.each {|match| files_to_test[match] if File.exist?(match)}
  end
end

# exit if no files to test
puts "No matching files to test, exiting" and exit if files_to_test.empty?
  
msg = "#{files_to_test.size} file" 
msg << "s" unless files_to_test.size == 1
msg << " to test"
puts msg + ":"
puts "\t" + files_to_test.map{|k,v| k}.sort.join("\n\t")

puts "Press ENTER to continue, or CTRL+C to quit"
begin
  $stdin.gets # note: Kernel#gets assumes that ARGV contains a list of files from which to read next line
rescue Interrupt
  exit 1
end
puts "Running tests and specs, please wait..." 

cmd = at.make_test_cmd(files_to_test)

at.hook :run_command

# copied from Autotest#run_tests and updated to use ansi colours in TURN enabled test output and specs run with the format option set to specdoc
old_sync = $stdout.sync
$stdout.sync = true
results = []
line = []
begin
  open("| #{cmd}", "r") do |f|
    until f.eof? do
      c = f.getc or break
      # putc c
      line << c
      if c == ?\n then
        str = if RUBY_VERSION >= "1.9" then
                          line.join
                        else
                          line.pack "c*"
                        end
        results << str
        line.clear
        if str.match(/(PASS|FAIL|ERROR)$/)
          # test output
          case $1
            when 'PASS' ; color = :green
            when 'FAIL' ; color = :red
            when 'ERROR' ; color = :yellow
          end
          print "\e[#{COLORS[color]}m" + str + "\e[0m"
        elsif str.match(/^\- /)
          # spec output 
          if str.match(/^\- .*(ERROR|FAILED) \- [0-9]+/)
            color = $1 == 'FAILED' ? :red : :yellow 
            print "\e[#{COLORS[color]}m" + str + "\e[0m"
          else
            print "\e[#{COLORS[:green]}m" + str + "\e[0m"
          end
        else
          print str
        end
      end
    end
  end
ensure
  $stdout.sync = old_sync
end

at.handle_results(results.join)