Continuous Testing of lime unit tests in Symfony 1.4 using watchr, Growl, growlnotify and launchd plists


I have a web app written in Symfony 1.4 for which I have lime unit tests. lime is the built-in unit test framework that comes with the Symfony framework.

I have been running my unit tests ever so often in my project_root dir. I would like my file system to be continuously monitored so that whenever I edit files in my Symfony project, my lime unit tests are run automatically. If any unit test fails, I want to be notified instantly via Growl. After some googling, I found these blog posts related to the matter:

Autotesting with watchr, growl and PHPUnit

CakePHP: Autotest using Watchr

I take the above posts a step further in that I show how to launch watchr automatically whenever you login, using launchd.plists. I also further the watchr.rb script to notify the user whenever a model is modified that all unit tests have passed. If there is an error, the user is alerted to which tests have failed.


Growl and growlnotify

Mac OS X 10.6 (Probably valid for OS X < 10.6)

Ruby (I’m using 1.8.7) and RubyGems

Optional Symfony 1.4
Not really required to follow along, however my final watchr.rb script
is used to parse lime unit tests output.



watchr can be installed via ruby gems

$ sudo gem install watchr
Successfully installed watchr-0.7
1 gem installed
Installing ri documentation for watchr-0.7...
Installing RDoc documentation for watchr-0.7...

watchr is run from the command line, with its single argument being
the ruby script that it runs. Below is a simple script which outputs
the name of the file that is modified

# watch is the DSL function provided by watchr
watch('lib/model/doctrine/(.*).php') { |m| code_changed(m[0]) }
def code_changed(file)
  puts("MODIFIED: #{file}")

Here, I run watchr from the command lind and modify a file in another

/Library/WebServer/Documents/project_root$ watchr watchr.rb 
MODIFIED: lib/model/doctrine/MyModel.class.php

lime unit tests

In this demonstration, I am using a simple MyModel.test.php file which contains the following code

 * test for MyModel class
 * @author James Borden
require_once dirname(__FILE__).'/../../bootstrap/unit.php';
                              $t = new lime_test(1);
                              $t->is('1', '1', '1 == 1');

When a test passes, it looks like this

When a test fails, it looks like this

If I add a a new test to a MyModel.test.php, but forget to update the amount of planned tests (i.e. update lime_test(n) to n total tests), I receive the following error.

Other times, the PHP compiler itself chokes on the test, printing
messages to STDERR. Here I add the fake function “fakeFunction()” to
MyModel.test, which results in a PHP Fatal Error

My watchr.rb

Below is the code for my new watchr.rb script. This script is given a set of files to monitor. If the files that are being monitored are edited, watchr will run each test file, parse any error messages that it may receive and send the error message via growl to the user.

Click below to view code. It is commented for the curious.

# Needed for displaying dates, used later on in the system call to growlnotify
require 'date'
# List of files to watch.
watch('lib/model/doctrine/(.*).php') { |m| code_changed(m[0]) }
watch('lib/(.*).php') { |m| code_changed(m[0]) }
watch('lib/form/(.*).php') { |m| code_changed(m[0]) }
watch('lib/form/doctrine/(.*).php') { |m| code_changed(m[0]) }
watch('lib/validator/(.*).php') { |m| code_changed(m[0]) }
watch('lib/widget/(.*).php') { |m| code_changed(m[0]) }
# what do when code is changed
def code_changed(file)
  # Array used to store the directorys that contain unit tests
  testDir = []
# Assume there won't be any errors
  error = false;
  # Go through all the test dir
  testDir.each do |test_dir|
    # select all *.php files from test dir
    test_dir_files = File.join("**",test_dir,"*.php")
    phpTestFiles = Dir.glob(test_dir_files)
    # For each PHP test file
    phpTestFiles.each do |phpTestFile|
      # run the system command php on each file
      # The test had an error
        error = true;
  # If there are no errors, let the user know
    growl("All Passed")
# command is run on a file and the output is parsed
def runCmdOnFile(cmd, file)
  if(cmd == "php")
  # we want to redirect stderr to stdout so that watchr can handle error messages
  # when the php interepreter itself barfs on our test code
  run_output = `#{cmd} #{file} 2>&1`
       return true
       # Try to parse the message for errors
       errorMessage = parseError(run_output, file)
       return false
# Simple check to see if the test passed
def checkMessage(run_output)
  if run_output.include?("# Looks like everything went fine.")
    return true
    return false
# Try to figure out what went wrong when the program was executed
def parseError(run_output, file)
  # Array to contain the error message
  errorMessage = []
  # First, note the name of the test that failed
  errorMessage.push("#{file}\n \n")
  output = []
  output = run_output.split("\n")
  run_output = output;
  # Look through the output for common errors.
  # This can be expanded as new problems arise
  run_output.each do |line|
    if(line =~ /# Looks like you planned/)
      errorMessage.push("You probably forgot to update lime_test.\n")
    if(line =~ /not ok/)
      errorMessage.push("Test Failed\n")
    # This is real bad, the php interpreter barfed on the code
    if( line =~ /PHP Fatal error/)
      errorMessage.push(line + "\n")
  return errorMessage;
# Function to send messages to user via growl
def growl(message)
  # Check to see if the message is a "All Passed" signal
  # and set the image accordingly
  if(message.include?("All Passed"))
    image = "~/.watchr/images/passed.png"
    # Set the passed flag to true
    passed = true
    # There was some kind of error
    image = "~/.watchr/images/failed.png"
  # Transform any messages that are strings into an array
    messageArray = []
    messageArray.push(message + "\n")
    message = messageArray;
  # If the error output of test is too long,
  # growlnotify will break. Limit the message to the first 2
  message = message.first(2);
  # Timestamp our message
  # Where does your growlnotify live? "which growlnotify" will tell you
  growlnotify = '/usr/local/bin/growlnotify'
  # Options to pass to growlnotify
    # Let's not make an "All Passed" signal sticky.. better to just reassure the
    # user everything is ok when a file is saved
    options = "-n Watchr -m '#{message}' --image #{image}"
    # If there is an error, make it persist so that the user has to least acknowledge it
    options = "-s -n Watchr -m '#{message}' --image #{image}"
  # Make the system call to growlnotify
  system("#{growlnotify} #{options}")
  return true

Here are the passed.png and failed.png that I used with growl.

Growl output

The new watchr.rb will check my tests and send output via growl.

Whenever I make changes to the files being watched and all tests passed, I like to be reassured everything is fine

If I add a new test to a MyModel.test.php, but forget to update the amount of planned tests (i.e. update lime_test(n) to n total tests), I receive the following error.

If I add a a new test to a MyModel.test.php, but forget to update the amount of planned tests (i.e. update lime_test(n) to n total tests), I receive the following error.

Finally, whenever PHP chokes completely on the file, growl outputs this

Creating a plist for launchd to automatically load watchr at login

If I had to manually run watchr every time that I code, I would eventually forget about running it altogether! I want watchr to always be there, checking over my code to make sure that all of my tests are still passing as I edit my project. For this, I use the built-in launchd daemon/agent manager program.

launchd is “the official” way to handle background programs (daemons) in Mac OS X. launchd works by loading xml-based plist files for every process. The plist file dictates how the process is handled by launchd. I wrote the following example watchr.plist to run watchr in the background automatically whenever I log in. watchr.plist lives in ~/Library/LaunchAgents

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" \
<plist version="1.0">
  <!-- Contains a unique string that identifies my plist to launchd.
       This key is required. -->
    <!-- watchr is started in the same dir as the watchr.rb script -->
    <!-- Contains the arguments [to execvp()] used to launch the daemon.
    This key is required.  -->
    <!-- This plist is loaded whenever I log in, but the command is not
    automatically run. This key ensures that it is run whenever this plist file is loaded -->

I use the system command, launchctl, with the subcommands load and unload from the command line to manage the watchr background process. Here, I load the newly created watchr.plist and the watchr process starts automatically. This is due to the fact I have the RunAtLoad key set to true.

~/Library/LaunchAgents$ launchctl load watchr.plist
~/Library/LaunchAgents$ ps -wax | grep ruby | grep -v grep
49081 ??         0:00.20 /System/Library/Frameworks/Ruby.framework/Ver
sions/1.8/usr/bin/ruby /usr/bin/watchr

I can also unload watchr.plist, and the process is stopped automatically.

~/Library/LaunchAgents$ launchctl unload watchr.plist
~/Library/LaunchAgents$ ps -wax | grep ruby | grep -v grep

If I had not specified RunAtLoad key to be true, the start and stop subcommands could have been used to start and stop the watchr process manually. Because watchr.plist lives in my personal ~/Library/LaunchAgents dir, it will be loaded whenever I login and turned off whenever I log out. About as automatic as it gets!


I have demonstrated how to use watchr to monitor files and run the corresponding lime unit tests whenever the files are changed. I have shown how to use Growl to notify the user of the tests’ status. I have also shown how to use launchd to automatically run watchr when the
user logs in. Thus, I have shown how to setup a continuous testing environment on the Mac for Symfony 1.4 projects.

It is my hope that I have given the reader enough knowledge to implement this system for their own unit tests. You certainly need not be limited to only lime unit tests. My watchr.rb script can be used as a template for your own projects. Happy coding and may all of your tests continually pass!

Update: You can find a github repository with this code here.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: