(Note CIA might be superceded by the Continuous Builder Plugin
, as evidenced by this post by David )
CIA runs on the server where your Subversion repository lives. It watches for commits, checks out the most recent tree of the related project, and runs all the unit & functional tests. If there are failures, they are logged to the database, as well as emailed to a pre-configured address.
(For more about the philosophy of continuous integration, see Martin Fowler’s article .)
Setting up CIA is a bit of a process. These notes are all for Unix systems, specifically OS X Tiger 10.4.2; I have no idea whether any of this could be made to work on Windows.
Set up for building
First, you need to make sure that you can actually build your app on the server where CIA runs. This may seem obvious, but it’s a lot harder to debug build problems when they’re happening under CIA. A good way to think about it is that CIA only does unit and functional tests, which is what happens when you run rake. So if you check out your project and type rake, does it pass the tests? No? First, make sure it does. Then continue.
Dependencies
Obviously, you’ll need Subversion installed and working. You’ll also need rake, which you’d need for Rails anyway. CIA ships with an SQL schema only for SQLite, so you’ll need to install a version of SQLite and the Ruby adaptor. See below for how to get it working with PostgreSQL.
Install CIA
You’ll have to get CIA by checking out a copy from the Subversion server. Go to wherever you keep outside source code and do this:
$ svn co http://dev.rubyonrails.org/svn/rails/tools/cia/trunk cia
If your Ruby is installed somewhere other than /usr/local/bin, you’ll have to change the #! lines in the files in scripts/.
Take a look at the end of config/environment.rb and make sure the paths are correct. You’ll almost definitely want to change NOTIFIER_FROM_ADDRESS, and maybe the paths for rake and svnlook if they aren’t in /usr/local/bin.
If you’re using SQLite, create the SQLite database:
$ sqlite db/cia.sqlite sqlite> .read db/dbsqlite.sql sqlite> .exit
However, I had a really hard time getting SQLite to work. CIA is run in part through the post-commit script, which may not be the same user as the WEBrick server. My way out was to simply use PostgreSQL. I translated the supplied db/sqlite.sql script to db/postgresql.sql, as follows:
CREATE TABLE projects ( id SERIAL PRIMARY KEY, project_id INTEGER DEFAULT NULL REFERENCES projects(id), name VARCHAR(255) NOT NULL, repository VARCHAR(255) NOT NULL, path VARCHAR(255) NOT NULL, email_addresses_to_be_notified TEXT DEFAULT NULL ); CREATE TABLE builds ( id SERIAL PRIMARY KEY, project_id INTEGER DEFAULT NULL REFERENCES projects(id) ON DELETE CASCADE, revision INTEGER DEFAULT NULL, success INTEGER DEFAULT NULL, running INTEGER DEFAULT 1, output TEXT DEFAULT NULL, created_at TIMESTAMP DEFAULT NULL );
...then initialized the database:
$ createdb -U YOUR-DB-USER cia_production $ psql -U YOUR-DB-USER cia_production < db/postgresql.sql
...and finally changed my configs/database.yml to use PostgreSQL under the desired userid.
Note that one of the unit tests in CIA currently fails. This seems to be because of an error in the ActiveMailer generated tests; I found a patch that seemed to fix the problem.
Because CIA actually checks out a copy of the most recent tree, it needs a place to write that tree, and CIA uses the data/ directory inside the CIA application tree. So you’ll have to make directory writable by the script/runner script when it’s run from the post-commit hook. If you’re using Apache/WebDAV to handle Subversion, the user/group will probably be www:www or the like. So:
$ sudo chown www:www data
Create the project in CIA
CIA is a Rails app itself, so you configure your projects through a web interface. So first you’ll need to start CIA under WEBrick:
$ cd YOUR-CIA-SOURCE $ script/server -e production
Then go to http://YOURSERVER:3000. Create a new project.
For Name, enter the descriptive name of the project.
For Repository, enter the path to the root of your repository, without a trailing slash.
For Path, enter the path to your project, relative to the path of your repository, without a leading or trailing slash.
For Email addresses to be notified, enter one or more email addresses who will be sent any failure reports.
Press the Create button to create your project entry.
Hook CIA up to Subversion
CIA runs by Subversion’s post-commit hook, which is set up in the base of each repository.
$ cd YOUR-REPOSITORY-ROOT $ cd hooks $ ls post-commit
If you don’t have a post-commit file, do this to create a new one:
$ sudo cp post-commit.tmpl post-commit $ sudo chmod +x post-commit
(That last chmod step is very important!—otherwise Subversion won’t run the hook script.)
Now edit that post-commit file, and append the following line, replacing YOUR-CIA-SOURCE with wherever you installed CIA, as above:
export PATH=/usr/bin:/bin export RAILS_ENV=production cd YOUR-CIA-SOURCE script/runner "Agent.build(\"$REPOS\", $REV)"
If you’re running out of DarwinPorts, you’ll need to prepend the right search path, so add this before the script/runner line (assuming you are using PostgreSQL as your database):
export PATH=/opt/local/bin:/opt/local/lib/pgsql8/bin:$PATH
If commits don’t seem to be getting through, you might try running your post-commit file by hand. To simulate what CIA is doing, you should add a revision to your project, and note the revision number. Then type this (but change www to the actual userid that would really be running post-commit; probably, it’s whatever your web server runs under):
sudo -u www env - YOUR-SVN-REPOS/hooks/post-commit YOUR-SVN-REPOS YOUR-REVISION
Test and use
You’ll need to have the CIA server running (see above).
Commit some sort of change in your project. Then go to http://YOUR-SERVER:3000 and you should see your projects listed; click the Show link and you should see at least one line that has either Failure or Success. If you see Application Error, obviously something’s gone wrong. If you don’t see any revisions/builds listed, it usually means the hook in the post-commit file isn’t being properly run; see the debugging instructions above.
If your integration has failed, you can click on the Failure link and it should show you the test output.
Check the email addresses you configured CIA with; they should have reports if tests have failed.
Gotchas
It appears that CIA never removes the source trees it checks out. If you’re concerned about this, you might want to clean out YOUR-CIA-SOURCE/data directory occasionally. (I don’t know whether this will cause any problems for CIA.)
I kept getting the following error in my builds:
Command failed with status (127): [ruby18 -Ilib:test "/usr/local/lib/ruby/gem...]
The solution was to set the path to the ruby18 executable in the post-commit hook script. This works on FreeBSD, ymmv:
export PATH=$PATH:/usr/local/bin
More info
Patrick Lenz’s article Setting up CIA: Continuous Integration Automater helped greatly in writing this HowTo?.
See also a post on BlogFish with some useful information.
My Continuous Integration machine has no development database
It doesn’t make sense for a CI server to have a development database, so the rake command is going to fail because the test tasks cause the test database to be cloned from the development one.
I wrote about how to work around this in DamageControl here
In the near future, I’ll have to update this to get the CI server to apply necessary migrations before running the tests, but this should be easier. This should actually be part of the Rails code – instead of cloning the DB structure, it should perform a migration before the tests are run.
—Tom Fakes
Running CIA on a shared server like Textdrive
If you are trying to get CIA to run on a shared hosting server, make sure the process that CIA runs as, i.e. “www” is added to your group. Also add “export HOME=Path to home directory” to POST-COMMIT file. .subversion dir is present in the home dir and svn needs this file to checkout from the repo.
Textdrive user
Running CIA on a Windows server
1) modify RAKE_PATH, SVN_LOOK_PATH and NOTIFIER_FROM_ADDRESS in config/environment.rb
2) setup SMTP in config/environments/production.rb
ActionMailer::Base.server_settings = {
:address => "your.smtp.server",
:port => 25,
:domain => "yourdomain.com"
}
3) create post-commit.bat file in YOUR_SVN_REPOSITORY_ROOT/hooks which should look like this:
@ECHO OFF SET REPOS=/path/to/svn/rep/root SET REV=%2 set RAILS_ENV=production cd c:\path\to\cia start ruby script\runner "Agent.build(\"%REPOS%\", %REV%)"
There was a problem using “SET REPOS=%1”, because of the windows path separator (backslash). Paths like c:\svn\rep were received as c: vn/rep in Ruby, so I have decided to hardcode the repository path in Unix style here. You must insert the same into the CIA web interface when creating projects. The “start” on the line running CIA build ensures that the build will not block the commiting SVN client.
—Karel Miarka
Error in CIA
I’m not sure what’s root of following problem, but I came with quick and dirty hack.
Using CIA I got into trouble one day. Committing to one project, return me an Failure of the another project. Looking into database I realise that everytime I commit, all the projects in cia are tested. Not only the one I commited.
The quick solution was rewrite method Agent#changed_projects in this way
def changed_projects
Project.find(:all,
:conditions => "repository='#{@repository}'")
end
—Radek Hnilica
Alternatives
If you think CIA is overkill for your needs and you just want a simple drop-in shellscript for your CI needs, Tuxie blogged about a Subversion post-commit hook he wrote:
Quick, dirty and nice Continous Integration with Subversion and Rails
Also if you prefer lightweight solutions for CI I recommend you to try Cerberus tool It is my favourite choice.
Another CI tool is Tesly Jr. Just install the plugin and you get CI reporting and a web-based test reporter.
A great open-source GUI tool for subversion for Mac OS X is svnX, available here:
svnX
category: Howto
(Note CIA might be superceded by the Continuous Builder Plugin
, as evidenced by this post by David )
CIA runs on the server where your Subversion repository lives. It watches for commits, checks out the most recent tree of the related project, and runs all the unit & functional tests. If there are failures, they are logged to the database, as well as emailed to a pre-configured address.
(For more about the philosophy of continuous integration, see Martin Fowler’s article .)
Setting up CIA is a bit of a process. These notes are all for Unix systems, specifically OS X Tiger 10.4.2; I have no idea whether any of this could be made to work on Windows.
Set up for building
First, you need to make sure that you can actually build your app on the server where CIA runs. This may seem obvious, but it’s a lot harder to debug build problems when they’re happening under CIA. A good way to think about it is that CIA only does unit and functional tests, which is what happens when you run rake. So if you check out your project and type rake, does it pass the tests? No? First, make sure it does. Then continue.
Dependencies
Obviously, you’ll need Subversion installed and working. You’ll also need rake, which you’d need for Rails anyway. CIA ships with an SQL schema only for SQLite, so you’ll need to install a version of SQLite and the Ruby adaptor. See below for how to get it working with PostgreSQL.
Install CIA
You’ll have to get CIA by checking out a copy from the Subversion server. Go to wherever you keep outside source code and do this:
$ svn co http://dev.rubyonrails.org/svn/rails/tools/cia/trunk cia
If your Ruby is installed somewhere other than /usr/local/bin, you’ll have to change the #! lines in the files in scripts/.
Take a look at the end of config/environment.rb and make sure the paths are correct. You’ll almost definitely want to change NOTIFIER_FROM_ADDRESS, and maybe the paths for rake and svnlook if they aren’t in /usr/local/bin.
If you’re using SQLite, create the SQLite database:
$ sqlite db/cia.sqlite sqlite> .read db/dbsqlite.sql sqlite> .exit
However, I had a really hard time getting SQLite to work. CIA is run in part through the post-commit script, which may not be the same user as the WEBrick server. My way out was to simply use PostgreSQL. I translated the supplied db/sqlite.sql script to db/postgresql.sql, as follows:
CREATE TABLE projects ( id SERIAL PRIMARY KEY, project_id INTEGER DEFAULT NULL REFERENCES projects(id), name VARCHAR(255) NOT NULL, repository VARCHAR(255) NOT NULL, path VARCHAR(255) NOT NULL, email_addresses_to_be_notified TEXT DEFAULT NULL ); CREATE TABLE builds ( id SERIAL PRIMARY KEY, project_id INTEGER DEFAULT NULL REFERENCES projects(id) ON DELETE CASCADE, revision INTEGER DEFAULT NULL, success INTEGER DEFAULT NULL, running INTEGER DEFAULT 1, output TEXT DEFAULT NULL, created_at TIMESTAMP DEFAULT NULL );
...then initialized the database:
$ createdb -U YOUR-DB-USER cia_production $ psql -U YOUR-DB-USER cia_production < db/postgresql.sql
...and finally changed my configs/database.yml to use PostgreSQL under the desired userid.
Note that one of the unit tests in CIA currently fails. This seems to be because of an error in the ActiveMailer generated tests; I found a patch that seemed to fix the problem.
Because CIA actually checks out a copy of the most recent tree, it needs a place to write that tree, and CIA uses the data/ directory inside the CIA application tree. So you’ll have to make directory writable by the script/runner script when it’s run from the post-commit hook. If you’re using Apache/WebDAV to handle Subversion, the user/group will probably be www:www or the like. So:
$ sudo chown www:www data
Create the project in CIA
CIA is a Rails app itself, so you configure your projects through a web interface. So first you’ll need to start CIA under WEBrick:
$ cd YOUR-CIA-SOURCE $ script/server -e production
Then go to http://YOURSERVER:3000. Create a new project.
For Name, enter the descriptive name of the project.
For Repository, enter the path to the root of your repository, without a trailing slash.
For Path, enter the path to your project, relative to the path of your repository, without a leading or trailing slash.
For Email addresses to be notified, enter one or more email addresses who will be sent any failure reports.
Press the Create button to create your project entry.
Hook CIA up to Subversion
CIA runs by Subversion’s post-commit hook, which is set up in the base of each repository.
$ cd YOUR-REPOSITORY-ROOT $ cd hooks $ ls post-commit
If you don’t have a post-commit file, do this to create a new one:
$ sudo cp post-commit.tmpl post-commit $ sudo chmod +x post-commit
(That last chmod step is very important!—otherwise Subversion won’t run the hook script.)
Now edit that post-commit file, and append the following line, replacing YOUR-CIA-SOURCE with wherever you installed CIA, as above:
export PATH=/usr/bin:/bin export RAILS_ENV=production cd YOUR-CIA-SOURCE script/runner "Agent.build(\"$REPOS\", $REV)"
If you’re running out of DarwinPorts, you’ll need to prepend the right search path, so add this before the script/runner line (assuming you are using PostgreSQL as your database):
export PATH=/opt/local/bin:/opt/local/lib/pgsql8/bin:$PATH
If commits don’t seem to be getting through, you might try running your post-commit file by hand. To simulate what CIA is doing, you should add a revision to your project, and note the revision number. Then type this (but change www to the actual userid that would really be running post-commit; probably, it’s whatever your web server runs under):
sudo -u www env - YOUR-SVN-REPOS/hooks/post-commit YOUR-SVN-REPOS YOUR-REVISION
Test and use
You’ll need to have the CIA server running (see above).
Commit some sort of change in your project. Then go to http://YOUR-SERVER:3000 and you should see your projects listed; click the Show link and you should see at least one line that has either Failure or Success. If you see Application Error, obviously something’s gone wrong. If you don’t see any revisions/builds listed, it usually means the hook in the post-commit file isn’t being properly run; see the debugging instructions above.
If your integration has failed, you can click on the Failure link and it should show you the test output.
Check the email addresses you configured CIA with; they should have reports if tests have failed.
Gotchas
It appears that CIA never removes the source trees it checks out. If you’re concerned about this, you might want to clean out YOUR-CIA-SOURCE/data directory occasionally. (I don’t know whether this will cause any problems for CIA.)
I kept getting the following error in my builds:
Command failed with status (127): [ruby18 -Ilib:test "/usr/local/lib/ruby/gem...]
The solution was to set the path to the ruby18 executable in the post-commit hook script. This works on FreeBSD, ymmv:
export PATH=$PATH:/usr/local/bin
More info
Patrick Lenz’s article Setting up CIA: Continuous Integration Automater helped greatly in writing this HowTo?.
See also a post on BlogFish with some useful information.
My Continuous Integration machine has no development database
It doesn’t make sense for a CI server to have a development database, so the rake command is going to fail because the test tasks cause the test database to be cloned from the development one.
I wrote about how to work around this in DamageControl here
In the near future, I’ll have to update this to get the CI server to apply necessary migrations before running the tests, but this should be easier. This should actually be part of the Rails code – instead of cloning the DB structure, it should perform a migration before the tests are run.
—Tom Fakes
Running CIA on a shared server like Textdrive
If you are trying to get CIA to run on a shared hosting server, make sure the process that CIA runs as, i.e. “www” is added to your group. Also add “export HOME=Path to home directory” to POST-COMMIT file. .subversion dir is present in the home dir and svn needs this file to checkout from the repo.
Textdrive user
Running CIA on a Windows server
1) modify RAKE_PATH, SVN_LOOK_PATH and NOTIFIER_FROM_ADDRESS in config/environment.rb
2) setup SMTP in config/environments/production.rb
ActionMailer::Base.server_settings = {
:address => "your.smtp.server",
:port => 25,
:domain => "yourdomain.com"
}
3) create post-commit.bat file in YOUR_SVN_REPOSITORY_ROOT/hooks which should look like this:
@ECHO OFF SET REPOS=/path/to/svn/rep/root SET REV=%2 set RAILS_ENV=production cd c:\path\to\cia start ruby script\runner "Agent.build(\"%REPOS%\", %REV%)"
There was a problem using “SET REPOS=%1”, because of the windows path separator (backslash). Paths like c:\svn\rep were received as c: vn/rep in Ruby, so I have decided to hardcode the repository path in Unix style here. You must insert the same into the CIA web interface when creating projects. The “start” on the line running CIA build ensures that the build will not block the commiting SVN client.
—Karel Miarka
Error in CIA
I’m not sure what’s root of following problem, but I came with quick and dirty hack.
Using CIA I got into trouble one day. Committing to one project, return me an Failure of the another project. Looking into database I realise that everytime I commit, all the projects in cia are tested. Not only the one I commited.
The quick solution was rewrite method Agent#changed_projects in this way
def changed_projects
Project.find(:all,
:conditions => "repository='#{@repository}'")
end
—Radek Hnilica
Alternatives
If you think CIA is overkill for your needs and you just want a simple drop-in shellscript for your CI needs, Tuxie blogged about a Subversion post-commit hook he wrote:
Quick, dirty and nice Continous Integration with Subversion and Rails
Also if you prefer lightweight solutions for CI I recommend you to try Cerberus tool It is my favourite choice.
Another CI tool is Tesly Jr. Just install the plugin and you get CI reporting and a web-based test reporter.
A great open-source GUI tool for subversion for Mac OS X is svnX, available here:
svnX
category: Howto