Recently, we introduce continuous integration into the development process. CI solution for rails include CruiseControl.rb from thoughtwork, Integrity, Cerberus and RunCodeRun which even provides the build service. We choose integrity since it’s light-weighted, easy to configure, good support for git and works for ourselves on our own install.

The latest gem version for integrity is 0.1.10, whose source code resides at github. There is also another forked project, which has wonderful features (e.g. background build) which make significant progress. Till now, it's still in development and has no official announcement yet.

Although integrity is easy to configure, we still encounter many problems, most of that are about the building environment. Hope the practice here can help you in case you want to try it.

Server Configuration

• SUSE Linux
• Apache
• Passenger
• mysql 5.x

Install Integrity

$ gem install --passenger integrity
$ integrity install [integrity-diretory]

These commands generate 3 empty directories (build, log, public) and 2 files (config.ru, config.yml).

We use mysql instead of default sqlite. Gem package do_mysql (~> 0.9.11) is required.

$ integrity migrate_db config.yml

Unfortunately, the generated migration sql is not accepted by mysql 5.x. The error is:

== Performing Up Migration #1: initial
   CREATE TABLE `integrity_projects` ENGINE = InnoDB CHARACTER SET utf8 COLLATE utf8_general_ci (`id` serial PRIMARY KEY, `name` VARCHAR(50) NOT NULL, `permalink` VARCHAR(50), `uri` VARCHAR NOT NULL DEFAULT NULL, `branch` VARCHAR(50) NOT NULL DEFAULT 'master', `command` VARCHAR(50) NOT NULL DEFAULT 'rake', `public` TINYINT DEFAULT 1, `building` TINYINT DEFAULT 0, `created_at` DATETIME, `updated_at` DATETIME, `build_id` INT(11), `notifier_id` INT(11))

/opt/local/lib/ruby/gems/1.8/gems/dm-core-0.9.11/lib/dm-core/adapters/data_objects_adapter.rb:92:in `execute_non_query': (mysql_errno=1064, sql_state=42000) You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '`id` serial PRIMARY KEY, `name` VARCHAR(50) NOT NULL, `permalink` VARCHAR(50), `' at line 1 (MysqlError)
Query: CREATE TABLE `integrity_projects` ENGINE = InnoDB CHARACTER SET utf8 COLLATE utf8_general_ci (`id` serial PRIMARY KEY, `name` VARCHAR(50) NOT NULL, `permalink` VARCHAR(50), `uri` VARCHAR NOT NULL DEFAULT NULL, `branch` VARCHAR(50) NOT NULL DEFAULT 'master', `command` VARCHAR(50) NOT NULL DEFAULT 'rake', `public` TINYINT DEFAULT 1, `building` TINYINT DEFAULT 0, `created_at` DATETIME, `updated_at` DATETIME, `build_id` INT(11), `notifier_id` INT(11))

The correct syntax should be “CREATE TABLE (...) options”. Apply this patch to dm-migrations will resolve this problem.

diff --git a/lib/sql/mysql.rb b/lib/sql/mysql.rb
index 9311bc0..ca2a531 100644
--- a/lib/sql/mysql.rb
+++ b/lib/sql/mysql.rb
@@ -22,7 +22,11 @@ module SQL
     end

     def create_table_statement(quoted_table_name)
-      "CREATE TABLE #{quoted_table_name} ENGINE = InnoDB CHARACTER SET #{character_set} COLLATE #{collation}"
+      "CREATE TABLE #{quoted_table_name}"
+    end
+
+    def create_table_options
+      "ENGINE = InnoDB CHARACTER SET #{character_set} COLLATE #{collation}"
     end

     # TODO: move to dm-more/dm-migrations
diff --git a/lib/sql/table_creator.rb b/lib/sql/table_creator.rb
index 5b210c9..9c71413 100644
--- a/lib/sql/table_creator.rb
+++ b/lib/sql/table_creator.rb
@@ -21,7 +21,7 @@ module SQL
     end

     def to_sql
-      "#{@adapter.create_table_statement(quoted_table_name)} (#{@columns.map{ |c| c.to_sql }.join(', ')})"
+      "#{@adapter.create_table_statement(quoted_table_name)} (#{@columns.map{ |c| c.to_sql }.join(', ')}) #{@adapter.create_table_options if @adapter.respond_to?('create_table_options')}"
     end

     # A helper for using the native NOW() SQL function in a default

There are still issues in dm_migration for sql generation of “URI” and ":default". This patch to integrity does workaround. Then, you can tuning the database as you want directly.

diff --git a/lib/integrity/migrations.rb b/lib/integrity/migrations.rb
index f8497e3..0cd9918 100644
--- a/lib/integrity/migrations.rb
+++ b/lib/integrity/migrations.rb
@@ -39,7 +39,7 @@ module Integrity
           column :id,          Integer,  :serial => true
           column :name,        String,   :nullable => false
           column :permalink,   String
-          column :uri,         URI,      :nullable => false
+          column :uri,         String,   :nullable => false, :default => ""
           column :branch,      String,   :nullable => false, :default => "master"
           column :command,     String,   :nullable => false, :default => "rake"
           column :public,      Boolean,                      :default  => true
@@ -53,10 +53,10 @@ module Integrity
 
         create_table :integrity_builds do
           column :id,                Integer,  :serial => true
-          column :output,            Text,     :nullable => false, :default => ""
+          column :output,            Text
           column :successful,        Boolean,  :nullable => false, :default => false
-          column :commit_identifier, String,   :nullable => false
-          column :commit_metadata,   Yaml,     :nullable => false
+          column :commit_identifier, String,   :nullable => false, :default => ""
+          column :commit_metadata,   Yaml
           column :created_at,        DateTime
           column :updated_at,        DateTime
 
@@ -66,7 +66,7 @@ module Integrity
         create_table :integrity_notifiers do
           column :id,         Integer, :serial => true
           column :name,       String,  :nullable => false
-          column :config,     Yaml,    :nullable => false
+          column :config,     Yaml
 
           column :project_id, Integer
         end
@@ -110,7 +110,7 @@ module Integrity
           column :started_at,   DateTime
           column :completed_at, DateTime
           column :successful,   Boolean
-          column :output,       Text,    :nullable => false, :default => ""
+          column :output,       Text
           column :created_at,   DateTime
           column :updated_at,   DateTime

Create Your Project

local git repository

If you project resides on your build machine, you can specify it as local path.

/var/git/[project-name].git 

remote git repository

More often, the git repository you want to build is not on the build machine. You may want to use ssh public/private key authentication to facilitate source pulling. Be careful that the running environment is not identical to that when you login in as the owner of the web app process. If you put the private key under /home/[owner]/.ssh, it will not be used when pulling the code. Put the private key under /root/.ssh definitely resolves this problem.

ssh://[user]@[hostname]/var/git/[project-name].git

Running the Build

After configuring your first project, you can try to build it. One standalone machine is best for running CI. If deploy integrity on one server and use another DB server (sort of production environment), you’ll have many problems. For example, “rake test:units” will invoke “rake db:test:prepare”, which needs the development db. You can’t locate your development db on another machine if you are not planning to do so when you develop. Make sure the building environment is almost the same as your development environment.

Git Hook Up

After each push, the building machine needs to be notified to run the build. The “Push URL” integrity provides is supposed to be used against Github project. If you host the git repository on your own, you can check A Git Hook to Push to Integrity. Unfortunately, when running the script, we get 500 Internal error from integrity. As an alternative, we use the below URL in the script to simulate clicking "Fetch and Build" button.

POST_RECEIVE_URL = 'http://[user]:[password]@[integrity-server]/[project-name]/builds'

Email Notification

After installing integrity-mail and configuring the local SMTP server in the project setting, we got “502 Command Not implemented” error when sending mail because the SMTP server has no TLS support yet. Integrity uses sinatra-ditties for mail, where TLS is required by default.

Refer to Postfix With SMTP-AUTH And TLS on openSuSE Linux 10.x.

As an alternative, you can comment out the line in mailer.rb to workaround.

#Net::SMTP.enable_tls(OpenSSL::SSL::VERIFY_NONE)

Integrate Metric_fu

Install metric_fu and add the task to the build script. Below is a sample script. Be sure to add "test" to include path for rcov, otherwise you'll meet "no such file to load -- test_helper".

require 'metric_fu'

MetricFu::Configuration.run do |config|
  #define which metrics you want to use
  config.metrics  = [:churn, :saikuro, :stats, :flog, :flay, :reek, :roodi, :rcov]
  config.flay     = { :dirs_to_flay => ['app', 'lib']  } 
  config.flog     = { :dirs_to_flog => ['app', 'lib']  }
  config.reek     = { :dirs_to_reek => ['app', 'lib']  }
  config.roodi    = { :dirs_to_roodi => ['app', 'lib'] }
  config.saikuro  = { :output_directory => 'scratch_directory/saikuro', 
                      :input_directory => ['app', 'lib'],
                      :cyclo => "",
                      :filter_cyclo => "0",
                      :warn_cyclo => "5",
                      :error_cyclo => "7",
                      :formater => "text"} #this needs to be set to "text"
  config.churn    = { :start_date => "1 year ago", :minimum_churn_count => 10}
  config.rcov     = { :test_files => ['test/**/*_test.rb', 'spec/**/*_spec.rb'],
                      :rcov_opts => ["--sort coverage", 
                                    "--no-html", 
                                    "--text-coverage",
                                    "--no-color",
                                    "--profile",
                                    "--rails",
                                    "--exclude /gems/,/Library/,spec",
                                    "--include test"]
                    }
end

desc "Special task for running tests on integrity server"
task :ci do
  Rake::Task["db:migrate"].invoke
  Rake::Task["test"].invoke
  Rake::Task["metrics:all"].invoke

  rm_rf "/var/app/integrity/public/[project-name]/metric_fu"
  mv "tmp/metric_fu", "/var/app/integrity/public/[project-name]"
  
  puts "Please visit http://[integrity-server]/[project-name]/metric_fu/output/index.html for metric_fu results"
end

Everything goes well now. Enjoy!