Proof of concept cookbook to run test-kitchen inside CircleCI using kitchen-docker gem.
You can use this in your cookbook by using a circle.yml file similar to the following:
machine:
services:
- docker
ruby:
version: 2.3.0
dependencies:
override:
- bundle check --path=vendor/bundle || bundle install --path=vendor/bundle --jobs=4 --retry=3:
timeout: 900
test:
override:
- KITCHEN_LOCAL_YAML=.kitchen.docker.yml bundle exec kitchen test:
timeout: 900
Look below for more complete examples.
The following files will help you understand how this works:
This example cookbook only installs nginx. It also includes some Serverspec tests to check everything is working correctly.
- kitchen-in-travis: Runs test-kitchen inside Travis CI using User Mode Linux, without using the new native Docker service. The build times are longer but more customizable than in
kitchen-in-travis-native
. Recommended if you want to run tests against many instances. For example, to test multiple instances for each build. - kitchen-in-travis-native: Runs test-kitchen inside Travis CI using the native Docker service and kitchen-docker. The builds are faster (~2 mins to start), but a little less customizable than in
kitchen-in-travis
.
First you need to install Docker.
Then you can use bundler to install the required ruby gems:
$ gem install bundle
$ bundle install
$ bundle exec rake
This example will run kitchen with Vagrant in your workstation. You can use $ bundle exec rake integration:docker
to run kitchen with Docker, as in CircleCI.
$ bundle exec rake -T
rake integration:docker[regexp,action] # Run integration tests with kitchen-docker
rake integration:vagrant[regexp,action] # Run integration tests with kitchen-vagrant
First, create a .kitchen.docker.yml
file with the platforms you want to test:
---
driver:
name: docker
platforms:
- name: centos-6.6
run_list:
- name: ubuntu-14.04
run_list:
- recipe[apt]
# [...]
If not defined, it will get the platforms from the main .kitchen.yml
by default.
You can get the list of the platforms officially supported by Docker here.
Then, I recommend you to create a task in your Rakefile:
# Rakefile
require 'bundler/setup'
# [...]
desc 'Run Test Kitchen integration tests'
namespace :integration do
desc 'Run integration tests with kitchen-docker'
task :docker do
require 'kitchen'
Kitchen.logger = Kitchen.default_file_logger
@loader = Kitchen::Loader::YAML.new(local_config: '.kitchen.docker.yml')
Kitchen::Config.new(loader: @loader).instances.each do |instance|
instance.test(:always)
end
end
end
This will allow us to use $ bundle exec rake integration:docker
to run all the tests.
The circle.yml file example:
machine:
services:
- docker
ruby:
version: 2.3.0
dependencies:
override:
- bundle check --path=vendor/bundle || bundle install --path=vendor/bundle --jobs=4 --retry=3:
timeout: 900
test:
override:
- bundle exec rake integration:docker
timeout: 900
If you are using a Gemfile, you can add the following to it:
# Gemfile
group :integration do
gem 'test-kitchen', '~> 1.2'
end
group :docker do
gem 'kitchen-docker', '~> 2.1.0'
end
This will be enough if you want to test only 2 or 3 platforms. If you want more, continue reading:
If you want to test many platforms, you will need to split up the tests in multiple CircleCI builds. For those cases, I recommend you to use a Rakefile Rake task similar to the following:
# Rakefile
require 'bundler/setup'
# [...]
desc 'Run Test Kitchen integration tests'
namespace :integration do
# Gets a collection of instances.
#
# @param regexp [String] regular expression to match against instance names.
# @param config [Hash] configuration values for the `Kitchen::Config` class.
# @return [Collection<Instance>] all instances.
def kitchen_instances(regexp, config)
instances = Kitchen::Config.new(config).instances
return instances if regexp.nil? || regexp == 'all'
instances.get_all(Regexp.new(regexp))
end
# Runs a test kitchen action against some instances.
#
# @param action [String] kitchen action to run (defaults to `'test'`).
# @param regexp [String] regular expression to match against instance names.
# @param loader_config [Hash] loader configuration options.
# @return void
def run_kitchen(action, regexp, loader_config = {})
action = 'test' if action.nil?
require 'kitchen'
Kitchen.logger = Kitchen.default_file_logger
config = { loader: Kitchen::Loader::YAML.new(loader_config) }
kitchen_instances(regexp, config).each { |i| i.send(action) }
end
desc 'Run integration tests with kitchen-vagrant'
task :vagrant, [:regexp, :action] do |_t, args|
run_kitchen(args.action, args.regexp)
end
desc 'Run integration tests with kitchen-docker'
task :docker, [:regexp, :action] do |_t, args|
run_kitchen(args.action, args.regexp, local_config: '.kitchen.docker.yml')
end
end
This will allow us to run different kitchen tests using the $ rake integration:docker[REGEXP]
command. For example we can use $ bundle exec rake integration:docker[ubuntu]
to run only the Ubuntu integration tests.
Then, you can use the following circle.yml file:
machine:
services:
- docker
ruby:
version: 2.3.0
environment:
TESTS: ubuntu centos
dependencies:
override:
- bundle check --path=vendor/bundle || bundle install --path=vendor/bundle --jobs=4 --retry=3:
timeout: 900
test:
override:
- TESTS=(${TESTS// / }) ; bundle exec rake integration:docker[${TESTS[$CIRCLE_NODE_INDEX]}]:
parallel: true # Project Settings -> Tweaks -> Adjust Parallelism -> 2
timeout: 900
This will allow us to configure the integration tests to run in parallel in the TESTS
environment variable.
For this to work, you need to go to Project Settings -> Tweaks -> Adjust Parallelism in the CircleCI dashboard and set the paralellism to the number of tests (2
in our case: ubuntu
and centos
).
You can use more complex regular expressions for the tests such as:
machine:
-- [...]
environment:
TESTS: (centos-6|debian-[67]) (debian-8|oraclelinux-6) ubuntu-1[24]04 (ubuntu-1504|scientific-6)
-- [...]
The example above will run 4
tests in parallel:
- CentOS
6
, Debian6
and Debian7
. - Debian
8
and Oracle Linux6
. - Ubuntu
12.04
and Ubuntu14.04
. - Ubuntu
15.04
and Scientific Linux6
.
Look real-world examples below for more complete examples.
ssl_certificate
cookbook (circle.yml, .kitchen.docker.yml, Rakefile): Runs kitchen tests against many suites and platforms (~40). Includes a Serverspec and bats tests.
Cookbooks requiring systemd may not work correctly on CentOS 7 and Fedora containers. See Systemd removed in CentOS 7.
You can use alternative images that include systemd. These containers must run in privileged mode:
# .kitchen.docker.yml
# Non-official images with systemd
- name: centos-7
driver_config:
# https://registry.hub.docker.com/u/milcom/centos7-systemd/dockerfile/
image: milcom/centos7-systemd
privileged: true
- name: fedora
driver_config:
image: fedora/systemd-systemd
privileged: true
Some cookbooks requiring Ubuntu Upstart may not work correctly.
You can use the official Ubuntu images with Upstart enabled:
# .kichen.docker.yml
- name: ubuntu-14.10
run_list: recipe[apt]
driver_config:
image: ubuntu-upstart:14.10
It's recommended to install net-tools
on some containers if you want to test listening ports with Serverspec. This is because some images come without netstat
installed.
This is required for example for the following Serverspec test:
# test/integration/default/serverspec/default_spec.rb
describe port(80) do
it { should be_listening }
end
You can ensure that netstat
is properly installed running the netstat
cookbook:
# .kitchen.docker.yml
- name: debian-6
run_list:
- recipe[apt]
- recipe[netstat]
If a command can take a long time to run and is very quiet, you may need to run it with some flags to increase verbosity such as: --verbose
, --debug
, --l debug
, ... You can also increase the timeout:
value in the circle.yml file.
Currently I'm using this for my own projects. It may not work correctly in many cases. If you use this or a similar approach successfully with other cookbooks, please open an issue and let me know about your experience. Problems, discussions and ideas for improvement, of course, are also welcome.
This cookbook example does not contain anything new. It is based on Torben Knerr's work on the sample-toplevel-cookbook
cookbook.
Author: | Xabier de Zuazo (xabier@zuazo.org) |
Copyright: | Copyright (c) 2015, Xabier de Zuazo |
License: | Apache License, Version 2.0 |
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.