Friday, 8 April 2016

Best Practices of Continuous Integration and Testing

A good video summary of what CI is, can be found here

The Practices of Continuous Integration include the following 11 detailed in Martin Fowler's
Practices of Continuous Integration
This page of notes is still being compiled asat: 8 Apr 2016 

Practice 1Maintain a Single Source Repository

Using Version Control Systems (VCS') such as ...

Google Trends - VCS ToolsGoogle Trends - VCS

Practice 2Automate - Version-Control / Build / Deploy / Test

Using CI tools such as

  • Anthill: Now distributed by IBM and implemented in Java. It has a proprietary license.
  • Bamboo: Atlassian’s CI server isn’t open source, but the Java code is provided to customers for free. It is also free for open source and community projects to use.
  • Buildbot: An open source offering - Brian Warner’s Python based CI tool.
  • Continuum: An Apache build tool developed in Java.
  • Cruise Control: First shipped in 2001, CruiseControl can fairly claim to be the server that started it all. It was originally developed by employees at ThoughtWorks, but is now an independent project. It's open source, and has versions written in Java, C# and Ruby.
  • easyCIS: Freeware CI server developed by Václav Zahradník on top of the .NET framework.
  • FinalBuilder Server/Continua CI: CI server from VSoft written in C#.
  • Go: ThoughtWorks' continuous delivery tool written in Java.
  • Gump: The Apache Software Foundation's first CI tool. Written in Python it supports Apache Ant and Maven, amongst others.
  • Hudson: Java CI server now managed by the Eclipse Foundation.
  • Jenkins: An open source tool written in Java. The project was forked from Hudson after a dispute with Oracle and is now led by CloudBees.
  • Mojo: OpenMake’s free Java-based CI server.
  • Parabuild: Java-based server from Viewtier Systems.
  • Pulse: Java-based CI server from Zutubi.
  • QuickBuild: PMEase’s Java-based CI tool.
  • Sin: CI tool written in C# by Casper Hornstrup. It supports Subversion and is licensed with GPL.
  • TeamCity: Java-based CI server from JetBrains, makers of IntelliJ.
  • Team Foundation Server: Microsoft’s CI sever, widely used by .NET developers.
  • TravisCI: Hosted continuous integration service for the open source community.
  • Zed: Java-based CI Server from Hericus Software.

Controlling tools to:


  • Maven - is a "build management tool" (orchestrator) for defining how your .java files get compiled to .class 'es, packaged into .jar (or .war or .ear) files, then (pre/post)processed with tools to manage the CLASSPATH, and all sorts of other tasks required to build a project.
  • Ant:
  • Ruby:
  • Rake:
  • Powershell:
  • WinAutomation:
The following is an example hypothetical build process you might use with a simple software system
  1. Get the source. You may need to download or fetch the source from a source code repository. For this, you might need to know the tag or version of the source code you want to build.
  2. Prepare a build area. You will probably want to create a set of directories, perhaps according to some standardized directory layout.
  3. Configure the build. In this step, you will determine what optional components can be built based on the current environment. You might want to set build numbers and version numbers to be included in the build.
  4. Validate the source code. You may have a standard style guide and you wish to ensure all code conforms to this before you build a release.
  5. Compile the source code
  6. Build the compiled code into libraries potentially including non-code resources such as properties, images and sound files.
  7. Run the system's tests to validate the build.
  8. Build the documentation for the software. This may range from something as simple as collecting text files up to processing content through some form of publishing system to produce the documentation in its final form
  9. Package up all of the components of the software – code, resources, images, documentation, etc. – into a deployable package. You might need to produce several packages in different formats for different target users
  10. Deploy the software to some standard location for use or distribution


  • Powershell
  • Octopus: normal
  • Ansible: a radically simple IT automation engine
    • Used for cloud provisioning, configuration management, application deployment, intra-service orchestration, and many other IT needs.
    • It uses no agents and no additional custom security infrastructure, so it's easy to deploy - and most importantly, it uses a very simple language (YAML, in the form of Ansible Playbooks) that allow description of automation jobs in a way that approaches plain English.
    • Ansible manages Linux/Unix machines using SSH by default.
    • From version 1.7, Ansible also supports Windows machines. Native PowerShell remoting is used, rather than SSH.
      Ansible will still be run from a Linux control machine, and uses the “winrm” Python module to talk to remote hosts.
  • dssd


  • Ranorex
  • BOLD: normal
  • BOLD: normal
Jenkins Tutorial videos
Jeff Jantz - Continuous Integration with Jenkins on Amazon EC2 [1 / 5]
Jeff Jantz - Installing and configuring a Jenkins CI Server on an Amazon EC2 [2 / 5]
Jeff Jantz - Configuring Jenkins on Amazon EC2 [3 / 5]
Jeff Jantz - Continuous Integration with Jenkins on Amazon EC2 [4 / 5]
Jeff Jantz - Continuous Integration with Jenkins on Amazon EC2 [5 / 5]
Sriram Gunda - Installing Jenkins as a Windows Service

TeamCity Tutorial videos
TeamCity UserGuide - Introduction [1 / 9]
TeamCity UserGuide - Web Interface Overview [2 / 9]
TeamCity UserGuide - Build Configurations [3 / 9]
TeamCity UserGuide - Build Results [4 / 9]
TeamCity UserGuide - Working with Tests [5 / 9]
TeamCity UserGuide - Code Analysis [6 / 9]
TeamCity UserGuide - Working with Changes [7 / 9]
TeamCity UserGuide - Personal Builds [8 / 9]
TeamCity UserGuide - Notification and User Profiles [9 / 9]
TeamCity UserGuide - Getting started with TeamCity

Practice 3Make Your Build Self-Testing

Here is just a short excerpt of tools and frameworks:
  • UI tests: Selenium, Robot Framework, Protractor, WinAutomation
  • Service tests: JBehave, FitNesse, SoapUI, JMeter Performance Tests
  • Unit tests: JUnit, TestNG, Mockito (Mock Framework)

Practice 4Everyone Commits To the Mainline Every Day

Practice 5 Every Commit Should Build the Mainline on an Integration Machine

Since branching and merging has become easier with Git, there are various workflows available on how to develop software with Git. The most popular one is called GitFlow.
Here is a list of various Git workflows and comparisons that you can examine, in case you want to learn more on the topic:

Practice 6 –  Fix Broken Builds Immediately

Practice 7 Keep the Build Fast

Practice 8Test in a Clone of the Production Environment

Practice 9Make it Easy for Anyone to Get the Latest Executable

Practice 10Everyone can see what’s happening

Practice 11Automate Deployment

To deploy your software you should use a single deployment script.
This deployment script will be used for deploying to every environment.
As a result you automatically test the script before doing an actual production deployment.