Jenkins Tutorial

This document describes the process of configuring Jenkins and setting up build slaves on INRIA’s continuous integration platform.

It is not meant to be a comprehensive docementation for Jenkins. If you need further details about Jenkins, then you should consult the official documentation.

Prerequisites

This tutorial assumes that:

• Step 1: you have a user account on https://ci.inria.fr/ and that you created a project (or been invited to an existing project)
• Step 2: you can open remote sessions (ssh/rdesktop) to your slave virtual machines

Connecting to your Jenkins instance

Each project registered on https://ci.inria.fr/ has its own Jenkins instance running on the platform. The users who have Admin rights on the project also have Admin rights on the Jenkins instance.

The jenkins instance of your project can be accessed by to different ways:

• by clicking on the button in the Web Portal
• directly by typing the URL:

https://ci.inria.fr/project_shortname/

If you are working on a qualification instance, this qualification instance will be located at a separate url:

https://ci.inria.fr/project_shortname-qualif/

User Authentication

The credentials to open a session on the Jenkins instance are the same as for the Web Portal:

• User: your e-mail address
• Password: your password on ci.inria.fr

Configuring Jenkins

Modifying the global Jenkins settings is done through the page Manage Jenkins / Configure System

Restrictions in Jenkin’s configuration

Most of Jenkins can be configured to your liking. However, to be properly integrated within the CI platform, there are a few settings that MUST NOT be changed:

• # of executors must be set to zero (build jobs must run on the slaves, not on the master Jenkins instance)
• Access Control must be done through the CI LDAP server
• Jenkins URL must be kept to https://ci.inria.fr/project_name/
• E-mail Notifications must be sent through INRIA’s smtp server smtp.inria.fr

Setting up tools (e.g. JDK, Maven)

Automatic installation

Jenkins can automatically install tools such as JDKs, Ant, or Maven.

Note: in the case of the Oracle JDK, an Oracle account is needed. Jenkins will prompt for credentials. Also, Jenkins versions prior to version 1.464 have an issue which prevents automatic installation of the Oracle JDK.

Manual installation

You may prefer to install these tools manually. In that case Jenkins allows you to enter at which location (on the slaves) these tools are installed.

Note: in any case, there must be at least a JRE available on each slave, so that Jenkins can run its agent software (if you are using a featured template, then the JRE is already installed)

Providing build slaves

Build is performed by build slaves that are controlled by Jenkins.

Builds CAN NOT be performed on the master Jenkins instance

(to help users avoid this mistake, the master Jenkins instances are configured with 0 executors by default).

Depending on your needs you can do it: - automatically using the Web Portal (recommended) - manually using Cloudstack portal (advanced use cases)

Providing build slaves using the CI Portal

The easiest way to add new slaves to you Jenkins instance it to use the web portal to instantiate them from a template (featured templates, community templates or your own templates).

The procedure is mostly automated:

1. Create the slave on the Web Portal (this is already done if you have followed the Web Portal Tutorial)
2. Reboot your Jenkins instance (stop, then start when status is offline the Jenkins instance on the Web Portal)

Providing custom build slaves using the CloudStack portal

If you have specific needs that prevent you from using an existing template, then you will have to do most of the configuration manually:

1. Create the slave on the CloudStack portal
2. Add your new slave into Jenkin’s configuration

Launching Jenkin’s Agent

The list of slaves in Jenkins is displayed below the menu on the left side of the screen.

To control what is happening on its slaves, Jenkins runs an agent program on each machine. These are normally started when Jenkins boots up.

If a slave is marked as (offline) then it means that the agent is not started and that the slave is not currently useable. This happens if the slave was not reachable during Jenkin’s boot or if the connection is lost (in particular the first boot of windows slaves takes a long time, so when you create a Windows slave it will likely be offline when Jenkins finishes its reboot). In that case you will need to start it manually.

To start an agent, click on the offline slave you are interested in. This will open the status page of this slave. Then click on Launch slave agent. If the action fails, check the logs to get the reason of the failure. If the virtual machine is really down, you will need to restart it from the Web Portal.

• If you are facing issues with Linux/OSx virtual machines, please check the ~ci/.ssh/authorized_keys file on the virtual machine. You can retrieve what shall be in this file from the Slave tab of your project on the Web Portal https://ci.inria.fr/project//slaves

• If you are facing issues with windows virtual machines, please visit the FAQ - Jenkins failed to launch a Windows slave

Creating a new job

Overview

A new job can be created selecting “New Job” in the top left menu of the top page.

Three main kinds of job are available:

• typical build server work, such as continuous/official/nightly builds, tests, or some repetitive batch tasks (aka “free-style software project”),

• dedicated Maven 2/3 jobs,

• multi-configuration jobs.

Once the kind of job chosen, several options are proposed:

• build management options (build storage, parametrization, etc),

• advanced options (e.g. dependency management),

• optional SCM, such as CVS or Subversion where your source code resides,

• optional triggers to control when Jenkins will perform builds,

• some sort of build script that performs the build (ant, maven, shell script, batch file, etc.) where the real work happens,

• optional steps to collect information out of the build, such as archiving the artifacts and/or recording javadoc and test results,

• optional steps to notify other people/systems with the build result, such as sending e-mails, IMs, updating issue tracker, etc.

SCM configuration

Let us focus on the configuration of some source code management tools.

git configuration

By default, Jenkins does not provide interface to manage git repository. It is therefore necessary to install a dedicated plugin, namely “Git Plugin”. To install the plugin, go to the plugin configuration page (Manage Jenkins → Manage Plugins → Available), select the “Git Plugin” and click on the install button at the bottom of the page. If you cannot find the plugin in the list, go to the Advanced tab and click the Check now button at the bottom of the page.

Once the installation carried out, you can configure Jenkins. Firstly, a valid url to the target git repository has to be set.

It is advised to specify one or more branches to track in the repository. By default, all branches will be examined for changes and built.

If you want to track only the master branch, it must be configured as follows:

svn configuration

Indicate the repository URL of your subversion repository:

Click on the link to specify different credentials:

Trigger configuration

The Build Triggers section controls when your job is going to be built. If no box are checked, builds will be triggered manually only (when you click on Build now).

The recommended configurations are:

• Trigger builds remotely for continuous integration jobs, i.e. the jobs you want run as soon as a change is committed in the svn/git repository. This configuration provides the best reactivity and avoids loading the servers by pervasive polling.

• Poll SCM for time-consuming jobs that should be run at night (running an extensive test-suite for example).

1. SVN Poll
2. Git Poll

The general recommendation is to keep your continuous integration builds as short as possible (ideally less than 10 minutes), so that developers get feedbacks as quickly and to improve the chances of fixing bugs before they interrupt the other developers.

If your build is too long, you may consider several strategies to improve your feedback loop:

• remove the time-consuming/non-critical tasks and move them into a separate job, which you can run on a best-effort basis or (for heavier jobs) nightly. This will allow spotting critical errors (e.g. compilation error) quickly while still having all regression tests run on a regular basis.

• (for huge projects) split the build into multiple jobs (for example: if your software is made of multiple libraries, you can configure one job for each library)

• (for c++ projects) use pre-compiled headers to speed up the compilation. Note that pre-compiled header may introduce side effects in the build. So you may want to keep a nightly build without pre-compiled headers and advise your team to use pre-compiled headers too.

Build actions

The Buid section configures the actual commands to be run for your job.

Initially the section is empty. You can add a new action by clicking on Add build step. The base action is to run a script (shell script on Linux/MacOS and batch script on Windows). Depending on the plugins installed on your Jenkins instance, you may have more “user-friendly” menus to configure your jobs, depending on the build system you are using (Ant, Maven, ...). Please refer to the documentation of these plugins if you want more details.

Post-build actions

The Post-buid Actions section tells what Jenkins shall do once the job is run. In this section can send e-mail notifications (typically to report a broken build), publish test results, trigger other builds, etc.