CloudStack Tutorial

CloudStack is the cloud middleware used by the Inria Continuous Integration service. It permits to create and administrate virtual machines used as Jenkins/Gitlab runners (i.e., virtual machines for automated builds). This page describes different ways of virtual machines creation with CloudStack from a template or an ISO. This is an alternative much more complicated compared to Virtual Machine creation from the web portal and you may have to deal with a fine tuning of your virtual machine administration.

Advanced Cloudstack users should refer to the CloudStack manuals.

Accessing the CloudStack platform

Just click here to connect to the CloudStack platform.

To identify on the CloudStack platform, you should use the email and the password used on the continuous integration portal as the login/password pair. The CloudStack domain is ci/project-name where project-name is the unix name (aka the short name) of your project on the continuous integration platform.

Once connected to the CloudStack platform, you get access to a control panel to administrate the virtual machines of your project.

Creating a virtual machine from a template

A project is limited in terms of RAM, CPU and Disk size. Please note also that there is a 24 hours delay when deleting a virtual machine (enabling undo) that counts in that limit.

The easiest way to create a virtual machine is to use a template. A template is a preconfigured machine that you simply duplicate so no operating system installation is needed. In this section, we go through the steps needed to create a virtual machine from a template.

First, go to Compute/Instances section

Click on the button

You should get this dialog window:

1. Ignore this step

   

2. select the desired template. The template in the Community tabs are the one provided by the users of the continuous integration service and probably the ones you want to select

   

3. select the size of your VM, please select the minimum amount as much as possible to avoid useless resources consumption

   

4. if you need extra disk space (by default, 20Gb disks are created but a part of it is used by the operating system), you can select some

5. Ignore this step

   

6. Ignore this step

   

7. give a name for your VM. Note: your VM name must be prefixed with your project name → projectname-VM_name

8. click on Launch Virtual Machine button

You can see a notification appears in the upper-right corner. Clicking on, you have a small window with the list of all current notifications

Note: As some operations on virtual machines can be quite long, notifications are quite useful to check the status of it.

This VM will also appear in the Compute/Instances. When the creation is over, you can administrate it (Stopping / Deleting / Starting / …) by clicking on the following buttons.

opens a web terminal on the VM.

If you click, on the VM name, a new page displays all the VM information.

If you stop the VM, you will be able to change the service offering for the VM (scale VM button)

If you wish to use the newly created VM in Jenkins please read providing custom VM to Jenkins.

Creating a virtual machine from an ISO

To create a virtual machine from an ISO, you should first upload an ISO. Select Images/ISOs in Cloudstack left menu. Users can upload their own ISOs based on a URL or a local file.

Note: it is a good practice to prefix your ISO name with your project name → projectname-iso_name, especially if the ISO needs to be public.

ISO upload from a URL

Click on .

.

Don’t forget to fill the operating system type before clicking on OK.

ISO upload from a local file

Click on .

.

Don’t forget to fill the operating system type before clicking on OK.

ISO status

Your ISO will start getting downloaded and you will be able to track its status in the ISO View panel:

Virtual machine creation

First, go to Compute/Instances section

Click on the add instance button .

You should get this dialog window:

1. Ignore this step

   

2. Select the desired ISO. By default, your ISO will be in My templates tab, in Community tab if you made it public, or in Featured tab if you set it featured (requires admin privileges) Important: Select VM ware hypervisor only for OSx Virtual machine (not for linux or windows ones)

   

3. Select the size of your VM, please select the minimum amount as much as possible to avoid useless resources consumption

   

4. Take into account that a part of your disk will be used by the operating system

   

5. Ignore this step

   

6. Ignore this step

   

7. Give a name for your VM. Note: your VM name must be prefixed with your project name → projectname-VM_name

8. Click on Launch Virtual Machine button

The only change with the way you create a VM from a Template are:

• Choose ISO in the first step
• Select the desired ISO in the second step
• Select a disk size in the fourth step, you cannot choose no disk because no disk is provided with the ISO contrary to a template.
• Select the correct hypervisor (VM ware are limited and reserved for OSx)

After that, your VM will be started with the ISO and you can do the installation of the operating system using the console .

Once the installation is finished, you can detach the ISO using the “Detach ISO” button in the VM control panel.

If you wish to use the newly created VM in Jenkins please read providing custom VM to Jenkins.

Saving your Virtual Machine

A word about the template & snapshots backup policy

If your VM needs a subtancial amount of work to setup, it is advised to create template for it, may it be public or private. It will then be easy to replicate or re-create other VM in minutes from the template. If the “featured templates” provided fulfill your needs, or if it requires only a small effort to setup, then the template creation might not be justified.

Only the templates and snapshots are automatically saved every day. The VM themselves are not, since they can be easily restored by the templates (the featured templates, or your owns).

Creating a template

To create a template, first prepare the VM you want to turn into a template (installation, configuration, etc...). Then stop it with the button in the Compute/Instances view and confirm the popup message (can be forced).

Once the VM status is stopped, you can go to the Storage/Volumes view and select your VM volume drive, you will get the following screen:

Click on the “Create Template” button and you will get the template creation dialog:

Select a name, a description and the corresponding operating system. Select Public if you wish to make your template available in the community tab of the template selection.

Click on OK and wait for the notification that the template is created from the volume. Then you should see your template in the Images/Templates view.

Taking snapshot

The data of a volume (a disk storage, or system disk) can be backed up at a particular state by taking a snapshot of it.

• The “Take Snapshot” button let you take a individual snapshot of the VM.
• the “Setup recurring Snapshot” button let you program automatic recurring snapshot.

Note that the first (manual) approach gives you the possibility to stop the virtual machine before taking its snapshot (recommended), while the second one will take the snapshot while the virtual machine is running, which may not be as robust.

A snapshot can then be restored in two ways:

• Create a volume from it.
• Create a template from it.

Add a storage disk to an existing virtual machine

It is possible to create an new storage disk, and add it to an existing virtual machine.

1. In Storage/Volumes Create a new volume clicking on
2. Attach the new disk to the virtual machine instance clicking on
3. Start and stop the virtual machine instance, so that the new disk is detected. (restart is not enough).
4. Then it’s up to the user to make the new disk being mounted automatically. On GNU/Linux i.e., the device path is /dev/vdb or another path found by lsblk --fs   (you can follow this tutorial for instance to format and mount it - Do not use GParted but the command line instead)

Formatting and using a disk for a MacOSX VM

Prerequisites: creating and attaching a disk with CI portal (extra disk space) or by CloudStack portal as explained above.

1. Find the disk device with diskutil list, by example /dev/disk0 ;
2. Format and mount the disk on /Volumes/builds:

diskutil eraseDisk JHFS+ builds /dev/disk0

3. Copy the initial /builds on /Volumes/builds/:

rsync -a /builds/ /Volumes/builds

4. Mount automatically the new disk partition /dev/disk0s2 on /builds, you need to know Volume UUID to add e line in /etc/fstab:

diskutil umountDisk /dev/disk0
diskutil info disk0s2 | grep 'Volume UUID'
   Volume UUID:              <YOUR-VOLUME-UUID>
sudo EDITOR=nano vifs
## Add the line below at the end of fstab:
UUID=<YOUR-VOLUME-UUID> /builds hfs rw
## You need to reboot because mount -a doesn't work
sudo reboot

You can use this procedure for Unix VM replacing diskutil by equivalent in your system.

Next Step

Once your virtual machines are correctly instantiated and tuned, you can proceed to the next step:

→ Virtual Machines Access Tutorial

This part will teach you how to open a remote session (through a SSH session or a remote desktop) on your virtual machine. This will allow you to handle the administration tasks (change the passwords, install new packages, ...).