This is a guide on the workflow of deployment and development for HammerCloud.

On this page:

1. Introduction

The development and part of the deployment of HammerCloud is based on Git and and internal code review system based on Gerrit. The workflow used for the development is similar to the one used in the OpenStack project, with the difference that, at the moment, there is not continuous integration system for the deployment.

The diagram of the workflow shows better different parts of the development process:

2. Central services

There are two basic central services: the central Git repository, and the Gerrit review system. The interaction between them is the following.

2.1. Initial setup

The git and git-review packages should be installed on the local machine. For instance, for Red Had and Debian based machines, respectively:

# yum install git git-review

# apt-get install git git-review

Also, in order to clone repos and, to make general communication with HammerCloud servers, the CERN CA certificates must be installed.

2.2. Cloning the central repo

In order to clone the central repo, one must have access to the CERN central Git service on https://forge.cern.ch. After, it will be possible to clone the repo:

$ git clone https://git.cern.ch/reps/hammercloud

That will create a folder called hammercloud, with the code inside. There will be a remote, origin, that points to the central repo. In principle, pushing to this repo should not be done.

2.3. Setting up gerrit

There are two ways to setup the Gerrit remote on the recently cloned repo: via HTTPS and via SSH. The recommended way is via HTTPS since the server can be accessed outside the CERN network, however, it needs some tweaks and might be a bit less efficient if using Git clients prior to 1.6.6.

2.3.1 Configuring the Gerrit profile

You will need an account for Gerrit. For the moment, until some issues with CERN SSO and Gerrit authentication integration are solved, we are using plain HTTP authentication, in this way, to request an account you should send an email to ramon.medrano@cernNOSPAMPLEASE.ch with your CERN account username and an Apache encrypted password (we know this sucks and we are improving this area soonish). To generate the Apache password file, you can use:

$ htpasswd -c /tmp/gerritpasswd

Then, attach that file (e.g. /tmp/gerritpasswd) in the email. The file will contain a hash of the password, and not the password itself, which should remain secure.

When you are able to login on the gerrit web site (https://hc-services.cern.cg/gerrit), please fill your profile data and upload a public SSH key that you will use to connect to the server. Please also generate an HTTP password that you will use if you clone the repo via HTTPS on Settings > HTTP Password. This password will only be used by the Git client so you must login on the Web site using the password you sent generated by Apache.

2.3.2 Accessing the Git remote via SSH

This option would be preferred for those on CERN network. We need to add a virtual host on our SSH config (~/.ssh/config):

Host hc-gerrit
     User <your gerrit username>
     Hostname hc-services.cern.ch
     Port 29418
     PubkeyAuthentication yes
     IdentityFile <path to your private key>
     ForwardX11 no
     ForwardX11Trusted no
     GSSAPIAuthentication no
     GSSAPIDelegateCredentials no

Then we can create the Gerrit remote easily by using the .gitreview file included. It can be automatically configured by running this command:

$ git review -s

This will create a remote pointing to the virtual host created on the SSH configuration. Since the port of this server is non standard, access outside CERN is forbidden. Yo can validate the ssh access by:

$ ssh hc-gerrit

  ****    Welcome to Gerrit Code Review    ****

  Hi <your name as set on the Settings>, you have successfully connected over SSH.

  Unfortunately, interactive shells are disabled.
  To clone a hosted Git repository, use:

  git clone ssh://<username>@hc-services.cern.ch:29418/REPOSITORY_NAME.git

Connection to hc-services.cern.ch closed.

This will not work outside CERN's network.

2.3.3 Accessing the Git remote via lxplus

To access the remote outside CERN, we'll need to tunnel it through lxplus, in this way, we can add to our SSH config file (~/.ssh/config) the last line with the ProxyCommand directive, which should make transparent the access to the gerrit server.

Host hc-gerrit
     User <your gerrit username>
     Hostname hc-services.cern.ch
     Port 29418
     PubkeyAuthentication yes
     IdentityFile <path to your private key>
     ForwardX11 no
     ForwardX11Trusted no
     GSSAPIAuthentication no
     GSSAPIDelegateCredentials no
     ProxyCommand ssh lxplus.cern.ch /usr/bin/nc %h %p 2> /dev/null

Now, executing ssh hc-gerrit shoud work perfectly.

3. Workflow

Pushing changes directly to the central Git repo is not allowed, first they have to pass and approve the code revision. For that, we'll use git review.

3.1 Sending changes for review

Develop your changes and commit them. The following will send your commits to the review server:

$ git review 
remote: Resolving deltas: 100% (1/1)
remote: Processing changes: new: 1, refs: 1, done    
remote: 
remote: New Changes:
remote:   https://hc-services.cern.ch/gerrit/XX
remote: 
To ssh://rmedrano@hc-gerrit:29418/HammerCloud.git
 * [new branch]      HEAD -> refs/publish/master

In the server, you shall see now the patch ready to be reviewed. You can add explicitly the reviewers or wait until it gets 2 +1. Then it can be merged.

3.1 Listing the status of our pending changes

This will show the changes you have opened for review.

$ git review -l
5  master  <topic of the change>
Found 1 items for review

3.2 Approving and merging

Once the patchset is reviewed and gains the 2 +1, a button called 'submit' will appear on the web site. Just click it and the code will be merged on the main ref of the review server. The change will be on merged status.

Then, the server itself will get validate the change (in the future, with a CI server passing tests) and will push it to the main repository. For now, this is done automatically, but there is not testing infrastructure yet in place.

3.3 Getting changes from the main repo

Just pull from the origin remote:

$ git pull origin master