LHCbPR

LHCb Progress & Regression is a web based client designed to provide a means of easily comparing different builds of LHCb software. As part of the contribution made by Birmingham, we are responsible for maintaining some of the benchmark tests used to monitor the variation in Geant4 and Gauss simulations.

Development

Using lb-dev to Develop LHCb Software

LbLogin -c <any platform the nightly was built on> 
lb-dev --nightly-cvmfs --nightly <nightly name> [day] Gauss/<name of version in nightly> [--name]
git lb-use Gauss
git lb-checkout Gauss/<name of Git branch> <package>  
# Make Changes to Package  
make -j8  
# Test changes! If they work and you're happy with them  
git add /all/the/files/I/worked/on  
git commit -m "Updated blah..."  
git lb-push <package> <JIRAID>.<Description>

Note: Do not use '*' when adding files to a commit! This results in other files you did not develop also being added.

As an example case let's say we would like to make changes to the master version of Sim/SimChecks package within Gauss, and we would like to work with the nightly lhcb-gauss-dev that has built Gauss/HEAD using x86_64-slc6-gcc62-opt:

 LbLogin -c x86_64-slc6-gcc62-opt 
lb-dev --nightly-cvmfs --nightly lhcb-gauss-dev Gauss/HEAD 
cd GaussDev_HEAD 
git lb-use Gauss 
git lb-checkout Gauss/master 
# code, code, code... 
git add Sim/SimChecks/a/file 
git commit -m "Updated file" 
git lb-push LHCBGAUSS-XXYY.FileUpdate

Building a Full Package

Sometimes development will require compiling a whole package, for example some changes to C++ code require a full compilation, i n order to obtain a full version of Gauss you will need to use the lbn-install command specifying the nightly slot you require from https://lhcb-nightlies.cern.ch/nightly/summary/.

In the following example we shall obtain the Wednesday build of Gauss/HEAD from the lhcb-gauss-dev slot under the x86_64-slc6-gcc62-opt platform:

# Make a Directory to Work In  

mkdir Gauss_Sandbox 

cd Gauss_Sandbox 

# Obtain a nightly build version of Gauss from a given slot 

lbn-install --insecure --projects=Gauss --platform=x86_64-slc6-gcc62-opt lhcb-gauss-dev Wed 

# Use the environment  

lb-run --no-sp --dev-dir=/afs/cern.ch/lhcb/software/DEV/nightlies -c x86_64-slc6-gcc62-opt --user-area=lhcb-gauss-dev/Wed GAUSS/HEAD bash --norc

Next we need to set the environment variables so CMake/CMT is able to run the compile process, at the time of writing this information was available within the LHCbNightlies? page
by clicking on the build sub-heading for the relevant build of Gauss and then Show details of environment under the Environment sub-heading.
Copy both the CMTCONFIG and CMAKE_PREFIX_PATH variable entries excluding the first term which is the directory in which the nightly was built (not relevant for us!):

export CMTPROJECTPATH=/home/lblocal/cmtuser:/cvmfs/lhcb.cern.ch/lib/lhcb:/cvmfs/lhcb.cern.ch/lib/lcg/releases:/cvmfs/lhcb.cern.ch/lib/lcg/app/releases:/cvmfs/lhcb.cern.ch/lib/lcg/external:$CMTPROJECTPATH 

export CMAKE_PREFIX_PATH=/afs/cern.ch/lhcb/software/DEV/nightlies:/cvmfs/lhcbdev.cern.ch/lib/lcg/releases:/cvmfs/lhcb.cern.ch/lib/lcg/releases:/cvmfs/sft.cern.ch/lcg/releases:/afs/cern.ch/work/m/marcocle/workspace/LbScripts/LbUtils/cmake:/cvmfs/lhcb.cern.ch/lib/lhcb/LBSCRIPTS/LBSCRIPTS_v9r2p4/LbRelease/data/DataPkgEnvs:/cvmfs/lhcb.cern.ch/lib/lhcb/LBSCRIPTS/LBSCRIPTS_v9r2p4/LbUtils/cmake:/cvmfs/lhcb.cern.ch/lib/lhcb/LBSCRIPTS/LBSCRIPTS_v9r2/LbRelease/data/DataPkgEnvs:/cvmfs/lhcb.cern.ch/lib/lhcb/LBSCRIPTS/LBSCRIPTS_v9r2/LbUtils/cmake:/cvmfs/projects.cern.ch/intelsw/psxe/linux/x86_64/2017/vtune_amplifier_xe:/afs/cern.ch/sw/lcg/releases:$CMAKE_PREFIX_PATH

If all has gone well you should then be able to build Gauss!

cd lhcb-gauss-dev/Wed/GAUSS/GAUSS_HEAD 

make -j8

The build process can take anything above 40 minutes to complete.
You can then fetch any remote branches:

git fetch origin

You can then apply certain commits by 'cherry-picking', be warned! You should apply the commits in the order they have been made:

git cherry-pick <commit-id>

Running a test manually

You might want to schedule your test to run to see if the whole chain (test -> handler -> lhcbpr) works without having to wait for whenever your test is scheduled to run in the week. To do this you will need to navigate to https://lhcb-nightlies.cern.ch/periodic/summary/ and ensure you are logged in. You can then click the orange button on the right which says Start new periodic test.

You will then see a drop down menu:

• Slot: Refers to the nightly slot you would like to run the test on, you can find the nightlies slots at https://lhcb-nightlies.cern.ch/nightly/summary/
• Build: This a number which designates which day's nightly build to use, can also be found at the above address
• The Rest: The rest of the information can be found in https://gitlab.cern.ch/lhcb-core/LHCbNightlyConf/blob/master/test_schedule2.xml for example (for the muon test here)

From this I can tell:

1. The project is Gauss
2. The platform is x86_64-slc6-gcc62-opt
3. The group is GAUSS-MuonMSc
4. The enviroment is lb-run (You only take the part before the pipe)
5. The runner is lhcbpr
6. The os label is perf

Once all the fields have been filled in you can click start test.

There may be some delay in starting as you will need to wait for a machine to become available but it should be much quicker than having to wait for your scheduled time each week. You can monitor the progress from https://lhcb-nightlies.cern.ch/periodic/summary/ , in progress jobs are highlighted in blue, yellow means test success but handler failure, red means test failed, green means test was successful and handler worked.

Simulation Links

Gauss Nightly Build Configurations (Environment Information) https://gitlab.cern.ch/lhcb-core/LHCbNightlyConf/blob/master/Gauss.py

LHCb Progress & Regression (LHCbPR2? ) https://lblhcbpr.cern.ch

Scheduling of the tests in LHCbPR: https://gitlab.cern.ch/lhcb-core/LHCbNightlyConf/blob/master/test_schedule2.xml

LHCb Nightlies https://lbnightlies.cern.ch/

-- Main.Kristian Alexander Zarebski - 16 Feb 2018