LHCbPR < LHCb

---+ !LHCbPR & Test Development

%TOC%

LHCb Progress & Regression is a web based client designed to provide a means of easily comparing different builds of LHCb software. As part of the contribution made by Birmingham, we are responsible for maintaining some of the benchmark tests used to monitor the variation in Geant4 and Gauss simulations. At time of writing, use of the option *--nightly-cvmfs* is now deprecated, nightlies no longer existing on CMFS.
---++ Development
---+++ Using lb-dev to Develop LHCb Software
 <verbatim>LbLogin -c <any platform the nightly was built on>
lb-dev --nightly <nightly name> [day] Gauss/<name of version in nightly> [--name]
git lb-use Gauss
git lb-checkout Gauss/<name of Git branch> <package>  
# Make Changes to Package  
make -j8  
# Test changes! If they work and you're happy with them  
git add /all/the/files/I/worked/on  
git commit -m "Updated blah..."  
git lb-push <package> <JIRAID>.<Description></verbatim> *Note: Do not use '*' when adding files to a commit! This results in other files you did not develop also being added.* <br /><br />As an example case let's say we would like to make changes to the *master* version of *Sim/SimChecks* package within Gauss, and we would like to work with the nightly *lhcb-gauss-dev* that has built *Gauss/HEAD* using *x86_64-slc6-gcc7-opt*:<br /><verbatim>LbLogin -c x86_64-slc6-gcc7-opt 
lb-dev --nightly lhcb-gauss-dev Gauss/HEAD 
cd GaussDev_HEAD  
git lb-use Gauss  
git lb-checkout Gauss/master Sim/SimChecks 
# code, code, code...  
git add Sim/SimChecks/a/file  
git commit -m "Updated file"  
git lb-push LHCBGAUSS-XXYY.FileUpdate</verbatim> 
---+++ Building a Full Package

Sometimes development will require compiling a whole package, for example some changes to C++ code require a full compilation, in order to obtain a full version of Gauss you will need to use the *lbn-install* command specifying the nightly slot you require from https://lhcb-nightlies.cern.ch/nightly/summary/ to fetch a local approporiate version of GEANT4 to build from. Note also that if a special unreleased version of other packages is used you will need to include this in the list of projects also. For example, at the time of writing *lhcb-sim09-cmake* slot required *LHCb/sim09-patches*, as the name suggests this is a special build for testing.
<pre>mkdir Gauss_Sandbox</pre><pre>cd Gauss_Sandbox</pre><pre>lbn-install --nodeps --projects=GEANT4[,other-projects-required] --platform=x86_64-slc6-gcc62-opt lhcb-g4r103 Today</pre><pre>cd lhcb-g4r103/Today </pre> Now we need to obtain a copy of Gauss from GitLab, it is important to know which version you are developing (at the time of writing Sim09 and HEAD were the two main builds):<br /><pre>git clone ssh://git@gitlab.cern.ch:7999/lhcb/Gauss.git GAUSS/GAUSS_HEAD </pre><pre>cd GAUSS/GAUSS_HEAD</pre>

Now we merge in any merge requests we want to include or if you want to merge in changes from another build e.g. changes in the sim09 branch into master) you should cherry-pick these changes to avoid conflicts:

<pre>git config --add remote.origin.fetch +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/* </pre><pre>git fetch --all </pre><pre>git merge --no-ff origin/merge-requests/request-ID</pre><pre>git cherry-pick commit-ID</pre>Now we initialise the project to create a makefile:<br /><pre>lb-project-init</pre>

In order for the build to use the correct versions of required packages we need to edit the CMakeLists.txt file, the information on which versions were used in the nightly can be found in Gauss Nightly Build Configurations <a rel="noreferrer" target="_blank" href="https://gitlab.cern.ch/lhcb-core/LHCbNightlyConf/blob/master/Gauss.py#L113" style="box-sizing: border-box; background-color: #ffffff; color: #2389d7; text-decoration-line: none; word-break: break-word; font-family: 'Open Sans', sans-serif; font-size: 12px; white-space: pre-wrap">https://gitlab.cern.ch/lhcb-core/LHCbNightlyConf/blob/master/Gauss.py</a>, the version of GEANT4 to be used should match the local one you obtained earlier:
<pre>-gaudi_project(Gauss v52r0  </pre><pre>+gaudi_project(Gauss HEAD </pre><pre>                          FORTRAN</pre><pre>  -                        USE LHCb v42r6p2</pre><pre>  -                        Geant4 96r4p7</pre><pre>  +                       USE LHCb v43r1</pre><pre>  +                       Geant4 HEAD</pre><pre><font color="#000000" size="2">in the same location you will find the environment variable information, you will need to set these correctly also: </font></pre><pre>unset VERBOSE </pre><pre>export CMAKEFLAGS="-DCMAKE_USE_CCACHE=ON" </pre><pre>export CCACHE_DIR="$PWD/.ccache"</pre><pre>export CMTPROJECTPATH = /path/to/workdir/Today:/afs/cern.ch/lhcb/software/DEV/nightlies:/cvmfs/lhcbdev.cern.ch/lib/lcg/releases:/cvmfs/lhcb.cern.ch/lib/lcg/releases:/cvmfs/sft.cern.ch/lcg/releases:${CMAKE_PREFIX_PATH}:/afs/cern.ch/sw/lcg/releases:$CMTPROJECTPATH</pre>

Now we can build Gauss:
<verbatim>make -j4 && make unsafe-install && make post-install</verbatim>Now you can edit the software and simply rebuild once you are happy with the changes.

---++ Test Handlers

In order for !LHCbPR to be able to identify what outputs from the performed tests are required to be uploaded to the server for displaying the plots/important data are selected and the packaged via python scripts called Handler files. These exist within the <b>LHCbPR2HD</b> !GitLab repository. Each test has its own handler file containing an instance of the core handler class. Instructions and information are available on the page for the repository found [[https://gitlab.cern.ch/lhcb-core/LHCbPR2HD][here]].

---++ Running a test manually

You might want to schedule your test to run to see if the whole chain (test -> handler -> lhcbpr) works without having to wait for whenever your test is scheduled to run in the week. To do this you will need to navigate to https://lhcb-nightlies.cern.ch/periodic/summary/ and ensure you are logged in. You can then click the *orange* button on the right which says *Start new periodic test*.

<img alt="starttest.png" src="%ATTACHURL%/starttest.png" />

You will then see a drop down menu:

%ATTACHURL%/testform.png

   * *Slot:* <span style="background-color: transparent"> Refers to the nightly slot you would like to run the test on, you can find the nightlies slots at </span><span style="background-color: transparent"> </span>https://lhcb-nightlies.cern.ch/nightly/summary/ 
   * *Build:* This a number which designates which day's nightly build to use, can also be found at the above address 
   * *The Rest:* <span style="background-color: transparent"> The rest of the information can be found in </span><span style="background-color: transparent">https://gitlab.cern.ch/lhcb-core/LHCbNightlyConf/blob/master/test_schedule2.xml </span>for example (for the muon test here) <span style="background-color: transparent; white-space: pre"> </span> 
%ATTACHURL%/testsched.png

From this I can tell:

   1 The project is Gauss 
   1 The platform is x86_64-slc6-gcc62-opt 
   1 The group is GAUSS-MuonMSc 
   1 The enviroment is lb-run (You only take the part before the pipe) 
   1 The runner is lhcbpr 
   1 The os label is perf 

<br />Once all the fields have been filled in you can click *start test*.<br /><br />There may be some delay in starting as you will need to wait for a machine to become available but it should be much quicker than having to wait for your scheduled time each week. You can monitor the progress from https://lhcb-nightlies.cern.ch/periodic/summary/ , in progress jobs are highlighted in blue, yellow means test success but handler failure, red means test failed, green means test was successful and handler worked.

---++ <br /> Simulation Links
 <br />Handlers for !LHCbPR <a rel="noreferrer" target="_blank" href="https://gitlab.cern.ch/lhcb-core/LHCbPR2HD" style="box-sizing: border-box; background-color: #ffffff; color: #2389d7; text-decoration-line: none; word-break: break-word; font-family: 'Open Sans', sans-serif; font-size: 12px; white-space: pre-wrap">https://gitlab.cern.ch/lhcb-core/LHCbPR2HD</a><br />
 <br />Gauss Nightly Build Configurations (Environment Information) <a rel="noreferrer" target="_blank" href="https://gitlab.cern.ch/lhcb-core/LHCbNightlyConf/blob/master/Gauss.py#L113" style="box-sizing: border-box; background-color: #ffffff; color: #2389d7; text-decoration-line: none; word-break: break-word; font-family: 'Open Sans', sans-serif; font-size: 12px; white-space: pre-wrap">https://gitlab.cern.ch/lhcb-core/LHCbNightlyConf/blob/master/Gauss.py</a><br /><br />LHCb Progress & Regression (LHCbPR2) <a rel="noreferrer" target="_blank" href="https://lblhcbpr.cern.ch" style="box-sizing: border-box; background-color: #ffffff; color: #2389d7; text-decoration-line: none; word-break: break-word; font-family: 'Open Sans', sans-serif; font-size: 12px; white-space: pre-wrap">https://lblhcbpr.cern.ch</a><br /><br />Scheduling of the tests in LHCbPR: <a rel="noreferrer" target="_blank" href="https://gitlab.cern.ch/lhcb-core/LHCbNightlyConf/blob/master/test_schedule2.xml" style="background-color: #ffffff; box-sizing: border-box; color: #2389d7; text-decoration-line: none; word-break: break-word; font-family: 'Open Sans', sans-serif; font-size: 12px; white-space: nowrap">https://gitlab.cern.ch/lhcb-core/LHCbNightlyConf/blob/master/test_schedule2.xml</a><br /><br />LHCb Nightlies <a rel="noreferrer" target="_blank" href="https://lbnightlies.cern.ch/" style="background-color: #ffffff; box-sizing: border-box; color: #2389d7; text-decoration-line: none; word-break: break-word; font-family: 'Open Sans', sans-serif; font-size: 12px; white-space: nowrap">https://lbnightlies.cern.ch/</a><br /><br />-- Main.Kristian Alexander Zarebski - 16 Feb 2018
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who
png	starttest.png	r2 r1	manage	108.2 K	28 Feb 2018 - 13:06	UnknownUser
png	testform.png	r3 r2 r1	manage	61.2 K	28 Feb 2018 - 13:10	UnknownUser
png	testsched.png	r1	manage	44.3 K	28 Feb 2018 - 12:33	UnknownUser
This topic: LHCb > WebHome > LHCbPR
Topic revision: r15 - 29 Mar 2019 - _2fDC_3dch_2fDC_3dcern_2fOU_3dOrganic_20Units_2fOU_3dUsers_2fCN_3dkzarebsk_2fCN_3d763242_2fCN_3dKristian_20Alexander_20Zarebski
Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback