GQSub - Simplified Submission

A simplified grid submission system, based on the work conducted at the University of Glasgow.

Introduction

The University of Glasgow have spent some time developing wrapper scripts for grid job submission. These scripts have been designed to mimic the familiar torque tools qsub and qstat.

Submitting a simple job

This example assumes that you have already applied for, and installed, a valid grid certificate. It also assumes that you are member of a supported Virtual Organisation.

This example will submit the following simple batch script to the grid. Note that this script could just as easily be submitted to a local torque cluster.

######################
# A simple batch script      
######################

#!/bin/bash

#PBS -l cput=00:30:00

date
pwd

######################
# End of script                    
######################

Note that this script specifies a maximum CPU time limit of 30 minutes ( PBS -l cput=00:30:00). Including this directive is a requirement of the gqsub script - jobs will cannot be submitted to the grid without specifying a maximum CPU time.

Start by setting up a Grid User Interface. This can be achieved on the Particle Physics system by simply opening an SL4/5 terminal window and typing the command source lcguisetup. On BlueBEAR this can be achieved by using the command source /apps/hep/lcgui/lcguisetup.

Before submitting your very first job, you should setup a valid grid proxy for your VO with the command voms-proxy-init --voms _your_vo_. The gqsub command will normally take care of proxies for you by using the default VO specified in ~/.gqsubrc. If this file doesn't exist, it will be created, taking the default VO as being the first VO listed in any existing proxy.

A job can be submitted using the command gqsub _script_name_. The gqsub command will package up your job, create a JDL file automatically, and submit your job to the grid. Note that you may be asked for your grid certificate password if it detects that your proxy does not have sufficient time left.

Once the job has been submitted, you can check on it's progress using the gqstat command. This will print out details of the status of all submitted jobs. Once a job has completed, the stdout and stderr files will be retrieved automatically by gqstat, and deposited into a directory with a random string name (eg cjc_asqhknFN243£52).

Using the Sandbox

Sometimes it's desirable to send files with a grid job to a worker node. Providing that the total size of the files is small (~20 MB), they may be staged in via the grid sandbox. In order to specify an input file, the directive:

#GQSUB -W stagein=myinputfile.dat

should be added to the batch script. Note that one directive must be added for each file, so if lots of files are required then they should be transferred in a tar ball.

It is also possible to transfer small output files back via the sandbox, using the directive:

#GQSUB -W stageout=myoutputfile.dat

Again note that only small volumes of data (~20MB) should be transferred this way. Larger volumes should be transferred via a grid SE.

Submitting to Other Sites

By default, jobs are submitted to the Birmingham BlueBEAR Grid Cluster. However, it is possible to submit jobs to other grid sites. In order to do this, the directive:

#GQSUB -q serv07.hep.phy.cam.ac.uk:2119/jobmanager-lcgcondor-calice

must be added to your batch script, specifying a valid Computing Element. In this example, the job will be send to the Cambridge. Note that it is only possible to submit jobs to Computing Elements which support your VO. These CEs can be identified using the command:

lcg-infosites --vo voname ce

VO Software

If your VO manages software installation centrally, it will be installed in a directory specified by the environment variable $VO_XXX_SW_DIR, where XXX is the VO name (for example, $VO_ATLAS_SW_DIR). If software is not managed by the VO, it is assumed that users will manage their own software, staging it in or downloading from the SE as required.

More information

More information about the gqsub tools can be found on the University of Glasgow project pages.

-- ChristopherCurtis - 26 May 2010


This topic: Computing > WebHome > LocalGridTopics > Gqsub
Topic revision: r3 - 01 Jun 2010 - _47C_61UK_47O_61eScience_47OU_61Birmingham_47L_61ParticlePhysics_47CN_61christopher_32curtis
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback