The VO Frontend

Description

The VO Frontend is, as the name suggests, the user interface of a glidein based WMS.

Index

• Overview
• Implementation
• The VO Frontend management
• Starting a VO Frontend Daemons
• Reference

Overview

The main task of the VO frontend is to look for user jobs and ask the Glidein Factories to provide glideins, if needed. See the picture below for a general overview.

The VO frontend knows nothing about the glideins or Grid sites; the only information in has are the attributes that the various Glidein Factories publish. This makes the life of the frontend much easier, as it only has to worry about matching user job requirements to the factory attributes.

Implementation

The current implementation of the VO frontend is Python based. It uses a tree of process like the factory does.

The sub-process (group) code is composed of 4 logical pieces:

• an element that holds the Glidein Factory Ads,
• an element that holds the user jobs Ads,
• a matchmaking procedure,
• a procedure that advertises the requests, and

• the glue that links the above together.

The top two elements simply contact the appropriate Condor Collector and store the data in memory.

The matchmaking procedure loops over all the factory ads and counts the number of idle user jobs that match the attributes of that particular factory entry point.

The matching expression is a Python expression, having as inputs

• an object holding the job attributes, named job, and 

• an object holding the entry point attributes, named glidein.

Both objects are dictionaries, with keys being attribute names, and values being attribute values.

A few examples:

job["CDFMaxHours"]<glidein["attrs"]["WMSMaxHours"]
job["HasData"] or (glidein["site"] in string.split(job["SitesWithRequiredData"]))
(job["Arch"] == glidein["Arch"]) and (job["gccVersion"]==glidein["attrs"]["gccVersion"])

Finally, the advertise procedure loops over the counts and converts them into appropriate Condor Class Ads.

The glue code is responsible of parsing the input parameters, loading the configuration values and loop over the above steps.

The frontend also can handle multiple proxies. The proxy selection is handled by a plugin mechanism.

The VO Frontend management

The VO Frontend configuration involves creating the configuration directory and files and then creating the daemons. As in the Glidein Factory set up, an XML file is converted into a configuration tree by a configuration tool.

For the installer to create the VO Frontend instance from the configuration directory and grid mapfile, the following objects can be defined:

• <frontend frontend_name="your name" advertise_delay="seconds" loop_delay="nr" >
  The frontend_name is a combination of the frontend and instance names specified during installation. It is used to create VO Frontend instance directory and files. The delay parameters define how active the VO Frontend should be.

• <frontend><match><factory><collectors><collector classad_identity="joe@collector1.myorg.com" node="collector1.myorg.com" />
  This shows what WMS Collector the VO Frontend will map to. It is the mapped name for the identity of the classad.

• <frontend><match><job query_expr="expression" >
  A Condor constraint to be used with condor_q when looking for user jobs. (like 'JobUniverse=?=5', i.e. consider only Vanilla jobs). If you want to consider all user jobs, this can be set to TRUE.

• <frontend><match><job><schedds><schedd fullname="schedd name" />
  When you provide the user pool collector to the installer, it will find all the available schedds. You can specify which schedds to monitor here for user jobs. The schedd fullname is the name under which schedd is registered with the user pool collector. 

• <frontend><monitor base_dir="web_dir" flot_dir="web_dir" javascriptRRD_dir="web_dir" jquery_dir="web_dir" />
  The base_dir defines where the web monitoring is. The other entries point to where javascriptRRD, Flot and JQuery libraries are located.

• <frontend><security classad_identity="joe@collector1.myorg.com" classad_proxy="proxy_dir" proxy_selection_plugin="ProxyAll" >
  The classad identity the VO Frontend will use to talk to the WMS Collector and User Pool Collector. It is located in the classad_proxy directory and you must specify the full path to the proxy.

• <frontend><security><proxies><proxy absfname="proxy" />
  The location of the proxy the VO Frontend forwards to the factory for use in submitting glideins. You can have multiple proxy entries listed here.

• <frontend><stage base_dir="web_dir" web_base_url="URL" />
  The location of the web server directories. It is the staging area where the files needed to run you job are located (i.e. condor). You may need to change this according to your requirements.

• <frontend><work base_dir="directory" />
  This defines the path to the VO Frontend directory.

• <frontend><attrs><attr name="attr_name" value="value" parameter="True" type="string" glidein_publish="boolean" job_publish="boolean" >
  The following three attributes are required to be set in the frontend requests and/or published for jobs. Others can be specified as well.

  • <attr name="GLIDEIN_Collector" >
    The contains the name of the pool collector, i.e. mymachine.mydomain.

  • <attr name="GLIDEIN_Expose_Grid_Env" >
    This determines if you want to expose the user to the grid environment.

  • <attr name="USE_MATCH_AUTH" >
    This determines whether or not you want to use match authentication. You specify the match expression in a group section of the config file.

• <frontend><files><file absfname="filepath" relfname="filename" executable="boolean" />
  The file parameters are used to specify the location of additional files. The grid mapfile location must be specified.

The following group parameters are used to configure multiple frontends. If only one group is specified, they apply to all frontends. The objects specified are used for creating and monitoring glideins.

• <frontend><groups><group name="name" enabled="boolean" >
  This specifies the name of the group and whether it is enabled.

• <frontend><groups><config>
  The parameters listed here define how many glideins to create and maintain, such as idle_glideins_per_entry and running_glideins_per_entry.

• <frontend><groups><match match_expr="expr" >
  A Python boolean expression is used to match glideins to jobs. The glidein and job dictionaries are used in the expression, i.e. glidein["attrs"]["GLIDEIN_Site"] in job["DESIRED_Sites"].split(","). If you want to match all, just specify True.

• <frontend><groups><group><match><factory query_expr="expr" >
  A Python expression that specifies a factory constraint.

• <frontend><groups><group><match><factory><match attrs><match attr name="name" >
  A list of glidein factory attributes used in the factory match expression.

• <frontend><groups><group><match><job query_expr="expr" >
  A Python expression that specifies a job constraint.

• <frontend><groups><group><match><job><match attrs><match attr name="name" >
  A list of job attributes used in the factory match expression.

Starting a VO Frontent Daemons

Once you have the desired configuration file, move to the VO frontend directory  and launch the command:

./frontend_startup start

group_*/log/frontend_info.<date>.log

group_*/log/frontend_err.<date>.log<

The frontend logs are deleted after a week.