The VO Frontend is, as the name suggests, the user interface of a glidein based WMS.
The main task of the VO frontend is to look for user jobs and ask
the Glidein Factories to provide
glideins, if needed. See the
picture below for a general overview.
The VO frontend knows nothing about the glideins or Grid sites; the only information in has are the attributes that the various Glidein Factories publish. This makes the life of the frontend much easier, as it only has to worry about matching user job requirements to the factory attributes.
The current implementation of the VO frontend is Python
based. It uses a tree of process like the factory does.
The sub-process (group) code is composed of 4 logical pieces:
- An element that holds the Glidein Factory Ads
- An element that holds the user jobs Ads
- A matchmaking procedure
- A procedure that advertises the requests
- ... and the glue that links the above together.
The top two elements simply contact the
appropriate Condor Collector and store the data in memory.
The matchmaking procedure loops over all the factory ads and counts the number of idle user jobs that match the attributes of that particular factory entry point.
The matching expression is a Python expression, having as inputs
an object holding the job attributes, named job, and
an object holding the entry point attributes, named glidein.
Both objects are dictionaries, with keys being attribute names,
and values being attribute values.
A few examples:
job["CDFMaxHours"]<glidein["attrs"]["WMSMaxHours"] job["HasData"] or (glidein["site"] in string.split(job["SitesWithRequiredData"])) (job["Arch"] == glidein["Arch"]) and (job["gccVersion"]==glidein["attrs"]["gccVersion"])
Finally, the advertise procedure loops over the counts and converts
them into appropriate Condor
The glue code is responsible of parsing the input parameters, loading the configuration values and loop over the above steps.
The frontend also can handle multiple proxies. The proxy selection is handled by a plugin mechanism.
The VO Frontend configuration involves creating the configuration directory and files and then creating the daemons. As in the Glidein Factory set up, an XML file is converted into a configuration tree by a configuration tool.
For the installer to create the VO Frontend instance from the configuration directory and grid mapfile, the following objects can be defined:
- <frontend frontend_name="your name" advertise_delay="seconds" loop_delay="nr" >The frontend_name is a combination of the frontend and instance names specified during installation. It is used to create VO Frontend instance directory and files. The delay parameters define how active the VO Frontend should be.
- <frontend><match><factory><collectors><collector my_identity="firstname.lastname@example.org" node="collector1.myorg.com" factory_identity="email@example.com" DN="/DC=org/DC=doegrids/OU=Services/CN=collector/collectory1.my.org"/>This shows what WMS Collector the VO Frontend will map to. It is the mapped name for the identity of the classad. It alos tells what should be the idetnity used by the frontend itself and the expected identity of the factory
- <frontend><match><job query_expr="expression" >A Condor constraint to be used with condor_q when looking for user jobs. (like 'JobUniverse=?=5', i.e. consider only Vanilla jobs). If you want to consider all user jobs, this can be set to TRUE.
- <frontend><match><job><schedds><schedd fullname="schedd name" />When you provide the user pool collector to the installer, it will find all the available schedds. You can specify which schedds to monitor here for user jobs. The schedd fullname is the name under which schedd is registered with the user pool collector.
- <frontend><security proxy_DN="/DC=org/DC=doegrids/OU=Service/CN=frontend/frontend1.my.org" classad_proxy="proxy_dir" proxy_selection_plugin="ProxyAll" security_name="vofrontend1">Grid proxy to use is located in the classad_proxy directory and you must specify the full path to the proxy. security_name signifies the name under which the frontend is registered with the factory.
- <frontend><security><proxies><proxy absfname="proxy" security_class="frontend"/>The location of the proxy the VO Frontend forwards to the factory for use in submitting glideins. You can have multiple proxy entries listed here.
- <frontend><stage base_dir="web_dir" web_base_url="URL" />The location of the web server directories. It is the staging area where the files needed to run you job are located (i.e. condor). You may need to change this according to your requirements.
- <frontend><work base_dir="directory" />This defines the path to the VO Frontend directory.
- <frontend><files><file absfname="filepath" relfname="filename" executable="boolean" />The file parameters are used to specify the location of additional files. The grid mapfile location must be specified. This can be used to add custom scripts to the vo frontend. See below or the page dedicated to writing custom scripts for more information.
- <frontend><attrs><attr name="attr_name" value="value" parameter="True" type="string" glidein_publish="boolean" job_publish="boolean" >The following three attributes are required to be set in the frontend requests and/or published for jobs. Others can be specified as well.
- <attr name="GLIDEIN_Collector" >The contains the name of the pool collector, i.e. mymachine.mydomain.
- <attr name="GLIDEIN_Expose_Grid_Env" >This determines if you want to expose the user to the grid environment.
- <attr name="USE_MATCH_AUTH" >This determines whether or not you want to use match authentication. You specify the match expression in a group section of the config file.
- <attr name="GLIDEIN_Glexec_Use" >This determines whether or not you want to mandate the use of GLEXEC. Possible values are NONE (do not use GLEXEC) or OPTIONAL (use GLEXEC if the site is configured with it) or REQUIRED (Always use GLEXEC). Mandating the use of GLEXEC also enforces the factory to submit jobs to sites that have GLEXEC configured.
Other attributes can be specified as well. They are used by the VO frontend matchmaking and job matchmaking. The format is similar to the attributes on the Factory config file. The table below describes the <attrs ... > tag in more detail.
Name of the attribute
Value of the attribute
If set to True, the attribute will be available in the constants file created in the staging area. Used only if parameter is True.
Set to True if the attribute should be passed as a parameter. Always set this to True unless you know what you are doing.
If set to True, the attribute will be available in the condor_startd's classad. Used only if parameter is True.
If set to True, the attribute will be available in the user job's environment. Used only if parameter is True.
You can specify description of the attribute here.
Type of the attribute. Supported types are 'int', 'string' and 'expr'. Type expr is equivalent to condor constant/expression in condor_vars.lst
An example attribute would be:
The following group parameters are used to configure multiple frontends. If only one group is specified, they apply to all frontends. The objects specified are used for creating and monitoring glideins.
- <frontend><groups><group name="name" enabled="boolean" >This specifies the name of the group and whether it is enabled.
- <frontend><groups><config>The parameters listed here define how many glideins to create and maintain, such as idle_glideins_per_entry and running_glideins_per_entry.
- <frontend><groups><match match_expr="expr" >iA Python boolean expression is used to match glideins to jobs. The glidein and job dictionaries are used in the expression, i.e. glidein["attrs"]["GLIDEIN_Site"] in job["DESIRED_Sites"].split(","). If you want to match all, just specify True.
- <frontend><groups><group><match><factory query_expr="expr" >A Python expression that specifies a factory constraint.
- <frontend><groups><group><match><factory><match attrs><match attr name="name" >A list of glidein factory attributes used in the factory match expression.
- <frontend><groups><group><match><job query_expr="expr" >A Python expression that specifies a job constraint.
- <frontend><groups><group><match><job><match attrs><match attr name="name" >A list of job attributes used in the factory match expression.
<file absfname="script name" executable="True" comment="comment"/>
The script will be copied to the Web-accessible area and added to the glidein's file_list, and when a glidein starts, the glidein startup script will pull it and execute it. If any parameters are needed, they can be specified using <attr />.
For more detailed information, see the page dedicated to writing custom scripts.
You can also create wrapper scripts or tar-balls of files, see the factory entry page for syntax. (Use groups/group tags instead of the factory's entry tag).
Once you have the desired configuration file, move to the VO frontend directory and launch the command:
All the activity messages will go into
while the warnings go into