GlideinWMS The Glidein-based Workflow Management System

Search Results

Glidein Frontend

Installation

1. Description

The Glidein Frontend will match user jobs with glidein factory ads. It is responsible for the submission of new glideins.

The VO frontend also keeps part of the configuration of a glidein, and can also provide the glidein Factory with the pilot proxies.

Note: if you are installing only the frontend and planning to connect to the OSG factory, please refer to this section: Connecting to OSG Factory

2. Hardware requirements

CPUs Memory Disk
1 1GB ~20GB

This machine needs a fast CPU and a moderate amount of memory (1GB should be enough).
The disk needed is just for binaries, config files and log files (20GB should be enough)

3. Needed software

Software Notes Install Before glideinWMS
Linux OS A reasonably recent Condor-supported OS Linux OS (RH/SL4 and RH/SL5 tested at press time). X
Python interpreter v2.3.4 or above X
The perl-Time-HiRes rpm. This rpm may already be included in perl, depending on the perl version X
The OSG client software. This can be installed prior to glideinWMS, but the installer can install it inline with the glideinWMS install  
A HTTP server, like Apache or TUX. This should be installed prior to glideinWMS (see below) X
The Condor distribution as a tarball. The installer will use the tarball to install and configure Condor inline  
The RRDTool package v1.2.18 or later (see additional install notes) X
The M2Crypto python library v0.17 or later (see additional install notes) X
The javascriptRRD package v0.6.3 or later with flot (see additional install notes) X
The glideinWMS software.    

NOTE:

  • Condor version v7.3.1 has a known issue with incorrect return/exit codes of condor_status and condor_q
  • If you are using Condor version v7.3.2 disable VOMS checking in condor_config file used by Condor daemons other than that used by user schedd. VOMS checking adds unrequired overhead. To do so, set
    USE_VOMS_ATTRIBUTES = False
    or for individual condor daemons like collector
    COLLECTOR.USE_VOMS_ATTRIBUTES = False

3.2 HTTP Server

The glidein Frontend needs a HTTP server, like Apache or TUX. The server should be installed on the same node, but a different node can be used as long as the web area is writable from this one. Servers often come pre-installed with HTTP server software, so if you have one running, just reuse it. Otherwise, the installer can help you install one (as root). (See GlideinWMS Component Install)

3.3 RRDTool

You will also need the python module for RRDTool (v1.2.18 or later). Many systems come with packages for it; if possible use that. Otherwise see additional install notes for alternative installs.

3.4 M2Crypto

You will need the M2Crypto python library. A few systems include it in the software distribution; if possible install the system one. Otherwise see additional install notes for alternative installs.

3.5 javascriptRRD

You will need the javascriptRRD package. It contains the javascript libraries needed by the monitoring. Just download the tarball (with flot), and untar it. You will need to point the installer to this directory.

4. Before you begin...

4.1 Required Users

The installer will ask you for several non-privileged users during the install process. These should be created prior to running the glideinWMS installer.

User Notes
Vo Frontend User The Glidein Frontend should be installed as a non-priviliged user.
Note:If you are not re-using an existing condor client and instead installing a condor client specific for the frontend, you should install it as this user as well.

4.2 Required Certificates/Proxies

Each service in the GlideinWMS will use a x509 certificate in order to identify itself using GSI authentication (see the Quick Reference Guide" for an overview. The installer will ask for several DNs for GSI authentication. You have the option of using a service certificate or a proxy. These should be created and put in place before running the installer. The following is a list of DNs the installer will ask for:

  • WMS Collector cert/proxy DN
  • User Pool Collector cert/proxy DN
  • User Submitter cert/proxy DN
  • Glidein Frontend Condor cert/proxy DN (cannot use a cert here)
Note: In some places the installer will also ask for nicknames to go with the DNs. For the most part the name given doesn't really matter. There is one case where is does matter. If you are using privilege separation, then, on the WMS Collector, the nickname for each Glidein Frontend must be the username that you created for the frontend.
Note 2: The installer will ask if these are trusted Condor Daemons. Answer 'y'.

4.3 Required Directories

When installing the Glidein Frontend you will be presented with a question asking for the directory location for various items. The example below puts many of them in /var. All the directories in /var have to be created as root. Therefore, if you intend on using /var, you will have to create the directories ahead of time.

Note: The web data must be stored in a directory served by the HTTP Server.

Example:

Where will the web data be hosted?: [/var/www/html/glidefactory] /var/www/html/glidefactory

4.4 Miscellaneous Notes

At some point the installer will prompt you for the OSG VDT Client location or if you want to install it. The installer will install the client for you. (See GlideinWMS Component Install)

When asked if you want OSG_VDT_BASE defined globally? Answer 'y' unless you want to force your users to find and hard code the location.

By default, match authentication will be used. If you have a reason not to use it, be sure to set to False the USE_MATCH_AUTH attribute in both in both the Factory and Frontend configuration files.

5. Condor installation instructions

The Glidein Frontend will need the the Condor binaries. You can reuse an existing installation, if available, like if you host the Glidein Frontend on the a submitter node.

Else you need to install a local copy. The suggested way is to install as the same non privileged Glidein Frontend user (see below). The whole process is managed by a install script described below.

Move into

glideinWMS/install

and execute

./glideinWMS_install

You will be presented with this screen:

What do you want to install?
(May select several options at one, using a , separated list)
[1] glideinWMS Collector
[2] Glidein Factory
[3] GCB
[4] pool Collector
[5] Schedd node
[6] Condor for Glidein Frontend
[7] Glidein Frontend
[8] Components

Select 6.

Now follow the instructions. Additional description is below:

Field Installation Text Description
Condor Where do you have the Condor tarball?
Where do you want to install it?
Though the frontend does not actually start any Condor processes, it needs a condor installation in order to use Condor tools and commands. For this, you will need a condor distribution and a location to install to. It will also prompt for a administrator email.
GSI Security Where can I find the directory with the trusted CAs? GSI security is based on x509 certificates. First, you will need a list of trusted certificates. VDT comes with a list of certificates, so, if you install that now (or have installed it previously), you can install that now. Note that you may have to update your certificates if you have an old VDT installation.
You will next need a certificate or proxy for the VO frontend. See the previous section for more information on required certificates and proxies.
Collector What node is the collector running (i.e. CONDOR_HOST)? The installer will ask, "What node is the collector running (i.e. CONDOR_HOST)?". The collector referred to by this question is the user pool collector. Answer with the fqdn of the user pool collector. You will need to open all port(s) that the collector will use on the firewall.

6. Glidein Frontend installation

The Glidein Frontend needs a x509 proxy to communicate with the glidein Factory. You need to create such proxy before starting a VO Frontend and then keep it valid for the life of the frontend. If used for job submission (i.e. if it is passed to the glidein Factory), this proxy must at any point in time have a validity of at least the longest expected job being run by the glideinWMS (and not less than 12 hours).

How you keep this proxy valid (via MyProxy, kx509, voms-proxy-init from a local certificate, scp from other nodes, or other methods), is beyond the scope of this document.

The VO frontend can also host the x509 proxies used for glidein submission. If you do use this (recommended) method, you need to keep these proxies valid at all time, as you do for the main frontend proxy.

The Glidein Frontend should be installed as a non privileged user. The provided installer can be used to create the configuration file, although some manual tunning will probably be needed.

Before starting the installation of the Glidein Frontend make sure that the WMS Collector is started and running.

Note that the OSG client is required, but it is not recommended to actually source the setup.sh before installation. This can cause problems with conflicting python versions. VDT sometimes installs its own version of python which will not have rrdtool or M2Crypto. You can determine if this will be a problem by sourcing the VDT setup script and then doing a 'type python' or 'which python' to see which python is being used.

To begin the installation procedure:

cd glideinWMS/install
./glideinWMS_install

You will be presented with the service selection screen. Follow the instructions. Further detail and a walk-through is presented below:
Field Installation Text Description
javascriptRRD Do you have already a javascriptRRD installation?
Where is javascriptRRD installed?
The javascriptRRD library is required to display statistics for the web-based display. It must be installed before the installation can continue.
Directories Where will you host your config files?
Where will the web data be hosted?
What Web URL will you use?
Where will you host your log files?
You will have to give the directories where the frontend will store its files. By default, some of these directories are in /var, and, if you want to keep them there, you will need to create them first as root. The web data should be stored in a directory served by your web server.
You may want to consider putting them in a directory other than a user home directory.
VO Configuration Give a name to this Glidein Frontend
Give a name to this Glidein Frontend instance?
IMPORTANT: When the installer asks, "Give a name to this Glidein Frontend?", you must provide the name that you gave to the installer for the Factory when it asked, "Frontend security name". Otherwise, the factory will reject requests from the frontend.
WMS Collector What node is the WMS collector (i.e. the gfactory) running?
What is the classad identity of the glidein factory?
What is the WMS collector DN (i.e. subject)?
At this point, the location (fqdn) as well as the DN for the WMS collector is needed. The installer will ask you for the classad identity of the glidein factory. This should be the be the username the factory was installed as. It should be in this format: <username>@<factoryfqdn>. You will need to make sure all ports are open on any firewalls for these machines.
For a visual guide to the configuration options that need to match in the frontend and factory, see this color coded chart.
GSI Configuration Where is your proxy located?
What is the mapped name?
To use the Glidein Frontend you need a valid GSI proxy. Glidein Frontend will use this proxy to talk to the WMS Collector and User Schedd Make sure this DN is in the WMS collector condor_mapfile. The installer will ask, "What is the mapped name?". If you are using privilege separation, answer with the username of the user you created for the VO frontend on the Factory. The answer should have the following format: <username>@<factory fqdn>
Pool Collector What is the pool collector DN (i.e. subject)?
List and secondary pool collector the glideins should use.
This will be used to contact the pool collector to query jobs. You will need to match the DN used in the condor mapfile.
Job monitoring What kind of jobs do you want to monitor?
Give a name to the main group:
This is a condor expression for which jobs you'd like reported in the web-based tool. The default is:
(JobUniverse==5)&&(GLIDEIN_Is_Monitor =!= TRUE)&&(JOB_Is_Monitor =!= TRUE)
This tranlates to all vanilla universe jobs that are not monitoring processes. This should be fine unless a more restrictive set is desired.
Job Matching What expression do you want to use to match glideins to jobs? You will need to specify the selection and matching criteria for the user jobs. The suggested values:
jobs_constraint = (JobUniverse==5)&&(DESIRED_Sites=!=UNDEFINED)
match_string = glidein["attrs"]["GLIDEIN_Site"] in job["DESIRED_Sites"].split(",")

should be fine for some basic matchmatching, providing that your user jobs add

+DESIRED_Sites = "site1,site2,...siteN"
requirements = stringListMember(GLIDEIN_Site, DESIRED_Sites)
Proxy Configuration VO frontend proxy = 'XXXX'
Do you want to use is to submit glideins?
Please add all the proxies that this glidein will use
A proxy is required to submit jobs to the glideins. Find more on this in the previous section.
Other DNs Please add all the DNs that this glidein will connect to These DNs will be put in the condor mapfile so that they are authorized to talk to the frontend. This is for security purposes.
Final Config Do you want to expose the Grid env. to the user jobs?
Do you want to create the Glidein Frontend instance?
Exposing the grid environment to user jobs will export shell variables to the user job running on the glidein. Saying 'n' will keep the environment cleaner. Most configurations use 'y' so the user jobs can use those variables if needed.
Creating the instance will generate scripts and configurations for the VO frontend. Otherwise, you will need to manually run a creation command post-install.
(Look at the glideinWMS manual for more details.)

Here a possible set of answers is presented; your setup will probably be slightly different:

Welcome to the glideinWMS Installation Helper

What do you want to install?
(May select several options at one, using a , separated list)
[1] glideinWMS Schedds and Collector
[2] Glidein Factory
[3] GCB
[4] User Pool Collector
[5] User Schedd
[6] Condor for Glidein Frontend
[7] Glidein Frontend
[8] Components
Please select: 7

The following profiles will be installed:
[7] Glidein Frontend


Installing Glidein Frontend

Do you have already a javascriptRRD installation?: (y/n) y
Where is javascriptRRD installed?: /home/frontend/javascriptrrd-0.6.3
Where will you host your config files?: [/home/frontend/frontstage] /home/frontend/frontstage
Where will the web data be hosted?: [/var/www/html/vofrontend] /var/www/html/vofrontend
What Web URL will you use?: [http://frontend1.my.org/vofrontend/] http://frontend1.my.org/vofrontend
Where will you host your log files?: [/home/frontend/frontlogs] /home/frontend/frontlogs
Directory '/home/frontend/frontlogs' does not exist, should I create it?: (y/n) y

Give a name to this Glidein Frontend?: [myVO-vofrontend1] vofrontend1
Give a name to this Glidein Frontend instance?: [v1_0] v1_0

What node is the WMS collector (i.e. the gfactory) running?: gfactory1.my.org
What is the classad identity of the glidein factory?: [gfactory@gfactory1.my.org] gfactory@gfactory1.my.org
What is the WMS collector DN (i.e. subject)?: "/DC=org/DC=doegrids/OU=Service/CN=gfactory/gfactory1.my.org"

To use the Glidein Frontend you need a valid GSI proxy.
Glidein Frontend will use this proxy to talk to the WMS Collector and User Schedd
Where is your proxy located?: /home/frontend/security/grid_proxy

Using DN '/DC=org/DC=doegrids/OU=Service/CN=frontend/frontend1.my.org'
Make sure this DN is in the WMS collector condor_mapfile

For security reasons, we need to know what will the WMS collector map us to.
It will likely be something like joe@gfactory1.my.org
What is the mapped name?: frontenduser1@gfactory1.my.org


We are done with the WMS collector (i.e. gfactory) config
Let's configure the access to the local (user) pool

Using pool collector collector1.ny.org:9618
What is the pool collector DN (i.e. subject)?: /DC=org/DC=doegrids/OU=Services/CN=collector1.my.org

List and secondary pool collector the glideins should use
instead of the main collector.
(leave empty if you want to use the main collector only)
Valid syntaxes are:
 master1.my.org:9620-9624
 master1.my.org:9619
 master3.my.org
Leave an empty collector name when finished.
Collector name:
The following schedds have been found:
 [1] schedd_jobs1@frontend1.my.org
 [2] schedd_jobs2@frontend1.my.org
 [3] schedd_jobs3@frontend1.my.org
 [4] schedd_jobs4@frontend1.my.org
Do you want to monitor all of them?: (y/n) y
Using ['schedd_jobs1@frontend1.my.org', 'schedd_jobs2@frontend1.my.org', 'schedd_jobs3@frontend1.my.org', 'schedd_jobs4@frontend1.my.org']

What is the DN (i.e. subject) for schedd schedd_jobs1@frontend1.my.org?: '/DC=org/DC=doegrids/OU=Services/CN=schedd1.my.org'

What is the DN (i.e. subject) for schedd schedd_jobs2@frontend1.my.org?: '/DC=org/DC=doegrids/OU=Services/CN=schedd1.my.org'
What is the DN (i.e. subject) for schedd schedd_jobs3@frontend1.my.org?: '/DC=org/DC=doegrids/OU=Services/CN=schedd1.my.org'
What is the DN (i.e. subject) for schedd schedd_jobs4@frontend1.my.org?: '/DC=org/DC=doegrids/OU=Services/CN=schedd1.my.org'
What kind of jobs do you want to monitor?: [(JobUniverse==5)&&(GLIDEIN_Is_Monitor =!= TRUE)&&(JOB_Is_Monitor =!= TRUE)]
Give a name to the main group: [main] main
What expression do you want to use to match glideins to jobs?
It is an arbitrary python boolean expression using the dictionaries
  glidein and job
A simple example expression would be:
  glidein["attrs"]["GLIDEIN_Site"] in job["DESIRED_Sites"].split(",")
If you want to match all (OK for simple setups),
  just specify True (the default)
Match string: [True] glidein["attrs"]["GLIDEIN_Site"] in job["DESIRED_Sites"].split(",")

What job attributres are you using in the match expression?
I have computed my best estimate for your match string,
please verify and correct if needed.
Job attributes: [DESIRED_Sites] DESIRED_Sites
What glidein/factory attributres are you using in the match expression?
I have computed my best estimate for your match string,
please verify and correct if needed.
Factory attributes: [GLIDEIN_Site] GLIDEIN_Site
My DN = '/DC=org/DC=doegrids/OU=Service/CN=frontend/frontend1.my.org'

VO frontend proxy = '/home/sfiligoi/.globus/x509_service_proxy'
Do you want to use is to submit glideins: (y/n) [y] n
You may want to use other proxies
Please add all the proxies that this glidein will use

An empty entry means you are done.
proxy fname: /home/frontend/security/x509_glidein_proxy1
proxy fname: /home/frontend/security/x509_glidein_proxy2
proxy fname: /home/frontend/security/x509_glidein_proxy3

proxy fname:
My DNs = ['/DC=org/DC=doegrids/OU=Service/CN=pilot1/myVO.my.org', '/DC=org/DC=doegrids/OU=Service/CN=pilot2/myVO.my.org', '/DC=org/DC=doegrids/OU=Service/CN=pilot3/myVO.my.org']

You will most probably need other DNs in the glidein grid mapfile
Please add all the DNs that this glidein will connect to
(usually the Pool Collector and the Submit nodes)

Please insert all such DNs, together with a user nickname.
An empty DN entry means you are done.

DN: /DC=org/DC=doegrids/OU=Services/CN-collector1.my.org
nickname: [condor001] collector

DN: /DC=org/DC=doegrids/OU=Services/CN=schedd1.my.org
nickname: [condor002] schedd1

DN (leave empty when finished):

Do you want to expose the Grid env. to the user jobs?: (y/n) y
Do you want to create the Glidein Frontend instance (as opposed to just the config file)?: (y/n) [n] n
To create the Glidein Frontend instance, you need to run
/home/frontend/glideinWMS/creation/create_frontend

Configuration files are located in /home/frontend/frontstage/instance_v1.cfg
Remember to set X509_USER_PROXY before starting the daemon
    

If you followed the example above, you ended up with a configuration file in

/home/frontend/frontstage/instance_v1.cfg/frontend.xml

Edit this file to suit your needs and than create the frontend instance with:

/home/frontend/glideinWMS/creation/create_frontend /home/frontend/frontstage/instance_v1.cfg/frontend.xml

6.1. Starting the Glidein Frontend

Use the startup script:

source $VDT_LOCATION/setup.sh
export X509_USER_PROXY=<Proxy that this frontend instance should use>
cd <install dir>
./frontend_startup start

The same script can be used to stop, reconfig and restart the Glidein Frontend.

6.2. Reconfiguring the Glidein Frontend

The files in the frontend working directory must never be changed by hand after the directory structure has been created.

The proper procedure to update the frontend configuration is to make a copy of the official configuration file (i.e. frontend.xml) as a backup. Then edit the config file and run

<frontend working directory>/frontend_startup reconfig config_copy_fname

This will update the directory tree and restart the frontend and group dameons. (If the frontend wasn't running at the time of reconfig it will only update the directory tree.)

Please notice that if you make any errors in the new configuration file, the reconfig script will throw an error and do nothing. If you executed the reconfig command while the frontend was running, it will revert to the last config file and restart with those settings. As long as you use this tool, you should never corrupt the installation tree.

The frontend_startup script contains a default location for the frontend configuration and is set to the location used for the initial install. This allows you to not have to specify the config location when doing a reconfig. To change the default location in the file, run the command:

<frontend working directory>/frontend_startup reconfig config_copy_fname update_default_cfg

NOTE: The reconfig tool does not kill the frontend in case of errors. It is also recommended that you disable any groups that will not be used. Never remove them from the config file.