GlideinWMS The Glidein-based Workflow Management System

Components

Prerequisite Component Installation

Prerequisite components

The following is needed by GlideinWMS. The installer will install HTCondor but all others should be installed before you begin. The javascriptRRD directory will also need to be provided to the installer.

Software Version WMS collector Factory User collector Submit Glidein Frontend Comments
GlideinWMS v3.0+ y y y y y  
Linux OS y y y y y A reasonably recent HTCondor-supported Linux OS (RH/SL 6 and 7 tested at this time).
HTCondor v7.8+/
v8.2.3+(BOSCO or Ganglia)
y   y y y The installer will use the tarball or rpm to install and configure HTCondor inline. It is recommended to use v8 and above.

Batch Submission via BOSCO:
If you want to add batch resources (via BOSCO) you need HTCondor v8.2.2+. This version requirement is for the WMS Pool, submitting glideins and the HTCondor client, used to setup the BOSCO resource (bosco_cluster_add). Other components can use previous recommended versions of HTCondor.

HTCondor
monitoring via Ganglia. GlideinWMS provides configuration files for it.
OSG client current version y   y y y Needed to generate GRID/VOMS proxies. See below for instructions.
Python v2.4.3+ y y y y y  
HTTP server   y     y Apache, TUX or whatever server of choice. This port will need open access to the internet, so worker nodes can download data from this service. See below for instructions.
RRDTool V1.2.18+   y     y The perl-Time-HiRes rpm is needed for this package. Please verify all dependencies: rrdtool, perl-rrdtool and python-rrdtool. These packages are available via yum using EPEL in EL6/7 or can be searched in RPMFind.
M2Crypto v0.17+   y     y This can be installed using yum, if available. Or via tarball.
javascriptRRD (with flot) v0.6.3+   y     y Note: JavascriptRRD and flot, are available from different sources however they may not always work when installed separately for GlideinWMS. We only support JavascriptRRD with Flot available from the sourceforge link provided.

1. HTTP server

You will need to install an HTTP server for GlideinWMS to be used for staging and monitoring. This needs to be installed on a port accessible to the outside. The server should be installed on the same node, but a different node can be used as long as the web area is writable from this one. In particular, the staging directory will need to be accessible to glideins so they can download appropriate software.

The choice of web server is independent of GlideinWMS, but this documentation will only show the example of Apache Web Server. For most systems, this can be installed via yum:

        yum install httpd 

Other options are available from Apache here.

There are several recommended tasks you may want to also do post-installation to secure and prepare your web server.

  • Disable all unneeded modules. Glideinwms does not any non-standard modules and many can be disabled. A list of example modules to keep enabled is below: 'access_module','include_module','log_config_module', 'mime_magic_module','expires_module','deflate_module', 'headers_module','setenvif_module','mime_module', 'status_module','autoindex_module','negotiation_module', 'dir_module','imap_module','alias_module','rewrite_module', 'cache_module','file_cache_module','mem_cache_module', 'authz_host_module','authz_default_module'.
  • Disable any (unneeded) extensions in /etc/httpd/conf.d
  • Recommended: disable indexes (Options -Indexes) in all directories/locations, such as frontend/factory stage/monitor directories.

2. OSG Client

The OSG Worker Node Client (formerly known as VDT Client and OSG Client) is now available as a set of RPMs. OSG provides a complete set of instructions that that we recommend. Here you can follow the instructions to install OSG Client on RHEL 6 and 7 based systems (including CentOS and SL). First you will need to enable the EPEL and OSG repositories. It is also advisable to enable the yum priorities module in order to guarantee the correct packages. For more detail, you have everything explained step by step here.

3. RRDTool

You will also need the python module for RRDTool (currently v1.7.0). Many systems come with packages for it; if possible use that.

You can install it via yum. It is available in the Scientific Linux Repository (slf-primary):

 yum install rrdtool rrdtool-python 

If installing as root is not an option, you can build from the sources, as the glidein factory user, and put the python libraries into the PYTHON_PATH.
The source code can be downloaded from here. Beware: The build process has several dependencies that make it complicated. Use this option only if you cannot install from RPMs (or similar).

4. M2Crypto

You will need the M2Crypto python library. However, if you have the OSG repositories enabled, you can install it via yum:

 yum install m2crypto 

If your system does not come with m2crypto package, or you do not can/want install it as root, compile it from source as the glidein factory user.
The source code can be downloaded from here.

The build process is very easy:

python setup.py build
python setup.py install --root <base dir>

Finally, add M2Crypto site-packages into the PYTHONPATH, possibly in the .profile, .bashrc and/or in the .cshrc.
The exact path is system specific. On a 32bit system with python 2.3.x it will look like this:

export PYTHONPATH=$PYTHONPATH:<base dir>/usr/lib/python2.3/site-packages/

5. javascriptRRD

You will need the javascriptRRD package. It contains the javascript libraries needed by the monitoring. If you enabled the OSG repositories (above), you can install it via yum:

 yum install javascriptrrd 

Otherwise, download the tarball (with flot), and untar it. You will need to point the installer to this directory. The installer will copy the library files to the Web directories during the installation or reconfig steps.