GlideinWMS The Glidein-based Workflow Management System

Search Results

Installation

Configuration-Based Installer

Overview

This installer uses a configuration file (ini format) to perform the installation of the glideinWMS services in lieu of a question/answer (q/a) type installer. It requires knowledge and planning of the glideinWMS services just as the the q/a installer did. The major difference is that you essentially pre-answer all the questions previously asked and document them in the ini file. In addition to having them documented, this performs the actual installation faster and allows for re-installs more quickly when some of the attribute values are incorrect (an inevitable consequence of a complex set of services such as these).

Another major difference, with this installer, is that it does NOT modify any scripts that set a user environment upon log in, e.g., .bashrc file, /etc/profile.d files, et al. Instead, an environment script is created for each service in its respective "home" location. If inclusion of these scripts is required at a location, it will need to be performed manually. The only exception to this is when privilege separation is in effect, in which case, the /etc/condor/privsep_config file is created. This location is hard-coded in Condor and cannot be changed. For each of the glideinsWMS services, the scripts for setting the environment are:

  • wmscollector, usercollector, submit: condor_location/condor.sh
  • factory: install_location/factory.sh
  • vofrontend: install_location/frontend.sh

For glideinWMS services using Condor, the CONDOR_LOCATION/config.d directory will contain the Condor attributes required for that service.

manage-glideins script

The script used to perform the installations is glideinWMS/install/manage-glideins.

./manage-glideins --OPTION SERVICE --ini INIFILE [--ssh [user]] [--debug]
    This usage can be used to install, start, stop or check the status of the glidein services based on the configuration in the specified ini file.

    OPTION can be one of:
    - validate: Allows you to validate the ini file prior to installation
    - install: Install the service
    - configure: This allows you to reconfigure your service based on changes to the ini file without re-installing condor.
    For services using Condor, it will update the config.d local config files,
    For the factory and vofrontend, it will update the respective xml config files.
    - start: Start the service. Remote starting of services is possible if remote access (via ssh) is allowed.
    - stop: Stop the service. Remote starting of services is possible if remote access (via ssh) is allowed.
    - status: Return the status of the service. Remote starting of services is possible if remote access (via ssh) is allowed.

    SERVICE can be one of:
    - wmscollector
    - usercollector
    - factory
    - submit
    - vofrontend
    - rpm (Used for OSG Frontend RPM sites. Note: The 'install' action is not allowed for this service.)
    - all: can only be used with start/stop/status actions

    --ssh: allows the start/stop/status actions to be performed remotely providing the user has valid access to the other service's node via 'ssh -l' using the service's username. The '--ssh' will use the ini file specified username attribute unless an optional 'user' is specified.

    --debug: When used with start/stop/status actions, it will display the series of commands used.

./manage-glideins --install-node --ini INIFILE
    This usage allows you to install all services for the node you are installing on. There are some limitation to this.
./manage-glideins --create-entries --ini INIFILE
    This usage can be used to select new glidein entry points after the initial installation of a factory service. If will walk you through the same question and answer process querying ReSS and allowing for manual entries. It will then create a file containing the entry elements for those selected. This can then be merged with the existing Factory configuration file and a reconfiguration performed..
./manage-glideins --create-group --ini INIFILE
    This usage can be used to select new group selection criteria after the initial installation of a frontend service. If will walk you through the same question and answer process as during the installation. It will then create a file containing the group element for the criteria selected. This can then be merged with the existing frontend configuration file and a reconfiguration performed..
./manage-glideins --show-ini INIFILE
    This option allows you to view the ini file options/values. This is especially useful when the DEFAULT section is used to apply values to all sections/services.
./manage-glideins --create-template SERVICE
    This option allows you to create an ini file template for installing a single service. It contains all the required attributes for that service. It should be understood that many of the validations that would normally insure a working installation are bypassed since those validations are normally performed on the node for those services.

glideinWMS ini file

The ini file determines the installation and configuration of the various services. All ini file attributes are required. However, in several cases the value may be left empty. These should be noted in the comments column of the tables.

The configuration based installer requires that the same ini file be used for all service installations. There are several areas where data is required from other services. Since most services can be installed on separate hosts, the installer can only valididate data for the node being installed.

Default Section

The attributes in this section apply to all subsequent sections in the ini file unless they are overridden specifically in that section. So, if the location/value of any option(s) in this section varies from host to host, you will need to override them in that section of the ini file. The only options in the glideinWMS.ini template will be the pacman options in the next section.

Pacman options

The 2 pacman related attributes are used to bring down the OSG/VDT client software and CA certificates if they are not already installed on the node. If you already have the OSG/VDT client and CA certificates installed on a host or you have already installed the CA certificates and are using other non-VDT client software for proxy renewals, then:
 - these options are still required but may be empty, i.e., contain no value.
 - the vdt_location option is still required but may be empty.
 - the install_vdt_client should be set to 'n'.

These are parts of the pacman related options below that should not be changed unless advised to by glideinwms-support as there may be compatiblity issues between pacman and VDT distributions. The comments column will provide the specifics.

Attribute Example Description Comments
pacman_location /path-to-pacman/pacman-3.28 This will be the directory in which pacman is installed.
The base level (e.g., pacman-3.28) will be used to select the pacman tarball from the pacman_url option.
The format is pacman-version.tar.gz.
The tarball will be retrieved using the pacman_url option, extracted and then removed.

You will need to specify the path to the directory, pacman-3.28. If you already have pacman-3.28 installed on your node, the installer will not attempt to bring a new pacman down. You may utitize that directory.
pacman_url http://physics.bu.edu/pacman/sample_cache/tarballs URL to retrieve pacman.
This is the one pacman option you should not change.

WMSCollector section

Under most circumstances, installation of wmscollector using manage-glideins will be need to run as root, since Condor installations require some files to be owned by root in order to properly use privilege separation.
./manage-glideins --install wmscollector --ini glideinWMS.ini
See below for the ini file attributes that installing/running WMS collector requires:

Attribute Example Description Comments
install_type tarball Indicates this is a Condor tarball installation.
At this time, for the WMS Collector, only tarball installations are supported.
Valid values: tarball
hostname wmscollectornode.domain.name hostname for WMS Collector. The WMS collector and Factory must be collocated at this time.
username condor (or whatever non-root user you decide on) UNIX user account that this services will run under. DO NOT use "root". For security purposes, this value should always be a non-root user.
However, if privilege separation is used (see the privilege_separation option), the manage-glideins script itself will need to be run as root (see above), since the Condor switchboard requires some files to be owned by root and some files to be owned by the non-superuser username for privilege separation to work correctly.
service_name condor-wms Used as the 'nickname' for the GSI DN in the condor_mapfile of other services. .
condor_location /path/to/condor-location Directory in which the condor software will be installed. IMPORTANT: The WMS Collector and Factory are always installed on the same node. The condor_location must not be a subdirectory of the Factory's install_location, logs_dir or client_log(proxy)_dir. They may share the same parent, however.
collector_port 9618 (condor default) Defines the Condor Collector port. Optional: default is 9618 (Condor default)
If multiple glidein services are installed on the same node, this should be unique for each service.
privilege_separation y See the Condor Privilege Separation Documentation for more information Valid values:
* y - privilege separation is used
* n - privilege separation is not used
frontend_users frontend_service_name : unix_account Maps the vofrontend's service name to the UNIX account that has been created for it. Only one frontend service can be specified on install. The format is: service_name : unix account
If privilege_separation is specified, this must specify the unique UNIX user account you set up for that frontend service.
If privilege_separation is not specified, this must be the factory username.
x509_cert_dir /path/to/certificates-location The directory where the CA certificates are maintained. The installer will validate for the precesence of *.0 and *.r0 files. If the CAs are installed from the VDT distribution, this will be the VDT_LOCATION/globus/TRUSTED_CA directory.
x509_cert /path-to-cert-location/cert.pm The location of the certificate file being used. This file must be owned by the user installing (starting/stopping) this service. Permissions should be 644 or 600.
x509_key /path-to-cert-location/key.pm The location of the certificate key file being used and associated with the certtificate defined by the x509_cert option above. This file must be owned by the user installing (starting/stopping) this service. Permissions should be 600 or 400.
x509_gsi_dn dn-subject-of-x509_cert-using-openssl This is the identity of the certificate used by this service to contact the other Condor based glideinWMS services. This is the subject of the certificate (x509_cert option).
openssl x509 -subject -noout -in [x509_cert]
It is used to populate the condor_config file GSI_DAEMON_NAME and condor_mapfile entries of this and the other glideinWMS services as needed.
condor_tarball /path/to/condor/tarballs/condor-7.5.0-linux-x86-rhel3-dynamic.tar.gz Location of the condor tarball. The installation script will perform the installation of condor using this tarball. It must be a zipped tarball with a *.tg.tz name.
condor_admin_email whomever@email.com The email address to get Condor notifications in the event of a problem. Used in the condor_config.local only.
number_of_schedds 5 The desired number of schedds to be used. There must be at least 1 schedd.
install_vdt_client y Indicates if an OSG/VDT client should be installed if it is not already present in the vdt_location option location. Valid values:
* y - installer will install a VDT client using the vdt_location and pacman_location/_url options specified.
* n - will not attempt to install the VDT client

This installer will not attempt to reinstall this software if it is already present.
vdt_location /path/to/glidein/vdt The location of the OSG/VDT client software. The installer looks for the existence of 2 files to verify if this is a valid OSG/VDT client installation:
1. setup.sh
2. existence of a voms-proxy-init executable.

If the install_vdt_client option is 'n', then this option's value should be empty/blank.
glideinwms_location /path/to/glideinWMS Directory of the glideinWMS software. Since this is a Condor service only, this software is only used during the installation process.

Factory Section

The manage-glideins script should be run as the factory user when installing the WMS Factory (see the username option below).
./manage-glideins --install factory --ini glideinWMS.ini
See below for the ini file attributes that installing/running WMS factory requires:

Attribute Example Description Comments
install_type tarball Indicates this is a Condor tarball installation.
At this time, for the Factory, only tarball installations are supported.
Valid values: tarball.
hostname wmscollector.domain.name hostname for Factory. The WMS collector and Factory must be collocated at this time.
username factory user
(non-root account)
UNIX user account that this service will run under. DO NOT use "root". For security purposes, this value should always be a non-root user. Although the WMS Collector and Factory must be co-located, they can be run as independent users.
service_name factory-wms Used as the 'nickname' for the GSI DN in the condor_mapfile of other services. .
install_location /path/to/glidein/factory HOME directory for the factory software. When the factory is created the following files/directories will exist in this directory:
* factory.sh - environment script
* glidein_[instance_name].cfg - the factory configuration file
* glidein_[instance_name] - directory containing the factory files

The install script will create this directory if it does not exist.
logs_dir /path/to/factory/logs User settable location for all Factory log files. Beneath this location there will be multiple sets of logs:
* for the factory as a whole * for each entry point the factory utilizes

The install script will create this directory if it does not exist.
client_log_dir
client_proxy_dir
/path/to/client/log_location
/path/to/client/proxy_location
User settable location for all client (VOFrontend) log and proxy files. If privilege separation is used,
* then the entire path (inclusive of this directory) must be root-writable-only (0755 and owned by root).
* these directories cannot be sub-directories of the Factory's install_location or logs_dir.

If privilege separation is not used,
* then the directory can be independent, or nested as a subdirectory, of the Factory's install_location or logs_dir.

If the above requirements are satisfied, the install script will create the necessary directories. If not, a permissions error will likely result.
instance_name v2_5 Used in naming files and directories. .
use_vofrontend_proxy y Specifies if the VO Frontend or Factory proxy should be used on the glidein pilot jobs submitted to the entry points. Valid values:
* y - only VO Frontend proxies will be used
* n - VO Frontend proxies will be used if available otherwise the Factory will provide the proxy.
x509_proxy /path-to-proxy/x509_proxyfile The location of factory pilot proxy. If the Factory use_vofrontend_proxy option is:
* y - this must be empty
* n - this must be populated
An X509_USER_PROXY variable using this value will set be in the factory environment script that is created during the installation.
x509_gsi_dn dn-issuer-of-x509_proxy-using-openssl The issuer of the Factory pilot proxy specified by the Factory x509_proxy option If the Factory use_vofrontend_proxy option is:
* y - this must be empty
* n - this must be populated
To obtain the issuer, use:
openssl x509 -issuer -noout -in [x509_proxy]
This is used in the UserCollector condor_mapfile and condor_config.local so the glidein pilot can communicate with the UserCollector.
use_glexec y Used to specify how user submitted jobs (not glidein pilots) are authorized on the WN nodes for an entry point
With gLexec, the individual user's proxy submitted with their job is used to authorize the job and is reflected in the accounting.
Without gLexec, the glidein pilot job's proxy is used and only that user account is reflected in the accounting.
Valid values:
* y - downloads and uses gLexec.
* n - glidein pilot proxy is used
use_ccb n Indicates if CCB should be used or not. Valid values:
* y - uses CCB
* n - does not use CCB
ress_host osg-ress-4.fnal.gov Identifies the ReSS server to be used to select entry points (CEs) to submit glidein pilot jobs to. The only validation performed is to verify if that server exists.
Valid OSG values:
* osg-ress-1.fnal.gov - OSG Production
* osg-ress-4.fnal.gov - OSG ITB
entry_vos cms, dzero A comma delimited set of VOs used to select the entry points that glideins can be submitted to. These are the used as the initial criteria in querying ReSS for glidein entry points.
entry_filters (int(GlueCEPolicyMaxCPUTime) <(25*60)) An additional entry point (CE) filter for insuring that specific resources are available. After the initial set of entry points have been selected using the entry_vos criteria, these filters are applied. The format is a python expression using Glue schema attributes.
install_vdt_client y Indicates if an OSG/VDT client should be installed if it is not already present in the vdt_location option location. Valid values:
* y - installer will install a VDT client using the vdt_location and pacman_location/_url options specified.
* n - will not attempt to install the VDT client

This installer will not attempt to reinstall this software if it is already present.
vdt_location /path/to/glidein/vdt The location of the OSG/VDT client software. The installer looks for the existence of 2 files to verify if this is a valid OSG/VDT client installation:
1. setup.sh
2. existence of a voms-proxy-init executable.

If the install_vdt_client option is 'n', then this option's value should be empty/blank.
glideinwms_location /path/to/glideinWMS Directory of the glideinWMS software. This software is used for both the installation and during the actual running of this glidein service.
web_location /var/www/html/factory Specifies the location for the monitoring and staging data that must be accessible by web services. The installer will create the following directories in this location:
1. web_location/monitor
2. web_location/stage
Important: This should be created before installing this service as the service's username and the web server user are generally different. This script will not be able to create this directory with proper ownership.
web_url http://%(hostname)s:port Identifies the url used by the glidein pilots to download necessary software and to record monitoring data.
In order to insure consistency, the installer will take the unix basename of the web_location and append it to the web_url value. So, for the value shown below in the web_location, the actual value used by the glidein pilots, will be web_url/factory/stage(monitor).
Important: It may be a good idea to verify that the port specified is accessible from off-site as some sites restrict off-site access to some ports.
javascriptrrd_location /path/to/javascriptrrd Identifies the location of the javascript rrd software. This installation must include the flot processes in the parent directory.

User Collector Section

The pool Collector can be installed either as root or as a non privileged user. Either case, make sure that the user has access to the needed GSI credentials. There is no real advantage to install as root, so non-privileged installation is recommended if installed separately.
./manage-glideins --install usercollector --ini glideinWMS.ini
See below for the ini file attributes that installing/running user collector requ ires:

Attribute Example Description Comments
install_type tarball or rpm If this is a VOFrontend RPM installation and you are doing a '--configure', then rpm should be specified.
If this is a stand-alone User Collector install, only tarball installations are supported.
Valid values: tarball, rpm.
hostname usercollector.domain.name hostname for User Collector. .
username collector (non-root account) UNIX user account that this services will run under. DO NOT use "root". For security purposes, this value should always be a non-root user.
service_name userpool Used as the 'nickname' for the GSI DN in the condor_mapfile of other services. .
condor_location /path/to/condor-userpool Directory in which the condor software will be installed. IMPORTANT: The User Collector can share the same instance of Condor as the Frontend. The condor_location must not be a subdirectory of the Frontend's install_location or logs_dir. They may share the same parent, however.
collector_port 9618 (condor default) Defines the Condor Collector port. Optional: default is 9618 (Condor default)
If multiple glidein services are installed on the same node, this should be unique for each service.
number_of_secondary_collectors 5 The desired number of secondary collectors to be used. Optional: default is 0 (zero)
A rough estimate is to use one collector per 100 glideins with a hard limit on 200 glideins per collector.
x509_cert_dir /path/to/certificates-location The directory where the CA certificates are maintained. The installer will validate for the precesence of *.0 and *.r0 files. If the CAs are installed from the VDT distribution, this will be the VDT_LOCATION/globus/TRUSTED_CA directory.
x509_cert /path-to-cert-location/cert.pm The location of the certificate file being used. This file must be owned by the user installing (starting/stopping) this service. Permissions should be 644 or 600.
x509_key /path-to-cert-location/key.pm The location of the certificate key file being used and associated with the certtificate defined by the x509_cert option above. This file must be owned by the user installing (starting/stopping) this service. Permissions should be 600 or 400.
x509_gsi_dn dn-subject-of-x509_cert-using-openssl This is the identity of the certificate used by this service to contact the other Condor based glideinWMS services. This is the subject of the certificate (x509_cert option).
openssl x509 -subject -noout -in [x509_cert]
It is used to populate the condor_config file GSI_DAEMON_NAME and condor_mapfile entries of this and the other glideinWMS services as needed.
condor_tarball /path/to/condor/tarballs/condor-7.5.0-linux-x86-rhel3-dynamic.tar.gz Location of the condor tarball. The installation script will perform the installation of condor using this tarball. It must be a zipped tarball with a *.tg.tz name.
condor_admin_email whomever@email.com The email address to get Condor notifications in the event of a problem. Used in the condor_config.local only.
install_vdt_client y Indicates if an OSG/VDT client should be installed if it is not already present in the vdt_location option location. Valid values:
* y - installer will install a VDT client using the vdt_location and pacman_location/_url options specified.
* n - will not attempt to install the VDT client

This installer will not attempt to reinstall this software if it is already present.
vdt_location /path/to/glidein/vdt The location of the OSG/VDT client software. The installer looks for the existence of 2 files to verify if this is a valid OSG/VDT client installation:
1. setup.sh
2. existence of a voms-proxy-init executable.

If the install_vdt_client option is 'n', then this option's value should be empty/blank.
glideinwms_location /path/to/glideinWMS Directory of the glideinWMS software. Since this is a Condor service only, this software is only used during the installation process.

Submit Section

The scheduler node software should be installed as root. While it is possible to run the schedds as a non-privileged user, it has some serious security implications; see the
Security section of the Condor manual for details.
./manage-glideins --install submit --ini glideinWMS.ini
See below for the ini file attributes that installing/running the submit node requires:

Attribute Example Description Comments
install_type tarball or rpm If this is a VOFrontend RPM installation and you are doing a '--configure', then rpm should be specified.
If this is a stand-alone Submit install, only tarball installations are supported.
Valid values: tarball, rpm.
hostname submitnode.domain.name hostname for Submit node .
username condor (non-root account) UNIX user account that this services will run under. DO NOT use "root". For security purposes, this value should always be a non-root user.
service_name submit Used as the 'nickname' for the GSI DN in the condor_mapfile of other services. .
condor_location /path/to/condor-submit Directory in which the condor software will be installed. IMPORTANT: The Submit can share the same instance of Condor as the Frontend. The condor_location must not be a subdirectory of the Frontend's install_location or logs_dir. They may share the same parent, however.
x509_cert_dir /path/to/certificates-location The directory where the CA certificates are maintained. The installer will validate for the precesence of *.0 and *.r0 files. If the CAs are installed from the VDT distribution, this will be the VDT_LOCATION/globus/TRUSTED_CA directory.
x509_cert /path-to-cert-location/cert.pm The location of the certificate file being used. This file must be owned by the user installing (starting/stopping) this service. Permissions should be 644 or 600.
x509_key /path-to-cert-location/key.pm The location of the certificate key file being used and associated with the certtificate defined by the x509_cert option above. This file must be owned by the user installing (starting/stopping) this service. Permissions should be 600 or 400.
x509_gsi_dn dn-subject-of-x509_cert-using-openssl This is the identity of the certificate used by this service to contact the other Condor based glideinWMS services. This is the subject of the certificate (x509_cert option).
openssl x509 -subject -noout -in [x509_cert]
It is used to populate the condor_config file GSI_DAEMON_NAME and condor_mapfile entries of this and the other glideinWMS services as needed.
condor_tarball /path/to/condor/tarballs/condor-7.5.0-linux-x86-rhel3-dynamic.tar.gz Location of the condor tarball. The installation script will perform the installation of condor using this tarball. It must be a zipped tarball with a *.tg.tz name.
condor_admin_email whomever@email.com The email address to get Condor notifications in the event of a problem. Used in the condor_config.local only.
number_of_schedds 5 The desired number of schedds to be used. There must be at least 1 schedd.
schedd_shared_port 9615 Specifies the port number to be used by the shared port daemon for schedds. This is only available in Condor 7.5.3+.
This can drastically reduce the number of ports used and thus improves scalability.
The default port is 9615.
Leave this option blank if you do not wish to utilize this feature or if is is not supported in the version of Condor being used.
For more information on use of the shared_port_daemon, see the GlideinWMS - Advanced Condor Configuration manual.
install_vdt_client y Indicates if an OSG/VDT client should be installed if it is not already present in the vdt_location option location. Valid values:
* y - installer will install a VDT client using the vdt_location and pacman_location/_url options specified.
* n - will not attempt to install the VDT client

This installer will not attempt to reinstall this software if it is already present.
vdt_location /path/to/glidein/vdt The location of the OSG/VDT client software. The installer looks for the existence of 2 files to verify if this is a valid OSG/VDT client installation:
1. setup.sh
2. existence of a voms-proxy-init executable.

If the install_vdt_client option is 'n', then this option's value should be empty/blank.
glideinwms_location /path/to/glideinWMS Directory of the glideinWMS software. Since this is a Condor service only, this software is only used during the installation process.

Glidein Frontend

The installation of the frontend should be done as the frontend user (see username below).
./manage-glideins --install vofrontend --ini glideinWMS.ini
See below for the ini file attributes that installing/running the glideinWMS frontend requires:

Attribute Example Description Comments
install_type tarball or rpm If this is a VOFrontend RPM installation and you are doing a '--configure', then rpm should be specified.
If this is a stand-alone VOFrontend install, only tarball installations are supported.
Valid values: tarball, rpm.
hostname vofrontend.domain.name hostname for VOFrontend. .
username vofrontend (non-root account) UNIX user account that this services will run under. DO NOT use "root". For security purposes, this value should always be a non-root user.
service_name vo_frontend Used as the 'nickname' for the GSI DN in the condor_mapfile. This name MUST correspond to frontend service name listed the factory configuration for the frontend to be able to talk to the factory.
install_location /path/to/vofrontend HOME directory for the frontend software. When the frontend is created the following files/directories will exist in this directory:
* frontend.sh - environment script
* frontend_[instance_name].cfg - directory containing the frontend configuration file
* frontend_[instance_name] - directory containing the frontend files

The install script will create this directory if it does exist.
logs_dir /path/to/vofrontend/logs User settable location for all Frontend log files. Beneath this location there will be multiple sets of logs:
* for the frontend as a whole
* for each group the frontend utilizes

The install script will create this directory if it does exist.
instance_name v2_5 Used in naming files and directories. .
condor_location /path/to/condor-submit Directory in which the condor software will be installed. Note: The Frontend never runs any Condor daemons. It only uses the Condor client tools.

IMPORTANT: The Frontend can share the same instance of Condor as the User Collector and /or Submit. The condor_location must not be a subdirectory of the Frontend's install_location or logs_dir. They may share the same parent, however.
condor_tarball /path/to/condor/tarballs/condor-7.5.0-linux-x86-rhel3-dynamic.tar.gz Location of the condor tarball. The installation script will perform the installation of condor using this tarball. It must be a zipped tarball with a *.tg.tz name.
condor_admin_email whomever@email.com The email address to get Condor notifications in the event of a problem. Used in the condor_config.local only.
x509_cert_dir /path/to/certificates-location The directory where the CA certificates are maintained. The installer will validate for the precesence of *.0 and *.r0 files. If the CAs are installed from the VDT distribution, this will be the VDT_LOCATION/globus/TRUSTED_CA directory.
x509_proxy /path-to-proxy/x509_proxyfile The location of the proxy used to identify this service to the other glideinWMS services. This is the only glideinWMS service that currently requires a proxy to communicate with the other glideinWMS services. This is NOT the glidein pilot proxy.
The x509_gsi_dn option that follows should be populated with the issuer of this proxy.
x509_gsi_dn dn-issuer-of-x509_proxy-using-openssl This is the identity used by this service to contact the other Condor based glideinWMS services. This is the issuer of the proxy identified by the x509_proxy option above.
openssl x509 -issuer -noout -in [x509_proxy]
It is used to populate the condor_config file GSI_DAEMON_NAME and condor_mapfile entries of this and the other glideinWMS services as needed.
glidein_proxy_files /path-to-proxy/proxyfile1 Identifies the proxy(s) to be used to submit the glidein pilot jobs. If the Factory (use_vofrontend_proxy option) is expecting the VOFrontend to provide the glidein pilot proxies, at least one proxy must be specified.
If not, then this should be empty.

When specifying multiple proxies, whitespace is the delimiter.
glidein_proxy_dns dn-issuer-of-glidein_proxy_files-using-openssl Identifies the issuer(s) of each proxy specified in the glidein_proxy_files option. This is the issuer of the proxy. (x509_cert option).
openssl x509 -issuer -noout -in [glidein_proxy_files]
Validation is performed based on the relative position of each entry in those options.
This is used in the UserCollector condor_mapfile and condor_config.local so the glidein pilot can communicate with the UserCollector.

When mulitple proxies are specified, a semicolon is the delimiter.
glexec_use optional Tells the Factory service if use of the glexec authorization process is to be considered in selecting entry points to submit glidein pilots for the frontend user jobs.. Valid values:
* required - will only request glidein pilots to entry points that use glexec
* optional - will request glidein pilots regardless of glexec use
* never - will never request glidein pilots for entry points using glexec
expose_grid_env True Tells the Factory service to export shell variables from the glidein pilots to the user jobs or not. Valid values:
* True - jobs will export shell variables to the user job running.
* False - no shell variables will be exported.
group_name main This defines a grouping for user jobs that you wish to monitor using a specified critera. The criteria used is defined by the match_string and userjob_constraints options that follow.
userjob_constraints , This represents the criteria used for matching glideins to jobs. This is a Condor expression. A simple expression would be:
(JobUniverse==5)&&(GLIDEIN_Is_Monitor =!= TRUE)&&(JOB_Is_Monitor =!= TRUE)
match_string True This is the selection and matching criteria used for the user jobs. This must be in python format so using the exact case (True and False must be capitalized) is critical. One example is:
glidein["attrs"]["GLIDEIN_Site"] in job["DESIRED_Sites"].split(",")
To select all jobs, just use 'True'.
If you use other that 'True', the criteria will need to specified in your user's condor job statement.
install_vdt_client y Indicates if an OSG/VDT client should be installed if it is not already present in the vdt_location option location. Valid values:
* y - installer will install a VDT client using the vdt_location and pacman_location/_url options specified.
* n - will not attempt to install the VDT client

This installer will not attempt to reinstall this software if it is already present.
vdt_location /path/to/glidein/vdt The location of the OSG/VDT client software. The installer looks for the existence of 2 files to verify if this is a valid OSG/VDT client installation:
1. setup.sh
2. existence of a voms-proxy-init executable.

If the install_vdt_client option is 'n', then this option's value should be empty/blank.
glideinwms_location /path/to/glideinWMS_code Directory of the glideinWMS software. This software is used for both the installation and during the actual running of this glidein service.
web_location /var/www/html/factory Specifies the location for the monitoring and staging data that must be accessible by web services. The installer will create the following directories in this location:
1. web_location/monitor
2. web_location/stage
Important: This should be created before installing this service as the service's username and the web server user are generally different. This script will not be able to create this directory with proper ownership.
web_url http://%(hostname)s:port Identifies the url used by the glidein pilots to download necessary software and to record monitoring data.
In order to insure consistency, the installer will take the unix basename of the web_location and append it to the web_url value. So, for the value shown below in the web_location, the actual value used by the glidein pilots, will be web_url/factory/stage(monitor).
Important: It may be a good idea to verify that the port specified is accessible from off-site as some sites restrict off-site access to some ports.
javascriptrrd_location /path/to/javascriptrrd Identifies the location of the javascript rrd software. This installation must include the flot processes in the parent directory.

Sample ini-files

Sample ini-files cover following three use cases:

  • CASE 1: Single node install with single ini files
  • CASE 2: Multiple node install with single ini file
  • CASE 3: Multiple node install with different init files

CASE 1: Single node install with single ini files

; ## Global options
[DEFAULT]
pacman_location =
pacman_url      =

install_type = tarball

hostname = cms-xen38.fnal.gov
username = glidein

version = v2_5_1
instance_name = all_four_node
install_dir = /opt/no-vdt-glideinWMS
glideinwms_location = %(install_dir)s/glideinWMS
javascriptrrd_location = %(install_dir)s/javascriptrrd

tarballs = /opt/install_packages
condor_tarball = %(tarballs)s/condor-7.5.5-x86_rhap_5-stripped.tar.gz
condor_admin_email = condor-admin@fnal.gov

vdt_location =
install_vdt_client = n

x509_cert = /etc/grid-security/hostcert.pem
x509_key  = /etc/grid-security/hostkey.pem
x509_gsi_dn = /DC=org/DC=doegrids/OU=Services/CN=cms-xen38.fnal.gov
x509_cert_dir  = /etc/grid-security/certificates

;--------------------------------------------------
[WMSCollector]

service_name = wms_service

condor_location = %(install_dir)s/condor-wms
collector_port = 9618
number_of_schedds = 5

privilege_separation = y
frontend_users = frontend_service : vo_cms,

;--------------------------------------------------
[Factory]

service_name = factory_service

install_location = %(install_dir)s/factory
logs_dir         = %(install_dir)s/factory-logs

client_log_dir   = %(install_dir)s/client-logs
client_proxy_dir = %(install_dir)s/client-proxies

use_vofrontend_proxy = y
x509_proxy  =
x509_gsi_dn =

use_glexec = n
use_ccb    = n

ress_host = osg-ress-4.fnal.gov
entry_vos = cms
entry_filters = (int(GlueCEPolicyMaxCPUTime)==0) or (int(GlueCEPolicyMaxCPUTime)<(72*60))


web_url       = http://%(hostname)s:8319
web_location  = /var/www/html/factory

;--------------------------------------------------
[UserCollector]

service_name = userpool_service

condor_location = %(install_dir)s/condor-therest
collector_port = 9640
number_of_secondary_collectors = 5

;--------------------------------------------------
[Submit]

service_name = submit_service
number_of_schedds = 5
schedd_shared_port =

condor_location = %(install_dir)s/condor-therest

;--------------------------------------------------
[VOFrontend]

service_name = frontend_service

install_location = %(install_dir)s/frontend
logs_dir         = %(install_dir)s/frontend-logs

condor_location = %(install_dir)s/condor-therest

x509_proxy = /etc/grid-security/tony_pilot_proxy
x509_gsi_dn = /DC=org/DC=doegrids/OU=People/CN=Anthony Tiradani 329103

glidein_proxy_files = /etc/grid-security/tony_pilot_proxy
glidein_proxy_dns = /DC=org/DC=doegrids/OU=People/CN=Anthony Tiradani 329103

group_name   = main
userjob_constraints = (JobUniverse==5)&&(GLIDEIN_Is_Monitor =!= TRUE)&&(JOB_Is_Monitor =!= TRUE)
match_string = glidein["attrs"]["GLIDEIN_Site"] in job["DESIRED_Sites"].split(",")

glexec_use = never
expose_grid_env = True

web_location  = /var/www/html/frontend
web_url       = http://%(hostname)s:8319

CASE 2: Multiple node install with single ini file

; ## Global options
[DEFAULT]

pacman_location = /opt/pacman/pacman-3.28
pacman_url      = http://physics.bu.edu/pacman/sample_cache/tarballs

install_type = tarball

version = v2.5.1
instance_name = v2.5.1-cream-cloud
install_dir = /opt/glideinWMS-v2plus
glideinwms_location = %(install_dir)s/glideinWMS
javascriptrrd_location = %(install_dir)s/javascriptrrd

tarballs = /opt/install_packages
condor_tarball = %(tarballs)s/condor-7.5.5-x86_rhap_5-stripped.tar.gz

vdt_location = /opt/vdt
install_vdt_client = n

x509_cert_dir=/opt/vdt/globus/TRUSTED_CA

condor_admin_email = condor-admin@fnal.gov

;--------------------------------------------------
[WMSCollector]

hostname = cms-xen6.fnal.gov
username    = tony_xen_factory
service_name = wms_service

condor_location = %(install_dir)s/condor-wms
collector_port = 9618

privilege_separation = y
frontend_users = frontend_service : tony_xen_cms_frontend

x509_cert = /etc/grid-security/glideincerts/cms-xen6glideinwmscollectorcert.pem
x509_key = /etc/grid-security/glideincerts/cms-xen6glideinwmscollectorkey.pem
x509_gsi_dn = /DC=org/DC=doegrids/OU=Services/CN=glideinwmscollector/cms-xen6.fnal.gov

number_of_schedds = 5

;--------------------------------------------------
[Factory]

hostname = cms-xen6.fnal.gov
username    = tony_xen_factory
service_name = factory_service

install_location = %(install_dir)s/Factory
logs_dir = %(install_dir)s/factory-logs
client_log_dir = %(install_dir)s/client-logs
client_proxy_dir = %(install_dir)s/client-proxies

use_vofrontend_proxy = y
x509_proxy =
x509_gsi_dn =
use_glexec = n
use_ccb    = y

ress_host       = osg-ress-4.fnal.gov
entry_vos = cms
entry_filters = (int(GlueCEPolicyMaxCPUTime)==0) or (int(GlueCEPolicyMaxCPUTime)<=(24*60))

web_url       = http://%(hostname)s:8319
web_location  = /var/www/html/Factory

;--------------------------------------------------
[UserCollector]

hostname = cms-xen8.fnal.gov
username    = tony_xen_pool_collector
service_name = userpool_service

condor_location = %(install_dir)s/condor-userpool
collector_port = 9618

x509_cert = /etc/grid-security/glideincerts/cms-xen8glideinpoolcollectorcert.pem
x509_key = /etc/grid-security/glideincerts/cms-xen8glideinpoolcollectorkey.pem
x509_gsi_dn = /DC=org/DC=doegrids/OU=Services/CN=glideinpoolcollector/cms-xen8.fnal.gov

number_of_secondary_collectors = 0

;--------------------------------------------------
[Submit]

hostname = cms-xen8.fnal.gov
username    = condor
service_name = submit_service

condor_location = %(install_dir)s/condor-submit

x509_cert = /etc/grid-security/glideincerts/cms-xen8glideinsubmittercmscert.pem
x509_key = /etc/grid-security/glideincerts/cms-xen8glideinsubmittercmskey.pem
x509_gsi_dn = /DC=org/DC=doegrids/OU=Services/CN=glideinsubmittercms/cms-xen8.fnal.gov

number_of_schedds = 5
schedd_shared_port = 9615

;--------------------------------------------------
[VOFrontend]

instance_name = v2_5_1-cream-cloud

hostname = cms-xen7.fnal.gov
username    = tony_xen_cms_frontend
service_name = frontend_service

install_location = %(install_dir)s/Frontend
logs_dir     = %(install_dir)s/frontend-logs

condor_location = %(install_dir)s/condor-frontend

gsi_credential_type = proxy
x509_proxy = /etc/grid-security/glideincerts/tony_pilot_proxy
x509_gsi_dn = /DC=org/DC=doegrids/OU=People/CN=Anthony Tiradani 329103

glidein_proxy_files = /etc/grid-security/glideincerts/tony_pilot_proxy
glidein_proxy_dns = /DC=org/DC=doegrids/OU=People/CN=Anthony Tiradani 329103

group_name = main
userjob_constraints = (JobUniverse==5)&&(GLIDEIN_Is_Monitor =!= TRUE)&&(JOB_Is_Monitor =!= TRUE)
match_string = glidein["attrs"]["GLIDEIN_Site"] in job["DESIRED_Sites"].split(",")

glexec_use = never
expose_grid_env = True

web_url       = http://%(hostname)s:8319
web_location  = /var/www/html/Frontend

CASE 3: Multiple node install with different ini files

Multiple node install with services using their own ini files is a work in progress. Sample ini files for different services for a working installation is shown below. WMSCollector and Factory configuration
[DEFAULT]
install_type = tarball

installation_version = v2.5.1
install_vdt_client = n
vdt_location = /home/gfactoryuser/vdt
x509_cert_dir = %(vdt_location)s/globus/share/certificates
glideinwms_location = /home/gfactoryuser/v2.5.1/glideinWMS
pacman_location =
pacman_url      =

;--------------------------------------------------
[WMSCollector]

hostname = cmssrv97.fnal.gov
username = condoruser
service_name = WMS_Collector
condor_location = /home/condoruser/%(installation_version)s/glidecondor
collector_port = 8618
privilege_separation = y
frontend_users = Frontend-v2_5_1-cmssrv99:test1user
x509_cert = /etc/grid-security/cmssrv97condorcert.pem
x509_key = /etc/grid-security/cmssrv97condorkey.pem
condor_tarball = /home/gfactoryuser/installers/condor-7.5.5-x86_64_rhap_5-stripped.tar.gz
condor_admin_email = condor-admin@fnal.gov
x509_gsi_dn = /DC=org/DC=doegrids/OU=Services/CN=condor/cmssrv97.fnal.gov
number_of_schedds = 5

;--------------------------------------------------
[Factory]

hostname = cmssrv97.fnal.gov
username = gfactoryuser
service_name = GlideinFactory-v2.5.1-cmssrv97
install_location = /home/gfactoryuser/v2.5.1/glideinsubmit
logs_dir = /var/gfactory/v2.5.1/glideinlogs
client_log_dir = /var/gfactory/v2.5.1/clientlogs
client_proxy_dir = /var/gfactory/v2.5.1/clientproxies
instance_name = v1_0
use_vofrontend_proxy = y
use_glexec = y
use_ccb = y
ress_host = osg-ress-4.fnal.gov
entry_vos = dzero
entry_filters = (int(GlueCEPolicyMaxCPUTime)<(25*60))
web_location = /var/www/html/glidefactory-v2.5.1
web_url = http://%(hostname)s:8000
javascriptrrd_location = /home/gfactoryuser/javascriptrrd
x509_proxy =
x509_gsi_dn =

;--------------------------------------------------
[UserCollector]

hostname = cmssrv99.fnal.gov
service_name =
x509_gsi_dn =
condor_location =
collector_port =
number_of_secondary_collectors =

;--------------------------------------------------
[Submit]

hostname = cmssrv99.fnal.gov
service_name =
condor_location =
x509_gsi_dn =

;--------------------------------------------------
[VOFrontend]

hostname = cmssrv99.fnal.gov
x509_gsi_dn = /DC=org/DC=doegrids/OU=People/CN=Parag Mhashilkar 209917
glidein_proxy_files = /tmp
glidein_proxy_dns = %(x509_gsi_dn)s
User Collector ini file
[DEFAULT]

pacman_url      =
pacman_location =
install_vdt_client = n
vdt_location = /home/frontenduser/vdt
x509_cert_dir = %(vdt_location)s/globus/share/certificates
glideinwms_location = /home/frontenduser/v2.5.1/glideinWMS

;--------------------------------------------------
[WMSCollector]

hostname = cmssrv97.fnal.gov
collector_port =
username =
privilege_separation =
condor_location =
frontend_users =
service_name =
x509_gsi_dn =

;--------------------------------------------------
[Factory]

use_vofrontend_proxy =
x509_gsi_dn =
service_name =
username =

;--------------------------------------------------
[VOFrontend]

hostname = cmssrv99.fnal.gov
service_name = GlideinFactory-v2.5.1-cmssrv97
x509_gsi_dn = /DC=org/DC=doegrids/OU=People/CN=Parag Mhashilkar 209917
glidein_proxy_dns = %(x509_gsi_dn)s

;--------------------------------------------------
[UserCollector]

hostname = cmssrv99.fnal.gov
username = condoruser
service_name = User_Pool
condor_location = /home/condoruser/v2.5.1/glidecondor
collector_port = 9618
x509_cert = /etc/grid-security/cmssrv99condorcert.pem
x509_key = /etc/grid-security/cmssrv99condorkey.pem
x509_gsi_dn = /DC=org/DC=doegrids/OU=Services/CN=condor/cmssrv99.fnal.gov
condor_tarball = /home/frontenduser/installers/condor-7.5.5-x86_64_rhap_5-stripped.tar.gz
number_of_secondary_collectors = 2
condor_admin_email = condor-admin@fnal.gov

;--------------------------------------------------
[Submit]

hostname = cmssrv99.fnal.gov
service_name = UserSchedd
x509_gsi_dn = /DC=org/DC=doegrids/OU=People/CN=Parag Mhashilkar 209917
condor_location =

Submit ini file
[DEFAULT]
pacman_url      =
pacman_location =
install_vdt_client = n
vdt_location = /home/frontenduser/vdt
x509_cert_dir = %(vdt_location)s/globus/share/certificates
glideinwms_location = /home/frontenduser/v2.5.1/glideinWMS

;--------------------------------------------------
[WMSCollector]

hostname = cmssrv97.fnal.gov
username =
privilege_separation =
condor_location =
frontend_users =
service_name =
x509_gsi_dn =

;--------------------------------------------------
[Factory]

username =

;--------------------------------------------------
[VOFrontend]

hostname = cmssrv99.fnal.gov
service_name = GlideinFactory-v2.5.1-cmssrv97
x509_gsi_dn = /DC=org/DC=doegrids/OU=People/CN=Parag Mhashilkar 209917

;--------------------------------------------------
[UserCollector]

hostname = cmssrv99.fnal.gov
service_name = User_Pool
condor_location =
collector_port = 9618
x509_gsi_dn = /DC=org/DC=doegrids/OU=Services/CN=condor/cmssrv99.fnal.gov
number_of_secondary_collectors =

;--------------------------------------------------
[Submit]

hostname = cmssrv99.fnal.gov
username = testuser
service_name = UserSchedd
x509_gsi_dn = /DC=org/DC=doegrids/OU=People/CN=Parag Mhashilkar 209917
condor_location = /home/testuser/v2.5.1/glidecondor
x509_cert = /etc/grid-security/cmssrv99testusercondorcert.pem
x509_key = /etc/grid-security/cmssrv99testusercondorkey.pem
x509_gsi_dn = /DC=org/DC=doegrids/OU=Services/CN=condor/cmssrv99.fnal.gov
condor_tarball = /home/frontenduser/installers/condor-7.5.5-x86_64_rhap_5-stripped.tar.gz
condor_admin_email = condor-admin@fnal.gov
number_of_schedds = 2
schedd_shared_port = 4444

VO Frontend ini file
[DEFAULT]
pacman_version  =
pacman_url      =
pacman_location =
install_vdt_client = n
vdt_location = /home/frontenduser/vdt
x509_cert_dir = %(vdt_location)s/globus/share/certificates
glideinwms_location = /home/frontenduser/v2.5.1/glideinWMS

;--------------------------------------------------
[WMSCollector]

hostname = cmssrv97.fnal.gov
service_name = WMS_Collector
collector_port = 8618
x509_gsi_dn = /DC=org/DC=doegrids/OU=Services/CN=condor/cmssrv97.fnal.gov
username = condoruser
privilege_separation = y
frontend_users = Frontend-v2_5_1-cmssrv99:test1user
condor_location =

;--------------------------------------------------
[Factory]

hostname = cmssrv97.fnal.gov
username = gfactoryuser
use_vofrontend_proxy = y

;--------------------------------------------------
[VOFrontend]

hostname = cmssrv99.fnal.gov
username = frontenduser
service_name = Frontend-v2_5_1-cmssrv99
install_location = /home/frontenduser/v2.5.1/frontstage
logs_dir = /home/frontenduser/v2.5.1/frontlogs
instance_name = v1_0
condor_location = /home/frontenduser/v2.5.1/glidecondor
condor_tarball = /home/frontenduser/installers/condor-7.5.5-x86_64_rhap_5-stripped.tar.gz
condor_admin_email = condor-admin@fnal.gov
x509_proxy = /home/frontenduser/security/grid_proxy
x509_gsi_dn = /DC=org/DC=doegrids/OU=People/CN=Parag Mhashilkar 209917
glidein_proxy_files = %(x509_proxy)s
glidein_proxy_dns = %(x509_gsi_dn)s
glexec_use = optional
expose_grid_env = True
group_name = main
userjob_constraints = (JobUniverse==5)&&(GLIDEIN_Is_Monitor =!= TRUE)&&(JOB_Is_Monitor =!= TRUE)
match_string = glidein["attrs"]["GLIDEIN_Site"] in job["DESIRED_Sites"].split(",")
web_url = http://cmssrv99.fnal.gov:8000
web_location = /var/www/html/vofrontend-v2.5.1
javascriptrrd_location = /home/frontenduser/javascriptrrd

;--------------------------------------------------
[UserCollector]

hostname = cmssrv99.fnal.gov
service_name = User_Pool
x509_gsi_dn = /DC=org/DC=doegrids/OU=Services/CN=condor/cmssrv99.fnal.gov
collector_port = 9618
condor_location =
number_of_secondary_collectors = 2

;--------------------------------------------------
[Submit]

x509_gsi_dn = /DC=org/DC=doegrids/OU=People/CN=Parag Mhashilkar 209917
hostname = cmssrv99.fnal.gov
service_name = UserSchedd
condor_location =