GlideinWMS The Glidein-based Workflow Management System

WMS Factory Design

Data Exchange

Jump to:
  1. Program flow
  2. Find and perform work
  3. Advertise myself
  4. Write statistics
  5. Source code distribution
  6. Configuration files
Factory Design Topics:

Data exchange overview

In the previous section we described the general architecture of the WMS. Let now have a look at the type of information that the two pieces publish.


See here for a detailed description of the protocol.

The Glidein Factory Class-Ad

A factory publishes one class-ad per glidein entry. The class-ad contains

  • the name of the glidein (and the factory),
  • the attributes that describe the glidein,
  • the list of parameters that the glidein accepts. All the parameters have also default values associated with them, in case a frontend does not redefine them, and
  • the (optional) public key algorithm, ID and value, together with the supported symmetric algorithms, that the frontend can use to push back encrypted values.
  • list of monitoring values (like how many glideins are in the queues, etc)
  • limits configured in the factory

Look at the picture below for a generic description.



Please notice that the glidein attributes can be completely arbitrary; the only predefined attributes are the glidein and factory name, and the convention that anything that starts with GlideinParam is a parameter; anything that starts with GlideinMonitor is a monitoring attribute; anything that starts with GlideinConfig is a limit configured in the factory.

Once the factory starts serving frontends, it will publish also another ClassAd for every frontend served. This ClassAd contains only monitoring information, and is not used by the glideinWMS itself.
Find below a graphical representation of these ClassAds.

The VO Frontend Class-Ad

A VO frontend will obtain the list all available glideins and select the ones that fit its needs, based on the published attributes. For each fitting glidein, a frontend class-ad will be published. Such a class-ad will contain

  • the name of the frontend and a request ID,
  • the desired glidein name,
  • the (optional) URL and signatures for the frontend specific scripts and data,
  • the desired rate and limits of glidein submissions,
  • the glidein parameters (in clear),and
  • the (optional) factory public key ID used, together with
    • the symmetric encryption algorithm and key,
    • the encrypted identity, and
    • the encrypted parameters.
  • list of monitoring values (like how many jobs are currently running, etc)
  • limits configured in the frontend

If encryption is used, the the encrypted identity must must match the AuthenticatedIdentity attribute inserted by the HTCondor collector (needs HTCondor version 7.3.1 or later)

Have a look at the picture below for a generic description.

The most important parameters that the VO Fronted sends to the Factory are:

  • The address of the User Pool collector(s) - GLIDEIN_Collector.
  • The pilot proxies. If present, these are always encrypted. Three types of information are sent:
    • Number of proxies sent - nr_x509_proxies
    • The proxy identifiers; given an identifier, the proxy DN must not change between updates. - x509_proxy_0_identifier ... x509_proxy_N_identifier
    • The security classes; proxies within the same class may have access to one another - x509_proxy_0_security_class ... x509_proxy_N_security_class
    • The proxies themselves - x509_proxy_0 ... x509_proxy_N
  • The security name associated with the proxies - SecurityName. The factory uses it for frontend whitelisting. If present, it is always encrypted.

In the current implementation, the only glidein rate setting parameters supported are ReqIdleGlideins, that tells the factory how many idle glideins to keep in the queue at any given time, and ReqMaxGlideins, that tells the factory to stop submitting new glideins when the number of running glidiens reaces that level. Future versions may contain more sophisticated controls, like the maximum number of glideins to keep in the queue or the maximum rate at which the glideins should be submitted.

The glideresource class-ad:

For every factory entry that accepts requests from the VO frontend, frontend will publish glideresource class-ads to the User Pool Collector. Such a class-ad will contains

  • Information about the factory and frontend element served and the downtime status
  • Monitoring information from the factory available in glidefactoryclient classad
  • Monitoring information from the frontend about the glideins and jobs
  • Matching information from the frontend about factory and job information:
    • GlideClientMatchingGlideinCondorExpr: The HTCondor start expression on the glideins to restrict the jobs that start on glideins.
    • GlideClientConstraintJobCondorExpr: The HTCondor expression of which jobs to consider for glidein matching.
    • GlideClientMatchingInternalPythonExpr: The Python expression that the frontend uses to determine which factory entries are acceptable for submission to.
    • GlideClientConstraintFactoryCondorExpr: A HTCondor classad expression to determine which factory entries can be considered.
  • Any custom information about the entry that is available through the glidefactory class-ad that a user can use for match-making purposes

Picture below shows a glideresource classad

New limits and curbs on number of glideins triggered appear in glideresource class-ad:

  1. To advertize Frontend limits/curbs

    2. Frontend adds the following attributes in glideresource classad about Frontend limits/curbs These new attributes in glideresource all start with the prefix "GlideClientLimit" to indicate that this attribute provides information about limit triggered. examples> If TotalGlideinsPerEntry on Frontend side is triggered, you will see in glideresource classad GlideClientLimitTotalGlideinsPerEntry = "count=11, limit=10" It basically says that "the limit is 10 and the actual count was 11, so the code had to do something about this"

  2. To advertize Factory limits,

    3. Factory adds the following attributes in glidefactoryclient classad These new attributes in glidefactoryclient all start with 2 prefixes GlideinMonitorStatus_GlideFactoryLimit GlideinMonitorStatus is prepended by Factory code for this attribute to be picked by Frontend code to become attribute of glideresource classad. The second prefix GlideFactoryLimit indicates that this attribute provides information about limit triggered. examples found in glidefactoryclient> If TotalGlideinsPerEntry limit is triggered on Factory side, you will see GlideinMonitorStatus_GlideFactoryLimitTotalGlideinsPerEntry = "count=11, limit=10" It basically says that "the limit is 10 and the actual count was 11, so the code had to do something about this" Frontend code looks at glidefactoryclient and picks up attributes that start with GlideinMonitorStatus and convert them to start with GlideFactoryMonitorStatus examples found in glideresource> Again, If TotalGlideinsPerEntry limit is triggered on Factory side, you will see GlideFactoryMonitorStatus_GlideFactoryLimitTotalGlideinsPerEntry = "count=11, limit=10"