Glidein Recipes -SciTokens Authorization

Description

This page documents a recipe for enabling and managing SciTokens Authorization between Glidein VOFrontend Groups and the Compute Elements(CE)s they request glideins to run on.

Requirements

Requirement	Description
Compatible GlideinWMS Versions For both Factory and VOFrontend	This documentation refers to GlideinWMS v3_7_3 and above for both Factory and VOFrontend
Compatible HTCondor Versions	GlideinWMS v3_7_3 installs condor 8.9.11 at the time of this writing. Condor version must be 8.9.11 or greater
An HTCondor Compute Element (htcondor-ce) Compatible with and correctly configured for SciTokens	Minimal version requirements at the time of this writing htcondor-ce.noarch 4.4.1-3.el7 @htcondor-development htcondor-ce-client.noarch 4.4.1-3.el7 @htcondor-development htcondor-ce-condor.noarch 4.4.1-3.el7 @htcondor-development condor.x86_64 8.9.11-1.el7 @htcondor-development
A SciToken that the htcondor-ce will accept	Link to method for generating a working token is given below. This is not how SciTokens would be generated and used in a production environment.

Obtain a SciToken

These instructions are expected to change rapidly, so for release 3.7.3 we will link to older instructions

Configure VOFrontend to use SciTokens

The SciToken must be configured in the "credentials" section of the frontend.xml config file. Example:

<credentials>
  <credential absfname="/var/lib/gwms-frontend/tokens.d/fermicloud127.scitoken" security_class="frontend" trust_domain="OSG" type="scitoken"/>
</credentials>

In this example, absfname is the full path to the SciToken. The type must be set to "scitoken". The trust_domain and security_class are used to communicate routing and matching when the credential is passed the factory. They behave the same way for SciTokens as they do for other credential types.

Multiple credential lines can be placed in a frontend <credentials> section. Example

<credentials>
  <credential absfname="/var/lib/gwms-frontend/proxy1" security_class="frontend" trust_domain="OSG" type="grid_proxy"/>
  <credential absfname="/var/lib/gwms-frontend/proxy2" security_class="frontend" trust_domain="OSG" type="grid_proxy"/>
  <credential absfname="/var/lib/gwms-frontend/tokens.d/fermicloud127.scitoken" security_class="frontend" trust_domain="OSG" type="scitoken"/>
</credentials>

In such a configuration, all the above credentials will be passed to the factory. The factory will try SciTokens authentication to the CE first, if this is unsuccessful, the other credentials will be tried as dictated by the xml configuration of the factory.

If the SciToken is refreshed on the VOFrontend it will be updated at the Factory. Currently running glideins will not automatically pick up the refreshed SciToken.

The frontend.xml configuration file has two <credentials> sections that can pass on a token, the "global" and "frontend group" sections. A "frontend group" configured SciToken will be preferred over a "global" SciToken if they exist in both <credentials> sections.

Configure Factory to submit your SciToken

Beginning of an example SciTokens entry configuration:

<entry name="el7_osg35ST" auth_method="scitoken" grid_type="condor"  trust_domain="OSG"  ...../>

Note that the auth_method is "scitoken", meaning no GSI credentials are sent to this entry. If the auth_method were "grid_proxy" or other GSI based method, and the frontend was configured to send a SciToken to the factory, SciTokens authorization would be attempted prior to any other method, with a fallback to GSI if there was a problem with the token auth.

Verify Token Functionality

SciTokens can be used to interrogate the CE's collector using condor_ping from either the Factory or VOFrontend to verify that the token is valid and the CE is correctly configured.

[root@fermicloud359 ~]# MY_CE=fermicloud127.fnal.gov
[root@fermicloud359 ~]# _condor_SCITOKENS_FILE=/path/to/scitoken condor_ping -address "<$(host $MY_CE | awk '{print $4}'):9619>" -table ALL
         Instruction Authentication Encryption Integrity Decision Identity
               ALLOW      SCITOKENS       none       MD5    ALLOW osg@users.htcondor.org
                READ           none       none      none    ALLOW unauthenticated@unmapped
               WRITE           FAIL       FAIL      FAIL     FAIL FAIL  (use -verbose for more info)
          NEGOTIATOR           FAIL       FAIL      FAIL     FAIL FAIL  (use -verbose for more info)
       ADMINISTRATOR           FAIL       FAIL      FAIL     FAIL FAIL  (use -verbose for more info)
               OWNER           FAIL       FAIL      FAIL     FAIL FAIL  (use -verbose for more info)
              CONFIG           FAIL       FAIL      FAIL     FAIL FAIL  (use -verbose for more info)
              DAEMON           FAIL       FAIL      FAIL     FAIL FAIL  (use -verbose for more info)
    ADVERTISE_STARTD      SCITOKENS       none       MD5    ALLOW osg@users.htcondor.org
    ADVERTISE_SCHEDD           FAIL       FAIL      FAIL     FAIL FAIL  (use -verbose for more info)
    ADVERTISE_MASTER           FAIL       FAIL      FAIL     FAIL FAIL  (use -verbose for more info)
[root@fermicloud359 ~]#

Another check is to disable GSI communication between the Factory and the CE by changing the auth_method from "grid_proxy" to "scitoken", reconfig the factory, and submit a job. If the factory can still submit jobs to the CE, SCITOKENS authentication must be working.

Useful Links

SciTokens Claims and Scopes Language A good description of what claims you may want to put in your SciTokens and why.
WLCG Common JWT Profiles WLCG Tokens are 'nearly' interoperable with SciTokens and are more mature.

GlideinWMS The Glidein-based Workflow Management System

SciToken Authorization