Jump to:
Reference Guide to HTCondor Configuration Attributes
The tables below acts as a reference guide for condor_config attributes for GlideinWMS.
| GlideinWMS Service |
Attribute | Description |
| ALL | CONDOR_IDS = 500.500 | Condor user (uid.gid) |
| CONDOR_ADMIN = user at domain | Contact, via email, when problems occur | |
| UID_DOMAIN=$(FULL_HOSTNAME) | ||
| FILESYSTEM_DOMAIN=$(FULL_HOSTNAME) | With glideins, there is nothing shared | |
| LOCK = $(LOG) | Condor lock files to synchronize access to various log files. They use the log directory so they are collocated. |
|
| DAEMON_LIST = MASTER, COLLECTOR, NEGOTIATOR, SCHEDD | Condor daemons. The WMS Collector services will have the list shown above. The User Collector service uses all but the SCHEDD. The Submit service uses only the SCHEDD. The Frontend service is a client only instance and will have a 'DAEMON_SHUTDOWN = True' to insure no daemons are started accidently. |
|
| SEC_DAEMON_SESSION_DURATION = 50000 | Limit session caching to 12 hours | |
| SEC_DEFAULT_AUTHENTICATION = REQUIRED SEC_DEFAULT_AUTHENTICATION_METHODS = FS,GSI SEC_READ_AUTHENTICATION = OPTIONAL SEC_CLIENT_AUTHENTICATION = OPTIONAL DENY_WRITE = anonymous@* DENY_ADMINISTRATOR = anonymous@* DENY_DAEMON = anonymous@* DENY_NEGOTIATOR = anonymous@* DENY_CLIENT = anonymous@* |
Authentication settings | |
| SEC_DEFAULT_ENCRYPTION = OPTIONAL SEC_DEFAULT_INTEGRITY = REQUIRED SEC_READ_INTEGRITY = OPTIONAL SEC_CLIENT_INTEGRITY = OPTIONAL SEC_READ_ENCRYPTION = OPTIONAL SEC_CLIENT_ENCRYPTION = OPTIONAL |
Privacy settings | |
| GSI_DAEMON_TRUSTED_CA_DIR = full_path_to_CA_directory | Certificate directory | |
| GSI_DAEMON_CERT = full_path_to_your_hostcert.pem GSI_DAEMON_KEY = full_path_to_the_hostkey.pem |
Credentials | |
| GSI_DAEMON_DIRECTORY = full_path_to_your_condor_mapfile_directory | Set daemon cert location | |
| CERTIFICATE_MAPFILE = full_path_to_the_condor_mapfile | Condor mapfile | |
| HOSTALLOW_WRITE = * ALLOW_WRITE = $(HOSTALLOW_WRITE) |
With strong security, do not use IP based controls Condor 7.4+ only |
|
| GSI_DAEMON_NAME= | Identity of the GlideinWMS services used to authenenticate when communicating between serrvices. This is a comma separated list of their identities (DNs). | |
|
WMS Collector User Collector |
NEGOTIATOR_POST_JOB_RANK = MY.LastHeardFrom | Prefer newer claims as they are more likely to be alive |
| NEGOTIATOR_INTERVAL = 60 NEGOTIATOR_MAX_TIME_PER_SUBMITTER=60 NEGOTIATOR_MAX_TIME_PER_PIESPIN=20 |
Increase negotiation frequency, as new glideins do not trigger a reschedule | |
| PREEMPTION_REQUIREMENTS = False | Prevent preemption | |
| NEGOTIATOR_INFORM_STARTD = False | Negotiator/GCB optimization | |
| NEGOTIATOR.USE_VOMS_ATTRIBUTES = False | Disable checking for VOMS extended attributes (FQAN) | |
| NEGOTIATOR_CONSIDER_PREEMPTION = False | Causes Negotiator to run faster. PREEMPTION_REQUIREMENTS and all condor_startd rank expressions must be False for NEGOTIATOR_CONSIDER_PREEMPTION to be False | |
| COLLECTOR_NAME = wms_service | ||
| COLLECTOR_HOST = $(CONDOR_HOST):9618 | ||
| COLLECTOR.USE_VOMS_ATTRIBUTES = False | Disable checking for VOMS extended attributes (FQAN) | |
| ##EVENT_LOG=$(LOG)/EventLog ##EVENT_LOG_JOB_AD_INFORMATION_ATTRS=Owner,CurrentHosts,x509userproxysubject, AccountingGroup,GlobalJobId,QDate,JobStartDate,JobCurrentStartDate, JobFinishedHookDone,MATCH_EXP_JOBGLIDEIN_Site,RemoteHost ##EVENT_LOG_MAX_SIZE = 100000000 |
Event logging, if desired | |
| ##COLLECTOR_MAX_FILE_DESCRIPTORS = 20000 | Allow more file descriptors (only works if Condor is started as root) | |
| COLLECTOR0 = $(COLLECTOR) COLLECTOR0_ENVIRONMENT = "_CONDOR_COLLECTOR_LOG=$(LOG)/Collector0Log" COLLECTOR0_ARGS = -f -p 9641 DAEMON_LIST = $(DAEMON_LIST), COLLECTOR0 |
For each secondary Condor collector desired, the attributes specified must be set.
For more detailed information on this, refer to the Advanced Condor Configuration - Mulitple Collectors document |
|
|
WMS Collector Submit |
MAX_JOBS_RUNNING = 6000 | Allow up to 6K concurrent running jobs. |
| JOB_START_DELAY = 2 JOB_START_COUNT = 50 |
Start max of 50 jobs every 2 seconds | |
| JOB_STOP_DELAY = 1 JOB_STOP_COUNT = 30 |
Stop 30 jobs every seconds This is needed to prevent glexec overload, when used. Works for Condor v7.3.1 and up only, but harmless for older versions |
|
| MAX_CONCURRENT_UPLOADS = 100 MAX_CONCURRENT_DOWNLOADS = 100 |
Raise file transfer limits. No upload limits, since JOB_START_DELAY limits that. But do limit downloads, as they are asyncronous |
|
| APPEND_REQ_VANILLA = (Memory>=1) && (Disk>=1) JOB_DEFAULT_REQUESTMEMORY=1 JOB_DEFAULT_REQUESTDISK=1 |
Prevent checking on ImageSize With 7.8.+, the APPEND_REQ_VANILLA is replaced by the JOB_DEFAULT_REQUESTMEMORY/DISK attributes. Having all 3 attributes defined does not present a problem. |
|
| MAXJOBRETIREMENTTIME = $(HOUR) * 24 * 7 | Prevent preemption | |
| SCHEDD_SEND_VACATE_VIA_TCP = True STARTD_SENDS_ALIVES = True |
GCB optimization | |
| ENABLE_USERLOG_FSYNC = False | Reduce disk IO - paranoid fsyncs are usully not needed | |
| SHADOW.GLEXEC_STARTER = True SHADOW.GLEXEC = /bin/false |
Prepare the Shadow for use with glexec-enabled glideins | |
| MAX_SHADOW_LOG = 1000000000 | Limits size of shadow logs | |
| SEC_ENABLE_MATCH_PASSWORD_AUTHENTICATION = TRUE | Submit service only and only if you have enabled the USE_MATCH_AUTH attribute in the Factory and Frontend configuration files. | |
| SHADOW_WORKLIFE = 0 | This is a workaround to a Condor problem
if SEC_ENABLE_MATCH_PASSWORD_AUTHENTICATION = TRUE and Condor
version <= 7.5.3 See Condor ticket 1481 |
|
|
SHADOW.USE_SHARED_PORT = True SCHEDD.USE_SHARED_PORT = True SHARED_PORT_MAX_WORKERS = 1000 SCHEDD.SHARED_PORT_ARGS = -p 9615 DAEMON_LIST = $(DAEMON_LIST), SHARED_PORT |
SCHEDD Shared Ports Available only in Condor 7.5.3+. |
|
|
WMS Collector |
SCHEDDGLIDEINS2 = $(SCHEDD) SCHEDDGLIDEINS2_ARGS = -local-name scheddglideins2 SCHEDD.SCHEDDGLIDEINS2.SCHEDD_NAME = schedd_glideins2 SCHEDD.SCHEDDGLIDEINS2.SCHEDD_LOG = $(LOG)/SchedLog.$(SCHEDD.SCHEDDGLIDEINS2.SCHEDD_NAME) SCHEDD.SCHEDDGLIDEINS2.LOCAL_DIR = $(LOCAL_DIR)/$(SCHEDD.SCHEDDGLIDEINS2.SCHEDD_NAME) SCHEDD.SCHEDDGLIDEINS2.EXECUTE = $(SCHEDD.SCHEDDGLIDEINS2.LOCAL_DIR)/execute SCHEDD.SCHEDDGLIDEINS2.LOCK = $(SCHEDD.SCHEDDGLIDEINS2.LOCAL_DIR)/lock SCHEDD.SCHEDDGLIDEINS2.PROCD_ADDRESS = $(SCHEDD.SCHEDDGLIDEINS2.LOCAL_DIR)/procd_pipe SCHEDD.SCHEDDGLIDEINS2.SPOOL = $(SCHEDD.SCHEDDGLIDEINS2.LOCAL_DIR)/spool SCHEDD.SCHEDDGLIDEINS2.JOB_QUEUE_LOG = $(SCHEDD.SCHEDDGLIDEINS2.SPOOL)/job_queue.log SCHEDD.SCHEDDGLIDEINS2.SCHEDD_ADDRESS_FILE = $(SCHEDD.SCHEDDGLIDEINS2.SPOOL)/.schedd_address SCHEDD.SCHEDDGLIDEINS2.SCHEDD_DAEMON_AD_FILE = $(SCHEDD.SCHEDDGLIDEINS2.SPOOL)/.schedd_classad SCHEDDGLIDEINS2_ENVIRONMENT = "_CONDOR_GRIDMANAGER_LOG=$(LOG)/GridManagerLog.$(SCHEDD.SCHEDDGLIDEINS2.SCHEDD_NAME).$(USERNAME)" DAEMON_LIST = $(DAEMON_LIST), SCHEDDGLIDEINS2 DC_DAEMON_LIST = + SCHEDDGLIDEINS2 |
Atttributes for defining multiple schedds. The capability to define multiple SCHEDD daemons as a Condor attribute is available in Condor 7.5+. Effective with Condor 7.7.5+, a JOB_QUEUE_LOG attribute was added. When multiple schedds are used, each additional schedd must specify the unique location for that file. The execute, lock and spool directories for each schedd must be created manually. For more detailed information on this, refer to the Advanced Condor Configuration - Mulitple Schedds document |
|
SCHEDDGLIDEINS2_SPOOL_DIR_STRING = "$(SCHEDD.SCHEDDGLIDEINS2.SPOOL)" SCHEDD.SCHEDDGLIDEINS2.SCHEDD_EXPRS = SPOOL_DIR_STRING |
Allows the Factory to query the schedds directly bypassing the collector and thus improving performance. | |
|
Submit |
SCHEDDJOBS2 = $(SCHEDD) SCHEDDJOBS2_ARGS = -local-name scheddjobs2 SCHEDD.SCHEDDJOBS2.SCHEDD_NAME = schedd_jobs2 SCHEDD.SCHEDDJOBS2.SCHEDD_LOG = $(LOG)/SchedLog.$(SCHEDD.SCHEDDJOBS2.SCHEDD_NAME) SCHEDD.SCHEDDJOBS2.LOCAL_DIR = $(LOCAL_DIR)/$(SCHEDD.SCHEDDJOBS2.SCHEDD_NAME) SCHEDD.SCHEDDJOBS2.EXECUTE = $(SCHEDD.SCHEDDJOBS2.LOCAL_DIR)/execute SCHEDD.SCHEDDJOBS2.LOCK = $(SCHEDD.SCHEDDJOBS2.LOCAL_DIR)/lock SCHEDD.SCHEDDJOBS2.PROCD_ADDRESS = $(SCHEDD.SCHEDDJOBS2.LOCAL_DIR)/procd_pipe SCHEDD.SCHEDDJOBS2.SPOOL = $(SCHEDD.SCHEDDJOBS2.LOCAL_DIR)/spool SCHEDD.SCHEDDJOBS2.JOB_QUEUE_LOG = $(SCHEDD.SCHEDDJOBS2.SPOOL)/job_queue.log SCHEDD.SCHEDDJOBS2.SCHEDD_ADDRESS_FILE = $(SCHEDD.SCHEDDJOBS2.SPOOL)/.schedd_address SCHEDD.SCHEDDJOBS2.SCHEDD_DAEMON_AD_FILE = $(SCHEDD.SCHEDDJOBS2.SPOOL)/.schedd_classad DAEMON_LIST = $(DAEMON_LIST), SCHEDDJOBS2 DC_DAEMON_LIST = + SCHEDDJOBS2 |
Atttributes for defining multiple schedds. The capability to define multiple SCHEDD daemons as a Condor attribute is available in Condor 7.5+. Effective with Condor 7.7.5+, a JOB_QUEUE_LOG attribute was added. When multiple schedds are used, each additional schedd must specify the unique location for that file. The execute, lock and spool directories for each schedd must be created manually. For more detailed information on this, refer to the Advanced Condor Configuration - Mulitple Schedds document |
|
SCHEDDJOBS2_SPOOL_DIR_STRING = "$(SCHEDD.SCHEDDJOBS2.SPOOL)" SCHEDD.SCHEDDJOBS2.SCHEDD_EXPRS = SPOOL_DIR_STRING |
Allows for queries against the schedds directly bypassing the collector
and thus improving performance. Nothing has been implemented at this time to utilize this. Only Factory/WMS Collector communication uses this. |
|
| WMS Collector |
GRIDMANAGER_MAX_SUBMITTED_JOBS_PER_RESOURCE=5000 GRIDMANAGER_MAX_PENDING_SUBMITS_PER_RESOURCE=5000 GRIDMANAGER_MAX_PENDING_REQUESTS=500 GRIDMANAGER_PROXY_REFRESH_TIME=604800 SCHEDD_ENVIRONMENT = "_CONDOR_GRIDMANAGER_LOG=$(LOG)/GridmanagerLog.$(USERNAME)" |
Condor-G attributes The GRIDMANAGER_PROXY_REFRESH_TIME attribute was introduced in Condor 7.7.5 as a configurable attribute with a default of 6 hours. This forces Condor-G to re-delegate the proxy as soon as the Frontend provides one, defaulting to 1 week, since we do not expect proxies with longer lifetimes |
| QUEUE_SUPER_USERS = $(QUEUE_SUPER_USERS), factory_user | Only required when privilege separation is in effect. Allows the Factory user to bypass privilege separation making administration simpler without compromising security. | |
| Submit |
JOB_Site = "$$(GLIDEIN_Site:Unknown)" JOB_GLIDEIN_Entry_Name = "$$(GLIDEIN_Entry_Name:Unknown) JOB_GLIDEIN_Name = "$$(GLIDEIN_Name:Unknown) JOB_GLIDEIN_Factory = "$$(GLIDEIN_Factory:Unknown) JOB_GLIDEIN_Schedd = "$$(GLIDEIN_Schedd:Unknown) JOB_GLIDEIN_ClusterId = "$$(GLIDEIN_ClusterId:Unknown) JOB_GLIDEIN_ProcId = "$$(GLIDEIN_ProcId:Unknown) JOB_GLIDEIN_Site = "$$(GLIDEIN_Site:Unknown) SUBMIT_EXPRS = $(SUBMIT_EXPRS) JOB_Site JOB_GLIDEIN_Entry_Name JOB_GLIDEIN_Name JOB_GLIDEIN_Factory JOB_GLIDEIN_Schedd JOB_GLIDEIN_Schedd JOB_GLIDEIN_ClusterId JOB_GLIDEIN_ProcId JOB_GLIDEIN_Site |
User Job Class Ad Attributes |