GlideinWMS The Glidein-based Workflow Management System

Search Results

1.Description

NOTE: It is strongly recommended to use CCB available in new version of condor over GCB. CCB provides same functionality as GCB and has performance benifits. Using GCB requires additional installation of condor daemon whereas this feature is integrated in newer versions of condor.

This node will serveas a Generic Connection Brokering (GCB) node. If you are working over firewalls or NATs, and are using an older version of condor (before v7.3.0) you will need one or more of these. If in use, GCB is needed every time you have a firewall or a NAT. If this node dies, all the glideins relying on it will die with it. If possible use CCB instead.

2. Hardware requirements

This machine needs a reasonably recent CPU and a small amount of memory (256MB should be enough).

It must have a reliable network connectivity and must be on the public internet, with no firewalls. It will work as a router. It will use 20k IP ports, so it should not be collocated with other network intensive applications.

The machine must be very stable. If the GCB dies, all the glideins relying on it will die with it.. (Multiple GCBs can improve this by minimizing the damage of a downtime, but this machine should still be on the most stable machine affordable).

About 5GB of disk space is needed for Condor binaries and log files.

As these specifications are not disk/memory intensive, you may consider collocating it with a VO Frontend.

3. Needed software

You will need a reasonably recent Linux OS (SL4 used at press time), and the Condor distribution.

4. Installation instructions

The GCB should be installed as a non privileged user.

The whole process is managed by a install script described below. You will need to provide a valid Condor tarball, so you should download it before starting the installer.

Move into the "glideinWMS/install" directory and execute

./glideinWMS_install
You will be presented with this screen:
What do you want to install?
(May select several options at one, using a , separated list)
[1] glideinWMS Collector
[2] Glidein Factory
[3] GCB
[4] pool Collector
[5] Schedd node
[6] Condor for VO Frontend
[7] VO Frontend
[8] Components
Select 3. Now follow the instructions. The installation is straightforward.

The installer will also start the Condor daemons.

4.1 Starting and Stopping

To start the Condor daemons, issue:

cd <install dir>
./start_condor.sh

To stop the Condor daemons, issue:

killall condor_master

4.2 Verify it is running

You can check that the processes are running:

ps -u `id -un`|grep gcb
You should see one gcb_broker and at least one gcb_relay_server.

You can also check that they are working by pinging it with gcb_broker_query:

<install dir>/sbin/gcb_broker_query <your_ip> freesockets

4.3 Fine tuning

4.3.1 Increase the number of available ports

The default installation will set up GCB to handle up to 20k requests. Look in the <install dir>/etc/condor_config.local for

GCB_MAX_RELAY_SERVERS=200
GCB_MAX_CLIENTS_PER_RELAY_SERVER=100
This is enough for approximately 4000 glideins (each glidein uses 5-6 ports).

If you want a single GCB to serve more glideins that that, you can increase those numbers. However, be aware that the OS also has its limits. On most Linux systems, the limit is set in /proc/sys/net/ipv4/ip_local_port_range.

$ cat /proc/sys/net/ipv4/ip_local_port_range
32768 61000
For example, the typical port range listed above has only ~28k ports available. If you want to configure GCB/CCB to serve more than that, first change the system limit, then the GCB/CCB configuration.

4.3.2 Increase the number of available file descriptors

Note that every port used by the GCB/CCB also consumes available file descriptors. The default number of file descriptors per process is 1024 on most systems. Increase this limit to ~16k or value higher than number of ports GCB/CCB is allowed to open.

This can be done by issuing a "ulimit -n" command as well as changing the values in the /etc/security/limits.conf file