GlideinWMS The Glidein-based Workflow Management System

Glidein Recipes

Overview

Roll new VM from an existing VM (EC2)

This recipe describes the process to take an existing VM and roll a new VM.

Start the VM on Amazon EC2 and SSH to it. This recipe assumes that the VM is running and that you are logged in. Make the changes that are desired. Then follow the folowing steps:

Make a directory to store your cloud credentials and scp them to that directory. In this recipe, we assume that the directory is:

/root/.ec2

The next step will be to bundle the running image using ec2-bundle-vol.

ec2-bundle-vol --generate-fstab -e /root/.ec2 -r x86_64 --kernel aki-9800e5f1 \
-k /root/.ec2/pk-XXXXXX.pem -c /root/.ec2/cert-XXXXXX.pem -u XXXX-XXXX-XXXX -p <image_name>

The following table explains what the various required options do.

Option Value Description
--generate-fstab N/A This generates the fstab for the image. This is dynamically created and inserted by EC2.
-e /root/.ec2 This option instructs that bundle process to ignore the credentials directory. This way the credentials are *not* bundled with the new image.
-r x86_64 Specifies the architecture for the new image.
--kernel aki-980035f1 Specifies the kernel that the image will request. In Amazon EC2, the kernels are dynamically associated with the images.
-k /root/.ec2/pk-XXXXXX.pem Specifies the private key to use. Replace the "X"'s with the appropriate file name.
-c /root/.ec2/cert-XXXXXX.pem Specifies the public key to use. Replace the "X"'s with the appropriate file name.
-u XXXX-XXXX-XXXX Specifies the account number to use. Replace the "X"'s with the appropriate account number.
-p <image_name> Specifies a name for the new image. This name will be re-used in subsequent commands.

Now that the image has been bundled, we need to upload the image to S3 storage.

ec2-upload-bundle -b <bucket_path> -m /tmp/<image_name>.manifest.xml -a XXXXXX -s XXXXXX

As before, the following table describes the required options:

Option Value Description
-b <bucket_path> S3 storage uses the concepts of "buckets". You can think of them as if they were directory. There are technical differences, but practically, they behave the same. If the bucket is one level (e.g. not nested), this command will create the bucket for you. If you wish to have a nested structure, then you will have to create the bucket path structure.
-m /tmp/<image_name>.manifest.xml This specifies the image manifest file. It is created during the bundle command.
-a XXXXXX Specifies Access Key for your account. Replace the "X"'s with the appropriate value.
-s XXXXXX Specifies Secret Key for your account. Replace the "X"'s with the appropriate value.

Now that the image has been modified, bundled, and uploaded, we have to register the image. Executing the following command will register the image and return the AMI_ID that will be used in any subsequent launch requests.

ec2-register <bucket_path>/<image_name>.manifest.xml -K /root/.ec2/pk-XXXXXX.pem \
-C /root/.ec2/cert-XXXXXX.pem -n <image_name>

As before, the following table describes the required options:

Option Value Description
<bucket_path>/<image_name>.manifest.xml This is the path to the manifest file. You must specify the full S3 path to this file.
-K /root/.ec2/pk-XXXXXX.pem Specifies the private key to use. Replace the "X"'s with the appropriate file name.
-C /root/.ec2/cert-XXXXXX.pem Specifies the public key to use. Replace the "X"'s with the appropriate file name.
-n <image_name> Specifies the name for the image that you are registering.

Now we are done with the current image. Shutdown the image and start using the new one!

New VM from scratch EC2

This recipe describes the process to roll a new VM from scratch.

TO BE ADDED.