[SSL Observatory] Using the SSL Observatory in the cloud

[Peter Eckersley]

This is a draft HOWTO for accessing and analysing SSL Observatory data using
Amazon EC2.  Feedback or corrections are appreciated.

1. Make an EC2 account:

   http://aws.amazon.com/ec2

2. Make a copy of the EBS snapshot that contains our data

   https://console.aws.amazon.com/ec2/home#s=Snapshots

   select Viewing: "Public Snapshots"

   search for:  snap-9010a2fc

   select the snapshot and click "Create Volume".

   You'll need to choose which data center to work in.  Virginia is a bit
   cheaper than other parts of the world.

3. Create a virtual machine instance for yourself.  The available list of
   systems and hourly prices is here:

   http://aws.amazon.com/ec2/
   http://aws.amazon.com/ec2/pricing/

   EC2 images come in odd sizes.  For queries over the valid_certs table
   you'll probably get good performance with ~8GB of RAM.  For queries over
   all_certs 17GB *might* suffice; but 34GB is safer.

   We've mostly been using "spot instances" which are noticeably cheaper but
   have the possibility of disappearing if the price spikes above your bid.
   Or you can pay more to avoid that possibility.

   We've been using the following "Community AMI" virtual machine image for our work:

   alestic-64/debian-6.0-squeeze-base-64-20090804.manifest.xml
   ami-2946a740

   Conceivably you could use any of the linux images provided it can mount JFS
   filesystems.

   Make sure you put your volume in the same data center that you selected in
   step 2.

   Along the way you'll make an SSH key that you can use to log in to this
   system.  With the debian image you log in as root.  The IP address is
   visible under Amazon's "Instances" menu.

4. Attach your volume to your virtual machine:
  
   https://console.aws.amazon.com/ec2/home#s=Volumes

   I'm going to assume you attach it as /dev/sdf

5. Log in to your virtual machine.

   mkdir /space
   mount /dev/sdf1 /space
   cd /space
   ./setup-script

   You will be asked a question about language settings. en_US-UTF8  works.

   You will be asked to set the msyql root user password.  Set it to "root"
   (or change dbconnect.py and obsdb to match your alternative password).

   Now you can run "obsdb" 


-- 
Peter Eckersley                            pde at eff.org
Senior Staff Technologist         Tel  +1 415 436 9333 x131
Electronic Frontier Foundation    Fax  +1 415 436 9993