Experiments with Running WordPressMU on EC2

Now that Amazon offers fixed IP addresses for their EC2 service, I am starting the process of researching what it would realistically take to implement WordPressMU on EC2.

I’m going to capture some notes here about the process. This post is a raw unfiltered list of random thoughts.

I am not writing this per se for anyone else, but since I’ve found these items out there and they’re useful to me, they may be useful to someone else. I may assume you already are very familiar with WordPressMU and/or MySQL, and certainly that you got admin skillz if you’re reading this.

Meta Point: You have no dedicated disk drive!

So the big issue with Ec2, and arguable grid computing vs dedicated servers is that you’re server instances don’t have a physical hard drive associated with them. Sure, they have disk space when they’re running, but as soon as they shut down that disk image is gone FOREVER… Obviously, if you plan to run a database on one of these machines, you have to think about this carefully.

It’s not like if you accidentally reboot your dedicated server at least you know your DB probably shut down safely, and will probably wake up working just fine. That disk is dedicated to you. But not so with EC2… that disk doesn’t exist anywhere. So you need to really think about how to make that disk image persistent.

But wait a second? Is that really an issue?

Well, if you’re lazy, or even if you’re not lazy and you’re counting on regular backups and the idea that your system could lose some data and you’d be ok… then maybe you haven’t built a system that withstand a complete drive failure. You figure, hey, I’ve got my dedicated box, I do regular backups, I copy those backups off of the box… Maybe I’m already using S3 to store those backups… no big deal. I could rebuild my box if I needed to.

Well, if that’s your attitude, then you would be flying pretty much by the seat of your pants on a pure cloud system like Ec2.

But, if you already have a redundancy and backup strategy that can handle a complete disk failure (as in there is nothing left on the disk) then you’re in perfect shape for running in the clouds.

So… here are some notes that are mostly EC2, MySQL, Linux specific about implementing various pieces of this puzzle.

Some great articles out there

General EC2 Articles

• The Official Amazon Ec2 Getting started guide -This is a pretty basic article but it explains everything you really need to know about setting up a basic EC2 instance.
• The Official Amazon EC2 Documentation (v2008-02-01) – This contains the actual documentation for all of the command line utilities, etc, that you need to get rolling. I’m the kind of guy that likes to know all the parameters available to me, so if I was walking through a how to guide that’s saying what to type in where, I still want to know all the options and why they work the way they do.

Mysql Database “strategy” articles:

• CodeWord’s Article on Running MySQL for large datasets on EC2 – This article is more than a year old, so it describes limitations in Ec2 that have been fixed, but the concepts of running MySQL in a double paired master/slave mode is a clever solution and totally appropriate for the project I’m working on.
• MySql documentation on mysqld_multi – This is the tool you’ll want to use when you run multiple mysqld instances on a single machine.
• Setting up Master-slave
• Setting up Master-Master

Articles about LVM backups:

• DBA Dojo’s Article on Setting up EC2 with LVM snapshots – Also totally appropriate, includes some handy scripts which demonstrate things like setting up your logical volumes, etc.
• PointyHair Article on Snapshot backups – Here’s another article about snapshots. It’s interesting as it shows how to do this from mysql, using mysqls ability to call shell commands in between SQL commands. This article also describes the (should be obvious) aspect of actually doing the real backup on the snapshot results…. so for example this is where you might copy to content to S3 using one of the below products.
• Another article that talks about LVM, MySQL Replication, and backups.
• Mike Kruckenberg’s post on the topic of LVM snapshot backups.

Articles related to file systems useful as “backing stores”:

• Great Thread Discussing different types of “backing stores” using S3 to make your a persistent image of your Ec2 data.
• ElasticDrive – Tool mentioned in the above thread… not exactly free, but looks like it has some useful features at scale.
• JungleDisk – Tool mentioned in above threads… practically free, solves a slightly different problem then ElasticDrive, but certainly would work nicely in the model described by CodeWord.
• PerstistFS – This is a FUSE based file system designed for S3/Ec2

Setting up my Image

I wanted to utilize the EC2 m1.large type for my database so that I would have plenty of room for DB performance. And I wanted to use Fedora Core (6 or 8).

So I started with the amazon public AMI for fedora core 6, which pretty much doesn’t include anything but the OS. I then installed: httpd, mysql, php, and subversion (since we like to update our code from our svn server). [hint: yum install httpd mysql mysql-devel mysql-server php subversion]

From this I made my own image that was fedora core, plus the basic LAMP components. This means following the standard directions building a bundle uploading it to S3, registering it, etc. Now I can start up new instances without having to reinstall all that stuff.

Installing WordPress MU: The code

Well, this is actually pretty easy, I just checked out our version of wpmu from our svn server. Our version includes all of our custom plugins and themes we’ve built.

Installing WordPress MU: The database

More to come.

Random Issues:

• For some reason I had some problems with the default amazon image for fedora core 6 when using an assigned IP attempting to do yum. I kept getting manifest checksum errors. Not sure if it’s a F6 issue or an assigned IP issue. I was able to yum a bunch of stuff with both FC6 and FC8 when not assigned a fixed IP. I had no problem when I recreated the instance (without associating it with the IP) and was able to yum and ultimately create an image I wanted. But it was a pain. Since then I’ve also been able to yum from an instance that is assigned to an IP… so really I have no idea what was going on here.

Advertisement