Most folks think that because they have access to more than one cloud in vCloud Express, or any other cloud provider for that matter, you automatically get disaster recovery. In doing past articles on how to transform your blog into a vCloud Express based setup on Bitnami, I discovered a few things that I wanted to share about application availability. First and foremost, folks like Virtacore have multiple clouds. Even though you log in through one portal you have access to both their public vCloud Express in Los Angeles as well as Virginia. I actually showed this in the articles about using vCloud Connector and you could see their respective url’s in the screen shots.
You can see below from the Virtacore login that although the vApps are rolled together you can see the different cloud’s networks and firewalls. If you expand a single vApp you will see what cloud it is actually running in. If you did not pay attention when you deployed it for the first time, it may be in one or the other. This is easy to move between clouds of course using vCloud Connector, but really what if you wanted to maintain some level of redundancy to your vApp? That is what I set out to do with my new WordPress installation specifically.
Location, Location, Location
In an ideal world with stateless web applications this is fairly easy to do if you have a load balancer appliance, web servers in each cloud, and a shared database that is replicated between clouds. The challenge today is that not all vCloud Express providers have exposed the load balancer function of vShield Edge, or have a vApp deployed at the external network level that performs the functions of load balancing. Even in the case of WordPress that does not help because the local files still need to be replicated for both servers to be in sync with each other. Since this was needed either way I decided to go the poor man’s route and use DNS to help with failover. The only downside is not all ISP’s honor the TTL on DNS which means some folks did have trouble getting to my site in the first test. Below is the Virtacore portal view, but you need to look close to see which Virtual Machines are in what Datacenter. The locations for these were picked when I created them.
Network DNS Setup
What I decided to do was nest the DNS for the two clouds so that I only needed to make some minimal changes to get things up and running quickly. Below is examples of the records I created to facilitate the failover. In my case I also have a WordPress Multi-Site install with Domain Mapping which makes things a lot easier since all my sites are on one server.
@ – A Record to Virtacore LA External Address
iad.PrimaryDomain.com – A Record to Virtacore LA External Address
lax.PrimaryDomain.com – A Record to Virtacore LA External Address
www.PrimaryDomain.com – CNAME to iad.PrimaryDomain.com
For each domain mapping to a WordPress site I also did the same thing, but on less records and all the TTL’s are set to thirty minutes for those ISP’s that do honor the time to live.
@ – A Record to Virtacore LA External Address
www.MappedDomain1.com – CNAME to iad.PrimaryDomain.com
This means that in the event I need to switch over for something I only need to update the CNAME record for the primary domain. This will in turn update the mapped domains down the chain. I would need to update the various @ records if I want the non “www” to function but that could come later depending on the amount of time I needed to be running on the other site.
My application example here is WordPress which is a bit harder to deal with because the application is not stateless. Even with a Content Delivery Network, (CDN), your images are still stored locally. The content is in your database which you could replicate with MySQL, but I chose an even easier route. For one, I use a plugin called BackupBuddy to mange daily backups of the content and ship them off via FTP to another location. Secondly, as you can see from above, I have a completely duplicated server in the Virtacore Los Angeles cloud. Since this was built from a Virtual Machine, copying it with vCloud COnnector was easy. Once I had the duplicate server, I simply secure copy all the WordPress content folders nightly from VA to LA. Doug Smart, a buddy of mine has a nice little article on how to do Secure SCP with a CRON job that I used.
This means daily I not only have an offsite backup, but a near real time copy in the other datacenter if I needed to flip the DNS and cut over. You can apply some of the same logic to other applications, the goal is simply to make sure you recognize you need to THINK about the fact one provider with two clouds does not always mean you are by default redundant with your application deployments.
They key thing here to realize is that just because your provider has more than one datacenter, does not mean you get high availability or Disaster Recovery. As a consumer of a cloud provider, it is still somewhat on your shoulders to make sure your applications are deployed and managed in such a way that you can recovery them. One should not assume that the individual datacenters for any provider are automatically moving your applications from one datacenter to another. Could they be? Maybe but is that an assumption you want to make? Virtacore does a nice job of allowing you to see which datacenter your vApps are in and you do actually have access to the vCloud Director out of box portal for some functions their custom portal does not yet have. Between the two points of entry you have all the access you need to add workloads. Some might say this is overkill for a blog site…..but hey it needs to be up for people to read it right? Anyone that was part of the AWS east coast outage now knows this first hand that customer’s need to also make their applications ready.