Nov 12

Cluster 101: Building your first dumb cluster

Experimental, Linux No Comments »

I would like to introduce you tonight to a x-part series (I don’t know how many parts there are at the moment, we will see as time progresses) entitled “How to build a www cluster in x days“.

In this series, we will be looking at a wide variety of different techniques that can be used to create a cluster for hosting your next large project on.
When I talk about clusters, I like to categorise them into two particular sections: dumb and intellegent clusters. A dumb cluster is a group of machines that essentially know very little of their surroundings. They do not take into account how many machines are currently running, where the machines are located or how the data is to be devlivered to it’s location. Put simply: you set it up, and it works. No complex heartbeat configurations (except when necessary) and nothing fancy.
Intelligent clusters on the other hand know alot of information about where other machines on the network are. They know where the vistor to the website is coming from, and what the fastest way is to deliver content to that user.
Put simply: a “dumb cluster” is a group of machines that are set up to deliver content. If one machine goes down, the load balancer will detect this and bring that node out of the server pool until it is reported back up again. an “intelegent cluster” knows where the user is visiting. It can take into consideration what the shortest internet route is to it’s desired location, or what route should be taken to get the best availible speeds.
In this series, we will be looking more at the dumb clustering side of things. Dumb clusters are easy to set up, and well, if you really want an intellegent cluster, hire me :] So lets look at what we are going to discuss in this series.
  1. Setting up our first webserver
  2. Setting up our shared filesystem
  3. Setting up our http/https load balancer
  4. Setting up our mail server
  5. Setting up our backup system
  6. Setting up our mysql cluster
  7. Setting up our shell server
  8. Setting up our TCP/IP load balancer
For the purpose of writing this series, I will be using Xen installed on my machine, using CentOS 5.4 as the base opperating system. At any one time, I will have up to five domains running on Xen. I will be testing loads and response times using my laptop.
Stay tuned, the first part of the series comes on Tuesday.

written by Tim Groeneveld \\ tags: , , , , ,

May 28

Strange behaviour: Parallels Business Automation

Humor, Linux No Comments »

At Digital Pacific, where I work, I have been working on migrating from our current billing system to a new; more feature complete billing system that will provide many benefits to users. With our new billing system, Parallels Business Automation, users will be able to see all their invoices (past, present and future) and will be able to literally click one button and switch between all of their hosting control panels.

This move, however, has not been without quite a number of funny mishaps while moving from our current billing system (ModernBill) to Parallels Business Automation (”PBA”). The following text is going to go into quite a bit of technical information, so if you don’t know what XML is, well, you might just want to close this window go to Wikipedia’s article on XML and look it up!

Observe this image very closely:

 
 

 

So what I have done, is I have made a set of files with 100 customers each. Every separate file gets imported by running the command on the PBA server ‘hspc-import.pl’. Here, I am running the 59th file (the files start at 00). hspc-import.pl is complaining that the email for person “Becky A. Douglas” is not defined, however, when I go to edit the file, the email is infact correct, and it is definitely defined. Just like it is for the clients before her.

This issue has had me baffled for weeks trying to solve this issue. Sometimes it works, sometimes it does not. Sometimes changing the name in the XML file will just magically solve the issue, as if it was not there to begin with.

So what I do, is I get on Skype and call our Sales Engineer at Parallels, and tell him what the issue is. He points out a few issues with the XML file (which is fair enough, but why did all the clients before these “just” import without any issues?).

I take into account the issues that he pointed out, and go and create a new set of import documents to get our clients into our new billing system. I run all the commands again to start the process rolling, and now the issue is coming up again!

But now something very, very bad is going on! The import is actually failing faster then it was before the fixes that were suggested by our Sales Engineer. So I back track and change a few lines in the XML file. Instead of the top of the file saying this:

<?xml version="1.0" encoding="UTF-8"?>
<data>
<account>
<type>CUSTOMER</type>
<status>active</status>
<is_corporate>0</is_corporate>
<name>Bob Smith</name>

It now says this:

<?xml version="1.0" encoding="UTF-8"?>
<data>
 
 
 
 
 
 
<account>
<type>CUSTOMER</type>
<status>active</status>
<is_corporate>0</is_corporate>
<name>Bob Smith</name>

After adding exactly 15 new lines after the <data> child in the failing XML file in the import batch, you wouldn’t believe it. The import actually succeeded.

So now my new motto is, “Every bug in the world can be solved with just fifteen newlines!“.

written by Tim Groeneveld \\ tags: , , , , , , , ,