Our Colo adventure Pt I – The Hardware

After renting for a number of years you eventually decide that it is more financially viable to own longterm. And whether you are talking about housing or hardware, the logic is all the same. Everyone needs housing. Everyone also obviously needs a 100TB+ monster of a server with massive Internet connectivity. Following with the analogy of home ownership, you can really customize your own server to suit your particular needs. Shopping for myself, and my co-worker, we decided that storage was most important. You can rent servers with half as much storage as we built, but it will cost you dearly.

st72

If you aren’t worried about getting a fixer-upper, you can save even more. You can get a “turn-key” chassis secondhand on ebay and get dual sockets and redundant power. Something like the Supermicro 846 series is a good choice for that. Or you can do “new construction” and build your own in a serviceable new chassis like a Norco 4224. This allows you to splurge on only the amenities that you are looking for.

We decided to go with a 5 year old Supermicro 846 that came fully furnished with dual 6 core Xeons and 48GB of memory. For the money, it’s an absolute screamer. I mean that in several ways. The noise is really oppressive if you were going to live with it. Fortunately, we planned on shipping ours off and only visiting when it was being very naughty. As the Supermicro included IPMI for out-of-band management, our need to actually visit the datacenter is going to be very minimal.

With dual hyperthreaded Westmere generation Xeons, we have 24 logical cores to play around with. This is especially important as we both plan on utilizing Plex for media streaming. The on-the-fly video transcoding can shape bandwidth for any situation, but it needs plenty of CPU. As Plex recommends 2000 passmarks per stream, we will be “limited” to 6 transcoded 1080p streams at any one time.

The really fun part for me was provisioning storage. It’s hard to really get a good idea of what 24 hard drives are like until you really work directly with them. It took me 45 minutes just to get them slotted in their hot-swap bays. We ended up with 24 Toshiba 5TB monsters as they were 7,200 RPM and had an excellent price at the time of purchase. Each of these things has a 128MB cache, so we have 3GB of caching before we even get to the controllers or OS.

We needed something to drive the drives, so to speak. The Supermicro allowed us to utilize a backplane with a SAS expander so that we had the option of feeding it a single SAS 8087 cable and run everything that way. Unfortunately, we would be limited to hardware RAID and have limited drive monitoring and error recovery options within the OS. The decision was made to utilize three IBM M1015 HBA’s. These are quite cheap on eBay and three all together have enough ports to give a dedicated 300MB/s lane to each hard disk. Flashed to IT firmware, they are perfect for next-generation RAID.

Overall I’m very happy with the hardware that we have, considering that our savings over something like a Dell or HP are enough to purchase a car.

kyle