Last week enough parts arrived that I could start putting together the first of my two new servers. In the end, I decided to buy SuperMicro barebones servers instead of HP or Dell (or IBM) servers because of the size of the hard drives we wanted. My co-worker and I came to the figure of 4TB for where we expected company-wide data storage to be in five years time, based upon the current size and the growth we’ve had and are expecting.
In order to build a RAID5 array of 4TB, we had to have five 1TB drives. Of course, 1TB drive doesn't actually HAVE 1TB of drive space on it, it’s only about 927Gb or so because of the whole 1000 vs 1024 multiplier. Sure it was fine in the days of 100Mb drives, but now it’s just ridiculous. Five 1TB hard drives yields a 3.6TB array. We are “missing” or “losing” 400Gb or almost half of one of those terabyte drives when extrapolated across the array.
The hard drives that Dell and HP (and I’m assuming IBM/Lenovo) use have custom firmware on them so that the onboard diagnostics can talk to the drive and receive information from them. This means that the same Seagate Barracuda or Western Digital Caviar 1024Gb drive that costs about $166 at NCIX or Tiger Direct costs $924 from Dell Direct or CDW. You also need the hot-swap caddy for that particular server, and they don’t sell those separately (unless you find some on Craigslist or eBay). That would have meant that I spent more on those hard drives than I ended up spending on the entire SuperMicro server.
This is the second time I’ve dealt with SuperMicro. When PC Powerhouse closed it’s doors, we (my old company) bought up their server rack, patch panels & switches and there were two SuperMicro 2U servers in there. We called it the Sharktank and used it to set up a completely separate network with a copy of our Active Directory on it to use for testing purposes. We also bought a third SuperMicro 2U server and stuffed it full of 500Gb hard drives to use as a disk-based backup solution. I was impressed with the build quality then and when I needed a cheaper alternative to brand-name servers here at my new job, I went to SuperMicro again.
Fortunately CDW carries SuperMicro servers. NCIX does as well I found out which means I have two suppliers I can have compete against each other for better pricing. The first one arrived mid-week last week and I put it together in one afternoon. These particular servers are Intel Xeon quad-core processors, 4Gg RAM and two 250Gb hard drives in a RAID1 (mirrored) configuration with Windows Server 2008 x64 Standard Edition running on it. The RAID controller is an Adaptec 3805. In addition, they also have the five terabyte drives configured in a RAID5 array. These servers have redundant 750 watt power supplies and are plugged into an APC 2U Rackmounted UPS pushing 2200VA.
So begins the headache. The maximum disk size that windows XP, 2003 and Vista (non 64-bit versions) can see is 2Tb. My array is 3.6Tb. Try as I might, I could not break through that 2Tb maximum. The drive just didn’t show up in the Disk Management snap-in. I tried everything I could think of, it just wouldn’t show up.
I deleted the array that I had created in the controller BIOS settings and re-created it in Windows using the Adaptec Storage Manager (ASM). No good. As soon as I added the 4th drive to the array, the available disk size went from 1.8Tb to 2.0Tb and ignored the remaining 1.6Tb. I searched and searched and searched all weekend and asked every SysAdmin I knew and had access to via IM, email, phone and shouting over a live band at a pub Saturday night. No one had any insight.
I found out about GPT during this time though, and how it works and what it does. There are a lot of limitations to using GUID Partition Tables instead of MBR mostly due to BIOS limitations. EFI bios can boot from GPT disks, so that means all Macs can, but only Windows XP x64, Vista x64, Server 2003 SP2 X64 and Server 2008 x64 can BOOT from a GPT. This had no bearing on my setup as I wasn’t booting from this disk, it was simply a big data drive. There’s supposed to be a way to right-click an unrecognized disk in the Disk Management snap-in and Convert to GPT (or Convert to MBR) but since my Disk1 was not showing up there, I couldn’t do it. FRUSTRATION SETS IN.
I came in over the weekend to relocate the server from my workbench into the rack and re-created the RAID5 array and initiated a Build/Verify rather than a Quick Init. After two hours of solid disk LED lights, the progress meter changed to 1%. Oi. I left and went home for the weekend, thinking that it should be done by Monday morning, and once the drive array is Optimal, then maybe it will magically appear.
No suck luck. I arrived this morning to an Optimal array but still nothing in the Disk Management snap-in. I opened the Device Manager and checked through there to make sure that the Adaptec 3805 had the correct and up-to-date driver. It did. When I clicked “check online for a new version” it returned a message that I already had the best driver for the job. Fortunately I’m not that trusting of Windows Update.
I went to the Adaptec website and navigated through to the 3805 downloads. there was a newer firmware available, but there was a new, windows-certified driver for Server 2008 x64 that was dated Oct 2, 2008. I downloaded that driver and copied it over to the server. The documentation suggested that I could either do it via rebooting the server and booting from a floppy, or I could do it via the Adaptec Storage Manager console itself. I updated the driver and Windows Server 2008 said “your new driver is installed but will not be working correctly until you restart your computer.” Since this is a new server and there’s no data on it yet (hell there’s nowhere to PUT the data) I clicked OK and when it asked me to reboot, I clicked yes.
I was disconnected from the Remote Desktop, and since I don’t have a console KVM in my rack just yet, I kept my fingers crossed and waited a few minutes for the server to come back up.
I re-connected via Remote Desktop (as an aside, as of November 19, 2008 RealVNC’s free version does not work with Windows Vista or Server 2008, just their pay versions do) I fired up the new Server Manager and expanded the + sign next to storage and clicked on Disk Management…
HOLY JUMPING JESUS ON A POGO STICK I HAVE A NEW DRIVE SHOWING!
Disk 1 unknown 3723.99Gb NOT INITIALIZED. The Initialize Disk Wizard popped up on it’s own and asked me how I wanted to initialize this disk: MBR or GPT? Even the note at the bottom is good: The GPT partition style is not recognized by all previous versions of Windows. It is recommended for disks larger than 2TB, or disks used on Itanium-based computers. Honestly, Itanium? Who even USES those? In this case, I’m going to go with GPT because I’m never going to boot off this drive, and Windows Server 2008 sees the GPT partition just fine. The XP Client computers and other Windows server 2003s that will be working with the data on these drives will all be accessed over the network via SMB anyway so it’s all good.
FINALLY the drive is ready to be formatted as NTFS (which should take another bunch of hours, even as a quick format) and I can start preparing my checklist for migrating the old file server on to this one.