Wright Hassall Infrastructure Refresh

Anatomy of an HP Blade Server

2009-02-01T14:04:00.004Z

You've all heard the hype about Blade Servers but I'm sure some of you may be wandering what they look like, inside and out. Well, here it is.

As you can see, it is very narrow, not very tall and probably 2/3rd the depth of a regular server. The design is very, very neat and simple. 16 x DDR2 RAM slots will allow 8GB per slot, giving you potential for 128GB RAM (we've gone for 16GB for now) in each physical server. 2 x Quad-Core Processors per Blade at upto 2.7GHz (the spec we've gone for) and 2 x on-board 10Gbit NICs (Network Interface Cards) mean heavily specified but very small, efficient hardware.

On top of this spec you can also specify additional connectivity hardware in the form of 'Mezzanine' cards, which are essentially PCI-Express slots in a different form factor. There are 2 of these slots per Blade and we have utilised both slots on all of our Blades.

Slot 1 is a 2-port 1Gbit NIC. (for the purposes of VMware VMotion and High Availability)
Slot 2 is a 2-port 4Gbit Fiber Channel Card. (for fast connection to the Fiber Channel SAN)

New Toys (blade servers) up and running!

2009-01-29T00:28:00.004Z

Well, it's finally happened. Wright Hassall have some top quality servers in the building. This kit is pretty well regarded as perfect for what we are trying to achieve, plus a lot more. Lots of connectivity options, lots of flashing lights and enough fans to rival a small aircraft at the nearest airstrip! When those fans run full bore, they shift some serious air...

The outline of the gear is as follows:

2 blade enclosures (housing 9 blade servers), a nice big SAN (Storage Attached Network) and a fast tape drive. All attached into the new infrastructure on 4 redundant FAST links.

Specs and configuration to follow.

Enjoy.

{Penultimate Configuration}

2009-01-24T17:02:00.005Z

It's all in and working nicely. Documentation is being produced as I type by the consultants with overall configuration. Below is a quick outline of how it all fits together:

Server Room (Core)
a. 2 core switches in the server room which the servers will plug into directly.

b. 2 x 10Gbit Ethernet links between them in trunking mode creating 20Gbit throughout and Spanning Tree Protocol to enable one link to go down and still have full communication between the two.

c. 4 x Power Supplies in each core switch. Switches can function on 2 power supplies fully, so we have massive redundancy here.

d. 2 x phased power supplies, one to each core switch (UPS). One phased supply can fail taking one core switch out with it and no loss of connectivity will be experienced anywhere.

e. 5 x 10Gbit Fiber links, one to each floor in EACH switch (10 links total) thus providing fully redundant links to every data room.

f. Routing for all 8 VLANS.

g. 1 x 8Gbit Ethernet Trunked link into old switch which currently has old servers and printers on it. Soon to be retired.

h. 4 x 10Gbit Ethernet links to forthcoming 2 x Blade enclosures, housing 9 x Blade servers. 7 x Production, 2 x spare.

Data Rooms (Tree Branches)
a. 1 x switch in each data room

b. 4 x power supplies in each switch on single phase supply (UPS).

c. 2 x 10Gbit Fiber links from the switches in each data room creating 10Gbit throughput, one into each of the core switches. This provides Spanning Tree Protocol again allowing one fiber pair to break without any interruption in each data room.

d. Between 48 and 144 x 1Gbit links in each data room to each desktop machine (currently old workstations so 90% only running at 100Mbit until workstations replaced).

e. 1 x VLAN per data room, routed at core in server room.

All of the above equipment is PoE (Power Over Ethernet) certified (with sufficient power supplies to run this immediately) for potential upgrade of phone system to VoIP (Voice Over Internet Protocol). This would mean ridding ourselves of potentially 50% of the cabling in each data room!

Our Illustrious Leader.

2009-01-24T16:37:00.002Z

Matt (our IT Director), the man who made it all possible, through hours of Finance Consultancy meetings, begging letters, promises of a better world and general all-round persuasion has managed to fund this project and agreed to be photographed. Well, not exactly, but hey.

The look of horror on his face may be due to the sight of the equipment we removed from our production environment. I still can't believe we used this stuff in anger and it didn't totally fail! None of this equipment is managed, not routed, nothing. We can all breathe a sigh of relief now!

Server Room Photos And Serious Bandwidth...

2009-01-24T16:13:00.004Z

Ok, so it's time for some photos of the jewel in the crown, the Server Room.

Cabling will be tidied on a rainy day but ultimately this is finished now.

Enjoy.

Let's Refresh This Infrastructure then!

2009-01-24T15:31:00.008Z

Ok, so it's been rather a busy week and it's finally done and dusted.

The decision was made to move from our current IP structure which saw every machine and server sitting on the same subnet and creating rather a lot of broadcast traffic all over the building unnecessarily. This in turn accentuated the lack of intelligent routing in place throughout the entire infrastructure and was cause for some serious change.

Each wing has been assigned its own switch and VLAN which in reality splits it off into a subnet which in turn is routed properly by the core switches in the server room. The new servers will be assigned a VLAN in much the same way and legacy equipment (just servers now) on the old IP range is routed properly through the core switch aswell, until it is phased out completely.

Static routes were then applied to each of our internet connections, giving full access to both from the new routed network. One 2Mbit fiber feed from one provider and one 8Mbit ADSL feed from a different provider. One or both of these will shortly be replaced by a 10Mbit feed from a new different provider to move inline with our refresh and enable better throughput for external client access and internal internet access. This is another sub-project which will tie in nicely with the replacement of the firewall and client-access gateway moving forward.

First we had to devise when to rip out all of the old kit and replace it with the new kit giving no downtime at all to users. One wing per night was decided after everybody had gone home, starting with the IT Department on the ground floor and the IT Training Room on the top floor (which uses a just a few sockets in its own wing). This way any errors would be insignificant and all the production users would not be affected.

All went swimmingly and each machine started up with a new IP from the VLAN range set on the respectuve wings. Routing to the servers (and other machines in different wings) worked perfectly from the new IP range and vice versa. Success!

Below are some photos of the Data Rooms dotted about the building and the new Hewlett Packard equipment now installed in them.

You will notice that the old Netgear switches are still installed in the racks with cables plugged in. These are the printers' connections until we change the IP addresses, they will remain there and then be migrated onto the new switches.

Visit to supplier and New Switch Gear preview.

2009-01-13T17:55:00.005Z

Well, myself and Stuart had a nice visit to our new preferred IT supplier who are running the infrastructure project and we got a sneak preview of the switching/routing gear along with inputs into its configuration. This side of the project is about ready to go, so sometime next week we will receive the gear and install it into the racks in all of the data rooms ready for switchover...

The setup we have gone for includes the following hardware:

HP ProCurve 5412ZL IntelligentEdge Switches
HP ProCurve 3500YL IntelligentEdge Switches
HP ProCurve 10Gb Ethernet Transceivers
HP ProCurve 10Gb Fiber Transceivers
HP ProCurve 1500W Power Supplies
HP ProCurve VRRP Protocol Licensing

The configuration we have gone for will be illustrated after the installation has been completed.

The 'before' shots! Lean on me (I dare you!)...

2009-01-13T00:05:00.000Z

Well, I guess it's time to show off some 'before' shots to give you an idea of what we're up against.

Working in IT, I have visions of clearly documented equipment and configurations, neatly laid cabling, fiber patching that isn't in danger of being damaged by straying hands, floors that are clean and tidy, servers that are logically laid out, etc, etc, blah, blah...

As you can see from the following photos, this is not currently the case. But we're on the case! Bear in mind we have had a MASSIVE clear out and have spent lots of time really pushing the boat out with tidying and consolidation. The idea was: get all the servers we actually use into a small space, turn off the ones we don't use and dispose of them appropriately. Some old ones are absorbed into our test network (if they're good enough), some are given the proverbial 'axe'.

The photos, running in no particular order, illustrate the following:

Data Room - patch panels with switching gear (no routing used currently at all) and fiber patching.
Server Room - patch panels with core switching gear and fiber, servers, storage, backup solutions, lots of spare frames and space.

Pre-requisite check and prep work.

2009-01-12T23:06:00.000Z

First job was to nip round the building and see exactly what we were dealing with.

It quickly became obvious that bottlenecks were present, dodgy equipment was being used and all the standard networking, Microsoft, IBM-PC and all other best practices were not being used, even slightly.

A particularly mismatched set of networking infrastructure is currently in use with misconfigurations, less than reliable interconnection methods and a total lack of redundancy throughout the building which leaves us with a large pause for thought. If we're to land ourselves ahead of the legal IT circle then we need to get our act together. And this thought is the basis for our large project.

Examining the server and data rooms reveals that we have an abundance of fiber running throughout the structure of the building which is being HUGELY under-utilised. The first part of the project will be to use this to our advantage without any new fiber patching, re-routing or splicing (therefore giving us a good headstart for minimal outlay).

All equipment we currently use is being retired and replaced with industry-standard 'recognisable' Manufacturer's hardware (read Hewlett Packard, here) in preparation for the forthcoming 'Software and Services Refresh' later this year.

This 'refresh' equates to the following being updated:

Data Rooms - Switching and Routing Gear. Fiber cabling.
Server Room - Switching and Routing Gear. All Servers. Shared Storage (SAN). Links between Servers, Storage and Main Network. Fiber and quick Ethernet cabling. Uninterruptible Power Supply (UPS) on different phases.

Check back soon!

James.

The beginning!

2009-01-12T21:20:00.000Z

Well, finally got off my backside and decided to do one of these 'blog' things and what better subject matter to use straight off the bat than the immense project ahead of us at Wright Hassall LLP. The next few months will see us move from a slow, aging infrastructure into the future-ready solid centre-piece of legal IT that we can benefit from.

There is a massive amount of work involved and every member of the team is getting stuck into the most exciting IT project the firm has witnessed in recent years.

Stay in touch to see what we've been upto.

James.