It's looking very impressive on the outside and even more so on the inside. It's huge, it's one of a kind and state of the art. But this new entity has not come without its shortfalls in certain areas along the way. Areas: namely IT.
Strangely all the way through the project, in an IT based company with almost everything running off of technology, IT was not taken into consideration when designing the new building. Even stranger, when they finally realised that computers will be needed to run the business, and that a full network infrastructure would need to be put in place, they didn't bother to speak to the experts: Us!
A server room is finally added to the plans. Whoever designed it had obviously been watching way too much Channel 4: Our new Server Room and Offices are now located in the basement. How very cliché, all we need now are some incompetent users. Oh wait...
The fun started when we were told that we wouldn't need much of an electrical supply, apparently servers don't take much and a double 13 Amp plug socket would do the trick. Sadly not, think again. With the potential for there to be eleven or more full racks containing hardware including Servers, 2 SANs, several Switches and countless other goodies that run the company, we would have been tripping before we had turned 5% of it on.
They offered us a 40 Amp supply instead, because "that was what they had calculated." This was still not enough and we told them so, only to be told we were wrong. They argued the point until we finally showed them the current server room, and the documentation. We needed 75 Amps per rack. It was a struggle, but they finally agreed to our demands.
I honestly think that these people believe a network is run on a single, desktop PC under the IT manager's desk and a single network cable that magically connects every PC together. And the Internet, well that's 'just there' isn't it.
Next there was a lot of air being sucked through teeth at the mention of a cooling system, and a lot of umming and arrhhing over whether it would even be required. The new building is to be eco-friendly with temperature control and nothing extra would be required. After all that single server under the managers desk wouldn't put out much heat would it?
We solved this one easily, we just explained to the directors that we would get everything set up and when the servers got too hot they would just simply, shut themselves down to prevent damage to the equipment. Cutting off everyone's email access and other vital systems: Payroll for instance and Ebay. Just think of the money they would be saving on their electricity bill.
They promptly decided to put air-conditioning units in (what a good idea, why didn't we suggest that?) For safe measure they put in three, on the wrong wall, directly behind where the racks were going to be placed. Communication; you can't beat it. Common sense; you can't buy it.
Christmas Eve:
It had been planned for about two months. Network ports had been changed, cables had been run and labelled, and diagrams depicting the complete rack configurations were gracing our walls in blazing A3 techno colour. A work of art. The great server move was all set to go ahead, amid the Christmas break.
We make our way over to make the final preparations only to discover workmen replacing our vinyl flooring with temporary floor tiles blocking our way in.
It is half an hour before we finish for Christmas, and we then discover that the electricity supply was neither running nor reliable as the new UPS' keeps tripping and switching off. To top it off we were then told that the contractors were going to be running air pressure tests the same day as the move: Nice of them to tell us at all really. This meant that instead of a progressive move, we would have to shift everything in one go before we can connect it up again. We would then be sealed inside the new structure until the tests were complete. It would be a rush and hard going, but we were fairly certain we could get it done before the lock down.
Five minutes before we leave we are told that the electric is still not running, and that the move is in doubt, it's left hanging in the air as we make our separate ways home, looking forward to a sausage roll, and the Doctor Who Christmas Special.
A Few Days Later:
It's a cold Wednesday morning and Christmas day has already faded into history. It's 8:45 am and seven of the best have cut their seasonal holiday short to move the contents of the server room to its new home.
First things first: Is there any power? We had been assured that there is, but why get three racks of equipment disconnected if you have to put it all back together again? We suit up, and go and check.
The electric was indeed up and running as promised, and of no concern. It was the two inches of foul smelling, muddy water flowing freely across the outer corridor that worried us slightly. As we waded into the new office we followed the flood to the doorway of the new server room. We thought we had struck lucky, and we were getting a company swimming pool too, but sadly we were wrong. The nice new floor tiles were already being ripped up. Two months later they are replaced with another vinyl lino floor.
The pressure testers had blocked off all the drains before finishing for Christmas, ready for the testing, with the temperature rising slightly outside, the snow that was piled on the roofs, and around the site had melted and had nowhere to go. This resulted in a major flood and several hundred pounds worth of damage. The room opposite ours had a false floor and a team of four spent the majority of the day bailing the water out in bucket loads.
The move could go ahead, but we needed a canoe and several dinghies rather than trolleys to move the stuff.
The move went well, somehow we ended up with less rack mounts than we started with and some of the cables mysteriously disappeared, but on the whole the day went smoothly: Until it came to switching everything on. We had a couple of hours panic when nothing would show up on our network, and one of the servers decided it was too much effort to switch on at all.
Several cable alterations later, and the domains start to appear. Yes, even the best of us make mistakes and plug cables in backwards. On the grand scale of this endeavour we were lucky that these were all the problems we had. The lazy server finally switched on the next day after a little coaxing and several threats of violence with a large hammer.
Six Weeks Later:
Its 8:30 on a Monday morning and we already have 67 email alerts from the various probes informing us that the temperature in the new server room was rising rapidly. When it finally hit 30 degrees I made my way down to the new sauna before the internet went down, and ebay became inaccessible.
The heat hit me in the face as I entered, to find a workman sitting in front of a patch cabinet in a t-shirt and high-viz, texting on his mobile phone. Between clicks and wiping the sweat from his brow, he denied all knowledge of the heat wave. Even when I discovered that the aircon had been switched off at the wall.
I reported the incident to the site supervisor, only to get a phone call an hour later with the following excuse:
"Hello. After careful analysis and investigation, we believe that someone had been in there and lent their ladders up against the control panel, accidently switching off the air conditioning."
"OK mate" I thought. You keep telling yourself that. As far as we knew only one person had been in there that morning and there were no ladders. We believe that the workman had switched the aircon off because he was getting cold from where he was texting his friends. Sod our servers; it was his comfort that was important.
They put a barrier over the control panel so that is doesn't happen again.
Three Weeks before the Move:
As is tradition with these types of building projects, the place isn't finished and the handover date is delayed by a couple of weeks. Ironically it's been changed to the day we are supposed to begin moving in to our new offices.
Furthermore, there is still no furniture in our new office. No one knows when or if we are getting any and no one seems at all concerned that it is possible, on moving day, that our new desks will be the cold, vinyl floor.
Moving in to a new building is supposed to be exciting, is this the sign of things to come..?
The Week of the Big Move:
Pretty much what we expected, by Tuesday evening the office and department are in complete chaos. There are boxes, crates and half-filled trolleys everywhere, and still no sign of the desks. We have already packed and stacked our own crates in the new office, in the blind hope that they will miraculously appear out of thin air sometime the near future. We ask when they are arriving and strangely the only answer we can get is "Yes."
Mid-Wednesday morning in true IT style myself and a colleague lug three old desks over to the new office. It was meant to be two, but the first didn't fit. Not wanting to carry it back, we placed it in another office and claimed 'it looked good there.' The other two fitted perfectly.
Afternoon comes and there is action at last, the desks have arrived and hurry down to place our claim.
By mid-morning the next day we were moved in, setup and updating our Facebook status' as if nothing had happened. When the man in charge of the new build came round to check our office out, he commented that there was "only supposed to be three desks." Well, there were, plus the two extra we added.
Now all that remains is to get the rest of the employees moved in.
Oh Joy!
No comments:
Post a Comment