Monday, June 06, 2005

Perpetual Motion

Our Leeds Office is having a pretty rotten time of it at the moment. You may recall that last week they spent a day and a half without email access – which is never something that leaves users happy. On Wednesday, around about the time that the email problem was solved, their file server needed a couple of reboots to try and get a new backup tape library to work properly. (My experience of Sony Tape Libraries is not good. We are now on our fourth in seven months!)

After all that, to properly round-off the week, on Thursday the office started suffering intermittent interruptions to their connection to the Internet – which also carries their traffic to all of our central systems. The outages continued into the weekend, despite them being serious enough for the ISP to get an engineer on site on Friday. Since nothing happened over the weekend, the problems were still there this morning when I arrived for work and thus became my problems, as the most experienced network engineer present.

In the end, it boiled down to some simple synchronisation problems between our firewalls in London and Leeds – although that can’t have been the initial problem, or it would have been solved on Thursday! Anyway I arranged a quick reboot of both boxes and hey presto! everything was running smoothly again…

… For about an hour. Then it was Edinburgh’s turn.

“We’ve lost email and Internet access,” came the call and my heart sank as I recalled the pointless forty-eight hours from a week or two back. While it was at first quite confusing, this time it didn’t take too long to narrow down the problem to a routing issue on BT’s part. (An analogy would be: some of the ‘road signs’ on BT’s network had been changed and were sending traffic in the wrong direction. As a result, data was getting lost down dead-ends.) There was nothing I could do about it but wait and, sure enough, after twenty minutes or so the problem fixed itself.

By mid-afternoon I was ready to start doing the work I had actually planned to do today and began looking at the trial of a new login script I had written. The script, which runs every time a user logs on to the network, replaces about a dozen different scripts that are currently in place, with a single one that is a bit more intelligent and modular and (most importantly!) writes a log of what it is and isn’t doing for the user.

I found no solace here either. One of my test users wasn’t recognised as being based in their correct office. I eliminated a few of the programming ‘likely suspects’ before having to conclude that my coding was in fact fine (but of course!) and that it was the programming language itself that had the problem. (Actually to call it a programming language makes it sound way too grand. It is actually a scripting language, which is a much simpler kind of programming language, but one which lets you access information about your computer and user more easily.)

In the end I got so frustrated with the whole thing I gave up on it and spent my last hour catching up on Dilbert and the blogs – oh, and briefing my boss ready for the lynching he is likely to receive when he visits the Leeds office tomorrow…

Tonight we had a Chorus rehearsal, which was no real respite: We seemed to be concentrating on new pieces with tricky rhythms and then, in the break, Marc D gave Brett and I lots of over-the-top drama about not being invited to Brett’s birthday dinner which was amusing for a short spell, but rapidly got old.

I was glad when it was all over and we could come home and go to bed.

No comments: