I’m currently on my way home from work and it’s already past midnight. Things have been going relatively well recently, with our new system in place and the pressures of a major deliverable disappearing into the past. Today, however, it all went a bit wrong.
We were in the middle of a bug fix release that we’ve been working on for the past month or so when one of thr developers noticed some screwy data. Not major screwy, but enough for us to stop what we were doing and work out what was wrong. It seems that an old version of a piece of code had crept into both our production and our UAT systems, meaning that it has to be backed-out and that our testing was invalid. Not good! We’ve now got to work out how it happened, why we didn’t see any problems in testing and how to persuade our users to do more testing. It’s going to be a long couple of weeks!
Still, it could be worse and we must keep a sense of perspective. Everyone makes mistakes – that’s why they put rubbers on the end of pencils. The important thing is to learn from them and work out what we can do to stop it from happening again.
Update: due to a problem in trying to get my iPaq to talk to my new W800i phone I didn’t get the chance to post this until now. I’ve had four and a half hours of sleep and I’m on my way in again!