Transformers are very expensive, and usually impossible to repair, but this one has not given up its exact problem, and after a good once-over, flush and inspection, it was test-energized successfully, although it made some funny sounds that concerned a substation operator enough that he made a call into the control center about.
But today is apparently the day that the bosses want it switched the rest of the way in, and although you're knee deep in other jobs, the switching orders and procedure to put it all the way back in service have just landed on your desk, and crews are already en route to switch it in within an hour or so.
I wasn't the guy on the desk. The dispatcher who had this land in front of him took the position of 'no way in hell am I going to approve this on my own'. He notified several other area supervisor-boss types with direct interest in the situation (and the power to call it off) of the decision to put it back, shared the particulars of his concern, and made sure that they were on record either by email or on the recorded phone as either approving the procedure or at least not calling a stop to it.
Transformers, you see, don't have moving parts with which to generate noises (except phase shifters, but that's another story and those moving parts would never remotely do anything like this one is doing... and oh yeah, this one is not a phase shifter). When energized at high voltage, like most anything else exposed to 60Hz AC power, it should hum, but that is all. Anything else is, well, Bad. However, the district manager who decided that it needed to go back disagreed with that assessment. His exact words in email: "Noise coming from an unloaded transformer alone isn't a worry without more data."
So, onward.
The dispatcher sufficiently covered his arse with phone and email records, and proceeded with the job, communicating his concerns with field crews, who were on the same page. When it was time to energize the transformer, the dispatcher had all crews leave the substation and get out of the immediate area before closing it in by remote control.
Select: Click. Close: Click. Confirm: Click. Three Mouse Clicks.
All the dispatcher got to see and hear of this event was from the SCADA computer. Beeeeeeeeeeep. Transformer lockout alarm, open circuit breakers. Well, that didn't work.
Then the phone rang. The field crew, a couple hundred yards away, reported a large explosion and lots of smoke. After being reassured that equipment was de-energized, they returned to the substation yard to find the side violently blown out of the now-smoking transformer and BIG environmental mess to clean up. A transformer under load is a very different beast than one just heated up from one side.What should have been an immediately-scheduled replacement job for $2 million that would have been done months ago, was instead delayed months for all the testing. Obviously, it will have to proceed now anyway, but it will have to wait until the cleanup is complete to EPA requirements. The attempt at $aving$ has resulted in way larger lo$$es.
Oops. Should've listened to the dispatchers and field crews, boss.
If you said make your feelings known, follow orders if forced, yet do everything necessary to document who owns responsibility and make absolutely certain your field crews are out of harms way, you made the same call that the on-duty dispatcher did that day. This was not a bad call.
If you canceled the job and told the bosses to pound sand, as I like to say I would have done with the luxury of hindsight, you would have caught some serious crap about it only as long as it took them to find someone else to take over for you and blow it up, and you would have made the right call.
No doubt there are some interesting memos going around, that we'll never get to see. Oh well.
(Disclaimer: That picture above is just for fun, not a picture of the described event.)
