We were bouncing reminisces of epic power company failures around recently, and keeping with the Happy Medic's Rule of Threes, I will share with you the three that stuck out as worth passing along. Unlike HM's threes, though, my stories often go too long, so you only get one tonight.
Now, it should be recognized that lots of stuff goes wrong all the time, of course, but we usually can play by the seat of our pants and get out of it without many people knowing. That's the whole point of why we're here and what we get paid for doing. But heaven help the dispatcher on the desk when the stars align, whether its his fault or not, because big fun ensues.
Episode 1: When "Preferred" is not Preferable
A 69kV subtransmission loop had a fairly good-sized community fed off of the end of one side next to the normal open switch that kept the loop from being tied through. If you've been reading here for a while, you remember that normal-open loops are good, because a fault anywhere in the loop only takes out half of it instead of the whole thing, and once you find and isolate the problem, you can serve the customers on the remote end of the bad section from the other side until affairs are settled.
Problem was, in this case, the entire loop was fed through some rough terrain and there were frequent outages, relatively speaking. Each time the thing tripped, it would take a few hours to find the problem and then switch in the city from the other side. The city was a pretty squeaky wheel, and eventually pushed us to install an automatic throwover switch.
An auto throwover switch is actually two switches, mounted up on one or two power poles, and equipped with PTs (potential transformers, that detect voltage) on either side of the customer's tap. One switch is open, the other is closed on the preferred source side. If the preferred source goes dead, the PT on that side calls to the other one and says "Yo, I'm dead over here, how about you?" The other PT says "I'm fine over here, my Xbox is still on!", and the throwover switch scheme, overhearing the conversation, opens up on the dead side and then closes in on the live one, picking up the customers after just a short delay. Instead of being out hours, now the occasional interruption lasts about five seconds.
Important: a throwover switch is motor-operated, and motors require.... electricity.
The best implementation is to mount a poletop transformer right there to serve a charger for the switch's 12VDC battery.. Except in this case, someone went cheap and decided to just go with a transformer straight to an inverter to serve the 12VDC switch motors, supplying the throwover scheme right off the 69kV and saving the cost of batteries.
They put it all together, tested it, works great. Now we just have to wait for the next outage and show those guys how cool this is!
Within a few weeks, a storm blew through, and the preferred side faulted as reliably as ever. The guy on duty pulled up the customer's substation display on screen to observe when the feeders picked up again upon the throwover executing its beauty. Nothing. Waiting.... tick tick tick, still nothing. Of course, you saw this coming, right? The engineer designed the throwover to be fed off the preferred side (after all, it was preferred). When the preferred side went dead, the PTs would have liked to have had their discussion abotu who still had a working Xbox, but the switch motors had no power and were helpless to throw.
Oops. Two hours out. Again.
The next day the throwover source was moved to the alternate side, and a few months later, it also got batteries like it should have had in the first place.
Watch for Episode 2: The Important Sign in a week or so. Episode 3 involves a backup generator at a power plant, but I haven't thought up even a bad name for it yet.