Firefighter    ■    Power Dispatcher    ■    Husband    ■    Daddy    ■    Grandpa    ■    Crazy Man

Tuesday, February 23, 2010

Who's got the power?

Part of the reason so many power dispatchers have such self-righteous bad attitudes is the rule that grants such strong power to the on duty dispatcher. The guy on the desk has not just the authority, but the responsibility, to unilaterally take whatever actions are necessary to save his portion of the grid and/or prevent damage to his portion of the grid. Failure to take these prescribed actions within the required timeline (generally within 15 minutes but the sooner the better), when warranted, can result in multi-million penalties to the company that failed to perform.

The directives are like standing EMS orders for paramedics. When time is absolutely of the essence, critical minutes are shaved off by leaving predetermined directions so no time is wasted getting approvals.

The precedent type of incident for requiring this rule in the power industry has been repeated over and over again, when preventable blackouts and/or major component damage have occurred because a weak-willed dispatcher felt that a supervisor or manager (invariably one who is not on duty and doesn't answer their cell phone) cannot be contacted to hand down a blessing. So the dispatcher leaves a voice mail. The supervisor eventually calls back..... to ask why all the lights are out. What, you left a message?

Every system control center with NERC-certified dispatchers on duty is now required to have a notice posted in a conspicuous location in the control room that states, in essence, that the dispatcher has clear authority to do basically whatever they determine is necessary to save the free world without requiring any kind of approval from above. The notice must be signed by the highest-ranking official of the company. Even failure to have this notice posted (whether company policy is in line as intended or not), can result in a fine.

They take this stuff pretty seriously, I guess. Incidentally, this rule is the same one that allows me to tell the same CEO trying to micromanage me from behind my chair to either get lost or kindly assume full responsibility for my desk while I go out to Delmonico's for a bite.

So, with that background out of the way, the story. You knew a story was coming.

A notoriously under-supported section of our system, a long, 46kV subtransmission loop, has long suffered from low voltage. In fire terms, think of constantly trying to provide adequate pressure and GPMs to 1,000' of 2.5" line. It doesn't matter how much pump pressure you've got at the source, up to the point that you burst the hose line at the pump panel, you just can't get more than a piddle out of the tip.

Electric transmission suffers the same thing if there are not adequate shunt capacitor banks (in-line relaying pumps, if you will) along the way. Adding cap banks is just a cheap workaround, anyway, bound to get you in trouble in the long run as far as system planning goes. The intelligent thing to do is shorten the transmission distances by plugging in a new substation somewhere in the loop, fed off of a good 115-230kV high voltage source, to inject voltage (pressure) right where it is needed.

Generally, many subtransmission and virtually all distribution loops are operated with a "normal open" switch somewhere in the middle. This is done so that a fault in the loop only dumps half of the loop instead of the whole shebang. Then, when you find the problem, you can isolate it by opening nearby switches on either side, and then close the "normal open" to feed the back portion of the bad side off of the other half of the loop that is still hot. Fix stuff up, then switch it all back to normal.

Well, this bad section of ours is in such bad shape from load growth (new construction, new customers, more demand, etc.) that the "normal open" switch lost its status years ago, and the loop is kept closed all the time. Before this was done, the far remote ends of each side of the open loop had horrible voltage problems. Closing in the loop helped only slightly. The Engineering conclusion was, if Planning won't upgrade the loop, better to close it in and risk the whole loop to occasional trips than to damage customer equipment at the remote ends with chronic ongoing low voltage conditions.

Still with me? The story is still coming.

So, this particular 46kV loop has a small power plant on it. A weak, persnickety, temperamental power plant. It likes chocolate and to be told it is pretty before it will come online, but you can improve your chances with a nice card in an embossed envelope.

Well, once in a while, as circumstances dictate, this loop has something break on it and has to go out for repair. This is hardly surprising since, as you recall, it is already old and decrepit. This open configuration resulted in the Persnickety Steam Plant hanging on one end of the loop, charged with the unhappy task of propping up voltage at its end of the line.

Murphy stepped in, and Persnickety tripped offline.

Voltage went to crap. But not quite to crap that I felt compelled to shed load (black some people out) to prop voltage up. Back to fire terms.... my interior attack guys only had about 60psi at the tip. They aren't happy, but they can sort of work with it for a while as long as things don't get worse. If it goes downhill, we'll pull them out, but for now our fingers are crossed.

Still, I really need Persnickety back.

So, after a couple of hours, the Persnickety Control Room Operator called and said they were ready to come back online. One problem though.... voltage is too low on their station service (external power they're getting off the weak end of the loop to start the unit), and the source is too weak to roll the unit online.

Persnickety is not coming back until I raise voltage. I have no cap banks to boost voltage with. There is only one way to raise voltage.

Drop load.

Lights out for a few feeders, several hundred customers, at least until Persnickety rolls up. That might take an hour or so.

So be it. I gave the orders to the distribution dispatcher for Small Municipal Utility with the feeders that I wanted to whack.

He balked. Had to check with his boss, he said.

Uh, no you don't. I am giving you a Directive (very important perk-your-ears-up word in this business), to trip feeders blank, blank and blank until further notice.

SMU dispatcher was not happy, but knows we are both on recorded lines and follows the order. He knows that the issue is in my court and that I am taking responsibility for the outage orders.

Hint: That's what that notice on the wall is all about, baby.


Ninety minutes later, Persnickety Steam Plant got a good shoulder massage and box of bon bons, and decided to go ahead and come online. After another half hour or so when it was clear that Persnickety was going to stay with us, I gave permission to the SMU dispatcher to restore his feeders.

They'll send us a bill for the troubles it caused, forward whiny customer lawsuits, whatever. That's fine, that's exactly how the game is played.

However, the next day I got an email directly from the SMU supervisor/manager, blasting me for having the gall to issue those orders, and providing me with a list of phone numbers that must be called to secure permission should I (or anyone here, for that matter) ever try that stunt again. He was on a pretty high horse, atop a gilded pedestal, in his ivory tower, and was speaking down to lowly me from on high accordingly.

Yeah buddy. Fail. I forwarded that note to my boss.

Mr. SMU boss got some joint correspondence from our legal department and our upper management detailing the FERC/NERC rules and our obligations to them, that SMU is in our footprint and subject to our authority accordingly, and that if he wanted to stand his ground, we would be happy to forward the big multi-million dollar fines to him.

Oops.

The written apology he sent us back was rather polite, actually.

Remember, one person's ivory tower is someone else's remote guard shack with a space heater and AM radio on the shelf.

And that long rambly story, is to explain why we have a bad self-righteous attitude. Because as a rule, our ivory tower really is at the top, and we're used to getting our way.


No comments:

Post a Comment