Firefighter    ■    Power Dispatcher    ■    Husband    ■    Daddy    ■    Grandpa    ■    Crazy Man

Wednesday, December 14, 2011

Call Grumpy

I'm the only seriously active firefighter in my part of the district, and living so close to Station 51 means when either rig rolls out of there, my voice is pretty much always the one on the other end of the radio.

When you've served this long in one place, the people in your neighborhood get to know you either by direct contact or word of mouth for both good and bad moves. Still, it surprises me sometimes how it comes about.

My father-in-law works in fleet management for a large transportation outfit a few hours from here, and was hundreds of miles from home at a business function, chatting casually with the regional safety manager for his company. Somehow they got to talking about home, and the safety guy mentioned that he lived in the same county that we do.

"Oh really," my father-in-law said, "my daughter lives around there, on Lower Valley Road."

"You don't say," said the safety guy, "I live right about there, too."

"Yeah, her husband is a firefighter there and usually runs out of the station up the road from you."

"Oh, you mean Grumpy and Mrs. Grumpy? You're telling me he's your son-in-law??"

The way it was told to me, the conversation went on with the safety guy saying nice things about us and how he and his neighbors appreciate both that I run so many calls for the neighborhood and that Mrs. Grumpy helps that effort by being awesomely supportive.

I've actually never heard of the guy. Never responded to his house. Point taken, though, that your reputation precedes you one way or another. It was a really nice thing to hear about.

========

So last week I took in a middle of the night call for a medical emergency. The address was familiar, as the guy has had a few medical issues in the past, but I hadn't been there since 2009. I walked in and said "Hey Barry, it's Grumpy from the fire department, been a while since I've seen you, what's going on this morning?"

I heard the wife's voice in the dawning of comprehension as she said "You're Grumpy??"

Seems Barry was suffering stroke symptoms, and while he seemed to be bouncing back from what I assume ended up being a TIA, he was at that moment still struggling to speak clearly. As I put him on a high flow NRB and collected vitals while awaiting Medic 97, his wife explained that she woke up when he was acting strangely. The only thing she was able to make out before deciding to call 911 was "Call Grumpy." She had no recollection of my name, and was thoroughly perplexed. Now it made sense.

When Barry was fearfully struggling against the fog of a stroke, of all the things he might have wanted to say, it was my name that he punched out in his plea for help.

I am seriously and substantially honored and humbled at how that played out. I'm just a small cog in the big machine, but point taken that when the chips are down and the people call us, our arrival leaves a powerful impression for good or bad.

Reputations and impressions last a long time. Carefully ensure that you are investing in making the right ones.

Barry's home again and made a full recovery according to what I heard. I'll have to go by and see him this week. I am, after all, the neighborhood beat fireman, and should know my people.


Tuesday, December 6, 2011

Charlie Foxtrot


Engine 54, Engine 21, Engine 51, Medic 97, traffic accident.....

This is in the middle of 54's area. 21 will come from the north and I will come from the south in 51, to meet at the scene if 54 still needs us.

The next update has a chilling effect.

Rollover and over an embankment, six patients, some ejected. However, dispatch does not actually know where the scene is. The call came through Hazzard County 911, and Mayberry FD is already looking for it on their side of the line along with the Mayberry ambulance service.

The place they dispatched us to is associated with info that Hazzard County gave our dispatchers regarding a cell phone ping. But then our dispatchers mention a road name that was in the conversation that is nowhere remotely near where we've been sent. In addition, the caller is apparently one of the patients, and is describing terrain in no-man's land way down south that fits with the alternate possible location.

54, 21 and the medic will continue to the original scene to check, but the new spot is now behind me and closer to Mayberry. I divert to the new location, which will make me drive back right past Station 51, and flop from our TAC channel over to the Hazzard County channel to see what they know. Not much, except that the Hazzard County ambulance is already on scene asking for assistance yet no one seems to quite know where they are or how to interpret their directions. Units are scrambling to communicate on our TAC channel, Hazzard County's dispatch channel, and a shared inter-op channel that we should all be using but not everyone has access to. This in addition to the fact that we're on the fringe of radio coverage for any of these channels.

54 and 21 arrive, UTL. 21 returns, and 54 starts my way with Medic 97 right behind them.

I arrive at the end of the world, where roads stop having reliable names. A couple of Mayberry units are there. The Mayberry Chief hustles up to me to let me know a second Hazzard County ambulance just went by them a few minutes before, but they don't know where it went.

Seriously? You didn't think of following the medic if it seemed to know where it was going? Oh well, this is their call, we are the mutual aid, so I tell the MFD Chief that we have another engine on the way and ask how we can help.

He gives me a blank stare for a half second, and then as if he didn't hear me, he asks me where the call is at and what we want them to do.


We have thus descended into Charlie Foxtrot status.

We thought it was their call, and they apparently thought it was our call. No one has Command, no one knows where the ambulances are, and everyone is scattered on three radio channels.

Cue the Southwest Airlines catchphrase: Wanna get away?

We are - in our present literal location - technically in our district by about 15 feet, though the roads into no-man's land head mostly into Mayberry territory. This could be anyone's call. But OK, I can take it.

Now with a reliable starting place to work from, I managed to get enough information from the ambulance to guess where they are, and with that, we're off into darkness.

After about five miles of that we come over a rise and find an ambulance coming toward us. Great, are we ALL lost?

No, as it turns out, this medic has two patients aboard. The other Hazzard medic has one patient aboard and is coming up behind them. There were never more than three patients, they are all loaded, and there is nothing left to do. They thank us for showing up, and off they go to the hospital.

You could just sense the deflation of everyone's adrenaline when I got on the radio and told everyone to just go home.

So, what happened?

True to past performance, massive communication breakdowns between 911 agencies. Hazzard County 911 gave our dispatchers an address near a cell phone tower in our area, and the other road names out of our area carried no meaning and were ignored. We got sent to the tower. The tower.

Too many radio channels, with too many people talking on them. The Command vacuum empowered everyone to chatter until confusion reigned.

The first unit on scene failed to initiate Command. The first officer on scene also failed, as did the first Chief. It should never have fallen to me as first-arriving M/A representative to recognize that and try to fill it.

We're not perfect, I make plenty of mistakes, but this Charlie Foxtrot was worse than usual, and oh-so-preventable with basic communication skills.

It could have been worse, I guess. Argh.


Wednesday, November 30, 2011

Tutorial 10: System Protection, Part III


OK, we're back for another rare installment of the tutorials that make up "Grid 101: Introduction to How The Power Grid Works", or "What Is This Confounded Sorcery?"

You don't have to re-read all the tutorials for this to make sense, but I would suggest you at least go over Tutorial 9: System Protection, Part II.

First is the detection relays. I explained that the relays watch for faults and decide what to do about them, but did not expand on how this magic works.

Remember that electricity flows can be compared to water flows that you fireman-types can relate to. Voltage is equal to water pressure, Amperage is equal to flow rate, and Resistance (Ohms) is equal to friction loss.

Resistance is hard-calculated much like water friction loss. We know the more-or-less fixed value of water psi loss for a given section of 1.75" hose and associated appliances. Similarly, power system engineers can calculate the resistance that a given power line of x distance and y conductor size will create. So file that away for a moment.

Voltage (pressure) is measured by potential transformers (PT). These are everywhere, nearly always attached to any buses as well as on the line-side of any breakers at the station. They serve to detect the presence of voltage as well as its strength. Their output is fed to the relays.

Amperage (flow) is measured by current transformers (CT). These are located anywhere an overload flow condition could arise that would need to be cleared, such as above and below every substation transformer and also on the line-side of any breakers at the station.

The differential protection mentioned in Tutorial 9 is made possible by the CTs on both (or all) sides of a given section, providing the measurements that should sum to zero indicating that everything going in is also going out, which is a good thing.

For each line out of the station, the values from the PT and CT are fed to the relays. For decades, relays were electromechanical solid-state devices, and specific electrical signals from the CT/PT would cause one or more contacts in the relay to close, enabling a trip signal to be sent to the necessary circuit breaker(s). In the past fifteen years or so, digital relays have arrived, and they have uber-fast computers that analyze the CT/PT inputs and decide what to do about it.

In all cases, electromechanical and digital relays have to be programmed or tuned by relay technician engineers who input tolerable levels of flow and voltage based on engineering system studies. Being a relay tech is fascinating work. Not only does each sold state relay need to be set up to do its thing, it also needs to be able to detect that its requested action happened or not and be able to send a trip signal to the next thing upstream if necessary. Digital relays too, but it is much easier to program that aspect than fine-tune the mechanicals.

The reclosing ability of lines and feeders relies on PT inputs to help the relay decide whether it should try or not. If a line trips and a reclose is permitted, the relay will first check the upstream PT for the presence of voltage. If no voltage upstream (bus is dead), there is no benefit to reclosing, and it won't happen.

Often, transmission line reclosing is staged so that the test is performed from the strongest side, so that a repeated fault doesn't disrupt a larger area or a generation plant. For an "auto-first" recloser, the automatic recloser will reclose into a dead line if the bus behind it is hot (PT senses voltage). An "auto-second" recloser waits for both the line and bus to be hot, meaning that the line was successfully tested OK by the other side. Auto-second is found at all power plants, and at weaker substations. A third variety, less common, is where a recloser sees a hot line and a dead bus. If the bus was merely de-energized by a large scale outage and never suffered a clearing fault, the breaker will close to pick up the bus, at which point any auto-first reclosers will see the bus pick up and try their lines. Of course, reclosers can be set up to operate at the direction of the relays in multiple or all of these situations.


Random amusing unrelated picture, to interrupt the long post.

Seriously, it's OK if you want to go refill your coffee at this point.


OK, all rested up? Onward.

So, we know that relays process input from PTs and CTs to decide if there is a fault, and how bad it is. Take it back to fire flows and it makes sense. If you have high water flow but water pressure is good, that is probably OK. Likewise, if water pressure is a little low and water flows are also low, it isn't ideal but doesn't indicate a major problem either. Conversely, if your water flow suddenly surges while water pressure plummets, you have a pretty clear indication of a burst hose. Translated to the PT/CT inputs, high amps and volts are OK, as are low amps and volts, relatively speaking, but sudden high flows with plummeting voltage is a clear indication of a line fault.

Within the transmission protection zones, relays watch the line from both ends, and can use the PT/CT inputs to not only detect a fault but also figure out with fairly good accuracy how far out the fault is from the station. The location of the fault is important, and influences the behavior of the relay.

A relay looking out into a line for a fault typically has three zone of protection in regard to the line. Zone 1 watches for faults from the breaker on out to cover 80-90% of the line. Zone 2 over-reaches Zone 1 by 10-25% and can see faults in or near the remote bus of the next station past the far breaker of the monitored line. Zone 3 looks backwards from the breaker into its own local buswork. See the illustration below for the protection zones watched by the relays assigned to circuit breaker "A" at West Substation (in blue), and the zones for circuit breaker "B" at East Substation (in green).

A fault in Zone 1 is indisputably on the protected line and will be cleared without delay by the breaker that sees it, usually within about 3 cycles (0.05 seconds). Most of the line is covered by both Zone 1's, so generally both breakers will independently see it and trip. Both Zone 2's also saw it in their area, but have a time delay to prevent false tripping and let the Zone 1's handle it.

If a fault occurs right outside the station, let's say right outside the fence from East Substation, things are different. "B" Zone 1 picks it up and trips immediately, but "A" Zone 1 doesn't see it. "A" Zone 2 knows about it, though and trips "A" after perhaps 6 or 10 cycles (up to 0.15 seconds). The delay is important, because Zone 2 can see into and probably slightly beyond the next station. In this case, if a fault occurred just outside of East Substation on some other line, "A" Zone 2 probably sees it, but it would not be appropriate to knock off this line when the fault is somewhere else. The time delay allows the local Zone 1's for that other line to take care of business. If they don't, "A" Zone 2 trips after its delay.

From here, things get more complicated. For advanced protection arrangements, there is equipment to block or force tripping independently of what a relay is seeing for itself. Relays at different substations can communicate with each other in a variety of ways, such as fiber optic networking, microwave radio, or a carrier tone through the actual power line itself.

If false tripping is a problem or seriously needs to be avoided, is where Zone 3 comes into play with blocking schemes. Zone 3 generally looks backwards as far as the remote Zone 2 looks through it. In the previous case of an unrelated fault elsewhere right outside of East Substation, West "A" Zone 2 sees it, but so does East "B" Zone 3, and it knows that the fault is not on this line. It will communicate to West "A"s relay scheme that a trip would be bad, telling that relay to ignore any Zone 2 trip it might want to try (at least for a bit, it will eventually allow it if the fault persists). Thus, no false tripping and better stability of the grid.

Cool, huh? It gets better.

For lines where relay techs have determined that a potential Zone 2 clearing duration of up to 0.15 seconds is too long and the disturbance too great, the permissive transfer-trip scheme is used. Sensitive transmission lines and very high voltage lines at or above 230kV is where you'll see these. In these cases if either Zone 1 picks up, it automatically tells the other side to trip without waiting for the remote side to go through its Zone 2 time delay. This ensures that a fault on either end that falls outside of the dual Zone 1 coverage still gets cleared as if both relays saw it as Zone 1.

Think about that. A fault occurs. The PT/CT inputs are picked up by the relay. The relay concludes it is a Zone 1 fault and must trip. It sends a signal to its local circuit breaker to trip, and also sends a signal to the relay at the other end of the line. The local circuit breaker trips. The remote relay gets the "trip now!" message from the local relay that detected the fault, and tells its own circuit breaker to trip, and it does. From fault to cleared, with all that communication and mechanical response from the circuit breakers, probably no more than 4 cycles - 0.066 seconds - elapsed.

Once you have these fundamental ideas about zones of protection and calculating fault locations and intensities, you can imagine that some relay techs can dream up some amazingly elaborate protection schemes. When they work, it is pure beauty. Even when the special schemes fail for some reason, the fallback default Zone 1-2-3 systems still work.

What we learned: (1) Power lines act a lot like fire hoses in some ways, but we already sort of knew that. (2) Pressure and flow rates (volts and amps) measured at the right places can provide a lot of information about what's going on if analyzed the right way. (3) Protection zones back each other up, which we also sort of knew. (4) Special schemes that cost a lot of money to engineer and build can shave fractions of a second off a trip time, and that fast action can be very important to save the grid.

Whew. See what happens after such a long hiatus of tutorials? Blah blah blah blah... I talk too much. I see your coffee cup is empty again. Go fill it up and hide in a quiet place to absorb what you've learned today. Stay safe out there.

(Click this link to see all posts tagged "tutorial")



Monday, November 21, 2011

Checking in

This new boss gig is taking a lot of my time! But it is a good gig. This is evidenced by the dropoff in posts, as this blog was started for the same reason many are: to rant. I'm just not pressed to rant since starting the new job, because it is really agreeing with me and I am surrounded by good people and an intelligent leadership structure that more or less has a plan. It is AMAZING.

I am very lucky.

Also, not too many exciting fire or power grid stories to tell, but the rough season is approaching. Of course, there was the guy who complained when we fixed a streetlight in his area. He told the customer service rep that the light was damaging his DNA. We offered to shield his house from the light, and he insisted that he provide the shielding. I expected something wrapped in foil, but it turned out to be a flap of rubber from an inner-tube when they brought it to Dispatch so we could see it. OK, whatever floats your boat. Timo went out to install it, and we had a deputy along by request, just in case.

The blog got some pings from a discussion about wind power, leading me to realize that I have really dropped the ball on talking about future power grid trends as well as the tutorial series. Some new stuff is up with wind which I actually find encouraging. I still need to finish talking about circuit protection schemes and the relays that make them go. And the so-called "smart grid" you hear about from time to time? I'll fill you in on what it is supposed to look like, since no one in the media seems to have a clue except to use buzzwords like "demand management" and "smart supply", or some other useless catchphrase of the week.

And as my parting shot, check this out. While on the road this summer, putting a few thousand miles on the family transport to see the in-laws, I ran across this very creative way to recycle an old power pole. The Grumpy Dispatcher approves.

Thursday, November 3, 2011

The New Guy

As I arrived in Tanker 54 as the second piece on scene, the smoke was rolling pretty good. The fire had self-vented in back and was starting to really roll. Engine 54 was just getting set at the front door, their officer awaiting my passenger to pack up so they could have their two outside and go in.

Engine 54's engineer, Trev, was one of our newer part-time guys, but he had advanced well enough through the academy sessions and had passed his engineer stuff. We're not a huge department, so it is possible to get up to engineer within a year or two if you apply yourself, and he was one of these guys. Still, Trev didn't always come off as especially confident.

I set my pump into gear, and pulled the short section of 3" out of the far side pump panel's sideboard tray, laying it down at E54's panel. By the time I was ready to give Trev water and went back to verify he was ready, he had hooked up my supply line to one of his discharge ports. Nothing to make a scene over, I just told him where to move it. It was not the time for a learning discussion. Trev swore at himself and moved it. Guess we might need some more training for familiarity with the equipment, right?

-------------

Trev was at the live fire and pump ops drill, with a handful of new guys and recent volunteer recruits also in attendance. For several of them, this would be their first live-fire exercise.

I was standing next to the instructor as he barked at the fresh faces. He wasn't being mean, he was just being true. I was sure my BS meter was broken, because when I turned it on, the needle didn't budge. Is this thing on? That's what kind of instructor was in charge that day. He told them that this was practice time, and not learning time. If you think you know what you're doing, do it and be evaluated. If not, stand back... watch... learn, and we'll get to you later. All business.

You see, Trev is in his fifth year with us now, and was hired full-time a year ago. He has turned out to be an excellent beginner instructor, and a fairly proficient engineer. I love watching him with the recruits because he takes ownership of them, and I like seeing how he has changed. He barks, but with passion and love. I wanted to give him a big hug and smooch because he turned out so well, but it would have ruined the mood.