Firefighter    ■    Power Dispatcher    ■    Husband    ■    Daddy    ■    Grandpa    ■    Crazy Man

Tuesday, December 29, 2009

Monday, December 28, 2009

How to Capture and Visualize 60Hz

Boring electrical geek ramble here.

100 years ago, many AC systems were operated at 25Hz (25 cycles per second), and pretty much anyone could detect it when looking at a light bulb. You could see it pulsing really fast. At today's 60Hz, it is pretty much impossible to detect that pulse in an incandescent light bulb, because the filament does not have enough time to noticeably dim between the 60-times-per-second pulses.

Unlike incandescents, fluorescent lights visibly pulse to some people whose eyes are sensitive enough. Like mine. I can see the flicker of computer monitors set to 60Hz, too. Both give me headaches. Many monitors have configurable frequency settings, so I change them to as high a setting as possible. If I can't, the monitor is disposed of.... somehow I usually find a way. And of course I avoid using fluorescent lights as much as possible.

But there's a relatively new flicker-detectable 60Hz light source out there, and I probably can't run away from it:  Cheap LED lights.

LEDs are either ON or OFF. There is no 'warmup' or 'cooldown' interval.

Have you ever seen one of those spinning light gadgets sold at gift stores, with the LEDs on it? As they spin, the lights make various multi-colored solid lines or dashes or dots, rotating back or forward, and they are fabulous for mesmerizing cranky 2-year-olds for a few minutes.

Next time you get your hands on one of those, grab the spinner so it can't turn, and then turn the thing on. These are battery-operated, so they are not strictly 60Hz, but run at variable frequencies controlled by the internal circuitry. You'll note that, when the spinner is not spinning, the LEDs are constantly changing in their intensity. Brighter, dimmer, to varying degrees. What you are seeing is the pulse rate affecting how much total light they give off. When the LEDs are stationary, your eyes detect the difference in total light delivered as brightness, but when spinning you can see the actual pulses as the dark gaps between the lines and dots in its rotation. Cool, huh?

So there's a lot of Christmas lights that are LEDs now, and I can tell them from the old ones instantly by their flicker. I was at a big Christmas light display with hundreds of thousands of these LED lights, and I about went nuts. Since these are AC, they always pulse at 60Hz. How to visualize that when they don't spin? I used my cell phone camera, and swung my arm quickly sideways while taking a picture of the lights. Can't rotate the lights, then rotate yourself. The result is below.

There's lots of reds in there, but only a few blues or greens. Count the number of blue dots captured from one of these lights, and you'll see that my cell phone camera captured six pulses. At 60 pulses per second, that means my camera caught 1/10th of the pulses delivered every second.... so now I know that my cell phone camera's exposure time is 1/10th of a second. Incidentally, there are a couple of white incandescents that got caught in there on the lower left, and you can see the intensity of one of them oscillating, though it never has enough time to go out between pulses, hence the pulses are invisible to the normal naked eye.

Cool huh?

Or boring.

I know it's late, but Merry Christmas!

Friday, December 25, 2009

The Pre-Plan

Showed up for work at the power company this fine Christmas morning.

Keeping the lights and the heat on for you while you do whatever you do this holiday. Don't mention it. I get paid extra to be here and I am happy to have a job.

Anyway, for some reason, the refrigerator was replaced during my days off. Looks like the person in charge ordered a really nice stainless steel side-by-side to replace the old one. What was wrong with the old white enamel one is not clear. Apparently the boss simply wanted us to have a new one. With stainless steel.

They didn't go cheap. Bought a side-by-side to replace the old over/under. After all, we're a multi-billion $$$ company, so we can afford it.

If you've never compared refrigerators... typical mid-line over/unders are about 33" wide, while side-by-sides are rarely (if ever) less than 38" wide.

Yeah, you can see where this is going.

So in the corner of the control center kitchen, where the old fridge used to be, the new one is in front of the old hole, turned 90 degrees and against the side wall. Now we have the loss of space in the kitchen, the open and inaccessible hole blocked by the sideways fridge in front of it which will no doubt gather accidental trash and crud, and you know..... the new one has less fridge space than the old one overall, in exchange for a bigger freezer that we pretty much never use.

Not to mention, the all-important pizza box doesn't fit in a side-by-side.

Nice upgrade.

Pre-plan FAIL.

I'll keep you posted on what comes next: (A) Swap for smaller fridge (with or without consulting the guys who actually use it), (B) Spend a bundle to remodel a kitchen that does not need a remodel so that the wrong fridge will now fit, or (C) just leave the goofy situation as is.

Saturday, December 19, 2009

Fuzzy Brain Radio Follies

Engine 51, Engine 57, Mayberry Engine 13, Medic 98.... traffic accident.... tac channel is TAC2.

Ugh. Engine 51 is first-due. I look at the clock. 02:24. Ugh. OK, here we go.

The fuzz is heavy. I pull out in trusty 51 and grab the mic to come online.

Engine 51 is responding.

10-4 Engine 51.

I then hear a couple of other sort-of-nearby units from our agency start our way, ready to turn around if not needed.

Then I look at the radio. Oops... I used the FireMed channel.... but they answered me anyway and just went with it, without a poke. Thanks, guys.

I pick up the mic again to advise dispatch that I am cutting over to Mayberry's response channel to touch base with Engine 13 when they come up. You see, they don't have the fancy-schmancy 800MHz system we have, so on M/A someone has to go meet them on their channel.

As I key the mic, I abruptly realize that I never changed channels in the first place. I am still on FireMed instead of TAC2. Since I came online first, as first-due I am the one responsible to initiate the correction and make sure everyone is moved over to TAC2.

I have two messages in my fuzzy head, and I have already keyed the mic. (#1) I am going over to Mayberry Channel 4 for a moment. (#2) Let's move all responding units to TAC2.

My fuzzy brain, under pressure (mic is keyed), smashes these two messages violently together.

Dispatch from Engine 51, let's move all responding units to Mayberry Channel 4.

Oh crap, did I really just say that on the air?

I really fubar'd this. I can visualize every guy on a responding unit, as well as anyone else listening, staring at their radios as if they just channeled radio traffic from the Burger King drive-thru. Say what? You want us to... did he just say that?? Not all units responding even have the capability to go to Mayberry's channel.

And I'll be danged if the dispatcher didn't just go with that, too, nary skipping a beat.

10-4 Engine 51. All units responding on the traffic accident, move to Mayberry Channel 4 per Engine 51.

I wish he had put the brakes on and called me on the goof... but off we go before I can pipe up again to fix it.

So... some guys just follow orders and go to Mayberry Channel 4 as directed, some stay on FireMed to see what my next move is because they realize I am fuzzed up and should come out of it soon, while still others go to TAC2 as we were supposed to have done all along. Once I let the livestock out of the pasture, I wasn't sure how to get them corralled again. From there on out, it was a great big communications Charlie Foxtrot.

Thankfully it was a one-car, non-injury wreck, not a lot of coordination involved in finding units, whichever channel they went to, and telling them to cancel.

Lesson to self (and I rant about this to the new guys constantly): Engage brain and think over your message before you key the mic.

The Captain sat me in the office for a few minutes the next day, just to make sure I still knew the protocol for M/A and TAC channels. He knows it was 0-dark-30, but he had to check. I don't blame him.


Tuesday, December 15, 2009

Commentary on Death by Ignorance, and System Protection

There's a little tie-in commentary I want to make in regard to the previous post and all of the system protection stuff talked about in the last two tutorials.

Everything in a substation is protected by zones, so why in the world would the carnage depicted in the photos on the last post be so severe? Why didn't something detect the fault and clear it before it got so bad?

Well, my friends, what probably happened in that case and does happen in many others like it, is that technically there was never a fault. There was never a hard connection to the fatal arc from the generator-to-component chain that is watched by fuses and relays.

Part of the transmission of electricity involves a powerful electromagnetic field which literally rotates around energized equipment and power lines, out to a distance of several yards, though the strength of this field weakens quickly as the distance from the energized equipment increases.

You hear about people complaining about EMF exposure when living near power lines, so you can imagine the distances the fields can reach out to. And then you can imagine the fields you're immersed in when you're in a substation and fairly close to the big stuff. Induction potential is normally negated on the structures and fences at stations by the grounding straps which dissipate the potential immediately

If the induction potential is not dissipated by grounds (because some Darwin Award candidate removed them), any electromagnetic induction potential will look for a way out, and will create an arc when it finds it. Since it draws no actual fault current from the energized equipment, there is nothing for the relays to detect. It will just burn and burn and burn, either until (1) there's nothing left to burn, (2) the damage created by the arc grows so large that the arc can no longer jump the gap, (3) the arc reaches energized equipment where the relays can see it, (4) the damage causes a structural failure of some variety that indirectly creates a real fault, or (5) a power company guy observes the arc and takes action to de-energize the associated equipment.

So, as you might conclude, exposure to electricity by induction is far more hazardous than exposure to straight-up energized equipment. The energized stuff will pretty much kill you 99.999% of the time, though rare exceptions have been noted and made possible by quick fault clearing and only on relatively low voltages. But you've got no chance at all when you get whacked by undetectable and non-clearing induction current.

Just some things to think about.

Friday, December 11, 2009

Death by Electricity and Ignorance

The Smooth Substation Operator sent these to me recently.

Be clearly warned: The photos behind these obfuscated thumbnails are full color, full detail, are not for the faint of heart, and are every bit as visually traumatic as the appearance of victims of fire. 

I do not know where this took place, how long ago, or who these souls were. They attempted to steal electrical cable to trade for cash at a scrap dealer, for reasons I won't guess on, as it is no longer important to them or us.

It is fairly common for power companies to have the neutral grounding straps cut and removed from substation structures and facilities by metal thieves. Disconnecting the neutrals can allow voltage induction, with no way for the stray potential to be grounded and made harmless. In some cases, the removal of grounding straps might give you a buzz when touching something normally "safe", but in other cases without the grounds, touching something normally as innocuous as the station fence can be fatal. Always look for intact grounding straps attached to the fence before you touch it. Or... just don't touch.

Anyway, I digress. I am willing to bet that if you asked these two before they got into it, if what they wanted the cable for was worth risking their lives, they likely would have said no.

As such, they did not know what they were dealing with, gambled in a game where they didn't understand the stakes, and lost. These aren't the first guys to die doing this, and won't be the last.

Now you know what you're dealing with. Stay away.

Again, the full-size photos are uncensored, visually graphic, and may be upsetting to some of my readers. If in doubt, just don't open them. Some things are hard to forget, you know.

Wednesday, December 9, 2009

Not grid. Not fire. But, dude.....seriously?

My previous post, Mobile Command Post Fail so far brought in just one comment.

But it was a good one.

Learned about a new site. No idea why I had not heard of it before, because it is my kind of funny.

But seriously, you want a REAL mobile command post? With style? It's even red:

You just gotta go see the rest of the pics. Don't miss the interior driver's seat picture. Holy crap!

Wander around Hamsexy a bit. You'll laugh, you'll cry, you'll basically never get those ten minutes back and you'll wish you had gone your whole life without seeing some of that stuff.

Thanks, Jacob.

Tuesday, December 8, 2009

Mobile Command Post Fail

How many antennas are enough? How many more are required to impress the ladies? How many more to impress the ladies enough that they'll overlook the rust?

But you're a ham radio guy? Yeah, so am I. And you're a card-carrying storm spotter/chaser? Yeah, so am I, though I loathe the punks who drive like hell just to watch carnage under the spotter pretext. I work with the kinds of guys who actually report data to the NWS and, you know, get tornado sirens activated to, you know... try to save lives. I don't even bring a camera with me. Anyway, I don't have that many antennas (just the one is fine, thanks), and can still do everything I need to do despite not having 'Skywarn' stickers nor whackerbar lights for 360 coverage. But hey, I bet you get more dates than me.

Monday, December 7, 2009

Tutorial 9: System Protection, Part II

After a long layover, the tutorial is back.
I hope you enjoyed the break.

A quick total review of what we have learned so far:

In Tutorial 1 - DC vs. AC, we learned... well, the difference between Direct Current (DC) and Alternating Current (AC) and why AC is used for the grid.

In Tutorial 2 - AC Supply and Demand we learned that power on the AC grid is produced exactly at the rate it is consumed, and that it cannot be stored.

In Tutorial 3 - Generation Supply, Control and Scheduling we learned how power companies decide which power plants to run, and how they exchange power between companies.

In Tutorial 4: Basic ACE, and Reserve Sharing we learned how power companies determine if they are generating too much or not enough power, and how they quickly recover from the loss of a big power plant with help from their neighbors.

In Tutorial 5: Introduction to Power Lines we learned about the different types of lines and voltages are used to move power efficiently, and about transformers.

In Tutorial 6: Identifying the Wires and Tutorial 7: Identifying The Wires, Part II we learned how to... well, identify the wires.

And lastly, in Tutorial 8: Introduction to System Protection we learned why the system protects itself.

Wow, that rehash was long enough to be its own blog post. But it's been over a month since we did one of these, so I feared you might be rusty.

Well, I am, anyway.

Seriously, you might want to at least re-read Tutorial 8 to spin things up again before starting this one.

And you are hereby forwarned, this will be a LONG post. Go refill your coffee now and stop in the restroom on the way. We'll see you in a minute or two.


OK, ready?

The electrical grid is protected at every point on its journey from the power plant's armature windings to the end terminals of the power cord inside of Firegeezer's coffee pot. For most of us, this power travels a pretty long distance, goes through several voltage changes up and down, passes through switches and circuit breakers, and does a few other exciting things before getting to you. It is not practical to dump the entire grid between you and the power plant if your coffee pot shorts out, so your household circuit breaker does the job. That action defines the first of many segments, or 'zones of protection' in power company parlance, the first (or last) of which is the zone between your breaker panel and your coffee pot, protected by your panel breaker.

An aside on circuit breakers. On this blog I pretty much always refer to the large breakers generally found in substations. These breakers have just one job: break the circuit. They cannot sense problems, so they only open when commanded to by other equipment. I might get into those big breakers later, but I want to differentiate the big breakers from the tiny breakers found on the breaker panel in your home. Your tiny breakers do both jobs; that is, they detect the problem AND break the circuit. Tiny breakers detect problems in two ways: The electromagnetic portion detects sudden surges, and the bi-metal portion detects high loads over time. Look it up if you want to know more about those, just so you know that aside from the size difference, there is also a functional difference between the big breakers and the tiny household ones.

Back to the zones.

The names I will use below I am more or less making up to keep things simple for the uninitiated, and may or may not be the terms used locally by your power company. In fact, they probably aren't. If you try to mention these specific zone names to linemen, other dispatchers or protection engineers, they will probably look at you with a raised eyebrow and try to tactfully change the subject.

So, the first zone we have identified. We'll call it the COMPONENT ZONE, referring to components that you plug into your outlets at home. The elements protected are, going backwards: Coffee pot, power cord, electrical outlet, wiring in your wall leading from your breaker panel to that outlet. If there is a problem here, as long as things are installed properly, your household circuit breaker should take care of it.

The next zone is relatively tiny, the PANEL ZONE, and pretty much just watches the stuff inside your breaker panel, between the main breaker and the individual circuit breakers. If something goes awry in the panel, the main breaker will dump the house. Also, if one of the circuits in your home in the COMPONENT ZONE has a serious problem and the assigned circuit breaker fails to clear it in a timely manner, and if things are configured properly, the main breaker should dump the panel to take care of business.

The next segment is the SERVICE DROP ZONE. This starts at your main breaker and goes upstream from there, to the source wires going through your wall and to the service drop or other point of entry, out via overhead or underground service wires to wherever your local service transformer is, be it a pad mount transformer or a poletop can transformer, and then up to the fused cutout serving the transformer. The fused cutout was touched on in the "Wires" tutorials, it is a switch where the opening part is also a large fuse. If something happens downstream from this cutout that draws enough load, the fuse will blow. It generally will blow for problems with the transformer, but if there is a problem downstream from that drawing a lot of fault current and not being cleared, it will eventually cook the transformer and cause the fuse to blow. Sadly, by the time the cutout blows for a problem at your panel or somewhere downstream that was not handled by the main breaker, your house is on fire already, causing me to drop what I'm doing and come see you in my Big Red Truck anyway. See, I'm involved either way.

The next segment is the FEEDER ZONE, which travels upstream from your cutout switch. That includes the jumper or other tap wire from the cutout to the primary distribution feeder conductor, and that feeder circuit back to the next breaking point. For most applications, that breaking point is a relatively small substation feeder breaker. Alternatively, there may be an in-line field recloser (basically a light-duty feeder breaker mounted on a pole) or other switch designed to sectionalize the feeder into pieces. Sectionalizing makes sense for long feeders or multi-branched feeders. If you can detect a problem and isolate just the troubled portion while still reaching the rest, it makes it unnecessary to dump the entire feeder. These breakers and reclosers cannot generally detect problems on their own, but rely on signals from other equipment, known as relays, which measure things like voltage, amps, flow magnitudes and such, for Bad Things. When Bad Things happen, the relay will then tell the appropriate device(s) to open. The relays at this point may take up to several seconds to decide there is a problem and clear the circuit. Also, the relays are often set up to attempt to reclose the breaker once or twice to see if the problem was transient (we talked about that in past posts), but will give up after enough failed attempts.

The FEEDER ZONE segment backs up the fused cutouts if they don't blow and draw enough fault current, but this backup is not very reliable. It is impractical to reliably get the relays to sense that kind of small fault that far away.

This picture placed here to give you some brief unrelated entertainment, because this post contains no other photos. Entertained adequately? Good.

The next segment is the DISTRIBUTION BUS ZONE, and is a small area running from the circuit breaker at your distribution station into the bus work at that station, which serves all of the other feeders. Typically this bus has a breaker on every component attached to it: Any and all feeder circuits, input sources from transformers, and voltage regulation devices. The relays watch the sum total of the electrical inputs and outputs of all of the things attached to the bus. They should sum to zero, as the bus does not keep any electricity for itself. If they don't sum to zero, the only plausible conclusion is a fault to ground, and all of the breakers attached to the bus are ordered by the relay to open, like breaking up the party and clearing all the uninvited teenagers out of the house. Bus faults are the worst thing that can happen to us dispatchers, particularly at large transmission stations. It just breaks the crap out of everything, often causing a serious disturbance when major transmission buses crater on us. Thankfully, bus faults and bus clearing events are rare.

The DISTRIBUTION BUS ZONE backs up the FEEDER ZONE. The relay that watches the feeder is responsible for telling that feeder's breaker to open. If that breaker fails to open for whatever mechanical odd reason, the feeder relay lets the bus relay know about the problem. The bus relay then rolls its eyes and clears the bus to handle the problem. Also, the bus relays watch the feeder and can detect the same faults that the feeder relays should operate for. If the feeder relay fails to open the circuit, and also fails to ask the bus relays to help out, the bus relay will eventually shove the unconscious feeder relay out of the way, grab the steering wheel and take over, again by dumping the bus to solve the problem. Relays on buses, as a rule, never attempt a reclose on a bus fault. One trip goes straight to lockout, and the station requires inspection before attempting to pick it up again.

The next segment is the TRANSFORMER ZONE. Like the bus zone, it is small. Its only job is to protect the multi-million $$$ transformer. Or at the least clear it before it boils and blows up and takes the whole station out. Like the bus zone it measures the in and out flows, but can also be activated by other sensed problems such as gassing in the transformer, or a sudden pressure increase internal to the transformer. And like the bus zone relay, transformer relays do not reclose breakers into a problem seen on the transformer.

The TRANSFORMER ZONE backs up the DISTRIBUTION BUS ZONE in two ways. If the bus attempts to clear and is unable for whatever reason to do so, the bus relay will ask the transformer relay to dump the source into the bus (the transformer). Similarly, the transformer relay can detect high loads, particularly high fault current at the bus, and will unilaterally dump the bus if the bus relay doesn't take care of issues at home fast enough. And, if for whatever reason the FEEDER ZONE fails to trip, and the DISTRIBUTION BUS ZONE fails to help out, the transformer relay will usually still sense the crazy high current and low volts and eventually throw everyone out of the pool.

Do you see a pattern yet?

Upstream from the transformer there may be a "high side" bus (higher voltage than the other side of the transformer). If so, there will be a TRANSMISSION BUS ZONE. It works on the same principles of the distribution bus.

Leaving the station, on the other side of the breakers attached to the transmission bus, are the high voltage lines, making the TRANSMISSION LINE ZONE. They work similarly to feeders in regard to how problems are sensed, but they usually are cleared by high-speed relaying and high-speed breakers, and usually also include automatic reclosing. These transmission-clearing setups are really amazing. The problem occurs, the relays sense it, the relays decide where the problem is, the relays decide which breakers need to be opened to solve the problem, the relays tell the chosen breakers to open, the breakers get the signal to open, the breakers mechanically do their thing and break the circuit.... from fault to clearing, sometimes as fast as 1/10th of a second or less, TOTAL. FAST.

All along the way, transmission lines, transformers and buses are all protected by their respective zones and the neighboring zones poking their nose in the goings-on at each point, ready to take over as needed. Finally, at the power plant end, there are zones protecting the power plant's station components, which are more or less configured in reverse of distribution stations but rated for higher loads. And then a zone from the yard to the generator. The generator zone has a bunch of nifty things it watches for that I won't get into here right now, but there are just bunches of things that will excite one or another relay and trip a power plant, especially the steamers (coal/oil/gas/nuclear/etc). At pretty much all points above the feeder circuit breaker, there are multiple relays that can sense faults not just in their designated area, but for good distances through other zones as well. The relays with overlapping detection have built-in time delays to give the local equipment time to handle it, but will do what they can to solve the problem if too much time goes by (usually measured in just a few seconds). I think you get the idea without me spelling it all out. Your eyes glazed over five paragraphs ago anyway.

Goodness gracious, what a long post! But wait, there's more! Next time.

What we learned: (1) The path from power plant to power outlet at home is broken into many segments, each protected independently and usually backing each other up. (2) The clearing is designed to remove only the problem, to interrupt as few people as possible. (3) We remember now why I didn't get back to the tutorial series for a while. They are way too freaking long.

That's enough. Out.

(Click this link to see all posts tagged "tutorial")

Thursday, December 3, 2009

The Six. Never Forget.

I just don't have much wit to share these days, it is a hard time of the year. The Lakewood WA incident still stings, and will for a while. And the beginning of December always harkens to the Worcester event. Has it already been ten years? Rest in peace, brothers.