The Network Maintenance Corner

The Importance of Alarm Audits —

COLUMN INTRODUCTION:
The Network Maintenance Corner is focused on the daily struggles faced by Network Technicians, Network Managers, Network Analysts (NOC), and Network Engineers. New and newer technologies will be a part of the subject matter. However, one primary goal is to address the legacy technology that is being expected to do more than ever, is difficult to maintain, and requires human intelligence and intuition to diagnose and fix issues.

The days when most Central Offices (CO) had an on-site, live resident Network Technician are over. One of my previous customers had 2 Network Technicians covering the entire state of Wyoming. That begs the question. How much maintenance is happening in that situation?

My situation is much better than that. In my territory, I watch over:
• 14 towns
• 3,000 Square Miles
• 11 Nortel DMS 10s
• 1 Calix concentrated CO
• 1 ADTRAN concentrated CO
• 1 Nortel OPM
• 0 traffic lights

Alarm Audits
Alarm audits are nothing new, and several large system wide initiatives related to them have occurred over recent years. The problem with most of them is that they are completed by shorting pins on the horizontal side of the Main Distribution Frame (MDF). This only verifies that a jumper is connected to the assigned alarm point. What (if any) equipment is connected, and whether the alarm engages when the assigned equipment has a problem, are in no way verified using this method.

Though the process is much more time-consuming and could be dangerous to subscriber traffic (if done improperly), the only way to make sure that the alarms are working as designed is to cause or fake the condition that is supposed to bring in an alarm.

During an audit of my territory, I found that about 20% of the alarms were either not working or reporting the wrong condition.

Let’s talk about the "big hitter" alarms. All alarms are important, and you ignore them at your own peril, but power and climate control are the big ones. In 2018, climate control is at least as important as power. And speaking of, let me share an experience from which you can learn. Recently, the high temp alarm engaged in one my COs on a Sunday; 2 of 3 cooling units went down. The threshold is set at 85 degrees F, but by the time I had been called out and got there, the inside temperature was 103 degrees F. This was true even though it was only 83 degrees F outside. The cause: 1 cooling unit not working.

As you know, today’s equipment now generates so much heat (and the most crucial transport equipment is the worst) that even on cold days it has the potential to cook itself without climate control. Unfortunately, my offices do not have alarms tied directly to cooling units. This makes us completely reliant on the single threshold high temp alarms. If the alarm comes in during non-working hours, or it is caught by the NOC, by the time someone is dispatched to the site, it is too often already too late.

Power
Prescribed battery backup times vary, but generally it is 4 hours for a site with a permanent generator and 8 hours if a portable generator must be pulled to the site. As we all know, commercial power normally comes back on before the batteries are drained to critical voltage, and about the time you pull up with a generator. Still, we can’t count on that.

Many of the battery strings were engineered when the CO amperage draw was 60% of current draw. With that, you can’t count on 4 or 8 hours. The crucial transport equipment can start to have problems at 46 VDC and can be the first to drop.

Even where there is a permanent generator, be careful. When undersized (again due to unaccounted for increases in amperage draw), a perfectly maintained generator can overheat and shut down under live load. With only a 4-hour battery backup, and often no external generator connections, the office can be in jeopardy before you can reach the site. There have been times when I’ve had to turn down the voice switch to save the transport equipment. Trust me, those old Class 5 switches do not like be turned down, and they are tricky to bring back to life after full power is restored.

The Network Maintenance Corner appears quarterly as web exclusive content at ise.mediaquest.co. The next 3 articles are planned for May, August, and November of 2018. Although my territory includes a broad spectrum of systems and equipment, I ask for input from YOU to make these articles as interesting and as informative as they can be, so send me an email at [email protected]. Encourage your team members to get a free subscription to ISE magazine by subscribing at ise.mediaquest.co/subscriptions.