Being Powerfully Redundant Matters

Rectifier Battery Drop

Understanding the Risks
2017 tied a record (with 2011) for the most extreme weather events causing damages in excess of $1 billion in a single year. Although driven to some extent by increasingly crowded coastlines, inland disasters have also wreaked havoc. Of the 16 different extreme weather events in 2017, the majority occurred away from the coasts, including hail storms in Minnesota, tornadoes in Missouri and Illinois, flooding in Arkansas, thunderstorms across Oklahoma and Texas, and a freeze in Georgia.

The increase in extreme weather events in recent years has driven a corresponding increase in weather-related power outages. Exacerbating the impact of the increase in extreme weather is the diminished condition of the country’s power grid. In its 2017 U.S. infrastructure report card, the American Society of Civil Engineers gave the U.S. Energy Infrastructure a D+ ranking. With over 640,000 miles of high-voltage transmission lines that were mostly constructed in the 1950s and 1960s and some local distribution infrastructure constructed before the turn of the 20th Century, today’s power grid is severely oversubscribed. Power utilities are now playing catch-up to generate and distribute the 40% of additional power required by US businesses and consumers.

Weather-related power outages are directly impacting communications network uptime, while demand for high-speed data continues to grow relentlessly. High-speed data demand is being driven by well understood macro trends such as cord-cutting, smartphone penetration, and a strong economy. A less understood driver of data consumption is the growth in connected devices including consumer, commercial, and industrial Internet of Things sensors and appliances. At ISE EXPO this year in Denver, Jack Waters, CTO & President, Fiber Solutions, Zayo Group, cited a 40% CAGR in this usage.

40% of America’s utility technicians are eligible to retire in the next 5 years. This wouldn’t be a tough problem to solve if millennials were interested in entering this workforce, but they are by and large not attracted to this type of work. This means that service providers can no longer rely as much on human capital to maintain network elements. Consolidation of service providers is designed to ultimately drive scale and efficiencies, but it creates near-term financial pressures until "synergies" are realized. This financial pressure often drives prioritization of cash away from the capital budget toward debt service and dividend payments. Underinvestment in the network impacts service quality, customers defect, revenue erodes, and margins shrink. It’s a vicious cycle.

Where does this leave service providers? With batteries aging in the field, an aging technical field force, and increasingly tight capital budgets, it may seem like an impossible situation. However, hardening your network infrastructure to improve network reliability is not without its ROI potential. In fact, as service providers eventually start to reinvest in their networks it is critical that they do so intelligently, utilizing automation, data, and IoT, to multiply the impact of their investments. Backup power is no exception.

InvisiLight® Solution for Deploying Fiber

Go to Market Faster. Speed up Network Deployment

Episode 10: Fiber Optic Closure Specs Explained…

Food for Thought from Our 2022 ICT Visionaries

Backup Power Made Easy
So, what are these processes and tools mentioned earlier? Proactive data-influences and holistic power network oversight encompass how providers can be more effective with the few resources they have at their disposal. Let’s break this down into examples for contextual purposes:
1. Testing batteries has become time- and cost-prohibitive.
2. Power-related problems often drive wasted site visits.
3. Batteries fail prematurely in the field.
4. Battery problems may not really be battery problems.
5. Triage of sites during an outage.

1. Battery Testing
For decades telecom regulators clearly mandated 8 hours of backup power for voice customers, which was accomplished using various types of batteries at the central office or remote terminals. While these regulatory requirements are increasingly muddied these days, service providers tend to adhere to the 8-hour guideline. And lead acid batteries continue to be a work horse for ensuring network uptime when the power goes out.

Lead acid telecom batteries are expected to last 7 years on average, but only if properly maintained. One such maintenance step, battery testing, has fallen victim to financial pressures for many carriers in recent years.

Battery testing can be completely automated and can provide significant insights into network performance that can improve reliability and the customer experience.

2. Wasted Site Visits
Rural networks often require site visits that can be >100 miles apart. The shrinking technical workforce stretches to try and address all of these sites with less help, more sites because of financial consolidation, and a lack of insight before they make the trip.

Remote monitoring can provide insights that eliminate the initial "discovery" visit and inform the technician to make the maintenance or repair visit impactful.

3. Failing Batteries
Batteries are not meeting their life expectancies in the field. Float charging batteries, especially in extreme environmental conditions, has been shown to shorten expected battery life by 50% or more. This phenomenon can be reversed by resting the batteries. While at rest, remote test measurements avoid being tainted by the charging process, and provide much more accurate and predictive data.

4. Identifying the Root Cause of the Issue
Often what appears to be a battery issue is actually a problem with another power-related component at a site. For example, rectifier problems can often mask as battery issues, and are difficult to diagnose without accurate, trended data measurements.

For example, an over-compensating rectifier blade may only get worse as the heat picks up in the summer. This blade may be swapped out, returning battery voltages to a normalized level.

5. Outage Response
This faulty rectifier could have turned disastrous later this year during colder months by over-charging the batteries, increasing probability of thermal runaway.

Automated prioritization places sites with the least amount of backup time remaining on the top of the list for triage.

Hurricane Florence
This mighty storm whipped winds over 100 mph at landfall back in September. Its slow pace concentrated rainfall along the coastal Carolinas, amounting to several feet of standing water even weeks after the major event. The confluence of the strong gusts of wind and massive amounts of precipitation created storm surges as tall as homes when the storm was at its worst. Close to 1 million people lost power, and today some are still recovering from what happened.

Geographies ridden with natural disasters go by a different playbook than those that rarely experience it. To whatever capacity your network lies in, there is something to be learned by all of us from Florence.

Sites outside of Wilmington, North Carolina, were in the direct path of the storm and very susceptible to outages and lengthy downtime, but that didn’t stop a carrier from ensuring service. Churches, convenience stores, and even city hall, fell within the service area of these sites, along with a large number of residential customers for 1 data transmission site. Batteries, generators, and power management software, equipped them with the DC power arsenal they needed to readily respond.

Healthy batteries discharged to provide immediate power to maintain communications uptime. This came after utility power was disrupted at 2:50 a.m. ET on that Friday morning. Most technicians do not have remote visibility for this insight or battery health measurements that indicate backup time. Power management software empowered this carrier by alerting them in real time when commercial power was lost, telling them how much battery backup was available and whether a generator was located at the site. Their technicians now had insight that helped them make more informed decisions on what actions to take, especially for sites like those serving city government during a natural disaster. Their preparation paid off for their community.

Backup power configurations react swiftly when outfitted with batteries, intelligent infrastructure, properly fueled generators, and software, to visualize all of this streaming data. Our processes and tools outlined earlier through the 5 examples proved to be integral for a streamlined recovery in the Carolinas. State of health data from multiple battery tests per day gave them confidence that they had the backup power they needed before and after Florence. Moreover, resting their batteries ensured more backup time was available when needed. By streamlining and automating more mundane power-related maintenance functions, technical staff were able to focus their time reviving communications equipment.

Data Integration for Customer Satisfaction
These 5 use cases illustrate how data can be used to inform network managers and drive more efficient network maintenance. This data, gathered using Servato’s Smart Power System line of products, including the newly announced integrated rectifier with battery management (SPS-R™), can be integrated into NOCs and trouble ticket systems using SNMP traps. In addition to eliminating wasted truck rolls, this data can be used to make all truck rolls more impactful. Resting batteries can extend their life and provide more accurate and predictive state of health measurements to drive other battery-related insights. The CapEx and OpEx savings generated by these efficiency gains can drive attractive ROIs. More importantly, an investment in intelligent infrastructure puts the customer first.