Latest from C&E/NetDev Ops/GIS/Open-Source Networks
Windstream’s Network Optimization Best Practices
Find the Needle and Move It —
When the CLEC boom was on the rise, my career began in telecom. At that time, Congress made regulatory changes to its communication policy with the goal of reducing the barrier to entry and increasing competition. These changes created a frenzy of activity which felt like a new CLEC was starting daily. ISPs, power companies, newspapers, start-ups — anyone who thought they could make a run at it were throwing their hats in the ring and starting their own communications company. The competition was fierce, and to survive meant out-pacing the competition.
As the CLEC boom slowed and as the various economic pressures took their toll, industry roll-up began to occur. As it turns out, building and operating telecommunications networks isn’t cheap. The fine balance between network experience and network expense largely decided many of the winners and losers. As the roll-up occurred, many of the "winners" found themselves with a stockpile of network assets built over the course of 20-30 years, largely overlapping. How many voice switches does one company really need for a given market?
Fast forward to 2019: the landscape of the communications industry looks a lot different than it did in 1999. But, there is a buzz that resembles the excitement of that time.
Today, the excitement is about competing in new ways. Connectivity alone is no longer enough. Customers have a variety of choices when it comes to connectivity; the new race is about the customer experience and differentiation through software. Cloud services, IoT, SD-WAN, 5G, network orchestration — all of these are part of the new set of technologies and products driving that excitement. That said, shifting focus typically requires a balance of caring for the old while developing the new.
The skill sets required to harness the latest and greatest technology and to craft new and meaningful product sets are not the same skill sets that installed and maintained the telecommunications networks of yesterday. On top of that, the physical network isn’t going away. The equipment will modernize, but there will always be some physical underpinnings that hold everything up. That physical network, old and new, must still be cared for. Much of the stockpile of overlapping network assets is aging. The equipment is mostly End of Life, End of Support, and the skilled workforce who know these devices are aging out of the industry.
InvisiLight® Solution for Deploying Fiber
April 2, 2022
Go to Market Faster. Speed up Network Deployment
April 2, 2022
Episode 10: Fiber Optic Closure Specs Explained…
April 1, 2022
Food for Thought from Our 2022 ICT Visionaries
April 1, 2022We know the network needs to be modernized, optimized, and consolidated, but the network is very complex. Most service provider networks are similar in nature. Riding directly on the fiber are typically 3-4 layers of transport technologies aggregating the various systems and circuits (DWDM, OTN, SONET, ATM, Frame Relay, DCS). Each layer is typically a unique vendor/platform combination with its own unique provisioning tools, layered in different ways. Riding on these systems are your service layers, typically voice and/or data being delivered over common access lines via all sorts of access technologies (Fractional T1s, xDSL, EoC, Access nodes). Don’t forget about the supporting infrastructure of BITS clocks, test units, echo cans, repeaters, power, and HVAC.
With all of the complexity, how do we deal with our legacy networks? They still hold significant amounts of customers and revenue. Do we pick devices that we don’t like and attack them? Do we pick specific layers of the network and attack those? Do we overbuild the entire thing and groom? There are several different approaches to optimizing and modernizing these legacy networks, but they all come with their own challenges and they all come with a price tag. The goal is to pick an approach and move the needle, right? But asking ourselves which needle we are trying to move can be the difference between a successful strategy and just a lot of time, effort, and expense.
Focus on the Right Needle
At Windstream, we tried a variety of approaches and we chased a variety of needles. We attacked individual devices, we attacked specific vendor/platforms, we even started attacking high-cost sites, but through each attempt we realized that we either weren’t moving any needles, or the needles we were moving weren’t meaningful enough. They weren’t making enough of a difference to justify the costs and effort. We didn’t start really finding success until we oriented our efforts around the goal of cost savings.
We achieved this cost savings through a program dedicated to site exits. As mentioned earlier, network overlap is typically high. Multiple legacy sites from multiple legacy companies are still being operated. Every site carrying its own lease, its own utility bill, its own access expense, and also its own customers and revenue. Each site filled with legacy network elements that are stacked and intertwined with one another. Each site requiring routine truck rolls for maintenance or break-fix issues. When you dig into it, each site carries a significant amount of both direct and indirect expense.
5 Learnings
Following are 5 learnings that may help you as your company considers Windstream’s network optimization strategies.
Learning #1: DEDICATE RESOURCES
We pulled resources from all over the company to build a team comprised of all the necessary skill sets to plan and execute these projects. We wanted to own as much of the project as possible. This included transport engineers, voice engineers, IP engineers, designers, project managers, and technicians. The projects are very complex because the network is very complex. Dedicated resources ensure that the complex problems get solved and not ignored.
Learning #2: MANAGE THE VOLUME
When we started digging into the list of network sites, there were thousands. We knew that a deep dive into each site wasn’t feasible to get started. We needed a way to compare the effort vs. reward for each site, so we developed our score-carding technique based upon critical data points about the site. Things like:
• Is there a voice switch?
• How many OSP fiber cables?
• How many circuits?
• How much are we paying?
• When can we terminate the lease? etc.
We assigned values to each data point and created our own scoring. This effort gave us an ability to find the lowest hanging fruit.
Learning #3: REQUIRE A BUSINESS CASE
We had to educate ourselves on what we pay for, and how we pay for it. Does freeing up rack space matter? How are we billed for power? When you have more good things to do than you could ever get done, you must set rules for yourself.
Executing due diligence to understand our billing allowed us to tie our efforts more directly to the savings, and helped us to avoid doing work that wasn’t ultimately going to generate anything. It also helped us to avoid just pushing expenses around. If 90% of my expense in a site is cross-connects to customers, spending the capital to consolidate racks may not actually save me anything. We might spend more doing the work than we’ll save.
Learning #4: USE TODAY’S TOOLS TO SOLVE YESTERDAY’S PROBLEMS
We developed our own research tools using the power of big data platforms. The horsepower we obtained allowed us to ask bigger questions of the network than ever before. It allowed us to more rapidly understand the network we were trying to address and thus arrive at a solution faster. It also helped us remove the swivel chair of information from one team to the next. These tools give us a common place to access the information. We also worked with partners to develop tools and automation that would take configurations out of one device, translate them into a different format, and load them into a new device. This saved hours of manual conversion.
Learning #5: DEVELOP GOOD RELATIONSHIPS AND PROCESSES WITH CORPORATE REAL ESTATE TEAMS
Consolidating spaces within a building is often as much a technical discussion as it is a contractual one. Understanding what space is attractive to your landlord vs. what you might have to move is critical in identifying space reduction opportunities. The Number One detractor from a predictable site exit plan is an unplanned forced exit. Ensure that you are monitoring your lease renewal options and notification schedules. You don’t want your network space being shopped out from under you.
Like this Article?
Subscribe to ISE magazine and start receiving your FREE monthly copy today!
The Results Thus Far
In our previous attempts at site consolidation work, we were only able to handle maybe 5-6 sites in a year. In our first year, we exited 24 sites. In 2019, we are on pace to exit another 30 sites. Doing the hard work of cutting bottom line network expense out of the business through network optimization not only gave us an opportunity to modernize our network, it generated a self-sustaining program that was easy to justify. In addition, it is helping to solve the earlier problem of making budgetary space for new skill sets required to transition the business.
