Overhaul operations management for a nationwide backbone network to coincide with a move to a new data center. The branch office access network is used by more than 8,000 network devices, totaling 45,000 modules and 300,000 interfaces. The number of managed elements was increasing with each year and the configuration was growing progressively more complex, leading to a greater management load as well. It had turned into a massive, complex network and more efficient management was needed.
“Expediting the troubleshooting process was another critical issue. Without network access business comes to a halt, and with the imminent move toward desktop virtualization and network consolidation, stable network operations had become more important than ever before,” adds Ishimaru.
With increasing access path diversity, they faced issues of how to determine the extent of impact of a failure and quickly pinpoint its cause from among a diverse device configuration. It was also imperative to enhance security. Nomura needed a system that would allow security management to be carried out more effectively and more reliably.
The key would be to automate processes that were being handled manually, such as rapid response to vulnerability information, policy compliant device configuration and the management of network device access logs.
It was decided to overhaul the management system for branch office access network in order to expedite troubleshooting, streamline operations, and enhance security. Implementation of the new management system along with comprehensive support was provided by the Nomura Research Institute (NRI), which has assisted Nomura Securities with systems configuration and operations for many years.
Nomura Securities is Japan’s leading securities company, with 5,390,000 accounts and 100.6 trillion yen of customers’ assets in custody (as of end of March 2016). The company meets a diverse range of customer needs with approximately 160 branches across the nation. It offers strong research capabilities and an extensive product lineup, including investment banking, real estate transaction advisory services and comprehensive financial services. The IT Management & Infrastructure Strategy Department at Nomura Securities singlehandedly oversees operations management of the branch office access network that connects all domestic head and branch offices with the data center.
Tomohiro Ishimaru, an associate of the department’s Information Infrastructure Section, explains the need for expansion of the network environment in order to meet diversified business needs.
“In January 2014, we set out to build a new data center and a more powerful, faster network core with our sights on eventually offering virtualized desktop environments and building a common infrastructure system. This was also a good opportunity to re-evaluate our network management.”
Kenji Suzuki, a senior technical engineer for the IT Platform Services Division at NRI, chose Network Node Manager i (NNMi) as the network management software it would implement to help expedite troubleshooting.
According to Suzuki, it was the advanced filtering functionality of NNMi that appealed to him the most. Being able to define filter conditions to group managed elements together made it easier to manage the wide variety of network devices, which in turn led to speedier troubleshooting.
“Once an administrator has completed the initial configuration, the IP address is the only thing that needs to be manually input when registering a device on the network. NNMi automatically detects which branch office the device is located at and what the device is, and then automatically assigns it to a group. In the event of an error, it uses that configuration information to determine the extent of impact and then represents the impact visually on a network map, which allows for a speedy response.”
It's said that troubleshooting generally comprises around 10% of network operations management, while the remaining 90% is maintenance. The burden of device information collection, configuration, and other everyday maintenance tasks for the Nomura Securities branch office access network had begun to grow unwieldy. In order to make the handling of these tasks more efficient, Nomura Securities also decided to implement Network Automation (NA) network operations management software.
This project gave me a renewed appreciation for the value of Micro Focus (now OpenText) software as products that can be implemented with confidence in a large-scale system environment. The Nomura Securities network will continue to grow, and I look forward to Micro Focus (now OpenText) Japan continuing to provide cutting-edge products with exceptional reliability to support it.
A main advantage of NNMi is that it is capable of monitoring the massive network in its entirety:
“Previously, we had been using a combination of four systems to monitor the whole network due to performance constraints but with NNMi we can achieve the same monitoring with a single system. We cut the number of admin servers and the amount of work required for registrations down to a quarter of what it was previously, which resulted in a reduction in operational costs,” says Suzuki.
NNMi has come to support countless IT infrastructures since its release in 1990. The OpenView brand name that the product was previously released under is considered synonymous with highly-reliable network management software by many administrators today. According to Suzuki, this also contributed to his confidence in choosing the software.
“We considered a number of open source solutions as well, but they were all technologically outdated and had stability issues. Due to its long history of success and reliability as a management platform supporting businesses, NNMi was really the only logical choice.”
Upon implementation, the engineers drew on their familiarity with operations to define filtering rules that would lead to speedier troubleshooting. NRI also proposed the integration of multiple dashboards onto a single screen.
“We had been using different dashboards for each of several monitoring tools. Using the product integration features of NNMi, we were able to integrate network monitoring information from NNMi and server monitoring information from other tools into a single dashboard,” says Suzuki.
Failure operations were also unified. In the event of a failure, the dashboard will specify an appropriate engineer depending on the type of failure, which allows for quick escalation by an operator.
“Time to recovery varies depending on the type of failure, but the time required to investigate the cause of a failure has been reduced by 30% thanks to the integration of monitoring information and the standardization of operations procedures.”
NA has proven extremely effective for configuration management. Managing approximately 300,000 interfaces and 45,000 modules with Excel just isn't feasible. It automatically collects firmware, CPU, power supply, and other detailed information from network devices, making real-time information management possible. As a result, Nomura was able to reduce information collection man-hours by 30% as well as increase the efficiency of a variety of other management tasks using the configuration information collected by NA. For example, comparing configuration, contract, and support information allows the company to generate a list of devices nearing end of support with a single click.
“We were able to automate part of the network device configuration process as well. The standard task of simultaneous network device configuration changes was previously carried out by our engineers, but now our operators can use NA to do this automatically, making the process much more efficient,” says Suzuki.
Nomura also uses NA to automatically check device settings. Relegating this arduous task to the software has helped the company to avoid network issues caused by misconfigurations due to human error.
Suzuki concludes: “We're also responding to vulnerability information quicker. Before, we were spending large amounts of time searching for the corresponding devices from among a myriad of network devices, but NA can generate a list of devices for us with a single click. The rapid response to vulnerability information has reduced our business risks.”
The company also automated policy-compliant device configuration and the management of network device access logs. Using the automation capabilities of NA to their full extent has resulted in more exhaustive security management with less effort.
Nomura Securities has now made the move to a new data center and transitioned to the new network management system built around NNMi and NA. In doing so, it achieved its initial objectives of speedier troubleshooting, more efficient operations, and enhanced security.
Tomohiko Kamikawa is an associate of the IT Management & Infrastructure Strategy Department's Information Infrastructure Section and helped build the new network management system. According to Kamikawa, the NRI, which has assisted Nomura Securities with systems operations for many years, played a critical role in this project.
“NRI is always considering how to make improvements when approaching network operations. Their solution this time was to implement NNMi and NA, and the result has been a clear improvement in both the quality and efficiency of our operations,” explains Kamikawa. “We needed to solve the issue of integrating multiple dashboards onto a single screen using product integration, and NNMi provides the functionality to do this. The software meets the needs of those wanting to integrate operations within an existing environment.”
We were able to automate part of the network device configuration process as well. The standard task of simultaneous network device configuration changes was previously carried out by our engineers, but now our operators can use NA to do this automatically, making the process much more efficient.
Nomura Securities is Japan’s leading securities company, with 5,390,000 accounts and 100.6 trillion yen of customers’ assets in custody (as of end of March 2016). The company meets a diverse range of customer needs with approximately 160 branches across the nation. It offers strong research capabilities and an extensive product lineup, including investment banking, real estate transaction advisory services and comprehensive financial services.