Tokio Marine North America Services (TMNAS)

Solutions

UFT One is engineered to facilitate distributed test execution of multiple tests simultaneously. A proof-of-concept demonstrated that this could substantially speed up TMNAS’s testing cycle. Initially just one license was purchased, and Soisson was delighted with the results: “We went from three to four days of manual testing to an eight-hour automated overnight testing session. When we showed the CIO the ROI we were able to achieve with one license, he got really excited and challenged us to put a roadmap together to purchase more licenses, and start automating the entire regression test process. The early success with UFT One developed confidence in my team and quickly paved the way for additional Micro Focus (now OpenText) solutions like ALM and LoadRunner Enterprise.”

ALM is an integrated IT software quality management solution that enables consistent QA procedures across the application lifecycle by enabling test case management and defect fixing in a centralized location. LoadRunner Enterprise is an automated performance and test automation solution that provides an accurate picture of end-to-end system performance under varying loads, so that issues can be identified and resolved before an application goes live.

TMNAS identified core test cases and built a framework to automate these. Over the next couple of years more and more test automation was added and a Quality Center of Excellence was established that included the newly formed automation team. Performance, functional, and end-to-end regression testing are all included and can be remotely executed and monitored through a Virtual Automation Lab, hosting 12 dedicated test automation VMs. Proactive dashboards and metrics provide a real-time view of test automation within TMNAS. Soisson: “We started with just one automation engineer and have grown the team to nine resources. We run the automation team as a development team rather than a test team, and we feel the value added to the business is what gives us our credibility.

With 780,000 test checkpoints and 100 percent of regression tests being automated, all tier 1 applications undergo a monthly performance validation before being released into general production. This includes 150 hours of automated regression testing to cover all core applications, running high risk business workflows worth $4.5 billion to TMNAS.

The response from the business has been overwhelmingly positive: “If we have to make an emergency change to a key application, our internal clients feel safe in the knowledge that our test automation processes will run overnight and nothing catastrophic will have happened to their application when they come in in the morning. They trust our team and the processes we’ve introduced,” says Soisson.

Hemant Anugonda, Senior Manager of Quality Services at TMNAS, adds: “We’ve introduced parallel threading, leveraging TestNG, UFT Developer, IntelliJ, and Eclipse, running up to three testing threads on the same machine. This lowers test execution times significantly and enables us to release our versions much faster. The use of headless browsers (i.e. browsers without a graphical user interface) helps us run our test automation scripts without even rendering them on a screen. This allows us to reduce our regression test time by 70 percent, time we can use to execute deeper and broader application tests.”

TMNAS has ambitious plans for the future and the Quality Center of Excellence will play a central role in supporting them. It is moving towards a SaaS model to create more efficiencies and enable faster upgrades. In preparation for this, the foundations are being laid for a Test Data Management team to service test data requirements for TMNAS. They will enhance the automated testing framework as a self-service solution.

In response to increased use of mobile applications, TMNAS is expanding its mobile test capabilities. This triggers additional security requirements and the Quality Center of Excellence is expanding its remit to include security testing as well. Integration of automated smoke tests with Octopus Deploy supports continuous integration and deployment, shortening the time to release new versions. A large infrastructure transformation program is underway and the Quality Center of Excellence aims for all associated application testing to be automated and in a self-service model, through the use of OpenText (formerly Micro Focus) solutions.

Results

Soisson comments on the results: “In the first year, we were able to demonstrate a saving of over 1,500 man hours, leveraging UFT One. This meant we could focus our team efforts on value-add projects for our internal customers.”

ALM, he notes, has been especially helpful: “We were totally dependent on spreadsheets prior to buying ALM—it was a lot of manual work opening every spreadsheet to compile metrics across our enterprise. ALM took us to the next level—standard metrics are included and you can design whatever else you can imagine.”

LoadRunner Enterprise has been so successful in helping the team test and ensure the performance of TMNAS core systems and other applications, that it’s now an integral part of the company’s standard operating procedure. Anugonda explains: “LoadRunner Enterprise makes it easy to benchmark the performance of your software, then rerun the same scripts after each deployment to make sure any deltas are acceptable—we’ve built an entire process around that and it’s now an integral part of our Quality Center of Excellence.

The TMNAS team feels ready to support a move towards DevOps which is a roadmap item. Soisson: “With Micro Focus (now OpenText) solutions, we have built a foundation to introduce a continuous integration testing and deployment cycle in the near future. Aspects of this are in place and we have already shortened the time it takes to release applications, and have reduced our test execution time by an incredible 80 percent.”

He concludes: “We are very proud of the Quality Center of Excellence and the trust our employees have placed in us. The partnership with Micro Focus (now OpenText) was key to its success. They listened, understood our long-term vision, and worked closely with us to help us achieve our goals as a business.

In the first year, we were able to demonstrate a saving of over 1,500 man hours, leveraging UFT One. This meant we could focus our team efforts on value-add projects for our internal customers.