OpenText home page.
Customer stories

TMNA services LogoTokio Marine North America Services (TMNAS)

Supports Quality Center of Excellence, delivering 80% reduction in test execution time and faster time-to-market

TMNA services Logo

Outcomes

  • Reduced test execution time by 80%
  • Introduced Quality Center of Excellence (QCoE)
  • Prepared for DevOps and SaaS model introductions
  • Accelerated releases via centralized, automated test management

Challenge

Introduce an application test automation framework to support ambitious growth plans from a central IT service.

IT within TMNAS is delivered as a central service to all TMNAS group companies. When this was first introduced, Todd Soisson, IT Vice President, Digital & AI of Business Experience for TMNA Services, had a very small Quality Assurance (QA) team that soon ran into capacity issues: “At the time we were using a freeware software testing tool for automated regression testing, and it just wasn’t cutting it. We were getting a new build every other day, and many new fixes were breaking older functions that had otherwise been working. Problem was, the freeware tool wasn’t catching everything and the scripts were difficult to maintain.”

Not long afterward, they also began receiving complaints from agents in the field regarding the slow performance of another core system. Further complicating the matter was the fact that test cases and associated metrics were being manually managed in spreadsheets. Soisson: “This testing model was obviously not scalable and we saw our workload piling up, while releases were in danger of running late.” TMNAS realized test automation could be the answer and started investigating the market for solutions.

With OpenText solutions, we have built a foundation to introduce a continuous integration testing and deployment cycle in the near future. Aspects of this are in place and we have already shortened the time it takes to release applications, and have reduced our test execution time by an incredible 80 percent.

Todd Soisson
IT Vice President, Digital & AI, TMNA Services

Details

Solutions

OpenText™ Functional Testing is engineered to facilitate distributed test execution of multiple tests simultaneously. A proof-of-concept demonstrated that this could substantially speed up TMNAS’s testing cycle. Initially just one license was purchased, and Soisson was delighted with the results: “We went from three to four days of manual testing to an eight-hour automated overnight testing session. When we showed the CIO the ROI we were able to achieve with one license, he got really excited and challenged us to put a roadmap together to purchase more licenses, and start automating the entire regression test process. The early success with OpenText Functional Testing developed confidence in my team and quickly paved the way for additional OpenText solutions like OpenText Application Quality Manager and OpenText Enterprise Performance Engineering.”

OpenText Application Quality Manager is an integrated IT software quality management solution that enables consistent QA procedures across the application lifecycle by enabling test case management and defect fixing in a centralized location. OpenText Enterprise Performance Engineering is an automated performance and test automation solution that provides an accurate picture of end-to-end system performance under varying loads, so that issues can be identified and resolved before an application goes live.

TMNAS identified core test cases and built a framework to automate these. Over the next couple of years more and more test automation was added and a OpenText Application Quality Management of Excellence was established that included the newly formed automation team. Performance, functional, and end-to-end regression testing are all included and can be remotely executed and monitored through a Virtual Automation Lab, hosting 12 dedicated test automation VMs. Proactive dashboards and metrics provide a real-time view of test automation within TMNAS. Soisson: “We started with just one automation engineer and have grown the team to nine resources. We run the automation team as a development team rather than a test team, and we feel the value added to the business is what gives us our credibility.

With 780,000 test checkpoints and 100 percent of regression tests being automated, all tier 1 applications undergo a monthly performance validation before being released into general production. This includes 150 hours of automated regression testing to cover all core applications, running high risk business workflows worth $4.5 billion to TMNAS.

The response from the business has been overwhelmingly positive: “If we have to make an emergency change to a key application, our internal clients feel safe in the knowledge that our test automation processes will run overnight and nothing catastrophic will have happened to their application when they come in in the morning. They trust our team and the processes we’ve introduced,” says Soisson.

Hemant Anugonda, Assistant Vice President at TMNA Services, adds: “We’ve introduced parallel threading, leveraging TestNG, OpenText™ Functional Testing for Developers, IntelliJ, and Eclipse, running up to three testing threads on the same machine. This lowers test execution times significantly and enables us to release our versions much faster. The use of headless browsers (i.e. browsers without a graphical user interface) helps us run our test automation scripts without even rendering them on a screen. This allows us to reduce our regression test time by 70 percent, time we can use to execute deeper and broader application tests.”

TMNAS has ambitious plans for the future and the OpenText Application Quality Management of Excellence will play a central role in supporting them. It is moving towards a SaaS model to create more efficiencies and enable faster upgrades. In preparation for this, the foundations are being laid for a Test Data Management team to service test data requirements for TMNAS. They will enhance the automated testing framework as a self-service solution.

In response to increased use of mobile applications, TMNAS is expanding its mobile test capabilities. This triggers additional security requirements and the OpenText Application Quality Management of Excellence is expanding its remit to include security testing as well. Integration of automated smoke tests with Octopus Deploy supports continuous integration and deployment, shortening the time to release new versions. A large infrastructure transformation program is underway and the OpenText Application Quality Management of Excellence aims for all associated application testing to be automated and in a self-service model, through the use of OpenText solutions.

Results

Soisson comments on the results: “In the first year, we were able to demonstrate a saving of over 1,500 man hours, leveraging OpenText Functional Testing. This meant we could focus our team efforts on value-add projects for our internal customers.”

OpenText Application Quality Manager, he notes, has been especially helpful: “We were totally dependent on spreadsheets prior to buying OpenText Application Quality Manager—it was a lot of manual work opening every spreadsheet to compile metrics across our enterprise. OpenText Application Quality Manager took us to the next level—standard metrics are included and you can design whatever else you can imagine.”

OpenText Enterprise Performance Engineering has been so successful in helping the team test and ensure the performance of TMNAS core systems and other applications, that it’s now an integral part of the company’s standard operating procedure. Anugonda explains: “OpenText Enterprise Performance Engineering makes it easy to benchmark the performance of your software, then rerun the same scripts after each deployment to make sure any deltas are acceptable—we’ve built an entire process around that and it’s now an integral part of our OpenText Application Quality Management of Excellence.

The TMNAS team feels ready to support a move towards DevOps which is a roadmap item. Soisson: “With OpenText solutions, we have built a foundation to introduce a continuous integration testing and deployment cycle in the near future. Aspects of this are in place and we have already shortened the time it takes to release applications, and have reduced our test execution time by an incredible 80 percent.”

He concludes: “We are very proud of the OpenText Application Quality Management of Excellence and the trust our employees have placed in us. The partnership with OpenText was key to its success. They listened, understood our long-term vision, and worked closely with us to help us achieve our goals as a business.

In the first year, we were able to demonstrate a saving of over 1,500 man hours, leveraging OpenText Functional Testing. This meant we could focus our team efforts on value-add projects for our internal customers.

Todd Soisson
IT Vice President, Digital & AI, TMNA Services

About TMNA Services

TMNAS image Logo

TMNAS offers the security of nearby expertise, enhanced by the diversity and power of one of the world’s most respected insurance groups. Tokio Marine’s companies offer access to leading commercial insurance solutions spanning the property and casualty landscape including professional liability, workers’ compensation, and property coverage.

How can we help?