Enables testing of all functions on hundreds of different mobile devices
Delivers test environments to remote teams at least ten times faster than before
Reduces testing costs significantly while boosting quality and cutting time to market
Enables widescale automation of testing for improved speed and efficiency

Challenge

Ensure comprehensive mobile app testing capabilities across multiple devices – without requiring testers to have physical access to devices.

Details

Extending testing to more devices

Banking customers worldwide love the convenience and speed of managing their finance on their mobile devices – and Brazilians are no exception. The mobile channel at Santander Brazil continues to grow rapidly, and the bank offers five mobile apps. These include the Santander app, which empowers individuals to do practically everything they could in a branch, and the Way app, which aims to provide the best experience for managing credit and debit cards, payments and transfers. The bank also offers apps for business and corporate account holders.

Compared with many international markets – for example, the US, where Apple and Samsung currently share 79% of the market – Brazil has a great diversity of mobile devices, and many people have two different handsets. This diversity was making it difficult for the bank to test its apps. With many customers relying on mobile apps as their primary point of contact with Santander Brazil, the bank wanted to ensure it could test against the largest possible number of devices.

Felipe Inocencio Fabricio, IT Lead, Santander Brazil, explains: “We need to be sure that our apps work reliably across the vast majority of all devices that are used in Brazil, because user experience is a key competitive differentiator in a mobile-first world. And to do this, it’s vital to have access to all elements of each physical device; simulating a device in software does not cover every possible usage scenario.”

To avoid the cost and complexity of owning and running its own mobile device testing center, Santander Brazil was using AWS Device Farm to test its app software on real physical mobiles hosted in an AWS data center. To run their tests, QA teams had to request the relevant device and wait for the cloud service to provision it.

“It took between three and five minutes to get a device ready for testing, which was causing delays in our process and limiting the number of tests we could complete,” says Diego Sanchez Pereira, VP T&O, Santander Brazil. “AWS Device Farm didn’t give testers full remote access to all physical handset components, so it was impossible to reliably test app functions involving SMS, GPS, camera or fingerprint/face unlock functions. However, the really pressing issue was the cost: if a team needed to run multiple tests to fix a bug, the service fees would mount up very rapidly. Worst of all, the fees were charged in USD, and a massive shift in the exchange rate with BRL effectively doubled our costs last year.”

OpenText made it possible for us to have all the benefits of hands-on physical testing on mobile devices without the disadvantages.

Felipe Inocencio Fabricio
IT Lead, Santander Brazil

Saving time and cutting costs

Following a successful proof-of-concept (POC) exercise, Santander Brazil replaced its existing solution with OpenText™ Functional Testing Lab for Mobile and Web testing software for around 200 users, and a mobile device farm hosted by Keeggo.

Keeggo hosts Santander Brazil’s chosen mobile devices in racks in its Tier3 Brazilian data center, giving the bank access to the devices and OpenText Functional Testing Lab for Mobile and Web as a managed service through a secure private network. A dashboard enables testers to view metrics and track results from their tests, helping them to make better data driven decisions.

“The POC demonstrated huge improvements in performance and cost-efficiency,” says Felipe Inocencio Fabricio. “We signed up for a one-year contract initially, and we’ve just extended that for a further three years.”

Starting with 50 mobile devices, Santander Brazil has grown its environment to 140 devices; it is expected that the bank will soon expand to 200 or even 300 devices.

“We can test all aspects of every device remotely and without significant latency from any of our three sites – one of which is 200 km away,” says Felipe Inocencio Fabricio.

To help Santander Brazil manage the available resources, Keeggo recently created an optimized reservation system that identifies under-utilized devices – for example, when a QA tester has requested a device but not used it after 15 minutes – and returns them to the pool for other testers to access.

Achieving higher quality and faster delivery

The combination of OpenText™ Functional Testing Lab for Mobile and Web and hosted devices has transformed Santander Brazil’s mobile app testing. The solution provides full access to all the functionality of every handset, making it possible for QA teams to test every function accurately – which ultimately translates into higher quality and a better experience for end users. In addition, OpenText Functional Testing Lab for Mobile and Web has improved the user experience for Santander Brazil’s Dev and QA teams, making their work more efficient.

“A key benefit of the solution is that we can test the full range of handset functionality,” says Diego Sanches Pereira. “For example, we can simulate the use of QR codes and fingerprint or face-recognition security features on any handsets that support them. This was an important differentiator in selecting the OpenText solution, as it enabled us to automate all aspects of testing our mobile apps.”

The solution offers lower fees than the AWS solution it replaced, and pricing in BRL rather than USD means that Santander Brazil is no longer exposed to swings in the exchange rate.

Speed of access to testing resources is another major benefit of the solution, as Diego Sanches Pereira explains: “Our testers now get access to the target mobile device in less than 20 seconds compared with at least 3 minutes previously. This also boosts overall quality because we can run more tests in the available time.”

As the Brazilian mobile banking landscape continues to evolve, Santander Brazil’s flexible contract enables it to keep refreshing the physical devices in the testing farm. Felipe Inocencio Fabricio concludes: “We look forward to ongoing success with the solution, which gives us the flexibility we need, combined with easy scalability and hybrid cloud options. Keeggo also provides committed and skilled local implementation and support services in our own language.”

OpenText Functional Testing Lab for Mobile and Webでは、テストを自動化し、複数のデバイスで並行して実行できるため、テストの速度も向上します。これにより、新しいリリースの市場投入までの時間を短縮することができます。

Felipe Inocencio Fabricio
IT Lead, Santander Brazil

About Santander Brazil

Established in 1982, Santander Brazil is the country’s third largest privately held bank by assets and an autonomous subsidiary of Santander – the largest bank in the eurozone and one of the global leaders. Santander Brazil is present throughout the country, serving individuals, SMEs and large enterprises with a full range of banking and treasury products.