University of Bristol
University of Bristol condenses years of manual regression testing into minutes by using ALM Octane automation software
Products and services
Solution group
Outcomes
- Uses automation to reduce four man-weeks of manual testing to just 11 minutes
- Reduces test costs by as much as 75%
- Enables nightly test runs to avoid thousands of issues at project end
- Supports Agile approach to project management
Challenge
Ensure effective regression testing during a two-year upgrade project for the university's Strategic Information Technology Systems (SITS) student management system.
In common with over 60% of the UK's higher education establishments, Bristol uses the Strategic Information Technology Systems (SITS) student management system to store, administer and manage all aspects of the student lifecycle from initial enquiry and application, right through to their degree. However, having been in operation since 1997, the university's legacy SITS application was outdated, fragmented and no longer fit for purpose. The vision was to implement a new integrated system that would support staff and students to a level that had never been possible before.
“We looked at all the alternatives and decided to replace the old SITS system with a more modern version rather than migrating to a different solution,” says University of Bristol's test manager, Nigel Summers.
A team of between 20 and 30 technicians is now 20% through the major two-year SITS replacement project called Student Lifecycle Support Program. Addressing all the student facing systems in the university is a highly complex undertaking that involves interfacing with many university systems so stringent regression testing was vital and automation was the only viable option.
“Because of the length of the program, there were going to be many phases of delivery, so it was clear that a very large amount of regression testing would be required,” explains Summers. “The alternative of doing it manually would have meant many man-years of manual testing work and would have extended the timeframe considerably. Automating the testing allows us to do it much more quickly and cheaply.”
The university had been using OpenText Quality Center (QC) and Unified Functional Testing (UFT) but as well as increased automation, it wanted to migrate to a test tool that would support its Agile approach to project management. After researching the market, a Proof of Concept confirmed that ALM Octane was the best fit. ALM Octane is an Application Lifecycle Management (ALM) tool designed to support Waterfall and Agile projects and make it easier to track and manage programs of work.
Details
Solution
To configure ALM Octane and integrate it with other tools, OpenText™ recommended its Gold Partner, Infuse, a specialist in test automation and performance engineering.
Senior consultant, Colin Lindsay explains: “The development process at the University of Bristol has always been handled in Jira so we had to bridge the gap between ALM Octane and Jira. I set up integrations that allow the defect management in ALM Octane to be mirrored into the Jira flow. I also set up an automated regression system with a Continuous Integration (CI) server using Jenkins, Git and Bitbucket linked to ALM Octane, so now we can run a nightly regression pack which reports directly into ALM Octane.”
ALM Octane is ideal for us because it's designed to support the Agile process. Octane was the easy winner in our Proof of Concept tests.
Results
Automated workflows enable the team to run regression testing every night and have brought a rapid acceleration of the process because test cycles that would have taken four man-weeks now run in just 11 minutes. It's also a considerable saving in manpower and is about one quarter of the cost because four man-weeks of testing would have had to be done every two weeks.
“The regression pack runs every day and, so far, we've found a number of defects that were caught by the Octane automation pipeline and, more than likely, would otherwise have gone unchecked,” adds Lindsay.
Continuous testing throughout the program will also pay dividends further down the line, as Summers explains: “Without automation, experience says that at the end of the program you'd have a massive manual testing task that could take up to a year with a project of his size. Typically, you could unearth a couple of thousand defects so it's essential to use continuous automated testing.”
When completed in late 2019, the Student Lifecycle Support Program will increase the number of things students can do on-line from inquiring about courses to registration and even using their mobiles to locate exam rooms and the actual desk they will sit at. It will also revolutionize the efficiency of staff who have previously relied on paper-based systems.
A very large amount of regression testing was required and doing it manually would have extended the timeframe considerably. Automated testing with ALM Octane allows us to do a lot of testing much more quickly and cheaply.
About University of Bristol
With 18,000 undergraduate students and 2,100 postgraduates, the University of Bristol offers over 200 degree courses and is one of the UK's most successful universities. Ranked among the top 50 in the QS World University Rankings 2018, it was formed in 1876 and with nearly 7,000 staff, it has 40 Fellows of the Royal Society and 15 Fellows of the British Academy.
