Case Study

Background

A global investments solutions provider is developing an application that can deliver real-time and historical financial data to their trading desks. Portfolio Managers and Traders will use this data to map out their trading strategies on a variety of assets. An Agile development approach was adopted to satisfy the user’s request for new functionality in a timely fashion. The software testing effort will need to preserve quality while also adapting to the quick turnaround time. A new release is deployed to production every three weeks. Since the traders are using this application to make decisions about their assets under management it is crucial that releases are delivered on time and without issue.

Challenges

An application will supply critical financial information that portfolio managers will use to plan out their trading strategies. A new version of the application will be delivered to all of the trading desks every three weeks. An Agile testing solution that can handle frequent releases while also preserving software quality is needed.

Strategy

Create an automated test suite to utilize as a smoke test against new builds. Develop a test suite while working closely with business and development teams to cover new functionality and regression testing.

Solution

• QuickTest Professional (QTP) was the primary tool used to create the automated smoke test suite covering 86 different screens within the application.
• Quality Center was the main lifecycle management tool used for running both Manual and Automated test cases. Over 500 test cases (300 manual, 200 automated) were developed with input from the business to cover new functionality and regression testing.
• Defect and change requests were managed in PIVOTAL. This made it difficult for stakeholders to connect defects & change requests to test case coverage. RTTS was able to create custom reports that would link Quality Center cases with PIVOTAL items.

When dealing with Agile software development the testing team needs to adapt quickly. Builds can be deployed multiple times a day and change requests are frequently updated. It was determined that a basic smoke test set was needed to quickly identify any obvious impacts to the application on a build by build basis.

RTTS utilized QuickTest Professional (QTP) to build an automation suite that could test each screen in the application. We were able to cover each of the 86 different screens with verifications for basic functionality to serve as the smoke test. This greatly improved the time it took to find broad impact defects within each new build.

In addition, we were able to use the smoke test set framework to expand our functional test scenarios. Setting up a common Object Repository and a Function Library allowed us to re-use navigation and verification calls from existing cases. This allowed us to develop new test cases quickly which helped in meeting our tight deadlines.

Completing the automated test suite runs in a timely manner was also a concern for the client. In Quality Center we were able to configure multiple host machines and assign them to a host group. In coordination with the client’s change management team we set up the smoke test suite to automatically run each time a new build was deployed. Using multiple host machines allowed us to reduce our execution time to under an hour.

Test Suites were managed on a release by release basis in Quality Center. Testing coverage was reviewed with both the business and the dev team on a daily scrum call. Part of the challenge involved linking the tools used by the different teams to accurately display testing progress and coverage to the stakeholders. RTTS was able to create custom reports linking the Pivotal data with the test cases in Quality Center. These reports helped to identify current build health and testing needs. The development team found these reports useful when prioritizing their workload in order to meet release deadlines.

Benefits

• Introduced an automated smoke test suite which allowed for rapid health analysis of new builds. Reduced the time it took to smoke test a new build from 8 hours (manual) to 1 hour (automated).
• Created a flexible automation framework that allowed for speedy development of additional functional test scripts.
• The QA team developed custom reports providing stakeholders with a snapshot of the current defect counts and testing coverage. This allowed the development team to accurately asses their workloads in relation to the release date.
• QA team performed post release testing in order to verify production was functioning as expected and ready for use the next trading day.