Case Study

Background

A company in the medical industry has a .net application which is responsible for protocol information management. Test scenarios for this application can be quite complex as the system contains a multitude of conditional options that are triggered based on the data selections from various fields in the standard workflow. The company would like to implement an automated testing framework using the Visual Studio 2010 Coded UI testing tool. The goal is to develop a framework that is reusable and flexible enough to handle various workflow scenarios.

Challenges

How can automation be used to improve efficiency and expand coverage for a complicated protocol information management system?

Our Strategy

Develop an automation framework that can handle the testing workflow but is also flexible enough to build upon. Mentoring will be provided to ensure the client is comfortable with the solution and that they will be able to expand on the work done by RTTS.

Solution

• RTTS created an automation framework based on one test flow that involved multiple UI window forms and different users.
• While developing the automation framework we worked closely with the client’s QA team to provide guidance on best practices.
• A data-driven testing approach was taken to allow the reuse of scripts for multiple scenarios with varying input data.
• Automation scripts were modularized to allow for flexibility in the future implementation of new solutions.

The client was aware of a need to automate their testing process due to the number of different test scenarios and the limited amount of resources available. Prior to this engagement the majority of testing for this .net application was done manually. Certain test scenarios could take as much as a week to complete with a single resource. Due to a tight release schedule their QA team was not able to get full coverage on a number of different testing scenarios.

RTTS was brought on to help with the implementation of an automated solution and to provide some mentoring as to the best approaches when creating an automated testing framework. 

We decided to focus on automating one particular workflow to start. It was one of the more complex flows as there were a large amount of possible test scenarios due to the number of available fields the user can select. The entire flow involved 5 different checklist forms from 3 different users. The answers for these forms were then displayed in a window which could be accessed by an administrative user. The goal would be to automate the entire process for a single workflow then test additional scenarios by changing the user’s selections for each form. 

The automation framework was created using Microsoft’s Visual Studio 2010 coded UI testing tool and written in C#. The approach was to break down the overall scenario into smaller parts that would be handled by separate coded UI test scripts. Separate UI Maps were created to work with the scripts for each of the different forms encountered on the UI. This will allow the scripts to be used for more than one workflow in the future. The separate scripts were pulled together and run by driver script. We also created standard automation scripts that can be reused for most of their other workflows such as a universal login script and database access scripts. 

All of the expected values were stored in a database table based on a test case ID number. The same script was used to test multiple scenarios based upon the input data provided from the table. Results were also stored within database table for each of the verification points. 

The same techniques used to build to this workflow scenario can be applied when automating other processes within their testing. Throughout the length of the project RTTS provided mentoring for using the Coded UI testing tool. By the end of the engagement we were able to get a complete run for the single scenario as well as additional scenarios based on varying input data for different users. The client felt comfortable going forward in their ability to build upon the framework put in place by RTTS.

Benefits

• Certain workflow scenarios could take as long as 1 week to manually test with a single resource. The automated scripts reduced end-to-end testing time to about 2 hours for that same scenario.
• A defect was discovered on the first run of an alternate workflow scenario. This highlighted the advantage of the new automation as it was able to execute a scenario find a defect which was missed during manual testing.
• The client QA team will be able to model additional workflows using the framework created for this scenario.