Case Study

Background

The client provides training, nutrition, and physical therapy programs by a team of specialists. As part of their program, they utilize software that integrates with workout machines to provide the user with recommended training exercises based on previous workouts, weekly workout challenges, and member goals.

The client is looking to implement a functional test automation framework for their application in order to perform regression testing as new builds are released.

Challenges

The functional test automation framework must support the Google Chrome web browser. The framework must also be implemented in such a way that script maintenance be kept at a minimal and current employees will be able to continue creating scripts once the contract for the RTTS resource comes to an end.

Strategy

Selenium WebDriver was used to automate business transactions. RTTS also took advantage of the page object design pattern to help minimize the maintenance required once an application undergoes UI changes. Private training was also conducted to provide the client’s employees the means to create new test scripts.

Solution

The implemented automation framework is 100% open source and the components are as follows:

• Eclipse
• Java
• Selenium 2
• JUnit
• Apache Ant
• Subversion

Once the framework was in place, the page object design pattern was utilized in which classes were created for each page in the application. The page object classes provided an interface for the testers to interact with each page.

Test scripts were then created by calling methods from the page object classes that performed specific transactions, such as login as registered user, create a new user, create a workout, etc. All work was committed to a Subversion repository for version controlling.

As Selenium WebDriver lacks any built-in logging mechanism, a custom solution was used to record logs into an excel file using the open source java api JExcel. Each commands and verifications performed were logged. This provided a detailed view of where the test script had failed and the steps performed that caused the failure.

Once the framework was in place and several test scripts created, training was provided to the client’s employees on the usage of Selenium 2.

After training was provided, the client’s QA testers began creating test scripts while continuous support was provided for any issues that were encountered.

During scripting, most of the issues encountered were due to heavy AJAX usage in the application. Since AJAX would only update a section of the application instead of the entire page, the Selenium WebDriver test script was executing commands faster than AJAX was updating the UI. The ExpectedConditions class, part of the Selenium package, was used to wait for certain conditions to be met prior to executing the next Selenium WebDriver command, such as visibility.

Management had also requested that a report be generated after executing a test suite. In order to create the report, Apache Ant was used to execute the JUnit tests and generate a JUnit report. The report displayed metrics on the number of tests that failed and passed. Reports can also be drilled down to display additional information about each failure and what caused the failure to occur.

Lastly, a local server was set up with virtual machines of different operating systems that supported different browser versions. These virtual machines will serve to provide an environment in which full regression testing will be performed on using the Selenium Grid.

Benefits

By sticking with an open source framework, there was no cost in obtaining the required components to set up the framework.

Additional savings were made by implementing the Selenium Grid. Originally, the client had opted to utilize a third party company to execute Selenium WebDriver test scripts. The cost of using the third party company came in at $149/month with limits on the amount of tests that could be executed. With the Selenium Grid, the client is now able to run Selenium WebDriver test scripts without any limits or fees.