Case Study

Background

A major electronics retailer/eTailer offers a variety of services through its consumer website, including product information, retail locations for their products and online sales and support. In order to improve its online presence, the company relies on a third-party web tracker to obtain data about where its customers are searching, how much time they spend on each page, how long it takes them to fill out web forms, how many visitors are repeat visitors, and more. As part of the ongoing development of this website, the software quality effort’s goals are to verify the functionality of the site with each new build as well as the accuracy of data generated by the third-party web tracking software.

Challenges

How can the software quality effort for an eCommerce website efficiently verify application data to meet ongoing deployment schedules? Specifically for the development team, how can test automation be used to verify the accuracy of a web tracking tool?

Strategy

RTTS’ strategy was to implement an automated test suite using IBM Rational Functional Tester (RFT) with the following capabilities:

• Verify the consistency of the website’s data while switching between the ​“buy”, ​“product support” and ​“information” sections of the site using both a specific data set and a random one.
• Utilize Windows APIs to find and verify data in the tracking tool, delivered as a browser add-on.

Solution

RTTS implemented an automated test solution with IBM Rational Functional Tester to:

• Dynamically verify data integrity when navigating through the ​“buy”, ​“product support”, and ​“information” sections of the site
• Identify key HTML tables and the links within those tables flexibly, regardless of dynamic data content
• Identify the tracking ​“tags” being sent to the tracking service

Dynamic Navigation Across the Web Application

In order to build dynamic navigation into the automation suite, RTTS implemented an automation strategy that takes advantage of one of RFT’s power features: the ability to ​“pattern-match” object recognition properties. With pattern matching, RFT recognizes objects by matching a pattern, instead of matching a static, hard-coded value. This approach allows RFT to navigate the electronic store’s pages dynamically and flexibly, based on a small number of generic patterns, instead of a large number of hard-coded recognition data values. For example, by making the recognition for a product table generic by using a pattern, RFT could identify similar tables in multiple product categories without having to record and maintain each table as a new entry in the object map.

Pattern matching also removed the need to include detailed product data in the object map. This allowed test scripts to search each HTML table’s child objects flexibly and find links to different products identified for separate scripts, without requiring a data-heavy object map.

Data-driven Testing with RFT Datapools

Once RTTS created a pattern-based Object Map for the ​“buy”; section of the web application, data-driven testing was implemented with RFT datapools. ​“Buy” test scripts order a specific item or group of items based on RFT datapool entries, or order randomly selected items from among multiple datapool entries. The scripts then check the accuracy of the selections in the shopping cart.

Reusable Automation Code

Using the same approach, navigation was built for the ​“product support” and ​“information” sections of the site. This automation was quickly developed by extending the framework libraries built for the ​“buy” section, taking advantage of the consistent design used across all sections of the application. Switching between these sections allows for test scripts to validate data consistency between the sections, by navigating through one section and then switching to another in order to capture related data (i.e. the RFT script’s virtual consumer could navigate to a product in the ​“buy” section and then look up information on that product in the ​“information” section). These validations were then applied to the shopping cart portion of the site as well.

Automating the Web Tracking Tool

As noted above, a goal of this project, in addition to developing test automation for the website’s functionality, was to build automated tests for the site usage tracking tool. The tracking tool ​“tags” that send information to the tracking service can be captured from a browser plugin which displays the tracking data.

RTTS’ experts used the RFT API to build a simple handler for the browser plugin’s windows that could navigate and capture data from them. The handler was built to manage both IE 6 and IE 7, using regular expressions to identify window objects that use instance IDs. In the end, a method was created that would launch the plugin (if it wasn’t launched already), identify it, and return the tracking information. Through this approach the tracking ​“tags”; were successfully verified by the RFT scripts.

Benefits

• The automated framework helped increase testing coverage over time as more test scripts were built over multiple development iterations.
• Test automation released team members from the burden of executing repeated manual regression test suites and allowed them to pursue other high-value tasks.
• The initial automation implementation was used as a training tool within the development group to help members new to automated testing learn how to use the tools and how to design tests.