Case Study

Background

ProAct Technologies Corporation provides applications for managing and delivering employee benefits programs for health and wealth. Their applications work with major ERP and HRMS systems, integrating into any portal architecture. ProAct has developed enterprise-wide employee relationship management software for Fortune 1000 firms in all major industries.

In order to secure an important sale of an HR application to a major global publisher, ProAct needed to deliver a proof-of-concept to their prospect. They had to prove their application would work for the publishing house’s specific environment before they would buy it. So testing the solution’s configuration and use for the publisher before going in to do the proof-of-concept was critical to the win.

Challenges

ProAct needed pre-production load testing done on an array of transaction types within the application; and they needed the testing done fast, but lacked the in-house expertise and manpower to do the work. ProAct turned to Rational Test Bureau partner RTTS for help. RTTS, a New York City based IT application and architecture testing firm, has distinguished itself as a top provider of automated testing for Fortune 500 corporations and had already proven itself an effective testing service on many previous engagements with ProAct. RTTS’ world-class level of testing expertise and knowledge made them a welcome candidate for participation in the Rational Test Bureau program.

Rational Test Bureau partners deliver access to Rational’s solution for functional, performance, reliability testing and runtime analysis, Rational Suite™ TestStudio®, combined with the expert testing services of our partners. Businesses and organizations using the Test Bureau don’t have to purchase, install, or learn how to implement software tests. Many companies whose software is their livelihood find the services advantageous when faced with urgent projects coupled with resource constraints, even if they have in-house testing.

ProAct was in exactly that position of having an urgent project and resource constraints when their Vice President of Professional Services, Carlo DeMarco, went to RTTS for assistance. The scope of the testing included both load and functional testing, to ensure a flawless proof-of-concept for the publishing house.

ProAct’s software for the publishing house had to meet these performance standards:

• Handle up to 500 concurrent users
• Fully load any one page within 21 seconds
• Fully load any single object on any page within 10 seconds.
• Perform up to 5,000 page requests per hour
• Editing and updating employee mailing information
• Updating and retrieving employee paycheck information,
• Retrieving against a logic expression for medical benefit information
• Accessing 401k information from an external site
• Proving the single sign-on capability via Netegrity’s SiteMinder® worked with the application

Strategy

For RTTS project lead Jon Harris, Rational Suite TestStudio was the testing tool of choice for their client because it performs functional and load testing simultaneously and allows the virtual users to do more than one thing at once, the way people use applications in real life. The tool is highly scalable and creates runtime analysis and code-coverage metrics in the background, correlating bugs to the code causing them and letting the RTTS testers know what code was exercised during testing. This meant that Harris’ RTTS team could test ProAct’s business critical application with confidence in their ability to find any bugs that could keep ProAct from winning its sales engagement with the publishing house.

Solution

The tests simulated users logging-into the system and executing the various types of representative transactions. According to RTTS’ Harris, Rational Suite TestStudio’s graphic scheduler was a powerful asset in testing the transactions under varied user loads. ​“Using Rational Suite TestStudio’s graphic scheduler, we were able to schedule users to run different transactions and different mixes of transactions to truly simulate real life,” said Harris. ​“Most other products on the market today allow each virtual user to run only one script.”

Throughout the testing process, RTTS testers recorded scripts of virtual user actions at critical points as they scaled-up in increments to 500 concurrent users. Between each test they verified that everything executed correctly, and that they had the correct volume of transactions. But then a problem occurred: something was getting in the way of scaling up to the maximum required performance level.

Thanks to the runtime analysis capability that Rational Suite TestStudio provides during testing, the RTTS testers were able to pinpoint the cause to an information-sharing problem between two third-party software components required for the proof-of-concept. The problem in the compatibility of the other vendors’ products was keeping ProAct’s application from scaling as required.

The RTTS testers solved the problem after they discovered a pattern of error messages occurring during login and after the first few pages of testing. By looking at response times and error messages, and after conducting still further tests with Rational Suite TestStudio on the third-party server software, RTTS was able to narrow the problem to the way one third-party application shared information with the required third-party server. After consulting on the project, the server software vendor issued a patch that resolved the problem, allowing the test of ProAct’s application to immediately and successfully scale and be tested at 500 concurrent users within the assigned performance constraints.

Benefits

Rational Suite TestStudio performed as promised for RTTS and RTTS enabled ProAct to be confident of their application’s performance during the proof-of-concept. ProAct in turn conducted a successful proof-of-concept, a critical accomplishment that motivated ProAct’s client to purchase the application. It was a winning situation all-around, with particular benefits for ProAct. ​“We couldn’t undertake this kind of testing on our own,” said Carlo DeMarco, Vice President, Professional Services for ProAct Technologies. ​“RTTS and the Rational Test Bureau Program saved us valuable time, manpower, and resources and allowed us to concentrate on what we do best-develop world class enterprise-wide software for employee resource management.”

“This scenario clearly illustrates the reasons we created the Rational Test Bureau,” said Bill Chandler, Rational Software’s Program Manager for the bureau. ​“Here you had ProAct, a company that realized the importance of testing, yet didn’t have adequate expertise, manpower, or infrastructure to do it themselves. Additionally, they were up against a tight deadline and delivery of the proof-of-concept was critical to their winning a large order for their products. RTTS, like all of our Test Bureau Partners, was uniquely positioned to provide timely, cost-effective testing services using Rational’s industry-leading testing platform.”

This article has been re-printed with permission from Rational Software. Copyright January 2002