Case Study

Background

The client creates software for municipal, city, and county districts of north-east Texas that enables the courthouses of those jurisdictions to enter, update, and track the progress of legal cases, their judgments, and the fines attendant upon these judgments.

One of the larger jurisdictions (“the assessor”) in the area wished to move their legacy system data to a customized system designed by the client. The assessor wished to understand the performance characteristics of the system under specific data loads and user populations.

Challenges

How can a service provider company show potential clients that its solution can scale to the case load (in terms of both database size and projected clerk population) of a large government law enforcement angency? 

RTTS acted as an unbiased third-party to conduct a series of empirical studies to determine the performance characteristics of the client’s custom-built system for law enforcement case management in order to demonstrate a good fit with the assessor’s legal systems, case data, and projected needs in terms of both population and case load data.

Strategy

After investigating the issues involved, RTTS decided that our standard process and practices for performance testing would accurately determine the workload necessary to properly exercise the PLM solution.

Solution

After extensive requirements capture sessions with the assessor and the client and a brief survey of the system under test, RTTS wrote a test plan. Leveraging the benefits of their partnership with IBM, the client decided to use IBM Rational Performance Tester (“RPT”), a tool that allows users to capture business transactions, automate and modify these transactions, create synthetic workloads based on population size and roles, and measure the effects of these workloads on production-ready systems.

RTTS worked with the domain subject matter experts to determine the most common actions performed on the existing system, automate them, and generalize them for broad use across many user roles. RTTS determined courthouse load based on current statistics and crafted a network workload to reflect this.

Working with the client, RTTS applied this workload in 54 experiments against up to a 450 virtual courthouse employees and a database of up to five million legal cases, collecting data to determine the performance characteristics at the transaction and system level.

Using in-house tools and expertise, RTTS summarized its findings and generated production-level tables and charts to facilitate discussion between the assessors and the clients. At the assessors’ request, RTTS created a spreadsheet to interpolate the results as far as possible for heavier and lighter workloads.

During discussions of the data, RTTS acted to help interpret any questions regarding the test and related data.

Benefits

After investigating the network traffic generated by several transactions, RTTS noted that specific transactions were transmitting much more data than was required for the needs of the specific transaction, and worked with the client to streamline the protocol traffic for that transaction by several orders of magnitude.

During the iterative process of adapting and generalizing the automated business transactions for use across many different kinds of court procedures, legal cases, and payment systems, RTTS determined that case searches were returning too much data to workstations connected to the system.

All of these findings were caught during initial studies of the system, allowing the development team time to pivot and to address these concerns before the performance studies were to be conducted. Despite the changes in the protocol and the system, RTTS had automated the transactions in such a way that handling these changes required minor changes of a few minutes, rather than the beginning of a new performance automation development iteration.

RTTS assisted the client in running several pilot studies on the system to provide feedback to the database administrators so appropriate optimizations could be made on the database management systems.

With the quality and speed of the feedback, the systems team was able to resolve a number of performance and functional issues before release. The end result was that the formal empirical performance study of the system showed local sub-second responses in all but the most data-laden high-usage conditions. RTTS provided a detailed study of the performance characteristics of the client system to the assessor team and the client for discussion.