Handling the Dirty Work of Dynamic Content

One reason performance and scalability testing can be so challenging these days is that the content to be tested is so dynamic and ever-changing. It's a dirty job, but somebody's gotta do it - for a test to be realistic and effective it can't be done the "old fashioned" way, just by recording a single path through the application. It has to be automatically generated from the content itself, and that's what we're working on right now.

There are two types of dynamic content, server-side and client-side. An example of server-side content is online news, which changes constantly but is stored on the server and thus can be used for creating performance tests. Client-side dynamic content comes from AJAX or web services calls and shows up only in the browser, like a list of tickets available for a sporting event or concert. This particular site "sports" a rich client interface on the front end and uses Amazon EC2 on the back end. When you click on an event, the list of available seats is totally dynamic - it isn't stored anywhere except in the browser at that moment. Like an airline flight, it can disappear in an instant if the event gets sold out.

One of the companies we are working with is dealing with server-side dynamic content (online news) and another is dealing with the client-side content described above (online tickets). In the first case, what we envision is fairly straightforward - a small script will query the server database and generate a comma delimited text file to fill in the parameters of "template sessions" on the CapCal server, then call another script that automatically uploads it and kicks off the test. This will generate thousands of unique test cases that can be run by thousands of virtual users against a staging site before moving new content or application changes into production. Not only will the most current and dynamic content be tested, it will be tested in enough combinations and at high enough load levels to flush out any bugs or bottlenecks that might be lurking undetected.

In the case of the online ticket site and the countless other similar applications, we are at work on a solution that will be quick and easy for users while not so trivial for us developers (as it should be, of course)! Basically it will be an extension to the browser add-on in the CapCal client that allows fields or links in the browser to be identified and grouped so they can be dynamically accessed at runtime by each virtual user and manipulated by a new command called "Click-On". I can't divulge the magical powers of this new command until they fully exist and have been confirmed by users. But suffice it to say that it will easily solve the seat selection problem and that's what matters most right now!

I'm especially excited about this because it is true automation, with no manual intervention at all other than writing the extraction query or identifying the dynamic fields in the browser and recording the session template. Add to that the enormous economies of scale made possible by the cloud and all of a sudden something that has never been done effectively (if at all, as in the case of dynamic client content) can suddenly be done very thoroughly and cost effectively with no manual intervention.

Simple, powerful and clean - like soap and AJAX!