eValid™ -- Automated Web Quality Solution Browser-Based, Client-Side, Functional Testing & Validation, Load & Performance Tuning, Page Timing, Website Analysis, and Rich Internet Application Monitoring.

Synopsis
Intermittent non-repeatable field-reported failures that were having a significant impact on E-business profitability were effectively identified by running regular eValid transaction oriented monitoring tests over a period of time combined with careful analysis of the eValid playback event logs corresponding to the failures detected.

Background
Over a period of time the customer had observed server failures that caused loss of E-business transactions -- but which were not directly identifiable internally. The most common failure appeared to the website was an abandoned shopping cart, but the reason for the abandonment was note clear.

Regular human navigtion of the site failed to uncover any problems, and even recreating individual transactions "always seemed to work OK" when done manually. The common thread to the failures was that they all seemed to happen during periods of heavy load and during a user sequence in which the website had accumulated a great deal of context.

eValid Application Description
To assist in the diagnosis and repair process, eValid created three realistic E-business simulation scripts. These scripts were "deep" transactions in the sense that they performed a large number of steps, created a local context, and worked all different parts of the web server "back end" functions.

We set these scripts up in "monitoring mode" using an in-house machine connected to the web via standard DSL lines. This effectively simulated real users, over a typical last mile connection.

Results Achieved
Here is a summary of the results achieved after several weeks of continuous monitoring with these transaction scripts. All of these observations were made by careful study of the "red screen" event logs.

1. Abandoned shopping carts are not necessarily dangerous, but initial eValid tests revealed that the website's assumptions did not include a specific provision for closing "timed out" shopping carts. This led to clogs in the processing in the shopping cart database processor.
2. Many of the "red screen" tests resulted from the website search engine timing out, or exceeding the permitted maximum download time (60 seconds).
3. Several pages caused a sequence of page redirections to occur, although not on a consistent basis.
4. Some of the recorded navigate commands were not completing within the required time limits, evidently held up at the server side for some reason.
5. We found that the "shipping cost" step in the E-business transaction would often require so much time that the maximum time thresholds that were set were exceeded.
6. We noted that particular FORM-submit actions failed to complete in the maximum time limits set for them.

All of the events noted were detected essentially "at random" because the majority of the tests, 10/hour, 24x7, or 1,680/week PASSed without incident.

Results Obtained
The net result of this forensic event log analysis work was to identify, from the evidence collected, a number of relatively simple required changes in the server front-end and back-end processing activity. After changes to the server complex were completed, over a period of several weeks, the rate of "red screen" playbacks dropped significantly.

The overall result of these improvements was a significant increase in sales for the company due to avoidance of loss of "in process" sales, and improved customer satisfaction due to user reliability enhancements.