eValid™ -- Automated Web Quality Solution Browser-Based, Client-Side, Functional Testing & Validation, Load & Performance Tuning, Page Timing, Website Analysis, and Rich Internet Application Monitoring.

Introduction, Caveats
Nothing's perfect, but eValid is close to it. Even so, there are several "gotcha's" that act like potholes in the road to a smoothly running functional testing, regression testing, performance monitoring, or server loading project.

Underlying Principles, Goals
There are millions of different ways to implement web applications. What's common among them is that they are viewed with a browser. This means that the main events that affect how to test them are rooted in what occurs inside the browser. eValid's universality lies in the fact that at the end of the day there always is a sequence of eValid commands that tests ANY web application. We have a 100% Guarantee of this.

Known Troublesome Areas
Before you decide that there is something wrong with eValid because a recording you make FAILs on the first playback, please consider the possibility that this might be due to "pilot error."

Here are known troublesome areas with descriptions of the solutions:

• Non-Determinism In General
  The majority of difficulties that eValid faces fall into the general area of a user attempting to do something deterministic in a situation which is inherently non-deterministic. I.e. the state transition is essentially random.

  The good news about eValid, that it is a very stateful simulation of user interactions, is also often bad news: it is too stateful! Sometimes eValid's statefulness gets in the way of what you want to do.

• Login/Logout With Session Cookies
  If the recording logs in and does not log out, and the logged-in state is kept in a session cookie, then the playback will fail (you're already logged in). To clear session cookies you need to restart eValid to reset the internal states. Batch mode runs often work when foreground/interactive mode runs fail.

• Modal Dialogs, Modal Popups
  Modal dialogs take over control of your recording session and prevent eValid from recording your inputs/responses. Here is a collection of hints about how to handle Modal Dialog Boxes.

• Drag/Drop Operations Use Absolute Coordinates
  If you have to use it, eValid has a drag/drop capability (see the Settings) that necessarily uses absolute coordinate operations. It's also worth remembering that, if you drag somewhere and the browser remembers it, so when you do it again, your drag will be additive to what you already did. This is non-deterministic behavior and defeats repeatability.

• Stateful Operation In General
  Not only do Login/logout operations are not the only way eValid manages a test session state. Here are two key resources:
  • Initial Conditions deals with correctly setting up initial conditions.
  • Managing State deals with more general approaches to handling operations with stateful sessions replication.

• AJAX Synchronization
  AJAX (Asynchronous JavaScript & XML) applications are asynchronous -- by definition. Real time recordings don't play back reliably because the very strong page download syncs aren't able to also account for what the AJAX engine is doing in the background. Syncing with Waits is generally not reliable. eValid provides a range of methods to help:
  • AJAX Support provides a general description of how to synchronize AJAX applications.
  • Sync Mode Summary summarizes available synchronization techniques.
  • Synchronization Settings shows how to use eValid's Synchronization settings.

• Idempotent Tests
  "Idempotent" is a descriptor of tests don't alter the state of the test object by having the test return it to the original starting state. A simple rule is, if your test adds something, be sure that the same test takes it out. The test [of the test] is: can you run the same test end to end several times without failure.

• Recording AJAX Menu Manipulations
  AJAX implementations use background JavaScript engine activity to help the user choose which how to navigate. For example, many AJAX applications create menu selection structures dynamically. So, it often happens that you make an incorrect recording because you haven't waited for the background activity to stop long enough for the eValid recording to have synchronized with the page that is being created behind the browser face.

• Opaque Objects
  If the PageMap shows you a property like "Object" then the odds are that recording of it will require use of Application Mode commands. Flash, Applets and ActiveX controls are three examples of "Object" elements that require use of Application Mode.

• Unusal Link Composition Structures
  Pages that are rendered on manually-composed HTML, or which are generated in a non-standard way. For example, we have seen cases with a simple anchor tag has extra whitespace (blanks, tabs, newlines) for "formatting purposes," which cause recorded script commands to fail at playback time. Non-standard HTML layouts are not supported, but may work normally in some cases.

• Unusal Page Composition Structures
  Pages with unusual (non-conventional) internal structure will prevent eValid from correctly recording URL or FramePath information. We have seen cases where apparent violation of the "same origin policy" prevents eValid from correctly identifying the actual FramePath. Or, we have seen situation in which the frame structure in a page changes dynamically (sometimes due to AJAX implementations), such that information that is true at record time is invalid a few seconds later, resulting in a recording whose playback effect is page-response-time dependent. In all such cases manual modification of the recorded script overcomes technical limits and temporal variations.

• Null Redirections
  Certain redirections to null objects (e.g. "About: Blank") may sometimes confuse eValid's native playback synchronziation mode. On the other hand, a site that performs a redirection to such a page doesn't make a great deal of practical sense. In this case it may be necessary to add a NO_NAV parameter to the GotoLink command and resort to an auxiliary form of synchronization.

• Quotes in URLs
  If the visible text in an anchor tag contains a " (quote) character and the total length of the visible text plus the total length of the actual URL exceeeds 110 characters, the Windows string processing utilities will strip escape characters from the "/s [this appears to a known Windows dialog box processing error] and the extra "'s will cause a syntax error on playback. The solution is to manually escape the "'s with \'s.

• Non-Deterministic Application Component
  Flight simulator or other games fail every time because there is an internal simulation engine that can't be synced with the test playback accurately enough. Even a 1 msec error every second will pile up into a big drift in the internal state of the game, and eventually your game crashes. This is a well known limitation in automated testing technology, going back many, many years.

• Unusual AJAX Constructions
  We have seen some very skillful AJAX-like web pages that use some direct-HTML-access tricks to bypass the DOM entirely. An example is one popular email program, for which the only possible way in eValid to click the "Delete" button is with a desktop-face click (an absolute click). Even though the rest of the application is done in very clever JavaScript this one button has a mind of its own and declinces to accept an event fed to it from the browser face.

• Movies, Videos
  In most cases, from the browser's perspective, only the starting image (before Play) or the last image is constant enough to be recorded accurately enough for reproducibility.

  You can record pause/play/resume clicks and the playback will work but the point in the playback imagery won't be reproducibly checkable. eValid's time resolution of 1 msec is insufficient to stop an image at a particular frame. Frame by frame moving forward is perfectly reproducible, however, if the player provides that option.