eValid™ -- Automated Web Quality Solution Browser-Based, Client-Side, Functional Testing & Validation, Load & Performance Tuning, Page Timing, Website Analysis, and Rich Internet Application Monitoring.

Overview
eV.Manager is a test suite manager specifically aimed at making it very easy and intuitive to handle suites of eValid tests. eV.Manager's target test suite size is ~1,000 scripts. eV.Manager follows all eValid conventions but is a free-standing application that runs eValid scripts by making system calls to eValid each time a script is to be played back.

Test Suite Structure
A test suite's file structure is organized around a Project Name [ProjectID] folder that identifies the name of the project. In each project there are one or more Group Names [GroupID] folders. Within each Group Name folder there are individual Tests, each with a unique Test Name [TestID].

Suite Folder Structure
A typical 1000 test suite folder would probably be organized into ~25-100 group folders of ~4-10 tests per group. It is best if a test suite is organized so that the topmost folder is the Project Name, its folders are the Group Names, and within each Group Name folder individual test files are named after the Test Names in the scripts. This makes forward and backward cross-referencing easier.

Sub-Suite Execution Selection
Both the complete test tree (all available tests) and the active test tree (the subset of tests that is to be run) are shown in the eV.Manager GUI. The user selects the entire test suite, one or more groups of tests, or one or more tests within groups. Tests can be added to or deleted from the active test tree from the complete test tree with the eV.Manager GUI.

Suite Execution
On user command, eV.Manager runs the selected tests by invoking eValid with the correct script name. eV.Manager then waits for test playback completion and PASS/FAIL determination, and then saves the results internally. eV.Manager can repeat a selected set of test runs a user-specified number of times.

Results Reporting
eV.Manager accumulates log files that contain playback results (PASS/FAIL data, return codes, runtime warnings/errors).

The Test Report shows a summary of the complete current test information, including data on the most recent tests run. It shows the current PASS/FAIL percentage.

The Regression Report pinpoints any tests that "regress," i.e. previously had PASSed and now FAIL or previously FAILed and now PASS.

Filenames
eV.Manager uses these files and filename extensions:

• <suite>.evm Tabbed user-input file.
• <suite>.ats Internal (but user-editable in an emergency) Automated Test Script file.
• <suite>.rep Internal eV.Manager log file.
• <suite>.report.html Standard Report.
• <suite>.output.html Saved output.

Automatic Test Suite Extraction
eV.Manager can automatically create a test tree from the existing file system structure. To do this the user selects a project folder using the File > New selection. eV.Manager creates the tree and asks you for the location at which to save the generated .evm file. The test tree is now ready for use.

Test Check-In & Check-Out Process
To add a new eValid test script to the current test suite:

1. Add the test script in the appropriate folder in the current project.
   (Note: This may not be necessary as eValid saves script files in this criteria)
2. Add the name of the script in the eV.Manager test tree setup editor.
3. Click Save and eValid will refresh the tree including the newly added test.
4. Run the new test alone to re-initialize the baseline data if necessary.

Alternatively, the user can:

1. Use any text editor to modify the tabbed user-input *.evm file. Add the Group Name (if necessary) and the Test Name in the appropriate place.
2. From within eV.Manager open the .evm file using the File > Open option.
3. Run the new test alone to re-initialize the baseline data if necessary.

To delete a test from the current test suite follow the above process but delete the test rather than add it.

Batch Mode, Unattended Operation
eV.Manager supports batch mode operation for use in unattended test suite executions. The eV.Manager Command Line Interface for eV.Manager permits you to specify the particular *.evm file to run, the total number of times you wish to have the suite to run, and whether you want eV.Manager to cease operation after completing all of the specified tests.

Image Difference Utility
If your tests involved detailed manipulation of images, you can use the Image Difference Utility to compare old and new images, to determine differences between images, and to compare images with masks.