eValid™ -- Automated Web Quality Solution Browser-Based, Client-Side, Functional Testing & Validation, Load & Performance Tuning, Page Timing, Website Analysis, and Rich Internet Application Monitoring.

Summary Of Experiment
Using eValid SetUserAgent capability gives a user the ability to test large numbers of mobile devices from a single platform.

In the experiment we chose a set of 100 mobile devices and, after appropriate setup by eValid, had them examine the starting page for a well known website www.amazon.com.

For each such mobile device page load we used eValid's internal performance checking capability to save the size of the page that was downloaded, and the time it took the server to finish the download.

Results Generated
Here is a page that shows screenshots that results from running the script. You will see that for each sample screen we've shown the actual User Agent used to generate the screen. The actual run took about 30 minutes to complete, and auto-generated these screenshots.

The size of the material downloads, and the download time (and thus the download rate), varied considerably across the 100-device run, as shown in this Results Chart. (Note that the target page, when downloaded into the eValid browser, a clone of IE, forms the benchmark of comparison shown in the chart.)

The download times and data volumes included some surprizes, too. The page sizes delivered ranged from 8,071 bytes to 569,186 bytes (a 70.5:1 ratio) and the download times ranged from 0.947 sec to 3.466 sec (a 3.6:1 ratio). These are wider ranges than we had expected, considering that the target website is certainly served by some very powerful machines.

Script Parameters
The script below employs eValid's Data Synthesis and Environment Variables capability to parameterize this script so that one test run can examine how 100's (or 1,000's) of mobile devices behave. The script is parameterized in a number of ways, shown in this table.

Parameter	Data Source	Explanation
$PAGE	Environment Variables	Web page to be viewed in multiple devices
$RESULTS	Environment Variables	The file where SaveRecord data is written
$DEVICE	Data Synthesis	Name of the device to be simulated with special UserAgentString setting
$UAS	Data Synthesis	UserAgentString for $DEVICE

The values for the $UAS parameter are shown in the Device Definition file, where the string definition uses %20 in place of blanks. Note that the devices are arranged so that the most commonly used devices appear at the top of the list.

Script
Here is the script used to run the multiple-device performance test (the parameters are shown here in bold).

To allow eValid to take on the identify of a non-IE device requires using a special Start Page that allows a sub-window be launched after assigning the appropriate UserAgentString,

# Recording by eValid V9 Build #308 # Copyright (c) 2012 by Software Research, Inc. # Recording made on: Microsoft Windows Vista Service Pack 1 ProjectID "LoadTests" GroupID "work" TestID "mobileUAS" LogID "AUTO" ScreenSize 1280 1024 FontSize 0 DeleteCache DeleteCookies # Pick up the local parameters... EnvironmentVariableFile "deviceset.txt" DataSynthesisFile "100devices.txt" # Start here as IE... InitLink "http://www.e-Valid.com/Products/Documentation.9/Mobile/" \ "Multiple.Devices/start.html" Wait 2000 # Set up being some other device... SetUserAgent "$UAS" Wait 1500 IndexSet 0 IndexFindElement 0 DOWN "innerText" "Start Page" "" ValueSet "_window_$DEVICE" ValuePutElement 0 "target" "" IndexFollowLink 0 "" Wait 1000 # Set up the timing step... ResetTimer SaveRecord $RESULTS.txt "-" GotoLink 1 "$PAGE" "" SaveRecord $RESULTS.txt "$DEVICE\tDOWNLOADED (Bytes): %k\tTIME (msec): %e\t" MessageWindow "DEVICE: $DEVICE ****>> USERAGENT: $UAS" 0 200 500 1000 150 SaveWindowFace 1 "$DEVICE" Wait 2000 SubWindowCloseAll # End of script.