Your e-Business Quality Partner eValid™ -- Automated Web Quality Solution
Browser-Based, Client-Side, Functional Testing & Validation,
Load & Performance Tuning, Page Timing, Website Analysis,
and Rich Internet Application Monitoring.

eValid -- Single Platform Testing -- 100 Different Mobile Devices
eValid Home

Summary Of Experiment
Using eValid SetUserAgent capability gives a user the ability to test large numbers of mobile devices from a single platform.

In the experiment we chose a set of 100 mobile devices and, after appropriate setup by eValid, had them examine the starting page for a well known website

For each such mobile device page load we used eValid's internal performance checking capability to save the size of the page that was downloaded, and the time it took the server to finish the download.

Results Generated
Here is a page that shows screenshots that results from running the script. You will see that for each sample screen we've shown the actual User Agent used to generate the screen. The actual run took about 30 minutes to complete, and auto-generated these screenshots.

The size of the material downloads, and the download time (and thus the download rate), varied considerably across the 100-device run, as shown in this Results Chart. (Note that the target page, when downloaded into the eValid browser, a clone of IE, forms the benchmark of comparison shown in the chart.)

The download times and data volumes included some surprizes, too. The page sizes delivered ranged from 8,071 bytes to 569,186 bytes (a 70.5:1 ratio) and the download times ranged from 0.947 sec to 3.466 sec (a 3.6:1 ratio). These are wider ranges than we had expected, considering that the target website is certainly served by some very powerful machines.

Script Parameters
The script below employs eValid's Data Synthesis and Environment Variables capability to parameterize this script so that one test run can examine how 100's (or 1,000's) of mobile devices behave. The script is parameterized in a number of ways, shown in this table.

Parameter Data Source Explanation
$PAGE Environment Variables Web page to be viewed in multiple devices
$RESULTS Environment Variables The file where SaveRecord data is written
$DEVICE Data Synthesis Name of the device to be simulated with special UserAgentString setting
$UAS Data Synthesis UserAgentString for $DEVICE

The values for the $UAS parameter are shown in the Device Definition file, where the string definition uses %20 in place of blanks. Note that the devices are arranged so that the most commonly used devices appear at the top of the list.

Here is the script used to run the multiple-device performance test (the parameters are shown here in bold).

To allow eValid to take on the identify of a non-IE device requires using a special Start Page that allows a sub-window be launched after assigning the appropriate UserAgentString,

# Recording by eValid V9 Build #308
# Copyright (c) 2000-2012 by Software Research, Inc. 
# Recording made on: Microsoft Windows Vista Service Pack 1 

ProjectID "LoadTests"
GroupID "work"
TestID "mobileUAS"

ScreenSize 1280 1024
FontSize 0

# Pick up the local parameters...
EnvironmentVariableFile "deviceset.txt"
DataSynthesisFile "100devices.txt"

# Start here as IE...
InitLink "" \
Wait 2000

# Set up being some other device...
SetUserAgent "$UAS"
Wait 1500
IndexSet 0
IndexFindElement 0 DOWN "innerText" "Start Page" ""
ValueSet "_window_$DEVICE"
ValuePutElement 0 "target" ""
IndexFollowLink 0 ""
Wait 1000

# Set up the timing step...
SaveRecord $RESULTS.txt "-"
GotoLink 1 "$PAGE" ""
SaveRecord $RESULTS.txt "$DEVICE\tDOWNLOADED (Bytes): %k\tTIME (msec): %e\t"
MessageWindow "DEVICE: $DEVICE ****>> USERAGENT: $UAS" 0 200 500 1000 150
SaveWindowFace 1 "$DEVICE"
Wait 2000
# End of script.