Wednesday, August 17, 2011

Web Application Testing Automation

Architecture Choices

We have chosen the following architecture for our web application testing:
  • Primary test client is a Windows 2003 server running in Amazon's EC2 environment which if fully patched and has Internet Explorer 8, and the latest versions of Google Chrome and Firefox.
  • SeleniumHQ with test scripts written in PHP since it is already a core technology that we used to build the site.
  • The suite of tests developed on the primary test client will be written in such a way that they can also be copied and run from Windows and Ubuntu Linux machines located outside of the Amazon environment.
  • In the future, testing will be expanded to include IE9 and possibly additional browsers and versions.
We feel this will give us a testing process with the following qualities:
  • Comprehensive testing capability
  • Cost Effective
  • Reliable
  • Accessible by many from anywhere
Sources for Inspirations, Ideas and getting started

Selenium Server Configuration

The following was captured during setup of the Selenium Server on the primary test client...

Before installing the Selenium Server we need to install a recent version of Java from

Now download Selenium Server (2.4.0) from the Selenium Downloads page and put it into a folder where you are likely to find should you need to down the road.

Create a batch file containing the following in the same folder that you put the Selenium Server jar file into:
java -jar selenium-server-standalone-2.4.0.jar -trustAllSSLCertificates 

Create a shortcut to the above batch file in a convenient location.  When the Selenium Server is running it will occupy a command window that can be shut down using Ctrl-C when you are done with it.

Install PHP
Download the zip file from and unzip it into C:\Program Files\PHP\.
Copy php.ini-development to php.ini.
C:\Program Files\PHP>copy php.ini-development php.ini
        1 file(s) copied.

Install PEAR
Make sure you install the latest version of PEAR by first replacing the C:\PROGRA~1\PHP\PEAR\go-pear.phar file with one obtained from  Your output may not look like mine because I ran mine twice, first with an out of date version of the PEAR installer.
C:\PROGRA~1\PHP>php PEAR\go-pear.phar

Are you installing a system-wide PEAR or a local copy?
(system|local) [system] :

Below is a suggested file layout for your new PEAR installation.  To
change individual locations, type the number in front of the
directory.  Type 'all' to change all of them or simply press Enter to
accept these locations.

 1. Installation base ($prefix)                   : C:\PROGRA~1\PHP
 2. Temporary directory for processing            : C:\PROGRA~1\PHP\tmp
 3. Temporary directory for downloads             : C:\PROGRA~1\PHP\tmp
 4. Binaries directory                            : C:\PROGRA~1\PHP
 5. PHP code directory ($php_dir)                 : C:\PROGRA~1\PHP\pear
 6. Documentation directory                       : C:\PROGRA~1\PHP\docs
 7. Data directory                                : C:\PROGRA~1\PHP\data
 8. User-modifiable configuration files directory : C:\PROGRA~1\PHP\cfg
 9. Public Web Files directory                    : C:\PROGRA~1\PHP\www
10. Tests directory                               : C:\PROGRA~1\PHP\tests
11. Name of configuration file                    : C:\WINDOWS\pear.ini
12. Path to CLI php.exe                           : C:\PROGRA~1\PHP

1-12, 'all' or Enter to continue:
Beginning install...
Configuration written to C:\WINDOWS\pear.ini...
Initialized registry...
Preparing to install...
installing phar://C:/Program Files/PHP/PEAR/go-pear.phar/PEAR/go-pear-tarballs/A
installing phar://C:/Program Files/PHP/PEAR/go-pear.phar/PEAR/go-pear-tarballs/C
installing phar://C:/Program Files/PHP/PEAR/go-pear.phar/PEAR/go-pear-tarballs/P
installing phar://C:/Program Files/PHP/PEAR/go-pear.phar/PEAR/go-pear-tarballs/S
installing phar://C:/Program Files/PHP/PEAR/go-pear.phar/PEAR/go-pear-tarballs/X
install ok: channel://
install ok: channel://
install ok: channel://
install ok: channel://
install ok: channel://
PEAR: Optional feature webinstaller available (PEAR's web-based installer)
PEAR: Optional feature gtkinstaller available (PEAR's PHP-GTK-based installer)
PEAR: Optional feature gtk2installer available (PEAR's PHP-GTK2-based installer)

PEAR: To install optional features use "pear install pear/PEAR#featurename"

** WARNING! Old version found at C:\PROGRA~1\PHP, please remove it or be sure to
 use the new c:\progra~1\php\pear.bat command

The 'pear' command is now at your service at c:\progra~1\php\pear.bat

** The 'pear' command is not currently in your PATH, so you need to
** use 'c:\progra~1\php\pear.bat' until you have added
** 'C:\PROGRA~1\PHP' to your PATH environment variable.

Run it without parameters to see the available actions, try 'pear list'
to see what packages are installed, or 'pear help' for help.

For more information about PEAR, see:

Thanks for using go-pear!

For convenience, a REG file is available under C:\PROGRA~1\PHPPEAR_ENV.reg .
This file creates ENV variables for the current user.

Double-click this file to add it to the current user registry.


Double-click C:\PROGRA~1\PHPPEAR_ENV.reg as instructed above.

Install PHPUnit

Issue the 4 commands highlighted below as instructed on  Again your output may be different because what I show below is not from a virgin install. I was only able to get the beta version to install without errors, hence the "-beta" at the end of the fourth command.

C:\PROGRA~1\PHP>pear channel-discover
Adding Channel "" succeeded
Discovery of channel "" succeeded

C:\PROGRA~1\PHP>pear channel-discover
Adding Channel "" succeeded
Discovery of channel "" succeeded

C:\PROGRA~1\PHP>pear channel-discover
Adding Channel "" succeeded
Discovery of channel "" succeeded

C:\PROGRA~1\PHP>pear install phpunit/PHPUnit-beta
phpunit/PHPUnit can optionally use PHP extension "curl"
phpunit/PHPUnit can optionally use PHP extension "dbus"
pear/HTTP_Request2 can optionally use PHP extension "curl"
pear/HTTP_Request2 can optionally use PHP extension "fileinfo"
pear/HTTP_Request2 can optionally use PHP extension "openssl"
downloading PHPUnit-3.5.14.tgz ...
Starting to download PHPUnit-3.5.14.tgz (118,697 bytes)
..........................done: 118,697 bytes
downloading XML_RPC2-1.1.1.tgz ...
Starting to download XML_RPC2-1.1.1.tgz (68,431 bytes)
...done: 68,431 bytes
downloading HTTP_Request2-2.0.0RC1.tgz ...
Starting to download HTTP_Request2-2.0.0RC1.tgz (95,353 bytes)
...done: 95,353 bytes
downloading Net_URL2-0.3.1.tgz ...
Starting to download Net_URL2-0.3.1.tgz (8,488 bytes)
...done: 8,488 bytes
install ok: channel://
install ok: channel://
install ok: channel://
install ok: channel://


Install Firefox and the Selenium IDE Add-On
Install Firefox ( and the Selenium IDE add-on for it (Tools=>Add-Ons, search for "Selenium IDE").  Version can be an issue.  You need to ensure that the end result is a version of Selenium IDE that supports exporting to PHPUnit.

Running Tests
The following file is named IWA_Test.php which is located in the "H:\Program Files\PHP\" directory and was created largely by collecting and editing code snippets recorded in the Firefox Selenium IDE add-on.  This example runs tests on Chrome, Firefox and IE7.  Make sure you start the  Selenium Server before running the tests.  Run the tests in a command session from the above mentioned directory by typing "phpunit IWA_Test.php".


// Run from DOS prompt using:  phpunit IWA_Test.php //

  require_once 'PHPUnit\Extensions\SeleniumTestCase.php';

  class WebTest extends PHPUnit_Extensions_SeleniumTestCase {

    // Set up the browsers that we want to test in
    public static $browsers = array(
        'name'    => 'Google Chrome on Windows',
        'browser' => '*googlechrome H:\Documents and Settings\dave\Local Settings\Application Data\Google\Chrome\Application\chrome.exe',
        'host'    => 'localhost',
        'port'    => 4444,
        'timeout' => 30000,
        'name'    => 'Firefox on Windows',
        'browser' => '*firefox H:\Progra~1\Mozill~1\firefox.exe',
        'host'    => 'localhost',
        'port'    => 4444,
        'timeout' => 30000,
        'name'    => 'Internet Explorer 7 on Windows',
        'browser' => '*iexplore',
        'host'    => 'localhost',
        'port'    => 4444,
        'timeout' => 30000,

    protected function setUp() {

    public function test_IWA() {
      // Determine which browser is being tested //
      $tmp = serialize($this);
      $chrome_pos   = strpos($tmp, 'Google Chrome on Windows');
      $firefox_pos  = strpos($tmp, 'Firefox on Windows');
      $iexplore_pos = strpos($tmp, 'Internet Explorer 7 on Windows');
      if($chrome_pos   < $firefox_pos && $chrome_pos   < $iexplore_pos) { $curBrowser = 'GoogleChrome'; }
      if($firefox_pos  < $chrome_pos  && $firefox_pos  < $iexplore_pos) { $curBrowser = 'Firefox'; }
      if($iexplore_pos < $firefox_pos && $iexplore_pos < $chrome_pos)   { $curBrowser = 'IE7'; }

      echo PHP_EOL.PHP_EOL.PHP_EOL.'============== '.$curBrowser.' ==============='.PHP_EOL.PHP_EOL.PHP_EOL;

      // Request Account //
      echo '  Request Account...'.PHP_EOL;
      // Build a testUser with a name that should be unique
      $testUser = $curBrowser."_".date('dHis');
      $this->type("css=input[name=user_name]", $testUser);
      $this->type("css=input[name=first_name]", $testUser);
      $this->type("css=input[name=last_name]", $testUser);
      $this->type("css=input[name=email]", $testUser."");
      $this->type("phone_number", "123-456-7890");
      $this->type("password", "9/3=Two_");
      $this->type("password2", "9/3=Two_");
      $this->fireEvent("password2", "keyup");
      for ($second = 0; ; $second++) {
        if ($second >= 60) $this->fail("timeout");
        try {
          if ($this->isElementPresent("css=select[name=company]")) break;
        catch (Exception $e) {}
      $this->select("css=select[name=company]", "");
      $this->type("css=input[name=selected_comp]", '');
      $this->select("timezone", "label=-04:00 ==> Eastern Daylight Time");
      $this->fireEvent("css=input[name=user_name]",  "blur");
      $this->assertTrue($this->isTextPresent("exact:Getting Started"));
      $this->assertTrue($this->isTextPresent("exact:When is Dave available?"));

      // Login //
      echo '  Login...'.PHP_EOL;
      $this->type("user_name", $testUser);
      $this->type("password", "9/3=Two_");
      $this->assertTrue($this->isTextPresent("Getting Started"));

      // Edit Profile //
      echo '  Edit Profile...'.PHP_EOL;
      // Change every value
      $this->type("firstname", $testUser."x");
      $this->type("css=input[name=last_name]", $testUser."x");
      $this->type("css=input[name=email]", $testUser."@industrialwebapps.comx");
      $this->type("phone_number", "123-456-7890x");
      $this->select("timezone", "label=-08:00 ==> Pacific Standard Time");
      // Confirm that the changes were made
      $this->assertTrue($this->isTextPresent("Pacific Standard Time"));

      // Change Password //
      echo '  Change Password...'.PHP_EOL;
      $this->type("password", "6/2=Two+");
      $this->type("password2", "6/2=Two+");
      $this->fireEvent("password", "keyup");
      $this->fireEvent("password2", "keyup");
      $this->assertTrue($this->isTextPresent("exact:Password Change"));
      $this->assertTrue($this->isTextPresent("exact:Password successfully changed."));
      $this->assertTrue($this->isTextPresent("exact:A password change notification has been sent to your configured email address."));
      $this->assertTrue($this->isTextPresent("exact:When is Dave available?"));

      // Logout //
      echo '  Logout...'.PHP_EOL;
      $this->assertTrue($this->isTextPresent("exact:User Login"));
      $this->assertTrue($this->isTextPresent("exact:When is Dave available?"));



No comments:

Post a Comment