Simulating Mac Safari browser on Windows

In my project there was a requirement to automate few test cases that would validate functionality of the website on Safari browser on Mac machine. But automation framework supports only Windows machines.

To support the new requirement, I leveraged 'user-agent' feature of the browser to simulate safari browser on Windows client machine. The intention of testing here was not to validate the layout or look and feel of the website but rather validating features such as presence or absence of a button based on the OS on which we are browsing.

User-agent is basically a string that represents the browser engine, browser version, OS and OS version etc.
Browser will send user-agent header with every http request to server. Most of the websites depend on user-agent string to find out from which OS and Browser the request is coming and then change the business logic accordingly or even redirect to different website.

For example, when browsing on mobile you will observe facebook.com will automatically redirect to m.facebook.com. So to simulate any browser, user-agent should be altered accordingly before making request. There are many add-ons like this one for firefox and for chrome available to do that.

Switching user-agent can be achieved in Automation without using any add-ons. WebDriver allows setting user-agent preference before invoking the browser. Once it is set, the browser will send altered user-agent string to the website for every subsequent request and essentially mimicking the browser we wanted to test on.

Currently, I have tested it with Firefox and Chrome browsers and it is working fine. Here is the sample code to switch user-agent for different OS such as MAC, Linux, Android and iOS:

public void OpenFirefox()
{
string osType = "MAC";
FirefoxProfile _ffProfile = new FirefoxProfile();
_ffProfile.SetPreference("general.useragent.override",  GetUserAgent(osType));
IWebDriver _driver = new FirefoxDriver(_ffProfile);
}

public void OpenChrome()
{
string osType = "MAC";
 var options = new ChromeOptions();
options.AddArgument("--user-agent=" + GetUserAgent(osType));
 IWebDriver _driver = new ChromeDriver(options);
}

public string GetUserAgent(string osType)
        {
            switch (osType.ToUpperInvariant())
            {
                case "MAC":
                    return "Mozilla/5.0 (iPad; CPU OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5355d Safari/8536.25";
                case "LINUX":
                    return "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:20.0) Gecko/20100101 Firefox/20.0";
                case "ANDROID":
                    return "Mozilla/5.0 (Linux; Android 4.1.1; Galaxy Nexus Build/JRO03C) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.166 Mobile Safari/535.19";
                case "IOS":
                    return "Mozilla/5.0 (iPhone; CPU iPhone OS 5_0 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A334 Safari/7534.48.3";
                default:
                    return "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31";
            }
        }

Find installed browser version from Registry

More often than not, there would be need to find version of browser installed on a machine on which Automation is run.

After lot of googling and validations of suggestions on different blogs, I could figure out registry paths to get browser version for Internet Explorer, Firefox and Chrome on 32-bit and 64-bit machines.

I thought it would be good idea to blog this so that others can save time rather than browsing through multiple sites and do trail and error.

Internet Explorer:
For 32-bit and 64-bit Machines, registry path:
HKLM\SOFTWARE\Microsoft\Internet Explorer\Version

For Windows 8:
HKLM\SOFTWARE\Microsoft\Internet Explorer\svcVersion

Method to retrieve version programmatically:

private static void GetIEVersion()
    {
        RegistryKey regKey = Registry.LocalMachine;
        Console.WriteLine("ie-" + regKey.OpenSubKey(@"SOFTWARE\Microsoft\Internet Explorer").GetValue("Version")); //For Windows 8 the key name is 'svcVersion'
    }

Firefox:
For 32-bit machine:
HKLM\SOFTWARE\Mozilla\Mozilla Firefox\CurrentVersion
For 64-bit machine:
HKLM\SOFTWARE\Wow6432Node\Mozilla\Mozilla Firefox\CurrentVersion


Method to retrieve version programmatically:

private static void GetFirefoxVersion()
    {
        string wowNode = string.Empty;
        if (Environment.Is64BitOperatingSystem) wowNode = @"Wow6432Node\";
        RegistryKey regKey = Registry.LocalMachine;
        Console.WriteLine("firefox-" +
          regKey.OpenSubKey(@"Software\" + wowNode + @"Mozilla\Mozilla Firefox")
                .GetValue("CurrentVersion"));
    }

Chrome:
For 32-bit machines:
HKLM\SOFTWARE\Google\Update\Clients

For 64-bit machines:
HKLM\SOFTWARE\Wow6432Node\Google\Update\Clients

Not done yet, chrome is little complex. 
The above path lists multiple GUIDs {CLSID}. Search for GUID subkey which has key "name" value as "Google Chrome"
Under same subkey chrome version will be present in key named "pv"

If chrome is installed only for current user, change the root of registry to HKCU
HKCU\Software\Google\Update\Clients

On 64-bit machine if there is no key as mentioned above, try searching in same path as that of 32-bit machine.

Method to retrieve version programmatically:

private static void GetChromeVersion()
    {
        string wowNode = string.Empty;
        if (Environment.Is64BitOperatingSystem) wowNode = @"Wow6432Node\";

        RegistryKey regKey = Registry.LocalMachine;
        RegistryKey keyPath = regKey.OpenSubKey(@"Software\" + wowNode + @"Google\Update\Clients");

        if (keyPath == null)
        {
            regKey = Registry.CurrentUser;
            keyPath = regKey.OpenSubKey(@"Software\" + wowNode + @"Google\Update\Clients");
        }      

        if (keyPath == null)
        {
            regKey = Registry.LocalMachine;
            keyPath = regKey.OpenSubKey(@"Software\Google\Update\Clients");
        }

        if (keyPath == null)
        {
            regKey = Registry.CurrentUser;
            keyPath = regKey.OpenSubKey(@"Software\Google\Update\Clients");
        }

        if (keyPath != null)
        {
            string[] subKeys = keyPath.GetSubKeyNames();
            foreach (string subKey in subKeys)
            {
                object value = keyPath.OpenSubKey(subKey).GetValue("name");
                bool found = false;
                if (value != null)
                    found =
                        value.ToString()
                             .Equals("Google Chrome", StringComparison.InvariantCultureIgnoreCase);
                if (found)
                {
                    Console.WriteLine("chrome-" + keyPath.OpenSubKey(subKey).GetValue("pv"));
                    break;
                }
            }
        }
        else
        {
            Console.WriteLine("registry key not found for chrome");
        }
    }

Registry entries for Internet Explorer settings

As a webdriver user you probably aware that IE browser requires few tweaks I mean specific settings in order to avoid hangs, indefinite waits etc.

There are few settings which are mandatory like enabling protected mode (Tools > Options > Security > Internet Zone > Checkbox at the bottom) should be ON or OFF for all Zones and there are few which are optional as long as your flow doesn't need those. For ex: Disable alert which shows 'Do you want to show http content over https?'

I have compiled comprehensive list of these settings that are required to make webdriver tests work fine in IE. You can apply these settings using registry file as a one time task on all of your client machines during initial setup or at runtime in your project.

To apply these on your machine, save highlighted code below as a .reg file and double click to merge changes into registry.

Please read comments for each to know more about settings.

REGEDIT4

;1406 - Disable Popup blocker
;1609 - Enable Dispaly Mixed Content(http over https)
;2500 - Turn On Protected Mode
;http://support.microsoft.com/kb/182569
;Local Intranet Zone
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\1]
"1406"=dword:3
"1609"=dword:0
"2500"=dword:0

;Trusted sites Zone
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\2]
"1406"=dword:3
"1609"=dword:0
"2500"=dword:0

;Internet Zone
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\3]
"1406"=dword:3
"1609"=dword:0
"2500"=dword:0

;Restricated Sites Zone
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\4]
"1406"=dword:3
"1609"=dword:0
"2500"=dword:0

;Disable Check for Publisher's Certificate Revocation
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\WinTrust\Trust Providers\Software Publishing]
"State"=dword:23e00

;Disable Check for Server Certificate Revocation
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings]
"CertificateRevocation"=dword:0

;Disable Auto Form Fill and Remember Password
[HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main]
"Use FormSuggest"="no"
"FormSuggest Passwords"="no"
"FormSuggest PW Ask"="no"

;Disable dailog box 'A script on this page is causing Internet Explorer to run slowly. Do you want to stop the script?'
[HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Styles]
"MaxScriptStatements"=dword:FFFFFFFF

;Disable script error alerts in IE Containers
[HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main]
"DisableScriptDebuggerIE"="yes"
"Disable Script Debugger"="no"
"Error Dlg Displayed On Every Error"="no"

;Disable Check for Default Browser as IE
[HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main]
"Check_Associations"="no"

Clean session in Internet Explorer

Between tests and before starting test on client machine the browser should be in clean session in order for test not to interfere with previous session data. Otherwise tests might fail. There is a method in webdriver API called DeleteCookies to clear cookies of browser

However, this work perfectly fine for Firefox and Chrome but not always for Internet Explorer. For Firefox and Chrome, a new temporary profile while creating new browser instance in Webdriver and this is not the case in IE hence we might sometime see previous session details still persisting in new session.

QTP allows access to IE tools menu and options window to clear cookies and cache, however webdriver doesn't support interacting with menu. Hence we have to opt for command line option to clear cookies in IE.

Here is Code for same:

public void DeleteCookies()
        {
             if (_driver == null) return;
                _driver.Manage().Cookies.DeleteAllCookies();

                if (_driver.GetType() == typeof(OpenQA.Selenium.IE.InternetExplorerDriver))
                {
                     ProcessStartInfo psInfo = new ProcessStartInfo();
                     psInfo.FileName = Path.Combine(Environment.SystemDirectory, "RunDll32.exe");
                     psInfo.Arguments = "InetCpl.cpl,ClearMyTracksByProcess 2";
                     psInfo.CreateNoWindow = true;
                     psInfo.UseShellExecute = false;
                     psInfo.RedirectStandardError = true;
                     psInfo.RedirectStandardOutput = true;
                     Process p = new Process { StartInfo = psInfo };
                     p.Start();
                     p.WaitForExit(10000);
                }
        }

Only in case of IE, RunDll32.exe is invoked with arguments InetCpl.cpl,ClearMyTracksByProcess 2 to clear cookies and cache. To clear entire history including temp files, history, form data, passwords etc.. change arguments to InetCpl.cpl,ClearMyTracksByProcess 255. This should work on IE 7 and above on all OS.

Following are different arguments supported by InetCpl.cpl:

echo Clear Temporary Internet Files:
RunDll32.exe InetCpl.cpl,ClearMyTracksByProcess 8

echo Clear Cookies:
RunDll32.exe InetCpl.cpl,ClearMyTracksByProcess 2

echo Clear History:
RunDll32.exe InetCpl.cpl,ClearMyTracksByProcess 1

echo Clear Form Data:
RunDll32.exe InetCpl.cpl,ClearMyTracksByProcess 16

echo Clear Saved Passwords:
RunDll32.exe InetCpl.cpl,ClearMyTracksByProcess 32

echo Delete All:
RunDll32.exe InetCpl.cpl,ClearMyTracksByProcess 255

echo Delete All w/Clear Add-ons Settings:
RunDll32.exe InetCpl.cpl,ClearMyTracksByProcess 4351

Getting cookies from all domains

In automated tests there might be instances where we have to validate cookies of a website.

Webdriver has simple and powerful API to retrieve cookies of current page domain. Here is the sample code to read cookies:

 public Dictionary<string, string> GetAllPageCookies()
        {
            return _driver.Manage().Cookies.AllCookies.ToDictionary(cookie => cookie.Name, cookie => cookie.Value);
        }

The Problem:
However, if a website has some 3rd party plugins such as omniture for page tracking and analysis the cookies are stored under its own domain, not under the domain of your application. By default, as a security measure, browsers don't allow access to cookies from different domain from one domain. Which otherwise will lead to inevitable security issues.

The tricky part is, when you want to validate all the cookies from different domains that are created when a user visits your website.

The Solution:
Solving the problem from webdriver point of view means the feature has to work on all supported  browsers, which is very tough as different browsers implement cookies in different ways. So make it easier I have restricted the solution to Firefox browser which has very good support for plugins and profiles.

To accomplish this task I have created a plugin for firefox called Get all cookies in XML and published it in Mozilla Add-Ons website.

This plugin's job is to use firefox native API to to save all cookies in an XML file under current profile directory of firefox browser.

Once plugin is installed, to capture cookies at any moment open the URL chrome://getAllCookies/content/getAllCookies.xul or click on the menu option which will save all cookies in file by name 'cookie.xml' directly under current profile directory.

The XML schema is as follows:
<cookies>
  <cookie id="cookie_name">
    <host>domain.com</host>
    <isdomain>FALSE</isdomain>
    <path>/</path>
    <issecure>FALSE</issecure>
    <expires>1388032303</expires>
    <name>cookiename</name>
    <value>cookie_value_as_cdata</value>
    <policy>0</policy>
    <status>0</status>
  </cookie>
<cookies>

Cookie.xml is the snapshot of all cookies at a given moment. Having access to this file I can now parse or search for specific cookies using any XML parsing mechanism.

Steps to use this plug-in in your project:
1. Download the plugin from here
2. If you are invoking firefox in specific profile in your automation suite, install the plugin in the same profile
3. If you are not using any specific profile then use AddExtension method of FirefoxProfile class to install plugin at runtime.
4. Assuming firefox is opened wihtout any issues and the browser is navigated to a page where cookies are to be captured, call the following GetAllCookies() method to get XDocumnet object with all cookies.

private XDocument GetAllCookies()
        {
            IJavaScriptExecutor jscript = _driver as IJavaScriptExecutor;
            int lastWindowCount = _driver.WindowHandles.Count;
            if (jscript != null) jscript.ExecuteScript("window.open('')");
            int maxWaitTimeInMilliSeconds = 5000;
            do
            {
                if (lastWindowCount < _driver.WindowHandles.Count)
                    break;

                Thread.Sleep(100);

                maxWaitTimeInMilliSeconds = maxWaitTimeInMilliSeconds - 100;

            } while (maxWaitTimeInMilliSeconds > 0);

            if (maxWaitTimeInMilliSeconds <= 0)
            {
                throw new ApplicationException("Javascript Open Window Failed");
            }

            _driver.SwitchTo().Window(_driver.WindowHandles.Last());
            //Need not open URL; As soon as new window is opened, cookie.xml is re-created with all cookies.
            //If you want to capture cookies without opening a new window, browse to below URL in current window
            _driver.Navigate().GoToUrl("chrome://getAllCookies/content/getAllCookies.xul");
            //Wait for file to get updated
            Thread.Sleep(1000);
            _driver.Close();
            _driver.SwitchTo().Window(_driver.WindowHandles.Last());
            return XDocument.Load(Path.Combine(_ffProfile.ProfileDirectory, "cookie.xml"));
        }

Waiting for Page Load to complete

Webdriver has ways for implict and exlict wait but that wont be useful when page is taking too long to load. Also, when an exception or error is occured in the flow, we end up waiting unnecessarily for "specified" time though page has already loaded and nothing is going to change in the remaining time period.

One of the limitation of Webdriver API is no support for WaitForPageLoad out of the box. But we can implement that using WebDriverWait class and readyState property of DOM.

Here is the solution:

public void WaitForPageLoad(int maxWaitTimeInSeconds)
{
   string state = string.Empty;
 try{
 WebDriverWait wait = new WebDriverWait(_driver, TimeSpan.FromSeconds(maxWaitTimeInSeconds));

//Checks every 500 ms whether predicate returns true if returns exit otherwise keep trying till it returns ture


wait.Until(d =>{
try{
state = ((IJavaScriptExecutor)_driver).ExecuteScript(@"return document.readyState").ToString();
}
catch (InvalidOperationException){
//Ignore
}
catch (NoSuchWindowException){
//when popup is closed, switch to last windows
_driver.SwitchTo().Window(_driver.WindowHandles.Last());
}
//In IE7 there are chances we may get state as loaded instead of complete
return (state.Equals("complete", StringComparison.InvariantCultureIgnoreCase) || state.Equals("loaded", StringComparison.InvariantCultureIgnoreCase));
});
}
catch (TimeoutException){
//sometimes Page remains in Interactive mode and never becomes Complete, then we can still try to access the controls 

if (!state.Equals("interactive", StringComparison.InvariantCultureIgnoreCase))
throw;
}
catch (NullReferenceException){
//sometimes Page remains in Interactive mode and never becomes Complete, then we can still try to access the controls 

if (!state.Equals("interactive", StringComparison.InvariantCultureIgnoreCase))

throw;
}
catch (WebDriverException){
if (_driver.WindowHandles.Count == 1)
{
_driver.SwitchTo().Window(_driver.WindowHandles[0]);
}

state = ((IJavaScriptExecutor)_driver).ExecuteScript(@"return document.readyState").ToString();

if (!(state.Equals("complete", StringComparison.InvariantCultureIgnoreCase) || state.Equals("loaded", StringComparison.InvariantCultureIgnoreCase)))
   throw;
 }
}