Wednesday, 28 October 2009

Using Non-Functional Tests Tools with Axe and WatiN

As a functional automation tester have you ever been asked the following questions:

  • From Infrastructure - We have upgraded the network and are expecting an increase in throughput. Can you please run a quick test and let us know the improvement stats?
  • From a Developer – We have made some changes to the framework and are expecting some issues. Can you please run some tests and let us know the error codes?

I was faced with the second question, and after a bit of googling and discussions with the Dev team I found out about HttpWatch: http://www.httpwatch.com/

A quick read of their website gave me this:

“All web applications make extensive use of the HTTP protocol (or HTTPS for secure sites). Even simple web pages require the use of multiple HTTP requests to download HTML, graphics and javascript. The ability to view the HTTP interaction between the browser and web site is crucial to these areas of web development”

If you want to investigate more about the features that HttpWatch provides, go here: http://www.httpwatch.com/features.htm

A cool thing about HttpWatch is that it has a COM based automation interface, which allows you to use it from C#, as well as Javascript and Ruby. This means that I could integrate it into Axe, a bit more googling gave me this which formed the basis for the integration of HttpWatch and Axe:

Using HttpWatch with WatiN: http://blog.httpwatch.com/2008/10/30/using-httpwatch-with-watin/

The steps to integrate HttpWatch into Axe are as follows, or you can just download the sample Axe project, install HttpWatch and get testing:

1.Axe and WatiN installed and configured.
2.A WatiN project with a local Action Map and custom library file.
3.Download and install the free Basic Edition of HttpWatch.
4.Add two methods to the custom project library file.
5.Modify the local Action Map and Run config files.

The output from HttpWatch is a hwl.file per Axe test, so if you have 100 Axe tests, you will get a 100 hwl.files. Not a good thing, when you are pressed for time. To minimise the time spent pouring over the output files you should create a test run that exercises most pages, and add a switch to the Run config file to turn the HttpWatch logging on or off.


Installing HttpWatch

I downloaded the free Basic Edition of HttpWatch (http://www.httpwatch.com/download/), version 6.2.10 and installed it using the default options. A HttpWatch folder was created in my ProgramFiles folder on the C drive.


Adding Methods to the Project Library File

If you do not have a custom project library file, see the Axe documentation for creating one.

The first step is to add a reference to HttpWatch to your custom library project.

HttpWatch-Axe





















We are going to add two methods to the Project Library file called:

  • InitialiseHttpWatch(), and
  • CloseHttpWatch()
public static HttpWatch.Plugin InitialiseHttpWatch(Browser ie, AxeMainAPI axe)

{

// HttpWatch
if (String.Compare(axe.GetRunCategoryOption("Option", "HttpWatch"), "TRUE", true) == 0)
{
// Attach HttpWatch to instance of IE
HttpWatch.Controller ct = new HttpWatch.Controller();
HttpWatch.Plugin plugin = ct.IE.Attach((SHDocVw.IWebBrowser2)ie.IE.InternetExplorer);

// Start recording a log file in HttpWatch
plugin.Record();

return plugin;
}
else
{
return null;
}
}


public static void CloseHttpWatch(HttpWatch.Plugin plugin, string resultDir, string testID)
{
if (plugin != null)
{
plugin.Stop();
plugin.Log.Save(resultDir + @"\" + testID + ".hwl");
HttpWatch.Summary logSummary = plugin.Log.Entries.Summary;
}
}

There are two variables in the CloseHttpWatch() method , testID and resultDir that need to be declared and set. This is done in the __testPrefix section of the Local Action Map.


Modify the Run Config File

Add the following switch to the Run config file and set it to False:
CategoryOptionValue
OptionHttpWatchFalse

This is where you turn on the HttpWatch logging. The code looks for a value of TRUE to start logging, any other value and the logging will be turned off.


Modify the Local Action Map

Add the following to the __testPrefix section of the Local Action Map:
// Run settings
string testID = "%TESTID%";
string resultDir = @"%RESULTDIR%";
HttpWatch.Plugin plugin = null;

Modify the testEnd(id) section of the Local Action Map to look like the following:

}
catch(Exception ex)
{
axe.StepInfo(ex.Message);
axe.TestAbort();
NovaTest.NovaTestLib.CloseHttpWatch(plugin, resultDir, testID);
if(ie.IE != null) ie.IE.Dispose();
Environment.Exit(1);
}
axe.TestEnd()


You should now be able to Build and Run your Axe project and view the HttpWatch output in the Results directory.

Integrating HttpWatch into Axe does not make Axe a performance test tool, but it does give you some extra ammo in the hunt for bugs.

Happy testing

Marc Maurhofer

Senior Test Automation Analyst

Support Files
nFocusHttpWatch.zip

Note
Users must extract the zip file to c:\nfocushttpwatch because some paths are hardcoded for expediency.

Versions of software for reference:
Axe : 2.0.3
Watin : 1.3 (.NET 2.0)
HttpWatch Basic : 6.2.10

Wednesday, 21 October 2009

Announcements, a catch up and watch this space!

Firstly, I must start this entry with an apology for being away from the blog during the last few weeks... It's been a busy period over here at nFocus, especially during the last few weeks, but I am glad to be back, and writing.

Secondly, I need to thank all of our visitors for taking the time out to read our articles over the past few months. It makes me proud to announce that in the short time the blog has been online, we have achieved over a 1000 visits, and our last article, 'Automated Functional Testing Tools, QTP vs Selenium,' was our most popular article so far!

I am also proud to announce that the blog has been entered for this years Computer Weekly IT Blog Awards for the Corporate SME catergory, so thank you for nominating us, and I will do my best to keep you updated with how this goes.

So enough of the announcements, let's get to the main event. My entry today is to introduce an absolutely brilliant article written by a colleague of mine here at nFocus; Marc Maurhofer. Before the article is posted, I thought I would include Marc's short bio, in his own words:

"I have been a software tester for over 10 years, and learnt a long time ago that the only way not to drown in a sea of regression is to automate as much as possible, using a good test framework. I do not like the idea of creating automated regression packs after the product has gone live, but instead prefer to automate as you go along. When your overnight run takes longer than 12 hours, it is time to get another test rig."

The practical walkthough is titled, "Using Non-Functional Test Tools with Axe and WatiN," and we will publish it here within the next week.

So all this leaves me to say is, I hope you find his article useful and please feel free to make Marc feel welcome with plenty of comments after his article : )