Thursday, December 17, 2009

Design of Selenium tests for ASP.NET: Dealing with persistence

Every web site have persisted storage. It can be XML on disk, DB or Key-Value storage, whatever. This aspect brings a lot of troubles while designing Web Tests. The most common example – tests script must login in system with predefined credentials and this credentials should be in storage before test run. Or another example:

 [Test]
public void NewlyAddedUserCanLoginInSystem()
{
Start
.LoginAndGoToHomePage()
.AddUser("TestUserForLogin", "TestPwdForLogin")
.Logout()
.EnterCredentials("TestUserForLogin", "TestPwdForLogin")
.Login();
}

This test will pass only for first time – while second run user with name “TestUserForLogin” will be already in storage and inserting procedure will fail.

To resolve this issues it is needed to:
  1. Have some initial data in storage. e.g. test user
  2. Clean up data before test suite run.

To perform this actions it is needed to have direct access to storage to the application, which allows to manage storage. Definitely it is not acceptable to insert this functions in system under test – “clean database” button is the grenade waiting for the monkey:)

It is needed to create small standalone admin application:

Storage admin application Next function will be executed when button is clicked:

 protected void ClearStorage(object sender, EventArgs e)
{
const string emptyStorage = @"<users><user name=""admin"" password=""god"" /></users>";
var doc = new XmlDocument();
doc.LoadXml(emptyStorage);
doc.Save(new Configuration().Xml);
}

This function is designed to clean up XML, but if storage is database it will be need to execute simple script to restore database like this:

USE [master]
GO

IF EXISTS (SELECT name FROM sys.databases WHERE name = N'teststorage')
ALTER DATABASE [teststorage] SET SINGLE_USER WITH ROLLBACK IMMEDIATE
GO

IF EXISTS (SELECT name FROM sys.databases WHERE name = N'teststorage')
DROP DATABASE [teststorage]
GO

RESTORE DATABASE [teststorage] FROM DISK = N'C:\sl\BackUp\teststorage.bak'
WITH FILE = 1,
MOVE N'teststorage'
TO N'C:\sl\Sites\teststorage.mdf',
MOVE N'teststorage_log'
TO N'C:\sl\Sites\teststorage.ldf',
NOUNLOAD, REPLACE, STATS = 10
GO

Now it is only needed to click on this button before any test run. This is possible with nunit “SetupFixture” attribute.

This is the attribute that marks a class that contains the one-time setup or teardown methods for all the test fixtures under a given namespace. The class may contain at most one method marked with the SetUpAttribute and one method marked with the TearDownAttribute.

using NUnit.Framework;
using Tests.SmokeTest.Core;
using Tests.SmokeTest.PageObjects;

namespace Tests.SmokeTest
{
[SetUpFixture]
public class SetUpSUT
{
[SetUp]
public void CleanUpStorage()
{
CleanUp();
}

public static void CleanUp()
{
using (var navigator = new Navigator())
{
navigator.Start(Configuration.StorageAdminSiteUrl);
var storageAdminPage = navigator.Open<StorageAdminPage>();

navigator.Navigate<StorageAdminPage>(storageAdminPage.ClickClear);
}
}
}
}

Of course it is needed to create standalone Storage Sandbox for each developer, which runs web tests and for each continuous integration build to avoid interfering between tests.

Note that both SUT and Storage Admin sites should be deployed and online during the test run. It is very convenient to setup virtual folders for them in IIS, because if you use ASP.NET development server, it will run each site in instance with randomly generated port, and it is needed always fix configs for tests.

SUT and admin application in IIS

All sources are available in design-of-selenium-tests-for-asp-net sample solution.


kick it on DotNetKicks.com

2 comments:

Constantine said...

For the testing purposes it is often convenient to use some lightweight in-memory RDBMS, like HSQLDB. It is crossplatform, doesn't require installation and is a way faster to start, stop and restore. But of course you will run into trouble if some vendor-specific SQL or stored procedures are used.

Yauheni Sivukha said...

Good hint.

Approach with in-memory database has another benefit. It is possible to provide separate instance of database for each test. This guarantees that tests which are executed in parallel will not affect each other.

However I do not think that performance is a good reason to replace DB engine for tests. It is possible to create fresh fixture for huge SQL server database in milliseconds (We really did it for one of our projects, I can prove it :) ).