Synopsis
- Microsoft changed the behavior of Coded UI Tests between Visual Studio 2010 and Visual Studio 2012.
- The change is that the browser window or app window being used for the test case will automatically be closed after each test case unless you add code to keep it open.
- In any event, the browser window or app window used will always be closed at the end of a test run.
- During development you can use a Breakpoint to halt test execution prior to the windows being closed.
Issue
There is a behavior change between Coded UI Tests (CUIT) running in Visual Studio 2010 and Visual Studio 2012 . In Visual Studio 2010, a test case would leave windows open on the desktop unless they were explicitly closed in code in the test class. In Visual Studio 2012 , the window of the application under test (browser, WPF app, etc.) is automatically closed whether or not it was closed in code during the test run. If you are new to CUIT and you only use VS 2012 then you weren’t aware that the behavior changed. If you have upgraded from VS 2010 then this is something new.
Why did the folks at Microsoft make this change?
This change was added to make it easier to create an maintain test cases because it saves you from explicitly having to close your app window or browser at the end of every test. An automated test run that has many test cases launching browser windows but never closing them will leave the test client’s desktop in a messy state. I agree that it makes CUIT easier.
So why write a blog post about something you agree with?
The reason for this post is that it changes the way that folks from VS 2010 interact with coded UI tests, specifically if you use a manual test’s Action Recording as the starting point for your CUIT.
In VS 2010 I would create a new CUIT and use the action script from a manual test case to generate a base test without any validation as in the code below. I’d then run this test to make sure everything worked as expected. At the end the browser would close.
2: public void KioskSmokeTest()
4: // To generate code for this test, select "Generate Code for Coded...
5: // For more information on generated code, see http://go.microsoft...
7: this.UIMap.StartDinnerNowKiosk();
8: this.UIMap.ExerciseUI();
9: this.UIMap.CloseKiosk();
Figure 1: Basic coded UI test that explicitly closes the app under test
Now I need to add some validation to my test. Normally I would just comment out the line that closes the windows, in my example
this.UIMap.CloseKiosk() and re-run the test. This would leave the app window open and the app would be in the correct state for me to add validations.
1: [TestMethod]
2: public void KioskSmokeTest()
3: {
4: // To generate code for this test, select "Generate Code for Coded...
5: // For more information on generated code, see http://go.microsoft...
6:
7: this.UIMap.StartDinnerNowKiosk();
8: this.UIMap.ExerciseUI();
9: //this.UIMap.CloseKiosk();
10: }
Figure 2: Basic coded UI test that is supposed to leave the app window open at the end
My example is fairly simple, but what if the
ExerciseUI() method actually navigated through 4 or 5 screens and added test data. In this case, my process has allowed the test to perform all of the data entry and I am assured that the app is in the correct state for my validations. This saved me a lot of work and headaches.
In VS 2012, this behavior isn’t the same. The
CloseKiosk() method is no longer needed as the test engine will automatically close the app between test cases unless told to keep them open. In effect, the first code example is now equivalent to the second one.
But I liked the old behavior, is there a workaround?
Your only option is to set a breakpoint on the last executable line within your test case and then go to Test View and right-click on the test’s name then select
Debug Selected Tests. When the breakpoint gets hit, press
F10 to let the last executable line run then just stop debugging. That will leave your app in the correct state and you can now use the Coded UI Test Builder to create your Assertions.
Figure 3: Use a breakpoint and debug your test to get to the right app state for validation
Figure 4: Stop debugging when you have finished execution of all of the preparatory steps
What if I chain test cases together and assume that the app stays open between tests?
Some folks write completely self-contained tests like my KioskSmokeTest above. Other folks have more complex needs and write tests that get run in a specific order, each collaborating to setup the state of a system and then performing validations. This can be called “chaining tests”.
If you have chained tests, the new behavior will break your test runs since the application under test gets closed after each test case is run. Microsoft has added some new flags that can be set (depending on app platform) to tell the test engine to leave your app alone between tests cases in a single test run.
If your
target app is browser-based, you can tell the testing framework to leave the browser open at the end of the test case by setting the
BrowserWindow.CloseOnPlaybackCleanup property to
false (Line 5 below).
2: public void VerifyValidDefaultSearchTest()
4: //Leave the browser open
5: this.UIMap.UIDinnerNownetWindowsIWindow.CloseOnPlaybackCleanup = false;
7: this.UIMap.StartvalidsearchParams.UIDinnerNownetWindowsIWindowUrl = "http://WebTest/DinnerNow/Default.aspx";
8: this.UIMap.Startvalidsearch();
9: this.UIMap.ClicktheFindbutton();
Figure 5: Use CloseOnPlaybackCleanup to keep the browser open
The Microsoft Visual Studio ALM team has a nice blog post describing this in more detail.
If your target app is not browser-based (WPF/WinForms/etc.) then you need to use the ApplicationUnderTest.CloseOnPlaybackCleanup() property.
Note: There is a bug in VS 2012 RTM that causes ApplicationUnderTest.CloseOnPlaybackCleanup() property to be ignored. This bug is fixed in VS 2012 Update 1.
Unfortunately the ApplicationUnderTest object isn’t as easy to access as the BrowserWindow objects. To make this work you will have to capture the instantiation of your app in the UIMap.Designer.cs file and store the reference in a variable that can be seen from your test methods.
Below you see the code that was generated in the UIMap for my application. This code launches the DinnerNowKiosk application and uses a local variable to store the handle to the app. Once the StartDinnerNowKiosk method goes out of scope, the reference is no longer accessible. Also, you can’t just go and add a call to dinnerNowKioskApplication.CloseOnPlaybackCleanup = false because this is a generated code file so the added code will be lost of the UIMap is regenerated.
1: public void StartDinnerNowKiosk()
2: {
3: ApplicationUnderTest dinnerNowKioskApplication = ApplicationUnderTest.Launch
(this.StartDinnerNowKioskParams.ExePath,
this.StartDinnerNowKioskParams.AlternateExePath);
4: }
Figure 6: The generated application launch code in the UIMap.designer.cs file
Instead you have to use the UIMap.cs partial class to add your new startup code. Just copy the StartDinnerNowKiosk() method into the partial class and set the CloseOnPlaybackCleanup property to false.
1: public partial class UIMap
2: {
3: static ApplicationUnderTest dinnerNowKioskApplication_persistent = null;
4:
5: public void StartDinnerNowKiosk_Persistent()
6: {
7: if (null == dinnerNowKioskApplication_persistent)
8: {
9: dinnerNowKioskApplication_persistent = ApplicationUnderTest.Launch (this.StartDinnerNowKioskParams.ExePath, this.StartDinnerNowKioskParams.AlternateExePath);
10: dinnerNowKioskApplication_persistent.CloseOnPlaybackCleanup = false;
11: }
12: }
13:
14: }
Figure 7: The new app launch code from the non-generated partial class in the UIMap
And then change your test to call the new app launcher method.
1: [CodedUITest]
2: public class KioskSmoke
3: {
4:
5: [TestMethod]
6: public void KioskSmokeTest()
7: {
8: this.UIMap.StartDinnerNowKiosk_Persistent();
9: this.UIMap.ExerciseUI();
10: }
11:
12: [TestMethod]
13: public void KioskSmokeTest2()
14: {
15: this.UIMap.StartDinnerNowKiosk_Persistent();
16: this.UIMap.AdditionalSteps();
17: }
18: }
Figure 8: The updated tests that call to the new app launcher.
In the example above, the StartDinnerNow_Persistent() method is called from each test case because we don’t know the order these test will be run. If you need to have the tests run in a specific order then you need to create an additional test method that calls them in the order desired.
Can I keep the app under test open between test runs?
No. The app/browser is launched by the test agent process (QTAgent). At the end of a test run QTAgent is shut down, so any child processes are also shut down. I have confirmed this behavior in VS 2012 RTM and VS 2012 Update 1.
Resources
MSDN: BrowserWindow class -
http://msdn.microsoft.com/en-us/library/microsoft.visualstudio.testtools.uitesting.browserwindow.aspx
MSDN: ApplicationUnderTest class -
http://msdn.microsoft.com/en-us/library/microsoft.visualstudio.testtools.uitesting.applicationundertest.aspx
MSDN:
Visual Studio 2012 Update 1 Download -
http://www.microsoft.com/en-us/download/details.aspx?id=35774
Visual Studio 2012 ALM Blog: Coded UI Test: Why does application close after each test in Visual Studio 2012? -
http://blogs.msdn.com/b/visualstudioalm/archive/2012/11/08/using-same-applicationundertest-browserwindow-across-multiple-tests.aspx
![SNAGHTML709b95[4]](http://www.planeterph.com/blog/img/How-to-make-an-offline-installation-of-V_AD72/SNAGHTML709b954.png "SNAGHTML709b95[4]")
RSS 3.0
