Table of Contents
Microsoft Surface 1.0 SP1: SDK - Surface Simulator
Tl>
Overview
Microsoft Surface Simulator is a Microsoft Windows application. You can use Surface Simulator together with Microsoft Visual C# 2008 Express Edition (or Microsoft Visual Studio 2008) to test Microsoft Surface applications on a separate workstation. Surface Simulator runs the applications and enables you to use a mouse or mice to simulate finger, blob, and tagged-object contacts, like the contacts that a user uses on a Microsoft Surface unit.
 Important |
|---|
| Surface Simulator does not support the use of raw image input. If your application requires the use of raw image APIs like TryGetRawImage and UpdateRawImage, you must test it on an actual Microsoft Surface unit. |
Surface Simulator replicates the user interface and behavior of a Microsoft Surface unit that is in user mode. Surface Simulator has access points, Launcher, and the loading screen. When you start an application in Surface Simulator, the application displays like it is on a Microsoft Surface unit.
You can use Surface Simulator to evaluate how an application and its user interface respond to basic input. For example, if you simulate a painting application and if you touch multiple colors, one at a time, and then add the colors to a mixing bucket, you can test the logic of the application and how well it mixes the colors by using the touch-based interface.
Surface Simulator runs with the appearance and functionality of a Microsoft Surface unit in user mode (the way that it appears to users). You can switch applications by using Launcher and the access points that display on the Launcher screen and the applications.
Surface Shell is always turned on when you use Surface Simulator, so you see the attract application and Launcher (if you have touched an access point on the attract application tohen you start an application in Surface Simulator, the application displays like it is on a Microsoft Surface unit.
You can use Surface Simulator to evaluate how an application and its start Launcher).
Starting Surface Simulator
Microsoft Surface Simulator uses a 1024 × 768 simulation region to display Microsoft Surface applications. In this simulation region, Surface Simulator interprets mouse inputs as types of contacts (fingers, blobs, gestures, or tagged objects).
| Note |
|---|
| You cannot use Surface Simulator in a Remote Desktop or virtual environment. |
You do not have to have Microsoft Visual C# 2008 Express Edition (or Microsoft Visual Studio 2008) running when you start Surface Simulator. However, if Visual C# 2008 (or Visual Studio 2008) is running when you start Surface Simulator, make sure that you start Surface Simulator before you start a Microsoft Surface application from Visual C# 2008 (or Visual Studio 2008).
You must run Surface Simulator at the same privilege level as Visual C# 2008 (or Visual Studio 2008). If you start Visual C# 2008 (or Visual Studio 2008) as Administrator, you must also start Surface Simulator as Administrator. If you start Visual C# 2008 (or Visual Studio 2008) as a User, you must also start Surface Simulator as a User.
To start Surface Simulator, click Start, click All Programs, click Microsoft Surface SDK 1.0 SP1, click Tools, and then click Surface Simulator.
You can start a Microsoft Surface application from one of the following locations:
- From Launcher in Surface Simulator (as a user accesses it on a Microsoft Surface unit). To start the application from Launcher, the application must be registered with the unit.
- From outside Surface Simulator:
- From Visual C# 2008 (or Visual Studio 2008), by pressing F5. This method is useful if you are debugging the application.
- From the application executable file in Windows Explorer.
- From the application executable file at a command prompt.
- From Visual C# 2008 (or Visual Studio 2008), by pressing F5. This method is useful if you are debugging the application.
| Important |
|---|
|
Surface Simulator User Interface
This section describes the types of Microsoft Surface contacts you can use in Surface Simulator and describes Surface Simulator's interface.
Surface Simulator enables you to use a mouse to simulate three types of Microsoft Surface input:
- Finger. The contact of a finger.
- Blob. The contact of an object.
- Tagged object. The contact of an object that is marked with a byte tag or identity tag.
Parts of Surface Simulator
The following illustrations show the Surface Simulator interface with an attract application.
Surface Simulator interface includes the following pieces. (The numbers in the previous illustrations correspond to the numbers in the following list.)
- The Contact Selector button enables you to select and manipulate one or more contacts. Manipulations include move, resize, stretch, and rotate.
- The Finger button simulates a finger that contacts the application and triggers finger contact events.
- The Blob button simulates an arbitrary ellipse shape that contacts the application and triggers blob contact events. You can resize the Blob button to simulate shapes of different sizes.
- The Tag button simulates a tagged object that contacts the application and that uses user-specified hexadecimal values. This button triggers tagged-object contact events according to the specified hexadecimal values.
- The Tag Value data entry control enables you to enter a hexadecimal value (00 through FF) to assign to tagged-object contacts that you create by using the Tag button.
- The Identity Tag button enables you to simulate an identity tag contact that is placed on a Microsoft Surface screen. Enter identity tag series and value properties in the adjacent Series and Value data entry controls.
- The Identity Tag Series and Identity Tag Value controls enable you to assign hexadecimal values (0000000000000000 through 7FFFFFFFFFFFFFFF) to the series and value of an identity tag contact that you create by using the Identity Tag button.
- The Remove All Contacts button removes all contacts from the simulation region.
- The Hide/Show Contacts button determines whether the current contacts on the simulation region are visible. You can click the button to switch from showing the contacts to hiding the contacts. You can still put new contacts on the active area of the simulated screen when existing contacts are invisible.
- The simulation region is the area of the Surface Simulator user interface where the application appears. In this area, you can test the application by using finger, blob, and tagged object contacts. When you first start Surface Simulator, the active area displays the default attract application.
- The Surface Simulator access points that appear in each corner of the active area are Microsoft Surface controls that enable you to move between applications and Launcher. These access points and Launcher work the same way that they do on Microsoft Surface units.
- The recording panel enables you to record interactions and then play them for testing purposes. For more information about how to record and play interactions, see Recording and Playing Interactions below.
- The Minimize and Exit buttons minimize or close Surface Simulator and all Surface applications, respectively.
- The Help button opens the Surface Simulator section of the Surface SDK documentation.
- The Surface Simulator window is a movable window that encloses the Surface Simulator user interface and the application that you are testing. The window size is fixed to 1224 × 868, but you can minimize and restore the window.
How to Use Surface Simulator
This section describes how to work with Microsoft Surface contacts in Surface Simulator.
Adding Contacts to Surface Simulator
In the upper-left corner of the Surface Simulator window, you can use the buttons to simulate the different types of Microsoft Surface input (finger, blob, and tagged object). A contact is an instance of an input type that is interacting with the simulated region.
To simulate contact input, do the following:
- Select a contact tool by clicking one of the buttons on the toolbar in the upper-left area of the Surface Simulator window (for example, the Finger button).
- Move the cursor over the simulation region. The cursor displays the pointer that corresponds to the tool that you selected.
- Press the left mouse button to add the contact and then release the mouse button to remove the contact. Or, to put a contact on the screen and leave it there, press-and-hold the left mouse button and then right-click.
You can also click the Contact Selector button and then select a contact on the simulated Microsoft Surface screen and drag it across the screen.
To simulate more than one contact, you can connect additional mice to your computer's USB ports.
| Note |
|---|
| For additional USB ports, you can connect a USB hub to the computer. |
Contact States
A Surface Simulator contact can appear in one of the following states:
- White with a red shadow indicates that a contact is selected for manipulation (but is still on the simulated Microsoft Surface screen).
- Red indicates that the contact is placed on the screen.
- White indicates that the contact is hovering and is not yet placed on the screen.
Using Surface Simulator Controls and Contacts
This section describes how to use Surface Simulator controls and contacts.
Contact Selector
Click the Contact Selector button (or press CTRL+S) to choose the Contact Selector tool. You can then select contacts that are already placed in the simulation region by doing one of the following:
- Place the cursor directly over an already placed contact and then click it.
- Drag a selection region around one or more contacts that have already been placed.
You can remove selected contacts by using the DELETE key.
For examples of multitouch gestures that you can simulate by using Contact Selector, see Common Microsoft Surface Actions in Surface Simulator.
Finger Contact
Click the Finger button (or press CTRL+F) and then click in region by doing one of the following:
- Place the cursor directly over an already placed contact and then click it.
- Drag a selection region around one or more contacts that have already been placed.
You can remove selected contacts the active area to simulate a finger contact in the application. You can also click the Finger button and then drag it in the active area to simulate a dragging gesture across the application.
Press-and-hold the left mouse button and then right-click to place a finger contact, which leaves a finger pressed in that location of the application. You might use this procedure to press multiple controls at the same time (for example, buttons or pieces of content), to press-and-hold a control, or to anchor part of an interface. For example, you might use this procedure to hold a corner of a photo and then use the mouse to rotate the photo by interacting with the other corner.
Blob Contact
Click the Blob button (or press CTRL+B) to simulate an arbitrary ellipse shape. This button works the same way as the Finger button. You might want to use blob input when users are pressing different parts of their hands on the surface. For example, if a user leaves a handprint on the screen, the application might need to receive blob input.
Tagged-Object Contact (Byte Tag)
To simulate a byte tag
-
Click the Tag button (or press CTRL+T).
-
Click Tag Value, and enter the hexadecimal value you want.
-
Click the active area to simulate an object that is touching the screen and lifting up.
-
Press-and-hold in the active area, and then right-click to place the tagged object.
You can leave unlimited tagged-object contacts on the application. The Shopping Cart sample application demonstrates the code to use tagged-object contacts.
Tag Value
Click the Tag Value box and enter a valid hexadecimal value ("00" through "FF") for a tagged object.
| Note |
|---|
| You can select the tagged-object contact before or after you enter your hexadecimal value. |
| Important |
|---|
The updated hexadecimal value is applied to:
|
Tagged-Object Contact (Identity Tag)
To simulate an identity tag
-
Click the Identity Tag button (or press CTRL+I).
-
Click the Series box, and enter the hexadecimal series value that you want ("0000000000000000" through "7FFFFFFFFFFFFFFF").
-
Click the Value box, and enter the hexadecimal value that you want ("0000000000000000" through "7FFFFFFFFFFFFFFF").
-
Click the active area to simulate an object that is touching the screen and lifting up.
-
Press-and-hold in the active area, and then right-click to place the tagged object.
Identity Tag Series and Identity Tag Value
Click the Identity Tag Series or Identity Tag Value boxes, and enter a valid hexadecimal value ("0000000000000000" through "7FFFFFFFFFFFFFFF") for a tagged object.
| Note |
|---|
| You can select the tagged-object contact before or after you enter your hexadecimal series and value. |
| Important |
|---|
The updated series and value are applied to:
|
Hide/Show and Remove Contacts
Click the Remove All Contacts button to remove all contacts that you have placed in the active area of the Surface Simulator window. You can also press the Delete key to remove all currently selected contacts. Click the Hide/Show Contacts button to hide all your placed contacts or to show all the contacts that you have hidden. These features enable you to clear any extra contacts from the user interface.
Common Microsoft Surface Actions in Surface Simulator
The following table shows the corresponding mouse actions in Surface Simulator for common actions on a Microsoft Surface unit. Some of these actions are also listed in the preceding sections.
| Surface action | Mouse action | |
|---|---|---|
|
Select a contact type (finger, blob, or tagged object). |
Click the appropriate icon in the upper-left corner of the Surface Simulator window. |
|
|
Simulate a tagged object by using a byte tag with a specific value. |
|
|
|
Simulate a tagged object by using an identity tag with a specific series and value. |
|
|
|
Modify the size or orientation of a blob contact before placing it. |
| |
|
Modify the size or orientation of a placed blob contact. |
|
|
|
Touch the screen with finger. |
Click the Finger button, and then click the active area. |
|
|
Press a finger on the screen and hold indefinitely (place a finger contact). |
Press-and-hold the Finger contact, and then right-click. |
|
|
Drag a finger on the screen. |
Press-and-hold, and then drag the Finger contact. |
|
|
Drag two fingers on the screen. |
Do one of the following:
|
|
|
Lift a finger off the screen. |
Move the mouse over a placed contact until it becomes active, press-and-hold the contact, and then right-click it. |
|
|
Rotate a finger. |
Select a contact, and then rotate the mouse wheel up or down. |
|
|
Touch the screen with a blob. |
Click the Blob button, and then click the active area. |
|
|
Touch the screen with a tagged object. |
Click the Tagged Object button, and then click the active area. |
|
|
Lift a tagged object off the screen. |
|
|
|
Drag two fingers across the screen. |
Place the first finger on the simulated screen, place the second finger on the simulated screen, use the Contact Selector button to select both finger contacts, and then drag the contacts by using the symbol in the center of the selection area.  |
|
|
Rotate two fingers around a central point. |
Place the first finger on the simulator screen, place the second finger on the simulated screen, use the Contact Selector button to select both finger contacts, and then rotate the contacts by using the double-arrow pointer to the right of the selection area.  |
|
|
Drag two fingers apart (also known as a zoom gesture). |
Place the first finger on the simulated screen, place the second finger on the simulated screen, use the Contact Selector to select both finger contacts, and then drag the contacts apart by clicking and holding one corner on the selection area and dragging it.  |
|
|
p>
|
||
|
Drag two fingers apart (also known as a zoom gesture). |
Remove all contacts. |
Click the Remove All Contacts button in the upper-right corner of the active area. You can also press the DELETE key to remove all currently selected contacts. |
|
Show or hide contacts. |
Click the Hide/Show Contacts button in the upper-right corner of the active area. |
Recording and Playing Interactions
Surface Simulator supports recording and playback. You can record a sequence of Microsoft Surface interactions for rapid, iterative testing. These recordings are saved as scripts that you can then play later.
| Important |
|---|
| Do not open the script file for a recording and alter it. |
Recording Panel
When you want to record interactions on a computer, you can use the recording panel that appears at the bottom of Surface Simulator window.
You can also record interactions on a Microsoft Surface unit by using the recording panel that appears at the top of the Microsoft Surface screen on top of an application. To record and play interactions on a unit, log on the unit as an administrator and start Surface Simulator.
When you use the recording panel on a Microsoft Surface unit, you can use the following additional features:
- When your mouse is not moving, the recording panel disappears so that it does not obstruct your recording.
Note Move the mouse to make the recording panel reappear. - You can click the Close button (
) to close the recording panel.
Note The recording panel does not respond to touch input. You must use the mouse instead.
Recording Interactions
To record interactions in Surface Simulator
-
Click the Record button (
) to start recording.Note If you record a script when another script is already loaded, Surface Simulator starts a new recording. Surface Simulator does not erase or overwrite the previously loaded script. -
Complete gestures that you want to test by using the Surface Simulator controls.
-
Click the Pause button (
) to pause the recording, interact with the application, or change the interface so that the script moves to a new task when you play it. -
Click the Stop button (
) to name and save your recorded script.
Playing Interactions
To play interactions in Surface Simulator
-
Click the Open button (
), and then select a previously recorded script. -
Click the Play button (
) to start playing the script.Important - Do not use any Surface Simulator controls (such, as Finger, Blob, Tag, or Contact Selector) when you are playing a recorded script. These controls are disabled.
- Do not play a script if you have altered the interface.
- Play a script only on top of the application that the script was recorded with.
-
Click the Pause button (
) to pause the playback or to physically interact with the application. -
Click the Stop button (
) to stop recording. You must stop the script before you can load a new script.Note To play interactions by using the Surface Simulator automation API, see SimulatedContactScript.
When to Record Interactions
Surface Simulator's recording feature is useful for the following areas:
- Repeating test cases. You can record interactions to automate test cases.
- Tracking contacts. You can track contacts more easily by using Surface Simulator than by using a Microsoft Surface unit. However, you can create more realistic testing on a computer by using scripts that you recorded on the unit.
- Reproducing bugs. You can record actions that show the process of uncovering a bug. You can then easily send the script file to other developers to review.
The following procedure demonstrates an example of how you can use the recording and play features to create a test case that can be repeated:
-
Click the Record button (
) to start recording. -
Interact with the application by using Surface Simulator controls:
- Place three blob contacts on three movable pieces of content, such as photos.
- Use the Contact Selector button to select all three placed contacts.
- Drag any of the selection manipulation handles to transform the positions of the selected contacts.
- Place three blob contacts on three movable pieces of content, such as photos.
-
Click the Stop button (
) to finish and save the recording. -
Make any changes to your code and reopen your application in Surface Simulator.
-
Click the Open button (
), and then select the recorded script. -
Click the Play button (
) to run the same test case.