Microsoft Surface 1.0 SP1: SDK - Tagged Objects - TechNet Articles - United States (English) - TechNet Wiki

Table of Contents



Overview

The Microsoft Surface Vision System captures and processes raw images of a Microsoft Surface screen and can "see" the actual outlines of physical objects that are placed on the screen by using cameras that operate in the near infrared light (specifically, 850 nm).

(For an example of how to see object outlines, see Capturing and Rendering a Raw Image.)

However, if you want your Microsoft Surface application to be driven by objects, the Microsoft Surface Vision System also recognizes special tagged objects that are marked with a special pattern of dots called tags. A tag consists of a geometric arrangement of infrared reflective and absorbing areas. Tagged objects give your application several advantages over processing raw images:

Microsoft Surface supports two types of tags:

Your application's needs determine which type of tag that you should use. 

When you use tagged objects, you can use as many instances of the same tag as an application requires. However, if a user moves a tagged object too fast, the Microsoft Surface Vision System cannot track the object successfully. If you're writing a Microsoft Windows Presentation Foundation (WPF) application, you can create an on-screen visual object that tracks the tag motion by using the TagVisualizer control.

↑ Return to Top


How to Use Tagged Objects

A Microsoft Surface application can use tagged objects in several ways:

Because identity tags have a greater range of possible values, they also enable additional scenarios:

noteNote
Any application that uses tagged objects should register the tags that it uses. For more information, see Registering a Microsoft Surface Application.

 

↑ Return to Top


Security Considerations for Tagged Objects

Do not use tagged objects to access private data. Tags are not sufficiently secure to use as a method of authentication. You can use tagged objects to identify a user, but you should require the user to enter a password, personal identification number (PIN), or other secure data for authentication purposes.

When you use tags, consider the following important security issues:

Overall, we recommend that you can use tags for identification but you should not use them authentication. An identity tag is simply a way to enter shorthand, intended to save you from having to type something into a text box. It's the equivalent of having a hyperlink that you can click, instead of forcing you to enter a URL, or the equivalent of pressing CTRL+R to respond to an e-mail, instead of entering the recipient's address manually. The contents of an identity tag are not more secure than a URL or an e-mail address.

↑ Return to Top


Byte Tags

Byte tags are one type of tagged objects that Microsoft Surface recognizes. These tags store 8 bits of data (1 byte), so there are 256 unique tag values. (In contrast, identity tags store 128 bits of data.)

An example byte tag

The value of a byte tag is represented in code by a ByteTag structure. The Value property represents the value of the tag that your application can use to interpret the physical object. The Contact.Tag.Byte.Value property stores the value for a given Contact.

There are 256 possible byte tags, each with an 8-bit value and orientation markers.

↑ Return to Top


Byte Tag Geometry

Each byte tag that the Microsoft Surface Vision System identifies has an orientation and an 8-bit tag value. The following figure illustrates the basic tag that all other tags are based on.

Byte tags - Parts labeled
  1. Infrared-absorbing background

  2. One infrared-reflecting circle (0.125-inch radius) in the center of the tag. This circle locates the tag on the Microsoft Surface screen.

  3. Three infrared-reflecting circles (0.08-inch radius) located 0.28 inches from the center of the tag in each direction (left, right, and down). These "guide" circles determine the tag orientation.

  4. Return to Top


    Byte Tag Geometry

    Each byte tag that the Microsoft Surface Vision System identifies has an orientation and an 8-bit tag value. The following figure illustrates the basic tag that all other tags are based on.

    Byte l>
<p>In addition, each tag contains from zero to eight data bits that define the tag value. These data bits are white circles (0.075-inch radius) that are centered in one of the following locations:</p>
<ul>
<li>0.3058 inches horizontally and 0.2038 inches vertically (bits 1, 2, 5, and 6) from the center of the tag.<br />
<br />
</li><li>0.2038 inches horizontally and 0.3058 inches vertically (bits 0, 3, 4, and 7) from the center of the tag.<br />
<br />
</li></ul>
<p style=↑ Return to Top


    Byte Tag Values

    There are 256 possible unique byte tags that are encoded as a byte. The highest order bit (bit 7) is at the 1 o'clock position when you look at the printed side of the tag. Less significant bits are then read counter-clockwise from the 12 o'clock position. An infrared-reflective circle in a bit location represents 1. The absence of a circle represents 0 (zero).

    Byte tags - Bit locations

     

    Tagged objects - 0x00 / 0xFF / 0xC1 / 0xC6 values

    ↑ Return to Top


    Printing Byte Tags

    Microsoft Surface units include two full sets of byte tags (256 tags in each set). You can also create and print your own tags, according to the specifications in this article, up to the maximum of 256 unique patterns.

    If you want to create and print your own byte tags, you have some freedom when you design and work with the physical tags. The following illustration shows the dimensions of a physical tag.

    Byte tags - Dimensions

    The undefined region indicates where you can extend the geometry of the tag, if you need to.

    The tag region is the opaque black-and-white patterned area of the actual tag. You can remove or cut a 0.125 × 0.125 inch triangle from each corner of the 0.75 × 0.75 inch square tag without affecting performance, if you need to affix the tag to a smaller circular object. However, the entire tag region must touch the Microsoft Surface screen.

    ↑ Return to Top


    Identity Tags

    Identity tags are one type of tagged objects that Microsoft Surface recognizes. These tags store 128 bits of data (two 64-bit values), so there is an effectively unlimited range of unique tag values. (In contrast, byte tags store 8 bits [1 byte] of data.)

    An example identity tag

    The value of an identity tag is represented in code by an IdentityTag structure. The IdentityTag structure consists of two properties, Series and Value, which are 64-bit numbers.

    An identity tag can represent a much greater range of values that an byte tag. This greater range of possible values enables the following possible scenarios:

    ↑ Return to Top


    Creating Identity Tags

    The Printing Tagged Objects section describes general requirements for creating and printing tagged objects.

    If you want to create and print your own identity tags, you can use the Identity Tag Printing Tool (GenTagUI.exe) to create identity tags by using a regular printer. You can also create identity tags by using the IdentityTagGenerator class in your own application.

    After you create an identity tag, attach it to something solid, like a plastic card or heavy cardboard, because the tag must be perfectly flat for the Microsoft Surface Vision System to detect it. The black ink that is printed on an identity tag must be as dark as possible for the correct contrast between the white dots and the black background.

    The identity tag printing tool enables you to generate a new series (Series) and values (Value) within that series. For example, consider a scenario like a promotion where Series represents a code for the promotion itself and Value represents individual offerings or even individual customers that the promotion targets.

    After you create the Series and Value value, you can copy and paste them into your database or application code in XAML or C# format. You can also print the actual tag graphic to a black-and-white printer.

    ↑ Return to Top


    Designing Applications for Tagged Objects

    Tagged objects provide many new scenarios for Microsoft Surface applications. However, when you create an application that is driven by tagged objects, there are unique design considerations to remember.

    For an example of an application that is designed for tagged objects, see the Item Compare sample application. For a tutorial about how Item Compare is designed as a tag-driven application, see Designing Tag-Driven Applications for Microsoft Surface.

    ↑ Return to Top


    Choosing a Tag Type

    Microsoft Surface supports two types of tags: byte tags and identity tags. Your application's needs determine which type of tag that you should use.

    Because byte tags are more compact and are faster for the Vision System to recognize, they represent the best choice for an object-driven application if the number of unique tag values that you need is relatively small (for example, identifying pieces in a board game). However, byte tags have only 256 possible values, so they may not be appropriate for some applications.

    If your application must distinguish among a large number of unique values (for example, if the tag represent a unique customer ID from a large database), identity tags are the best choice.

    ImportantImportant
    Although you can use identity tags to provide logon credentials, you should not use them as the only means of authentication because malicious users can copy or guess tags. Any authentication scheme that is based on tag values should also require another form of information, such as a PIN or password.

     

    ↑ Return to Top


    Reacting to Tagged Objects

    If you are creating an application based on the Presentation layer, your application can respond to tagged objects most easily by using the TagVisualizer control. This control can recognize, track, and show user interface (UI) elements for tags that are placed on the control.

    If you want to use custom processing (or if you are writing an application based on the Core layer), your application can respond to tagged objects by watching for Contact objects whose IsTagRecognized property is set to true. When that property is true, you can use the Tag property on the contact to get information about the type and value of tag that has been recognized.
     

    ↑ Return to Top


    Starting Applications with Tagged Objects

    A user's first experience with your application is how to start it. If your application is designed to be driven primarily by tagged objects, consider integrating it with the object routing feature. Object routing enables users to start your application without going through Launcher. Ideally, after a user opens your application by using a tagged object, the application will use the tag value in some way.  
     

    ↑ Return to Top


    About Printing

    A Microsoft Surface application relies solely on the orientation and position of a tag (instead of the physical object that the tag is attached to), so the Microsoft Surface platform needs to know where the tag is located on the physical object the tag is placed. The location enables the platform to infer the location of the object based on the location of the tag. Depending on the physical requirements of your tagged object, you might need to place the tag somewhere other than its center. In such a case, you should correct for the location of the tag so your application can show the UI for the tag in the appropriate position.

    If you use the TagVisualizer control, you can use the PhysicalCenterOffsetFromTag and OrientationOffsetFromTag properties of the TagVisualization object to correct for tag placement.

    For more information about how to create and print tags for the physical objects, see Printing Tagged Objects below. For information about how to create and print identity tags, see Identity Tag Printing Tool.


    ↑ Return to Top


    Printing Tagged Objects

    Microsoft Surface units include two full sets of byte tags (256 tags in each set). You can also create and print your own byte tags and identity tags.

    In addition to the information in this topic, review the specifications for each tag type in the Byte Tags and Identity Tags sections above. If you are printing identity tags, you should also use the identity tag printing tool.

    Identity Tag Printing Tool.


    ↑ ↑ Return to Top


    Reflectance

    Make sure that printed tags meet the following requirements:

    A single tag on a physical object with a black or non-IR reflective background provides the best tag recognition performance.

    You can measure reflectance for surfaces with diffuse reflectance by using a reflectometer that has response in the near IR wavelength range (for example, the InspectIR-Vis instrument from Surface Optics Corp).
     

    ↑ Return to Top


    Materials and Placement

    Whenever possible, place tags on infrared-absorbent objects. A tag on an object that is infrared-absorbent typically performs better than a tag on a highly infrared-reflective item. If you place a tag on a highly infrared-reflective surface, the Microsoft Surface Vision System might have trouble recognizing the tag.

    noteNote
    The infrared reflectivity of an object is not necessarily the same as its brightness to the naked eye. For example, some materials that appear dark to the eye can actually be reflective to infrared light.

    You must place tags on a solid, consistent background (for example, with no patterns or stripes). Use a dark-colored or non-IR reflective background instead of a light background.

    Place tags on flat surfaces that help the tag maintain flush contact with the tabletop. Also, tags work better on physical objects that do not easily tip over while they move.

    The following materials and processes have been validated and shown to produce tags that function correctly with Microsoft Surface. If you use these materials, your tags should work properly. However, always test new tag types in their final format before you distribute them.

    ↑ Return to Top


    High IR-Reflective/White

    In general, most white paper works well. However, some are too bright and can cause a blur in the Microsoft Surface Vision System.
     

    ↑ Return to Top


    Low IR-Reflectance/Black

    ↑ Return to Top


    Top coats and protective layers

    In practice, top coats such as UV print varnish and laminates can successfully improve a tag's durability and minimally affect the optical properties of the tag. However, many types of protective layers can significantly affect optical performance. You should test any protective layer that you add to a tag to make sure it performs correctly.
     

    Low IR-Reflectance/Black

    ↑ ↑ Return to Top


    Printers

    Ink jet printers that use a mix of cyan, yellow, and magenta to create black generally do not work. The three-color combination is typically invisible in the infrared spectrum. You should test all printing methods for their appearance in the infrared 850 nm wavelength.

    Be careful that the tags cannot be rubbed off. Tags that are printed by using ink jet or laser printers wear off rapidly when they are used. If the tag printing wears off, you cannot use the tag anymore and the tag will smudge the Microsoft Surface screen. The tags that are included with the unit are printed on self-adhesive vinyl to protect them from wearing off.

    noteNote
    Do not use the Microsoft XPS Document Writer to create an .xps file for later printing of the identity tag. The XPS Document Writer introduces light gray lines that outline the small hexagonal areas of the tag. These lines can prevent the Vision System from detecting the tag.

    For more information about how to print identity tags, see Identity Tag Printing Tool.
     

    ↑ Return to Top


    Using Object Routing with Tagged Objects

    Typically, users open Microsoft Surface applications through Launcher. However, Microsoft Surface also includes object routing, which enables you to configure tagged objects that open your applications without using Launcher.

    ↑ Return to Top


    Tagged Objects and Applications

    When a tagged object is placed down on a Microsoft Surface screen, the Microsoft Surface software reads the object's tag value and checks if the tag is registered with Surface Shell.

    Note
    Even if your application does not use object routing, you should register tags that your application uses to prevent interference from applications that do use object routing.

    The Microsoft Surface software also checks whether any tag is registered for object routing. If a tag is registered for object routing, the Microsoft Surface software displays a menu that includes all of the applications that are registered for object routing with that tag.

    When you design an application that works with object routing, we strongly recommend that your application use the tag value for some clear purpose after users open the application from the object routing menu. Strong designs provide immediate, obvious information to the user based on that tag. Users should instantly see what options are available and how to proceed based on the tag that they placed on the screen. For example, the tag value could correspond to a particular promotion or set of user preferences.

    You can associate tagged objects with your application by using the application's XML registration file. For more information, see Registering a Microsoft Surface Application.
     

    ↑ Return to Top


    Object Routing UI

    If a user places a tagged object that is registered for object routing on a Microsoft Surface screen, the Microsoft Surface software displays the object routing UI, as the following illustration shows.

    Object routing - Example

    The object routing UI displays a menu of applications that users can open because they are registered with the tagged object. If an application does not have an icon image, the object routing UI displays a default icon.

    The object routing UI displays on top of all other elements in the experience except the access points, but it does not prevent users from interacting with the current application. However, the current application does not interact with the tagged object that is displaying the object routing UI.

    The Microsoft Surface software automatically sizes the object routing UI based on the tagged object's bounding box. For more information about the bounding box, see the Specifying Physical Attributes of Tagged Objects section below.

    noteNote
    We recommend that you register up to four applications for object routing with each tag.

    If the object routing UI for two tags appears too close to each other on a Microsoft Surface screen, they repel each other. (This behavior avoids confusion that could occur if the UI elements overlapped.)

    Unlike other UI elements, users cannot push an object routing menu beyond the edge of the screen, so all application icons in the menu are always visible.

    When a user starts an application by using the object routing UI, the orientation of the tagged object determines the orientation for the application.

    The object routing UI disappears when a user opens an application that is associated with the tag. The tagged object then interacts with the newly opened application. The object routing UI also disappears if the tagged object is removed from the unit.
     

    ↑ Return to Top


    Setting Up Object Routing

    To configure object routing for an application, you must do the following:


    ↑ Return to Top


    Specifying Physical Attributes of Tagged Objects

    When a user places a tagged object on a Microsoft Surface screen to interact with applications by using object routing, the Microsoft Surface software displays a shadow for the object, a text label, and icons to start or display applications.

    For each tagged object's value that you want to configure for object routing, you must define the tagged object in the registry of the Microsoft Surface unit. The values must be in hexadecimal format. Maintain the values for each defined byte tag value and identity tag series in the following registry locations:

    The following table describes the registry settings that you must define to describe the values of the tagged objects.

    Key Description

    BoundingBoxWidth

    The width of the bounding box, in inches. (For more information, see the following "Bounding Box" section of this topic.)

    BoundingBoxHeight

    The height of the bounding box, in inches. (For more information, see the following "Bounding Box" section of this topic.)

    PhysicalCenterOffsetFromTagX

    The offset of the bounding box center from the tag center, in the x-direction. (For more information, see the following "Horizontal and Vertical Offsets" section of this topic.)

    PhysicalCenterOffsetFromTagY

    The offset of the bounding box center from the tag center, in the y-direction. (For more information, see the following "Horizontal and Vertical Offsets" section of thi

    BoundingBoxWidth

    The width of the bounding box, in inches. (For more information, see the following "Bounding Box" section of this topic.)

    Bouns topic.)

    OrientationOffsetFromTag

    The offset of the bounding box orientation from the tag orientation, in degrees. (For more information, see the following "Orientation Offset" section of this topic.)

    Text

    The text to appear under the object.

    The following illustration demonstrates the following parts of a tag on a tagged object:

    1. The offset of the bounding box center from the tag center, in the x-direction (PhysicalCenterOffsetFromTagX)

    2. The offset of the bounding box center from the tag center, in the Y-direction (PhysicalCenterOffsetFromTagY)

    3. The center point of the bounding box

    4. The center point of the tag

    5. The offset of the bounding box orientation from the tag orientation, in degrees (OrientationOffsetFromTag)

    Tagged objects - Physical attributes labeled

    You can translate (localize) the text that appears under a tagged object and include a separate registry key for each locale that you support. For example, the following registry key includes text for France:

    If you do not specify a localized version for a locale, the default text is displayed.

    For more information about how to create the physical tags, see Printing Tagged Objects.
     

    ↑ Return to Top


    Bounding Box

    To determine the size of the object routing UI, the Microsoft Surface software requires the size of the rectangle around the object, called the bounding box. The center point of the bounding box serves as the anchor for the tethers that connect the menu items to the tagged object in the object routing UI.

    The bounding box is represented with values, in inches, for the BoundingBoxWidth and BoundingBoxHeight registry values.
     

    ↑ Return to Top


    Horizontal and Vertical Offsets

    You can place a tag anywhere on the relevant surface of a tagged object. It does not have to be centered within the bounding box. For the Microsoft Surface screen to accurately reflect the position of the object, you must calculate the offset from the center of the tag to the center of the tagged object's bounding box.

    Tagged objects - Offsets examples

    Offsets are represented in inches from the center of the tag to the center of the X or Y axis of the bounding box.

    ↑ Return to Top


    Orientation Offset

    You can also place a tag in any orientation with respect to a tagged object. For the Microsoft Surface screen to accurately reflect the position of the object, you must calculate the offset between the up orientation of the tag and the up orientation of the object's bounding box.

    Orientation offsets are represented in degrees from the orientation of the tag.

    ↑ Return to Top


    Installation Co href="http://social.technet.microsoft.com/wiki/contents/articles/8028.microsoft-surface-1-0-sp1-sdk-tagged-objects/edit.aspx#Top">Return to Top


    Orientation Offset

    You can also place a tag in any orientation with respect to a tagged object. For the Microsoft Surface screen to accurately reflect the position of the object, you must calculate the offset between the upnsiderations

    Ideally, when you develop your application, you should also create an installer that installs the application, creates the correct entries in the ApplicationName.xml registration file, and creates and sets the appropriate registry keys.

    The installer should also check the registry for registered tagged objects. If the value or series of a tag has been specified in the registry, enable the administrator to overwrite the registry values or cancel the installation.

    noteNote
    Because byte tags have fewer available values, the possibility of registry conflicts is greater. The possibility of identity tag collision when the tag values are generated randomly is close to zero.

     

    Automatically Registered Applications

    ↑ Return to Top


    When you start your application from Microsoft Visual Studio or if start it by running the executable file in administrator mode, the application is automatically registered with Surface Shell. However, applications that you open in this way cannot use object routing unless you have previously registered them with Surface Shell. For more information about how to register an application, see Registering a Microsoft Surface Application.

    ↑ Return to Top