Unity SDK for Desktop

DOCUMENTATION

User Manual

SDK version: 3.0 Beta

Good Choice :)

Thanks for choosing Tobii! You have invested your money and curiosity in an eye tracker device, let me now guide you through the steps to set you up to develop a whole new kind of game interactions using the player’s eye-gaze.

Contents

Getting Started

Follow these steps to get started exploring the demo scenes included in the SDK.

Step 1: Install and make sure your eye tracker is working

For peripheral eye trackers only

Make sure your system fulfills the System Requirements.

Follow the instructions included with your eye tracker to download and install the correct Tobii Engine runtime software. Make sure the eye tracker and software is working correctly.

Step 2: Download the Tobii Unity SDK for Desktop

If you haven’t already done so, download the Tobii Unity SDK for Desktop from Tobii Developer Zone (http://developer.tobii.com). You need to be logged in to access the Downloads page. You can create an account for free, it only takes a couple of minutes.

Step 3: Import the SDK assets to a new Unity project

Save the Tobii Unity SDK for Desktop unitypackage file in an easy to remember place on disk. This unitypackage includes all the Tobii Unity SDK assets and demo scenes.

Open Unity, and create a New Unity Project. Select Windows as Target Platform.

Import all assets from the Tobii Unity SDK for Desktop unitypackage.

Step 4: Get to know the API and the eye-gaze data

Locate the Assets/Tobii folder in the Unity Editor Project panel. Expand it and find the DemoScenes folder. Open the 01_EyeTrackingData scene.

Add all the demo scenes to the build using ToolsAdd Tobii SDK Demo Scenes to Build menu.

Press the play button and use the scenes’ next button to go through the provided demo scenes and get an overview of the core features of the SDK and popular eye tracking features in games. See also API Overview.

↑ Back to Top

Integrate the SDK into your game

When you have explored the features of the SDK, it is time to try it out in your game. If at any point things don’t work as expected, check out the Troubleshooting page.

Section contents:

Import the assets

Make sure Windows is the selected Target Platform in the build settings. Import the Tobii Unity SDK for Desktop unitypackage to your game (the DemoScenes folder is optional).

Implement eye tracking features

To start using the TobiiAPI, add using Tobii.Gaming; to a script, type “TobiiAPI.” and browse the static functions of the API:

You might also want to explore the possibilities of the the built-in object selection using eye-gaze by adding the Gaze Aware component to some of your game objects. The following sections give some hints what to explore.

↑ Back to Section start ↑ Back to Top

Drag and drop eye tracking features to your game

You get the latest Gaze Point by calling TobiiAPI.GetGazePoint() from any script where you have added using Tobii.Gaming; at the top of the script.

In the Tobii Gaming Team, we have implemented and helped implement eye tracking features in a number of games (from AAA titles to indie games) using this basic Gaze Point data. Over the years we have made a number of iterations and improvements to the algorithms. And now, for your convenience, we are making all our best algorithms freely available for you to use in your game (as long as they are used with Tobii eye trackers). Just drag, drop, and setup/adapt to fit your game. This is probably the fastest way you can add Tobii Eye Tracking features to your game.

Here is a list of features up for grab, and their scripts are all available in the DemoScenes folder of the SDK:

↑ Back to Section start ↑ Back to Top

Make game objects Gaze Aware

The SDK also comes with basic built-in gaze to object mapping. This enables eye tracking features like:

You can make a game object Gaze Aware by adding the Gaze Aware component in the Inspector: Add ComponentEye TrackingGaze Aware.

The component’s HasGazeFocus property lets the object know if the user’s eye-gaze is focused on the object or not. Read the HasGazeFocus property in the Update loop of a Gaze Aware game object and have it react to the user’s eye-gaze by changing its behavior or appearance. Or, let your underlying game algorithms query which game object the user is looking at using TobiiAPI.GetFocusedObject(), and have the game respond to the user’s eye-gaze in some other way.

// Example script on Gaze Aware object:
// Make object spin when looked at!

using Tobii.Gaming;
using UnityEngine;
 
[RequireComponent(typeof(GazeAware))]
public class SpinOnGaze : MonoBehaviour
{
    private GazeAware _gazeAware;
 
    void Start()
    {
        _gazeAware = GetComponent<GazeAware>();
    }
 
    void Update()
    {
        if (_gazeAware.HasGazeFocus)
        {
            transform.Rotate(Vector3.forward);
        }
    }
}

Note that Gaze Aware objects might not be a good fit for your particular game genre, game mechanics, or size and amount of game objects. But it is usually at least a very convenient way of prototyping object interaction ideas.

Configure Gaze Focus layers for Gaze Aware objects

If you use the Gaze Aware component, you should set up the Gaze Focus detection layers. By default, the first time you press ‘Play’ in a game that uses some Tobii Unity SDK feature, default settings for Gaze Focus are created. The default settings selects all existing layers to be tested for Gaze Aware objects. If a new layer is added to the game at a later point, and you want it to be tested for Gaze Aware objects, the layer has to be manually checked in the Gaze Focus layers dropdown menu. (Note that UI elements like buttons and labels do not work with the Gaze Aware component since the feature only works with objects with a UnityEngine.Collider).

Open the Eye Tracking Settings window from the Edit menu: EditEye Tracking Settings...

Click to open the Gaze Focus Layers multi-selection dropdown and make sure that all the layers are selected where you want Gaze Aware game objects to be focusable using eye-gaze. Also make sure that layers you do not want to track with eye-gaze are not selected (like transparent layers).

↑ Back to Section start ↑ Back to Top

Build for standalone

The Tobii Unity SDK assets include an editor script for copying the correct plugin dll’s to the build output folder. The Tobii Unity SDK features are currently only available and working for PC (Windows 10/8.1/7), but the framework compiles on Mac and Linux as well (without copying the dll’s).

↑ Back to Section start ↑ Back to Top

API Overview

It might be tempting to skip this part of the documentation, but I promise you that it will save you time later to at least browse through it.

This section gives an overview of the features available in the Tobii Unity SDK. It introduces the core concepts and gives you insight into what tools and features the SDK has to offer you, and how they work.

For API details on each class and function, see the Tobii Unity SDK Scripting Reference.

Section contents:

TobiiAPI

The static functions of the TobiiAPI class is the essential API of the Tobii Unity SDK.

Include the line

using Tobii.Gaming;

in a script anywhere in your game and you will have access to the static API functions.

Here are some of the API functions:

You can read more about gaze point data, head pose data, user presence, and focused object in the sections below.

Check validity on all data points

The TobiiAPI is implemented using so called ‘lazy initialization’. This means that the API is not initialized until the first call to any of its functions. Because of this, the first time the functions are called they will return values that are invalid (IsValid is false). Depending on how many game loops it takes to initialize the underlying framework, there will be a number of frames where invalid data is returned.

Gaze Point data and Head Pose data can be explicitly initialized by calling TobiiAPI.SubscribeGazePointData or TobiiAPI.SubscribeHeadPoseData respectively. If these functions are called during some appropriate time, during game startup for example, valid data can be available on the first calls to TobiiAPI.GetGazePoint() and TobiiAPI.GetHeadPose().

In most cases it’s also very important to know that you are working with recent data. TobiiAPI.GetGazePoint() and TobiiAPI.GetHeadPose() will return the latest data received from the eye tracker but that can be old. For example, if a user looks away from the monitor. We suggest using IsRecent() method to ensure that data is both valid and recent.

↑ Back to Section start ↑ Back to Top

Gaze Point data

GazePoint is the data type returned by TobiiAPI.GetGazePoint(). It represents the point on the screen where the user is looking. More technically it is the point on the screen where the eye tracker has calculated that a line along the user’s eye-gaze intersects with the screen plane.

GazePoint.Viewport returns the gaze point in Viewport coordinates, where (0,0) is the bottom-left and (1,1) is the top-right corner of the UnityEngine.Screen. The values can be higher than 1 and lower than 0 if the user looks a bit outside the physical bounds of the monitor.

GazePoint.Screen returns a Vector2 (x,y) coordinate on the UnityEngine.Screen. This might seem pretty much like getting a mouse pointer coordinate, but please do not make the mistake of using this point out-of-the-box as you would use a mouse pointer coordinate. Eye tracking data is not as precise as mouse pointer data - it is actually physically impossible due to how our eyes and seeing works. Instead, think of a series of GazePoints as representing an area where the user is looking, and take into account that the accuracy and precision of the data varies from user to user.

↑ Back to Section start ↑ Back to Top

Head Pose data

HeadPose data can be used to complement GazePoint data to further improve infinite screen experiences like Extended View.

HeadPose is the data type representing the position and orientation of the user’s head. Position is being measured in millimeters relative to the middle of the screen eye tracker is mounted on. The position is the point in space around which the head is rotated.

Rotation represents the rotation of the head of the user, expressed using Quaternion. Use eulerAngles property of the Quaternion to convert it to euler angles.

↑ Back to Section start ↑ Back to Top

User Presence

User Presence is a state that indicates if a user is present in front of the eye tracked screen or not. This state can be used to for example pause some feature if there is no user present.

↑ Back to Section start ↑ Back to Top

Gaze Focus and GazeAware component

The Tobii Unity SDK has built-in support for mapping eye-gaze data to game-objects. We call this feature Gaze Focus. Rather than just mapping every gaze point to a game-object the idea is that an object that has Gaze Focus is intentionally focused by the user. To do this, we are under the hood using algorithms that take series of gaze points, the history of focused objects and timings in human perception into account.

These gaze to object mapping algorithms were introduced in the 2.0 version of the SDK. The plan is to refine the algorithms continually in future versions of the framework. The goal is to have a fast and robust calculation that works for different eye tracker models with different characteristics in accuracy and precision, but also over a broad set of different eye-gaze characteristics of end-users.

The Gaze Focus system only maps game objects that gaze focusable. In order to make a game object gaze focusable you need to add the GazeAware component to it. This component will register the game object as Gaze Aware at OnEnable and unregister it at OnDisable. In the game object’s Update loop the GazeAware component’s HasGazeFocus property can be read to know if the game object is focused or not.

For game algorithms outside of the individual game objects, it is also possible to ask the gaze focus handler which object is currently focused using the static API function TobiiAPI.GetFocusedObject(), and have the game respond to this information. Only one object (or no object) is considered to have gaze focus at the time.

As a developer you should use Gaze Focus as it is out-of-the-box, since any additional filtering done on top of the gaze focus calculation might be a bad fit when the algorithms are refined in future releases. Highlighting of objects and visualisations should use timings related to human perception rather than be adapted to a specific gaze focus algorithm, eye tracker model or a specific user’s characteristic gaze-tracking.

Gaze Focus is only available for 3D game objects with a UnityEngine.Collider. It does not work with UI elements on canvases.

↑ Back to Section start ↑ Back to Top

System Requirements

Version 3.0 of the Tobii Unity SDK for Desktop requires the following setup on your computer:

The code samples in the SDK and the code snippets here in the documentation are all written in C#. It is also possible to write your scripts in UnityScript, if you prefer.

For download links and more information about other Tobii SDK’s visit the Tobii Developer Zone (http://developer.tobii.com).

↑ Back to Top

Appendix: Unity Editor Howto’s

Set Windows as Target Platform

In the Unity Editor menu, select: EditBuild Settings....

Select the following settings:

Import the SDK assets

In the Unity Editor menu, select: AssetsImport AssetsCustom package...

↑ Back to Top