text: This is the sixth post in the series of Holographic App Development Using Microsoft HoloLens. In one of my previous post, we have discussed How to build a Holographic Apps with Unity 3D and Microsoft HoloLens. Now let’s try to interact with the same holographic app we made so far. To start with, we will focus on Gaze Input. Like we use a mouse to move a cursor, it is now Gaze for the HoloLens. Gaze is one form of input on HoloLens. Gaze Input in HoloLens indicates where the user is currently focusing. A cursor is used to indicates the direction of the Gaze. The cursor follows the point of attention as users heads move in the scene.
  To determine the point of user’s attention or user currently focusing on, each frame does a raycast into the scene and place cursor on the object where the raycast is intersected with the objects. The same formula goes even when the user is moving along with HoloLens, and thus the focus is maintained.
  Watch this Video to get a quick glance on what is Gaze Input and how it works.
  Just to summarize, here is the list of points that were highlighted in this video.
  - Cursor Indicates the Gaze Direction.
  - Raycast determines point of attention.
  - Cursor can hug surfaces it intersects.
  - Non-directional cursors often work best.
  - Arrow can lead user to action.
  - Cursor can be temporarily hidden.
  To start with Adding a Gaze cursor into our application, we will refer to the previous developed 3D Holographic application.. from a bit early stage and HoloToolkit – Unity
  Developing Holographic Apps with Unity 3D and Microsoft HoloLens
  Start Using HoloToolkit – Unity
  The HoloToolkit – Unity is the repository that contains all Unity specific components and scripts to make development faster. The HoloToolkit is a collection of scripts and components intended to accelerate development of holographic applications targeting Windows Holographic…
  HoloToolkit contains the following feature areas:
  - Input
  - Sharing
  - Spatial Mapping
  - Spatial Sound
  - Utilities
  .. Read More about HoloToolkit Here …
  Including HoloToolkit in to the Unity Solution
  You can include the HoloToolkit – Unity, easily in to your Unity Project.
  - Download the Toolkit from
  - Unpack and Place it inside the Asset Folder of the Unity Project.
  Adding a Gaze Input Cursor
  To add the Gaze Input Cursor from HoloToolkit we need follow two steps
  - Placing the Gaze Manager Script
  - Placing the Cursor in the Scene
  Placing the Gaze Manager Script
  The Gaze Manager script used to calculates the Raycast hit position and normal. If the Raycast hits a hologram, it set the position and normal to match the intersection point.
  Add an Empty GameObject in the Scene, and named it as “GameRoot”
  Navigate to HoloToolkit –> Input and drag the GazeManager.cs script on to GameRoot object in the object Hierarchy.
  Once the Script is added, you should be able to see this as a components in the Object Inspector.
  Placing the Cursor in the Scene
  Cursor is the Visual Indication for the Gaze. When we Raycast hits the Hologram, we need to display this cursor to indicate the gazed object.
  Navigate to HoloToolkit –> Prefabs –> Input, and add the cursor into the object hierarchy
  At this point of time if you inspect the Cursor Object, you will find “Cursor Manager” is already attached with it.
  CursorManager class takes Cursor GameObjects and shows the appropriate Cursor when a Hologram is hit. This hit is calculated by the GazeManage Raycast.
  That’s it for now ! We can run and test the cursor appearance when Gaze.
  Run and Test the Gaze Input Cursor
  Run the application and you should be able to view the cursor as soon as focus is on any of the Cube. When there is no focused object, you won’t see the cursor appearing.
  Behind the scene… How it works ?
  If you want to check what is happening in behind, the entire things are managed by the Gaze Manager and Cursor Manager scripts. On each frame updates, the Update() method for Gaze Manager is invoked, which calling the UpdateRaycast() method to find the Raycast hit information. When Raycast hot the Hologram, LateUpdate() method from the Cursor Manager is invoked, that takes control of showing and hiding the cursor.
  Gaze Handler – Adding Visual Effects When Gaze Enter / Exit
  Let’s take it a step forward and giving a Visual effects on the object when it is being gazed.
  Add a new Script and name it as “’Gaze Handler” and have it as a Singleton implementation.
  Add Two method OnGazeEnter() and OnGazeExit() respectively for Gaze Enter and Gaze Exit.
  Now, let’s try to change the color of the object as soon as Gaze enter, and revert to the original color when Gaze off. Following code snippet are just trying to get the material color used for objects and updating with a fixed (yellow) color and when Gaze Exit, the previously stored color is getting reverted.
  Here is how the entire script class looks like…
  public class GazeHandler : Singleton<GazeHandler> { private Color startColor; // Use this for initialization void Start() { } // Update is called once per frame void Update() { } void OnGazeEnter() { var com = gameObject.GetComponent<Renderer>(); startColor = com.material.color; com.material.color = Color.yellow; } void OnGazeExit() { var com = gameObject.GetComponent<Renderer>(); com.material.color = startColor; } }
  Now refer to the script over here Git repository / Or Explore it in Visual Studio –
  You will find how the Gaze manager is checking the change of current Hit Object. Just update the method name that we write recently for Gaze Exit and Gaze Entry.
  Add this script as components to all the cube object.
  That’s it.
  Now, as soon as an old object is changed and the new object gets focus, the Gaze Enter and Exit will start invoking. Here is a complete set of code map that shows the entire flow starting from Raycast to On Gaze Enter.
  Here is how it look like – Video
  HoloToolkit is very useful and can make your HoloLens App development very fast ! Start exploring it!
  Hope this help !
  I followed your steps but its not working in my case. Few differences i found in Hololens kit and your script. For instance under gazeManager we SendMessageOptions.RequireReceiver in your sample but in Hololens kit its showing SendMessageOptions.DontRequireReceiver.
  I’m bit confuse here.
  Would you please help?
  Hi Amit, What is not working ? Are you able to see the cursor ? did you tried with breakpoint and see if the script are getting called as expected.
  With Respect of SendMessageOptions, the Requirer Receiver is used when there is a listener. In the next article I have created my own handler to handle the gaze. you may look at that as well. Please do let me know if that resolve your issues. Or else, I can help you further. Thanks again for asking.
  Cheers !
  I also followed your steps. My Cube color is changing to yellow when I gaze over the object. But when I leave, the color still remains yellow. I debugged the code and I saw that the color in “starColor” ist always the same. Even after changing the the color of the cube in unity. So i guess its saving the yellow color. Do you know what I am missing?
  Thanks
  Kevin
  Hello Kevin, at OnGazeEnter() method, could you please check what value are you receiving in the Start Color for first time ?
  var com = gameObject.GetComponent();
  startColor = com.material.color;
  hello,
  I’m very new in unity and hololens .
  There’s no cursor manager inside the holotoolkit assets, and when I want to just importing the “cursor manager”script from academy tutorial, an error 0246 occurs.
  where’s the cursor manager? And how to add gaze without this script?
  I’m expierencing the same problem, There is no script called “cursor manager” in the toolkit.
  I replied. Please have a look and let me know if that helps !
  Thanks !
  With the new version of HoloToolkit, you don’t required Cursor manager. Have a Empty Root object, Add Input Manager, Gaze Manager and Gaze Stabilizer.
  Add your 3D Object
  Add Cursor Prefabs from the Input > Prefabs
  Run the application, You should start seeing the Cursor.
  For Gesture, Use below interface and implement them in the class.
  NewBehaviourScript : MonoBehaviour, IFocusable, IInputClickHandler
  Let me know if you still have problem.
  It works!!!Thank you! I can see the cursor on and off my assembly!
  Thank you!
  I still have problem with gestures. My task is very basic and simple, I got a solidworks assembly, the effect what I need is, I can see the assembly hologram and interact with it, like move, rotate, explode the parts, etc in my hololens. Just like the effect in academy 210 and 211:”Interactible(choose different of assembly)”,”Navigation(rotate assembly in every direction)”,”Manipulation(move assembly)”,”Model expansion(expand the parts of assembly)”.
  For now, I know how to transfer the assembly to unity, and with your help I can add gaze input. But you said:”For Gesture, Use below interface and implement them in the class.
  NewBehaviourScript : MonoBehaviour, IFocusable, IInputClickHandler” confused me, it’s totally different between the academy tutorial and latest holotoolkit, As a very new beginner, I don’t know what to do with the holotoolkit.
  Thank you again!!!