Abstract
Transcranial Magnetic Stimulation (TMS) is a non-invasive procedure that uses time varying short pulses of magnetic fields to stimulate nerve cells in the brain. In this method, a magnetic field generator ("TMS coil") produces small electric fields in the region of the brain via electromagnetic induction. This technique can be used to excite or inhibit firing of neurons, which can then be used for treatment of various neurological disorders such as Parkinson's disease, stroke, migraine, and depression. It is however challenging to focus the induced electric field from TMS coils to smaller regions of the brain. Since electric and magnetic fields are governed by laws of electromagnetism, it is possible to numerically simulate and visualize these fields to accurately determine the site of maximum stimulation and also to develop TMS coils that can focus the fields on the targeted regions. However, current software to compute and visualize these fields are not real-time and can work for only one position/orientation of TMS coil, severely limiting their usage. This paper describes the development of an application that computes magnetic flux densities (h-fields) and visualizes their distribution for different TMS coil position/orientations in real-time using GPU shaders. The application is developed for desktop, commodity VR (HTC Vive), and fully immersive VR CAVE TM systems, for use by researchers, scientists, and medical professionals to quickly and effectively view the distribution of h-fields from MRI brain scans.
INTRODUCTION

Transcranial Magnetic Stimulation (TMS)
Transcranial Magnetic Stimulation (TMS) is a non-invasive neuromodulation technique that uses time varying short pulses of magnetic field to induce an electric field in the conductive tissues of the brain, thus modulating the synaptic transmission of neurons. This neuromodulation technique can be used to excite or inhibit the firing rate of neurons in the targeted regions of the brain. TMS technique has shown beneficial effects for the treatment for various neurological disorders such as major depressive disorder, Parkinson's disease, post-traumatic stress disorder and migraine 1, 2, 3, 4, 5 . It is currently FDA approved for depression 6 and the treatment costs are typically reimbursed by major medical insurance providers. TMS is a safe and economical treatment that can be administered during an out-patient visit hence, it is becoming an attractive treatment for depression. There are many multi-site clinical trials on going on the development of TMS treatment for other neurological disorders. Evidence also suggests that the exposure of the brain to pulsed electromagnetic fields enhances "brain plasticity" that can be exploited for the treatment of stroke rehabilitation 7 . TMS can be used to assist brain to adapt and take on functions in undamaged parts of the brain that were originally undertaken by the damaged parts of the brain.
Why are simulations of TMS necessary?
One of the persistent and prevalent challenges for TMS is to improve stimulation focality and penetration depth 8 . These two factors typically show tradeoff characteristics, where one preference sacrifices the other 9 . Standard single hoop and figure-eight TMS coils can stimulate only a surface area ranging from 1 -2 cm 2 , depending on various factors 10 . Also, electric fields induced by TMS decrease rapidly with distance (inversely varies as cube of distance), and hence their effect is generally limited to the surface of cerebral cortex, cerebellum, and spinal cord structures. Stimulation in smaller and deeper regions of brain is typically challenging to achieve, and literature exists showing promise for deeper penetration and focality of fields with better coil designs 11, 12, 13, 14 . These coil designs must thoroughly be tested to ascertain their viability for either mice or human clinical trials, making numerical simulations and interactive visualization a necessary prior step in advancing their development.
BACKGROUND

Mathematical modeling of TMS
TMS can be modeled mathematically using fundamental laws of electromagnetism. Specifically, Biot Savart's law can be used to determine magnetic flux densities (commonly, h-fields) and induced electric fields (e-field), for a given coil with electric current produced from a pulse generator and a constant air permittivity. Most commonly, mathematical modeling of TMS is achieved through finite difference methods 15, 16 .
Mathematical modeling of TMS
Induced electric fields, e-fields: When time varying magnetic fields are applied on the brain, an electric field is induced in the conductive parts of the brain by Faraday's second law of induction. These are dependent on the electrical conductivities of different brain organs. To numerically compute e-fields, conductivity values must be assigned to different brain organs. This means that computing e-fields will pre-require that the brain organs for a given dataset are distinguishable and segmented. However, tissue densities in brain organs vary only slightly making acceptable quality segmenting computationally and time intensive. Typical imaging equipment such as CT and MRI do not automatically distinguish and segment different brain organs in their scans. Hence, images sourced from such equipment need to be post-processed using algorithms such as marching cube to segment out the organs and are well documented 17, 18, 19 . Images sourced from MRI are typically preferred for this over CT because of their ability to capture subtle differences in tissue densities.
Mathematical modeling of TMS
Tissues that are segmented through post-processing of MRI images are assigned electrical conductivities, which can then be used in numerically computing e-fields. However, the intermediate step of determining an acceptable quality brain organ segmentation is hence typically not real-time and could take up to 6-10 hours 20 with current day technologies. As such, computing e-fields from an MRI dataset is not a real-time prospect quite yet.
Mathematical modeling of TMS
Magnetic flux densities, h-fields: These are not the parameters that directly determine the stimulation strength in the brain. Since the magnetic permeability of all the tissues of the brain do not change significantly, magnetic fields do not vary from tissue to tissue. They are a function of the distance between brain tissue and the location of the TMS coil. Depending upon the coil type, h-fields can simply be a function of scalar magnitude of the distance (e.g., single hoop coil) or direction vector dependent (e.g., figure 8 coil, halo coil). The symmetry of single hoop coil results in a net zero field, making the magnetic field reside only on the z-direction along the centerline of the hoop (see Figure 3) . Figure 1 shows the difference in how h-fields and e-fields are induced in the brain with a figure-8 TMS coil. Figure 1a shows different parts of the brain that are affected by TMS -the skull, cerebral spinal fluid, gray matter, and the white matter. There are a few software programs commercially/freely available to model electric/magnetic fields within the human brain such as Sim4Life 21 and SemCAD X 22 , SimNIBS 23 and a multiphysics FEM software COMSOL 24 . There are other magnetic FEM packages such as ANSYS Maxwell 25 that can compute magnetic fields and flux densities but are limited to use in simple geometries such as non-biological models.
Mathematical modeling of TMS
Although the single hoop coil is a specific case for modeling h-fields, it serves as a niche in the effort towards the development of a complete framework in modeling and simulation of TMS on dissimilar platforms. As evident from the literature, the features and function of software programs catered for TMS are very limited in scope. Together with the fact that present day technologies are far more computationally capable, it provides a tremendous opportunity to model TMS technique in a simulated environment and visualize them on dissimilar platforms. This paper presents the development of a visualization application of h-fields in human brain simulated via the use of a single hoop magnetic coil with electric current from a pulse generator subject to a constant air permittivity.
TMS visualization using virtual reality
Virtual Reality (VR) provides compelling means to experience and analyze three-dimensional data. The interest in using VR has exponentially grown in the last decade in both commercial and research sectors, and is considered a multi-billiondollar industry 26 . In fact, it is estimated that VR branded content market could reach as much as $22 billion USD by 2020, and $80 billion USD by the year 2025 27 . Facebook acquired Oculus Rift for $2 billion USD in 2014 28 , and Google invested $0.5 billion USD in Magic Leap 29 with the intent that VR is going to rise up to a significant prominence in the coming years.
TMS visualization using virtual reality
With VR increasingly becoming pervasive, it stands to reason for leveraging its capabilities for TMS simulations and visualization to benefit both research and commercial domains. When viewed in VR, users can spatially be immersed and interact with brain images represented as volumetric data overlaid with h-fields. This is a substantial leap from traditional desktop visualization tools in inferring the focality and penetration of magnetic fields.
TMS visualization using virtual reality
Fully immersive VR systems: According to Wikipedia, "immersion into virtual reality is a perception of being physically present in a non-physical world" 30 , and that the perception can be created by a combination of computer generated imagery, sound, or other forms of stimuli. A fully immersive VR system is where users walk into a cube like structure, called the CAVE Automatic Virtual Environment (CAVE TM ), and experiences 360 0 immersion from images displayed on all sides of the cube by a series of projectors. The C6 (six-sided CAVE) at the Virtual Reality Applications Center 31 , Iowa State University is one such system where each side of the cube is powered by four Sony SRX-105 projectors for a total of 24 projectors, capable of projecting CGI images at true 4K resolution. A 48-node graphics computer cluster each housing dual Nvidia Quadro GPUs pushes images, one per eye, to these hunterapril@example.com. The system sports an ultrasonic Intersense 900 tracking system that can track the position and orientation of head and 6 degree-of-freedom wand navigation devices. Figure 2a shows a schematic of the C6 structure indicating projector placement and lineup. Figure 2b is a photograph of an example gameday football VR application rendered in the C6, developed at Iowa State University 32 . Commodity VR systems: They are 3-5 orders of magnitude cheaper than fully immersive systems. For a fraction of cost, they provide a compelling VR experience compared to a multi-million USD VR systems. VR systems such as the HTC Vive 33 , the Oculus Rift 34 , and playstation VR 35 belong to tethered commodity VR systems. They are physically connected (hence 'tethered') to a PC using a cable (HDMI 1.4, dual DVI, or display port) and possess graphics computing capabilities of the host PC's GPU. Sensors track the location and orientation of their headsets and navigation systems. User wears a pair of glasses not physically tied to a PC User wears a headset physically tied to a PC Expensive ($0.5 Million -$6 Million USD) Low cost ($600 -$800 USD)
TMS visualization using virtual reality
Designed for high-impact research and industry Designed for research and consumers, especially gaming Table 1 shows a quick comparison of major differences between fully immersive and commodity VR systems. One of the primary differences between both systems are the field of view. A user can walk into the C6 and experience a full 360 0 field of view, while being positionally tracked. As such, he/she can experience more information per unit time interval without moving. On the contrary, a commodity VR system requires the user to move or spin the head himself/herself to access different locations in the visual information being presented. A user is self-aware of where he/she is within the bounds of a fully immersive system, but a commodity VR system obscures the outside world when wearing it.
TMS visualization using virtual reality
Each of these two VR systems offer convincing arguments for their use in modeling and visualization of TMS h-fields. It is also assumed that the end users do not always have access to VR equipment, and will prefer using traditional desktop environment for viewing h-field distributions within brain image data. Therefore, a desktop variant of the application is also desired. This paper presents the first proof of concept for modeling of magnetic flux densities (h-fields) induced from a single hoop TMS coil and visualized in real-time on desktop, commodity and fully immersive VR systems. It is to be noted that the user interface is not the primary focus of the paper, and hence formal user studies were not conducted. Figure 3 represents the circular hoop coil, represented by solid thick lines, used in numerically generating a magnetic field. Electric current, represented by thick arrows, is passed in the circular loop coil. This creates a magnetic field, which is concentrated at the center of the coil, and is represented by dotted lines with smaller arrows indicating the direction. 
METHOD DEVELOPMENT 3.1 h-field model and characteristic equations
At the center of the coil's loop, the angle is 90 0 for all points along the circumference of the coil, and the distance to a point on the field is constant. As such, a circular integral becomes A component of the magnetic field from the center is distributed along x, y, and z axes, which can be vectorially resolved, as shown in Figure 4b . The elements dB x and dB y follows along the path of the circular loop, which result in a net zero field because of symmetry. Hence, the only active component that persists is dB z , which resides along the axis of the current loop. More information on magnetic fields induced in a current coil can be obtained in any standard electrical engineering books 36 . Equation (4) is used as the basis for numerical computing of magnetic flux densities and visualization of the fields in a given brain dataset in this paper.
Choice of software development tools and hardware

Requirement listing
The application development hinged on the following requirements: a) Should be scalable in desktop, commodity VR and fully immersive VR systems, b) Relatively easily programmable or scriptable, and c) Clusterizable for use with the 48-node C6 computer graphics cluster.
Application development setup -VR and desktop environments
Open source and commercial graphics APIs such as OpenGL 37 , OpenScenegraph 38 , WorldViz Vizard 39 , were investigated as potential development environments. However, Unity 3D game engine 40, 41 showed a significant advantage over other development environments because of its native support for a variety of hardware and operating systems and its simplicity in scripting a visualization environment suitable for both desktop and VR systems alike. Its drag and drop feature of scripts significantly reduces burden to programmatically synchronize different components of the application together.
Application development setup -VR and desktop environments
Fully immersive VR: getReal3D 42 , a commercial third-party plugin developed by Mechdyne, was used for piping the graphics application across the C6 cluster. getReal3D is Windows based and is available only for Unity3D at the time of writing this paper. The plugin is capable of automatically spawning instances of the Unity3D application across the cluster nodes and manages viewport clipping, gameobject and state synchronization. The plugin also comes with an optional trackd software that supports various tracking devices (e.g., Intersense 900 head tracker and wand) and gamepads for use within the C6. getReal daemons on cluster nodes listen for game activity and automatically updates viewports and follows it with software and hardware synchronization thereby providing a seamless visualization experience. The plugin was tested and proven capable on a number of other projects developed at Iowa State University. Another clusterization tool MiddleVR 43 for Unity3D was investigated but its cost-to-benefit ratio was too high for a 48-node computer graphics cluster hosted at Iowa State University.
Application development setup -VR and desktop environments
Commodity VR: HTC Vive was chosen as a commodity VR environment for the application deployment. SteamVR 44 and VRTK 45 Unity3D plugins for the Vive were readily available and has significant development support. These plugins are also known to natively support an Oculus Rift VR system. Although the VRAC owns both these systems and the TMS visualization application is designed for use with both VR systems, focus was placed on the Vive as an initial proof of concept application development.
Application development setup -VR and desktop environments
Desktop: Unity3D has native support for desktop application development on platforms including Windows, Mac, and Linux operating systems. Apart from requiring a decent GPU and an optional use of an X-input based gamepad (e.g., Xbox 360/One, Logitech F-710, etc), the application was built with no additional requirement. The application was tested on desktop PCs with a separate Nvidia GPU as well as a Macbook Air with an onboard Intel Graphics chip.
Volumetric viewing of NIfTI brain image data
Neuroimaging Informatics Technology Initiative (NIfTI-1) is a data format proposed by the NIfTI data format working group as a "short-term measure to facilitate inter-operation of functional MRI data analysis software packages" 46 . A 64-bit update to NIfTI has been approved in 2011 and termed as NIfTI-2 format 47 . NIfTI format is specifically meant for brain data and can be either 3D structural images (static data) or 4D functional images (simple time-varying or multi-volume). The data used in the development of this application is 3D structural only and does not support functional temporal data.
Volumetric viewing of NIfTI brain image data
A 'Volume Viewer' asset 48 procured from Unity asset store was used for reading and loading brain image data. The asset pre-requires brain images be structural NIfTI-1 format and not include functional data. Typical DICOM datasets can be converted into NIfTI-1 formats relatively easily with a number of open source and commercial software as needed 49, 50 . A 160 x 256 x 256 (= 10,485,760 voxels) sample dataset that came along with the Volume Viewer asset was primarily used as a testbed to calculate h-fields in this application, and two other datasets from cited sources were tested.
Features implemented in the application
With the primary goal of computing h-fields and visualizing them on dissimilar platforms, the following sub-sections describe the development of the application.
GPUs for computing h-fields
The 'Volume Viewer' Unity3D asset came bundled with the capability of loading and visually rendering structural NIfTI-1 brain image data, deeming it unnecessary to custom write code to handle this functionality. Computing h-fields, for example on a 160 x 256 x 256 = 10,485,760 voxel sample dataset is computationally expensive and serial computations on a CPU is extremely inefficient. Besides, identical computations are performed on each adjacent voxel with no interdependencies. The mode of computations follows a Single Instruction Multiple Data format, which made it possible to use GPUs for computations. This means that the h-field values for all voxels can be computed simultaneously instead of serial computations on a CPU. At runtime, a Unity3D gameobject that loaded the NIfTI MRI brain data made voxel values and locations available to a Cg vertex shader. Also, the vertex shader receives the TMS coil position, radius, and current values. The vertex shader then computed h-field values for each voxel using an implementation of Biot Savart's law described in Equation (4) and passed them to the subsequent fragment shader. Color values were then computed in the fragment shader to indicate h-field gradients and visually merged them with voxel values of NIfTI brain data. Color gradients blue, green, yellow, orange, and red were used in that order to indicate low to high h-field values. The use of GPUs for computing the fields were commonly implemented for all three modes of visualization -the desktop, the Vive, and the fully immersive C6.
TMS coil placement and interaction
During TMS treatment, the magnetic coil is physically held at a specific location above the human head for the duration of a session. To determine the coil effectiveness however, it is desirable for a simulation to place it in any position and orientation around the head. This will better allow studying the focality and penetration of the magnetic fields. As such, a simple user interface with slider bars was built allowing a user to interactively move and orient the coil geometry around brain data. This approach instantly increases the usability and realize instant response from the system, as opposed to the popular TMS software SIMNIBS where a user is required to manually type in coordinate positions of the coil. The UI interactively allows passing values such as the coil position, radius, and current values are passed to the shader that computes h-field values for all brain data voxels.
Clipping planes (excluder or slicer)
Excluder, also termed as a slicer in this work, is a clipping plane feature offered by the 'Volume Viewer' asset. They clip a portion of the volumetric brain data from viewing, allowing the user to examine the magnetic fields induced on the interior portions of the data. The clipping planes can be moved within the bounds of the volumetric brain data and rotated in any of the Euler axes using slider bars. Partial transparent square panes are attached to the excluders, which visually indicate the location of clipping planes. The user interface (UI) for activating and using the clipping planes is similar to the TMS coil placement and orientation. Figure 6 shows the UI used for clipping along coordinate axes and arbitrary Euler angles. 
H-field spot probe
Spot computation of the h-field at a specific location within the dataset was implemented. A spherical probe can be moved within the volumetric brain data using the UI sliders and can spot compute h-field value at a specific location based on its distance from the TMS coil. A visual feedback on the interface indicating the probe location, coil radius, coil current, and the corresponding h-field values are provided. Since this is a computation for a single voxel location, it was implemented using the CPU.
Navigation
Navigation implementations substantially differ between three modes of visualization -desktop, commodity and fully immersive VR. On a desktop, viewing and interaction was scripted for use with a mouse and an optional gamepad. Standard viewing options such as zoom, pan and rotate along with the ability to interact with the UI was enabled. X-input gamepad navigation was also enabled to translate and rotate around the scene. Therefore, devices such as Xbox 360/One or Logitech F710 can be used natively with the application.
Navigation
The two wands that come with the commodity VR system HTC Vive were used for interacting with the UI and navigational purposes. The left wand allows a user to hold the UI in visualization space while the right wand lets interacting with options and slider bars. The wands do not come with traditional joystick inputs, but a touchpad is intended to replace it, allowing a user to 'teleport', or virtually jump from one location to another. However, the teleportation feature would have been useful for scenes that require moving in large virtual spaces (e.g., first person shooter games). However, viewing static volumetric data does not have a large virtual real estate to move around. Therefore, a rubber banding movement was scripted to allow a user to locally fly in different directions, while the handle buttons were pressed down.
Navigation
Navigating the application within the C6 uses a tracked Intersense-900 wand. The wand features a programmable joystick that can be made to perform basic navigational functions. getReal3D comes with wand navigational utilities such as walk through (simulates a character moving on a surface), Aim-n-go (motion following the direction of the wand using the joystick), and Wand Drive (motion following a button press, and does not require joysticks). All of these navigational options were included within the fully immersive application, and can lend themselves to be changed at runtime to suit to a user's preference.
RESULTS
Three NIfTI-1 structural datasets were evaluated and with the developed application:
RESULTS
1. Dataset1: 160 x 256 x 256 (1mm x 1mm x 1mm) image set, obtained from Volume Viewer sample. Table 2 shows a comparison of each mode of visualization and the differences in hardware/software. A single hoop coil is used for generating magnetic flux densities, as shown in Figure 7 . Proc. of SPIE Vol. 10219 102190P-9
RESULTS
slicer position & orien Figure 9 shows the desktop application running Dataset 3 with the coil close to the frontal lobe. The coil diameter for this configuration was 8 cm with a pulse current of 2,800 Amperes. Since this is a skull stripped dataset, a mesh based head was added to give a spatial sense for the brain's location. The figure also shows a h-field probe represented by a colored sphere at the center of the TMS coil. This probe spot computed the h-field and indicated in the left bottom corner of the UI. The x, y, z sliders can be used to move the probe's location and the h-value can be updated on the fly. For a pulse current of ~2650A and a coil radius of 0.09m as seen in the figure, an h-value of about 15 kA/m was computed. These results match with theoretical h-field calculations. The right wand controller allows interaction and parameters in the UI using a ray, which is casted in the direction the wand is pointed. When the ray encounters the corresponding gameobject, for example, a slider bar, it can be scripted to perform a certain action. Figure 10a shows a user wearing the Vive and running the application. Figure 11 shows the left-and righteye views perceived in the HTC vive. While the content is identical, the Vive separates the two images with a slight offset for each eye to get the correct immersive experience when a user wears the headset. For the above coil configuration and location, a flux density value of 97.6 A/m was computed for a probe located in the interior portion of Dataset 2, as seen in Figure 12 . The probe can be seen as a colored sphere in Figure 12 below. The image data is clipped using the excluder to reveal the probe location. This h-value is a 99.9% decrease from the one computed and shown in Figure 9 . This serves to show evidence that the h-field values substantially decrease as the distance from the coil increases. Figure 12 Clipped Dataset 2 with an h-field Probe Figure 13 shows Dataset 3 being displayed in the fully immersive C6, with volume data extending between the front, right, and the floor screens. Similar to the dataset shown in Figure 9 , the skull stripped brain image volume is overlaid inside of a transparent mesh based head model to provide a spatial reference. The TMS coil is placed above the frontal lobe. The user, as seen in the figure, is interacting with stereoscopic display of h-field data with an Intersense-900 head tracked active stereo shutter glasses and a tracked wand. The user can physically walk around the dataset and see the data from different vantage points within the confines of the 10ft x 10ft C6 real-estate. The wand can be used to virtually travel longer distances and to turn around. The green line pointing toward the UI refers to the direction the wand is pointing to and its buttons and joystick can be used to move the viewing position or the UI around. The UI elements are paginated and can be flipped through and different features such as the coil placement, clipping plane, and the h-field probe can be accessed using button presses on the wand. Figure 14 shows Dataset 1 in fully immersive C6, with a portion of the brain data cropped in an arbitrary clipping position and orientation. The cropped brain image data exposed the white spherical h-field probe, which computed an h-field value of 82 A/m with a coil radius of 11cm and a pulse current of 5,000 A. The TMS coil is placed pointing towards the right temporal lobe in this configuration. Since the camera viewpoint is customized for the user wearing a head tracker, photographs taken from the outside of the C6 do not always capture the full extent of user immersion and experience. It can be seen in Figure 13 and Figure 14 that the UI elements can be placed anywhere per user preference within the C6, and the data can be made smaller or larger if the user wished to see more detail.
CONCLUSIONS AND FUTURE WORK
Magnetic flux densities (h-fields) for Transcranial Magnetic Stimulation (TMS) is modeled and visualized in real-time on three different platforms -the desktop, the HTC Vive commodity VR system and a fully immersive six sided CAVE TM VR system. A basic single hoop coil is used for computing h-fields and the proof of concept application is tested on three different volumetric structural NIfTI MRI brain image datasets. The work presented uses voxel volumes constructed from slices of MRI images, but the methods developed can readily be applied to mesh-based geometry as well with very few changes in the code. Although the application in its current form can only compute and visualize h-fields, it sets itself apart from other TMS simulation software by providing a compelling visualization experience in real-time in both the desktop and immersive VR environments. This work is a preliminary first step towards developing a generic TMS simulationsboth h-fields and e-fields with a variety of coil types. While the UI was developed based on simple heuristics, it will benefit from formal UI design and user studies. Both these efforts will be made as a part of near future work.

{'paper_id': '173179956', 'title': 'Real-time visualization of magnetic flux densities for transcranial magnetic stimulation on commodity and fully immersive VR systems', 's2_url': 'https://api.semanticscholar.org/CorpusID:173179956', 'source': 'S2ORC'}