Towards a Multimodal Model of Cognitive Workload through Synchronous Optical Brain Imaging and Eye Tracking Measures

Philosophy

ORIGINAL RESEARCH published: 23 October 2019 doi: 10.3389/fnhum.2019.00375 Edited by: Frederic Dehais, National Higher School of Aeronautics and Space, France Reviewed by: Harsimrat Singh, Imperial College London, United Kingdom Tom Foulsham, University of Essex, United Kingdom *Correspondence: Murat Perit Çakır email@example.com Specialty section: This article was submitted to Cognitive Neuroscience, a section of the journal Frontiers in Human Neuroscience Received: 17 April 2019 Accepted: 03 October 2019 Published: 23 October 2019 Citation: I . şbilir E, Çakır MP, Acartürk C and Tekerek AŞ (2019) Towards a Multimodal Model of Cognitive Workload Through Synchronous Optical Brain Imaging and Eye Tracking Measures. Front. Hum. Neurosci. 13:375. doi: 10.3389/fnhum.2019.00375 Towards a Multimodal Model of Cognitive Workload Through Synchronous Optical Brain Imaging and Eye Tracking Measures Erdinç I . şbilir 1, Murat Perit Çakır2*, Cengiz Acartürk2 and Ali Şimşek Tekerek 1 1Advanced Technologies Directorate, Guidance and Photonics Division, Roketsan Missiles Industries Inc., Ankara, Turkey, 2Department of Cognitive Science, Informatics Institute, Middle East Technical University, Ankara, Turkey Recent advances in neuroimaging technologies have rendered multimodal analysis of operators' cognitive processes in complex task settings and environments increasingly more practical. In this exploratory study, we utilized optical brain imaging and mobile eye tracking technologies to investigate the behavioral and neurophysiological differences among expert and novice operators while they operated a human-machine interface in normal and adverse conditions. In congruence with related work, we observed that experts tended to have lower prefrontal oxygenation and exhibit gaze patterns that are better aligned with the optimal task sequence with shorter fixation durations as compared to novices. These trends reached statistical significance only in the adverse condition where the operators were prompted with an unexpected error message. Comparisons between hemodynamic and gaze measures before and after the error message indicated that experts' neurophysiological response to the error involved a systematic increase in bilateral dorsolateral prefrontal cortex (dlPFC) activity accompanied with an increase in fixation durations, which suggests a shift in their attentional state, possibly from routine process execution to problem detection and resolution. The novices' response was not as strong as that of experts, including a slight increase only in the left dlPFC with a decreasing trend in fixation durations, which is indicative of visual search behavior for possible cues to make sense of the unanticipated situation. A linear discriminant analysis model capitalizing on the covariance structure among hemodynamic and eye movement measures could distinguish experts from novices with 91% accuracy. Despite the small sample size, the performance of the linear discriminant analysis combining eye fixation and dorsolateral oxygenation measures before and after an unexpected event suggests that multimodal approaches may be fruitful for distinguishing novice and expert performance in similar neuroergonomic applications in the field. Keywords: neuroergonomics, fNIRS, eye tracking, human machine interface, cognitive workload Frontiers in Human Neuroscience | www.frontiersin.org 1 October 2019 | Volume 13 | Article 375 I . şbilir et al. A Multimodal Model of Cognitive Workload INTRODUCTION Understanding the neural underpinnings of complex cognitive tasks in the context of safety-critical settings is a key objective for neuroergonomics research (Parasuraman, 2011; Parasuraman et al., 2012; Mehta and Parasuraman, 2013). Operators' effective utilization of human-machine interfaces in such settings depends on interface design considerations, as well as the operators' level of expertise in carrying out key tasks via the interface. Cognitive or mental workload is an essential determinant of operator performance that needs to be optimized through effective training and user-centered design practices. Cognitive workload addresses the limited capacity of the human brain for processing information demanded by the task at hand (Wickens and McCarley, 2007). Decrements in performance, as evidenced in missed responses or delayed response times, are typically observed when operators are subjected to cognitive workload beyond the maximum information processing capacity of their brains (Hancock and Parasuraman, 1992; Wickens et al., 2013). In other words, cognitive workload is an essential determinant of cognitive performance. However, these limits are subject to change due to the development of expertise during training, and design elements that promote or hinder the utilization of the system's affordances, which altogether make their assessment a challenging issue (Fairclough et al., 2005; Ullén et al., 2018). Due to its essential role in cognitive performance, developing operational measures for monitoring cognitive load is an active area of investigation in neuroergonomics. Such measures have been traditionally monitored offline through subjective assessments with the help of instruments such as NASA-TLX (Hart and Staveland, 1988; Hill et al., 1992). Despite their convenience for administering these instruments in various real-life and experimental task settings, the subjective nature of the assessments obtained and the difficulties participants have in providing verbal accounts of their actions motivated the need for more objective means for measurement. Recent studies in psychophysiology and neuroergonomics have expanded on self-evaluation based methods by focusing on neurophysiological measurements targeting key components of the central and the peripheral nervous system, as well as the cardiovascular system (for extensive reviews, see Fairclough, 2009; Borghini et al., 2014; Lim et al., 2018; Charles and Nixon, 2019). Increase in operator workload is often associated with increase in heart rate, heart rate variability, respiration rate, blood pressure and galvanic skin conductance (Boucsein, 2012; Borghini et al., 2014; Durantin et al., 2014). However, the existence of other sources of variability competing with mental workload such as anxiety, muscle fatigue and room temperature present challenges for mental workload estimation methods based on these measures. Related work focusing on ocular correlates of cognitive workload tend to employ eye trackers and electrooculograms (EOGs) to relate measures such as fixation duration, saccadic amplitude, gaze entropy, pupil dilation, PERCLOS (the percentage of time that the eyelid covers 80% or more of the pupil), gaze blink frequency and latency with changes in workload. In particular, increase in cognitive workload is associated with increases in fixation duration, pupil dilation and blink latency, as well as decreases in blink duration, PERCLOS and gaze variability; whereas investigations on blink rate have produced mixed results (Marquart et al., 2015; Diaz-Piedra et al., 2019). External sources of variability such as changes in lighting conditions or the anatomic variation of the eyelids can influence the eye tracking, pupillometry and EOG measurements (Holmqvist et al., 2011). Cognitive workload assessment approaches based on brain physiology mainly focus on electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), due to their portability and flexibility that make them suitable for in-thefield neuroimaging studies. Studies on neural correlates of mental workload with EEG tend to focus on changes in the EEG power spectra, where an increase in workload is typically associated with an increase in the theta band power, especially at the topand mid-central electrodes (Borghini et al., 2017). Since EEG monitors changes in electric potentials originated in the brain via electrodes located over the scalp, it offers high temporal resolution. However, due to the complexities involved with the propagation of electric potentials within neural tissue, achieving good spatial resolution requires high-density EEG recording setups with sufficient number of electrodes (Gevins and Smith, 2008). The presence of artifacts such as eye movements, muscle contractions and heart-beat, the need for ensuring low impedance levels for suitable signal-to-noise ratio, and the need for using gels or liquids to improve conductance are other challenges involved with the use of EEG for monitoring workload in the field (Dussault et al., 2005; Borghini et al., 2011; Borghini et al., 2014). fNIRS monitors the hemodynamic changes over the cortex by using specific wavelengths of infra-red light that are known to interact with oxygenated and deoxygenated hemoglobin in the capillary beds (Obrig et al., 2000). fNIRS is a neuroimaging modality that enables continuous, noninvasive, and portable monitoring of changes in blood oxygenation and blood volume related to human brain function (Ayaz et al., 2013). fNIRS is not only non-invasive, safe, affordable and portable, it also provides a balance between temporal and spatial resolution which makes it a viable option for in-the field neuroimaging. Neuroergonomics studies employing the fNIRS modality reported a strong relationship between increasing bilateral dorsolateral prefrontal cortex (dlPFC) activity and systematic increases in cognitive workload (Izzetoglu et al., 2003; Gateau et al., 2015; Çakır et al., 2016). In combination with performance measures, this relationship could be fruitfully employed to evaluate alternative human-machine interaction designs (Menda et al., 2011; Ayaz et al., 2013) and to monitor expertise development as a function of the systematic decrease in prefrontal activity accompanied by improvements in performance in time (Ayaz et al., 2012). Moreover, the way infrared photons migrate in the neural tissue allow strategic placement of the sensor to target specific cortical regions close to the surface with a reasonable spatial resolution (Obrig et al., 2000). However, the latency involved with the hemodynamic response in reaction to neural activity is a limiting factor on the temporal resolution of the fNIRS methodology (Gratton and Fabiani, 2008). Frontiers in Human Neuroscience | www.frontiersin.org 2 October 2019 | Volume 13 | Article 375 I . şbilir et al. A Multimodal Model of Cognitive Workload To sum up, each modality offers specific advantages and limitations in terms of quantifying a complex construct such as mental workload due to the differences in their physical working principles and the relationship between the monitored biological processes to cognitive phenomena (Fairclough, 2009). Multimodal neurophysiological models that can bring the advantages of each modality together in contextually sensitive ways is an emerging necessity in neuroergonomics research (Fairclough et al., 2005). Existing efforts include simultaneous use of EEG and fNIRS (Aghajani et al., 2017; Liu et al., 2017a,b), as well as other physiological sensors (De Rivecourt et al., 2008; Gateau et al., 2015), and reported superior workload classification performance during working memory tests such as n-back as compared to predictions obtained from any single modality. However, conducting multimodal data recording and analysis brings significant challenges, such as sensor placement and shielding for reliable data acquisition, minimization of interference among the chosen modalities, time-synchronization of data streams, obtaining measurements without interfering with the operators' normal task execution, and most importantly, finding appropriate means to integrate the data and insights obtained from the chosen modalities. COGNITIVE WORKLOAD AND EXPERTISE In addition to the challenges involved with multimodal investigation of cognitive workload, another key source of variability in its assessment is due to individual differences in expertise with regards to the task at hand, which further complicates the development of metrics and/or thresholds for mental workload classification (Charles and Nixon, 2019). Related studies in cognitive neuroscience comparing experts and novices have identified both structural and functional changes due to expertise development, including the growth of gray and white matter in task-relevant cortical locations, as well as the reduction of activity in prefrontal and parietal cortex involved in domain-general cognitive control (Hill and Schneider, 2006; Bilalić et al., 2011; Bilalić, 2017; Bilalić and Campitelli, 2018). In particular, related work in expertise and distinguishing expertise levels in laparoscopic surgery skills showed that in contrast to novices experts exhibit greater performance under time pressure accompanied with significantly lower PFC activations and significantly higher primary motor area and supplementary motor area (SMA) activations, which are regions associated with bimanual motor dexterity performance (Modi et al., 2018; Nemani et al., 2018). Moreover, related work in gaze measures of expertise suggest that experts tend to exhibit more well-defined scan-paths towards task-relevant locations in the visual field, which is manifested in lower gaze entropy, lower fixation durations and shorter saccadic amplitudes (McCarley and Kramer, 2006). In a related research on expert and novice endoscopists' gaze patterns, the results showed that there are distinct gaze patterns that are associated with expert behavior (Lami et al., 2018). However, such differences can be highly contextual with respect to the specific task domain and the way expertly conduct is characterized in that domain (Hodges et al., 2006). Therefore, a broader view of expertise is needed to make sense of the underlying causes of the differences obtained when experts and novices are contrasted along various neurophysiological measures. The body of research in psychology and cognitive neuroscience of expertise suggest that most forms of expertise in domains such as sports, chess, math, dance and music entail: (i) a process of automation; (ii) reorganization of memory resources; and (iii) sensorimotor adaptations (Ullén et al., 2018). Sensorimotor adaptations involve changes due to extensive motor practice, such as the development of complex reflexes and sensitivities in sports like tennis, and the corresponding changes in the somatosensory, premotor and motor cortices such as the enlargement of the representations of the fingers of the left hand for string players (Elbert et al., 1995). Automation refers to a change from a more effortful, flexible, deliberate mode of carrying out tasks that relies on attention and working memory resources towards a less effortful, reflex like, low effort, difficult to modify mode of conduct that requires less attentional resources due to training and experience (Hill and Schneider, 2006). Since experts require little explicit control, surplus cognitive resources can be allocated for strategic decision making and planning. Together with before-mentioned sensorimotor adaptations presumably occurring in the premotor cortex guiding the eye movements, the notion of automation could also be manifested in the scan-paths of experts guiding the expert's motor actions (Hodges et al., 2006). In neuroimaging, such adaptations are usually manifested as a decrease in the fronto-parietal network that is associated with deliberate guidance of attention (Duncan, 2010, 2013). Experts also tend to exhibit superior recognition of complex sensory stimuli, which is claimed to be facilitated by mechanisms of efficient retrieval of relevant domain-specific information from long term memory into working memory during performance (Guida et al., 2012, 2013). According to this view, with developing expertise specialized long-term memory elements with complex knowledge structures called chunks are formed in the medial temporal lobe, which can be accessed by working memory, effectively increasing the information processing capacity of the expert. As the knowledge elements get bound into chunks within working memory, an increase in medial temporal lobe activation is accompanied by decreasing activations in prefrontal and parietal areas that are typically associated with memory retrieval and working memory management (Just and Carpenter, 1992). In terms of eye movement metrics, changes in the memory organization may also influence the way the visual information is sampled as perceptual input. This could be partly the underlying reason for the experts' lower number of fixations and shorter fixation durations, as they can pick up and interpret the visual cues in a more efficient manner according to the related work in psychology of expertise (Hodges et al., 2006). Overall, the growing body of literature in neuroergonomics have utilized increasingly portable/wearable neurophysiological measurement modalities in various work settings. However, the need for applying these methods and insights in multitude of field settings to test their robustness and assess their degree of domain specificity are still important considerations for the Frontiers in Human Neuroscience | www.frontiersin.org 3 October 2019 | Volume 13 | Article 375 I . şbilir et al. A Multimodal Model of Cognitive Workload burgeoning field of neuroergonomics. To that end, in this study, we explored the simultaneous use of fNIRS and eye tracking in a human-machine interaction scenario involving a military land platform. The goal of the study is to observe in what ways experts and novices differ in terms of their prefrontal hemodynamic responses and eye movement patterns during regular and adverse task conditions in the field. The study aimed to investigate to what extent expertise levels modulated some of the expected neurophysiological correlates of cognitive workload as manifested in fNIRS and eye tracking modalities in a realistic setting with professional operators. Finally, the study explored a possible multivariate integration of prefrontal hemodynamics and gaze data to support expert vs. novice distinction. MATERIALS AND METHODS Participants The study included a human-machine interaction scenario where the participants (MeanAge = 28.9, SD = 3.6) were asked to operate a training version of a military land platform by using a computer-based control interface. The experimental procedures were conducted in the field with 14 professional, real-life operators, eight of whom were identified as experts due to their prior training on the platform, whereas six were novices/beginners who were on their first training day. All operators had prior experience in other comparable military platforms. Those operators who were considered as novices were all using the interface for the first time after receiving a formal presentation of the basic features of the system in a classroom setting. The experiment was conducted when this cohort moved on to the vehicles to perform hands-on exercises. An independent t-test showed that the age difference between the novices (Mean = 27.3, SD = 2.1) and the experts (Mean = 30.5, SD = 3.9) was not significant, t(12) = −1.76, p > 0.05. Participants were informed prior to the study that the main goal of the experiment was to perform a usability analysis of the system, their data would be kept anonymously and their observed performance would not have any implications on their professional careers. The study was approved by the METU Human Subjects Research Ethics Committee. Materials and Equipment While the participants were performing the tasks, their eye movements and the hemodynamic changes in their prefrontal cortices were monitored with a mobile eye tracker system (60 Hz, binocular by Pupil Labs GmbH, Germany) and a portable fNIRS system (fNIRS Imager 1002, fNIR Devices LLC, Potomac, MD, USA), respectively. The fNIRS system is composed of a flexible headpiece that holds four LED infrared light sources and 10 photodetectors to obtain oxygenation measures from 16 optodes located over the prefrontal cortex (PFC), a control box for hardware management, and a computer that runs the COBI Studio software (Ayaz et al., 2011) for data acquisition. fNIRS 1002 is a continuous wave NIRS system that uses wavelengths of 730 nm and 850 nm for optical imaging of the PFC. The sensor has a source-detector separation of 2.5 cm, which allows for approximately 1.25 cm penetration depth. This system can monitor changes in relative concentrations of oxy-hemoglobin (HbO) and deoxy-hemoglobin (HbR) at a temporal resolution of 2 Hz. According to co-registration studies that mapped the measurement locations of this fNIRS sensor on a standard brain template (Ayaz et al., 2006; Chen et al., 2017), the 16 optodes correspond to Broadmann areas 9, 10, 44 and 45 (Figure 1). The approximated mapping over a standard brain and the diffuse FIGURE 1 | The functional near-infrared spectroscopy (fNIRS) probe with four light sources and 10 detectors, and the corresponding 16 optodes mapped over the prefrontal cortex (PFC). Frontiers in Human Neuroscience | www.frontiersin.org 4 October 2019 | Volume 13 | Article 375 I . şbilir et al. A Multimodal Model of Cognitive Workload nature of photon migration paths in the tissue imply that the spatial resolution of the optodes are in the centimeter range (Ayaz et al., 2013). The eye movements of the participants were monitored with the Pupil Labs mobile eye tracking system. The mobile eye tracker had two infrared cameras and two video cameras facing towards both eyes for binocular recording, as well as an integrated world camera capturing the field of view from the participant's perspective (Figure 2). The two eye cameras monitor the pupil center and the corneal reflection by utilizing the dark pupil method in reference to a three-dimensional geometric model of the eyeball. The system has a sampling rate of 60 Hz, a reported accuracy of 0.6◦, and precision of 0.08◦. The external camera can record an external world image at 720 p HD resolution. The PupilCapture open source software accompanied with the mobile eye tracker was used to calibrate and record the eye movements. A nine-point calibration was performed at the beginning of each task and the recorded data was monitored in real time by the researchers for possible indicators of calibration loss or hardware failure. Since the fNIRS sensor and the eye tracker both rely on infra-red light for the monitoring of brain activity and eye movements, the IR camera of the eye tracker may interfere with the fNIRS signals. Moreover, since the study was conducted as a field study inside a training version of the real platform, the sensors were also partly exposed to sunlight entering through the windows. The fNIRS sensor was shielded with a cap supported with aluminum foil, which can minimize the effects of outside IR sources (Figure 3). Moreover, tests conducted with the eye tracker on and off while recording fNIRS data did not produce any discernable effects on the raw fNIRS signals. This is probably due to the fact that the Pupil Labs system's IR light source is located beneath the eye balls, which introduces a separation between the eye tracker IR camera and the fNIRS receivers. Due to their location over the forehead and orientation towards the cortex, the LEDs of fNIRS did not introduce any discernable interference on the eye tracking measurements. Excessive sunlight also hampers the eye tracker's performance and the head-mounted world camera recording the screen, so the built-in roller blinds of the vehicle were used to provide further protective shield from sunlight. Experiment Setup During the experiment, participants were seated inside the cabin of a training version of a military land platform designed and manufactured by Roketsan Inc. The training vehicle is an exact FIGURE 2 | The Pupil Labs mobile eye tracking system. FIGURE 3 | The installation of the fNIRS sensor and the mobile eye tracker for multimodal recording with minimal IR interference. Frontiers in Human Neuroscience | www.frontiersin.org 5 October 2019 | Volume 13 | Article 375 I . şbilir et al. A Multimodal Model of Cognitive Workload replica of the real system with the exception of live ammunition. The operators interacted with the system through a 15.6 inch laptop computer. A wire diagram of the graphical user interface is provided in Figure 4. Each control button on the left opens a window at the center of the screen dedicated to the specific features associated with that function. For instance, pushing the define mission button will bring the target definition screen as a pop-up window in the middle of the screen, which allows the user to either enter coordinates or select a pre-defined target from the list. Procedure During the experiment, the participants were asked to engage a pre-defined target by using the control interface in normal and adverse conditions. In the first task, participants executed the standard procedure for engaging a target, which required the completion of a sequence of operations including data entry, initialization, system checks, monitoring and target engagement. The second task required participants to engage with a second target in the same way, which was deliberately defined by the researchers as slightly out of range. Briefly after the participants unknowingly selected this target as instructed, the system triggered an unexpected interruption with a system failure message effectively aborting the mission. The translated error message prompted by the system stated ''Command Received/Command Aborted: guidance cannot be completed because the cradle's target information is out of guidance criteria,'' which was expected to be challenging for some of the operators to comprehend. Once the system failure message was prompted, the experimenter waited for about 60 s to observe the operator's response to the prompt, and then verbally asked what might have caused the error. The task was completed when the operator provided an answer. The expected response was related to the fact that the second target was slightly out of range, which was not immediately evident on the system interface. Although all operators had prior experience with comparable platforms, it was the first time the novice operators worked on the computer-based control interface used in this study. Prior to the experiment, novice operators attended a formal training presentation introducing the basic features of the platform. The experiments were conducted while the novices were using the training platform for the first time. For that reason, novice participants were provided verbal guidance by an expert when they needed while completing the first task. For the second task, participants were asked to follow the regular operation sequence as in the first task. It was observed that novice participants were able to perform the main operations until the error message without the need for guidance as in the first task. Overall, in line with existing studies on neural and ocular correlates of expertise, we expected experts to elicit less prefrontal activity and exhibit gaze patterns that are better aligned with optimal task sequence due to their familiarity with the system while performing the routine task sequence. In terms of eye movements, we hypothesized that since novices have just been exposed to the environment, they will be searching for the next action in the task sequence, which will require them to inspect more parts of the interface for longer periods of time. We expected this behavior to be reflected in higher saccadic amplitudes and longer fixation durations. For the experts, we expected eye movements to precede the next action with shorter fixation durations and saccadic amplitudes, since the expert is familiar with the task sequence. In the second task, we expected a stronger response from the experts towards the unexpected error when it first appears on the screen, and novices to be less alert since they do not yet have a full grasp of the possible error states and their underlying causes. FIGURE 4 | A wirediagram of the graphical user interface used by the participants during the experiment. Error messages were popped up as a separate window in the middle of the screen in the event of exceptions. Frontiers in Human Neuroscience | www.frontiersin.org 6 October 2019 | Volume 13 | Article 375 I . şbilir et al. A Multimodal Model of Cognitive Workload RESULTS Behavioral Performance Analysis Findings During the first task, all participants were able to successfully complete the regular operation sequence. An independent groups t-test conducted on the mean task completion times showed that the novices (Mean = 559.7 s, SD = 128.1 s) took significantly longer than the experts (Mean = 444.0 s, SD = 35.1 s), t(12) = 2.46, p < 0.05 to complete the first task as expected (Figure 5). During the second task, all novice participants failed to identify the cause of the system error, whereas 63% of the expert group was able to correctly interpret the error and provide a satisfactory explanation for the underlying reason. In addition to this, the time to reach the error message (which concluded the task) for the expert participants (Mean = 115.8 s, SD = 23.08 s) was significantly lower than the novice participants (Mean = 216.8 s, SD = 116.8 s), t(12) = 2.42, p < 0.05 (Figure 5). Although the groups differed in terms of the time to reach the error message, the error was consistently displayed in the same stage following the closing of the target selection screen on the interface. In short, the significant differences in accuracy and task completion times further support our initial grouping of the operators as experts and novices. fNIRS Analysis Findings Filtering of the raw fNIRS data and the conversion to oxygenation measures were carried out by using the fNIR Soft v4.11 (Ayaz, 2010). Firstly, the raw fNIRS data were visually inspected for cases including excessive noise and motion artifacts. Saturated optodes exceeding the signal-to-noise ratio limit of the sensors (if any) were excluded from analysis. The data obtained from one subject was eliminated due to such issues, so the analysis of fNIRS data was conducted over 13 subjects (six novices, seven experts). Sliding Windows Motion Artifact (SMAR) filter (Ayaz, 2010) was used to attenuate signal changes due to head movements. In total, 12 blocks were rejected due to saturation or motion artifacts among the 16 × 13 = 208 blocks, which corresponds to 6% of the collected data. Raw fNIRS data were low-pass filtered with a linear phase filter with order 20 and cut-off frequency 0.1 Hz to reduce the effect of high-frequency noise due to respiration and heart beat cycles. The filtered fNIRS data were then converted into HbO and HbR concentration changes by using the Modified Beer Lambert Law with the default differential path factor and chromophore absorption parameters for adults provided in fNIR Soft. The total hemoglobin (HbT) value, which is defined as the sum of HbO and HbR values observed at each optode and is known to correlate with total cerebral blood flow, was considered as the main dependent variable. The HbT measures were baseline-corrected with respect to the beginning of each block. Task 1–Normal Operation For the first task, the expert and novice groups were compared in terms of the average HbT levels observed at all optodes. For this purpose, the HbT signals between the beginning and end of the first task were extracted for each participant and block averages were calculated. The differences between the two groups were investigated by using independent t-tests conducted for each optode. Overall the average HbT levels tended to be higher for the novices as compared to the experts, especially at optodes 14 and FIGURE 5 | Task completion times for the novice and expert groups for task 1 (left) and task 2 (right). The whiskers indicate standard error. Significant differences are indicated with an asterisk. Frontiers in Human Neuroscience | www.frontiersin.org 7 October 2019 | Volume 13 | Article 375 I . şbilir et al. A Multimodal Model of Cognitive Workload 16 that are close to the right inferior frontal gyrus. However, independent t-tests showed that none of these trends reached statistical significance. Task 2–Response to Error Message For the second task, participants followed an operation sequence similar to the first task. However, the second task included a condition where the system generates an error message at a pre-defined stage. The average HbT values were obtained 20 s before and after the error message was prompted on the participants' screens. A 2 × 2 mixed analysis of variance (ANOVA) was conducted on average HbT levels to contrast experts and novices in terms of their hemodynamic responses to the error message. Although the average HbT levels tended to be higher for novices in contrast to experts across all optodes, the main effect of expertise did not reach significance in any one of the 16 optodes, which was consistent with the results obtained for the first task (Figure 6). However, the main effect of pre-/postdistinction and its interaction with expertise levels revealed significant differences, which are summarized in Figures 7, 8, respectively. The plot in Figure 7 summarizes the locations where significant HbT difference was observed when the post-error block averages were contrasted with the pre-error block averages. The plot is generated in fNIR Soft by using a BSpline interpolation of F-ratios computed for each optode for the main effect of pre-/post-error on mean HbT levels. Only those areas that exceed the statistical significance threshold are highlighted in the plot. The plot suggests that the error prompt elicited the strongest HbT response in the left dlPFC, followed by the left dmPFC and right dlPFC regions for the entire sample. The bilateral dlPFC response is a typical indicator of increased cognitive workload, and the stronger response in the left hemisphere could be interpreted as a reaction to the unanticipated error that aborted the mission. As shown in Figure 5, the difference between the expert group's preand post-HbT response was overall higher than the novice group. The regions where the interaction between the expertise level and the pre-/post-HbT level differences were significant are shown in Figure 7. The change in the HbT trend in the right dlPFC was particularly stronger in the expert group between preand post-error blocks, whereas the novice group exhibited almost no difference in HbT trends in this region. The implicated regions in the right dlPFC are known to be involved with the fronto-parietal network involved with the control of visual attention, so the difference among experts and novices between the levels of preand post-error episodes could be related to the attentional demand triggered by the error. Eye-Tracking Analysis Findings Prior to the extraction of gaze measures, the quality of the eye-tracking data was evaluated by going through the video recordings overlaid with raw gaze coordinates. The data of two participants were excluded due to technical issues encountered while finalizing the recorded data with the eye tracker software. Therefore, the analysis of the eye-tracking data was conducted on 12 participants including five novices and seven experts. The velocity-threshold identification (I-VT) filter as implemented by Komogortsev and Karpov (2013) was applied on the raw data to extract the eye fixation durations and the saccadic events. The data obtained from expert and novice operators were compared in terms of their average fixation durations and saccadic amplitudes observed during the first and second tasks. Task 1–Normal Operation During the first task, participants in both groups tended to exhibit similar gaze patterns while following the standard FIGURE 6 | Average HbT changes observed at 16 optodes during the second task before and after the prompted error message. Frontiers in Human Neuroscience | www.frontiersin.org 8 October 2019 | Volume 13 | Article 375 I . şbilir et al. A Multimodal Model of Cognitive Workload FIGURE 7 | BSpline interpolated F-ratios that exceeded the significance threshold obtained for each optode for the contrast between mean HbT changes during postand pre-error message. The significant responses in both groups were predominantly localized over the left dorsolateral prefrontal cortex (dlPFC) together with a narrower region within right dlPFC. flow on the interface for target engagement. The novice users tended to do more visual search scans to find the relevant menu items for each step of the task, whereas the experts followed a more well-defined gaze sequence directed towards the relevant interface elements. However, no significant difference was observed between expert and novice operators in terms of their mean fixation durations and mean saccadic amplitudes through the course of Task 1 (Figure 9). We also compared the average fixation duration and saccadic amplitudes during the first and the last minute of Task 1 through 2 × 2 mixed ANOVAs to check if the behavior of the novices and experts differed at different stages of the task. No significant effect of order (F(1,10) = 0.43, p > 0.05), group (F(1,10) = 0.26, p> 0.05) and interaction (F(1,10) = 0.42, p> 0.05) were observed for the average fixation duration measure. A significant main effect of order was observed for the case of saccadic amplitudes, F(1,10) = 22.97, p < 0.01, partial η2 = 0.69, but the main effect of group (F(1,10) = 0.01, p > 0.05) and interaction (F(1,10) = 0.45, p > 0.05) were not significant. Towards the end of Task 1 both experts and novices had larger average saccadic amplitudes, simply because they had to turn a series of manual switches to finalize the launch that was located towards the left side of the screen. Task 2–Response to Error Message Similar to the analysis of the fNIRS data, we focused on the influence of the error message on the eye movement patterns of expert and novice operators. For this purpose, the average fixation durations observed for the five fixation events that precede and succeed the error message respectively were compared by using a 2× 2 mixed ANOVA. The results indicated FIGURE 8 | BSpline interpolated F-ratios that exceeded the significance threshold obtained for each optode for the interaction of expertise level and pre-/post-episode on the mean HbT changes. The experts were most distinguished from novices with respect to their HbT response in the right dlPFC when their preand post-error average HbT responses were contrasted. that the main effect of group and pre-/postsequence were not significant, whereas the interaction effect was marginally significant, F(1,10) = 2.02, p = 0.09, one-tailed, partial η2 = 0.17. The majority of the novices tended to fixate on different parts of the interface for further cues, whereas two novices and one expert (who also could not identify the cause of the error) tended to fixate on the error message for longer durations of time. This led to an unclear pattern where the majority switched into a quick scan mode whereas the others fixated on a single location on the screen, which led to larger variability in the fixation duration measures of the novice participants. On the other hand, those experts who correctly identified the reason tended to switch to a more vigilant mode where they read the message and checked the relevant screen for inspecting the location of the given target. This was accompanied with a slightly increased average fixation duration. However, these qualitative observations did not translate into significant statistical differences in gaze data, possibly due to the small sample size (Figure 10). A similar analysis performed on saccadic amplitudes before and after the error message also did not detect any significant differences. Multivariate Analysis Results A multivariate model combining eye-tracking and fNIRS data was also considered based on the findings of our analysis for each modality. The analysis focused on a subsample of 11 participants (five novices, six experts) whose fNIRS and eye-tracking data were complete. Since no meaningful differences could be detected in task 1 in either modality, we focused on task 2 where subjects were prompted with an unexpected error message. A discriminant analysis model was constructed by using the Frontiers in Human Neuroscience | www.frontiersin.org 9 October 2019 | Volume 13 | Article 375 I . şbilir et al. A Multimodal Model of Cognitive Workload FIGURE 9 | Bar charts showing the average fixation durations (left) and saccadic amplitudes (right) observed for the novice and expert operators during the first task. FIGURE 10 | Bar charts showing the average fixation durations observed for the novice and expert operators during the second task. preand post-mean HbT changes across 16 optodes and the mean fixation durations as possible predictors in the model. The model that included all the features produced a single discriminant function that failed to significantly differentiate novice and expert groups, Wilk's Λ = 0.019, χ2(6) = 11.91, p > 0.05, with 64% prediction accuracy in the leave-one-out validation phase. When the model was restricted to those optodes over left and right dlPFC where significant changes in HbT levels were observed for both groups, a significant discrimination was obtained, Wilk's Λ = 0.005, χ2(7) = 18.79, p < 0.01 with leaveone-out validation accuracy of 73%. The model with the highest validation phase accuracy was obtained when the model was restricted to those optodes in the right dlPFC where we observed a significant interaction effect among pre-post and expert/novice dimensions. In particular, the model with mean HbT levels obtained from optodes 15 and 16 in the right dlPFC and the fixation duration provided a significant discrimination, Wilk's Λ = 0.043, χ2(6) = 15.73, p < 0.05, with 91% accuracy in leaveone-out validation. The histogram for the single discriminant function estimated by the LDA procedure for this model is shown in Figure 11, where experts and novices were clustered around smaller and lager values, respectively. When we excluded the pre-/postfixation duration information from the last model and focus only on the HbT features, the validation accuracy dropped to 55%, Wilk's Λ = 0.341, χ2(4) = 6.45, p> 0.05. DISCUSSION Based on the characterization of neural and ocular correlates of expertise in the related literature, in this study we hypothesized that there would be differences between expert and novice operators in terms of their gaze patterns and prefrontal oxygenation trends while they were engaging with regular and adverse versions of the target engagement task. In particular, since the experts were more familiar with executing the routine target engagement sequence, we expected them to exhibit lower prefrontal oxygenation levels and more structured gaze patterns over relevant parts of the interface as compared to novice operators. In contrast, we expected novices to have more scattered gaze patterns and higher prefrontal oxygenation trends due to the novelty of the interface. Finally, we expected the adverse condition to provide an even stronger contrast between the neurophysiological measures of expert and novice operators. The findings suggest a less straightforward picture for the normal task condition. Although experts completed both tasks significantly more accurately and more quickly, these behavioral differences were not simply translated into significantly lower prefrontal oxygen consumption and shorter fixation durations. Eye-tracking videos qualitatively suggested that the expert operators were executing the task sequence more smoothly where Frontiers in Human Neuroscience | www.frontiersin.org 10 October 2019 | Volume 13 | Article 375 I . şbilir et al. A Multimodal Model of Cognitive Workload FIGURE 11 | The histogram for the single discriminant function that separate novices (blue) and experts (red). their gaze patterns tended to precede/anticipate their actions on the interface, whereas the novices exhibited rather a visual search like behavior where they were possibly searching for cues to find the relevant interface elements for the next step. However, none of these qualitative differences translated into significant differences in terms of average fixation duration, saccadic amplitude and prefrontal oxygenation levels during the regular task. In contrast to the regular task, the adverse task condition turned out to be more effective in terms of revealing significant differences between experts and novices. The unanticipated interruption to the task sequence elicited a significant HbT response at bilateral dlPFC for the experts, whereas there was no change with the exception of a minor, non-significant increase over the left dlPFC for the novices. The interaction effect was most salient over the right dlPFC where experts had a stronger HbT increase following the error, a region known to be part of the right fronto-parietal network related to the management of visual attention. In contrast to hemodynamics effects, univariate analyses of fixation and saccadic amplitudes revealed only a marginally significant interaction effect, which is due to the crisscrossing trends in fixation durations for the experts and novices following the error message. Experts' fixation durations increased after the error message, whereas the novices' fixation durations got shorter. A multivariate model exploiting the covariance structure between fixation duration measures and the hemodynamic measures obtained from the right dlPFC could successfully discriminate novices from experts. The bilateral dlPFC response and elongated fixation durations in the case of experts seem to suggest that experts moved from an automated task processing mode towards a more deliberate attentional state in response to the unanticipated error message. Although we could not monitor the parietal cortex in this study, the overlap between the optodes where we observed a significant interaction and the prefrontal components of the right frontoparietal attention network seems to support this interpretation. The case is less clear in the case of novices as their prefrontal responses stayed more or less the same, and they tended to switch to shorter fixation durations. The fact that novices could not provide a reasonable explanation regarding the underlying cause of the error suggests that they had difficulty in interpreting the error message. The obscure language used in the error message itself seemed to have contributed to this issue. Moreover, the error message appeared not immediately after the target was selected, but only a few minutes later while the operator was carrying out the subsequent steps in the launch sequence, which seemed to have contributed to its surprise effect. With the exception of a single operator, the members of the expert group could interpret the meaning of the error message, reportedly based on their past experience with the same problem, whereas novice operators failed to comprehend even whether the error effectively aborted their mission. This highlights the need to consider expertise level and usability aspects hand in hand for cognitive workload assessment. A key limitation of the study is the small sample size, which is due to the limited availability of the operators and the limited access to the real system for research purposes. Since the system was recently acquired, the population of operators who received training and who were authorized to operate the vehicle were quite limited at the time of this study. This limitation precluded us from running a power analysis to determine the optimal sample size for this study. Frontiers in Human Neuroscience | www.frontiersin.org 11 October 2019 | Volume 13 | Article 375 I . şbilir et al. A Multimodal Model of Cognitive Workload Although the data set is rather small for a classification analysis, the multivariate discriminant analysis models revealed some key insights regarding the neurophysiological differences between novices and experts when they are presented an unanticipated error. The best classification performance was achieved when HbT averages observed at the right dlPFC were combined with average fixation durations before and after the error message. When we investigated unimodal classifiers the validation accuracy significantly dropped to chance levels. This suggests that a multivariate model tapping on the covariance structure of eye tracking and fNIRS measures may lead to more robust assessments of expertise. In future work, we plan to expand on additional gaze and neuroimaging features, such as gaze entropy and fronto-parietal as well as fronto-temporal connectivity changes, to further develop our understanding of the neurophysiological factors differentiating expert and novice performance in ecological settings. DATA AVAILABILITY STATEMENT The data that support the findings of this study are available from Roketsan Inc. Restrictions apply to the availability of these data since they were collected in a military platform with active personnel. The dataset can be made available with the permission and authorization of Roketsan Inc. and the TAF. ETHICS STATEMENT The studies involving human participants were reviewed and approved by Middle East Technical University Human Subjects Ethics Committee. The participants provided their written informed consent to participate in this study. AUTHOR CONTRIBUTIONS Eİ contributed to the data collection, data cleaning/synchronization, analysis and writing of the manuscript. MÇ contributed to the data collection, fNIRS analysis, multivariate model development and writing of the manuscript. CA contributed to the study design and analysis, interpretation and writing of eye-tracking data. AT contributed to the design and administrative coordination of the study as well as with data interpretation and writing. REFERENCES Aghajani, H., Garbey, M., and Omurtag, A. (2017). Measuring mental workload with EEG+fNIRS. Front. Hum. Neurosci. 11:359. doi: 10.3389/fnhum.2017. 00359 Ayaz, H. (2010). Functional Near Infrared Spectroscopy Based Brain Computer Interface. Philadelphia, PA: Drexel University. Ayaz, H., Izzetoglu, M., Platek, S. M., Bunce, S., Izzetoglu, K., Pourrezaei, K., et al. (2006). ''Registering fNIR data to brain surface image using MRI templates,'' in Proceedings of the 2006 International Conference of the IEEE Engineering in Medicine and Biology Society (New York, NY: IEEE), 2671–2674. Ayaz, H., Onaral, B., Izzetoglu, K., Shewokis, P. A., McKendrick, R., and Parasuraman, R. (2013). Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: empirical examples and a technological development. Front. Hum. Neurosci. 7:871. doi: 10.3389/fnhum.2013.00871 Ayaz, H., Shewokis, P. A., Bunce, S., Izzetoglu, K., Willems, B., and Onaral, B. (2012). Optical brain monitoring for operator training and mental workload assessment. Neuroimage 59, 36–47. doi: 10.1016/j.neuroimage.2011. 06.023 Ayaz, H., Shewokis, P. A., Curtin, A., Izzetoglu, M., Izzetoglu, K., and Onaral, B. (2011). Using mazesuite and functional near infrared spectroscopy to study learning in spatial navigation. J. Vis. Exp. 56:e3443. doi: 10.3791/ 3443 Bilalić, M., Langner, R., Ulrich, R., and Grodd, W. (2011). Many faces of expertise: fusiform face area in chess experts and novices. J. Neurosci. 31, 10206–10214. doi: 10.1523/JNEUROSCI.5727-10.2011 Bilalić, M. (2017). The Neuroscience of Expertise. Cambridge, MA: Cambridge University Press. Bilalić, M., and Campitelli, G. (2018). ''Studies of the activation and structural changes of the brain associated with expertise,'' in The Cambridge Handbook of Expertise and Expert Performance (2nd Edn.), ed. K. A. Ericsson (New York, NY: Cambridge University Press), 233–254. Borghini, G., Aricò, P., Di Flumeri, G., and Babiloni, F. (2017). Industrial Neuroscience in Aviation: Evaluation of Mental States in Aviation Personnel (Vol. 18). Cham: Springer. Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., and Babiloni, F. (2014). Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci. Biobehav. Rev. 44, 58–75. doi: 10.1016/j.neubiorev.2012.10.003 Borghini, G., Isabella, R., Vecchiato, G., Toppi, J., Astolfi, L., Caltagirone, C., et al. (2011). Brainshield: HREEG study of perceived pilot mental workload. Ital. J. Aerosp. Med. 5, 34–47. Available online at: www.ijasm.altervista.org/index.php/ijasm/article/view/161. Boucsein, W. (2012). Electrodermal Activity. New York, NY: Springer Science + Business Media. Çakır, M. P., Vural, M., Koç, S. Ö., and Toktaş , A. (2016). ''Real-time monitoring of cognitive workload of airline pilots in a flight simulator with fNIR optical brain imaging technology,'' in Proceedings of the 10th International Conference on Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience, Vol. 9743, eds D. D. Schmorrow and C. M. Fidopiastis (New York, NY: Springer-Verlag International Publishing), 147–158. Charles, R. L., and Nixon, J. (2019). Measuring mental workload using physiological measures: a systematic review. Appl. Ergon. 74, 221–232. doi: 10.1016/j.apergo.2018.08.028 Chen, M., Blumen, H. M., Izzetoglu, M., and Holtzer, R. (2017). Spatial coregistration of functional near-infrared spectroscopy to brain MRI. J. Neuroimag. 27, 453–460. doi: 10.1111/jon.12432 De Rivecourt, M., Kuperus, M. N., Post, W. J., and Mulder, L. J. M. (2008). Cardiovascular and eye activity measures as indices for momentary changes in mental effort during simulated flight. Ergonomics 51, 1295–1319. doi: 10.1080/00140130802120267 Diaz-Piedra, C., Rieiro, H., Cherino, A., Fuentes, L. J., Catena, A., and Di Stasi, L. L. (2019). The effects of flight complexity on gaze entropy: an experimental study with fighter pilots. Appl. Ergon. 77, 92–99. doi: 10.1016/j.apergo.2019.01.012 Duncan, J. (2010). The multiple-demand (MD) system of the primate brain: mental programs for intelligent behaviour. Trends Cogn. Sci. 14, 172–179. doi: 10.1016/j.tics.2010.01.004 Duncan, J. (2013). The structure of cognition: attentional episodes in mind and brain. Neuron 80, 35–50. doi: 10.1016/j.neuron.2013.09.015 Durantin, G., Gagnon, J. F., Tremblay, S., and Dehais, F. (2014). Using near infrared spectroscopy and heart rate variability to detect mental overload. Behav. Brain Res. 259, 16–23. doi: 10.1016/j.bbr.2013.10.042 Dussault, C., Jouanin, J. C., Philippe, M., and Guezennec, C. Y. (2005). EEG and ECG changes during simulator operation reflect mental workload and Frontiers in Human Neuroscience | www.frontiersin.org 12 October 2019 | Volume 13 | Article 375 I . şbilir et al. A Multimodal Model of Cognitive Workload vigilance. Aviat. Space Environ. Med. 76, 344–351. Available online at: www. ingentaconnect.com/content/asma/asem/2005/00000076/00000004/art00005. Elbert, T., Pantev, C., Wienbruch, C., Rockstroh, B., and Taub, E. (1995). Increased cortical representation of the fingers of the left hand in string players. Science 270, 305–307. doi: 10.1126/science.270.5234.305 Fairclough, S. H. (2009). Fundamentals of physiological computing. Interact. Comput. 21, 133–145. doi: 10.1016/j.intcom.2008.10.011 Fairclough, S. H., Venables, L., and Tattersall, A. (2005). The influence of task demand learning on the psychophysiological response. Int. J. Psychophysiol. 56, 171–184. doi: 10.1016/j.ijpsycho.2004.11.003 Gateau, T., Durantin, G., Lancelot, F., Scannella, S., and Dehais, F. (2015). Realtime state estimation in a flight simulator using fNIRS. PloS one 10, e0121279. doi: 10.1371/journal.pone.0121279 Gevins, A., and Smith, M. E. (2008). ''Electroencephalography (EEG) in neuroergonomics,'' in Neuroergonomics: The Brain at Work, eds R. Parasuraman and M. Rizzo (Cambridge, MA: Oxford University Press), 15–31. Gratton, G., and Fabiani, M. (2008). ''Optical imaging of brain function,'' in Neuroergonomics: The Brain at Work, eds R. Parasuraman and M. Rizzo (Cambridge, MA: Oxford University Press), 65–81. Guida, A., Gobet, F., and Nicolas, S. (2013). Functional cerebral reorganization: a signature of expertise? Reexamining Guida, Gobet, Tardieu, and Nicolas'(2012) two-stage framework. Front. Hum. Neurosci. 7:590. doi: 10.3389/fnhum.2013.00590 Guida, A., Gobet, F., Tardieu, H., and Nicolas, S. (2012). How chunks, long-term working memory and templates offer a cognitive explanation for neuroimaging data on expertise acquisition: a two-stage framework. Brain Cogn. 79, 221–244. doi: 10.1016/j.bandc.2012.01.010 Hancock, P. A., and Parasuraman, R. (1992). Human factors and safety in the design of intelligent vehicle-highway systems (IVHS). J. Safety Res. 23, 181–198. doi: 10.1016/0022-4375(92)90001-p Hart, S. G., and Staveland, L. E. (1988). ''Development of NASA-TLX (Task Load Index): results of empirical and theoretical research,'' in Human Mental Workload, eds P. Hancock and N. Meshkari (Amsterdam: North Holland), 139–183. Hill, S. G., Iavecchia, H. P., Byers, J. C., Bittner, A. C. Jr., Zaklade, A. L., and Christ, R. E. (1992). Comparison of four subjective workload rating scales. Hum. Factors 34, 429–439. doi: 10.1177/001872089203400405 Hill, N. M., and Schneider, W. (2006). ''Brain changes in the development of expertise: neuroanatomical and neurophysiological evidence about skillbased adaptations,'' in The Cambridge Handbook of Expertise and Expert Performance, eds K. A. Ericsson, N. Charness, P. Feltovich and R. Hoffman (New York, NY: Cambridge University Press), 653–682. Hodges, N. J., Starkes, J. L., and MacMahon, C. (2006). ''Expert performance in sport: a cognitive perspective,'' in The Cambridge Handbook of Expertise and Expert Performance, eds K. A. Ericsson, N. Charness, P. Feltovich and R. Hoffman (New York, NY: Cambridge University Press), 471–489. Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., and Van de Weijer, J. (2011). Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford, UK: Oxford University Press. Izzetoglu, K., Bunce, S., Izzetoglu, M., Onaral, B., and Pourrezaei, K. (2003). ''fNIR spectroscopy as a measure of cognitive task load,'' in Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Vol. 4, (Cancun: IEEE), 3431–3434. Just, M. A., and Carpenter, P. A. (1992). A capacity theory of comprehension: individual differences in working memory. Psychol. Rev. 99, 122–149. doi: 10.1037//0033-295x.99.1.122 Komogortsev, O. V., and Karpov, A. (2013). Automated classification and scoring of smooth pursuit eye movements in the presence of fixations and saccades. Behav. Res. Methods 45, 203–215. doi: 10.3758/s13428-012-0234-9 Lami, M., Singh, H., Dilley, J. H., Ashraf, H., Edmondon, M., Orihuela-Espina, F., et al. (2018). Gaze patterns hold key to unlocking successful search strategies and increasing polyp detection rate in colonoscopy. Endoscopy 50, 701–707. doi: 10.1055/s-0044-101026 Lim, Y., Gardi, A., Sabatini, R., Ramasamy, S., Kistan, T., Ezer, N., et al. (2018). Avionics human-machine interfaces and interactions for manned and unmanned aircraft. Prog. Aerosp. Sci. 102, 1–46. doi: 10.1016/j.paerosci.2018. 05.002 Liu, Y., Ayaz, H., and Shewokis, P. A. (2017a). Multisubject ''learning'' for mental workload classification using concurrent EEG, fNIRS, and physiological measures. Front. Hum. Neurosci. 11:389. doi: 10.3389/fnhum.2017.00389 Liu, Y., Ayaz, H., and Shewokis, P. A. (2017b). Mental workload classification with concurrent electroencephalography and functional near-infrared spectroscopy. Brain Comput. Interfaces 4, 175–185. doi: 10.1080/2326263x.2017.1304020 Marquart, G., Cabrall, C., and de Winter, J. (2015). Review of eye-related measures of drivers' mental workload. Procedia Manuf. 3, 2854–2861. doi: 10.1016/j. promfg.2015.07.783 McCarley, J. S., and Kramer, A. F. (2006). ''Eye movements as a window on perception and cognition,'' in Neuroergonomics: The Brain at Work, eds R. Parasuraman and M. Rizzo (New York, NY: Oxford University Press), 95–112. Mehta, R. K., and Parasuraman, R. (2013). Neuroergonomics: a review of applications to physical and cognitive work. Front. Hum. Neurosci. 7:889. doi: 10.3389/fnhum.2013.00889 Menda, J., Hing, J. T., Ayaz, H., Shewokis, P. A., Izzetoglu, K., Onaral, B., et al. (2011). Optical brain imaging to enhance UAV operator training, evaluation, and interface development. J. Intell. Robot. Syst. 61, 423–443. doi: 10.1007/97894-007-1110-5_26 Modi, H. N., Singh, H., Orihuela-Espina, F., Athanasiou, T., Fiorentino, F., Yang, G. Z., et al. (2018). Temporal stress in the operating room: brain engagement promotes ''coping'' and disengagement prompts ''choking''. Ann. Surg. 267, 683–691. doi: 10.1097/sla.0000000000002289 Nemani, A., Yücel, M. A., Kruger, U., Gee, D. W., Cooper, C., Schwaitzberg, S. D., et al. (2018). Assessing bimanual motor skills with optical neuroimaging. Sci. Adv. 4:eaat3807. doi: 10.1126/sciadv.aat3807 Obrig, H., Wenzel, R., Kohl, M., Horst, S., Wobst, P., Steinbrink, J., et al. (2000). Near-infrared spectroscopy: does it function in functional activation studies of the adult brain? Int. J. Psychophysiol. 35, 125–142. doi: 10.1016/s01678760(99)00048-3 Parasuraman, R. (2011). Neuroergonomics: brain, cognition, and performance at work. Curr. Dir. Psychol. Sci. 20, 181–186. doi: 10.1177/0963721411409176 Parasuraman, R., Christensen, J., and Grafton, S. (2012). Neuroergonomics: the brain in action and at work. Neuroimage 59, 1–3. doi: 10.1016/j.neuroimage. 2011.08.011 Ullén, F., de Manzano, Ö., and Mosing, M. (2018). ''Neural mechanisms of expertise,'' in The Oxford Handbook of Expertise, eds P. Ward, J. M. Schraagen, J. Gore and E. M. Roth (New York, NY: Oxford University Press), 1–24. Wickens, C. D., Hollands, J. G., Banbury, S., and Parasuraman, R. (2013). A model of human information processing. Engineering Psychology and Human Performance. 4th edn. (Upper Saddle River, NJ: Pearson). Wickens, C. D., and McCarley, J. S. (2007). Applied Attention Theory. New York, NY: CRC Press. Conflict of Interest: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. Copyright © 2019 I . şbilir, Çakır, Acartürk and Tekerek. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. Frontiers in Human Neuroscience | www.frontiersin.org 13 October 2019 | Volume 13 | Article


Philosophy

T h e J o u rn a l o f Te c h n o lo g y S tu d ie s 105 Abstract In his consideration of thought development, Auguste Comte proposed a three-stage model, in which the mechanism of development may lead to new types of thought. So the process that led to a philosophy of science may be repeated to create a new type of thought. The thought development is attributed to a process of accumulation of challenged but unanswered questions, followed by a decline of interest in that type of thinking. The science stage presented a huge accumulation of achievements, but at the same time it confronts huge challenges. The present work is a theoretical analysis for what is happening because of these challenges. The author believes that the technology may be regarded as a fourth stage for Comte's model. The technology stage corresponds to globalization as a social stage. Technology has its own methodological way. Owing to the connection between the technology and human need, the development of technology follows a Darwinian evolution model. The process of technology selection that leads to development is the user selection, which corresponds to Charles Darwin's natural selection. Keywords: Philosophy of technology, Threestage model, New scientific spirit, Globalization, Thought evolution, Technology development. Introduction The development of human civilization is associated with the development of knowledge. Auguste Comte traced thought development through time and formulated his law of three stages (Comte, 1855). Comte postulated three theoretical stages of development of thought and society: the theological, the metaphysical, and the positive, which correspond to the fictitious, abstract, and scientific ways of thinking, respectively. Each type has a characteristic logic and methodology. 1. The theological stage tries to attribute natural phenomena to personified deities. Thus, explanations of natural phenomena take the form of stories or legends. This stage may have three subdivisions: animism, polytheism, and monotheism. 2. The metaphysical stage uses a rational explanation style; abstract explanations are developed in this stage. 3. The positive stage tries to discover and explain nature according to science and experimental proofs. Comte (1855) considered social development in a similar set of three stages as well. In his most important work, Comte "explains why the law of the three stages is stated twice. Properly speaking, the law belongs to dynamic sociology or theory of social progress" (Comte, 2008, 4.1). Comte identifies a stage of material development corresponding to each social developmental stage: • The theological stage may also be called the military stage, and the military may be a feature of a primitive society; • The metaphysical stage corresponds to supremacy of the lawyers and jurists; • The positive stage is industrial; industry is based on scientific achievements. Comte, an engineer, was up to date with scientific developments of the Industrial Revolution of the 19th century. Table 1 shows the two sets of developments, that of thought and that of society. In the early years of the last century, there were revolutionary achievements in pure theoretical physics (PTP), including the development of the theory of relativity and quantum physics, which made pure physics the main branch of the modern history of physics. The strong agreement between theoretical predictions and experimental investigations led to great support for that branch of knowledge. The unusually large number of achievements in PTP during the first Is the Technology a New Way of Thinking? Mohammed Sanduk Table 1. Comte's stages Thinking stage Social stage 1. Theological stage Military 2. Metaphysical stage Supremacy of the lawyers and jurists 3. Positive stage Industrial T h e J o u rn a l o f Te c h n o lo g y S tu d ie s three decades of the last century motivated Gaston Bachelard's declaration of his new scientific spirit (Bachelard, 1985) or a new philosophy of science. A new type of physical thought based on probabilities of single particles and the accumulation of new achievements were behind his declaration. However, at that time, there were no recognized serious obstructions confronting quantum physics, one of Kuhn's (1962) scientific revolutions. The present work is a discussion of a possible new phase of thinking, the fourth stage according to Comte's scale. This work is organized as follows: • An overview of the mechanisms of thought development. • A review of the present situation of science research. • Postulation of a new stage, according to the mechanism of development. • Investigation of the features of this new stage. This work is not a philosophical study but a study in philosophy. Thus, the approach will not follow the philosophical argumentation. Instead, an analytical approach for the situation of science and technology is followed. The flow and obstruction model The development of thought can be explained by either dialectical or evolutionary processes. The materialism proposition states that the mere augmentation of a thing or things produces a change in quality and characteristics and, conversely, that a qualitative change produces a quantitative one (Thalheimer, 1936). Accordingly, the deflection of thought to a new direction may be considered a historical necessity. This process can also be considered using Toynbee's (1987) challenge and response theory. In this model, new types of thinking are responses to challenges. The change of thought mainstream is somewhat similar to changes in the direction of flow of water due to an obstruction. In this model, "obstructions" develop from an accumulation of challenging questions or unconvincing answers. To represent how the "mainstream" direction changes, Figure 1 represents the types of thought mainstream with different directions of flow. However, for Comte, "theology declined as it was challenged by scientific spirit, which fulfilled people's need much more effectively" (Pickering, 1993, p. 633). Theological (fictitious) interests declines in favor of metaphysical (abstract) works, and metaphysical interests decline in favor of scientific thought. The decline and challenges might have led Auguste Comte to his proposal of thought development. The diversion of the mainstream thinking to a new type of thinking appears to occur as a result of two types of accumulations: 1. The accumulation of challenged questions (lack of logic). This accumulation works as an obstruction and forces the mainstream to change direction. 2. The accumulation of potential (achievements) to find a new way of thinking. This accumulation is based on the previous achievements of the blocked thinking and provides the potential for the new direction of flow. 106 Figure 2. Flow of thought and effect of obstructions. Figure 1. Changing the direction of thought mainstream (as flow) according to Comte's three mentally conceived stages. T h e J o u rn a l o f Te c h n o lo g y S tu d ie s Figure 1 shows the diversion of thought flow, and Figure 2 represents the flow of thought and the obstruction effect for the three types of thinking. The process of development looks like a mechanism of flow and obstruction. The new stream due to the obstruction is of the new feature. Pure research and the accumulations In Europe, the remarkable shift toward scientific thought occurred during the scientific revolution in the 16th and 17th centuries (Shapin, 1998). Science started to change the direction of how people understood nature and started to affect religious belief. Shapin and many other historians (e.g., Lindberg, 1987) insisted on "Science as Religion's Handmaid," referring to Comte's second stage (metaphysics). Science as knowledge occupied the place of religion in providing alternative interpretation of the natural phenomena and provides solution for some of life's problems. Science without applications (without technology) does not have that effect on life developments of society. It may affect the understanding of nature and then social beliefs or social thought; that is quite clear in comparison the social effect of both of Darwin's evolution theory and Marconi's radio invention. The first led to a religious shock, whereas the second improved the quality of life. Like any new belief or thought, science affects society after any scientific revolution. The historical period that most marks the beginning of science's effect on society was the socalled Industrial Revolution (18th and 19th centuries). That revolutionary era led Comte to the concept of industrial society. The adoption of industry for the science achievements and discoveries led to serious influence of science on society. However, during the first half of the 19th century, the scientific environment became more notable and influential than it had been before for many reasons: • Science was able to explain nature in a more convincing way than either theological or metaphysics theories. • The benefit, persuasiveness, and applicability of scientific knowledge made scientific studies more favorable and popular than theological or metaphysical studies. Subsequently, most universities turned gradually from theological to scientific studies. • There were large developments and an accumulation of scientific achievements. This new type of thought rose as a new challenge to the dominant metaphysical type of thought. It led to slower growth or relative decline in metaphysical (abstract) works and interests compared with the growth of scientific achievements. Then, Positivism became the new direction of thought flow (Figure 2). Thus, the process of increase and decline may mark an era of a new type of thought. In describing the evolution process, Comte's law identifies only three stages, but these may be followed by many other stages as long as there is accumulation and obstruction. In conclusion, the advancement and accumulation of scientific achievements led to a new way of thinking (see Figure 3). New accumulations Through pure research, scientific developments led to two types of accumulation: • The accumulation of a huge number of scientifically proven achievements (scientific fact) that describe nature and can be used for many different applications. • The accumulation of theoretical works without experimental investigation. There are many philosophical attempts to classify and define sciences. Within science communities, the definition of pure or basic research is controversial, and it is far from the definition of basic science; there is no unified definition (Calvert & Martin, 2001). Here, the 107 Figure 3. The problems that change thought. T h e J o u rn a l o f Te c h n o lo g y S tu d ie s focus is on the epistemological and intentional features of research and the definition from the Organisation for Economic Co-operation and Development (OECD). Basic research "is experimental or theoretical work undertaken primarily to acquire new knowledge of the underlying foundations of phenomena and observable facts, without any particular application or use in view" (OECD, 1994). These two accumulations are epistemological in nature. The second accumulation is that of theoretical works that are impossible (at least in the present time) to investigate experimentally. For example, most of the theoretical predictions of PTP in the second half of the 20th century, such as string theory (Schroer, 2008; Smolin, 2006; Woit, 2007), black hole theories, and black matter theories, at different levels (microphysics and cosmological physics) faced and are still facing large obstructions to experimental investigations. Theoretical works accumulate and grow rapidly relative to the level of the slow development of technology of the experimental investigations. As in the example of PTP, either the needed technology is too advanced and sophisticated, or the needed technology is beyond the present level of science (e.g., space travels of superluminal velocity). These problems are not easily overcome and may form knowledge barriers that lead to knowledge boundaries (Sanduk, 2008), as shown in Figure 4. These boundaries resemble ultimate limits at both the microscopic and cosmological levels. Scientists start to confront the feeling their usual probing investigation may be at an end. In PTP, a huge amount of theoretical work is now carried out without investigations, and great numbers of research articles around the world are in need of experimental evidence. These boundaries are neither metaphysical nor faith based. Edmund Husserl (1970) predicted a crisis of science in his article, "The crisis of the European Sciences," but his point of view was based on intentional phenomenology. Aldous Huxley mentioned limits on studies in his fifth novel in which he wrote in 1931 that "we can't allow science to undo its own good work. That's why we so carefully limit the scope of its researches" (Huxley, 1998, p. 227). In this sentence, Huxley may mean an imposed restriction on research, which is different from the limits mentioned previously. However, PTP is a good example for the growth and decay of interest in science. In the first decades of the 20th century the great achievements of PTP made it very popular and garnered a great deal of research interest. However, PTP's honeymoon did not continue. Its decline began in the second half of the 20th century. This became noticeable during the 1990s, when pure physics research grants began to shrink, as did students' interests. The decline was quite obvious and continues today (Cressey, 2008). In addition to the two accumulations of epistemological nature, there is a growth of human demand for a better and comfortable life. Scientists distinguish between pure research and applied research. Applied research aims to solve a particular problem related to direct application. With aid of pure research outputs, applied research can find solutions for human demands. Technology tries to fulfill human needs (de Weck, Roos, & Magee, 2011) Contemporary technology (like IT, nanotech, genetic engineering . . . ) is based on large accumulations of science outputs, which can help in diverse applications and technology. Thus, a large accumulation provides technology with an extremely high potential for rapid growth. Technology depends on two types of research: basic science studies (which has a high accumulation of achievements) and research & development (R&D), which is adopted and supported by industry, government, and others. Interest in basic research declines in favor of applied and development research (or R&D), 108 Figure 4. The boundaries of scientific knowledge. T h e J o u rn a l o f Te c h n o lo g y S tu d ie s which is clearly observed in developed countries. Figures 5 & 6 show the funding of research in the United States during the second half of the last century (the period 1953-2000). Interest in basic research supported by industry declined, compared with research on applications and development. Based on these data, Figure 7 shows the ratio of the funding of development and applied research to the funding of basic research. It is obvious that the federal government, rather than industry, tries to support basic science. However, basic research still earns less support than applied research and development. There is no doubt that the vital role of basic research resulted in the federal government's change in its funding policy after 1958. Nevertheless, the balance remained in favor of applied and development research. Social interest in science may be reflected by the number of graduate students and by employment opportunities. For example, Figure 8 shows the number of earned bachelor's degrees in the United States in different fields during the period 1993-2007. Growth of student 109 Figure 5. U.S. federal government spending development for the basic, applied and development research (1953-2000) (Science and Engineering, Indicators 2002). Figure 6. The U.S. industry spending development for basic, applied and development research (1953-2000) (Science and Engineering Indicators, 2002). Figure 7. The ratio of U.S. spending on R&D to spending on basic science for both federal government and industry sectors (1953-2000). Figure 8. The number of the earned bachelor's degrees in USA for different field at period 1993-2007 (Science and Engineering Indicators, 2010). Figure 9. Employees in science and technology in the USA 1950-2000 (Science and Engineering Indicators, 2010). Figure 10. The ratio of physical scientists to engineers, mathematicians and computer scientists. T h e J o u rn a l o f Te c h n o lo g y S tu d ie s interest is observed in applied fields, such as engineering, biological sciences, and computer sciences, whereas no such significant growth is shown in the physical sciences, such as physics, chemistry, and mathematics. Tracing historical employment in science and technology shows a growth in engineering and computational jobs relative to physical science jobs (Figures 8, 9, and 10). The United Kingdom is a similar case. In a 2008 report titled "Review of UK Physics" (Research Councils UK, 2008), the authors concluded the following: • "There has been a significant decline in recent years of the number of students taking physics at A level (a stage before university) . . . ." • "Physics has a significant impact on the economy and society . . . . Possibly, the most valuable contribution appears to be physics trained graduates, who are highly sought after in many sectors of the economy." • "The Panel heard from many sources that more needs to be done to encourage university-based physicists to work more closely with industry . . . . We found that much of the research work in physics that was of direct interest to business was being performed in departments other than physics in the university sector. This has the effect of reducing the number and size of income streams to physics departments . . . " We focused on the PTP as an example. Yet many branches of sciences are reaching their peak of growth now (like genetic researches), but this is just a growth step similar to what it was like for PTP. The decline is a natural phenomenon. The present pure science situation is similar to that of the decline in metaphysics. Science is the third stage in Comte scale; Does the decline in pure science interest lead to a new deviation in the flow of thinking? Is there a new fourth stage? Because there is always a new accumulation and decline, the evolution of thought may lead to many new philosophical phases. The relationship between science and technology is the relationship between need and knowledge, and historically it is a deep relationship. Following the model of flow and obstruction, a new stream is expected. The previous discussion (the paragraph of New accumulations) shows clearly the real observable growth of applied science. That growth is faster than the growth of basic science, and it is more complicated. Due to the complicated structure of technology (technology is dependent on scientific facts, engineering, politics, trade, etc.), technology develops rapidly as an exponential trend over time. Figure 11 shows the rapid growth of the technology of global ICT and the retardation of others (old technology, like fixed telephone). Mobile technology looks as of exponential trend. The fourth stage is obviously based on science, but it goes beyond science with a more complicated structure. Thus, the next stage is technology (see Figure 12). Philosophy of technology In the late 19th and early 20th centuries, John Dewey was optimistic about the role of technology and adopted a pragmatic view (McDermott, 1981). The intensive interest in this type of thinking started around the middle of the last century, when technological achievements appeared to 110 Figure 11. Global ICT developments, 1998 – 2009 (ITU World Telecommunication/ ICT Indicators database, 2009). Figure 12. The possible fourth stage. T h e J o u rn a l o f Te c h n o lo g y S tu d ie s have manifested some serious effects on human society, such as the problems of the World War II, as identified by Martin Heidegger (Heidegger, 1993). The fast growth of technology and its side effects and misuse led to distinguish it from science. The first clear and focused report on the serious role of technology is the "Mount Carmel Declaration" of 1974 (50 Technion, 1974). The declaration insisted on technology without science. It is the first report written and signed by scientists. The declaration warns of the misuse of technology and puts it in the ranks of the threat of human welfare and survival. Interest in technological thinking was adopted academically, as many philosophers now are considered technology philosophers. Most of the works of these philosophers concern the social or humanitarian effects and history of technology. However, technology is more complicated than these limited considerations. The new generation of engineers and technologists are interested in the philosophical background and the social impact of their works or innovations. Some of them cannot distinguish between the logic of science and the methodology of engineering and technology. However, due to new applications or science products, a new type of methodology has been effectively implemented since the end of the last century (Sanduk, 2003). Technology is unlike science; it starts with applicable invention. Necessity drives invention, as the age-old maxim states. This drive has no full acceptance by the technology historian (The EMELSONMIT Program, 2004). In the present work, the drive of invention is regarded as a type of need (to make life easy, to address physiological needs, etc.). Proving a new idea depends on scientific laws, engineering, then industry, trade, politics, sociology, and more, for production, marketing, and so on. Just as science has its methodology, so technology has its own working logic (Figure 13). In his book Globalization and Technology, Rajneesh Narula presented a diagram relating R&D to technology and science (Narula, 2003, p. 3). A distinct type of thinking has been initiated, apart from the philosophy of science. The new type has multidisciplinary features. Although science is characterized by a single type of research, pure research, the R&D of technology includes the following: • Applied Research • Product Research • Manufacturing Research • Materials Research • Market Research • Operations Research. The technological era is the era of disciplinary mixtures-multidisciplinary or interdisciplinary. It is not a time of separated or isolated sciences. The development of technology Technology has its method of work. It is of a circular nature (Figure 13). Modifications or improvements are expected, and technology advancements occur over time for many different reasons. However, in all cases, the aim of modification is to achieve the fulfilment of need; the best technology is that which best meets the need. Therefore, there are many generations of each type of technological invention (e.g., television, computer, pen). The age of each generation depends on the quality of its performance, the social impact, ethics, market conditions, and so on. The selection of the technology invention in this case is a social selection. On the other hand, some technologies became obsolete (see Figure 11), and these 111 Figure 13. The logic of scientific knowledge and technological methodology. T h e J o u rn a l o f Te c h n o lo g y S tu d ie s technologies disappear gradually (or become extinct). Thus, technology development has features that resemble Darwinian evolution. The technology in each step of development will be accepted if it meets the need of that development, as in Herbert Spencer's phrase "survival of the fittest" (Peel, 1972). The process of technology selection that leads to development is the social (user) selection, which corresponds to Charles Darwin's natural selection. Scientific development occurs through a revolutionary process, according to Kuhn (1962), whereas technological development takes place through a Darwinian type of evolution. The appearance of a new technological invention resembles an evolutionary jump. With different and multiple uses, it will be circulated in many different developmental ways. Many different generations will branch off from that "grandfather." Figure 14 shows a branch of vacuum tube family and the evolution of the television. It is quite clear that the old members of the family are extinct, while the younger generation continues to develop. Globalization Our present societies are not industrial societies; they are more complicated. Technology today is structured in many ways, including industry, economy, politics, and others. The rapid advance of technology creates many types of instabilities (social, personal, global, etc.). The world has become smaller, and any event will receive a rapid response. In 1998, Longworth wrote of the shrinking of the world, "With computer and satellite, currency and stoke traders can do business virtually anywhere at any hour of the day" (Longworth, 1998, as cited in Ramos, 2003, p. 24). Not only have distances decreased but time has shortened as well. In Globalization and Technology, the author examines the interdependence of globalization and technology at two levels, the interdependence between locations, and between corporate entities (Narula, 2003). Humanity looks forward to globalization. Pascal Lamy of the World Trade Organisation (WTO) defines the globalization as "a historical stage of accelerated expansion of market capitalism, like the one experienced in the 19th century with the industrial revolution. It is a fundamental transformation in societies because of the recent technological revolution which has led to a recombining of the economic and social forces on a new territorial dimension" (Lamy, 2006). This definition suggests that the technology stage corresponds to globalization. Table 2 shows a modified model of Comte's stages. There is no technology without globalization and no globalization without technology; both grow interactively at the same time. The comparison between Comte's model as shown in Table 1 and the modified model (Table 2) reflects the continuous developments in the thought and society. Conclusions The previous discussion shows a decline in interest in pure science and a growth in interest in applied science and technology interests. Regarding Comte's model, we may reach to the following conclusions: • Technology can be regarded as a fourth stage for a modified Comte's model, following the positive stage. • The technology stage corresponds to globalization as a social stage. Thus, the technology has its own way of work and development. Technology methodology is quite different from the logic of science. 112 Figure 14. A branch of the vacuum tube family that includes the evolution of the television (Sanduk, 2003). Table 2. The modified model of stages Thinking stage Social stage 1. Theological stage Military 2. Metaphysical stage Supremacy of the lawyers and jurists 3. Positive stage Industrial 4. Technology stage Globalization T h e J o u rn a l o f Te c h n o lo g y S tu d ie s Owing to the connection between the technology and human need, the development of technology follows a Darwinian evolution model. The process of technology selection that leads to development is the social (user) selection, which corresponds to Charles Darwin's natural selection. Acknowledgment The author would like to thank Prof. D. Brandon of Technion-Israel Institute of Technology for supplying a copy of the original document of The Mount Carmel Declaration; thanks also for the National Science Foundation, Division of Science Resources Statistics for their advice on collecting the statistical data. 113 References Bachelard, G. (1985). The new scientific spirit (A. Goldhammer, Trans.) Boston: Beacon Press. Calvert, J., & Martin, B. (2001) Changing conceptions of basic research? Background Document for the Workshop on Policy Relevance and Measurement of Basic Research, Oslo 29-30 October 2001. www.oecd.org/dataoecd/39/0/2674369.pdf (Accessed 13 October 2011) Comte, A. (1855/2003). The positive philosophy of Auguste Comte. (H. Martineau, Trans.). Vol. I, p. 1. Kessinger Publishing. Comte, Auguste. (2008). In (E. N. Zalta, Ed.), Stanford encyclopaedia of philosophy. First published Wed Oct 1, 2008; substantive revision Thu Jun 2, 2011. http://plato.stanford.edu/entries/comte/ (Accessed 2 June 2011) Cressey, D. (2008). Government was warned of physics crisis. Nature. 16 January 2008. de Weck, O. L., Roos, D., & Magee, C. L. (2011), Engineering systems: Meeting human needs in a complex technological world. Cambridge, MA: MIT Press. The EMELSON-MIT Program (2004). Historical perspectives on invention & creativity, School of Engineering, Massachusetts Institute of Technology, 15. http://web.mit.edu/invent/n-pressreleases/downloads/history.pdf. (Accessed 30 September 2011) Heidegger, M. (1993). The question concerning technology (D. Krell, Ed.). (p. 321). New York: HarperCollins Publishers. Husserl, E. (1970). The crisis of European sciences and transcendental phenomenology; An introduction to phenomenological philosophy (D. Carr, Trans.). Evanston: Northwestern University Press. Huxley, A. (1998) Brave new world. New York: Harper Perennial Modern Classics, Ch.17. ITU World Telecommunication/ ICT Indicators database (2009), http://www.ispreview.co.uk/story/2009/10/08/global-mobile-broadband-subscriptions-to-top-600million.html (Accessed 22 October 2011) Kuhn, T. (1962). The structure of scientific revolutions. Chicago: The University of Chicago Press. Lamy, P. (2006). Humanising globalization. Geneva, Switzerland: The WTO; [updated 2006 Jan 30; cited 2006 June 1]. Retrieved September 10, 2011, from http://www.wto.org/english/news_e/sppl_e/sppl16_e.htm. Lindberg, D., (1987). Science as handmaiden: Roger Bacon and the Patristic tradition. Isis, 78(4), 518-26. McDermott, J. J. (1981). The philosophy of John Dewey. Chicago: University of Chicago Press. Narula, R. (2003). Globalization and technology. New York: John Wiley and Sons Ltd. Organization for Economic Co-operation and Development [OECD]. (1994). The measurement of scientific and technical activities: Proposed standard practice for surveys of research and experimental development Paris: OECD (p. 68). Peel, J. D. Y. (Ed). (1972). Herbert Spencer: On social evolution. Chicago: The University of Chicago Press. Pickering, M. (1993). Auguste Comte: An intellectual biography, Volume 1. Cambridge, England: Cambridge University Press, p. 633. T h e J o u rn a l o f Te c h n o lo g y S tu d ie s Ramos, C. (2003). Globalization and Technology. Manila: Rex Bookstore, Inc., p. 24. Research Councils UK (2008). Review of UK physics, panel. Chaired by Professor Bill Wakeham, Swindon. Sanduk, M. I. (2003). Philosophy of technology, Philosophica, 1 (V), pp. 11-20. Sanduk, M. I. (2008). Physics limitations and the philosophy of technology, Workshop on Philosophy & Engineering, WPE2008, The Royal Academy of Engineering London, November 10-12 2008. Retrieved October 13, 2011, from www.raeng.org.uk/societygov/philosophyofeng/pdf/ abstract_papers.pdf Schroer, B. (2008). String theory the crisis in particle physics and the ascent of metaphoric arguments. IJMPD 17, 2373-2431. Shapin, S. (1998), The scientific revolution. Chicago: The University of Chicago Press. Science and Engineering, Indicators (2002). Volume 2– Appendix tables, Chapter 4, Retrieved October 16, 2011, from http://www.nsf.gov/statistics/seind02/pdf_v2.htm Science and Engineering Indicators: 2010 (2010), Appendix tables, Table 2-12, Retrieved October 16, 2011, from http://www.nsf.gov/statistics/seind10/appendix.htm Smolin, L. (2006). The trouble with physics: The rise of string theory, the fall of a science, and what comes next. Boston: Houghton Mifflin Harcourt. Retrieved September 2, 2011, from www.rcuk.ac.uk/documents/reviews/physics/review.pdf 50 Technion Wunsch International Symposium Ethics in an age of pervasive technology, (1974). The Mount Carmel declaration on technology and moral responsibility, Technion – Israel Institute of Technology, Haifa. Thalheimer, A. (1936). Introduction to historical materialism-The Marxist view. New York: Covici Friede. Toynbee, A. (1987). A study of history, Vol. 1: Abridgement of Volumes I-VI. New York: Oxford University Press. Woit, P. (2007). String theory and the crisis in particle physics, Gulbenkian Foundation Conference on Is Science Near Its Limits? 25-26.