INTRODUCTION {#hbm23020-sec-0001}
============

Observing other people\'s actions is a visual behavior at the heart of human activities such as imitation and interactions between conspecifics \[for review see Caspers et al., [2010](#hbm23020-bib-0014){ref-type="ref"}; Grosbras et al., [2012](#hbm23020-bib-0024){ref-type="ref"}; Molenberghs et al., [2012](#hbm23020-bib-0042){ref-type="ref"}; Rizzolatti et al., [2014](#hbm23020-bib-0055){ref-type="ref"}\]. This behavior implies the ability to visually assess the nature of the actions performed by conspecifics, i.e., the goal of their action (what others are doing), as well as how the movements of the effectors allow achieving that goal (how others are doing it).

All imaging studies of action observation have so far used two‐dimensional (2D) videos. Frequently, 2D presentations of actions such as those on television are sufficient to judge what others are doing and how they are doing it. Nevertheless, three‐dimensional (3D) stereoscopic information becomes particularly relevant when viewing actions out of the 2D plane at close range, as when interacting with others nearby. Shaking hands with a friend or avoiding the blow of an opponent becomes difficult with one eye closed, because 2D visual input grossly misestimates the trajectories and kinematics of observed actions. As shown in Figure [1](#hbm23020-fig-0001){ref-type="fig"}A a single velocity vector in one eye corresponds to an infinite number of velocity vectors in 3D space, hence 2D velocity analysis cannot provide accurate speed or direction of bodily motion. Even using multiple 2D views, humans would be slow and imprecise in interacting with others. In fact, stereoscopic information is critical for judging exactly how actions out of the 2D plane are performed \[Jackson and Blake, [2010](#hbm23020-bib-0031){ref-type="ref"}\]. Monocular cues, including changing size, provide precise motion‐in‐depth information for rigid bodies \[Regan and Kaushal, [1994](#hbm23020-bib-0054){ref-type="ref"}\] but do not operate for deformable objects \[Regan and Gray, [2001](#hbm23020-bib-0053){ref-type="ref"}\]. Since the human body is only piecewise rigid, the angles between the rigid body parts cannot be recovered from 2D views, and hence observed actions, which are in essence changes in the angles of joints, cannot be assessed precisely in 2D vision.

![(**A**) Illustration of the projection of a moving object A onto the retinas of the left and right eye. An identical displacement in one eye *d* ~r~ can correspond to an infinite number of possible movements of the object (A~1~--A~2~). (**B**) Factors *Configuration* and *Kinematics* used in our original biological motion study \[Jastorff and Orban, [2009](#hbm23020-bib-0034){ref-type="ref"}\]. (**C**) Factorial design of Kinematics Experiment with the factors *Kinematics* and *Disparity*. (**D**) Factorial design of Configuration Experiment with the factors *Configuration* and *Disparity*. In C and D the dotted lines connect the dots representing the configuration of the human body (not shown in the final stimulus). In the Kinematics Experiment this configuration is maintained in all conditions but the speeds associated with the different dots are manipulated. In the Configuration Experiment the speeds associated with the dots are maintained in all conditions but the configuration is removed by the scrambling. Note that because the body is semi‐rigid in the BM conditions, the configuration in fact refers to constant pairwise distances between dots (see Supporting Information Fig. S1). In the 3D conditions the red and green dots and connecting lines indicate the images of the two eyes, which differ slightly.](HBM-37-203-g001){#hbm23020-fig-0001}

Typical imaging studies investigating action observation have reported activation by action videos within a network comprising three levels in occipitotemporal, parietal and premotor cortex \[Avenanti et al., [2012](#hbm23020-bib-0003){ref-type="ref"}; Buccino et al., [2001](#hbm23020-bib-0010){ref-type="ref"}; Gazzola et al., [2007](#hbm23020-bib-0019){ref-type="ref"}; Grafton and Hamilton, [2007](#hbm23020-bib-0022){ref-type="ref"}; Iacoboni et al., [1999](#hbm23020-bib-0029){ref-type="ref"}; Jastorff et al., [2010](#hbm23020-bib-0032){ref-type="ref"}; Nelissen et al., [2011](#hbm23020-bib-0045){ref-type="ref"}; Rizzolatti and Craighero, [2004](#hbm23020-bib-0056){ref-type="ref"}; Van Overwalle and Baetens, [2009](#hbm23020-bib-0069){ref-type="ref"}\]. Since the seminal study of Backus et al. \[[2001](#hbm23020-bib-0004){ref-type="ref"}\], disparity effects have been described at all three levels \[Chen et al., [2012](#hbm23020-bib-0015){ref-type="ref"}; Georgieva et al., [2009](#hbm23020-bib-0020){ref-type="ref"}; Likova and Tyler, [2007](#hbm23020-bib-0039){ref-type="ref"}; Rokers et al., [2009](#hbm23020-bib-0057){ref-type="ref"}; Tsao et al., [2003](#hbm23020-bib-0064){ref-type="ref"}; Tyler et al., [2006](#hbm23020-bib-0065){ref-type="ref"}\] making it difficult to predict where action‐related disparity effects might occur. However, a recent study has suggested that the two aspects of action observation, the goal and the effectors used to reach the goal, may be processed at the parietal and premotor levels respectively \[Jastorff et al., [2010](#hbm23020-bib-0032){ref-type="ref"}\]. Since stereo would be required for precise visual assessment of how the effectors used allow the actor observed reaching the action goal, our prediction was that disparity effects should occur predominantly at the premotor level. If verified this prediction sets apart 3D action observation from simple stereomotion which activates the occipitotemporal level \[Likova and Tyler, [2007](#hbm23020-bib-0039){ref-type="ref"}; Rokers et al., [2009](#hbm23020-bib-0057){ref-type="ref"}\].

To begin examining stereoscopic action observation, we used biological motion (BM) stimuli. These stimuli, initially developed by Johansson \[[1973](#hbm23020-bib-0036){ref-type="ref"}\], convey human actions by presenting only a few points of light moving as if attached to the principal joints of the person. fMRI experiments contrasting intact point‐light walker with scrambled versions of the same animations, have revealed activations in a number of areas inside and outside the visual pathway, with the extrastriate‐ and fusiform body areas as well as the posterior part of the superior temporal sulcus (pSTS) most consistently reported \[e.g. Beauchamp et al., [2003](#hbm23020-bib-0006){ref-type="ref"}; Grezes et al., [2001](#hbm23020-bib-0023){ref-type="ref"}; Grossman et al., [2000](#hbm23020-bib-0025){ref-type="ref"}; Howard et al., [1996](#hbm23020-bib-0027){ref-type="ref"}; Peuskens et al., [2005](#hbm23020-bib-0051){ref-type="ref"}; Saygin et al., [2004](#hbm23020-bib-0060){ref-type="ref"}; Vaina et al., [2001](#hbm23020-bib-0066){ref-type="ref"}\]. In our previous work, we have investigated BM by manipulating its main cues (Fig. [1](#hbm23020-fig-0001){ref-type="fig"}B): kinematics and configuration \[Jastorff and Orban, [2009](#hbm23020-bib-0034){ref-type="ref"}\]. Our results showed that both cues are processed in distinct anatomical locations within occipitotemporal cortex and that they are integrated within the body areas \[Jastorff and Orban, [2009](#hbm23020-bib-0034){ref-type="ref"}; Jastorff et al., [2012](#hbm23020-bib-0035){ref-type="ref"}\]. In the present study, we investigated the interaction between disparity and the two BM cues in two functional imaging experiments (Fig. [1](#hbm23020-fig-0001){ref-type="fig"}C,D). We opted for BM because the 3D positions and motions of the dots are known and can be manipulated precisely. Also, these stimuli gave us the advantage over natural action videos that we could specify which aspect of visual action processing is enhanced by stereopsis: action kinematics or posture changes resulting from the actions. Interactions between disparity and BM cues would indicate regions where stereo influences the processing of observed actions. Thus we expected enhancement of both aspects by disparity to occur primarily at the premotor level.

MATERIALS AND METHODS {#hbm23020-sec-0002}
=====================

Participants {#hbm23020-sec-0003}
------------

Twenty‐one volunteers participated in both main experiments (11 females; mean age: 23 years, range 19--29). Twelve to 16 volunteers took part in the Control Experiments, testing for confounding factors within the ROIs defined in the main experiments. All participants in the Response (*n =* 16) and Motion‐in‐depth (*n =* 14) Control Experiments had participated in the main experiments. Only three of the initial 21 volunteers participated in the Attention Control Experiment, which was performed later (*n =* 12). All participants were right‐handed, had normal or corrected‐to‐normal vision and no history of mental illness or neurological disorders. Before scanning, all participants underwent training to ensure that they could discriminate between 3D and 2D stimuli while maintaining fixation (see below). The study was approved by the Ethical Committee of Parma Province and all volunteers gave written informed consent in accordance with the Helsinki Declaration before the experiment.

Stimuli {#hbm23020-sec-0004}
-------

The stimuli for the experiments were point‐light (PL) displays \[Johansson, [1973](#hbm23020-bib-0036){ref-type="ref"}\], 7.5° in height, containing 13 light gray dots, 0.3° in diameter, on a dark gray background. Most BM studies have used brief stimuli (1--2 s), and reported only occipitotemporal activations \[Beauchamp et al., [2003](#hbm23020-bib-0006){ref-type="ref"}; Grossman et al., [2000](#hbm23020-bib-0025){ref-type="ref"}; Jastorff and Orban, [2009](#hbm23020-bib-0034){ref-type="ref"}; Peuskens et al., [2005](#hbm23020-bib-0051){ref-type="ref"}; Thompson et al., [2005](#hbm23020-bib-0062){ref-type="ref"}\]. To obtain BM activation at all levels of the action observation network, we used long BM sequences (21 s) portraying complex actions. For the biological motion (BM) conditions of the main experiments and the Response and Attention Control Experiments, dots moved according to motion‐tracking data (frame rate 60 Hz) recorded from the head, shoulders, elbows, wrists, hips, knees and ankles of human actors, obtained from: <http://mocap.cs.cmu.edu/>. For all movements, body displacement was cancelled by subtracting the translation of the hips in each frame, as if performing the movements in place. To include a variety of movements, we chose nine complex action sequences. These included (1) boxing: punching, bobbing, skipping; (2) basketball: forward dribble, 90‐degree turns, crossover dribble; (3) American football: throwing, catching, leaping; (4) aerobics: stretching, jumping jacks, rotation around the hips; (5) breakdance A: spins, flips, handstands; (6) breakdance B: flips, turns, somersaults; (7) dance A: leg and arm movements, rotation around the hips; (8) dance B: legs and arms up and down, jumping laterally, whole‐body rotations; (9) dance C: leg and arm movements, moving sideways. Thus, all sequences portrayed continuous whole‐body movements without interruption or repetition for 21 s.

Right‐ and left‐eye stereoscopic images were generated by projecting the sequences according to a viewing distance of 60 cm. This allowed subjects to fuse the binocular images without undue effort, as they were instructed that the convergence point was at an arm\'s length. To match the viewing distance, the original motion‐tracking data was rescaled to 8 cm in height and viewed binocularly in stereoscopic view (the disparities being rescaled to the same degree as the figure) with convergent fusion across the two eyes. The rescaling reduced the human figure to 7.5°, allowing subjects to perceive the full figure without shifting gaze. A small red square (0.2°) was superimposed onto all individual stimuli. This fixation dot remained at the center of the display, but the center of mass of the point‐light walker was randomly offset up to 1° in the frontoparallel (*x*/*y*) plane from that point to reduce low‐level retinotopic effects. For any given presentation, the offset was constant throughout the video. With respect to the third dimension (*z*), the fixation point was defined at the body center, thus, some point‐lights were in front of the fixation point while others were behind.

The main experiments followed the 2 × 2 factorial design of our previous studies \[Jastorff and Orban, [2009](#hbm23020-bib-0034){ref-type="ref"}; Jastorff et al., [2012](#hbm23020-bib-0035){ref-type="ref"}\], in which one factor modified the *kinematics* by translating a snapshot of the original stimulus in the fronto‐parallel plane and the other manipulated the global shape (*configuration*) by spatially scrambling the starting position of each dot (Fig. [1](#hbm23020-fig-0001){ref-type="fig"}B). In the present study, we introduced *disparity* as a third factor. In order to limit the number of conditions, the 2 × 2 × 2 design was split into two parts where one experiment combined the factors *kinematics* and *disparity* (Kinematics Experiment, Fig. [1](#hbm23020-fig-0001){ref-type="fig"}C) and the other the factors *configuration* and *disparity* (Configuration Experiment, Fig. [1](#hbm23020-fig-0001){ref-type="fig"}D).

### Kinematics experiment {#hbm23020-sec-0005}

This experiment contained five conditions (Fig. [1](#hbm23020-fig-0001){ref-type="fig"}C): (a) 3D biological motion (3D BM): Original motion tracking data of complex actions presented with binocular disparity, i.e. appropriate horizontal shifts between stimulus dots in the two eyes; (b) 2D biological motion (2D BM): same as a) but with disparity removed i.e. identical stimuli for both eyes, either that for the left or the right eye; (c) 3D rotating body (3D RB): a frame of the original video, representative of that sequence, rotated around a 3D axis at each time point (frame) with the same amount of rotation as in the original video (see Supporting Information). Dots in the stimulus moved rigidly in 3D space, but maintained speed differences (Supporting Information Fig. S1, hence structure from motion), unlike in translation in 3D. (d) 2D rotating body (2D RB): Identical to c) but without disparity. (e) 3D rotating shape (3D RS): in this condition, the 13 dots defining the body were rearranged into a different configuration while keeping the overall volume constant (Supporting Information Fig. S2). Thus, the dots were repositioned along their average 3D trajectories in the videos of the 3D RB condition in a manner incompatible with the spatial relationships between human body parts (see Supporting Information). This artificial 3D shape underwent the same rotation in 3D space as the body snapshot used in the 3D RB condition. This condition was a control for the presence of the human figure. In both the 3D RS and 3D RB conditions, the configuration of dots appeared rigid, unlike the 3D BM condition in which the configuration was semi‐rigid.

### Configuration experiment {#hbm23020-sec-0006}

Four conditions were used in this experiment (Fig. [1](#hbm23020-fig-0001){ref-type="fig"}D): (a) 3D BM, (b) 2D BM, (c) 3D scrambled motion (3D SCR): a constant random horizontal and vertical offset was added to the original movement of each dot, chosen such that dot density and stimulus size matched the original values. Scrambling preserved the original disparities (Supporting Information Fig. S1), (d) 2D scrambled motion (2D SCR): identical to condition (c) but without disparity. Additional details about stimulus generation and a comparison of eccentricity, absolute value of the disparity, and speed in *x*, *y*, and *z* directions between the conditions can be found in the Supporting Information.

### Response control experiment {#hbm23020-sec-0007}

In both main experiments subjects made judgments about the presence of 3D information, signaled by button presses. Premotor interactions might therefore reflect motor responses, or the task and its cognitive requirements in general. Therefore we presented in the Response Control Experiment the same stimuli as in the Configuration Experiment but subjects viewed the videos passively.

### Attention control experiment {#hbm23020-sec-0008}

In principle, interactions between disparity and the BM cues might reflect an attentional effect, whereby the 3D BM appears more interesting to subjects than control conditions. This is particularly true for the Configuration Experiment, in which intact 3D stimuli might be more salient than scrambled ones, yielding an interaction that could result from attentional shifts, in particular those in depth \[Chen et al., [2012](#hbm23020-bib-0015){ref-type="ref"}\]. To rule out this possibility we performed a control experiment, repeating the Configuration Experiment with one additional feature. During the presentation of each stimulus, 5 of the 13 PL were dimmed for 200 ms, by decreasing their luminance by 10%. The dimmed points were chosen pseudo randomly on different limbs, to ensure attention to the complete display. Five dimmings occurred at random intervals spanning from 3 s to 20 s from the start of a block and subjects were required to respond to each dimming by a button press. Dimming parameters were adjusted to yield 75% correct detection, based on a pilot study involving different subjects.

### Motion‐in‐depth control experiment {#hbm23020-sec-0009}

To control for low‐level nonarticulated stereoscopic motion‐in‐depth, the third control experiment used an annulus moving in depth \[Likova and Tyler, [2007](#hbm23020-bib-0039){ref-type="ref"}\] and included three relevant conditions (two other conditions with limited life‐time dots were also included but not analyzed). The first, 3D motion, presented the random dot stereograms (RDS, 3 arc min dots at 10% density) of Likova and Tyler \[[2007](#hbm23020-bib-0039){ref-type="ref"}\] to generate their cyclopean stereomotion condition, but dot lifetime equaled that of the stimuli (as in BM conditions) and the stimulus was restricted to an annulus with constant inner (3°) and outer (10.2°) diameters. The outer diameter was adjusted to the maximum size of the BM stimuli, while the central gap facilitated maintenance of fixation. The annulus disparity varied linearly at 0.33 Hz between ±20 min arc disparity (rather than jumping between extreme disparities) thus generating continuous motion in depth similar to the BM stimuli. The range of disparities matched those in the BM stimuli. Because of the long dot life‐time the 3D motion condition included two motion‐in‐depth cues: changing disparity and differences in monocular velocity. The two other conditions controlled for these two cues. The fixed disparity condition was similar to the one of Likova and Tyler \[[2007](#hbm23020-bib-0039){ref-type="ref"}\], with life‐time and size changes as in the 3D motion condition and displaying RDS at four fixed disparities (±20 min arc and ±6.6 minarc), presented in random order. In the final condition, 2D motion, the annulus moved in the frontoparallel plane by displaying the stimuli of the 3D motion condition except that the velocity was identical for both eyes.

Procedure {#hbm23020-sec-0010}
---------

Before scanning, subjects participated in a test session in the laboratory outside the hospital, during which all action sequences and all conditions of the Configuration Experiment were shown three times as red/green anaglyphs (nine movements × four conditions × three presentations = 108 trials). The subjects pressed buttons to indicate as quickly as possible whether the stimulus was 3D or 2D. Only those subjects averaging at least 90% correct were scanned. Subsequently, participants were familiarized with the conditions wearing the head mounted display outside the scanner and instructed to maintain fixation on the central target throughout the experiment.

### Kinematics experiment {#hbm23020-sec-0011}

A single time‐series (run) of the experiment included six conditions (four stimulus conditions of the factorial design plus 3D shape condition and fixation baseline), presented in blocks lasting 21 s for stimulus conditions and 24 s for fixation. A 3 s interstimulus interval (ISI), showing only the fixation dot, followed each stimulus condition. Subjects were asked to signal in this interval whether the preceding stimulus had been 3D or 2D by pressing one of two buttons on an MR‐compatible button box. Half responded using the right thumb, the other half responded with the left thumb. No response was required after the fixation condition. This task provided evidence that subjects used stereoscopic vision in the scanner in all experimental conditions. In each run, the six conditions were shown three times, yielding 18 (3 × 6) blocks per run. Order of the conditions was randomized across the six conditions and counterbalanced across subjects. For each run, three of the nine action sequences were selected pseudo‐randomly. The whole experiment included nine runs, thus every sequence was shown three times in total. Each run lasted 432 s and started with four dummy volumes to assure that the MR signal had reached steady state.

### Configuration experiment {#hbm23020-sec-0012}

Identical procedure to that of the Kinematics Experiment, with the exception that only the fixation condition was added to the four factorial conditions. Thus, one run included only 15 blocks and lasted 360 s.

### Response control experiment {#hbm23020-sec-0013}

The procedure was identical to the Configuration Experiment, except that subjects did not respond. Thus all blocks lasted 21 s, with no inter‐stimulus interval following experimental conditions (runs of 315 s).

### Attention control experiment {#hbm23020-sec-0014}

Conditions, timing and response to 3D/2D as in the Configuration Experiment, but subjects additionally reported each dimming by a button press within 2 s. Half used the right thumb, the others the left. We still asked subjects to report on the 3D/2D nature of the stimuli to ensure that they were able to detect whether the stimulus was presented with or without depth cues. Otherwise the absence of an interaction could have been related to differences in attentional allocation (the question the experiment was designed for) or simply to an inability to perceive the depth in the stimulus.

### Motion‐in‐depth control experiment {#hbm23020-sec-0015}

Six conditions were tested in 18 s blocks: three blocks devoted to the three experimental conditions of interest, each corresponding to six motion‐in‐depth cycles, two blocks with 3D motion and fixed disparity with limited life time that were not analyzed further, and one fixation block. These six conditions were presented three times in a single run. Eight runs (with different block orders) were acquired per subject in a single session.

Presentation and Data Collection {#hbm23020-sec-0016}
--------------------------------

Participants lay supine in the scanner bore with the response buttons (fMRI 4‐Button Diamond Fiber Optic Response Pad, Current Designs, Inc., Philadelphia, PA) under their thumb. Visual stimuli were presented via a head mounted display (60 Hz) with a resolution of 800 horizontal × 600 vertical pixels (Resonance Technology, Inc. Northridge, CA) for each eye. The display was controlled by an ATI Radeon 2400 DX dual output video card (AMD, Sun Valley, CA), allowing a stretched desktop presentation (1,600 horizontal pixels) corresponding to the images of the two eyes. Thus the display provided separate images for each eye, without shuttering. Sound‐attenuating headphones were used to muffle scanner noise and give instructions. Stimulus presentation and the recording of participants' responses were controlled by E‐Prime software (Psychology Software Tools, Inc., Sharpsburg, PA). To reduce head motion during scanning, the head was padded with PolyScan™ vinyl coated cushions. Throughout the session, eye movements were recorded with an infrared eye tracking system (60 Hz, Resonance technology Inc, Northridge, CA).

Scanning used a 3T MR scanner (GE Discovery MR750, Milwaukee, ILL) with an parallel channels receiver coil, located at the University Hospital of the University of Parma. Functional images were acquired using gradient‐echoplanar imaging with these parameters: 49 horizontal slices (2.5 mm slice thickness; 0.25 mm gap), repetition time (TR) = 3 s, time of echo (TE) = 30 ms, flip angle = 90°, 96 × 96 matrix with FOV 240 (2.5 × 2.5 mm in plane resolution), and ASSET factor of 2. The 49 slices contained in each volume covered the entire brain from cerebellum to vertex. A 3D high‐resolution T1‐weighted IR‐prepared fast SPGR (BRAVO) image covering the entire brain, acquired in one session, was used for anatomical reference. Its acquisition parameters were: TE/TR 3.7/9.2 ms; inversion time 650 ms, flip‐angle 12°, acceleration factor (ARC) 2; 186 sagittal slices acquired with 1 × 1 × 1 mm^3^resolution. A single scanning session lasted about 90 min.

Data Analysis {#hbm23020-sec-0017}
-------------

Data analysis was performed using the SPM8 software package (Wellcome Department of Cognitive Neurology, London, UK) running under MATLAB (The Mathworks, Inc., Natick, MA). Preprocessing steps involved: (1) realignment of images, (2) coregistration of the anatomical image and mean functional image, (3) spatial normalization of all images to standard stereotaxic space (MNI) with a voxel size of 2 × 2 × 2 mm and (4) smoothing of the resulting images with an isotropic Gaussian kernel of 8 mm (5 mm for the Attention Control Experiment testing only for a small premotor site).

### Statistical analysis of the main experiments {#hbm23020-sec-0018}

For every participant, onset and duration of each condition was modeled by a General Linear Model (GLM). The design matrix was composed of six regressors modeling the five experimental conditions and the fixation condition in the Kinematics Experiment and five regressors in the Configuration Experiment. In both experiments additional regressors included: (1) a regressor modeling the ISI between stimulus onsets to exclude variance related to the motor response (button press) of the subject; (2) six regressors obtained from motion correction in the realignment process to account for voxel intensity variations due to head movement; and (3) three regressors, modeling the local speeds in *x*, *y*, and *z* directions, because of a significant reduction in *z*‐speed in the 3D RB condition compared with the 3D BM condition (see Supporting Information Fig. S3). All regressors were convolved with the SPM canonical hemodynamic response function. Subsequently, we calculated contrast images for each participant for the two main effects and interaction, which entered the second‐level random effects analysis \[Holmes and Friston, [1998](#hbm23020-bib-0026){ref-type="ref"}\]. At this second level, the interaction was inclusively masked with the contrast 3D BM versus 2D RB and the contrast 3D BM versus 2D scrambled motion at a low threshold (*P \<* 0.05 uncorrected) in the Kinematics and Configuration Experiments respectively. This was done to ensure that the interaction, to which both 3D BM and 2D RB or 2D scrambled contribute positively, reflected increased activity in the 3D BM condition. The significance level for main effects and interaction was set at *P \<* 0.05 FWE corrected. Sites could reach this level in two ways: (1) their local maximum reached *P \<* 0.05 FWE corrected at the cluster level in the whole brain analysis; (2) their local maximum reached *P \<* 0.05 FWE small volume correction within predefined regions of interest (ROIs). Our a priori hypothesis was that the main effects of the BM cues and also the interactions with disparity should be located within the action observation network (AON). ROIs of the AON were defined based on Jastorff et al. \[[2010](#hbm23020-bib-0032){ref-type="ref"}\], investigating fMRI responses to the visual presentation of human hand, foot, and mouth actions, and included 2,601 and 3,048 voxels in left and right occipitotemporal cortex, 3,604 and 3,055 voxels in left and right parietal cortex and 846 and 1,090 voxels in left and right premotor cortex. Our a priori hypothesis for the main effect of disparity was derived from Tsao et al. \[[2003](#hbm23020-bib-0064){ref-type="ref"}\] and Tyler et al. \[[2006](#hbm23020-bib-0065){ref-type="ref"}\], indicating that a swath of occipitoparietal areas including KO and V7 are sensitive to depth structure. Hence, we used spherical ROIs with 10mm diameter as ROIs, centered on the coordinates reported in Tyler et al. \[[2006](#hbm23020-bib-0065){ref-type="ref"}, KO\] and Georgieva et al. \[[2009](#hbm23020-bib-0020){ref-type="ref"}, V7\]. Activation sites were projected (enclosing voxel projection) onto the PALS template \[Van Essen, [2005](#hbm23020-bib-0067){ref-type="ref"}, <http://sumsdb.wustl.edu:8081/sums/directory.do?id=636032>\] using Caret software \[Van Essen et al., [2001](#hbm23020-bib-0068){ref-type="ref"}, <http://brainvis.wustl.edu/caret>\].

### Statistical analysis of the control experiments {#hbm23020-sec-0019}

Control experiments included fewer subjects and served to test potential confounds related to the main experiments. Therefore, instead of investigating whole brain responses, statistical analysis focused on local maxima identified in the main experiments. In the Response and the Motion‐in‐depth Control Experiments we used the coordinates obtained from the main experiments, as these subjects also took part in the main experiments. In the Attention Control Experiment, testing a new group of subjects, we searched for effects in a spherical ROI surrounding the local maximum identified in the Configuration Experiment.

The statistical analysis of the Response Control Experiment was identical to the Configuration Experiment, except for the removal of the regressor modeling the button press. Comparison of the fMRI activation between the Configuration Experiment and the control was restricted to the 16 subjects who had participated in both.

For the Attention Control Experiment, we included one additional regressor modeling the dimming responses. Otherwise the analysis was identical to the Configuration Experiment.

In the statistical analysis of the Motion‐in‐depth Control Experiment, the design matrix consisted of 12 regressors corresponding to the five experimental conditions, one baseline fixation and 6 realignment parameters. Subsequently, the GLM was estimated for each participant at the first level. Next, % MR signal changes were determined for the local maxima obtained in the main experiments and averaged across subjects.

### BOLD activation profiles {#hbm23020-sec-0020}

For all experiments, the BOLD activation profiles represent the MR signal change relative to fixation in % of the average signal. They were first computed for individual subjects, averaging the response in 27 voxels surrounding the group local maxima obtained from the group analyses, and subsequently averaged across subjects. We averaged 27 voxels instead of using a single voxel (local maximum) to obtain a more representative estimate of the response profile of the region. BOLD activation profiles were intended to verify the visual nature of the responses by comparing them to fixation and to confirm that the interaction was driven by stronger 3D BM activation compared with other conditions.

RESULTS {#hbm23020-sec-0021}
=======

Main Experiments: Behavior {#hbm23020-sec-0022}
--------------------------

During scanning, the 21 subjects fixated the target in the center of the display. Eye movement recordings confirmed that subjects fixated well during the Kinematics Experiment, averaging 8 to 13 saccades/min in the six conditions, with no significant difference across conditions (one‐way repeated measure ANOVA *F* ~(5,15)~ = 1.4, *P \>* 0.22). Similarly, in the Configuration Experiment, subjects averaged 6 to 10 saccades/min in the various conditions, with no significant difference across conditions (one way repeated measure ANOVA *F* ~(4,16)~ = 1.3, *P \>* 0.25).

After each video presentation subjects judged it as 2D or 3D (for 3D RS they always responded 3D). They averaged over 85% correct responses for all conditions of the Kinematics Experiment (Fig. [2](#hbm23020-fig-0002){ref-type="fig"}A). A two‐way repeated measures ANOVA comparing performances across conditions showed a significant interaction (*F* ~(1,20)~ = 13.3, *P \<* 0.01), while main effects were nonsignificant (disparity *F* ~(1,20)~ = 0.37, *P \>* 0.5; kinematics *F* ~(1,20)~ = 0.04, *P \>* 0.8). This interaction resulted from enhanced performance in the 2D BM and 3D RB conditions, exactly the opposite pattern we were seeking in terms of fMRI responses. In the Configuration Experiment (Fig. [2](#hbm23020-fig-0002){ref-type="fig"}B), subjects reached over 90% correct in all conditions. Performance across conditions showed no significant differences (main disparity *F* ~(1,20)~ = 0.36, *P \>* 0.5, main configuration *F* ~(1,20)~ = 0.28, *P \>* 0.6, interaction *F* ~(1,20)~ = 1.4, *P \>* 0.2).

![(**A**--**C**) Behavioral performance measured as percent correct responses for the different conditions during the scanning sessions of the Kinematics Experiment (A), the Configuration Experiment (B) and the Attention Control Experiment (C); (**D**) percent correct detection of the dimming of the dots in the Attention Control Experiment. (**E**) Example of eye traces of left and right eyes during one block of the Attention Control Experiment.](HBM-37-203-g002){#hbm23020-fig-0002}

Main Experiments: Imaging {#hbm23020-sec-0023}
-------------------------

### Main effects of BM cues {#hbm23020-sec-0024}

The visual processing of actions requires the integration of kinematic and configural information \[Giese and Poggio, [2003](#hbm23020-bib-0021){ref-type="ref"}; Jastorff and Orban, [2009](#hbm23020-bib-0034){ref-type="ref"}\]. Therefore, our hypothesis was that both cues should lead to activation within the action observation network (AON). Activations for both main effects are shown in Figure [3](#hbm23020-fig-0003){ref-type="fig"}A, overlaid on a flatted left and right hemisphere of the PALS atlas. Indeed, both the kinematics and the configuration main effects yielded activation sites at the three cortical levels of the AON, indicated by the white outlines in Figure [3](#hbm23020-fig-0003){ref-type="fig"}A. For illustrative purposes, activations are plotted at a lower significance level (*P \<* 0.001 uncorrected), although, only sites that survived *P \<* 0.05 FWE correction will be discussed.

![Statistical parametric maps (SPMs) showing activated voxels (threshold *P \<* 0.001 uncorrected) for *Kinematics* and *Configuration* main effects (light and dark green, **A**), *Disparity* main effects (light and dark blue, **B**) in the main Experiments, projected onto the left and right hemispheres of the PALS atlas using Caret software. In A and B flatmaps are shown, in A the inflated hemispheres are also shown. Regions of overlap are indicated in yellow in A and in greenish blue in B. Numbered spheres give the location of the significant local maxima listed in Table [1](#hbm23020-tbl-0001){ref-type="table"}. Blue outlines indicate hMT+ in posterior ITS \[Jastorff and Orban, [2009](#hbm23020-bib-0034){ref-type="ref"}\]; white outlines in A indicate the a prior ROIs of the AON \[Jastorff et al., [2010](#hbm23020-bib-0032){ref-type="ref"}\]. Blue ellipses show the approximate locations along the intraparietal sulcus (IPS) of ventral IPS (VIPS), parieto‐occipital IPS (POIPS), dorsal IPS medial (DIPSM), dorsal IPS anterior (DIPSA) and putative human anterior intraparietal (phAIP) regions in caudal to rostral order \[Jastorff et al., [2010](#hbm23020-bib-0032){ref-type="ref"}\]. SFS: superior frontal sulcus; IFS: inferior frontal sulcus; preCS: precentral sulcus; CS: central sulcus; IPS: intraparietal sulcus; STS: superior temporal sulcus; OTS: occipitotemporal sulcus.](HBM-37-203-g003){#hbm23020-fig-0003}

In the Kinematics Experiment, the main effect of kinematics (light green, or yellow if it overlaps with the configuration main effect) was slightly biased towards the left hemisphere (LH). Significant activation sites (light green or yellow numbered spheres) in LH included fusiform gyrus, posterior middle temporal gyrus (MTG)/superior temporal sulcus (STS), and posterior superior temporal gyrus (STG) in occipitotemporal cortex, supramarginal gyrus in parietal cortex, and ventral part of the superior branch of precentral sulcus, (PrCSs), superior frontal gyrus (SFG) and cingulate sulcus in frontal cortex. Significant right hemisphere activation sites were restricted to posterior MTG/STS, posterior STG and supramarginal gyrus (Table [1](#hbm23020-tbl-0001){ref-type="table"}).

###### 

Significant activation sites in Main Experiments

                                                                        Left hemisphere   Right hemisphere                                             
  --------------------------- ----------------------------------------- ----------------- ------------------ ----- ----- ----- ---- ----- ----- ------ -----
  Kinematics experiment                                                                                                                                
  Main effect kinematics                                                                                                                               
  1                           Fusiform gyrus                            −42               −42                −20   5.4   x                             
  2                           Mid. temp. gyrus                          −48               −56                6     9.6   x     46   −62   8     12.0   x
  3                           Sup. temp. gyrus                          −52               −40                12    5.2   x     50   −42   20    6.5    x
  4                           Supramarginal gurys                       −60               −38                24    7.5   x     62   −36   24    7.9    x
  5                           Premotor cortex                           −40               −8                 50    4.4   SVC                           
  6                           Post. sup. front. gyrus                   −10               −4                 70    5.4   x                             
  7                           Cingulate gyrus                           −8                −26                46    5.2   x                             
  Main effect disparity                                                                                                                                
  8                           Cuneus                                    −24               −84                14    5.2   x     28   −84   18    5.5    x
  9                           V7                                        −24               −80                32    2.8   SVC   26   −84   24    4.6    SVC
  Interaction                                                                                                                                          
  11                          Mid. temp. gyrus/post. inf.temp. sulcus   −34               −80                −2    5.3   x     52   −68   2     7.0    x
  12                          Sup. par. lobe                            −30               −48                52    5.2   SVC                           
  13                          Premotor cortex                           −30               −10                50    4.2   SVC   40   −4    60    4.2    SVC
  Configuration Experiment                                                                                                                             
  Main effect configuration                                                                                                                            
  1                           Fusiform gyrus                            −40               −52                −22   8.1   x     40   −54   −20   11.8   x
  14                          Inf. occ. gyrus                           −48               −80                −4    8.2   x     42   −82   −6    11.0   x
  2                           Mid. temp. gyrus                          −46               −62                4     5.1   x     50   −60   4     6.0    x
  3                           Sup. temp. gyrus                                                                                 52   −40   22    5.9    x
  4                           Supramarginal gyrus                       −62               −32                26    5.7   x     60   −40   24    7.3    x
  15                          Premotor cortex                           −24               −4                 48    4.2   SVC   44   2     58    4.9    SVC
  Main effect disparity                                                                                                                                
  8                           Cuneus                                    −20               −88                14    6.2   x     24   −84   16    7.0    x
  9                           V7                                        −26               −78                30    4.9   SVC   26   −82   32    4.0    SVC
  10                          KO                                        −32               −86                4     4.0   SVC   30   −86   2     3.2    SVC
  Interaction                                                                                                                                          
  16                          Premotor cortex                           −40               0                  50    5.5   x                             

SVC: small volume correction.

In the Configuration Experiment, the main effect of configuration again yielded activation (dark green or yellow if there is overlap with the kinematics main effect) of all three levels of the action observation network, but emphasized occipitotemporal cortex (Fig. [3](#hbm23020-fig-0003){ref-type="fig"}A). Significant activation sites (dark green and yellow numbered spheres) in occipitotemporal cortex were centered on the human Middle Temporal (hMT)+ complex (blue outlines) bilaterally, extending into posterior MTG/STS/STG, fusiform gyrus/occipitotemporal sulcus (OTS) and inferior occipital cortex (Table [1](#hbm23020-tbl-0001){ref-type="table"}). Frontoparietal sites included supramarginal gyrus and premotor cortex bilaterally. The right premotor site was located on the ridge between inferior and superior branches of PrCS, with the left premotor site more dorsal in the PrCSs. These activation sites overlapped those of the Kinematics Experiment (yellow) in pMTG bilaterally, left occipitotemporal sulcus, right pSTG, left supramarginal gyrus, and left cingulate.

Thus the two main‐effect networks were relatively similar, but not identical. This finding was in agreement with Jastorff and Orban \[[2009](#hbm23020-bib-0034){ref-type="ref"}\], showing that regions involved in different aspects of the actions, such as the kinematics or relative position of body parts, are differently influenced by one or the other cue removal.

### Main effects of disparity {#hbm23020-sec-0025}

Figure [3](#hbm23020-fig-0003){ref-type="fig"}B shows activations (*P \<* 0.001 uncorrected) for the main effect of disparity. These were very similar in the two experiments and largely restricted to occipitoparietal cortex with a significant (*P \<* 0.05 FWE correction) activation in dorsal occipital cortex extending into the ventral intraparietal sulcus (VIPS)/parieto‐occipital intraparietal sulcus (POIPS) regions. When using our a priori ROIs as search volume, significant (*P \<* 0.05 FWE small volume correction) main effects of disparity were observed in area KO for the Configuration Experiment and in V7 for both experiments (Table [1](#hbm23020-tbl-0001){ref-type="table"}).

### Interactions of disparity and BM cues {#hbm23020-sec-0026}

In the Kinematics Experiment, interaction occurred bilaterally at all three levels of the action observation network (orange, Fig. [4](#hbm23020-fig-0004){ref-type="fig"}A). Significant sites included bilateral posterior inferior temporal (ITG)/MTG sites, overlapping the kinematics and configuration main‐effects and the hMT+ complex (blue outlines); a left SPL site at the angle between postcentral sulcus and intraparietal sulcus (IPS), anterior to DIPSA; and bilateral premotor premotor sites (numbered orange spheres Fig. [4](#hbm23020-fig-0004){ref-type="fig"}A). The left premotor site was located in the PrCSs near the junction with superior frontal sulcus (SFS) and between the kinematics and configuration main‐effect sites; the right site approached the ridge separating superior and inferior branches of precentral sulcus, extending onto the precentral gyrus (PrCG), overlapping with the configuration main‐effect site. The individual premotor interaction sites for the 21 subjects are shown in Supporting Information Figure S4. The mean vector distance of individual local maxima from the group local maximum was 7 and 9 mm in left and right hemispheres, respectively.

![(**A**) SPMs showing activated voxels (threshold *P \<* 0.001 uncorr.) for the interactions *Disparity* and *Kinematics* (orange, Kinematics Experiment), and between *Disparity* and *Configuration* (red, Configuration Experiment), projected onto the left and right hemispheres of the PALS atlas using Caret software. Numbered spheres indicate locations of the significant local maxima listed in Table [1](#hbm23020-tbl-0001){ref-type="table"}. (**B**) BOLD activation profiles for left and right occipitotemporal, left, parietal, left and right premotor interaction sites (Kinematics Experiment). The 3D shape condition was not part of the factorial design. (**C**) BOLD activation profiles for the left premotor interaction site (Configuration Experiment). Same conventions as Figure [3](#hbm23020-fig-0003){ref-type="fig"}.](HBM-37-203-g004){#hbm23020-fig-0004}

Whole brain analysis thus revealed interactions between disparity and kinematics in premotor, parietal and occipitotemporal sites. BOLD activation profiles of the premotor interaction sites (Fig. [4](#hbm23020-fig-0004){ref-type="fig"}B) showed that all conditions yielded visual responses and that the interaction was driven by a stronger response to 3D BM than the three other conditions. These profiles provide no new information, although they enable us to compare the differences in visual responses between the 3D BM and the three other conditions across the interaction sites by computing the "reduction" in these latter conditions. The reduction was defined as the difference between the 3D BM response and the average of the three other responses divided by the 3D BM response. This reduction gradually increased from occipitotemporal sites (LH: 10%, RH: 19%) over parietal (LH: 25%) to premotor sites (LH: 32%, RH: 31%).

BOLD activation profiles also show that the activity evoked by 3D RB exceeded that for 3D RS in all interaction sites (Fig. [4](#hbm23020-fig-0004){ref-type="fig"}B), indicating that these are sensitive to the form of the human body. This effect was stronger at parietal and occipitotemporal levels and was significant after correction for five comparisons in left parietal (*t* ~20~ = 5.07) and left and right occipitotemporal sites (LH: *t* ~20~ = 3.62, *P \<* 0.01; RH: *t* ~20~ = 8.93, *P \<* 0.001). It proved nonsignificant in right (*t* ~20~ = 2.67, *P \<* 0.05), and left (*t* ~20~ = 1.91, *P =* 0.07) premotor sites. Note that this difference could partially reflect the significantly smaller absolute disparity in the 3D RS condition compared with the 3D RB condition (Supporting Information Fig. S3, *t* ~8~ = 19.44, *P \<* 10^−5^).

In the Configuration Experiment, the interaction was limited to a single site in the anterior bank of left inferior branch of the precentral sulcus (PrCSi, Fig. [4](#hbm23020-fig-0004){ref-type="fig"}A), well below the premotor interaction of the Kinematics Experiment (vector distance of 14 mm). The local maxima for interactions in the individual subjects are shown in Supporting Information Figure S4. Their vector distance from the group local maximum averaged 7 mm. The premotor BOLD activation profile is shown in Figure [4](#hbm23020-fig-0004){ref-type="fig"}C. As expected, the interaction reflects the stronger MR response to the 3D BM condition. In comparison, responses to the three other conditions averaged 37% lower. Probing the symmetrical site in the right hemisphere showed that left‐right asymmetry in this experiment was not due to threshold effects, as the right showed no interaction (*F* ~(1,20)~ = 0.3, *P =* 0.57).

The Kinematics Experiment thus revealed interactions between *disparity* and *kinematics* at all levels of the action observation network. However, the difference between activation for 3D BM and the three other conditions gradually increased from occipitotemporal over parietal to premotor levels. The Configuration Experiment revealed an interaction between *disparity and configuration*, restricted to left premotor cortex. In order to compare the premotor interaction sites across the two experiments, we analyzed the BOLD activation profiles for the Configuration Experiment in the local maxima of the Kinematics Experiment and vice versa. Indeed, sites showing interaction between kinematics and disparity did not show an interaction between configuration and disparity (all five ANOVAs n.s.). On the other hand, the interaction site of the Configuration Experiment in left premotor cortex also showed a weak interaction between kinematics and disparity (*F* ~(20,1)~ = 4.9; *P \<* 0.05).

Response Control Experiment {#hbm23020-sec-0027}
---------------------------

The following two analyses were performed to investigate whether response or task was influencing the activation in premotor cortex. First, we evaluated % MR signal changes for the control experiment in the 27 voxels surrounding the local maximum of the left premotor site in the Configuration Experiment. The BOLD activation profile was extremely similar, with a 30% reduction in the three non‐BM conditions. The premotor site showed a significant interaction even when no response was required of the subjects (two‐way repeated measures ANOVA: *F* ~(1,15)~ = 4.98, *P \<* 0.05), indicating that disparity and configuration interacted, even without task. Yet the interaction could differ between the active (Configuration Experiment) and the passive (Response Control Experiment) conditions. To rule out this possibility we performed a three‐way repeated‐measure ANOVA at the premotor local maximum of the Configuration Experiment, with factors *configuration, disparity, and experiment*, including only the 16 subjects common to both experiments. This analysis showed significant interaction between the factors configuration and disparity (*F* ~(1,15)~ = 27.86, *P \<* 0.001), but no significant three‐way interaction (*F* ~(1,15)~ = 2.7, *P =* 0.12). Even though the same subjects were included in the Configuration Experiment and the Response Control Experiment, it seems unlikely that they were subconsciously preparing a response in the control experiment, because they were not provided response boxes and both scans were separated in time by several weeks. Thus, this first control experiment confirmed that premotor interactions obtained in the main experiments were not due to the motor response or the task, but related to the visual stimulus conditions.

Attention Control Experiment {#hbm23020-sec-0028}
----------------------------

In this control experiment we recorded eye positions binocularly, to explicitly control for vergence eye movements. The subjects, detected the dimming epochs with over 75% correct (Fig. [2](#hbm23020-fig-0002){ref-type="fig"}C) while still performing the 2D/3D task well, averaging 80% (Fig. [2](#hbm23020-fig-0002){ref-type="fig"}D). Performance in 2D/3D discrimination did not differ between conditions (main disparity: *F* ~(1,11)~ = 1.11, *P =* 0.31, main configuration: *F* ~(1,11)~ = 1.43, *P =* 0.26, interaction: *F* ~(1,11)~ = 3.93,*P =* 0.07). Dimming detection varied slightly (main disparity: *F* ~(1,11)~ = 3.37, *P =* 0.09, main configuration: *F* ~(1,11)~ = 1.2, *P =* 0.30, interaction: *F* ~(1,11)~ = 5.12, *P \<* 0.05). The interaction in the dimming detection performance was due to better performance in 3D scrambled and is thus an unlikely source of interaction in cortical sites more responsive in the 3D BM condition. The subjects fixated well, averaging 3 to 5 saccades/min in the various conditions, without significant differences between conditions (*F* ~(4,7)~ = 0.21, *P \>* 0.8). Eye movement traces documented vergence eye movements (Fig. [2](#hbm23020-fig-0002){ref-type="fig"}E). To test for differences in binocular vergence we measured the standard deviations of differences between left and right eye positions. The standard deviation of this position difference averaged 0.55° across conditions, with no significant difference between conditions (*F* ~(4~,~7)~ = 0.01, *P \>* 0.9).

To investigate, whether an interaction would also be observed when attention was equalized across conditions, we defined a search volume of 7.5 mm radius that was centered on the local maximum for the interaction in left premotor cortex obtained in the Configuration Experiment (−40, 0, 50). A site within our search volume with the local maximum at −48, 8, 52 displayed a significant interaction (*P \<* 0.05 FWE small volume correction). The BOLD activation profile (Fig. [5](#hbm23020-fig-0005){ref-type="fig"}) again confirms that the 3D BM condition drives the premotor site more strongly than the other conditions. Indeed, the visual response to the other three conditions hovered around zero and was sharply reduced (over 100%) compared with the 3D BM condition. We also tested the motor response in this site, using the regressor for button presses. The strong activation for this condition confirmed that the interaction site is indeed located in premotor cortex. This control experiment ruled out attentional and vergence eye movement confounds.

![Attention Control Experiment. (**A**) The sphere illustrates the location of the group local maximum in the left premotor cortex in the control experiment. Red patch indicates significant voxels for interaction in the Configuration Experiment. (**B**) BOLD activation profile for the left premotor interaction site for the Attention Control Experiment.](HBM-37-203-g005){#hbm23020-fig-0005}

Motion‐in‐Depth Control Experiment {#hbm23020-sec-0029}
----------------------------------

In this control experiment, we contrasted an annulus of constant size moving in depth (3D motion) with the annulus at four different fixed disparities (fixed disparity), or moving laterally in the front‐parallel plane (2D motion). For interaction sites to be selectively activated by non‐articulated motion in depth, the 3D motion condition should produce activations significantly higher than either control condition. This was tested in the ROIs corresponding to the six interaction sites yielded by the Kinematics and Configuration Experiments correcting for 12 tests. Figure [6](#hbm23020-fig-0006){ref-type="fig"} shows the BOLD activation profiles of these six interaction sites. No site was more strongly activated in the 3D motion condition compared with either of the control conditions (see figure legends for statistics).

![Motion‐in‐depth Control Experiment. BOLD activation profiles of the five interaction sites from the Kinematics Experiment and the premotor interaction site from the Configuration Experiment. Numbers refer to those in Table I. Asterisk indicates significant difference (correcting for six comparisons) with 3D motion. **[11]{.ul}**: LH: 3D vs. fixed: *t =* 3.1 *P \<* 0.01; 3D vs. 2D: *t =* 0.1 *P =* 0.94. RH: 3D vs. fixed: *t =* 5.0 *P \<* 0.001; 3D vs. 2D: *t =* 2.7 *P \<* 0.05. **[12]{.ul}**: 3D vs. fixed: *t =* 4.7 *P \<* 0.001; 3D vs. 2D: *t =* 3.0 *P \<* 0.05. **[13]{.ul}**: LH: 3D vs. fixed: *t =* 3.1 *P \<* 0.01; 3D vs. 2D: *t =* 1.9 *P =* 0.08. RH: 3D vs. fixed: *t =* 3.0 *P \<* 0.01; 3D vs. 2D: *t =* 1.5 *P =* 0.15. **[16]{.ul}**: 3D vs. fixed: *t =* 3.4 *P \<* 0.01; 3D vs. 2D: *t =* −0.2 *P =* 0.83. For comparison the two contrasts were jointly significant in −*22*, −*88, 6*, a site located close to our main effects of disparity (3D vs. fixed: *t =* 7.3 *P \<* 0.001; 3D vs. 2D: *t =* 4.0 *P \<* 0.003).](HBM-37-203-g006){#hbm23020-fig-0006}

DISCUSSION {#hbm23020-sec-0030}
==========

The main effects of BM cues indicate that the long‐lasting and complex BM stimuli activated the action observation network at all three cortical levels. Therefore, our design allowed us to test where interactions with stereo occur along the action observation pathway. The interactions between *disparity and configuration* reach significance solely in premotor cortex, those between *disparity and kinematics* reached significance in occipitotemporal, parietal and premotor cortex, but were strongest in premotor cortex.

Main Effects of BM Cues {#hbm23020-sec-0031}
-----------------------

The main effects of BM cues significantly activated the action observation network beyond occipitotemporal cortex, including supramarginal gyrus and premotor cortex. To our knowledge, parietal activation by BM has not previously been reported. Frontal activations by BM \[Jastorff et al., [2009](#hbm23020-bib-0033){ref-type="ref"}; Jastorff and Orban, [2009](#hbm23020-bib-0034){ref-type="ref"}; Saygin et al., [2004](#hbm23020-bib-0060){ref-type="ref"}\] were observed outside PMC \[but see Santi et al., [2003](#hbm23020-bib-0059){ref-type="ref"}\]. Activation of all three levels of the action observation network in this study probably reflects the deliberate choice of longer video sequences portraying complex actions rather than the usual brief clips of locomotion.

Several studies suggest that premotor activity in the action observation network reflects the effectors used in the actions \[Buccino et al., [2001](#hbm23020-bib-0010){ref-type="ref"}; Jastorff et al., [2010](#hbm23020-bib-0032){ref-type="ref"}; Wheaton et al., [2004](#hbm23020-bib-0072){ref-type="ref"}\]. The premotor locations of kinematics and configuration main‐effects correspond to regions activated by the observation of foot and hand actions (Fig. [7](#hbm23020-fig-0007){ref-type="fig"}) according to Jastorff et al. \[[2010](#hbm23020-bib-0032){ref-type="ref"}\], consistent with the whole body movements (hand and foot) portrayed in the BM sequences. On the other hand, the parietal level is likely organized according to action type \[Abdollahi et al., [2013](#hbm23020-bib-0001){ref-type="ref"}; Jastorff et al., [2010](#hbm23020-bib-0032){ref-type="ref"}\]. The activations located in supramarginal gyrus may reflect hand actions present in our BM sequences \[Grosbras et al., [2012](#hbm23020-bib-0024){ref-type="ref"}; Kroliczak and Frey, [2009](#hbm23020-bib-0038){ref-type="ref"}; Morrison et al., [2013](#hbm23020-bib-0043){ref-type="ref"}; Newman‐Norlund et al., [2010](#hbm23020-bib-0048){ref-type="ref"}\].

![Flatmaps (restricted to precentral sulci/gyri) summarizing the results of the Main Experiments in left and right premotor cortex. Light green and dark green outlines illustrate the *Kinematics* and *Configuration* main effects respectively, light red and dark red the interactions of the Kinematics and Configuration Experiments respectively. The elongated solid outline indicates the premotor part of the a priori ROI from Jastorff et al. \[[2010](#hbm23020-bib-0032){ref-type="ref"}\], with purple, blue and pink portions indicating regions equally activated by hand and foot action observation, dominated by foot or hand action, respectively. White stippled outline indicates right phF5c \[Ferri et al., [2015](#hbm23020-bib-0016){ref-type="ref"}\]. Blue and black circles show locations of the stereo effects in Chen et al. \[[2012](#hbm23020-bib-0015){ref-type="ref"}\] and interaction sites of Georgieva et al. \[[2009](#hbm23020-bib-0020){ref-type="ref"}\], respectively. Blue and black squares indicate sites involved in hand movement observation from Caspers et al. \[[2010](#hbm23020-bib-0014){ref-type="ref"}\] and Grosbras et al. \[[2012](#hbm23020-bib-0024){ref-type="ref"}\], respectively. Black stippled lines indicate the potential border between dorsal and ventral premotor cortex defined by Tomassini et al. \[[2007](#hbm23020-bib-0063){ref-type="ref"}\]. Black stippled ellipse indicates the frontal eye field defined in Hutchison et al. \[[2012](#hbm23020-bib-0028){ref-type="ref"}\] \[summarizing data from Amiez et al., [2006](#hbm23020-bib-0002){ref-type="ref"}; Braun et al., [1992](#hbm23020-bib-0009){ref-type="ref"}; Ford et al., [2005](#hbm23020-bib-0017){ref-type="ref"}; Luna et al., [1998](#hbm23020-bib-0040){ref-type="ref"}; Paus, [1996](#hbm23020-bib-0050){ref-type="ref"}\].](HBM-37-203-g007){#hbm23020-fig-0007}

Finally both of the main effects also activated the occipitotemporal cortex, but these regions showed some differences between the two main effects (Fig. [3](#hbm23020-fig-0003){ref-type="fig"}A). Both activated the rostral part of hMT+ (blue outline) extending into the pMTG bilaterally as well as left pSTG and right OTS/fusiform gyrus. The kinematics main‐effect activated posterior STS and STG of the left hemisphere, while the configuration main‐effect activated regions caudal to hMT+ and OTS/fusiform gyrus predominantly on the right. These results are in agreement with Jastorff and Orban \[[2009](#hbm23020-bib-0034){ref-type="ref"}\] indicating that the kinematics cue predominantly activated a strip of cortex crossing posterior MTG, STS, and STG, probably corresponding to rostral monkey STS upper bank, while configuration primarily activated the OTS/fusiform gyrus overlapping with LOC, putatively identified as the homologue of rostral monkey STS lower bank \[Caspari et al., [2014](#hbm23020-bib-0013){ref-type="ref"}; Jastorff et al., [2012](#hbm23020-bib-0035){ref-type="ref"}; Vanduffel et al., [2014](#hbm23020-bib-0071){ref-type="ref"}\].

Main Effect of Disparity {#hbm23020-sec-0032}
------------------------

The main effect of disparity, which reveals stereo‐effects independently of configuration and/or exact motion pattern of the dots, activated sites located bilaterally in dorsal occipital cortex (cuneus) extending into parieto‐occipital cortex at the level of VIPS. This site reaffirms earlier studies consistently reporting activation of occipitoparietal cortex centered on V3A and V7 by disparity stimuli \[Backus et al., [2001](#hbm23020-bib-0004){ref-type="ref"}; Baecke et al., [2009](#hbm23020-bib-0005){ref-type="ref"}; Nishida et al., [2001](#hbm23020-bib-0049){ref-type="ref"}\]. The extend of the disparity main effects matched the activation reported by Tsao et al. \[[2003](#hbm23020-bib-0064){ref-type="ref"}\] and included KO at its ventral edge as predicted from Tyler et al. \[[2006](#hbm23020-bib-0065){ref-type="ref"}\]. Activation patterns in this region are specific for correlated stereograms indicating that they reflect the perceived depth \[Preston et al., [2008](#hbm23020-bib-0052){ref-type="ref"}\]. Although sensitivity to absolute disparities was initially emphasized \[Neri et al., [2004](#hbm23020-bib-0046){ref-type="ref"}\], several subsequent studies indicated a sensitivity of this occipitoparietal region for disparity structure \[Tsao et al., [2003](#hbm23020-bib-0064){ref-type="ref"}; Tyler et al., [2006](#hbm23020-bib-0065){ref-type="ref"}\]. Additional studies have implicated this region in the discrimination of orientation in depth of surfaces \[Naganuma et al., [2005](#hbm23020-bib-0044){ref-type="ref"}; Shikata et al., [2001](#hbm23020-bib-0061){ref-type="ref"}\] and in the processing of 3D shape \[Georgieva et al., [2009](#hbm23020-bib-0020){ref-type="ref"}\]. Thus the available evidence suggests that the extensive parieto‐occipital site of the disparity main effect processes depth structure, which was present in 3D BM and 3D RB stimuli, but also in the 3D scrambled stimuli.

Premotor Interaction Sites {#hbm23020-sec-0033}
--------------------------

Although interactions with *configuration and kinematics* both reached significance in premotor cortex, the interactions differed in several respects. The *configuration* interaction was left lateralized, more restricted, and located ventrally to the main effect of kinematics (Fig. [7](#hbm23020-fig-0007){ref-type="fig"}). *Kinematics* interactions were bilateral and located between the main effects of configuration and kinematics in left premotor cortex, while overlapping with the main effect of kinematics on the right. Thus stereopsis enhances the visual processing of action kinematics and body configuration, or at least relative position of moving body parts, in different parts of premotor cortex. The parts devoted to enhanced kinematics were located near the boundary separating dorsal and ventral PMC \[horizontal black dotted line, Tomassini et al., [2007](#hbm23020-bib-0063){ref-type="ref"}\], and caudal and ventral to frontal eye field sites \[black dotteted ellipses, defined as in Hutchison et al., [2012](#hbm23020-bib-0028){ref-type="ref"}, Fig. [7](#hbm23020-fig-0007){ref-type="fig"}\]. They were associated with sensitivity to observation of hand and foot actions on both sides \[Jastorff et al., [2010](#hbm23020-bib-0032){ref-type="ref"}\]. This matches the properties of the stimuli used, as 12 of the 13 dots of the BM displays represented limbs and their movements. Furthermore, wrists carried the greatest disparities in these displays, followed by ankles, elbows, and knees (Supporting Information Fig. S3). The association of interaction with only the more dorsal sites sensitive to hand and foot actions may reflect the absence of distal joints (fingers and toes) in our PL stimuli \[Sakreida et al., [2005](#hbm23020-bib-0058){ref-type="ref"}\]. The same factor may also apply to the partial match with the meta‐analyses for hand movement observation (squares in Fig. [7](#hbm23020-fig-0007){ref-type="fig"}) of Caspers et al. \[[2010](#hbm23020-bib-0014){ref-type="ref"}\] and Grosbras et al. \[[2012](#hbm23020-bib-0024){ref-type="ref"}\]. Configuration interaction, reflecting enhanced processing of body posture, or at least relative position of moving body parts, showed no obvious association with action observation sites from the previous studies, but its premotor nature was supported by its activation by button presses.

Premotor stereo effects have been reported in studies mapping regions involved in 3D shape from disparity \[Georgieva et al., [2009](#hbm23020-bib-0020){ref-type="ref"}\] and deploying attention in depth \[Chen et al., [2012](#hbm23020-bib-0015){ref-type="ref"}\]. The kinematics sites are relatively close to the Chen et al. \[[2012](#hbm23020-bib-0015){ref-type="ref"}\] sites and the left dorsal site of Georgieva et al. \[[2009](#hbm23020-bib-0020){ref-type="ref"}; Fig. [7](#hbm23020-fig-0007){ref-type="fig"}\]. The configuration interaction site is also near the left Chen et al. \[[2012](#hbm23020-bib-0015){ref-type="ref"}\] site. It is unlikely that these interaction sites correspond to those of Chen et al. \[[2012](#hbm23020-bib-0015){ref-type="ref"}\]. Those sites were activated by attentional shifts between 50 and 150 cm from the subjects' eyes while the BM dots were portrayed between 45 and 75 cm. Furthermore a control experiment excluded any contribution of attention in 3D to the interaction between configuration and disparity (see below). The left kinematics interaction may correspond to the dorsal Georgieva et al. \[[2009](#hbm23020-bib-0020){ref-type="ref"}\] site, consistent with its sensitivity to 3D shape from motion, shown by the difference between the 3D RB and 3D RS conditions.

There are several indirect indications that mirror neurons \[Gallese et al., [1996](#hbm23020-bib-0018){ref-type="ref"}; Kilner and Lemon, [2013](#hbm23020-bib-0037){ref-type="ref"}\] in monkey F5 utilize disparity: a number of mirror neurons respond more strongly to live actions than to videos of the same action \[Caggiano et al., [2011](#hbm23020-bib-0011){ref-type="ref"}\] and mirror neuron responses depend on the distance to the action being performed \[Caggiano et al., [2009](#hbm23020-bib-0012){ref-type="ref"}\]. The right premotor kinematics interaction site is located just dorsal from the recently identified human homologue of monkey F5c \[Ferri et al., [2015](#hbm23020-bib-0016){ref-type="ref"}; white dotted outline, Fig. [7](#hbm23020-fig-0007){ref-type="fig"}\], a part of F5 housing mirror neurons \[Rizzolatti et al., [2014](#hbm23020-bib-0055){ref-type="ref"}\]. Monkey F5 is known to have two other parts, F5a and F5p, in addition to F5c. Since it has been suggested that the homologue of F5a is located ventrally from phF5c \[Ferri et al., [2015](#hbm23020-bib-0016){ref-type="ref"}; Neubert et al., [2014](#hbm23020-bib-0047){ref-type="ref"}\], one could speculate the kinematics interaction site located more dorsally is the homologue of F5p. Whatever the homologies, present results show that human premotor cortex has access to 3D information about kinematics and configuration, including relative body‐part position.

Kinematics interactions corresponded to smaller response differences in occipitotemporal and parietal cortex, while configuration interaction was absent at these levels. The stronger reduction of premotor activity in the non 3D BM conditions is unlikely to reflect the lower premotor response levels as reductions are computed across conditions for a given site. Alternatively these response differences could be due to greater variability in the premotor sites, but the ratio of SE/average response is similar in premotor and parietal levels. Globally the results support the view that stereo effects on BM are stronger at the premotor than the parietal or occipitotemporal levels. These results are consistent with (1) stereo effects on BM arising at the level of occipitotemporal cortex where BM is extracted by the integration of motion and shape cues, and (2) the proportion of neurons encoding BM in 3D being small at this level and increasing towards the premotor cortex. Several studies do suggest that both 3D shape and 3D motion cues are encoded in or near hMT+ where BM cues are integrated \[Jastorff and Orban, [2009](#hbm23020-bib-0034){ref-type="ref"}\]: this region is sensitive to motion in depth \[Likova and Tyler, [2007](#hbm23020-bib-0039){ref-type="ref"}; Rokers et al., [2009](#hbm23020-bib-0057){ref-type="ref"}\] and neighboring phPIT regions are sensitive to 3D shape from disparity \[Georgieva et al., [2009](#hbm23020-bib-0020){ref-type="ref"}\].

Lower‐order visual contributions to disparity‐BM interactions were minimized by stimulus design, factorial block designs, and by including local speeds as variables of no‐interest. Effects of motion in depth were specifically ruled out by a control experiment. Influence of eye movements in the frontoparallel plane was minimized by the fixation requirements, shown to be effective in all experiments. The Attention Control Experiment indicated that vergence eye movements explained very little of the premotor interactions.

Subjects performance in the main experiments was either similar across conditions (Configuration Experiment), or followed a pattern opposite to expectation for fMRI (Kinematics Experiment), with the Attention Control Experiment ruling out any accounting of results based on reduced attention in conditions yielding better performance. It could be argued that the 2D/3D discrimination tasks acted as a distracter from any processing associated with action observation. However, both task and motor effects were ruled out by the Response Control Experiment without a task.

Finally the Attention Control Experiment argues against interactions being due to attention effects. Dimming detection is an efficient way to draw attention to a continuous stimulus \[Vandenberghe et al., [2001](#hbm23020-bib-0070){ref-type="ref"}\], hence the unpredictability of the dimmings ensured that attention was allocated to the PL stimuli over most of each block. Similar detection performance across the various conditions strongly suggests that subjects deployed attention to the PL stimuli similarly in all four conditions. Yet, the results of the Attention Control Experiment were similar to the ones of the Configuration Experiment.

In conclusion, the difference in activation between the 3D BM and the 3 other conditions of the Kinematics Experiment underlying the interaction, increased gradually from occipitotemporal through parietal to the premotor level. In the Configuration Experiment, the interaction was significant only in premotor cortex. Thus the results meet our predictions that disparity should influence the action observation network predominantly at the premotor level.

Functional Interpretation {#hbm23020-sec-0034}
-------------------------

Since stereopsis is required for precise assessment of 3D kinematics and relative 3D positions of body parts, the observation that interactions between disparity and BM factors are robust at the premotor level, indicates that premotor cortex contributes to a more precise visual description of actions out of the 2D plane. Stimuli were seen at a smaller than natural size, exactly as when viewing videos on screens, and hence the present study provides the first insights into how we process 3D movies and 3D television \[IJsselsteijn et al., [1998](#hbm23020-bib-0030){ref-type="ref"}\]. Since we used reduced action stimuli on an empty background, more work is needed to understand how these findings generalize to other viewing situations, or to more realistic action videos.

The finding that premotor cortex provides a more precise visual description of actions out of the 2D plane is consistent with the view that this region plays a key role in the recognition of others actions \[Rizzolatti et al., [2014](#hbm23020-bib-0055){ref-type="ref"}\]. Alternatively, or additionally, this more precise description may benefit the role of the premotor cortex in motor planning. Indeed our findings show that the premotor cortex possesses the information necessary to plan interactions with conspecifics or to imitate their actions. This latter role is consistent with several studies implicating ventral premotor cortex in action imitation \[Brass and Heyes, [2005](#hbm23020-bib-0008){ref-type="ref"}; Iacoboni et al., [1999](#hbm23020-bib-0029){ref-type="ref"}; Molenberghs et al., [2009](#hbm23020-bib-0041){ref-type="ref"}\]. Planning interactions with conspecifics has also been suggested as a function for mirror neurons in monkey F5 \[Caggiano et al., [2009](#hbm23020-bib-0012){ref-type="ref"}\]. In this view, human ventral premotor cortex has the 3D information necessary to plan actions towards conspecifics (present study) as well as to objects \[Georgieva et al., [2009](#hbm23020-bib-0020){ref-type="ref"}\]. This parallel planning of actions towards conspecifics or objects might explain why many monkey F5 neurons have both mirror and canonical characteristics \[Bonini et al., [2014](#hbm23020-bib-0007){ref-type="ref"}\]. Whatever their exact functional role, the premotor regions exhibiting interaction between disparity and action‐specific factors are likely to play a key role in assessing 3D motion and relative 3D position of human body parts when observing others' actions.

Supporting information
======================

###### 

Supporting Information

###### 

Click here for additional data file.

J.J. is a postdoctoral fellow of the FWO. J.J., R.O.A., and G.A.O. designed the experiment, J.J. and R.O.A. generated the stimuli, R.O.A. acquired the data, F.F. assisted in data acquisition, J.J. and R.O.A. analyzed data, J.J. and G.A.O. wrote the article. All authors discussed the results and implications and commented on the article.

[^1]: Jan Jastorff and Rouhollah O. Abdollahi contributed equally to this work.
