Abstract
Human sensitivity to 3D gross shape changes was measured for the visual and haptic sensory channels. Three volume-invariant affine transformations were defined: compressing, shearing and stretching. Participants discriminated a reference 3D object (cube or sphere) from its deformed shape under three experimental conditions: visual only (on a computer monitor), haptic only (through a point-contact force-feedback device) and visuohaptic simulations. The results indicate that vision is more sensitive to gross shape changes than point-based touch, and that vision dominated in the visuohaptic condition. In the haptic alone condition, thresholds were higher for shearing and stretching than for compressing. Thresholds were otherwise similar for the three transformations in the vision only or visuohaptic conditions. These trends were similar for the two shapes tested. A second experiment, conducted under similar conditions but preventing participants from manipulating object orientations, verified that the main conclusion of our research still holds when visual inspection can rely only on a single perspective view of the object. Our earlier studies on 3D visuohaptic watermarking showed that the haptic channel is more sensitive to surface texture and roughness changes than vision. The thresholds from the present and our earlier studies can potentially be used as the upper limits for selecting watermark strengths in order to ensure watermark imperceptibility in a 3D visuohaptic watermarking system.
Introduction
Perception of 3D shapes through 2D retinal projections is a remarkable ability of human vision. From a computational point of view, the recognition of 3D scenes from 2D images is ill posed in the sense that the solution is not unique and prone to noise perturbation (see Pizlo, 2001 , for a review). Scientists have studied the nature of shape perception for about a thousand years, yet it remains an active research area in many fields including psychology, neuroscience, physics, computer science and engineering (see Pizlo, 2008) .
Introduction
As an initial study of human visual perception, Alhazen (1083 Alhazen ( /1989 ) developed a theory explaining the process of vision by rays of light reaching the eye from points on an object. He also defined shape constancy to refer to the fact that an object keeps its perceived shape despite changes in its retinal projections (see Pizlo and Stevenson, 1999 , for a review on shape constancy). A subsequent research area is the sources of information needed to resolve ambiguity associate with 3D shapes. Examples include variations of image intensity or shading, surface contours, color, size, and line drawings that represent edges and vertices of 3D objects (Todd, 2004) . Empirically, psychophysical experiments were conducted on shape perception with various cues such as depth, curvature, length from simple lines, sphere and cube and complex objects (Hecht et al., 1999; Kleffner and Ramachandran, 1992; Koenderink et al., 1996 Koenderink et al., , 1997 Koenderink et al., , 2000 Norman et al., 1996) .
Introduction
Compared to visual shape perception, haptic shape perception involves the combination of information from the cutaneous stimulation of skin surface and kinesthetic sense of joint positions. Although the haptic channel does not suffer from many of the visual ambiguities caused by projective transformations, there also exist many illusions in haptic shape perception (Hayward, 2008) . Nonetheless, studies by Klatzky et al. (1985) reported that haptic object recognition can be both rapid and accurate. Many scientists have investigated how the brain integrates inputs from individual receptors to form the percept of global shapes through the haptic channel (Reed et al., 2004; Zangaladze et al., 1999) . Others have studied multimodal shape perception and found that vision generally dominates perception when information from two modalities (e.g., vision and touch) are in conflict (Rock and Victor, 1964) , but progressively more weight is given to haptics when visual information becomes less reliable (Ernst et al., 2002; Helbig and Ernst, 2007) . Moreover, the integrated visuohaptic estimates are more reliable than those from either modality alone. These findings should be carefully considered when shape perception through both vision and touch is investigated.
Introduction
Our interest in visuohaptic perception of 3D object shapes originated from an ongoing research program on watermarking of 3D objects. As haptic technology matures, it is becoming easier to see and touch 3D virtual objects at the same time. With the development of 3D digital media suitable for both visual and haptic rendering, the need will soon arise to protect 3D visuohaptic contents from misuse. An important requirement of any 3D visuohaptic watermarking scheme is the imperceptibility of embedded watermarks. Previous studies of visuohaptic watermarking have investigated the perceptibility of additive noise embedded on object surfaces (Formaglio et al. 2006; Prattichizzo et al., 2005 Prattichizzo et al., , 2007 . This kind of modification to 3D meshes is akin to modifying the surface details of 3D objects. The results of the earlier studies suggest that touch is more sensitive than vision to changes in surface roughness due to watermarking.
Introduction
It is arguable that touch may not remain more sensitive than vision if watermarks are embedded in such a way that only the gross shape of the object is changed, since vision is better at global perception than touch. The goal of the present study is therefore to investigate the perceptibility of gross shape deformation of 3D objects through vision and touch. The results will inform the design of visuohaptic watermarking systems that embed watermarks in both the object shape and surface details of 3D objects.
Introduction
In the present study, three types of deformations (compressing, shearing, and stretching) of 3D gross shapes were designed and implemented in a PC environment. Two psychophysical experiments were conducted to estimate the discrimination thresholds for 3D gross shape in Euclidean space using two simulated basic shapes (cube and sphere) over three conditions (vision alone, haptics alone and visuohaptic). In the first experiment, participants were able to manipulate an object's orientation with the computer mouse and to reset the object's orientation by pressing the 'R' key on the keyboard. It could be argued that some participants may orient objects in such a way as to reduce a 3D shape discrimination task to a 2D length discrimination task. Therefore, in the second experiment, the object's orientation was fixed throughout the entire trial. In both experiments, the initial orientation of each object presentation was randomized.
Introduction
In Section 2, we describe 3D gross shape deformation and its simulation in a virtual environment, followed by the design of the psychophysical experiment. Thresholds of the two experiments are reported in Section 3. Concluding remarks are drawn in Section 4.
General Methods

Gross Shape Deformation
We use the term gross shape to refer to the global shape of a 3D object in Euclidean space, regardless of the surface details such as color or texture. In this section, we describe three linear deformations we had developed to manipulate the 3D gross shapes of cubes and spheres used in the present study.
Gross Shape Deformation
Among possible deformations of 3D object shapes, we focused on the more common types of compressing, shearing and stretching, which can be simulated by affine transformations. They are defined as follows:
Gross Shape Deformation
• compressing makes the gross shape of an object more compact by pressing in one direction;
Gross Shape Deformation
• shearing deforms the gross shape in which parallel planes remain parallel but are shifted in a direction parallel to themselves; • stretching makes the gross shape of an object longer along the stretched direction: the three deformations are illustrated in Fig. 1 using the cube as an example. Compression and stretching are similar in that the two external forces are applied along the same axis, but the direction of the force causes either flattening or elongation of the cube along the axis, respectively (see Fig. 1 (a) and (c)). Shearing, however, causes an angular change such that the resulting 3D object is no longer a rectangular prism (see Fig. 1 
(b)).
Mathematically, the 3D linear deformations can be formulated as S new = T (S old ), where S old and S new are polygonal surfaces before and after deformation, and T denotes a linear function. T can be set up as T 4×4 , a four-by-four matrix for homogeneous coordinates, x, y, z, 1 of a 3D point that is an affine transformation matrix without translation. An analysis of the matrix T 4×4 , parameterized as in equation (1), concluded that parameters a, e and i play a role in compressing and stretching, while the other parameters (b, c, d , f , g and h) cause shearing of 3D shapes. Additionally, all entries in the matrix can be minimized since they are symmetric with respect to an axis (x, y or z). Therefore, T 4×4 can be further simplified to form T 1 for compressing and stretching, and T 2 for shearing as shown below where k 1 and k 2 denote any constant (real number).
(b)).
(1)
(b)).
Of the matrices derived above, transformations based on T 1 does not guarantee volume-invariance; i.e., the 3D object's volume may change after the deformation. For the present study, we are interested in 3D shape transformations that preserve the object's 3D volume. This is similar to kneeling a play-dough to change its shape but not the volume. This is accomplished by deriving two more matrices, T compress and T stretch , from the matrix T 1 as shown below. The matrix T 2 , which preserves 3D volumes, is rewritten below as T shear with γ as the parameter, indicating that the original values on the x axis are changed with respect to the y plane. The three matrices were used to generate the 3D objects used in the present study by compressing, shearing, or stretching cubes and spheres.
Development of Simulator
For virtual simulations of gross shape changes, a simulator of the three deformations employing the matrices in equation (2) was developed with Visual C++, Chai3D (www.chai3d.org) and OpenGL libraries on a PC. Gouraud shading technique (Gouraud, 1971 ) was used for visual rendering. For haptic rendering, Ruspini's finger proxy rendering method (Ruspini et al., 1997) built into Chai3D libraries was used. Figure 2 shows the effects of the three deformations on the gross shape of a Bunny model by transforming the bounding rectangular prism. Any 3D shapes can be deformed by the three linear transformations introduced above by applying the transformation matrices to the bounding object. In the present study, cubes and spheres were used as the base reference objects for 3D shape discrimination as they can serve as the simplest bounding shapes for any 3D objects.
Participants
A total of twenty participants, divided into two groups, took part in the present study (Experiment I: 5 males and 5 females, age range 22-37 years old, average age 28.2 years old; Experiment II: 5 males and 5 females, age range 20-36 years old, average age 29.7). All but one participant were right-handed by self-report. Four participants in Experiment I and five participants in Experiment II had previous experience with haptic interfaces and perception experiments. None of the participants reported any deficiencies in vision or touch.
Apparatus
For haptic rendering of virtual 3D objects, a custom-designed 3-DOF (degrees of freedom) force-feedback device, the 'ministick' (see Fig. 3 ), was used for both Experiments I and II. The participant interacts with virtual 3D objects by moving a stylus that is magnetically connected to the end effector of the ministick. Whenever the stylus collided with the virtual object, a restoring force resisting the penetration was sent to the ministick. The participant perceived the gross shape of the virtual object in a manner that is similar to poking around a real object with the tip of a pen. The ministick was initially designed based on a parallel multi-loop mechanism invented by Adelstein (1998) and implemented by Traylor (2005) . Detailed documentation of the ministick can be found in Traylor's Master's thesis (Traylor, 2005) . The ministick has a usable, interference-free, and bowl-shaped hemispheri- cal workspace measuring approximately 9 × 9 × 6 cm. It is capable of an update of rate of 3.8 kHz, has a position resolution of ≈1.5 µm and a velocity resolution of ∼3 mm/s at the center of its workspace. The ministick produces a stable 8 N/mm stiffness at its typical update rate of 2 kHz. A standard TFT LCD 19 PC monitor (1280 by 1024 pixels) was used for visual rendering of virtual 3D objects. A keyboard was used by the participants to enter the responses.
Stimuli
Two reference 3D shapes, a cube and a sphere, were created with 3DS Max software (see Fig. 4 ) for both Experiments I and II. The undeformed shapes were presented as the reference stimuli to the participants through the visual and/or haptic interfaces. The test stimuli consisted of compressed, sheared, or stretched cubes or spheres with the amount of deformation controlled by γ (see equation (2)). Specifically,
Stimuli
where γ test denotes the value of γ in T compress , T stretch or T shear after deformation, γ ref the value of γ before deformation, and δ the change in γ for deformation. During the experiment, δ was increased or decreased from trial to trial based on the participants' responses. The values for the reference stimulus (γ ref ) were 1.1 for the T compress and T stretch matrices and 0.1 for the T shear matrix.
Procedures
A three-interval forced choice (3IFC) one-up three-down adaptive procedure (Levitt, 1971 ) was used for both Experiments I and II to estimate the discrimination thresholds of gross shape changes. Participants were tested with three conditions: vision alone (V), haptics alone (H) or both (VH). On each trial, for the V (or H) condition, the participants looked at (or touched) the sphere or cube presented through the monitor (or the ministick). For the VH condition, the participants looked at and touched the sphere or cube by using both the monitor and the ministick as shown in Fig. 5 . Two of the three stimuli contained the non-deformed reference cube or sphere. The remaining randomly-selected stimulus was a deformed object with the amount of deformation specified by δ. The participant's task was to indicate which of the three objects looked and/or felt different from the other two. The initial orientation of each object was randomly chosen on each trial for both Experiments I and II. There was only one difference between Experiments I and II concerning the manipulation of the object orientations. In Experiment I, the participant was able to rotate the stimulus at any time with a computer mouse and the object's orientation could be reset to its default position as seen in Fig. 1 by pressing the 'R' key on the computer keyboard. By allowing the participants to manipulate and reset object orientations for both visually and haptically rendered objects, we removed the difficulty in object shape perception due to a particular viewing and/or feeling angle. However, this could potentially allow the participants to align the axis of shape change with the computer monitor so that the 3D object shape discrimination task was accomplished by 2D length discrimination. In Experiment II, no option was given for the participant to manipulate the object orientation. The participant had to perform 3D shape discrimination based on the object orientation that was randomized at the beginning of each interval. According to the one-up three-down adaptive rule, the value of δ was increased after a single incorrect response and decreased after three successive correct responses; otherwise, the value of δ remained the same. The initial δ value was chosen to be large enough so that the gross shape change was clearly perceptible to the participant. The value of δ then decreased or increased by 6 dB, depending on the participant's responses. After the initial three reversals (a reversal occurred when the value of δ decreased after an increase, or vice versa), the value of δ changed by 2 dB. The initial larger change in δ was necessary for a fast convergence of the values, whereas the later smaller change improved the resolution of threshold estimates. Each adaptive series was terminated after 8 reversals at the smaller step size. The participants were comfortably seated in front of a computer monitor, the ministick haptic device, and a keyboard as shown in Fig. 5 . They wore headphones to block any sound from the equipment. Initial training was provided where a series of stimuli were presented to familiarize the participants with the three types of deformations (compress, shear, stretch) and the three experimental conditions (V, H, VH). The training time varied from participant to participant and averaged 20 min for both Experiments I and II. Each participant was tested once per defor-mation type, reference object shape and experimental condition, resulting in a total of eighteen adaptive series per participant. Each participant was tested over two sessions. It took between 3 to 4.5 h for each participant to complete either Experiment I or II.
Data Analysis
For each adaptive series, thresholds were calculated from the values over the last eight reversals at the 2 dB step size. Specifically, four threshold values were estimated by averaging the four pairs of peak/valley values recorded during the last 8 reversals. The mean and the standard deviation of the four discrimination thresholds were then calculated. According to Levitt (1971) , the resulting thresholds corresponded to the 79.4 perceptible point on the psychometric function. Figure 6 shows the results of Experiment I. The bars represent the average thresholds for all participants, and the error bars show standard deviations. The general trend of thresholds over the three types of deformations and the three experimental conditions were similar for the cube (top panel) and sphere (bottom panel) shapes. For both cubes and spheres, the thresholds for the H condition were significantly larger than those for the V condition over all three types of deformations. It was also apparent that the thresholds for the V and VH conditions were almost identical regardless of the shape or deformation type, indicating that the discrimination thresholds for the VH condition were very likely determined by those for the V condition. For the H condition, the thresholds for stretching and shearing were similar, which in turn were significantly larger than those for compressing. This was true for both cubes and spheres.
Results

Experiment I
A three-way ANOVA with the factors Condition (V, VH, H), Deformation (compress, shear, stretch) and Shape (cube, sphere) for Experiment I showed that each factor was significant (Condition: F (2, 166) = 104.88, p < 0.0001; Deformation: F (2, 166) = 18.89, p < 0.0001; Shape: F (1, 166) = 8.03, p = 0.0052). The only significant interaction was between Condition and Deformation (F (4, 166) = 11.28, p < 0.0001), indicating similar trends of thresholds for the two shapes. A posthoc Tukey test revealed two groups for Deformation (compress vs. stretch/shear) for the H condition (μ comp = 0.07947, μ shear = 0.27947, μ str = 0.23315; p < 0.0001) but a single group for Deformation for the V and VH conditions (p ∼ 0.9836 for all pair comparisons). A Tukey test also confirmed that there was a significant difference between the V and H conditions (μ V = 0.03051, μ H = 0.19737; p < 0.0001), but there was no significant difference between the V and VH conditions (μ V = 0.03051, μ VH = 0.02826; p = 0.9845). Figure 7 shows the results of the psychophysical Experiment II. The bars represent the average thresholds for all participants, and the error bars show standard deviations. It is apparent that the general trends of thresholds are similar in Figs 6 and 7. The thresholds of Experiment II were generally higher than those of Experiment I over all three types of deformations and the three experimental conditions, especially in the H condition.
Experiment II
A three-way ANOVA with the factors Condition (V, VH, H), Deformation (compress, shear, stretch) and Shape (cube, sphere) for Experiment II showed that each factor was significant (Condition: F (2, 166) = 259.25, p < 0.0001; Deformation: F (2, 166) = 36.90, p < 0.0001; Shape: F (1, 166) = 39.08, p < 0.0001). The only significant interaction was between Condition and Deforma- tion (F (4, 166) = 21.18, p < 0.0001), indicating similar trends of thresholds for the two shapes. A posthoc Tukey test revealed two groups for Deformation (compress vs. stretch/shear) for the H condition (μ comp = 0.14169, μ shear = 0.35207, μ str = 0.32203126; p < 0.0001) but a single group for Deformation for the V and VH conditions (p ≈ 0.0777 for all pair comparisons). A Tukey test also confirmed that there was a significant difference between the V and H conditions (μ V = 0.05296, μ H = 0.27193; p < 0.0001), but there was no significant difference between the V and VH conditions (μ V = 0.053, μ VH = 0.0607; p = 0.7583).
Experiment II
Finally, a posthoc Tukey test confirmed a statistically significant difference in the H thresholds between Experiments I and II (μ Expt.I = 0.1973663, μ Expt.II = 0.2719286; p < 0.0001) but found no significant differences for the V or VH condition (V: μ Expt.I = 0.031, μ Expt.II = 0.05296, p = 0.4348; VH: μ Expt.I = 0.0283, μ Expt.II = 0.060701, p = 0.0830).
Discussion
The present study investigated discrimination thresholds of 3D gross shapes with three deformations (compress, shear and stretch) over three conditions (vision only, haptics only and visuohaptic) using cubes and spheres. The results of Experiments I and II clearly demonstrate vision dominance in 3D gross shape discrimination in that the visuohaptic thresholds were determined by the visual thresholds. This result was to be expected since vision is better at capturing global information like 3D gross shapes than haptic perception of 3D shapes via a single contact point. Interacting with a stylus that touches only one point of a 3D object at a time is not conductive to the perception of the global shape (see Lederman and Klatzky, 2004) . The significant difference between the visual and haptic thresholds may be reduced by using a multiple point-of-contact haptic device (see Frisoli et al., 2005; Lederman and Klatzky, 2004) .
Discussion
It is apparent from Figs 6 and 7 that for the H condition, discriminating 3D gross shapes with compressing was easier than with stretching or shearing. One reason for this result may be due to the way most participants explored the 3D shapes. At the beginning of a trial, the participant would lower the stylus until it touched the top of the 3D virtual object. The participant would then start a lateral stroking motion (e.g., along the γ edge as seen in Fig. 1(a) ). Since compression affected the length of the top panel most significantly along the direction of the deformation, the participant could more easily discern a change in size on top of the object. Further investigation is needed in order to fully understand why the haptic sensory modality was more sensitive to compressing than to shearing or stretching.
Discussion
Comparing the thresholds from Experiments I and II, there was a general trend of increased thresholds in Experiment II although only the increase of the H thresholds was statistically significant. This is likely due to the fact that it was difficult for participants to identify an object's orientation and shape with a point-contact forcefeedback device. Regardless of the significant increase in H thresholds, however, the effects of experimental conditions on gross-shape discrimination thresholds for the three deformations examined in the present study are similar for Experiments I and II, thereby eliminating the possibility that the results of Experiment I were due simply to 2D object length discrimination at specific orientations.
Discussion
For our intended application of 3D visuohaptic watermarking, the results of the present study show that vision is more sensitive to gross shape (low frequency contents) deformation than touch with a single point of contact, and therefore the strengths of watermarks hidden in gross shape deformations should be selected with the visual thresholds as the upper limits. In addition, since users of watermarking applications can freely manipulate the object orientation on their watermarking system, the thresholds of Experiment I can be used as the maximum watermark strengths that can be embedded in order to ensure imperceptibility. The results of the present study complement those from earlier studies on haptic watermarking presented in Formaglio et al. (2006) and Prattichizzo et al. (2005 Prattichizzo et al. ( , 2007 showing that touch is more sensitive to distortions of surface details (high frequency con-tents) of 3D objects with additive random noise. A comprehensive 3D visuohaptic watermarking scheme can achieve watermark imperceptibility by taking advantage of the different sensitivity of vision and touch to distortions in gross shapes and surface details.

{'paper_id': '14108322', 'title': 'Visuohaptic discrimination of 3D gross shape.', 's2_url': 'https://api.semanticscholar.org/CorpusID:14108322', 'source': 'S2ORC'}