PMC9387771_01
PMC9387771_02
PMC9387771_03
PMC9387771_04
EMG-based human-machine interface scheme. (A) Muscle activity recording through a surface EMG. (A.1) Illustration of the rectus femoris (RF) (hip flexor and knee extensor), femoral biceps long head (FB) and semitendinosus (ST) muscles (hip extensors and knee flexors) and positions of the surface electrodes on these muscles responsible for controlling the movements of the virtual prosthesis knee. (A.2) Schematic diagram of the real-time processing of electromyographic activity and root mean square (RMS) calculations to estimate the level of muscle contraction. The RMS was normalized to the maximum voluntary isometric contraction (MVIC) of each muscle. Regarding recognition of the movement direction, the activity of the agonist muscle should be twice as high as the average of the baseline signal, and the antagonist muscle could not exceed a threshold relative to the agonist, which was initially set at 80%. The recognized EMG patterns were mapped into visual and vibrotactile feedback. (B) Feedback. (B.1) Visual feedback. Avatar modeled with a transfemoral prosthesis and visualization from the first-person perspective are shown. The range of motion available to the prosthetic knee was set between 0  and 90 . (B.2) Vibrotactile feedback scheme. The positioning of vibrotactile actuators on the participant's back was organized in a 4 x 4 matrix. The paradigm for the applied vibratory stimuli was associated with the movements of the virtual prosthesis: upward vibration during knee extension and downward vibration during knee flexion. The vibratory intensity peak of a given row corresponded to a specific angle of knee movement (row A, 0 ; B, 30 ; C, 60 ; and D, 90 ), with an overlap of 30  between adjacent rows.
fnhum-16-870103-g001.jpg
PMC9387771_04_fnhum-16-870103-g001.jpg
F1
Technologies  chip were used to amplify the electrophysiological signals, and the chip was connected to the OpenEphys  analog-digital converter board in communication with its software. The electrophysiological signals were sampled at a rate of 10 kHz (Figure 1A.1). A higher level of EMG activity associated with the hip extensor muscle resulted in knee flexion of the virtual prosthesis and, simultaneously, in a downward vibrotactile stimulation pattern on the subject's back (Figure 1A.2). For more details about the definition of the vibrotactile stimulation pattern, see Supplementary Figure 1 in the Supplementary Material. The subjects were able to control the knee extension and flexion movements of the prosthetic limb within a range between 0  and 90  (Figure 1B.1). Figure 1B.2). For more details on the vibrotactile stimulation device see Supplementary Table 3 and Supplementary Figure 2 in the Supplementary Material.
Training protocol with the EMG-based HMI scheme. (A) Training protocol diagram. Feedback within the virtual environment consisted of visual clues indicating the target angles that the participants had to reproduce. The target angles used were 0 , 30 , 60 , and 90 . Each angle was randomly presented four times during each task block (the participant had 20 s to establish each target angle). In addition to visual feedback, the participants received concomitant vibrotactile feedback on their back. The training sessions lasted 30 mins, and within that time, as many task blocks as possible were performed. (B) Difficulty of progression. Two criteria were adopted to increase the difficulty: (i) Tolerance of antagonist muscle contraction. Initially, the antagonist muscle could have up to 80% activation in comparison to the agonist muscle. The tolerance decreased progressively by 10% at each new difficulty level (the lower the tolerance was, the greater the need to isolate the agonist muscle contraction). (ii) Precision movement. To evaluate whether a target angle has been reached, different ranges of prosthesis position, in relation to the target angle, were adopted (15 , 10 , and 5 : the lower the range was, the greater the necessary precision of movement). Given a tolerance of antagonist muscle contraction, the different precision difficulties were progressively combined. If the participant had a success rate >= 75% on a task block with a certain combination of difficulties, the next block instituted a new combination of difficulties.
fnhum-16-870103-g002.jpg
PMC9387771_04_fnhum-16-870103-g002.jpg
F2
Figure 1B.2). For more details on the vibrotactile stimulation device see Supplementary Table 3 and Supplementary Figure 2 in the Supplementary Material. To guide the movements in real time, the participants were presented with a visual clue (semicircular ruler) indicating the position to which they should move the virtual prosthesis (Figure 2A). However, this became necessary as the difficulty gradually increased (Figure 2B).
Embodiment assessment. (A) Affective measurement-Skin conductance response. Two surface electrodes were placed on the intermediate phalanges of the second and third left hand fingers, and the SCR was recorded once a chandelier dropped on the virtual prosthesis, representing a threatening stimulus. (B) Spatial perception measurement-Crossmodal congruency task (CCT). During the CCT, visual stimuli were applied within the VR environment close to the avatar's feet (close to the hallux or heel) soon after the appearance of the visual distractor, and a vibratory stimulus was applied on the participant's back (thoracic or lumbar). The CCT was composed of sixteen different combinations of visual and vibrotactile stimuli, each presented four times at random, for a total of sixty-four trials. The participants were instructed to press a button corresponding to the location on their back where they received the vibratory stimulation as quickly as possible while ignoring the visual distractor. (C) Motor measurement. The participants moved the virtual prosthesis until they reached a specific predetermined position set at four target angles: 0 , 30 , 60 , or 90 . The participants' performances, execution time and success rates during the training were used to assess their ability to control the virtual prosthesis. (D) Self-perception. The participants quantified on a scale from 0 to 10, where 0 indicated "none" and 10 indicated "totally," how much they felt the virtual prosthesis was part of their own body and how much they felt that they could control it.
fnhum-16-870103-g003.jpg
PMC9387771_04_fnhum-16-870103-g003.jpg
F3
In these sessions, the participants learned to associate the residual limb muscular contraction with virtual prosthesis movements (for details, see Supplementary Figure 3 in the in the Supplementary Material). Finally, the magnitude of the SCR was analyzed (; Figure 3A). In this manner, the CCE was calculated as the difference in the reaction time between incongruent (for instance, when a visual distractor was localized on the upper part of the foot and the vibratory stimulation was in the lumbar region) and congruent conditions (for instance, when a visual distractor was localized on the upper part of the foot and the vibratory stimulation was in the thoracic region) (; Figure 3B and Supplementary Figure 4 in the Supplementary Material).
Affective, spatial perception measurements, and self-perception. (A) Skin conductance response (SCR) to a threat to the virtual prosthesis. (A.1) Two-way ANOVA with Tukey-Kramer correction. (A.2) Application of one-way MANOVA followed by canonical discriminant analysis. (B) Crossmodal congruency task (CCT) and crossmodal congruency effect (CCE) (two-way ANOVA with Tukey-Kramer correction). Comparison for stimuli applied on the same side (SS) and opposite side (OS). (C) Self-perception (absolute values quantified by the participants). (C.1) Sense of ownership. (C.2) Sense of agency. (D) Correlations between the SCR and CCE results (Pearson's correlation coefficients). *p < 0.05.
fnhum-16-870103-g004.jpg
PMC9387771_04_fnhum-16-870103-g004.jpg
F4
In this manner, the CCE was calculated as the difference in the reaction time between incongruent (for instance, when a visual distractor was localized on the upper part of the foot and the vibratory stimulation was in the lumbar region) and congruent conditions (for instance, when a visual distractor was localized on the upper part of the foot and the vibratory stimulation was in the thoracic region) (; Figure 3B and Supplementary Figure 4 in the Supplementary Material).