PMC7406178_01
Interfaces for controlling the robot and experimental setup for the
drinking and stacking tasks.. (a) Interface for main participant using the head-joystick. A head
mounted IMU was used to control the robotic arm, and switches (SW1, SW2,
SW3) placed behind the shoulder were used to toggle between different
control modes and for control of the grasper. (b) Interface for control
participant using the manual joystick. (c) Initial layout of drinking
task. Participants had to use the robotic arm to grasp the cup and bring
it to the mouth. (d) Initial layout of stacking task. Participants had
to use the robotic arm to stack the five blocks on top of each other in
order of decreasing size with the biggest block at the base.
pone.0226052.g001.jpg
PMC7406178_02_pone.0226052.g001.jpg
pone.0226052.g001
Our control participant was an able-bodied college-aged male volunteer (21 years old)- see Fig 1b. The individuals pictured in Fig 1 (and the S2 and S1 Videos) have provided written informed consent (or parental consent when appropriate, as outlined in PLOS consent form) to publish their images and videos alongside the manuscript. The robotic arm, shown in Fig 1, is anthropomorphic with 2 DOFs at the shoulder, 1 DOF at the elbow, 3 DOFs at the wrist, and a 1 DOF gripper; specifications of the robot can be found at www.kinovarobotics.com. Head-Joystick: For the child with congenital limb absence, the robotic arm was controlled via signals generated by a wireless inertial measurement unit (IMU) (YEI Technologies Inc., Ohio) worn on the top of a baseball cap with the bill removed:see Figs 1a and 2a. A second IMU, shown in Fig 1a, was placed on the table to determine the relative orientation between the participant and the robot reference frame. SW1 was a small button-type off-the-shelf switch (requiring a 1 N activation force) placed below the primary participant's left shoulder:see Fig 1a). Two additional switches (SW2 and SW3:see Fig 1a) were attached to the chair's backrest, behind the shoulders of the participant, and controlled the opening and closing of the end-effector. The participant was able to determine the state of the grasper from LEDs placed on the table:see Fig 1a. The able-bodied adult participant controlled the robotic arm using the manual joystick (shown in Figs 1b and 2b). The robotic arm was always initialized in the same starting position (X: 0.40 m, Y: 0.30 m, Z: 0.30 m from the base of the robot with the gripper open enough to grasp the cup and facing towards the right) while the cup was in front of and to the right of the participant on the table (mean X: 0.58 m, mean Y: 0.25 m, Z: 0.00 m):see Fig 1c. Fig 1d. From the same initial end-effector position as the drinking task, each block had to be grasped, reoriented, and placed on the previous larger block to build a tower. The blocks were placed directly across the table from the participant in five orientations not matching the target orientation:see Fig 1d.
PMC7406178_02
Interfaces for controlling the robot and experimental setup for the
drinking and stacking tasks.. (a) Interface for main participant using the head-joystick. A head
mounted IMU was used to control the robotic arm, and switches (SW1, SW2,
SW3) placed behind the shoulder were used to toggle between different
control modes and for control of the grasper. (b) Interface for control
participant using the manual joystick. (c) Initial layout of drinking
task. Participants had to use the robotic arm to grasp the cup and bring
it to the mouth. (d) Initial layout of stacking task. Participants had
to use the robotic arm to stack the five blocks on top of each other in
order of decreasing size with the biggest block at the base.
pone.0226052.g001.jpg
PMC7406178_02_pone.0226052.g001.jpg
pone.0226052.g001
Our control participant was an able-bodied college-aged male volunteer (21 years old)- see Fig 1b. The individuals pictured in Fig 1 (and the S2 and S1 Videos) have provided written informed consent (or parental consent when appropriate, as outlined in PLOS consent form) to publish their images and videos alongside the manuscript. The robotic arm, shown in Fig 1, is anthropomorphic with 2 DOFs at the shoulder, 1 DOF at the elbow, 3 DOFs at the wrist, and a 1 DOF gripper; specifications of the robot can be found at www.kinovarobotics.com. Head-Joystick: For the child with congenital limb absence, the robotic arm was controlled via signals generated by a wireless inertial measurement unit (IMU) (YEI Technologies Inc., Ohio) worn on the top of a baseball cap with the bill removed:see Figs 1a and 2a. A second IMU, shown in Fig 1a, was placed on the table to determine the relative orientation between the participant and the robot reference frame. SW1 was a small button-type off-the-shelf switch (requiring a 1 N activation force) placed below the primary participant's left shoulder:see Fig 1a). Two additional switches (SW2 and SW3:see Fig 1a) were attached to the chair's backrest, behind the shoulders of the participant, and controlled the opening and closing of the end-effector. The participant was able to determine the state of the grasper from LEDs placed on the table:see Fig 1a. The able-bodied adult participant controlled the robotic arm using the manual joystick (shown in Figs 1b and 2b). The robotic arm was always initialized in the same starting position (X: 0.40 m, Y: 0.30 m, Z: 0.30 m from the base of the robot with the gripper open enough to grasp the cup and facing towards the right) while the cup was in front of and to the right of the participant on the table (mean X: 0.58 m, mean Y: 0.25 m, Z: 0.00 m):see Fig 1c. Fig 1d. From the same initial end-effector position as the drinking task, each block had to be grasped, reoriented, and placed on the previous larger block to build a tower. The blocks were placed directly across the table from the participant in five orientations not matching the target orientation:see Fig 1d.
Configurations of head-joystick and manual joystick.. (a) Configuration of the head-joystick. It consisted of a three DOF
wireless inertial measurement unit (IMU) (YEI Technologies Inc.) on top
of a baseball cap with the bill removed. (b) Configuration of the manual
joystick. The joystick could move forward or backward, left or right, or
be twisted clockwise or counterclockwise. Buttons on the joystick
enabled the user to switch control modes as indicated by the
light-emitting diodes (LEDs) at the top.
pone.0226052.g002.jpg
PMC7406178_02_pone.0226052.g002.jpg
pone.0226052.g002
The individuals pictured in Fig 1 (and the S2 and S1 Videos) have provided written informed consent (or parental consent when appropriate, as outlined in PLOS consent form) to publish their images and videos alongside the manuscript. The robotic arm, shown in Fig 1, is anthropomorphic with 2 DOFs at the shoulder, 1 DOF at the elbow, 3 DOFs at the wrist, and a 1 DOF gripper; specifications of the robot can be found at www.kinovarobotics.com. Head-Joystick: For the child with congenital limb absence, the robotic arm was controlled via signals generated by a wireless inertial measurement unit (IMU) (YEI Technologies Inc., Ohio) worn on the top of a baseball cap with the bill removed:see Figs 1a and 2a. The able-bodied adult participant controlled the robotic arm using the manual joystick (shown in Figs 1b and 2b).
Description of switches used to toggle between modes.. (a) The three modes of end-effector control as toggled by switch one
(SW1). (b) Switches SW2 and SW3 were used to close and open the grasper.
The truth table shows how pressing and releasing each button on and off
causes the grasper to react.
pone.0226052.g003.jpg
PMC7406178_02_pone.0226052.g003.jpg
pone.0226052.g003
The robotic arm, shown in Fig 1, is anthropomorphic with 2 DOFs at the shoulder, 1 DOF at the elbow, 3 DOFs at the wrist, and a 1 DOF gripper; specifications of the robot can be found at www.kinovarobotics.com. The three modes allowed the participant to use the head-joystick to control all six DOFs of the robot, and the no-movement mode allowed the participant to freely move his body when not intending to control the robot:see Fig 3a. Pressing neither or both switches resulted in the current grasp being maintained:see Fig 3b. In the end-effector orientation control mode (achieved by toggling SW1:see Fig 3a), the IMU signals were translated to rotational velocities of the end-effector about its body-fixed frame.
Mapping between head motion and robotic arm motion for the head
joystick.. Three sets of possible motions of the head for controlling the three DOFs
of the head-joystick: (a) tilting the head up and down, (b) turning the
head right and left, (c) tilting the head right and left. (d) Coordinate
frame fixed to the robot base for controlling the position of the
end-effector showing the corresponding change in position of the
end-effector for each set of head motions. (e) Coordinate frame fixed to
the robot end-effector for controlling the orientation of the
end-effector showing the corresponding orientation of the end-effector
for each set of head motions.
pone.0226052.g004.jpg
PMC7406178_02_pone.0226052.g004.jpg
pone.0226052.g004
The head-joystick has three independent DOFs associated with the head tilting up and down (neck flexion/extension, see Fig 4a), the head turning right and left (rotation, see Fig 4b), and the head tilting right and left (lateral flexion/extension, see Fig 4c). Velocity commands for the end-effector position were computed with respect to the base frame of the robot (XYZ frame in Fig 4d), while velocity commands for the end-effector orientation were computed with respect to the body-fixed frame of the end-effector (e1e2e3 frame in Fig 4e). In the end-effector position control mode (achieved by toggling SW1), tilting the head upwards relative to a neutral home head orientation, as shown in Fig 4a, resulted in an upward (+Z) motion of the end-effector:see Fig 4d. Through the same process, tilting the head downwards (see Fig 4a) resulted in downward motion (-Z). Fig 4b) resulted in the end-effector motion towards the right (+X) and left (-X), respectively:see Fig 4d. Finally, following the right-hand-rule, tilting the head to the right and left (see Fig 4c) resulted in end-effector movement forward (+Y) and backwards (-Y):see Fig 4d. Tilting the head up and down (see Fig 4a) resulted in the end-effector pitching about its e1 axis in the positive and negative direction (see Fig 4e). Fig 4b) resulted in the end-effector rotating about its e3 axis in the negative and positive direction. Finally, tilting the head to the right and left (see Fig 4c) caused the end-effector to rotate about the e2 axis in the positive and negative direction, respectively.