Cloth Manipulation Using Random-Forest-Based Imitation Learning
Biao Jia Zherong Pan Zhe Hu Jia Pan Dinesh Manocha




  
























[1]▹ #1
SE[DOWHILE]DodoWhile[1] #1


#1#
  #1




FigFigure <ref>
figFigure <ref>
parSection <ref>
appenAppendix <ref>
secSection <ref>
subSection <ref>
tableTable <ref>
algAlgorithm <ref>
AlgAlgorithm <ref>
DefDefinition <ref>
ThmTheorem <ref>
stepStep <ref>
lnLine <ref>
eqEquation <ref>
pbProblem <ref>
itItem <ref>
teTerm <ref>
Eq:#1:(<ref>)
EqEquation #1:












































eqntiny
    eqntinyN
        Cloth Manipulation Using Random-Forest-Based Imitation Learning
        
            Biao Jia, Zherong Pan, Zhe Hu, Jia Pan, Dinesh Manocha 
        
        <http://cs.unc.edu/ biao/robustm>
        Biao Jia is with the Department of Computer Science, University of Maryland at College Park. E-mail:  biao@cs.umd.edu.Zherong Pan is with the Department of Computer Science, University of North Carolina at Chapel Hill. E-mail:  zherong@cs.unc.edu.Zhe Hu is with the Department of Mechanical Biomedical Engineering, City University of Hong Kong.
         E-mail:  zhe.hu@my.cityu.edu.hk.Jia Pan is with the Department of Computer Science, the University of Hong Kong. E-mail:  jpan@cs.hku.hk.Dinesh Manocha is with Departments of Computer Science and Electrical & Computer Engineering, University of Maryland at College Park. E-mail:  dm@cs.umd.edu.
            
        ===============================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
        
        emptyempty Cloth Manipulation Using Random-Forest-Based Imitation Learning
        
            Biao Jia, Zherong Pan, Zhe Hu, Jia Pan, Dinesh Manocha 
        
        <http://cs.unc.edu/ biao/robustm>
        Biao Jia is with the Department of Computer Science, University of Maryland at College Park. E-mail:  biao@cs.umd.edu.Zherong Pan is with the Department of Computer Science, University of North Carolina at Chapel Hill. E-mail:  zherong@cs.unc.edu.Zhe Hu is with the Department of Mechanical Biomedical Engineering, City University of Hong Kong.
         E-mail:  zhe.hu@my.cityu.edu.hk.Jia Pan is with the Department of Computer Science, the University of Hong Kong. E-mail:  jpan@cs.hku.hk.Dinesh Manocha is with Departments of Computer Science and Electrical & Computer Engineering, University of Maryland at College Park. E-mail:  dm@cs.umd.edu.
            
        ===============================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
        
        
        We present a novel approach for manipulating high-DOF deformable objects such as cloth. Our approach uses a random-forest-based controller that maps the observed visual features of the cloth to an optimal control action of the manipulator. The topological structure of this random-forest is determined automatically based on the training data, which consists of visual features and control signals. The training data is constructed online using an imitation learning algorithm. We have evaluated our approach on different cloth manipulation benchmarks such as flattening, folding, and twisting. In all these tasks, we have observed convergent behavior for the random-forest. On convergence, the random-forest-based controller exhibits superior robustness to observation noise compared with other techniques such as convolutional neural networks and nearest neighbor searches.
        
        
        § INTRODUCTION
        
        High-DOF deformable object manipulation, such as cloth manipulation, is an important and challenging problem in robotics and related areas. It has many applications, including assisted human dressing <cit.>, cloth folding <cit.>, sewing <cit.>, etc. Compared with rigid bodies or three-dimensional volumetric deformable objects <cit.>, cloth can undergo large deformations and form wrinkles or folds, which greatly increases the complexity of cloth manipulation tasks. The possibility of such large deformations is the major challenge in designing a cloth manipulation controller.
        In a real-life cloth manipulation task, a typical robot only observes a single RGB(D) image of the cloth. 
        As a result, we need robust methods that can perform such complex manipulation  tasks based on a single view observation. 
        This involves inferring the 3D configuration of the cloth from the image-based representation and compute the appropriate control action. 
        For example, if a robot manipulates a piece of cloth by holding two corners of the cloth mesh, then the controller should infer the desired end-effector positions of the robot.
        Several machine learning models have been proposed to parameterize such controllers, some of which have been used for cloth manipulations. Because of the recent development of deep (reinforcement) learning, one prominent method <cit.> is to represent feature extraction and controller parametrization as two neural networks, which are trained either jointly or separately. Other works, such as <cit.>, use one unified neural network architecture, but the structures of these neural networks are determined via trial and error. Recently, <cit.> represented the controller as a set of observations/control-signal pairs constructed manually. However, due to observation noise at runtime, it is not clear whether this constructed set can cover the experienced cases.
        
        Main Result: In this paper, we present a new method for cloth manipulation. Our method represents the controller as a random-forest. The random-forest takes the observation of the cloth configuration, an RGB(-D) image, as input. It then classifies the input by bringing it to a leaf-node of each decision tree. The optimal control signals are stored on the leaf-node and used as controller outputs. The random-forest is trained iteratively using imitation learning by collecting a dataset online. In each iteration, more data are collected and the random-forest is retrained to be more robust to observation noises.
        
        Compared with other parametric models such as neural networks, random-forest is non-parametric and the number of leaf-nodes can be dynamically adjusted. As a result, arbitrarily complex cloth configurations can be represented as more training data are provided. Compared with other non-parametric methods such as nearest neighbor, random-forest exhibits better robustness in terms of avoiding over-fitting. We show that as more iterations of imitation learning are performed, the number of leaf-nodes in a random-forest will converge. 
        
        We compare the performance of different controller models on three cloth manipulation tasks involving large deformations: cloth flattening, cloth folding, and cloth twisting. The results show that our model always outperforms nearest neighbor <cit.> and neural networks in terms of matching optimal control signals and robustness to noise. In addition, the number of leaf-nodes converges as imitation learning progresses.
        
        The rest of the paper is organized as follows. sec:related reviews related works. In sec:prob, we introduce the notation and formulate the problem. In sec:technique, we provide details for training the random-forest-based controller. Finally, we highlight the performance on challenging benchmarks in sec:result and compare the performance with prior methods.
        
        
        
        § RELATED WORK
        
        In this section, we give a brief summary of prior works on large deformation and manipulation,  dimension reduction, and controller optimization.
        
        Large Deformation and Manipulation: Different techniques have been proposed for motion planning for deformable objects. Most of these works (e.g., <cit.>) focus on volumetric objects such as a deforming ball or linear deformable objects such as steerable needles. By comparison, cloth-like thin-shell objects tend to exhibit more complex deformations, forming wrinkles and folds. Current solutions for thin-shelled manipulation problems are limited to specific tasks, including folding <cit.>, ironing <cit.>, sewing <cit.>, and dressing <cit.>. On the other hand, deformable body tracking solves a simpler problem, namely inferring the 3D configuration of a deformable object from sensing inputs. There is literature on deformable body tracking, which infers the 3D configuration from sensor data <cit.>. However, these methods usually require a template mesh as a priori and are mainly limited to handling small deformations.
        
        Dimension Reduction: Previous DOM methods use various feature extraction and dimensionality reduction techniques, including SIFT-features <cit.>, HOW-features <cit.>, and depth-based features <cit.>. Recently, deep neural networks have also been used as general-purpose feature extractors. They have also been used to manipulate low-DOF articulated bodies <cit.> and in DOM applications <cit.>. For simplicity, our random-forest uses HOW-features as inputs. Another feature recently proposed in <cit.> represents cloth using a small set of feature points. However, these feature points can only characterize small-scale deformations because there can be a lot of occlusions under large deformations. 
        
        Controller Optimization: In robotics, reinforcement learning <cit.>, imitation learning <cit.>, and direct trajectory optimization <cit.> have been used to compute optimal control actions. Trajectory optimization, or a model-based controller, has been used in <cit.> for DOM applications. Although the resulting algorithms tend to be accurate, these methods cannot be used for realtime applications. For low-DOF robots such as articulated bodies <cit.>, researchers have developed realtime trajectory optimization approaches, but it is difficult to extend them to deformable models due to the high simulation complexity of such models. Currently, realtime performance can only be achieved through learning-based controllers <cit.>, which use supervised learning to train realtime controllers. However, as pointed out in <cit.>, these methods are not robust in handling unseen data. Therefore, we further improve the robustness by using imitation learning. Apart from imitation learning used in this work, realtime cloth manipulation controllers can also be optimized using reinforcement learning methods as done in <cit.>. Recently, <cit.> proposed using non-rigid registration to transfer human demonstrations of cloth manipulations to real robots and <cit.> required an adaptive cloth simulator to predict the future state of a cloth. However, these methods require the knowledge of full 3D cloth geometries, which are not available in our applications.
        
        
        § PROBLEM FORMULATION
        
        In this section, we introduce our notations and formulate the problem. Our goal is to compute a realtime feedback controller to deform a cloth into an unknown target configuration. We denote the 3D configuration space of the cloth as 𝒞. Typically, a configuration c ∈𝒞 can be discretely represented as a 3D mesh of cloth and the dimension of 𝒞 can be in the thousands. However, we assume that only a partial observation 𝒪( c) is known, which is an RGB-D image from a single, fixed point of view in our case. The goal of the controller is to transform c into a target configuration c^*. We assume that, over the entire process of control, the robot grasps the cloth at a fixed set of N points whose coordinates are x, where | x|=3N and the control action is constituted by the desired positions of these grasping points, denoted as x^*. Therefore, the controller corresponds to a function:
        
            x^* = π(𝒪(c)|β),
        
        where β are its learnable parameters. Given x^*, the corresponding joint angles of the robot can then be determined via conventional inverse kinematics. Given the control action, the configuration of the cloth and the grasping points can be given by the following distribution:
        
            p( c_i+1,x_i+1| c_i,π(𝒪( c_i))).
        
        This distribution can be a cloth simulator <cit.> in a simulated environment or it can be obtained from a real-life robot. Note that, although the action is the desired grasping points (x^*), x^* and x_i+1 are generally not the same because the controller's output can violate a robot's kinematic or dynamic constraints.
        
        
        
         §.§ Controller Optimization Problem
        
        Our main goal is to optimize the learnable parameters β to optimize the performance of the controller, π. This controller optimization problem can take different forms depending on the available information about c^*. If 𝒪(c^*) is known, then we can define a reward function: R( c)=-dist(𝒪( c),𝒪( c^*)), where dist can be any distance measure between RGB-D images. In this setting, we want to solve the following reinforcement learning problem:
        
            α,β      E_τ∼π[∑_i^∞γ^iR( c_i)]
        
        where τ=( c_1, c_2,⋯, c_∞) is a trajectory sampled according to π, γ is the discount factor, and the subscript figures denote the timesteps. Another widely used setting assumes that 𝒪(c^*) is unknown, but that an expert is available to provide an optimal control action π^*(𝒪( c)). The expert is a ground truth controller following the definition of <cit.>. In this case, we want to solve the following imitation learning problem:
        
            α,β      E_τ∼π[∑_i^∞γ^idist(π^*(𝒪( c_i)),π(𝒪( c_i)))]
        
        This expert can be easily acquired in a typical human-robot collaboration task. Our method is based on the imitation learning formulation.
        
        
        
        § LEARNING RANDOM-FOREST-BASED CONTROLLER
        
        To find the controller parameters, we use an imitation learning algorithm <cit.>, which can be decomposed into two sub-steps: online dataset sampling and controller optimization. The first step samples a dataset 𝒟={⟨𝒪(c), x^* ⟩}, where each sample is a combination of cloth observation and optimal action. The second step optimizes the random-forest-based controller with respect to β, given 𝒟.
        
        
        
         §.§ Feature Extraction
        
        Before constructing the random-forest from 𝒟, we apply a feature transform to 𝒟. Our raw observation of the cloth, 𝒪(c), is an RGB-D image. it has been noted, (e.g., by <cit.>) that applying a simple feature transform can improve the accuracy of a classifier such as random-forest. In addition, our input is a 320×240 RGB-D image of the cloth mesh, which corresponds to 76800 entries each having three colors and one depth channel, which is high-dimensional. Therefore, a feature transform effectively reduces the dimensions of the input observation and makes the classifier more robust when the size of the dataset is small.
        
        In our approach, we use HOW-features <cit.> as the low-dimensional representation. HOW-features is a variant of HOG-features. HOW-features first applies Gabor filters to each patch of the image and then concatenates these patches, resulting in a 768-dimensional feature space. Since each image patch is spatially localized, HOW-features requires each image to be aligned as a pre-processing step. Because our input is an RGB-D image, we can perform a foreground extraction using the depth-channel and then align the image to the center of the screen using the same procedure as in <cit.>. We summarize this algorithm in Alg:Feature and denote this feature transform as a function ℱ. The dataset after the feature transform is defined as 𝒟̅={⟨ℱ∘𝒪(c), x^* ⟩}. 
        
        
        
        
         §.§ Random-Forest Construction
        
        Our key contribution is to use a random-forest as the underlying learnable controller in an imitation learning framework. A random-forest is an ensemble of K decision trees, where the k-th tree classifies ℱ∘𝒪(c) by bringing it to a leaf-node l_k(ℱ∘𝒪(c)), where 1≤ l_k(ℱ∘𝒪(c))≤ L_k and L_k is the number of leaf-nodes in the k-th decision tree. The random-forest makes its decision by classifying ℱ∘𝒪(c) using every decision tree and then computing the average over all the decisions of the trees in the forest. To use an already constructed random-forest as a controller, we define an optimal control action x_l,k^* so that the final action is determined by averaging:
        
            x^* = π(𝒪( c)|β) = 1/K∑_k=1^K x_l_k(ℱ∘𝒪( c)),k^*.
        
        To construct the random-forest, we use a strategy similar to that in <cit.>. We construct K binary decision trees in a top-down manner, each using a random subset of 𝒟. Specifically, for each node of a tree, a set of random partitions is computed and the one with the maximal Shannon information gain <cit.> is adopted. Each tree is grown until a maximum depth is reached or the best Shannon information gain is lower than a threshold. The optimal control action of a leaf-node is defined as the average of the control actions of the data sample belonging to that leaf-node.
        
        
        
        
         §.§ Imitation Learning
        
        We use an imitation learning algorithm <cit.> that includes two steps into an outer loop. During each outer iteration, 
        we query an expert, which in our case is a ground-truth hard-coded control algorithm. 
         Specifically, we generate a set of cloth simulation trajectories using a cloth simulator (eq:p). 
         During each timestep of these trajectories, we query the expert to get an optimal control action π^*(𝒪( c)). This optimal control action is combined with the action proposed by our random-forest π(𝒪( c)). The combined action is fed to the simulator to get the next observation. As a result, more data is added into 𝒟 and a new random-forest, β, is constructed from a new 𝒟. This algorithm is outlined in Alg:mainAlg.
        
        
        
        
        
         §.§ Analysis
        
        In typical DOM applications, data are collected using numerical simulations. Unfortunately, the high dimensionality of c induces a high computational cost for simulations (i.e. evaluating P in Equation <ref>) and generating a large dataset can be quite difficult. Therefore, we design our method so that it can be used with a small number of data samples. Our method's performance relies on the random-forest's stopping criterion (i.e. the threshold of gain in Shannon entropy). We choose to use a large Shannon entropy threshold so that the random-forest construction stops early, leaving us with a relatively small number of leaf-nodes. We expect that, with a large enough number of imitation learning iterations, the number of nodes in each decision tree of the random-forest will converge. Indeed, such convergence can be guaranteed by the following Lemma. 
        
        Lemma: When the number of imitation learning iterations N→∞, the distribution incurred by the random-forest-based controller will converge to a stationary distribution and the expected classification error of the random-forest will converge to zero.Proof:  By assuming that Alg:mainAlg generates a controller π^n at the n-th iteration,  Lemma 4.1 of <cit.> showed that π^n incurs a distribution that converges when n→∞. Obviously, the number of data samples used to train the random-forest also increases to ∞ with n→∞. The expected error of the random-forest's classification on a stationary distribution converges to zero according to Theorem 5 of <cit.>. In sec:result, we show that, empirically, the number of leaf-nodes in the random-forest also converges to a fixed value.
        
        
        
        § RESULTS
        
        We now describe our implementation and the experimental setup on both simulated environments and real robot hardware. We highlight the performance on several manipulation tasks performed by human-robot collaboration. We also highlight the benefits of using a random-forest-based controller by comparing our method with prior approaches. More implementation details are given in <cit.>. 
        
        
        
         §.§ Robot Setup
        
        We evaluate our method on a simulated environment. For the simulated environment, the robot's kinematics are simulated using Gazebo <cit.> and the cloth dynamics are simulated using ArcSim <cit.>, a highly accurate cloth simulator. We use OpenGL to capture RGB-D in this simulated environment. Our goal is to manipulate a 35cm×30cm rectangular piece of cloth with four corners initially located at: v^0=(0,0,0), v^1=(0.3,0,0), v^2=(0,0.35,0), v^3=(0.3,0.35,0)(m). Our manipulator holds the first two corners, v^0,v^1, of the cloth and the environmental uncertainty is modeled by having a human hold the last two corners, v^2,v^3, of the cloth so that we have x≜ (v^0, v^1)^T and each control action is 6-dimensional. The human could move v^2,v^3 to an arbitrary location under the following constraints:
        
            v^2-v^3≤ 0.3m
                  (v^2,v^3)^T_i+1-(v^2,v^3)^T_i_∞<0.1(m/s),
        
        where the first constraint avoids tearing the cloth apart and the second constraint ensures that the speed of the human hand is slow. 
        
        
         §.§ Synthetic Benchmarks
        
        To evaluate the robustness of our method, we design the 3 manipulation tasks listed below:
        
          *  Cloth should remain straight in the direction orthogonal to human hands. This is illustrated in fig:task (a). Given v^2,v^3, the robot's end-effector should move to:
        
            v^0=v^2+0.35z×(v^3-v^2)/z×(v^3-v^2) 
            v^1=v^3+0.35z×(v^3-v^2)/z×(v^3-v^2).
        
          *  Cloth should remain bent in the direction orthogonal to human hands. This is illustrated in fig:task (b). Given v^2,v^3, the robot's end-effector should move to:
        
            v^0=v^2+0.175z×(v^3-v^2)/z×(v^3-v^2) 
            v^1=v^3+0.175z×(v^3-v^2)/z×(v^3-v^2).
        
          *  Cloth should remain twisted along the direction orthogonal to human hands. This is illustrated in fig:task (c). Given v^2,v^3, the robot's end-effector should move to:
        
            v^0=v^2+v^3/2+
            0.31z×(v^3-v^2)/z×(v^3-v^2)+
            0.15(v^3-v^2)×(z×(v^3-v^2))/(v^3-v^2)×(z×(v^3-v^2))
            
            v^1=v^2+v^3/2+
            0.31z×(v^3-v^2)/z×(v^3-v^2)-
            0.15(v^3-v^2)×(z×(v^3-v^2))/(v^3-v^2)×(z×(v^3-v^2)).
        
        The above formula for determining v^0,v^1 is used to simulate an expert. Note that these equations for the expert requires the knowledge of the four corner positions of the piece of cloth, 
        and such information may not be available in a real robot system that only observes the cloth using a single RGB(D) image.
        Therefore, we train our random-forest in a simulated environment. These three equations assume that the expert knows the location of the human hands, but that robot does not have this information and it must infer this latent information from a single-view RGB-D image of the current cloth configuration. We also test the performance on complex benchmarks that combine flattening, folding, and twisting, or have considerable occlusion from a single camera.
        
        
        
         §.§ Transferring from Simulation to Real Robots
        
        Although we have only evaluated our method on a simulated environment, we can also deploy our controller on real robot hardware. For the real robotic environment, we use a RealSense depth camera to capture 640×480 RGB-D images and a 12-DOF ABB YuMi dual-armed manipulator to perform the actions, as illustrated in fig:task.
        
        To deploy our controller, we first use camera calibration techniques to get both the extrinsic and intrinsic matrix of the RealSense Camera. Second, we compute the camera position, camera orientation, and the clipping range of the simulator from the extracted parameters. Third, we generate a synthetic depth map using these parameters and train the three tasks using the random-forest-based controller parametrization and the imitation learning algorithm. Finally, we randomly perturb the human hand positions when collecting training data to make our random-forest robust to observation noises. A similar technique is used in <cit.>. We also add visual noises to the training samples and test the algorithm by posing objects between camera and object.  After that, we integrate the resulting controller with the ABB YuMi dual-armed robot and the RealSense camera via the ROS platform. As shown in fig:real, with these identified parameters we can successfully perform the same tasks that were performed in the synthetic benchmarks on the real robot platform.
        
        
        
        
        
        
        
         §.§ Multi-task Controller
        Unlike single-task controller, a multi-task random-forest-based controller stores multiple actions in a leaf-node. Each observed image is classified by each decision tree in a manner that is similar to that of a single-task controller.
        The leaf node chooses an action according to the id of the task.
        In this benchmark, we train a 3-task controller for the 3 synthetic tasks in sec:sim. And we transfer the controller to the real robot as benchmark (5) mentioned in fig:real. 
        We combines straightening, bending and twisting to show that our approach can perform complex tasks, as shown in the video. Moreover, we also show tasks which involve occlusion from a single camera viewpoint by adding noise to inputs.
        
        We compare the performances of a single-task controller and a multi-task controller, both of which are based on random-forests. Again, during each evaluation in the simulated environment, the human hands move to 10 random target positions v^2*,v^3*. As shown in fig:multi (red), we profile the residual (eq:err). Our controller performs consistently well with a relative action error of 0.4954%. We then train a joint 3-task controller. This is performed by defining a single random-forest and defining 3 optimal actions on each leaf-node. The performance of the 3-task controller is compared with that of the single-task controller in fig:multi. The multi-task controller performs slightly worse in each task, but the difference is quite small. 
        
        
        
        
        
         §.§ Complexity and Algorithm Properties
        
        As illustrated in Alg:mainAlg, the complexity of our overall approach mainly depends on three parts: dataset sampling, feature extraction, and random-forest construction. When constructing a single decision tree based on the sampled dataset D̅, the complexity has an upper bound of O(|D̅|^2). For the construction of a random-forest with K decision trees, the complexity is O(K|D̅|^2).
        
        To evaluate the performance of each component in our method, we run several variants of Alg:mainAlg. All the meta-parameters used for training are illustrated in table:param. In our first set of experiments, we train a single-task random-forest-based controller for each task and profile the mean action error:
        
            err   =   ∑_⟨𝒪(c), x^* ⟩1/| x^*||𝒟̅| x^*-1/K x_l_k(ℱ∘𝒪(c)),k^*^2,
        
        with respect to the number of imitation learning iterations (ln:outer of Alg:mainAlg). As illustrated in fig:perf (red), the action error reduces quickly within the first few iterations and later converges. We also plot the number of leaf-nodes in our random-forest in fig:perf (green). As more iterations are performed, the number of leaf-nodes in our random-forest also converges. 
        
        
        
        
         §.§ Comparison With Other Solutions
        
        A key feature of our method is that it allows the robot to react to random human movements while the effect of these movements is indirectly reflected via a piece of cloth. This setting is similar to <cit.>. However, <cit.> assumes the 3D geometric mesh of cloth c is known without any sensing error, which is not practical. 
        
        Our method falls into a broader category of visual-servoing methods, but most previous work in this area (such as <cit.>) has focused on navigation tasks and there is relatively little work on deformable body manipulation. <cit.> based their servoing engine on histogram features, which is similar to our use of HOW-features. However, they use direct optimization to minimize the cost function (dist(𝒪( c),𝒪( c^*))), which is not possible in our case because our cost function is non-smooth in general.
        
        Finally, our method is closely related to methods in <cit.>, which also use random-forest and store actions on the forest. However, our method is different from prior methods in two ways. First, our controller is continuous in its parameters, which means it can be trained using an imitation learning algorithm. Moreover, we use both feature extraction and controller parametrization in the imitation learning algorithm <cit.> so that both the feature extractor and the controller benefit from evolving training data.
        
        To show the benefits of random-forest, we compare three different models of controllers: random-forest, linear regression, and neural network <cit.>. During each evaluation in the simulated environment, the human hands move to 10 random target positions v^2*,v^3*. In table:cross, we plot of the residual (eq:err) of the tree methods against the number of imitation learning iterations. On the convergence of Alg:mainAlg, the random-forest-based controller outperforms the two other opponents, exhibiting a lower residual.
        
        To implement the neural-network-based controller, we use Tensorflow, which is a neural network toolkit. The structure of the neural network is fully connected and consists of a hidden layer of 128 neurons.
        To implement the linear-regression-based controller, we use the apply the implementation from scikit-learn <cit.>, which is a standard machine learning toolkit. We use the standard parameters from the linear regression module.
        
        
        
        
        
         §.§ Benefits of Random-Forest
        
        
        There are many standard techniques for computing low-dimensional controlling parameters from high-dimensional perceptual data such as RGB images and depth maps. These include standard regression models and neural-network-based models. We evaluate the performance of our algorithm along with the others.
        The test involves measuring the residual of the manipulator as it moves towards the goal configuration based on the computed control parameters, as given by eq:err.
        
        We obtain best results in our benchmarks using a random-forest-based controller. Using the random-forest-based controller and the imitation framework requires fewer parameters to configure a task. Further, the computed control parameters are limited to the labels of the random-forest, which makes the controller robust to the unseen data. 
        In practice, the random-forest-based imitation learning requires fewer computation resources which can enable the controller to be used in real-time applications. The performance is governed by the total number of iterations of the imitation learning. As the number of iterations of imitation learning grows, the residual eq:err reduces. After reaching a certain iteration, the imitation learning contributes less to the performance enhancement. In other words, when the imitation learning framework converges, the overall performance of the controller is guaranteed. 
        
        
        
        
        § CONCLUSION, LIMITATIONS AND FUTURE WORK
        
        We present a novel controller parametrization for cloth manipulation applications. In our parametrization, the optimal control action is defined on the leaf-nodes of a random-forest. Further, both the random-forest construction and controller optimization are integrated with the imitation learning algorithm and evolve with training data. We evaluate our method using a 3-task cloth manipulation application. The result shows that our method can seamlessly handle feature extraction and controller parametrization problems. In addition, our method is robust to random noises in human motion and observations. Moreover, our controller parametrization can robustly adapt to evolving training data and quickly reduce the mean action error for real-time human robot interaction. 
        During our evaluations, the controller performs consistently well in terms of accomplishing the cloth manipulation tasks, including the ones with very large cloth deformations. 
        In terms of comparing with the traditional regression-based controller, our approach can model complex relationships between high dimensional input data and configurations of the controller.
        Comparing with a neural-network-based controller, our approach can converge fast with limited input data, which makes it easier to adapt to unseen data. 
        
        
        One major limitation is that it is difficult to extend our method to reinforcement learning scenarios because our method is not differentiable when using a random-forest construction. Therefore, reinforcement learning algorithms such as the policy gradient method <cit.> cannot be used. Another potential drawback is that our method is still sensitive to the random-forest's stopping criterion. In addition, we need additional dimension reduction, i.e. the HOW-feature,  and action labeling in the construction of the random-forest. In this work, labeling is done by mean-shift clustering of optimal actions, but in some applications where observations can be semantically labeled, it can be advantageous to label observations instead of actions. For example, in object grasping tasks, we can construct our random-forest to classify object types instead of classifying actions. Finally, our method may not be suitable for high-level manipulation tasks such as cloth folding and laundry cleaning. These problems involve multiple smaller manipulation tasks which require a meta-algorithm that combines these tasks. In addition, these tasks usually require re-grasping between different stages of control, which is outside the domain of this paper.
        
        § ACKNOWLEDGEMENT
        
         This research is supported in part by ARO grant W911NF-19-1-0069, QNRF grant NPRP-5-995- 2-415, Intel, HKSAR General  Research  Fund (GRF) CityU 21203216, and NSFC/RGC Joint Research Scheme (CityU103/16-NSFC61631166002).
        
        
        
        
        IEEEtran
