Question: represent the text to find a very short summary (1-2 sentences max).

Question: Much recent research has been devoted to video prediction and generation, but mostly for short-scale time horizons. The hierarchical video prediction method by is an example of a state of the art method for long term video prediction. However, their method has limited applicability in practical settings as it requires a ground truth pose (e.g., poses of joints of a human) at training time. This paper presents a long term hierarchical video prediction model that does not have such a restriction. We show that the network learns its own higher-level structure (e.g., pose equivalent hidden variables) that works better in cases where the ground truth pose does not fully capture all of the information needed to predict the next frame. This method gives sharper than other video prediction methods which do not require a ground truth pose, and its efficiency is shown on the Humans 3.6M and Robot Pushing datasets. It is hypothesized that learning to predict the future and the effect of their actions is an important quality for intelligent agents that interact with their environment. This is a complicated task, as typical use cases require predicting the outcome of interactions between the agent and objects over multiple timesteps. In this work we are looking at the task of predicting the pixels of future video frames given the first few observed frames. We also consider the action conditional setting, in which we are given the action that the agent is taking and are tasked to predict the pixel level outcome of that action in the future. The method of BID20 is a novel way to generate long term video predictions, but requires ground truth human pose annotations. In this work we explore ways to generate videos using a hierarchical model without requiring a ground truth pose or other high level structure annotations for each frame. The method is hierarchical in the sense that it learns to generate a high level structure, then makes next frame predictions based on that structure. Patch level prediction The video prediction problem was initially studied at the patch level BID19 BID12 BID13 BID18. This work showed promising on synthetic data (e.g. bouncing balls), but did not scale to predicting higher resolution videos. Frame level prediction on realistic videos. More recently, the video prediction problem has been formulated at the entire frame level. Most of the recent work is based on the convolutional encoder/decoder framework. BID3 proposed a network that can perform next level video frame prediction by explicitly predicting movement. For each pixel in the previous frame, the network outputs a distribution over locations that pixel is predicted to move. The movements are averaged to get the final prediction. The network is trained end to end to minimize L2 loss. BID11 proposed adversarial training with multiscale convolutional networks to generate sharper pixel level predictions in comparison to conventional L2 loss. BID20 proposed a network that decomposes motion and content in video prediction and showed improved performance over BID11. BID10 proposed a deep predictive coding network in which each layer learns to predict the lower-level difference between the future frame and current frame. As an alternative approach to convolutional encoder/decoder networks, BID7 proposed an autoregressive generation scheme for improved prediction performance. Despite their promise, these work have not been demonstrated for long term prediction on high resolution natural videos beyond ≈ 20 frames. Long-term prediction. BID14 proposed action conditional convolutional encoderdecoder architecture that has demonstrated impressive long-term prediction performance on video games (e.g., Atari games), but it has not been applied for predicting challenging real-world videos.2.1 HIERARCHICAL VIDEO PREDICTION (VILLEGAS ET AL., 2017) BID20 demonstrated a long-term prediction method using hierarchical prediction where the ground truth human pose is assumed to be given as supervision. Our method is based off of that work, so we describe it in detail in the following section. To generate the image at timestep t, the following procedure is used. First, a convolutional neural network encoder generates an embedding vector from the previous ground truth image: e t−1 = CN N (img t−1). This encoding represents the pose of a person. Next, a multilayer LSTM predictor network predicts what the encoding will be in a future timestep. For some number of context frames, the predictor makes its prediction based off of the encoding from the ground truth image. After the predictor network has enough context, it makes its predictions based off of its previous predictions (Fig. 1 provides a helpful visual). For example, if there are C context frames, the following is used to generate the encoding at step t. DISPLAYFORM0 H t is the hidden state of the LSTM at timestep t. Note that only the encoding of the context frames are used, not the subsequent frames. Similar to e t in the above, p t represents the predicted pose. Once p t is obtained, a visual analogy network (VAN) BID15 is used to generate the corresponding image at time t. The VAN applies the transformation that occurred between two images to a given query image. In this case the first frame of the video should be transformed in the same way as the encoding was transformed from the first to t-th timestep. The VAN does this by mapping images to a space where analogies can be represented by additions and subtractions, and then mapping the back to image space. To obtain the predicted image at timestep t using the VAN one needs to use img t = V AN (e 1, p t, img 1), where the VAN is defined as DISPLAYFORM1 Where g is a hardcoded function to transform the pose into a 2 dimensional representation of the pose. The weights of f enc and f img are shared. The disadvantage of this method is that the training relies on ground truth pose annotations. The encoder is trained to produce the pose given the image, the predictor is trained to predict that pose into the future and the VAN is trained to generate the image given the pose. Our method uses a similar network architecture to BID20 but we present ways of training the network that do not require a ground truth pose. In our method, e t and p t have the same dimensionality and represent the network's own higher level structure (e.g., pose equivalent hidden variables) which the network learns as it is trained. In our case, there is no straightforward way to transform the encoding into a 2 dimensional representation of the pose. Therefore, the part of the VAN that maps the encoding is a fully connected network instead of a convolutional neural network. As a , the weights are not shared between the fully connected network which processes the encoding, and the ConvNet which processes the image. The equation for the VAN becomes: DISPLAYFORM0 Note that f enc is a fully connected network, and f img is a conv net. f dec is a deconv network. There are several ways these networks can be trained. In BID20, they are each trained separately with the ground truth human pose. In this work, we explore alternative ways of training these networks in the absence of any ground truth pose or other high level structure annotations. We use the same procedure as BID20 at inference time. One option is to connect the networks the same way as in inference time and train them end to end (E2E). In this method, the L2 loss of of the predicted image is optimized: DISPLAYFORM0 ). There are no constraints on what kind of encoding the encoder produces, or what kind of predictions the predictor makes. Because of how the networks are connected, the encoder will produce an encoding whose future state is easily predicted by the predictor. Likewise, the predictor will make predictions which the VAN can use to produce images which are similar to the ground truth. The encoder and predictor will not have to represent information that is present in the first ground truth frame, since the VAN will have access to the first frame. The size of e t and p t is a hyper parameter of this approach. Figure 1 represents a diagram of this method. Figure 1: The E2E method. The first few frames are encoded and fed into the predictor as context. The predictor predicts the subsequent encodings, which the VAN uses to produce the pixel level predictions. The average of the losses is minimized. This is also the configuration of every method at inference time, even if the predictor and VAN are trained separately. An alternative way to train the combined network is to explicitly train the encoder so that e t is easy to predict into the future, and so that the VAN can use e t to produce the next frame. We call this method Encoder Predictor with Encoder VAN, or EPEV. The encoder and predictor are trained together so the e t is easy to predict and the predictor predicts that encoding into the future. To accomplish this, the difference between e t and p t, L 2 (e t, p t) is minimized. The encoder is also trained with the VAN so the VAN can use e t to produce the image and so that the encoder generates an informative encoding. This is done by minimizing the loss of the VAN given the encoder output: L 2 (img et, img t) where img et = V AN (e 1, e t, img 1). The network is trained to minimize the sum of these two losses: min(DISPLAYFORM0, where α is a hyper-parameter that controls the degree to which the e t will be easy to predict vs. informative enough so the VAN can produce a good image. See figure 2 for a diagram of the encoder and predictor trained together, and figure 3 for the encoder and VAN trained together. Figure 2: The segment of the EPEV method in which the encoder and predictor are trained together. The encoder is trained to produce an encoding that is easy to predict, and the predictor is trained to predict that encoding into the future. The average of the losses is minimized. Figure 3: The segment of the EPEV method in which the encoder and VAN are trained together. The encoder is trained to produce an encoding that is informative to the VAN, while the VAN is trained to output the image given the encoding. The average of the losses is minimized. This method is similar to an autoencoder. Separate gradient descent procedures (or optimizers, in TensorFlow parlance) could be used to minimize L 2 (img et, img t) and L 2 (e t, p t), but we find that minimizing the sum works better experimentally. With this method, the predictor will predict the encoder outputs in future timesteps, and the VAN will use the encoder output to produce the frame. The end to end approach can also be augmented if the dataset has information about the ground truth pose or any other high level frame annotations. In this method, the e t and p t vectors would be split into two: the first path is optimized to represent the pose, and the rest of e t and p t is trained the same way as the E2E approach. At each training step a separate optimizer minimizes each loss. In this method, we can think of e t and p t as the concatenation of two vectors, one representing the pose, and the other containing additional information the network can represent. If e t = [e poset, e remainingt] and p t = [p poset, p remainingt], the following losses are minimized:The loss representing how well the encoder infers the pose: min(T t=1 L 2 (e poset, pose t)). The loss representing how well the predictor predicts the pose: min(T t=1 L 2 (p poset, pose t)). The end to end loss: min(T t=1 L 2 ( img t, img t)). These losses are minimized with separate optimizers in this method. Minimizing the end to end loss ensures that the VAN will learn to use the pose provided by the predictor network, and that the encoder and predictor will learn to produce additional information besides the pose that is useful to the VAN. In order to compare to a baseline, we also implemented the method where each of the networks are trained individually, as in BID20. The main difference between this method and Villegas et al. FORMULA0 is that we do not use an adversarial loss BID4. See section 5 for a discussion of how an adversarial loss could be added to our method. These methods were tested on two different datasets, the Robot Push dataset BID3 and the Humans 3.6M dataset BID6 BID1. Videos of the of our method are available by visiting the following URL: https://goo.gl/WA8uxc.The EPEV method works best experimentally if α starts small, around 1e-7, and is gradually increased to around.1 during training. As a , the encoder will first be optimized to produce an informative encoding, then gradually optimized to also make that encoding easy to predict. This dataset contains videos of a robot arm pushing objects on a table. The current joint angles and the location of the end effector are given, and we use these as the pose for the methods which require it. The action the robot arm is taking is fed into the predictor. Each of the methods considered was given two frames of context, and then trained to predict 17 subsequent frames. An encoding size of 16 was used for the E2E method. The size of the pose is 12, so the encoding size of the INDIVIDUAL method is 12. The other methods used an encoding size of 32.Additionaly, we randomly split the dataset into training, validation and test. We used 64x64 images, and the same frame rate as the original dataset. Results from our test set are shown in this section. Note that our experimental protocol is different from BID3, where the test set is composed of novel objects. We hypothesized that the methods where the network could learn its own pose equivalent would predict the movement of the objects the robot arm pushes more accurately than the INDIVIDUAL method. To test this, we manually compared the E2E and EPEV methods to the INDIVIDUAL method and evaluated where the movement of predicted objects most closely matched the ground truth. We evaluated 40 videos in which objects move. The are in TAB0. method, the predictor network can only produce the pose, so the VAN has to infer how the objects will move based on the start and end state of the arm. We were surprised by how well the VAN could infer this. However, from examining the videos, the EPEV method had better object predictions than the INDIVIDUAL method, which supports our hypothesis. The magnified part of ground truth frame 19 in FIG0 shows that the robot arm pushed the yellow object. The EPEV and E2E methods correctly predict this, but in the INDIVIDUAL method, the robot arm covers up the yellow object instead of moving it. Additional analysis is in appendix section F. The average Peak Signal to Noise Ratio (PSNR) of the different methods we introduce are similar on this dataset. In this data set, the model from BID3 gets a better PSNR than our model. The movement in this dataset can easily be represented by the movement of pixels, and it is relatively deterministic. So the model from BID3 which explicitly predicts the movement of objects and directly minimizes the L2 loss works well here. To confirm our claim that our method works well for long term predictions, we trained our method on a toy task with known factors of variation. We used a dataset with a generated shape that bounces around the image and changes size deterministically. We trained the EPEV method and the CDNA method in BID3 to predict 16 frames, given the first 3 frames as context. We do not show the E2E method since it usually predicts blurrier images than the EPEV method. Both methods are evaluated on predicting approximately 1k frames. We added noise to the LSTM states of the predictor network during training to help predict reasonable motion further into the future. Results form a held out test set are described in the following. After visually inspecting the of both methods, we found that when the CDNA fails, the shape disappears entirely, however when the EPEV method fails, the shape changes color. To quantitatively evaluate both methods, we used a script to measure whether a shape was present frames 1012 to 1022, and if that shape has the appropriate color. See table 2 for the averaged over 1k runs. The CDNA method predicts a shape with the correct color about 25% of the time, and the EPEV method predicts a shape with the correct color about 97% of the time. The EPEV method sometimes fails by predicting the shape in the same location from frame to frame. This does not happen very often, as the reader can confirm by examining the randomly sampled predictions in appendix section E. It is unrealistic to expect the methods to predict the location of the shape accurately in frame 1000, since small errors propagate in each prediction step. Our method was also tested on the Humans 3.6M Dataset. Only the E2E and EPEV methods were tested here, since BID20 has already shown the using the ground truth pose. We used subjects 1, 5, 6, 7 and 8 for training, subject 9 for validation. Subject 11 are reported in this paper for testing. We used 64 by 64 images. We subsampled the dataset to 6.25 frames per second. We trained the methods to predict 32 frames and the in this paper show predicting 64 frames. Each method is given the first 5 frames as context frames. So the in these images, the model predicts about 10 seconds into the future from.8 seconds of context. We used an encoding size of 32 for the E2E method and a encoding size of 64 for the EPEV method on this dataset. We compare our method to the CDNA method in BID3 in Fig. 6. Figure 6: A visual comparison of the EPEV method and CDNA from BID3 as the baseline. This example is cherry picked to show when there is significant movement in the ground truth. See appendix section G for non cherry picked . The contrast of these images was increased to make the humans easier to see. In CDNA from BID3, the person disappeared part way through the prediction. The EPEV method, produced relatively sharp predictions up until frame 42, and a blurry human prediction at frame 63.From visually inspecting the images we found that in images where there is not significant movement in the first 5 frames of the ground truth it is hard to tell the difference between our method and CDNA since both methods predict an image similar to the early ground truth frames. However, when there is significant movement in the first 5 ground truth frames, the predictions from EPEV are sharper further into the future than CDNA. See the appendix section G for images where there is significant movement in the first 5 ground truth frames so the methods can be compared. We also collected from the E2E method, but those blur out very quickly and are shown in appendix section G.The CDNA method from BID3 produces blurry images since it is trained to minimize L2 loss directly BID3. In the EPEV method, the predictor and VAN are trained separately. This prevents the VAN from learning to produce blurry images when the predictor is not confident. The predictions will be sharp as long as the predictor network predicts a valid encoding. We also compare our method to BID20. This method gives sharper than ours. We think that this is because BID20 uses a adversarial loss BID4 and since nothing besides the human is moving in this dataset, the pose works well as a high level structure. We propose to compare the methods quantitatively by considering whether the generated videos contain a recognizable person. To do this in an automated fashion, for each of the generated frames, we ran a MobileNet BID5 object detection model pretrained on the MS-COCO BID9 dataset. We recorded how confident the detector was that a person (one of the MS-COCO labels) is in the image. We call this the "person score" (its value ranges from 0 to 1, with a higher score corresponding to a higher confidence level). The on each frame averaged over 1k runs are shown in FIG2. The person score on the ground truth frames is about 0.4. This is likely due to the mismatch between the training set images of the model (the MS-COCO dataset images are very different in terms of image statistics compared to the Humans 3.6M data). The person score is 0.26 on average for the images generated by the EPEV method, and 0.18 for CDNA from BID3. The person score degrades very rapidly in the first 8 frames of CDNA, but degrades more slowly in the EPEV method. The person score of the EPEV method on frame 63 is about the same as on frame 8 of CDNA. This confirms our visual analysis that the EPEV method produces clearer predictions further into the future. The EPEV method was only trained to predict 32 frames into the future but there is no significant drop in the person score at frame 32, showing that the EPEV method generalizes well to predicting longer sequences. We also used a service similar to Mechanical Turk to collect comparisons of 1,000 generated videos from our EPEV method and the CDNA baseline. The task showed videos generated by the two methods side by side and asked raters to confirm whether one of the videos is more realistic. The workers rated the EPEV method as more realistic 53.6% of the time, the CDNA method as more realistic 11.1% of the time and the videos as being about the same 35.3% of the time. The high number of "same" responses could be because of it being difficult to tell the difference between the methods when there is little movement. On datasets where the pose does not capture all of the information needed to predict future frames, letting the network define its own high level structure in addition to the pose is an improvement upon a BID20. The EPEV method generates sharper images than BID3 on non deterministic datasets, and can generate further into the future on a toy dataset that we introduced. We posit an adversarial loss between the predictor and encoder would likely help with potentially uncertain scenarios and would fix the problem of the EPEV method sometimes generating blurry images,Under review as a conference paper at ICLR 2018 To see what the encoder has learned in the EPEV method, we can obtain from the visual analogy network given the input from the encoder. The encoder is given the ground truth image. The are shown in figure 9. The show that the encoder encodes where the person is Figure 9: Results from the EPEV approach when the VAN is given the output of the encoder on the ground truth frame.in the image, as well as the orientation of the arms, legs and head to some extent. The are not as good as one would expect from an autoencoder, since the encoder has the constraint that the encoding also has to be easy to predict. We trained all of the methods including BID3 for 3 million steps using async SGD, across 32 worker machines. We used a minibatch size of 8 sequences in each step. The minibatch size could be so small because there were multiple frames per sequence. In methods with multiple optimizers, a step is defined as running each optimizer once. The hyperparameters are optimized separately for both datasets on a validation set. We used the best learning rates for each method.

represent the tldr for finding the corresponding scientific article

We show ways to train a hierarchical video prediction model without needing pose labels.

We leverage deterministic autoencoders as generative models by proposing mixing functions which combine hidden states from pairs of images. These mixes are made to look realistic through an adversarial framework.


Question: Represent the scientific article to find a short TLDR

Question: Combining information from different sensory modalities to execute goal directed actions is a key aspect of human intelligence. Specifically, human agents are very easily able to translate the task communicated in one sensory domain (say vision) into a representation that enables them to complete this task when they can only sense their environment using a separate sensory modality (say touch). In order to build agents with similar capabilities, in this work we consider the problem of a retrieving a target object from a drawer. The agent is provided with an image of a previously unseen object and it explores objects in the drawer using only tactile sensing to retrieve the object that was shown in the image without receiving any visual feedback. Success at this task requires close integration of visual and tactile sensing. We present a method for performing this task in a simulated environment using an anthropomorphic hand. We hope that future research in the direction of combining sensory signals for acting will find the object retrieval from a drawer to be a useful benchmark problem A core aspect of human intelligence is the ability to integrate and translate information between multiple sensory modalities to achieve an end goal. For example, we have no trouble discriminating between a set of keys, a wallet or a watch kept in our pocket by simply feeling them with our hands. Similarly, we can easily retrieve a desired object present inside a dark drawer even if we can't see the objects using touch sensation from our hands. Not only can we retrieve a previously known object, but if we are shown an image of a previously unseen object, we would still have no trouble retrieving this object using only tactile exploration inside the drawer even in absence of any visual feedback. Such translation of information between sensory modalities is not specific to tactile and vision, but is noticed between other modalities as well. For instance, it is easy to imagine someone walking down the stairs and opening the door by simply hearing the sound that was generated. These examples demonstrate how easily humans can translate information between sensory modalities. Different sensory modalities provide a different view of the same underlying reality. The ability to transform between sensory modalities therefore provides an interesting way to learn useful representations of sensory inputs. Recent work in self-supervised learning has made extensive use of this observation and shown that useful visual features can be learned by predicting, from images, corresponding sounds BID18, ego-motion BID1 BID11, depth or even predicting color values from grayscale images BID23.In addition to learning feature representations, another and possibly more critical use of sensing from multiple modalities is performing goal directed actions in partially observable settings. In the running example of retrieving objects from a drawer, the agent receives only the image of the object as input and in absence of any light source in the drawer, the agent solely relies on its tactile sensing to find the object. Other examples are a pedestrian getting alerted when she hears the sound of a car coming from the back or animals in the jungle being alerted of a tiger behind the bushes by the sound of the movement. Yet another example showing close integration of two modalities (vision and touch) is a study that found it became almost impossible for human participants to perform the seemingly trivial task of picking up a matchstick and lighting it when their hands were anesthetized BID12 ). Figure 1: (Left) Shows our experimental setup. p objects are in a drawer and a dexterous hand equipped with tactile sensing can explore novel objects using deterministic routines. In this case, p = 3 but we compared performance by varying the number of objects (Middle) We are presented with a query image as seen by the inset in the top right of the image. We explore the objects in the drawer using tactile sensing only to identify the object (Right) We then retrieve the object by applying a grasping routineIn this work we use the task of retrieving objects from a drawer as an experimental setup to investigate joint learning from two sensory modalities of vision and touch. Because the agent is provided only with a visual image of the object to be retrieved, it must translate into the representation space of tactile sensing to retrieve objects only by touching them. In the general case of retrieving the object, the agent must first explore spatially to locate where the objects are. Once it finds the object, it must move its fingers in an exploratory manner to collect information required to determine if the object is the one that needs to be retrieved. Solving this problem in its full generality requires not only good goal directed exploration strategies and also a method for translating between different sensory signals. We therefore think that object retrieval from a drawer is a good challenge problem for investigating different models that combine visual and tactile information for a target end task. In our setup the agent learns a mapping from visual to tactile signals using unsupervised exploration. This mapping enables the agent to determine the representation of the image in the representation space of tactile sensing (i.e. expected tactile response). The agent explores each object present in the drawer by touching it and compares the of its exploration with the expected tactile response. Performing this comparisons requires a good representation of raw tactile signals. For learning such a representation, we leverage the in image classification, where it was found that a network pre-trained to classify images from the Imagenet dataset into one thousand image categories learns features useful for many other visual tasks. Similar to image classification, we pose the task of classifying objects from tactile signals collected by touching eleven objects. We show that tactile representation learned by performing the task of classification, generalize and can be used to retrieve novel objects. We present in a simulated environment and the agent explores the objects using an anthropomorphic hand. One of the earliest works presenting haptics as a sensory modality to explore the world was by BID8 ).Gibson showed that object recognition dramatically decreased when one could not actively interact with an object. Lederman and colleagues BID15 ]. They describe the various exploratory prcoedures (EP) that humans can perform to understand various object properties such as volume, temperature, friction, etc. Multi-modal learning is a key component for how biological agents learn and build models of objects. It can be argued by looking at failure modes to modern day robotics (rob) that it is exactly this lack in multi-modal learning that requires further study. Earlier work in haptic exploration includes BID3, BID9 ) who employed various hand engineered features to recognized objects using haptics. The challenges faced were largely due to robust sensors and the ability to control these sensors to explore objects effectively. BID5 ) measure various physical properties of objects using the bio-tac sensor using five different exploration procedures (EP). In addition, they also collect adjectives for each object and the corresponding They then compute precision, recall scores using a static hand-engineered feature and dynamic feature model employing Hidden Markov Models and compute precision, recall scores on a held out dataset. Similarly, Schneider et al. FORMULA1 et al. also classify objects using a bag-of-words appraoch. Romano et al. BID19 ) mimic human tactile sensing for grasping by hand engineering features that can appropriately measure slippage. They then design a control strategy that can grasp and place that employs the tactile responses. They show that in cases where objects are crushable, a naive controller crushes 100% of the time as compared to a controller that effectivel leverages tactile sensing. Others, such as Sinapov et al. BID21 ) have considered building object representations using vibrotactile sensation. They show that they can classify surface properties using data collected from five different EPs. Similarly, BID6 classify textures using the bio-tac sensor using a Bayesian exploration strategy. While, BID10 employ a palpatation sequence that is not learnt to effectively explore objects in a multi-fingered robot. Our work relates to work by BID7 who show that combining visual and haptic information can lead to better classification of haptic properties. More recently, Calandra et al. BID2 ) show that employing tactile inputs into a learnt model can help improve predictions of graspability. (OpenAI:) have shown that tactile features may not be required for certain constrained in-hand manipulation tasks. While this may seem contrary, this in fact is not a representative task. Further, the setup employed by the authors substitutes tactile sensing with a very rich 3D data along with a powerful learning method thus navigating around tactile sensing requirements. Task Setup: Figure 1 presents our task setup. A subset of objects from Figure 3 are placed in a drawer. An image of the object from a fixed pose is presented to the agent. The agent explores each object using a set of pre-determined routines combining palpation and grasping-like movements. The agent then identifies the object it needs to grasp and executes a grasping routine. In our setup the movement between the objects and grasping, is done using a pre-determined routine. The object is held translationally fixed in space but can rotate about its axes. The hand is initialized close to the object. The hand is translationally fixed, in that it cannot slide but it can rotate around the wrist joint. The fingers can explore their movements with only the restrictions imposed by the joints themselves. That is the fingers, say, cannot bend backwards towards the wrist. For each episode of 500 time steps long, the haptic forces H t and the corresponding Images I t are collected. Each object is presented in multiple random poses. The dataset consists of 500 samples per object, each sample is 500 timesteps long and has 19 dimensions. We use a simulated model of the anthropomorphic hand used as part of SouthHampton Hand Assesment Procedure (SHAP) test suite built by BID16 (see Figure 3). The SHAP procedure was established for evaluating prosthetic hands and arms. With this idea in mind, prior work built a prosthetic arm which could theoretically perform all useful human hand movements. Based on this hand and the DARPA Haptix challenge, a simulated model of a similar hand (but with fewer sensors) BID14 ) was built using the Mujoco physics engine BID22 ). This model was made publicly available and we use this for all our experiments. The hand has five fingers and 22 joints out of which many are tendon coupled. For example, curling the tip of a finger automatically actuates the other two joints on the finger so that the finger moves towards the palm. Because of these couplings, the ant dynamics can be quite complex and articulated. Out of the 22 joints, thirteen are actuated. Out of these thirteen, ten joints control the motion of fingers and the other three control the rotation of the hand. Additionally, there are three degrees of motion along the (x, y, z) axis and therefore overall 16 degrees of actuation. The hand is controlled by setting the position of these 16 actuators. In addition, the hand is equipped with 19 contact sensors (as seen in 3) that measure normal contact forces that are computed by Mujoco. These sensors form the basis of our tactile sensing. In our setup, we have two sets of networks. Network f 1 accepts as inputs, images at time t, defined by I t. It then learns to predict the haptic responses for the object being explored defined by H t. This network is optimized by minimizing the following objective function DISPLAYFORM0 Given, tactile responses can we discriminate a set of objects effectively? To do this, we train a separate network f 2. This network accepts, as inputs, H t and learns to predict object identities Y t. We then minimze the cross entropy loss as in 2. DISPLAYFORM1 To simulate how an agent would be able to identify an object during test time we present an image I to the model. Network f 1 predicts the haptic responses to this object -Ĥ. The predicted haptic responses,Ĥ are then used to compute the predicted object categoryŶ. We can then apply a learnt grasping routine to grasp the object and retrieve it. To train the haptics predictor network, F 1 the inputs were gray scaled, 64x64 images I t with the focus object centered in the image as seen in Figure 3. The network consisted of three convolutional layers of filters 32, 32, 32. The kernel size was 5,5 for each layer. The output of the convolutional layer was then fed into three sequential fully connected layers of sizes [..., 1024],, and to predict the 19 dimensional haptic forces. The groundtruth predictions were per-channel averaged haptic forces across an entire episode length of time T. We then trained the network using ADAM (Kingma & Ba FORMULA1) with an initial learning rate set to 1e-4.To train the object discriminator network, F 2 the inputs were average haptic forces over an entire episode which have 19 dimensions. These inputs were then passed into a network of fully connected layers of sizes,, and [250, K]. We then minimize the cross entropy loss between the ground truth and predicted object categories using the ADAM with an initial learning rate set to 1e-4.In both cases performing normalization of the input images and haptic forces was critical to training the network. For the images, a simple mean subtraction and division by the standard deviation sufficed. For the haptic cases since the forces were different across different dimensions and doing a simple normalization that ed in small values that were outside the range of tanh function ed in difficulties in good predictions. We introduced a scale term to the normalized output so that distribution of the target data was inline with the output range of the network. Figure 3: Displays the objects used in our experiments. We used a set of 25 objects. These were imported from the ShapeNet dataset BID4 ). Each object was presented in various different poses. The hand was initialized at the same location for each sample while the object was randomized in each trial. We present three sets of experiments in this section. First, we study how hard it is to identify an object using tactile sensing. We do this on novel poses that the model has not seen during training. Next, we explore the question of effective exploration length for these experiments. Finally, we study the problem of identifying novel objects in the dark. Before identifying novel objects in the dark, we wished to understand how challenging the problem of identifying an object through tactile sensing was. The inputs in this case were average haptic forces over the entire sampling routine. The training consisted of 400 training samples per object category. Each sample presents the object a random rotation about the z-axis. In total, 4400 were used in training. We used 50 samples per object to evaluate the model. During test time, another 50 samples from each object class but unseen random poses were provided. The model was asked to correctly identify the objects, this classification accuracy is reported in the table 4.1.For the object identification problem, we compare the classification accuracy of two networks. First, the pretrained f 2 network on ground truth haptics. Second, we provide the query in image space, we then employ f 1 to compute predicted haptics. We then used the predicted haptics to identify the object as seen in 4.1. We find that the network was able to predict the object identity using haptic forces on the known objects samples per category with near 100% accuracy. When employing the predicted means, this accuracy was a bit lower. Inputs 11 Object Accuracy Ground Truth Haptics.99 Predicted Haptics.54 Table 1: Table showing object identification test accuracies for training objects given ground truth and predicted haptic forces. There are 11 objects in the training set, many of which were imported from ShapeNet (Chang et al. FORMULA1). In our current setup the agent employs a predetermined sampling routine to explore the object using tactile sensors. A natural question is how much exploration is required to accurately identify the object just using tactile information. To answer this, we trained separate tactile classification networks (i.e. f 2) using different number of samples obtained by exploration. Results presented in table 4.2 show that about 100 samples are sufficient for accurate classification and performance improves only marginally when 500 samples are collected. The inputs to the network still consisted of averaged haptic forces but they were computed over different episode lengths. Every object from Figure 3 were presented in various poses. During test time, held-out samples were presented. Episode Length Accuracy 1 9% 10 14% 100 93% 500 95% Table 2: Table showing object identification accuracies given different episode lengths on the 11 objects imported from Shape-Net. From Table 4.2 we see that when only one time step is used the classification accuracy is just over chance (0.04%). This number increases significantly even with a few time steps and saturates fairly quickly after that. While the above experiments demonstrate that we are able to classify objects based on tactile sensing alone, they donot show if it possible to retrieve the object from interest when only an image of the object is presented (please see our experimental setup Figure 1).An input image I of the object to be retrieved is presented to the agent. A set of p objects where p ∈ K are presented to the hand. All the objects presented are novel. The model predicts the haptic responseĤ t of the image using F 1. We then use the predicted haptic response as an input to our classification network F 2. Since F 2 was not trained on the objects that we are trying to identify, we then identify the object using a nearest-neighbor classifier in the latent space of the network. We call this space embedded haptics. We compare the performance of this sampling classification by fitting the k-NN model with both raw haptic predictions and embedded haptic predictions for the p objects presented. We train F 1 and F 2 networks on 11 objects. During test time from the 14 novel objects, a set of p objects are are out in the drawer (see Figure 1). The higher p is, the harder the task is. The of our method for p = 2, 3, 5 are presented in Table 4.3.We generate a subset of held-out objects as well as three haptics templates per object. We then classify 1 query image from the set of objects with a kNN classifier with k = 3.To compute the mean precision and standard deviation in this precision we run 20 classification trials. For each trial, we classify 5000 query images following the above procedure. Mean precision and standard deviations are then computed across these trials. # objects Raw Hptx Embedded Hptx 2 58.5 % ±0.7% 65.9 % ±0.8% 3 42.5 % ±0.8% 50.0 % ±0.7% 5 26.6 % ±0.5% 34.2% ±0.6% Table 3: Table showing object identification accuracies given 2, 3 and 5 held-out objects to be discriminated from. We see that the haptic embedding yields a meaningful increase in classification accuracy. We present a model that when presented with a query image can identify a novel object from a set of p objects using tactile sensing only. As the number of novel objects presented in a single trial increased, this task quickly became more challenging. We show that a model with pre-determined exploration routine can identify the object but a richer exploration of the objects could allow us to answer more challenging inferences such as pose, texture, etc. One could imagine doing this by training a RL agent that uses the performance of {2 as a reward while generating trajectories that explore an object for identification. Presently, we train our two networks F 1 and F 2 independently. Since, our larger goal is to identify objects it would be interesting to jointly optimize the network to maximize object identification. This can help smooth prediction errors that are not consequential to classification or identification. While the MuJoCo simulator provides fast physics solvers that can compute realistic contact forces they only compute normal contact forces. Research in neuroscience has shown BID15) that a variety of forces are applied and measured while employing tactile sensing. It would be interesting to perform similar experiments on robot grippers equipped with tactile sensors. Our current setup still trains network F 2 in a supervised fashion. This is biologically implausible and tedious for practical scalability on real robots. It would be interesting if we posed this problem as a self-supervised problem and explored learning to identify novel objects using tactile sensing.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

In this work, we study the problem of learning representations to identify novel objects by exploring objects using tactile sensing. Key point here is that the query is provided in image domain.

Represent the TLDR for finding the corresponding scientific article

StrokeNet is a novel architecture where the agent is trained to draw by strokes on a differentiable simulation of the environment, which could effectively exploit the power of back-propagation.


Question: represent the scientific article to find a short tldr

Question: Locality sensitive hashing schemes such as \simhash provide compact representations of multisets from which similarity can be estimated. However, in certain applications, we need to estimate the similarity of dynamically changing sets. In this case, we need the representation to be a homomorphism so that the hash of unions and differences of sets can be computed directly from the hashes of operands. We propose two representations that have this property for cosine similarity (an extension of \simhash and angle-preserving random projections), and make substantial progress on a third representation for Jaccard similarity (an extension of \minhash). We employ these hashes to compress the sufficient statistics of a conditional random field (CRF) coreference model and study how this compression affects our ability to compute similarities as entities are split and merged during inference. \cut{We study these hashes in a conditional random field (CRF) hierarchical coreference model in order to compute the similarity of entities as they are merged and split during inference.} We also provide novel statistical analysis of \simhash to help justify it as an estimator inside a CRF, showing that the bias and variance reduce quickly with the number of bits. On a problem of author coreference, we find that our \simhash scheme allows scaling the hierarchical coreference algorithm by an order of magnitude without degrading its statistical performance or the model's coreference accuracy, as long as we employ at least 128 or 256 bits. Angle-preserving random projections further improve the coreference quality, potentially allowing even fewer dimensions to be used. Probabilistic models in machine learning, such as conditional random fields (CRFs), are widely successful at modeling many problems at the heart of knowledge base construction, including those in natural language processing, information extraction and data integration. However, when dealing with natural language data, the underlying feature representations are often sparse, high-dimensional and dynamic (i.e., they change during inference). In this paper we consider the task of coreference resolution, in which the goal is to partition a set of mentions into the entities to which they refer. We might represent each mention with a feature vector in which each dimension corresponds to a word or n-gram. Since only a small subset of the vocabulary is observed per mention, most elements of the vector are zero. Given the model and these representations, inference entails making decisions about whether two entities should be coreferent. To make such decisions, the model estimates a probability that involves computing the similarities between the aggregate feature representations of the two entities' mentions. Since all the vectors are both sparse and high-dimensional, these similarity operations are computationally expensive because the sparse data structures supplant the dense arrays that would otherwise support fast look-ups. Moreover, as the inference algorithm makes decisions about whether or not two entities are coreferent, we may have to split or merge the entities and thus we must update the feature vector to reflect these changes. Maintaining such sparse-vector representations in the inner-loop of probabilistic inference is expensive, especially as the entities grow in size. In order to cope with the computational problems associated with sparse, high dimensional dynamic feature representations, we propose using homomorphic compression, in which the compressed representations of intermediate inference can be computed directly from their operands, allowing inference to run directly on the compressed representations of the data even as they change. In this paper, we consider several such schemes to scale hierarchical coreference. First, we propose a novel homomorphic cosine-preserving hashing scheme based on simhash BID7 that also supports addition and subtraction to more efficiently represent the data and the evolving intermediate of probabilistic inference. Second, because various linear norm-preserving random projections also preserve angles BID24, we can directly compute cosine similarity on projected data -linearity of the projections means that they too can be updated dynamically. The ing angle estimates are superior to the homomorphic simhash representation, at the cost of reduced efficiency for certain operations. Third, we develop a homomorphic version of minhash BID5 to support Jaccard similarity. Our current algorithm is biased, but the bias appears small in practice for the situations we have considered. Although the minhash based set representations are not currently employed in hierarchical coreference, they might be useful in other models or applications that involve binary features over changing sets BID9.We provide error analysis for all three schemes, collating and extending known , and in the case of simhash, we provide novel statistical analysis for its use as a direct estimator for cos(θ) that shows the bias and variance decrease rapidly with the number of bits, helping to justify its use in a CRF for a task like coreference. On a hierarchical model for coreference resolution, the proposed simhash scheme improves the speed of probabilistic inference by an order of magnitude while having little effect on model quality. Moreover, we find that the underlying random projection representation can provide even better cosine estimates than simhash, at the cost of not being able to use certain fast bitwise-operations to compute similarities. Finally, we briefly evaluate homomorphic minhash and show that even though there are possible pathological cases, as long as we employ enough hash functions, the estimates are reasonably close to the true Jaccard, albeit, biased. Coreference resolution Coreference resolution is the problem of determining whether different mentions refer to the same underlying entity BID12. BID34. Coreference resolution arises in many other situations; for example, when merging two or more databases together it is desirable to remove duplicates that from the merge, a problem sometimes termed record linkage or deduplication BID32. Coreference is also foundational to knowledge base construction which requires that we combine information about entities of interest from multiple sources that might mention them in different contexts. For example, if we were to build a knowledge base of all scientists in the world -similar to Google scholar -we would need to perform the task of author coreference to determine who authored what papers BID16 BID8. Are the following two mentions of "J Smith" the same author? Although generally this is a difficult problem, it can be solved with machine learning since features of the mentions such as the words in the title (both have "Boson" in common), the topic of the title (both are about a similar subfield of physics), the journal (both are physics journals) and the co-authors (there appears to be a co-author in common) provide some evidence about whether or not the two "J Smith's" might be the same person. In order to solve the problem, it is thus common to extract such contextual features about each mention, such as in the above example, features from the title, co-author list, venue, year and author-name and employ them in a probabilistic model. These features are typically the raw words, character-ngrams and normalized variants thereof, sometimes with positive real-valued weights to indicate the importance (e.g., via TFIDF) of each feature. Then, given such features, a coreference model measures the similarities between mentions via functions such as cosine-similarity. In contrast to within document coreference discussed earlier, this type of coreference resolution problem is better suited for similarity based models, such as the ones we will use in the following. Moreover, since coreference decisions are not restricted by document-boundaries, the entities can grow unbounded in size, making compact representations of their growing feature sets especially important. Typically, the model is a discriminative conditional random field (CRF) that measure the probability of an assignment of mentions to entities conditioned on the observed features BID26. The model factorizes into potentials that score local coreference decisions. Local search procedures such as greedy-agglomerative clustering or Markov-chain Monte Carlo (MCMC) find the most likely assignment of mentions to entities BID26 BID8 BID42.In pairwise models, potential functions measure the compatibility of two mentions being in the same cluster. An issue with such models is that the possible pairwise comparisons scales quadratically with the number of mentions. An alternative class of models that avoids this quadratic blow-up are entity-based, in which entities are treated as first-class variables with their own set of inferred features, and potentials measure compatibility between mentions and entities. However, entity-based models come with their own scalability challenges. To illustrate the problem (and our solution), we focus on an entity-based model called hierarchical coreference, which recently won a challenge to disambiguate inventor names for the USPTO, due to its accuracy and scalability [, Monath and BID31 .Hierarchical Coreference In the hierarchical coreference model, mentions are organized into latent tree structures BID42 . There is one tree per entity with mentions at the leaves and intermediate nodes as "subentities" that organize subsets of the entity's mentions. Rather than modeling interactions between mention-pairs, the potential functions measure compatibility between child and parent nodes in the tree. The score of a given assignment of mentions into latent trees is the product of all model potentials which includes these child-parent compatibility scores as well as some additional priors on tree-shape and entities. These compatibility scores are parametrized cosine functions. Each mention is endowed with multiple feature variables that each capture a subset of the total features. For example, in author coreference, one feature variable might capture features of the author's name and another might capture features of the title and venue of the paper. Colloquially, we refer to these feature variables as "bags" since they inherit the usual bags-of-words assumption. We distinguish between different "bag-types" which each capture different types of features (e.g., the author name bag, title words bag, co-author bag, etc). The other nodes in the tree also contain feature variables (bags), but the values of these variables are determined by the current assignment of children to that parent node. In particular, a parent's bag is the sum of all its children's bags. In order to maintain this invariant, the bag representations must be updated to reflect the current state of inference, and hence for efficiency reasons, representations must be homomorphic with respect to operations that will be performed on the bags. Interpreting bags as vectors, the cosine distance between them is used to calculate their compatibility. The primary potential functions measure the compatibility between each child's bag and its parent's bag. There is one potential for each bag-type. For example, to measure the compatibility between a node z i and z j, let y ij be the binary variable that is 1 if and only if z j is the parent of z i, and let b j be a bag for z i and z j respectively, then the potential ψ for "bag 0" scores a coreference decision as: DISPLAYFORM0 where w is a real-valued weight and t is a real-valued translation parameter for potential ψ. The potentials for each bag-type have parameters w, t that we can fit to data. Because only a small handful of features are ever observed for a given mention, typical implementations of hierarchical coreference employ sparse-vector representations to represent bags (e.g., the implementation found in FACTORIE BID27 BID28).However, a key disadvantage of sparse vector representations is that they must store the indices and weights of the non-zero elements, which means that the data-structures must dynamically change in size as MCMC splits and merges entities. As the sizes of the entities grow, these operations become increasingly expensive. Similar issues arise in other entity-based models where features of entities are aggregated from those of mentions. Thus, while entity-based models avoid the quadratic comparison issue of pairwise models, a straight-forward sparse representation of their feature vectors is no longer efficient. Is there an alternative representation of feature vectors which (a) allows fast evaluation of cosine-similarity, and (b) can be efficiently dynamically updated? As we describe in the next section, the simhash hashing function BID7 provides a representation with property (a). However, the standard simhash cannot be updated as vectors are modified. We therefore develop a variant which we call homomorphic simhash, which has both properties (a) and (b). We also identify two other schemes that support these properties. We now discuss three different homomorphic representations that support addition and subtraction in the compressed space, while preserving similarity estimates. We propose homomorphic simhash and a related random projection for cosine similarity of high-dimensional vectors and multisets; and homomorphic minhash for Jacard similarity of sets. Background: simhash A locality-sensitive hash function for a distance metric d on a set of objects S is a function H such that given x, y ∈ S, we can estimate d(x, y) from the hashed representations H(x) and H(y). Simhash BID7 is a locality sensitive hash function for cosine similarity. To understand simhash, it is first helpful to consider the following randomized process: Imagine that we have two vectors a and b on the unit hypersphere in the Euclidean space R d with angle θ between them, and we want to produce an estimate of cos(θ). Select a random d-dimensional vector u by sampling each of its coordinates independently from N. Let the random variable X have value 1 if a and b are on different sides of the hyperplane orthogonal to u, and 0 otherwise. Then X is a Bernoulli random variable with expectation: DISPLAYFORM0 Let X 1,..., X n be the of independently repeating this process several times, and set DISPLAYFORM1 The idea behind simhash is to come up with a hash representation which lets us reproduce this randomized estimator: to construct a function H which produces n-bit hashes, we first randomly sample n vectors u 1,..., u n as above. Then, given a vector a, the hash H(a) is the length n bit sequence in which the ith bit is 1 if the sign of a · u i is positive and 0 1. Charikar BID7 notes that for some applications, 1 − θ π may be a good enough approximation of cos(θ), so that one can use Xn directly as an estimator of cos(θ).otherwise. Now, from two hashed representations H(a) and H(b), if the ith bit in the two hashes disagree, this is equivalent to X i in the randomized process above being equal to 1. Thus, by counting the number of positions where the hashes are distinct and dividing by n, we get X n, thereby producing an estimate of cos(θ).Rather than constructing the hash function H by sampling the u 1,..., u n vectors from the d-dimensional unit sphere uniformly, a common optimization is to instead sample them from {−1, 1} d. This has two advantages. First, it is faster to compute the dot product since no floating point multiplication is involved. Second, rather than having to explicitly sample and store each u i as a vector, we can replace each u i by a 1-bit feature hash function h i: the "value" of the vector represented by h i is 1 at coordinate j if h i (j) = 1 and is −1 if h i (j) = 0. We write a · h i for the dot product of a with the vector corresponding to h i.By restricting only to test vectors with coordinates of the form 1 and −1, the corresponding expected value of π(1 − X n) is no longer exactly θ (see [, Section 3.7 .3] for an example), but for high-dimensional spaces, this approximation is known to be effective in practice BID17. If we want to use simhash as a representation of feature vectors for entities in coreference resolution, then we need to be able to update the simhash representation as entities are merged and split. In particular, if we join nodes with feature vectors a and b, then the vector of their new parent will be a + b. However, if we only store H(a) and H(b), rather than the vectors a and b themselves, we cannot compute H(a + b): the ith bit of H(a) and H(b) just records the sign of a · h i and b · h i, and if these are different, we do not know what the sign of (a + b) · h i should be. A similar problem occurs when we split a child with vector b from a parent with vector a, since the updated parent's hash should be H(a − b).Our solution is instead to store the actual dot product of a · h i in the hash of a, rather than just the sign. That is, H(a) is now an array of length n instead of an n-bit sequence. And since DISPLAYFORM0 we can compute H(a + b) by adding component-wise the arrays for H(a) and H(b), and similarly for H(a − b). Finally, we can still efficiently compute the cosine distance between two vectors a and b by examining the signs of the entries of H(a) and H(b). We call this representation homomorphic because H is a homomorphism with respect to the additive group structure on vectors. Of course, storing each dot product instead of just the signs increases the size of our hashes. However, they are still small compared to the feature vectors and, more importantly, their sizes are fixed. We can also store both the dot product and the signs as a bit vector, making sure to update the sign vector after each operation based on the dot product. By storing the sign vector separately we can quickly count the signs in common between two vectors using bitwise operations. Statistical Properties Recall that since E[π(1 − X n)] = θ, we can derive a plausible estimate of cos(θ) from X n. In particular, let g(x) = cos(π(1−x)) and consider the estimator C n = g(X n). We now describe some statistical properties of C n. Our emphasis here is somewhat different from the standard analyses found in related work. The reason is that LSHs like simhash are most commonly used for duplicate detection BID17 and for approximate nearest neighbor search BID22, which is quite different from our use case. In those settings, one wants to show that if two items x and y are very similar, then the distance estimated from h(x) and h(y) will very likely be quite small, and conversely if x and y are very different, then their estimated distances will be large. In such cases, the linear approximation to cosine X n is sufficient. However, since we want to use the cosine distance estimates C n as part of the potential function in our CRF, we are interested in additional statistical properties of the estimator. Lemma 3.1. C n is consistent. In particular, C n a.s. Proof. By the strong law of large numbers, we have that X n a.s. DISPLAYFORM0 Since g is continuous, by the continuous mapping theorem BID40 ], DISPLAYFORM1 The first degree Taylor series for g(x) about µ is: DISPLAYFORM2 where R is the remainder term. We have then that: DISPLAYFORM3 Thus it suffices to bound |E[R(X n)]|, which can be done using Lagrange's remainder formula (see appendix). DISPLAYFORM4 For intuition, note that the Taylor series above for g shows us that DISPLAYFORM5, and plugging in the approximation we get: DISPLAYFORM6 To obtain the actual error bound, we carry out the same process but without dropping R(x) from the Taylor approximation for g, and then once again use Lagrange's remainder formula to bound the remainder (see appendix).Finally, since C n is equal to a Lipschitz continuous function of n independent Bernoulli random variables, we can use the the method of bounded differences [, Corollary 5.2], to derive the following Chernoff-like tail bound: DISPLAYFORM7 The statistics underlying simhash, whether the hyperplanes are drawn from Gaussians or the d-dimensional hypercube (i.e., Rademacher distributions), are actually random projections for which the Johnson-Lidenstrauss lemma applies BID19 BID18 BID0 ]. The lemma states that any set of m points in ddimensional Euclidean space can be embedded into n-dimensional Euclidean space, where n is logarithmic in m and independent of d, n = O(−2 log m), such that all pairwise distances are maintained within an arbitrarily small factor 1 ±, where 0 < < 1. Since the projections are linear, they are homomorphic with respect to addition and subtraction. Moreover, previous work shows that the norm-preserving property of a linear random projection A implies an angle-preserving property BID24; that is, for vectors u, v, let θ = (v, u) andθ = (Av, Au). The is the following: DISPLAYFORM0 where DISPLAYFORM1 and n ≥ 60 −2 log m. Therefore, we are justified in using the statistics underlying simhash to directly estimate the cosine similarity; viz., cos θ ≈ cosθ. More precisely, using a Taylor expansion, DISPLAYFORM2 Although we lose the bit-representation that supports the fast bit-operations that makes simhash so efficient in applications such as webpage deduplication that employ the linear estimator, we potentially gain an improvement in the quality of the cosine estimate. Certainly, the estimate will be smoother since simhash essentially quantizes the angles into a finite number of possibilities equal to the number of bits. Since the same representation supports both types of estimates, we could even alternate between the estimation strategies as dictated by the particular situation: if a large number of similarity comparisons are performed for each entity then simhash makes more sense, otherwise a direct computation of cosine makes more sense. Regardless, as we will later see, both schemes provide sufficiently fast and accurate cosine estimates for coreference resolution. Minhash BID4 BID6 ] is a locality-sensitive hash function for Jaccard similarity, defined for binary vectors (encoding sets). The Jaccard similarity between two sets S 1, S 2 ∈ Ω is J = |S 1 ∩S 2 | |S 1 ∪S 2 |. Minhash applies a random permutation (which in practice is accomplished using a random hash function, such as seeded 64-bit murmur hash) π: Ω → Ω, on a given set S ⊂ Ω and then extracts the minimum value h π (S) = min(π(S)). The probability that the minhash function for a random permutation of a set computes the same value for two sets is equal to the Jaccard similarity of the two sets BID37. In practice multiple (n) hash functions are employed to reduce the variance, ing in a vector v S = h π 0 (S), · · ·, h π n−1 (S) of n minimum values, one per hash function. There are three methods we need to design for homomorphic minhash: union, difference and score (to compute the Jaccard estimate). However, unlike simhash for cosine similarity, the operations underlying the statistics for minhash are non-linear due to the elementwise minimum operation that produces the underlying vector of hash values. Moreover, the set semantics on which the minhash score method relies are problematic because the union and difference methods need to maintain the invariance that a parent set is equal to the union of a collection of child sets: when we perform a difference operation between a parent set and a child set we cannot simply remove all of the child's elements from the parent since a sibling might also (redundantly) contribute some of those elements to the parent. We sketch a solution to these problems and include details in Appendix E. However, we note that our solution is not complete because there are several corner cases we do not yet handle. Nevertheless, we find that the representation works well in practice. First, we handle the problem of set-semantics by augmenting the n-dimensional minhash representation v S of each set S with an n-dimensional vector of counts c S, in which each dimension corresponds to a minimum hash value (essentially embracing multiset semantics, but in the hash space). The count indicates the number of child sets that contribute the associated hash value. For the union of two sets S = S 1 ∪ S 2, we either keep the count associated with the smaller hash value if they are different (that is, since we set v S = v S we set c S i = c S i where S = argmin S j ∈{S 1,S 2} (v S j i)), or sum the counts if they are the same (that is, c S i = c DISPLAYFORM0 i). For difference we employ the same strategy except we subtract the counts instead of add them. The counts are appropriately ignored when estimating the Jaccard since we want the estimate to reflect sets rather than multisets. Second, we must also address the fact that the minimum is a non-linear operator. The problem arises when the count associated with a hash value becomes zero. Then, how do we recompute what the new minimum value should be? Recomputing the minimum from scratch is a nonstarter because it would require keeping around the original sparse vector representations that we are trying to supplant. Rewriting the difference in terms of unions is also computationally expensive since it would require traversing the entire tree to check what the new minimum value should be. Instead, noting that minhash typically has hundreds of hash functions (e.g., n = 256) for each set, we propose to ignore the hash values associated with zero counts, and employ the remaining hashes to compute the Jaccard. This strategy has consequences for both bias and variance. First, since fewer hashes are employed, the variance increases, and if left unchecked may culiminate in a worst case in which all counts are zero and Jaccard can no longer be estimated. However, we can periodically refresh the counts by traversing the trees and hopefully we do not have to do such refresh operations too often in practice. Second, since the hashes associated with zero counts are correlated, the estimate is no longer unbiased. Therefore, as described in Appendix E, we modify the Jaccard estimate to make better use of the zero-counts rather than simply ignore them, and this eliminates the bias in some cases. However, we do not have a solution that works in every case. Finally, there is also the question of how to perform union and difference for cases in which the count is zero. For now, we ignore the zero counts during these operations, but are currently exploring strategies for incorporating them to further reduce bias and variance. Hierarchical coref involves other features, such as entropy and complexity-based penalties on the bag of words representations of context and topics associated with the entities. Although not a focus of this current study, we note that some of these representations depend on the ratios of p-norms that can be estimated with Johnson-Lindenstrauss style representations. Moreover, there exist hashing schemes to enable fast estimation of entropy. We save the homomorphic versions of these hashes for future work. In this section we empirically study the two cosine-preserving homomorphic compression schemes, simhash and its related random projection, on a conditional random field (CRF) model of author coreference resolution, the problem introduced in Section 2. First we study simhash and we hypothesize that the this representation will substantially improve the speed of inference for the reasons outlined earlier, but it is less clear how the simhash representation will affect the quality of the model. On the one hand, our theoretical analysis shows that the variance and bias reduce rapidly with the number of bits, but on the other hand, the sheer number of vector comparisons that take place during inference is substantial, making it likely for errors to occur anyway. With this in mind, we study how simhash affects the model quality in our first set of experiments. In our second set of experiments, we study the extent to which it improves inference speed. In a third set of experiments, we compare simhash with the random projection method. Finally, we present initial for homomorphic minhash to empirically assess how it compares to exact Jaccard, which is important given the increased bias and variance of our method. Data We employ the REXA author coreference dataset, 2 which comprises 1404 author mentions of 289 authors with ambiguous first-initial last-name combinations: D. Allen, A. Blum, S. Jones, H Robinson, S. Young, L. Lee, J. McGuire, A. Moore. We split the data such that training set contains mentions of the first five ambiguous names (950 mentions) while the testing set comprises the remaining three ambiguous names (454 mentions). The dataset is highly ambiguous since there are multiple entities with each name. In addition, we also employ the DBLP dataset which contains over one million paper citations from which we extract about five million unlabeled author mentions BID23.Model We investigate homomorphic simhash in the context of the hierarchical coreference model presented in Section 2. We employ two types of feature variables, a "name bag" that represents the features of the author's name and a "context bag" that represents the remaining features in the citation from which the author mention is extracted (co-authors, title, venue, topics, keywords). For more details about the features please see Appendix B.We employ the implementation of hierarchical coreference available in the FACTORIE toolkit, using FACTORIE's implementation of the variables, the model and the inference algorithm BID27. We additionally implement the simhash variables and potential functions inside this framework. We employ FACTORIE's default inference algorithm for hierarchical coreference which is essentially a greedy variant of multi-try Metropolis-Hastings in which the proposals make modifications to the sub-trees (e.g., move a subtree from one entity to another, or merge two trees under a common root node). More details are in previous work, and the implementation is publicly available in FACTORIE BID28 BID42. We estimate the parameters with hyper-parameter search on the training-set (Appendix B). representations. The closer points are to the identity reference line, the better. The dotted horizontal and vertical lines represent the decision threshold for inference. Points for which the two models agree are in blue, the agreements rates are: 88.6, 83.6, 97.0, 97.8 percent (respectively 32, 64, 128, 256 bits).Experiment 1: Simhash Estimation Quality Here we study the quality of models with simhash representations by comparing them to models with exact representations. First, we compare how these models evaluate and score the intermediate of MCMC inference. We run MCMC for 100,000 steps to optimize simhash-based models on the REXA test set (with either 256, 128, 64 and 32 bit hashes). The chains begin with the singleton configuration (all mentions in their own entity tree of size one), and at each step proposes changes to the current state which the model decides to either accept or reject. This process gradually produces larger and larger trees. For each proposed state change (sample), we record the log model ratio of both the simhash model and the exact model. We present every 10th sample in a scatter plot in FIG2. The closer the points are to the identity reference line y = x, the more accurate the simhash model is for those points. Varying the number of bits has a pronounced effect on the model's ability to judge MCMC states. For each step, we also record if the simhash model and exact model agree upon whether to accept the stochastic proposal (blue points) or not (red points). 3 The agreement rates are 97.8, 97.0, 93.6, 88.6 percent (respectively 256, 128, 64, 32 bits). We also plot the decision boundaries for this agreement as dotted lines. The upper-left and lower-right quadrants contain all the points for which the two models disagree, while the other two quadrants contain points for which they agree. In particular, the upper-right quadrants contain the points that both the simhash model and exact model believes should be accepted (true positives), while the lower-right quadrants contain the points that both models think should be rejected (true negatives). Most points lie in this quadrant since the chance of proposing a fruitful move is relatively low. Visually, there appears to be a qualitative gap between 64 bits and 128 bits on this data, leading to a recommendation of using at least 128 bits. We also compare the models in terms of their final coreference performance (according to B-cubed F1 BID1). The exact model achieves an F1 of 78.6, while the simhash variants achieve F1 scores of 77.6, 75.6, 62.8, 55.6 for 256, 128, 64, 32 bits respectively. For more detailed , with precision, recall and other types of F1, see Table 1 in the appendix. Overall, the accuracy of the 128 and 256-bit models are reasonable with 256 being competitive with the performance of the exact model. When using fewer bits, again, the performance decreases precipitously. Experiment 2: Simhash Speed In this experiment we study the extent to which the compressed simhash representation improves inference speed. As described before, we tune the models on the REXA training set. Then, to test the models on a larger dataset, we supplement the 454 labeled REXA test mentions with five million unlabeled mentions from DBLP. We run each model on this combined dataset, initializing to the singleton configuration and then running one billion samples. We record coreference quality on the labeled subset every 10,000 samples and plot how it changes over time in FIG3.Although Experiment 1 shows that the simhash models are slightly worse in terms of their final F1 accuracy on the REXA test set, we see a striking computational advantage for simhash. For each F1 value, we compute how much faster the simhash model achieves that value than the exact model and plot this speed-improvement as a function of F1-level in FIG3,2(d). As we can see, the speed improvement is more than a base-ten order of magnitude for most accuracy levels, and the speed difference increases with accuracy. This is because the exact representation slows down over time as the number of features in the representation grows during inference. Indeed if we look at the sampling rate over time for each model, we find that the simhash models run at about 20,000-25,000 samples per second the entire time, while the model with the exact representation starts at a rate of 3000-4000 samples per second, but then drops to under 1000 samples per second as the size of the entities get larger. This raises the question: can we improve the speed of the exact model by reducing the number of features? We address this question in Appendix C.2, but summarize here briefly: it is possible to improve the speed and reduce the gap, but simhash is still faster and there is a trade-off with coreference quality. In addition, whether feature ablation meaningfully improves performance depends on particular aspects of the data set, whereas the simhash representation can be applied generically. Experiment 3: JL Random Projections In this experiment, we compare simhash with an approach that directly computes cosine on the statistics that underly the bit representation. As argued previously in Section 3.2, this approach is justified because the statistics are random projections that are homomorphic and cosine-preserving. Intuitively, we expect this approach to work even better because simhash further compresses these statistics into single bits, whereas in the random projection approach, we compute cosine directly on the real-valued statistics. However, our current theoretical analysis does not allow us to compare one to another; therefore, we turn to the empirical studies in this section. We perform an experiment analogous to Experiment 1, except we employ the exact model to make decisions about what MCMC proposals to accept. This allows us to compare both approximation schemes on the same set of samples. For each accepted sample, we again ask the question how the approximate methods, simhash and JL, would have judged them. We find that JL does indeed perform better than simhash for the same number of bits. 4 In particuar, the Spearman's rho for JL increases from 93.2 to 97.2 over simhash when employing 256 bits (dimensions). For all but one case (128 bits), we find a similar improvement. Moreover, the coreference accuracy also increases in each case; for example, from 77.6 B3 F1 to 78.3. More detailed are in Table 3 in Appendix D, along with the associated scatter plots (Figure 6). We also investigate our homomorphic MinHash representation for Jaccard similarity. For lack of space, and because Jaccard is not employed by the hierarchical coreference, we relegate most of the evaluation to Appendix E.2. To briefly summarize, we find that 128 and 256 hash functions are sufficient for reasonable estimates and that the representation rarely needs to be refreshed to ensure that all the counts are above zero, enabling the computational efficiency we desire. However, there is room for improvement because only 16% of the hash functions on average have a non-zero entry after 100,000 samples. Thus, after 100,000 samples, a 256 hash version will begin to behave like a 40 hash version. Homomorphic representations have many possible uses in machine learning; for example, homomorphic encryption allows models to run directly on encrypted data for cases in which privacy is a concern [, Dowlin et al., 206]. Instead, our method is similar to homomorphic compression BID29 which allows our model to run directly on compressed data for cases in which computational efficiency is a concern. Our approach is based on simhash, a locality-sensitive hash function. Locality sensitive hashes such as simhash and minhash are useful in large scale streaming settings; for example, to detect duplicate webpages for web-crawlers or in nearest neighbor search BID5 BID25 BID22. They are sometimes employed in search and machinelearning applications including coreference for "blocking" the data in order to reduce the search space BID12. Note that this application of locality sensitive hash functions for coreference is complementary to ours, and does not require it to be homomorphic. Other hashing and sketching algorithms are also employed as a strategy to compress the sufficient statistics in Bayesian models so that they can scale better as the number of parameters increase with the dataset. For example, feature hashing, count-min sketches and approximate counters have been employed in scaling latent Dirichlet categorical models such as LDA by compressing, for example, the counts that assign latent variables to mixture components BID44 BID38.Our paper focuses on three homomorphic compression schemes including one for minhash. Our solution of furnishing the minhash values with counts resembles a strategy that similarly employs counts in a k minimum values (KMV) sketch for estimating the number of distinct values in a set BID2 BID3. A key difference is that that work develops an unbiased estimator for the number of distinct values that explicitly involves the counts as part of the estimate itself, whereas we develop a biased estimator that directly estimates Jaccard similarity and that employs the counts to bound our knowledge about possible minimum hash values when they vanish during difference operations. Our schemes are related to random projections, which are commonly used to improve computational efficiency in machine learning. Examples include feature-hashing BID41, fast matrix decomposition BID15 and fast kernel computations BID35. However, while some of these random projections happen to be homomorphic, this property is often not exploited. The reason is that they are typically used to compress static objects that remain fixed, such as the Gram matrix for kernel methods. In contrast, our setting requires compressing dynamic sets that change during inference. Word embeddings BID30 also provide low-dimensional dense representations that are useful for capturing context, but in practice, these embeddings are too smooth to be useful as the sole representation for disambiguating the names of peoples, places or organizations, as is necessary in coreference (e.g., the names "Brady" and "Belichick" would be highly similar according to a word-to-vec style embedding, even though they are unlikely to be coreferent). Deep learning might allow for more suitable representations to be learnt and its application to coreference is promising. However, the current research focus is on within-document noun-phrase coreference and entity linking, while emphasizing improvements in accuracy rather than inference speed and scalability BID13 BID43 BID21 BID36. Combining deep learning and conditional random fields for hierarchical coreference remains an open problem. Finally, in addition to the work mentioned throughout the paper, there is an abundance of related work on coreference resolution including noun-phrase coreference, cross-document coreference, record linking, entity linking and author coreference. While not possible to cover the breadth of the space here, we refer the reader to a few useful surveys and tutorials on the subject [, Getoor and BID12. More recently, as mentioned in the foregoing paragraph, deep learning approaches are beginning to show promise. There has also been promising recent work on scalable hierarchical clustering known as PERCH BID20. Since PERCH employs Euclidean distance rather than cosine similarity, it currently falls outside the purview of our study. However, the Johnson-Lindenstrauss applies to Euclidean distance making random projections a possibility for PERCH. In this paper we presented several homomorphic compression schemes for representing the sparse, high dimensional features in a graphical model for coreference resolution, even as these features change during inference. Our primary concern was cosine similarity for which we investigated simhash and angle-preserving random projections. We also proposed a homomorphic version of minhash for Jaccard similarity. In the particular case of simhash, we presented a new variant of the original estimator and analyzed its statistical propertiesincluding variance, bias and consistency -to help justify its use in a probabilistic model. We found that both simhash and angle-preserving random projections were sufficiently accurate, given enough dimensions, to represent features for coreference, and that the random projections produce slightly better estimates. Moreover, these representations were an order of magnitude faster than conventional sparse data structures, laying the foundation for greater scalability.

Represent the TLDR for finding the corresponding scientific article!

We employ linear homomorphic compression schemes to represent the sufficient statistics of a conditional random field model of coreference and this allows us to scale inference and improve speed by an order of magnitude.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We provide insightful understanding of sequence-labeling NER and propose to use two types of cross structures, both of which bring theoretical and empirical improvements.