Question:
represent the article for finding a single-sentence tldr:
For example, 'Training labels are expensive to obtain and may be of varying quality, as some may be from trusted expert labelers while others might be from heuristics or other sources of weak supervision such as crowd-sourcing. This creates a fundamental quality-versus-quantity trade-off in the learning process. Do we learn from the small amount of high-quality data or the potentially large amount of weakly-labeled data? We argue that if the learner could somehow know and take the label-quality into account, we could get the best of both worlds. To this end, we introduce “fidelity-weighted learning” (FWL), a semi-supervised student-teacher approach for training deep neural networks using weakly-labeled data. FWL modulates the parameter updates to a student network, trained on the task we care about on a per-sample basis according to the posterior confidence of its label-quality estimated by a teacher, who has access to limited samples with high-quality labels. "All samples are equal, but some samples are more equal than others." -Inspired by George Orwell quote, The success of deep neural networks to date depends strongly on the availability of labeled data and usually it is much easier and cheaper to obtain small quantities of high-quality labeled data and large quantities of unlabeled data. For a large class of tasks, it is also easy to define one or more so-called "weak annotators" BID10, additional (albeit noisy) sources of weak supervision based on heuristics or weaker, biased classifiers trained on e.g. non-expert crowd-sourced data or data from different domains that are related. While easy and cheap to generate, it is not immediately clear if and how these additional weakly-labeled data can be used to train a stronger classifier for the task we care about. Assuming we can obtain a large set of weakly-labeled data in addition to a much smaller training set of "strong" labels, the simplest approach is to expand the training set simply by including the weakly-supervised samples (all samples are equal). Alternatively, one may pretrain on the weak data and then fine-tune on strong data, which is one of the common practices in semi-supervised learning. We argue that treating weakly-labeled samples uniformly (i.e. each weak sample contributes equally to the final classifier) ignores potentially valuable information of the label quality. Instead, we introduce Fidelity-Weighted Learning (FWL), a Bayesian semi-supervised approach that leverages a small amount of data with true labels to generate a larger training set with confidence-weighted weakly-labeled samples, which can then be used to modulate the fine-tuning process based on the fidelity (or quality) of each weak sample. By directly modeling the inaccuracies introduced by the weak annotator in this way, we can control the extent to which we make use of this additional source of weak supervision: more for confidently-labeled weak samples close to the true observed data, and less for uncertain samples further away from the observed data. In this section, we describe FWL. We assume we are given a large set of unlabeled data samples, a heuristic labeling function called the weak annotator, and a small set of high-quality samples labeled by experts, called the strong dataset, consisting of tuples of training samples x i and their true labels y i, i.e. D s = {(x i,y i)}. We consider the latter to be observations from the true target function that we are trying to learn. We use the weak annotator to generate labels for the unlabeled samples. Generated labels are noisy due to the limited accuracy of the weak annotator. This gives us the weak dataset consisting of tuples of training samples x i and their weak labelsỹ i, i.e. D w = {(x i,ỹ i)}. Note that we can generate a large amount of weak training data D w at almost no cost using the weak annotator. In contrast, we have only a limited amount of observations from the true function, i.e. |D s | |D w |. Step 1: Pre-train student on weak data, Step 2: Fit teacher to observations from the true function, and Step 3: Fine-tune student on labels generated by teacher, taking the confidence into account. Red dotted borders and blue solid borders depict components with trainable and non-trainable parameters, respectively. Our proposed setup comprises a neural network called the student and a Bayesian function approximator called the teacher. The training process consists of three phases which we summarize in FIG0.Step 1 Pre-train the student on D w using weak labels generated by the weak annotator. The main goal of this step is to learn a task dependent representation of the data as well as pretraining the student. The student function is a neural network consisting of two parts. The first part ψ learns the data representation and the second part φ performs the prediction task (e.g. classification). Therefore the overall function isŷ = φ(ψ(x i)). The student is trained on all samples of the weak dataset D w = {(x i,ỹ i)}. For brevity, in the following, we will refer to both data sample x i and its representation ψ(x i) by x i when it is obvious from the context. Step 2 Train the teacher on the strong data (ψ(x j), y j ) ∈ D s represented in terms of the student representation ψ and then use the teacher to generate a soft dataset D sw consisting of sample,predicted label, confidence for all data samples. We use a Gaussian process as the teacher to capture the label uncertainty in terms of the student representation, estimated w.r.t the strong data. A prior mean and covariance function is chosen for GP. The learned embedding function ψ(·) in Step 1 is then used to map the data samples to dense vectors as input to the GP. We use the learned representation by the student in the previous step to compensate lack of data in D s and the teacher can enjoy the learned knowledge from the large quantity of the weakly annotated data. This way, we also let the teacher see the data through the lens of the student. The GP is trained on the samples from D s to learn the posterior mean m post (used to generate soft labels) and posterior co-variance K post (., .) (which represents label uncertainty) BID17. We then create the soft dataset D sw = {(x t,ȳ t)} using the posterior GP, input samples x t from D w ∪ D s, and predicted labelsȳ t with their associated uncertainties as computed T (x t) = g(m post (x t)) and Σ(x t) = h(K post (x t,x t)). The generated labels are called soft labels. Therefore, we refer to D sw as a soft dataset. g transforms the output of GP to the suitable output space. For example in classification tasks, g would be the softmax function to produce probabilities that sum up to one. For multidimensional-output tasks where a vector of variances is provided by the GP, the vector K post (x t,x t) is passed through an aggregating function h to generate a scalar value for the uncertainty of each sample. Note that we train GP only on the strong dataset D s but then use it to generate soft labelsȳ t = T (x t) and uncertainty Σ(x t) for samples belonging to D sw = D w ∪D s.Step 3 Fine-tune the weights of the student network on the soft dataset, while modulating the magnitude of each parameter update by the corresponding teacher-confidence in its label. The student network of Step 1 is fine-tuned using samples from the soft dataset D sw = {(x t,ȳ t)} wherē y t = T (x t). The corresponding uncertainty Σ(x t) of each sample is mapped to a confidence value, and this is then used to determine the step size for each iteration of the stochastic gradient descent (SGD). So, intuitively, for data points where we have true labels, the uncertainty of the teacher is almost zero, which means we have high confidence and a large step-size for updating the parameters. However, for data points where the teacher is not confident, we down-weight the training steps of the student. This means that at these points, we keep the student function as it was trained on the weak data in Step 1. The weak annotator, i.e. the unsupervised method used for annotating the unlabeled data. Full Supervision Only, i.e. the student trained only on strong labeled data (Ds). Weak Supervision Only, i.e. the or the student trained only on weakly labeled data (Dw).NN W/S + Weak Supervision + Oversampled Strong Supervision, i.e. thestudent trained on samples that are alternately drawn from Dw without replacement, and Ds with replacement. Since |Ds| |Dw|, it oversamples the strong data. Weak Supervision + Fine Tuning, i.e. the student trained on weak dataset Dw and fine-tuned on strong dataset Ds. NN W ω →NN S The student trained on the weak data, but the step-size of each weak sample is weighted by a fixed value 0 ≤ ω ≤ 1, and fine-tuned on strong data. As an approximation for the optimal value for ω, we have used the mean of η2 of our model (below). The student trained on the weakly labeled data and fine-tuned on examples labeled by the teacher without taking the confidence into account. This baseline is similar to BID16.More specifically, we update the parameters of the student by training on D sw using SGD: DISPLAYFORM0 where l(·) is the per-example loss, η t is the total learning rate, N is the size of the soft dataset D sw, w w w is the parameters of the student network, and R is the regularization term. We define the total learning rate as η t = η 1 (t)η 2 (x t), where η 1 (t) is the usual learning rate of our chosen optimization algorithm that anneals over training iterations, and η 2 (x t) is a function of the label uncertainty Σ(x t) that is computed by the teacher for each data point. Multiplying these two terms gives us the total learning rate. In other words, η 2 represents the fidelity (quality) of the current sample, and is used to multiplicatively modulate η 1. Note that the first term does not necessarily depend on each data point, whereas the second term does. We propose η 2 (x t) = exp[−βΣ(x t)] to exponentially decrease the learning rate for data point x t if its corresponding soft labelȳ t is unreliable (far from a true sample). In practice, when using mini-batches, we implement this by multiplying the loss of each example in the batch by its fidelity score and average over these fidelity-weighted losses in the batch when calculating the batch gradient based on that loss. β is a positive scalar hyper-parameter that controls the contribution of weak and strong data to the training procedure. A small β in a student which listens more carefully to the teacher and copies its knowledge, while a large β makes the student pay less attention to the teacher, staying with its initial weak knowledge. Hence, β gives a handle to control the bias-variance trade-off. In Appendix A, we apply FWL to a one-dimensional toy problem to illustrate its various steps. In this section, we apply FWL to document ranking task and evaluate its performance compared to the baselines presented in TAB1. Document Ranking is the core information retrieval problem and is challenging as the ranking model needs to learn a representation for long documents and capture the complex notion of relevance between queries and documents. Furthermore, the size of publicly available datasets with query-document relevance judgments is unfortunately quite small (∼ 250 queries). We employ a state-of-the-art pairwise neural ranker architecture as the student BID3 in which the ranking is cast as a regression task. Given each training sample x as a triple of query q, and two documents d + and d −, the goal is to learn a function F: {< q,d +,d − >} → R, which maps each data sample x to a scalar output value y indicating the probability of d + being ranked higher than d − with respect to q. The student follows the architecture proposed in BID3. The first layer of the network, i.e. representation learning layer ψ: {< q,d DISPLAYFORM0 − >} → R m maps each input sample to an mdimensional real-valued vector. In general, besides learning embeddings for words, function ψ learns to compose word embedding based on their global importance in order to generate query/document embeddings. The representation layer is followed by a simple fully-connected feed-forward network with a sigmoidal output unit to predict the probability of ranking d + higher than d −. The general schema of the student is illustrated in FIG1. More details are provided in Appendix C.1.The teacher is implemented by clustered GP algorithm. See Appendix C.2 for more details. The weak annotator is BM25 BID11 ), a well-known unsupervised method for scoring query-document pairs based on statistics of the matched terms. More details are provided in Appendix C.3. Description of the data with weak labels and data with true labels as well as the setup of the document-ranking experiments is presented in Appendix C.4 in more details. Results and Discussions. We conducted k-fold cross-validation on D s (the strong data) and report two standard evaluation metrics for ranking: mean average precision (MAP) of the top-ranked 1,000 documents and normalized discounted cumulative gain calculated for the top 20 retrieved documents (nDCG@20). TAB2 shows the performance on both datasets. As can be seen, FWL provides a significant boost on the performance over all datasets. In the ranking task, the student is designed in particular to be trained on weak annotations BID3, hence training the network only on weak supervision, i.e. NN W performs better than NN S. This can be due to the fact that ranking is a complex task requiring many training samples, while relatively few data with true labels are available. Alternating between strong and weak data during training, i.e. NN S + /W seems to bring little (but statistically significant) improvement. However, we can gain better by the typical fine-tuning strategy, NN W→S. We can gain improvement by fine-tuning the NN W using labels generated by the teacher without considering their confidence score, i.e. FWL \Σ. This means we just augmented the fine-tuning process by generating a fine-tuning set using teacher which is better than D s in terms of quantity and D w in terms of quality. This baseline is equivalent to setting β = 0. However, we see a big jump in performance when we use FWL to include the estimated label quality from the teacher, leading to the best overall . Sensitivity of the FWL to the Quality of the Weak Annotator. Our proposed setup in FWL requires defining a socalled "weak annotator" to provide a source of weak supervision for unlabelled data. In this section, we study how the quality of the weak annotator may affect the performance of the FWL on the Robust04 dataset. To do so, besides BM25 BID11 ), we use three other weak annotators: vector space model BID12 with binary term occurrence (BTO) weighting schema and vector space model with TF-IDF weighting schema, which are both weaker than BM25, and BM25+RM3 that uses RM3 as the pseudo-relevance feedback method on top of BM25, leading to better labels. FIG2 illustrates the performance of these four weak annotators in terms of their mean average precision (MAP) on the test data, versus the performance of FWL given the corresponding weak annotator. As it is expected, the performance of FWL depends on the quality of the employed weak annotator. The percentage of improvement of FWL over its corresponding weak annotator on the test data is also presented in FIG2. As can be seen, the better the performance of the weak annotator is, the less the improvement of the FWL would be. Training neural networks using large amounts of weakly annotated data is an attractive approach in scenarios where an adequate amount of data with true labels is not available, a situation which often arises in practice. In this paper, we introduced fidelity-weighted learning (FWL), a new student-teacher framework for semi-supervised learning in the presence of weakly labeled data. We applied FWL to document ranking and empirically verified that FWL speeds up the training process and improves over state-of-the-art semi-supervised alternatives. To better understand FWL, we apply FWL to a one-dimensional toy problem to illustrate the various steps. Let ft(x) = sin(x) be the true function (red dotted line in FIG4) from which a small set of observations Ds = {xj,yj} is provided (red points in FIG4). These observation might be noisy, in the same way that labels obtained from a human labeler could be noisy. A weak annotator function fw(x) = 2sinc(x) (magenta line in FIG4) is provided, as an approximation to ft.The task is to obtain a good estimate of ft given the set Ds of strong observations and the weak annotator function fw. We can easily obtain a large set of observations Dw = {xi,ỹi} from fw with almost no cost (magenta points in FIG4).As the teacher, we use standard Gaussian process regression 2 with this kernel: DISPLAYFORM0 where, kRBF(xi,xj) = exp xi −xj 2 2 2 k White (xi,xj) = constant value, ∀x1 = x2 and 0 otherwise We fit only one GP on all the data points (i.e. no clustering). Also during fine-tuning, we set β = 1. The student is a simple feed-forward network with the depth of 3 layers and width of 128 neurons per layer. We have used tanh as the nonlinearity for the intermediate layers and a linear output layer. As the optimizer, we used Adam and the initial learning rate has been set to 0.001. We randomly sample 100 data points from the weak annotator and 10 data points from the true function. We introduce a small amount of noise to the observation of the true function to model the noise in the human labeled data. We consider two experiments:1. A neural network trained on weak data and then fine-tuned on strong data from the true function, which is the most common semi-supervised approach FIG4 ). 2. A teacher-student framework working by the proposed FWL approach. As can be seen in FIG4, FWL by taking into account label confidence, gives a better approximation of the true hidden function. We repeated the above experiment 10 times. The average RMSE with respect to the true function on a set of test points over those 10 experiments for the student, were as follows: Algorithm 1 Clustered Gaussian processes.1: Let N be the sample size, n the sample size of each cluster, K the number of clusters, and ci the center of cluster i. 2: Run K-means with K clusters over all samples with true labels Ds = {xi,yi}.K-means(xi) → c1,c2,...,cK where ci represents the center of cluster Ci containing samples D c i s = {xi,1,xi,2,...xi,n}. 3: Assign each of K clusters a Gaussian process and train them in parallel to approximate the label of each sample. DISPLAYFORM1 where GP c i is trained on D c i s containing samples belonging to the cluster ci. Other elements are defined in Section 2 4: Use trained teacher Tc i to evaluate the soft label and uncertainty for samples from Dsw to compute η2(xt) required for step 3 of FWL. We use T as a wrapper for all teachers {Tc i}. We suggest using several GP = {GPc i} to explore the entire data space more effectively. Even though inducing points and stochastic methods make GPs more scalable we still observed poor performance when the entire dataset was modeled by a single GP. Therefore, the reason for using multiple GPs is mainly empirically inspired by BID13 which is explained in the following: We used the Sparse Gaussian Process implemented in GPflow. The algorithm is scalable in the sense that it is not O(N 3) as original GP is. It introduces inducing points in the data space and defines a variational lower bound for the marginal likelihood. The variational bound can now be optimized by stochastic methods which make the algorithm applicable in large datasets. However, the tightness of the bound depends on the location of inducing points which are found through the optimization process. The pseudo-code of the clustered GP is presented in Algorithm 1. When the main issue is computational resources (when the number of inducing points for each GP is large), we can first choose the number n which is the maximum size of the dataset on which our resources allow to train a GP, then find the number of clusters K = N/n accordingly. The rest of the algorithm remains unchanged. The employed student is proposed in BID3. The first layer of the network models function ψ that learns the representation of the input data samples, i.e. (q,d+,d −), and consists of three components: an embedding function ε: V → R m (where V denotes the vocabulary set and m is the number of embedding dimensions), a weighting function ω: V → R, and a compositionality function: (R m,R) n → R m. More formally, the function ψ is defined as: DISPLAYFORM0 where t q i and t d i denote the i th term in query q respectively document d. The embedding function ε maps each term to a dense m-dimensional real value vector, which is learned during the training phase. The weighting function ω assigns a weight to each term in the vocabulary. It has been shown that ω simulates the effect of inverse document frequency (IDF), which is an important feature in information retrieval BID3.The compositionality function projects a set of n embedding-weighting pairs to an m-dimensional representation, independent from the value of n: DISPLAYFORM1 which is in fact the normalized weighted element-wise summation of the terms' embedding vectors. Again, it has been shown that having global term weighting function along with embedding function improves the performance of ranking as it simulates the effect of inverse document frequency (IDF). In our experiments, we initialize the embedding function ε with word2vec embeddings BID6 pre-trained on Google News and the weighting function ω with IDF.The representation layer is followed by a simple fully connected feed-forward network with l hidden layers followed by a softmax which receives the vector representation of the inputs processed by the representation learning layer and outputs a predictionỹ. Each hidden layer z k in this network computes DISPLAYFORM2 where W k and b k denote the weight matrix and the bias term corresponding to the k th hidden layer and α is the non-linearity. These layers follow a sigmoid output. We employ the cross entropy loss: DISPLAYFORM3 where B is a batch of data samples. We use Gaussian Process as the teacher and pass the mean of GP through the same function g that is applied on the output of the student network. h is an aggregation function that takes variance over several dimensions and outputs a single measure of variance. As a reasonable choice, the aggregating function h in our sentiment classification task (three classes) is mean of variances over dimensions. In the teacher, linear combinations of different kernels are used in our experiments. We use sparse variational GP regression 3 with this kernel: DISPLAYFORM0 where, DISPLAYFORM1 +xi.xj k White (xi,xj) = constant value, ∀x1 = x2 and 0 otherwise We empirically found l = 1 satisfying value for the length scale of Matern3/2 kernels. We also set σ0 = 0 to obtain a homogeneous linear kernel. The constant value of K W hite (.,.) determines the level of noise in the labels. This is different from the noise in weak labels. This term explains the fact that even in true labels there might be a trace of noise due to the inaccuracy of human labelers. We set the number of clusters in the clustered GP algorithm for the ranking task to 50. The weak annotator is BM25 BID11 ), a well-known unsupervised retrieval method. This method heuristically scores a given pair of query-document based on the statistics of their matched terms. In the pairwise document ranking setup,ỹi for a given sample xj = (q,d DISPLAYFORM0 is the score obtained from the weak annotator. Collections We use two standard TREC collections for the task of ad-hoc retrieval: The first collection (Robust04) consists of 500k news articles from different news agencies as a homogeneous collection. The second collection (ClueWeb) is ClueWeb09 Category B, a large-scale web collection with over 50 million English documents, which is considered as a heterogeneous collection. Spam documents were filtered out using the Waterloo spam scorer 4 BID1 with the default threshold 70%.Data with true labels We take query sets that contain human-labeled judgments: a set of 250 queries (TREC topics 301-450 and 601-700) for the Robust04 collection and a set of 200 queries (topics for the experiments on the ClueWeb collection. For each query, we take all documents judged as relevant plus the same number of documents judged as non-relevant and form pairwise combinations among them. Data with weak labels We create a query set Q using the unique queries appearing in the AOL query logs BID9 . This query set contains web queries initiated by real users in the AOL search engine that were sampled from a three-month period from March 2006 to May 2006. We applied standard pre-processing BID3 a) on the queries: We filtered out a large volume of navigational queries containing URL substrings ("http", "www.", ".com", ".net", ".org", ".edu"). We also removed all non-alphanumeric characters from the queries. For each dataset, we took queries that have at least ten hits in the target corpus using our weak annotator method. Applying all these steps, We collect 6.15 million queries to train on in Robust04 and 6.87 million queries for ClueWeb. To prepare the weakly labeled training set Dw, we take the top 1,000 retrieved documents using BM25 for each query from training query set Q, which in total leads to ∼ |Q|×10 6 training samples. Setup For the evaluation of the whole model, we conducted 3-fold cross-validation. However, for each dataset, we first tuned all the hyper-parameters of the student in the first step on the set with true labels using batched GP bandits with an expected improvement acquisition function BID4 and kept the optimal parameters of the student fixed for all the other experiments. The size and number of hidden layers for the student is selected from {64,128,256,512}. The initial learning rate and the dropout parameter were selected from {10 −3,10 −5} and {0.0,0.2,0.5}, respectively. We considered embedding sizes of {300,500}. The batch size in our experiments was set to 128. We use ReLU BID7 as a non-linear activation function α in student. We use the Adam optimizer for training, and dropout BID14 as a regularization technique. At inference time, for each query, we take the top 2,000 retrieved documents using BM25 as candidate documents and re-rank them using the trained models. We use the Indri 5 implementation of BM25 with default parameters (i.e., k1 = 1.2, b = 0.75, and k3 = 1,000).' should have a representation like 'We propose Fidelity-weighted Learning, a semi-supervised teacher-student approach for training neural networks using weakly-labeled data.' but very far from 'A backward model of previous (state, action) given the next state, i.e. P(s_t, a_t | s_{t+1}), can be used to simulate additional trajectories terminating at states of interest! Improves RL learning efficiency.'.

Question: In this paper, we propose a nonlinear unsupervised metric learning framework to boost of the performance of clustering algorithms. Under our framework, nonlinear distance metric learning and manifold embedding are integrated and conducted simultaneously to increase the natural separations among data samples. The metric learning component is implemented through feature space transformations, regulated by a nonlinear deformable model called Coherent Point Drifting (CPD). Driven by CPD, data points can get to a higher level of linear separability, which is subsequently picked up by the manifold embedding component to generate well-separable sample projections for clustering. Experimental on synthetic and benchmark datasets show the effectiveness of our proposed approach over the state-of-the-art solutions in unsupervised metric learning. Cluster analysis has broad applications in various disciplines. Grouping data samples into categories with similar features is an efficient way to summarize the data for further processing. In measuring the similarities among data samples, the Euclidean distance is the most common choice in clustering algorithms. Under Euclidean distance, feature components are assigned with the same weight, which essentially assumes all features are equally important across the entire data space. In practice, such setup is often not optimal. Learning a customized metric function from the data samples can usually boost the performance of various machine learning algorithms BID1. While metric learning has been extensively researched under supervised BID19 BID18 BID17 BID14 and semi-supervised settings BID15 BID3 BID23 BID13, unsupervised metric learning (UML) remains a challenge, in part due to the absence of ground-truth label information to define a learning optimality. In this paper, we focus on the problem of UML for clustering. As the goal of clustering is to capture the natural separations among data samples, one common practice in the existing UML solutions is to increase the data separability and make the separations more identifiable for the ensuing clustering algorithm. Such separability gain can be achieved by projecting data samples onto a carefully chosen low-dimensional manifold, where geometric relationships, such as the pairwise distances, are preserved. The projections can be carried out linearly, as through the Principle Component Analysis, or nonlinearly, as via manifold learning solutions. Under the dimension-reduced space, clustering algorithms, such as K-means, can then be applied. Recent years have seen the developments of UML solutions exploring different setups for the lowdimensional manifolds. FME ) relies on the learning of an optimum linear regression function to specify the target low-dimensional space. BID0 model local sample densities of the data to estimate a new metric space, and use the learned metric as the basis to construct graphs for manifold learning. Application-specific manifolds, such as Grassmann space BID6 and Wasserstein geometry BID16, have also been studied. When utilized as a separate preprocessing step, dimensionality reduction UML solutions are commonly designed without considering the ensuing clustering algorithm and therefore cannot be fine-tuned accordingly. AML takes a different approach, performing clustering and distance metric learning simultaneously. The joint learning under AML is formulated as a trace maximization problem, and numerically solved through an EM-like iterative procedure, where each iteration consists of a data projection step, followed by a clustering step via kernel K-means. The projection is parameterized by an orthogonal, dimension-reducing matrix. A kernelized extension of AML was proposed in BID2. As the projection models are built on linear transformations, their capabilities to deal with complex nonlinear structures are limited. UML solutions performing under the original input space have also been proposed. SSO BID7 learns a global similarity metric through a diffusion procedure that propagates smooth metrics through the data space. CPCM BID4 relies on the ratio of within cluster variance over the total data variance to obtain a linear transformation, aiming to improved data separability. As the original spaces are usually high-dimensional, UML solutions in this category tend to suffer from the local minima problem. In light of the aforementioned limitations and drawbacks, we propose a new nonlinear UML framework in this paper. Our solution integrates nonlinear feature transformation and manifold embedding together to improve the data separability for K-means clustering. Our model can be regarded as a fully nonlinear generalization of AML, in which the transformation model is upgraded to a geometric model called Coherent Point Drifting (CPD) BID8. Data points are driven by CPD to reach a higher level of linear separability, which will be subsequently picked up by the manifold embedding component to generate well-separable sample projections. At the end, K-means is applied on the transformed, dimension-reduced embeddings to produce label predictions. The choice of CPD is with the consideration of its capability of generating high-order yet smooth transformations. The main contributions of this paper include the following.• Our proposed fully nonlinear UML solution enhances data separability through the combination of CPD-driven deformation and spectral embeddings.• To the best of our knowledge, this is the first work that utilizes dense, spatial varying deformations in unsupervised metric learning.• The CPD optimization has a closed-form solution, therefore can be efficiently computed.• Our model outperforms state-of-the-art UML methods on six benchmark databases, indicating promising performance in many real-world applications. The rest of this paper is organized as follows. Section 2 describes our proposed method in detail. It includes the description of CPD model, formulation of our CPD based UML, optimization strategy and the approach to kernelize our model. Experimental are presented in Section 3 to validate our solutions with both synthetic and real-world datasets. Section 4 concludes this paper. Many machine learning algorithms have certain assumption regarding the distribution of the data to be processed. K-means always produces clustering boundaries of hyperplanes, working best for the data set made of linearly separable groups. For data sets that are not linearly separable, even they are otherwise well-separable, K-means will fail to deliver. Nonlinearly displacing the data samples to make them linearly separable would provide a remedy, and learning such a transformation is the goal of our design. The application of such a smooth nonlinear transformation throughout feature space (either input space or kernel space) would change pairwise distances among samples, which is equivalent to assigning spatially varying metrics in different areas of the data space. In our framework, the CPD model is chosen to perform the transformation. Originally designed for regulating points matching, CPD moves the points U towards the target V by estimating an optimal continuous velocity function v(x): DISPLAYFORM0 where n is the number of samples in the dataset, and d is the data dimension. L represents a linear differentiation operator, and λ is the regularization parameter. The regularization term in CPD is a Gaussian low-pass filter. The optimal solution v(x) to matching U and V can be written in the matrix format as BID9: DISPLAYFORM1 where Ψ (size d × n) is the weight matrix for the Gaussian kernel functions, g(DISPLAYFORM2 . σ is the width of the Gaussian filter, which controls the smoothness level of the deformation field. K-means clustering aims to partition the samples into K groups S = {S 1, S 2, ..., S K}, through the minimization of the following objective function: DISPLAYFORM0 S c is the set of data samples in the c-th cluster. n c is the number of data instances in cluster S c, and µ c is the mean of S c.Allowing samples to be moved, we intend to learn a spatial transformation to improve the performance of K-means clustering by making groups more linearly separable, as well as by harnessing the updated distance measure under the transformed feature space. Let x i be the initial location of an instance. Through the motion in Eqn. FORMULA1, x i will be moved to a new position x DISPLAYFORM1 With Eqn. FORMULA5, Eqn. can be reformulated as: DISPLAYFORM2 Now DISPLAYFORM3 c is the mean vector of the instances in cluster S 1 c. Our proposed CPD based unsupervised metric learning (CPD-UML) is designed to learn a spatial transformation Ψ and a clustering S 1 at the same time. Eqn. can be reformulated into a matrix format through the following steps. First, put the input dataset into a d-by-n data matrix. Second, define a Gaussian kernel function matrix for the CPD deformation as: DISPLAYFORM4 The size of G is n-by-n. Third, let p be a vector of dimension n c -by-1 with all elements equal to one, then the mean of the data instances within a cluster S 1 c can be written as µ BID21. With these three formulations, and let E be a permutation matrix, Eqn. can be rewritten as: DISPLAYFORM5 DISPLAYFORM6 where X 1 is the transformed data matrix. Since ||A|| 2 F = trace(A T A), Eqn. FORMULA10 can be written in the form of the trace operation: DISPLAYFORM7 As trace(AB) = trace(BA), and p T p = n c, the J in Eqn. FORMULA11 can be further reformulated as: DISPLAYFORM8 Similar to BID21 ), we define a n-by-k orthonormal matrix Y as the cluster indicator matrix: DISPLAYFORM9 With X 1 = X + ΨG(X, X) and the cluster indicator matrix in Eqn. FORMULA13, Eqn. FORMULA12 can be written into the following: DISPLAYFORM10 To reduce overfitting, we add the squared Frobenius norm λ||Ψ|| 2 F = λtrace(Ψ T Ψ), to penalize any non-smoothness in the estimated transformations. λ is a regularization parameter. Finally, our nonlinear CPD-UML solution is formulated as a trace minimization problem, parameterized by Y and Ψ: DISPLAYFORM11 To search for the optimal solutions of Y and Ψ, an EM-like iterative minimization framework is adopted to update Y and Ψ alternatingly. The transformation matrix Ψ is initialized with all 0 elements, and the cluster indicator is initialized with a K-means clustering of the input data samples. Optimization for Y With Ψ fixed, Eqn. FORMULA1 reduces to a trace maximization problem: DISPLAYFORM0 Since Y is an orthonormal matrix: Y T Y = I K, the spectral relaxation technique BID21 ) can be adopted to compute the optimal Y. The solution is based on Ky Fan matrix inequalities below:Theorem. (Ky Fan) If A be a symmetric matrix with eigenvalues {λ 1 ≥ λ 2 ≥ ... ≥ λ n}. Let the corresponding eigenvectors be DISPLAYFORM1 where the optimal Y * is given by DISPLAYFORM2 This spectral relaxation solution can be regarded as a manifold learning method that projects data samples from the original d-dimensional space to a new K-dimensional space. In our case, the A matrix in Ky Fan Theorem takes the form of X T X. In implementation, we first compute the K largest eigenvectors of X T X, and then apply the traditional K-means method, under the induced K-dimensional space, to compute the cluster assignments. Optimization for Ψ With the Y generated from Eqn., Eqn. becomes a trace minimization problem w.r.t. Ψ: DISPLAYFORM3 Through a careful investigation of the gradient and Hessian matrix of Eqn. FORMULA1, we found the J could be proved to a smooth convex function, with its Hessian w.r.t. Ψ being positive definite (PD) everywhere. Therefore, the only stationary point of J, where the gradient is evaluated to 0, locates the global minimum, and provides the optimal Ψ *. The convexity proof is given as follows. Convexity proof of J w.r.t. Ψ: Firstly, we update J in Eqn., through several straightforward derivation steps (the details are given in Appendix A), to an equivalent form: DISPLAYFORM4 The gradient of J w.r.t. Ψ can then be computed as: DISPLAYFORM5 To facilitate the convexity proof, we rewrite this gradient equation as: DISPLAYFORM6 N is a matrix of size d × n. M is a symmetric matrix of size n × n, which can be proved positive definite, based on the theorem in :Theorem. "Suppose that A ∈ M m,n and B ∈ M n,m with m ≤ n. Then BA has the same eigenvalues as AB, counting multiplicity, together with an additional n − m eigenvalues equal to 0."We know Y T * Y = I K, whose eigenvalues are all 1s. Then, according to this Theorem, the eigenvalues of Y Y T are 1s (multiplicity is K), and 0 (multiplicity is n − K). In the matrix M of Eqn. FORMULA1 DISPLAYFORM7 T is a positive semidefinite matrix as it is symmetric and its eigenvalues are either 0 or 1. G is also positive definite because it is a kernel (Gram) matrix with the Gaussian kernel. With G being symmetric PD and λ setting to be a positive number in our algorithm, the matrix M is guaranteed to be a PD matrix. Expanding the gradient formulated in Eqn. to individual elements of Ψ, it can be further written as: DISPLAYFORM8 With Eqn. FORMULA1, Eqn. FORMULA1 can be resized into a vector of size d × n. Then, the Hessian matrix of J w.r.t. Ψ can be calculated as below: DISPLAYFORM9 DISPLAYFORM10 It is clear that H is a symmetric matrix with size (d * n) × (d * n). The diagonal of H is composed by d repeating M matrices. Let z be any non-zero column vector with size (d * n) × 1. To prove H is a PD matrix, we want to show that z T H z is always positive. To this end, we rewrite z as [z 1, z 2, ..., z d], where z i is the sub-column of z with size n × 1. Then z T H z can be computed as: DISPLAYFORM11 As M has been proved to be a PD matrix, each item in Eqn. FORMULA1 is positive. Therefore, the summation z T H z is also positive. Since z is an arbitrary non-zero column vector, this shows H is PD. With the Hessian matrix H being PD everywhere, the objective function J is convex w.r.t. Ψ. As a , the stationary point of J makes the unique global minimum solution Ψ *. Let Eqn. equal to 0, we get DISPLAYFORM12 The matrix M on the left is proved PD, thus invertible. The optimal solution of Ψ is given as: DISPLAYFORM13 Based on the description above, our proposed CPD-UML algorithm can be summarized as the pseudo-code below:Algorithm So far, we developed and applied our proposed CPD-UML under input feature spaces. However, it can be further kernelized to improve the clustering performance for more complicated data. A kernel principal component analysis (KPCA) based framework ) is utilized in our work. After the input data instances are projected into kernel spaces introduced by KPCA, CPD-UML can be applied under the kernel spaces to learn both deformation field and clustering , in the same manner as it is conducted under the original input spaces. We performed experiments on a synthetic dataset and six benchmark datasets. Comparisons are made with state-of-the-art unsupervised metric learning solutions. The two-moon synthetic dataset 1 was tested in the first set of experiments. It consists of two classes with 100 examples in each class. (see FIG1). All the samples were treated as unlabeled samples in the experiments. Both linear and kernel versions of our CPD-UML were tested. Linear version CPD-UML In this experiment, our CPD-UML was applied in deforming the data samples to achieve better separability under the input space. The effectiveness of our approach is demonstrated by comparing with the base algorithm K-means. The clustering of K-means and CPD-UML are shown in FIG1 (a) and 1 (b) respectively. The sample labels are distinguished using blue and red colors. The clustering are shown using the decision boundary. It is obvious that K-means cannot cluster the two-moon data well due to the data's non-separability under the input space. Our CPD-UML, on the contrary, achieves a 99% clustering accuracy by making the data samples linearly separable via space transformations. The deformation field of FIG1 in the input space is shown in FIG1 and (d). It is evident that our nonlinear metric learning model can deform feature spaces in a sophisticated yet smooth way to improve the data separability. Kernel version CPD-UML In this set of experiments, various RBF kernels were applied on the two-moon dataset to simulate linearly non-separable cases under kernel spaces. The clustering of kernel K-means with different RBF kernels (width = 4, 8, 16, 32) are shown in FIG2 (a) -2 (d). Colors and decision boundaries stand for the same meaning as those in FIG1. Obviously, the performance of kernel K-means was getting worse with sub-optimal kernels, as in 2 (b), 2 (c) and 2 (d). Searching for an optimal RBF kernel requires cross-validation among many candidates, which could in a large number of iterations. This procedure can be greatly eased by our kernel CPD-UML. The CPD transformation under kernel spaces provides a supplementary force to the kernelization to further improve the data separability, the same as it performs under the input space. FIG2 (f) -2 (h) demonstrate the effectiveness of our CPD-UML. Same RBF kernels as in FIG2 (b) -2 (d) were used, but better clustering were obtained. The ability to work with sub-optimal kernels should also be regarded as a computational advantage of our model. Experimental Setup In this section, we employ six benchmark datasets to evaluate the performance of our CPD-UML. They are five UCI datasets 2: Breast, Diabetes, Cars, Dermatology, E. Coli and the USPS_20 handwritten data. Their basic information is summarized in Appendix B.Both linear and kernel versions of our proposed approach were tested. For linear version, K-means method was used as the baseline for comparison. In addition, three unsupervised metric learning solutions, AML, RPCA-OM BID12 and FME were utilized as the competing solutions. For kernel version, the baseline algorithm is kernel K-means. NAML BID2, the kernel version of AML is adopted. Since RPCA-OM and FME do not have their kernel version, the same kernelization strategy in 2.4 was applied to kernelize these two solutions. RBF kernels were applied for all kernel solutions. Each dataset was partitioned into seen and unseen data randomly. Optimal cluster centers and parameters are determined by the seen data. Clustering performance is evaluated via the unseen data, which are labeled directly based on their distances away from the cluster centers. Similar setups have been used in BID11 BID6. In the experiments, we performed 3-fold cross validation, in which two folds were used as seen data and one fold as unseen data. In the competing solutions, the hyper-parameters were searched within the same range as in their publications. In our proposed approach, the regularization parameter λ and smooth parameter σ were searched from {10 0 ∼ 10 10} and {2 0 ∼ 2 10}, respectively. The RBF kernel width for all kernel methods is chosen from {2 −5 ∼ 2 10}. Since the performance of tested methods depends on the initialization clusters, the clustering of K-means was applied as the initialization clusters for all the competing solutions in each run. The performance of each algorithm was calculated over 20 runs. Results We measured the performance using the ground truth provided in all six benchmark datasets. Three standard performance metrics were calculated: accuracy, normalized mutual information and purity. To better compare the tested methods in statistic, we conducted a Student's t-test with a p-value 0.05 between each pair of solutions for each dataset. The solutions were ranked using a scoring schema from BID17. Compared with other methods, an algorithm scores 1 if it performs significantly better than one opponent in statistic; 0.5 if there is no significant difference, and 0 if it is worse. Tables 1, 2 and 3 summarize the clustering performance and ranking scores. The best performance is identified in Boldface for each dataset. It is evident that our CPD-UML outperforms other competing solutions in all three standard measurements with significant margins. Highest ranking scores in the performance tables are all achieved by our kernel version approach. In addition, significant improvements have been obtained by our proposed approach compared with the baseline algorithm K-means and kernel K-means. It is also noteworthy that, the linear CPD-UML achieved comparable with the other competing methods using RBF kernels, which further demonstrates the effectiveness of our nonlinear feature space transformation. The proposed CPD-UML model learns a nonlinear metric and the clusters for the given data simultaneously. The nonlinear metric is achieved by a globally smooth nonlinear transformation, which improves the separability of given data during clustering. CPD is used as the transformation model because of its capability in deforming feature space in sophisticated yet smooth manner. Evaluations on synthetic and benchmark datasets demonstrate the effectiveness of our approach. Applying the proposed approach to other computer vision and machine learning problems are in the direction of our future research.

a nonlinear unsupervised metric learning framework to boost the performance of clustering algorithms.

We propose nearest neighbor overlap, a procedure which quantifies similarity between embedders in a task-agnostic manner, and use it to compare 21 sentence embedders.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: As machine learning becomes ubiquitous, deployed systems need to be as accu- rate as they can. As a , machine learning service providers have a surging need for useful, additional training data that benefits training, without giving up all the details about the trained program. At the same time, data owners would like to trade their data for its value, without having to first give away the data itself be- fore receiving compensation. It is difficult for data providers and model providers to agree on a fair price without first revealing the data or the trained model to the other side. Escrow systems only complicate this further, adding an additional layer of trust required of both parties. Currently, data owners and model owners don’t have a fair pricing system that eliminates the need to trust a third party and training the model on the data, which 1) takes a long time to complete, 2) does not guarantee that useful data is paid valuably and that useless data isn’t, without trusting in the third party with both the model and the data. Existing improve- ments to secure the transaction focus heavily on encrypting or approximating the data, such as training on encrypted data, and variants of federated learning. As powerful as the methods appear to be, we show them to be impractical in our use case with real world assumptions for preserving privacy for the data owners when facing black-box models. Thus, a fair pricing scheme that does not rely on secure data encryption and obfuscation is needed before the exchange of data. This pa- per proposes a novel method for fair pricing using data-model efficacy techniques such as influence functions, model extraction, and model compression methods, thus enabling secure data transactions. We successfully show that without running the data through the model, one can approximate the value of the data; that is, if the data turns out redundant, the pricing is minimal, and if the data leads to proper improvement, its value is properly assessed, without placing strong assumptions on the nature of the model. Future work will be focused on establishing a system with stronger transactional security against adversarial attacks that will reveal details about the model or the data to the other party. Driven by the application of facilitating data exchange with clear consent, we focus on the most needed form of data transaction: a potential improvement to an already powerful model. A tiny model without sensible accuracy is not proven to work and is likely not deployed for important tasks. Models trained only a small scale of data are furthermore very sensitive to new data of any form, so the question of whether a new dataset brings forth improvement is easy to answer (yes!). A data transaction is the hardest and most meaningful in the setting of a robust model potentially improving with the addition of a relatively small set of data that makes the training set better mimic that of the real world i.e. deployed cases. This subsection shows that the naive approach of exchanging data upfront or paying blind upfront are both undesirable, as summarized in TAB0.In industry, because only tech giants with sufficient existing centrality gets to aggregate the most data, a convenient naive practice involves the data providers giving the data up in advance.1. An individual user using storage, social, or other valuable features may give up their data within the services for convenience. It can be argued that they are trading their data in for improved services they receive. 2. A small company that provides data to be evaluated, while trusting the auditing within the big companies to prevent useful data from not being paid. 3. Academic researchers in fields of linguistics, biology or other fields, upon hearing about the power of machine learning in industry, give their field studies data collected in decades to model owners, who tend to be entities of big corporations, for free, hoping for collaboration and interesting insights produced. The fairness in the pricing is highly questionable, as the implicit contracts get difficult to verify and reinforce. Furthermore, the data that is evaluated to be useless currently will likely still sit in the company's cluster, so while the company may decline reciprocating gifts such as academic collaboration, while using the data for some other service in the future. It is difficult for data providers to argue for a latent payment, since any data given up is given up. Improvement of services may never get delivered, and a user of a centralized service who has given up their data will have trouble telling if their data exchange was fair at all (even if their evaluation was purely psychological).A comparable approach is the other way around: a data company prices their dataset, such as phone call clips, in advance, and a model owner, such as a speech recognition service, pays for the data in full, before any evaluation is done. The fairness in the pricing is dubious, as the value of the data is not evaluated against the model. While this approach has high security guarantee for one of the involved parties, it is clearly inflexible. For one thing, in both cases, the financial incentives do not align: the model owners would rather not pay fairly for the data while the data owner would rather not price their data fairly. This is further complicated if we look into more granular fairness that relates to the data's interaction with the model: can we reason about the pricing of the data and the presumably positive effect it has on the model. For simplicity, we assume the model owner has a relatively robust large model that can use improvements. A hypothesis is made that a particular data provider may be able to improve the model. For the simplicity of a privacy argument and for the use case of user data, data here is assumed to be totally private before as far as the involved parties are concerned. That is, there is unknown information about the data that may benefit the model. DISPLAYFORM0 This ensures that the data change under MDE for a given model is similarly ranked against other data points, as with Spearman's Correlation, when compared to the ranking done directly on the model. Pricing on MDE is used to facilitate a one-time data transaction that is fair; that is, useless data is not paid much money, while useful data gets evaluated. Our technique imposes a pricing function that is applied to model approximation techniques, which is then applied on given data, as summarized in Figure 4.2. As summarized in Figure 4.2, a pricing can be put forward in advance before a transaction agreement is put forth; the data owners get paid for the data while the model owners, after paying, will have full flexibility in their usage of the data which has been estimated to be valuable. An improvement is evaluated eventually as the parameter updates that can leader to better performance on a particular test set, which we reduce to just desirable parameter updates. This choice is a deliberate one. In the next section, we discuss the usability of training on private data for the data transaction use case under our assumptions of a relatively robust but unknown model. The rest of the paper is structured to address other approaches, how practical, secure, private, and fair they are, and give details on our solution. 3 IS THERE USABLE TRAINING DATA PRIVACY?As machine learning models become more and more complicated, its capability can outweigh the privacy guarantees encryption gives us. Our approach of approximating the model, rather than using techniques to protect the data, is a calculated one: when facing black box models, encrypted training data is not so private. This section addresses the practical implications of private training data in our use case. Once a transaction of the raw data is made, the data no longer belongs to the data owner. The model owner is then free to use the data however many times in however many ways unless further restrictions are applied. It is fair to assume that from a security and privacy perspective, any data that is given up is given up forever. Under that assumption, a fair pricing scheme is needed to make sure a pricing happens before a transaction, thus giving data owners the opportunity to throughly trade the entirety of the data. If available, the test set should only be used once to maintain statistical validity, a test set is ideally only used once to prevent model improvements that are just overfitting. Similarly, our data evaluation framework affords a one-time evaluation between every pair of data and model, since repeated testing would leak a lot of contextual information. The proposed method, for instance, assumes the pricing and transaction to be both one-time activities. See the next sections for details. Note that even though giving the model owners the entirety of the data may lead to overfitting, as Numerai had tried to solve BID10, such concern is at a loss of the model owner's, and therefore not a concern of our paper, which aims to align the economic incentives of model owners and data owners. The motivation for training on encrypted data includes user data data and the desire to still improve a learned model. These ideals are tradeoffs. Specifically, for notable improvements on the model the data can demonstrate, some information about the data will need to be leaked, regardless of whether the model input was encrypted. More practically, it is well-known that popular networks on media data can memorize complete noise (but does not reveal data), as shown by BID12, making it further difficult to maintain the privacy of data for any meaningful model improvement that the model owner needs to observe, because the parameter updates alone reveal much information about the data. The following is a sketch of the proof from an information theory standpoint with real-world scales:Suppose we have a high resolution image passed to a visual network. A practical choice for an image would be a high resolution selfie, such as an 1024x1024 pixel black and white image. The network is assumed to be a smaller variant of AlexNet BID8, which has only 1 million parameters and only outputs binary updates. Note that we restrict the model capability to have a conservative estimate. Suppose every model parameter update is non-trivial, defined as greater than for some where 0 < 1. For a given input on the scale of 1MB, once encrypted and trained against an unknown model of 1 million parameters, it outputs binary parameter updates; that is, r i = 1 for updating θ i and r i = 0 for not updating θ i.As we know, the model parameters can encapsulate 1 million bits of information, so it has the capacity to model such an input by pixel. This means that there is no good information theoretical guarantee for encrypted data on a completely unknown model of arbitrary scale, especially in the use cases today. In fact, even for unencrypted data, the model owner should be incentivized to not reveal the details of its model and should instead use a proxy for the model in evaluating the efficacy of data. In practice, unrestricted raw training data is useful for model improvement. In addition, extremely restricted visibility into data can be dangerous, as in the case of training on encrypted data. A practical solution thus calls for more flexibility in data usage for the model owners. The model owners' incentives are against replacing the entire data with much less information, such as with the of feature extractions; because encrypted data is undesirable, the model owners will be less likely to employ it. Firstly, the engineering work involved each time such a filter of information is made is expensive. This adds to the cost of training on encrypted data, further disincentivizing model owners from sticking to the encryption scheme that requires regular updates. More broadly, the model owner would like to sometimes change the architecture of the model to improve it; not having the underlying training data stored makes such process hard.. If a feature extractor add one more entry, all the data would need to be extracted again. Similarly, if an image classifier's model architecture changes, all the data would need to be collected and purchased again. Even with the same model, extremely restricted data is inflexible for the model owners in improving their models. As deep learning matures as a technology, modern development techniques for a safe and robust model requires the model owners to have intimate access to data. The model developers usually require a lot of time, tweaking, test, and in general more iterations of runs with the training data in order to form a better model in the development phase. This development phase is not separable in practice: visibility into why their model works and does not work often rely on examining the data. In the case of media input, the raw data is often more human readable than their representations. In addition, not having the actual data at the disposal of model owners is undesirable and potentially unsafe. For example, adversarial training data cause misclassification BID4; autonomous driving cars that mis-classify a stop sign can potentially cause human deaths. To remedy such risks, practical model testing and debugging methods often rely on visibility into data BID7; for instance, adversarial images that cause mis-classification need to be examined BID1. Each of the testing technique require yet a new operation on the data; if we want the data to remain encrypted, each of these testing frameworks would need to be hand-crafted to accommodate. Fully homomorphic encryption, verifiable contracts and proofs, and federated learning techniques address the issue of privacy-preserving training. We talk about their privacy, practicality, and fairness tradeoffs in this section, as summarized in TAB2. In particular, we discuss the beneficial intuitions in federated learning, as summarized in TAB3. Homomorphic encryption is appealing in the privacy and security community for its power to complete verifiably correct computation without revealing the original inputs. Naively it can be applied to either the data or the model, yet the computational cost is prohibitively expensive; as BID11 show, to learn a classifier for recognizing 101 objects using 2048-dimensional deep features, a verifiable summing operation encrypted with a 1024-bit key takes about 3 ms for a single weight using a modern CPU on Mac OS X, thus expecting to take up more than 16 hours to encrypt all classifiers even in a simple case BID11. Encrypting the model is thus not practical. Encrypting the data for machine learning, while still slow, has led to many observations in the differential privacy community. For instance, some models have shown to have secure and efficient solutions, such as ridge regressionGiacomelli et al.. Here, we focus on those with strong security guarantees that are in practical use. The advances of bitcoin-related research touch on aligning financial incentives with machine learning algorithms' effectiveness using blockchain. Numerai addresses economic incentive against overfitting, while Openmined (et al., 2017) encourages a collaboration of open protocols and development that incorporate all of federated learning, homomorphic encryption, and blockchain smart contracts, which are all covered here. Advances in using blockchain smart contracts often have use cases that are diverge from our goal, making it suboptimal for security, flexibility, and pricing fairness. Security-wise, the data detail may Figure 3: OpenMined architecture, which aims to democratize data by making it easier to exchange user data. It consists of federated learning server Sonar running on the blockchain, Capsule for handling encrypted key exchange, Mine with user data, and Syft, which contains neural networks that can be trained in an encrypted state (et al., 2017).remain secret, yet it can be inferred from the model improvements. To mitigate that, a smart aggregator is designed to rely on the other users' data to mix up the updates, so that such information regarding data detail is differentially private. However, making strong assumptions about the distribution of other data not in our control is impractical, as we deal with a single transaction between a model owner and a data owner. In the case of training data transaction, we want the data to be preferably visible to maximize their flexibility. Absolute security of the data is also less relevant in the pricing case, as we hope to guard the data against its use case; that is, we aim to prevent its specific model improvements to known while still demonstrating that the model is improved by the data. Federated learning has seen great advances, without sacrificing much accuracy, while preserving privacy BID9. It potentially can be used to allow model parameter updates to be estimated before the central big model gets that information. That is, a transaction can happen between the gradient updates are passed back, and a pricing can be done based on the gradient, achieving the desired privacy guarantee for data owners. This method is an improvement on all the previous methods mentioned when it comes to privacy for data owners who can benefit a lot from the exchange. However, it is relatively unsafe for model owners, as a miniature version of the model is deployed per client, lacking true ownership, making the attack vector for knowledge about the model very large. In addition, it still requires the effect of the exact data on the exact model to be known before transaction. It further calls for a secure aggregating solution, though existing ones proposed appears to work in experimental in the case of visual learning BID11.Our approach can be seen as an improvement on federated learning scheme in that the data owners' privacy is prioritized over the model owner's model details. Federated learning often involves neural networks that can be trained in an encrypted state, in order to retain the information regarding the model, preventing it from being stolen. Thus they can be very slow. To optimize for them, the distributed model is hand-crafted, which is time consuming. This encryption guarantee is necessary for federated learning, since the architecture requires the model to be distributed per user, making the attack vector for model theft very large. Under such constraint, speeding up a neural network that is trainable in an absolutely encrypted state has little black box solution that is plug-and-play. Our method can be seen as an improvement on the federated learning scheme, utilizing the intuition that both the model and the data can be protected to achieve fairness. To secure the data, a pricing scheme is needed to reveal as little data detail to the model owners as possible before a transaction happens. The eventual system the data is tested on would output a scalar price for given data. It consists of a pricing component, a model-data efficacy middleware, and the model itself. The middleware is prepared by the model owners, who have control over the details of their model. A safer mediation is abstracting the middleware into two parts: a model approximation and an efficacy calculation layer. To model owner, the construction of MDE is under their control; for simplicity, we use a binary score. Then it suffices to have the data causing positive output be paid a uniform price while the data causing zero output to not be transacted, as they are likely useless. We are formulating the efficacy of data with respect to the model T. Such efficacy, denoted as ρ can be seen as a function that loosely maps data to a scalar. Since we care less about the cases when the data does not have value to us, including when the data is a duplication, of high similarly, leads to no parameter change, no model improvement, or even regression, we will only consider the non-negative cases. For the weak we wish to obtain: useful versus not very useful, many existing methods can be utilized. Interpretability solutions aim to alleviate the notoriety of reasonability of neural networks. Because the interpretation steps are generally faster or less resource-draining than running the data through the original model, some of these techniques are suitable for our purposes. Leaning on these, we can set up a general framework f where the model T's effect under data D is approximated: Influence functions are very flexible: it is shown that the approximation of the effect of the model works well on both seen and unseen data, making it suitable for our purposes BID0. The model access influence function needs are gradients and Hessian-vector products, effectively treating the model itself as a blackbox. Valuable are obtained in evaluating experimental data efficacy even on non-convex and non-differentiable models. The versatility of influence functions is a good direct MDE measure to be used for pricing. A similar approach that aims at interpretability of neural network models (often difficult to reason about) is model extraction BID0, which effectiveness approximates model as a decision tree. Model Extraction is also experimented to have better data efficacy approximation than just training a decision tree from the same set of data. This suggests that the extract model preserves the behavior of the original model well; that is, data that leads to improvements in model give the same inference in the extracted model as the original model would, and useless data that does not shift original model will not shift the extracted model. Leveraging that comes a fast approach of evaluating data efficacy, thus providing fair pricing approximations. Besides interpretability techniques through data, a more generic class of techniques that make the model smaller and faster (and less energy-consuming) can be utilized for pricing purposes, because they don't reveal the full model while still retaining its behavior. There are many techniques on model compression BID5 BID6, and our selection for pricing models requires the underlying model compression algorithms to take on relatively black-box approaches to maintain generality and secrecy. Model compression techniques are very powerful in reducing the resource footprint of large models while retaining the overall accuracy. Another extension to approximate data efficacy on model includes training an ensemble model that assumes sparsity assumption on models. Because this is closer to just replacing the existing model, rather than keeping it as is while shrinking it, we will leave it to future work to explore the possibilities. Some techniques are model-specific, such as model distillation for ensemble models BID6. A pricing model takes a data efficacy estimates and outputs a price. The model owners selects the desirable parameter updates. Because we have a much smaller component involved to evaluate the training data, thus encrypting it becomes a lot more practical than encrypting the whole model. Because we simplified the communication between data owners and model owners to be of a onetime scaler, the data security issue is largely simplified. Once the price is agreed upon, a verifiable transaction is trivial to implement to ensure that the same data is transacted after the money is paid. Encrypting the data or approximating the data is a lost cause for the data owner whose privacy is not guaranteed. Since the model owner has greater context on similar data distribution, they can infer much information about the data without actually seeing it. Because data cannot be practically secured without losing its value before being handed over, pricing and the transactional form relevant. In this scheme, no data is given up until money is paid. The suggested methods for Model-Data Efficacy include influence, which explores the change in parameter with training data, model extractions, which approximate a trained network with a decision tree, and model compression techniques that are learned. They all work to approximate the effect of data to the model owners without showing the exact makeup of the model. The crux of the usability of the solution lies in whether the approximation technique preserves model details, but combining secure transaction techniques is sufficient to make the approximated pricing model entirely private (beyond its output) without further approximating the effect of these pricing models, thus keeping them as accurate as the previous in the last section. Despite the potential accuracy loss, usability is much better. For any transaction reached through model approximation, we still maintain usable privacy guarantee. Securing a pricing function, which is very small, is easy. Enforcing by ways of contract to guarantee that a money-data transaction happens after agreeing on a price is much easier to enforce than contracts that bind within the large model owners? organization, such as trusting a security audit.

Represent the short summary for finding the corresponding scientific article (long)

Facing complex, black-box models, encrypting the data is not as usable as approximating the model and using it to price a potential transaction.

Incorporating the ability to say I-don't-know can improve the fairness of a classifier without sacrificing too much accuracy, and this improvement magnifies when the classifier has insight into downstream decision-making.