In this paper, we consider the problem of actively learning a linear classifier through query synthesis where the learner can construct artificial queries in order to estimate the true decision boundaries. This problem has recently gained a lot of interest in automated science and adversarial reverse engineering for which only heuristic algorithms are known. In such applications, queries can be constructed de novo to elicit information (e.g., automated science) or to evade detection with minimal cost (e.g., adversarial reverse engineering). We develop a general framework, called dimension coupling (DC), that 1) reduces a d-dimensional learning problem to d-1 low dimensional sub-problems, 2) solves each sub-problem efficiently, 3) appropriately aggregates the results and outputs a linear classifier, and 4) provides a theoretical guarantee for all possible schemes of aggregation. The proposed method is proved resilient to noise. We show that the DC framework avoids the curse of dimensionality: its computational complexity scales linearly with the dimension. Moreover, we show that the query complexity of DC is near optimal (within a constant factor of the optimum algorithm). To further support our theoretical analysis, we compare the performance of DC with the existing work. We observe that DC consistently outperforms the prior arts in terms of query complexity while often running orders of magnitude faster.
@cite_18: 
@cite_4: A process for continously annealing a fused cast refractory body, in order to obtain a crack-free product, in which the fused cast refractory body is placed on a suitable preheated carrier which has been previously heated to a temperature of greater than 500 DEG C in a preheat chamber, and is charged into an annealing tunnel which is divided into a hot zone and a cooling zone having at least one cooling section.
@cite_7: 
@cite_3: We provide new results concerning label efficient, polynomial time, passive and active learning of linear separators. We prove that active learning provides an exponential improvement over PAC (passive) learning of homogeneouslinear separators under nearly log-concave distributions. Building on this, we provide a computationally efficient PAC algorithm with optimal (up to a constant factor) sample complexity for such problems. This resolves an open question of (Long, 1995, 2003; , 2009) concerning the sample complexity of efficient PAC algorithms under the uniformdistribution in the unit ball. Moreover,it providesthe first bound for a polynomial-time PAC algorithm that is tight for an interesting infinite class of hypothesis functions under a general and natural class of data-distributions, providing significant progress towards a longstanding open question of (, 1989; , 1989). We also provide new bounds for active and passive learning in the case that the data might not be linearly separable, both in the agnostic case and and underthe Tsybakovlow-noisecondition. To derive our results, we provide new structural results for (nearly) log-concave distributions, which might be of independent interest as well.
@cite_6: We state and analyze the first active learning algorithm which works in the presence of arbitrary forms of noise. The algorithm, A2 (for Agnostic Active), relies only upon the assumption that the samples are drawn i.i.d. from a fixed distribution. We show that A2 achieves an exponential improvement (i.e., requires only O (ln 1 e) samples to find an e-optimal classifier) over the usual sample complexity of supervised learning, for several settings considered before in the realizable case. These include learning threshold classifiers and learning homogeneous linear separators with respect to an input distribution which is uniform over the unit sphere.
@cite_24: The invention is an improved apparatus for the removal of sodium carbonate from cyanide plating baths. The method involves the precipitation of sodium carbonate by the effect of cooling the cyanide plating bath liquid. A container is submerged in the solution with the container opening extending above the plating bath solution level. The container is filled with dry ice and water which produces a temperature of approximately zero degrees centigrade inside of the container. The solution adjacent to the exterior of the container is cooled so that excess sodium carbonate is precipitated as a crystalline deposit. After a desired interval, the container is removed with the encrustation of sodium carbonate for disposal.
@cite_5: We present a framework for margin based active learning of linear separators. We instantiate it for a few important cases, some of which have been previously considered in the literature.We analyze the effectiveness of our framework both in the realizable case and in a specific noisy setting related to the Tsybakov small noise condition.
@cite_16: We start by showing that in an active learning setting, the Perceptron algorithm needs Ω(1 ∈ 2 ) labels to learn linear separators within generalization error ∈. We then present a simple selective sampling algorithm for this problem, which combines a modification of the perceptron update with an adaptive filtering rule for deciding which points to query. For data distributed uniformly over the unit sphere, we show that our algorithm reaches generalization error e after asking for just O(d log 1 ∈) labels. This exponential improvement over the usual sample complexity of supervised learning has previously been demonstrated only for the computationally more complex query-by-committee algorithm.
@cite_10: The ability to cheaply train text classifiers is critical to their use in information retrieval, content analysis, natural language processing, and other tasks involving data which is partly or fully textual. An algorithm for sequential sampling during machine learning of statistical classifiers was developed and tested on a newswire text categorization task. This method, which we call uncertainty sampling, reduced by as much as 500-fold the amount of training data that would have to be manually classified to achieve a given level of effectiveness.
@cite_11: 
One very successful approach that does not suffer from the aforementioned computational challenge is pool-based active learning @cite_11 , where instead of ideally halving the space, effective approaximations are performed. Notable algorithms are @cite_10 and query-by-committee (QBC) @cite_4 . In fact, our problem is closely related to learning homogeneous linear separators under the uniform distribution in the pool-based setting. This problem is very well understood and there exist efficient pool-based algorithms @cite_5 @cite_16 @cite_24 . In particular, @cite_7 presented an efficient perceptron-based algorithm that achieve a near-optimal query complexity. Similar results can be obtained under log-concave distributions @cite_3 . Most of the pool-based methods require to have access to @math number of unlabeled samples in each iteration or otherwise they perform very poorly @cite_5 @cite_7 . This means that in order to have exponential guarantee in terms of sample complexity, we need to grow the pool size exponentially fast (note that there is no need to store all of these points). Moreover, with a few exceptions @cite_18 @cite_6 pool-based learning of linear separators in the noisy setting has been much less studied and the dependency of sample complexity on noise is not very well understood.