Question: represent the text to find a very short summary (1-2 sentences max)

Question: We propose a new application of embedding techniques to problem retrieval in adaptive tutoring. The objective is to retrieve problems similar in mathematical concepts. There are two challenges: First, like sentences, problems helpful to tutoring are never exactly the same in terms of the underlying concepts. Instead, good problems mix concepts in innovative ways, while still displaying continuity in their relationships. Second, it is difficult for humans to determine a similarity score consistent across a large enough training set. We propose a hierarchical problem embedding algorithm, called Prob2Vec, that consists of an abstraction and an embedding step. Prob2Vec achieves 96.88\% accuracy on a problem similarity test, in contrast to 75\% from directly applying state-of-the-art sentence embedding methods. It is surprising that Prob2Vec is able to distinguish very fine-grained differences among problems, an ability humans need time and effort to acquire. In addition, the sub-problem of concept labeling with imbalanced training data set is interesting in its own right. It is a multi-label problem suffering from dimensionality explosion, which we propose ways to ameliorate. We propose the novel negative pre-training algorithm that dramatically reduces false negative and positive ratios for classification, using an imbalanced training data set. The traditional teaching methods that are widely used at universities for science, technology, engineering, and mathematic (STEM) courses do not take different abilities of learners into account. Instead, they provide learners with a fixed set of textbooks and homework problems. This ignorance of learners' prior knowledge, pace of learning, various preferences, and learning goals in current education system can cause tremendous pain and discouragement for those who do not keep pace with this inefficient system BID6; BID5; BID7; BID18; BID43. Hence, e-learning methods are given considerable attention in an effort to personalize the learning process by providing learners with optimal and adaptive curriculum sequences. Over the years, many web-based tools have emerged to adaptively recommend problems to learners based on courseware difficulty. These tools tune the difficulty level of the recommended problems for learners and push them to learn by gradually increasing the difficulty level of recommended problems on a specific concept. The downside of such methods is that they do not take the concept continuity and mixture of concepts into account, but focus on the difficulty level of single concepts. Note that a learner who knows every individual concept does not necessarily have the ability to bring all of them together for solving a realistic problem on a mixture of concepts. As a , the recommender system needs to know similarity/dissimilarity of problems with mixture of concepts to respond to learners' performance more effectively as described in the next paragraph, which is something that is missing in the literature and needs more attention. Since it is difficult for humans to determine a similarity score consistent across a large enough training set, it is not feasible to simply apply supervised methods to learn a similarity score for problems. In order to take difficulty, continuity, and mixture of concepts into account for similarity score used in a personalized problem recommender system in an adaptive practice, we propose to use a proper numerical representation of problems on mixture of concepts equipped with a similarity measure. By virtue of vector representations for a set of problems on both single and mixture of concepts (problem embedding) that capture similarity of problems, learners' performance on a problem can be projected onto other problems. As we see in this paper, creating a proper problem representation that captures mathematical similarity of problems is a challenging task, where baseline text representation methods and their refined versions fail to work. Although the state-of-the-art methods for phrase/sentence/paragraph representation are doing a great job for general purposes, their shortcoming in our application is that they take lexical and semantic similarity of words into account, which is totally invalid when dealing with text related to math or any other special topic. The words or even subject-related keywords of problems are not completely informative and cannot contribute to embedding of math problems on their own; as a , the similarity of two problems is not highly correlated with the wording of the problems. Hence, baseline methods perform poorly on the problem similarity detection test in problem recommender application. We find that instead of words or even subject-related keywords, conceptual ideas behind the problems determine their identity. The conceptual particles (concepts) of problems are mostly not directly mentioned in problem wording, but there can be footprints of them in problems. Since problem wording does not capture the similarity of problems, we propose an alternative hierarchical approach called Prob2Vec consisting of an abstraction and an embedding step. The abstraction step projects a problem to a set of concepts. The idea is that there exists a concept space with a reasonable dimension N, with N ranging from tens to a hundred, that can represent a much larger variety of problems of order O(2 N). Each variety can be sparsely inhabited, with some concept combination having only one problem. This is because making problems is itself a creative process: The more innovative a problem is, the less likely it has exactly the same concept combination as another problem. The explicit representation of problems using concepts also enables state-dependent similarity computation, which we will explore in future work. The embedding step constructs a vector representation of the problems based on concept cooccurrence. Like sentence embedding, not only does it capture the common concepts between problems, but also the continuity among concepts. The proposed Prob2Vec algorithm achieves 96.88% accuracy on a problem similarity test, where human experts are asked to label the relative similarity among each triplet of problems. In contrast, the best of the existing methods, which directly applies sentence embedding, achieves 75% accuracy. It is surprising that Prob2Vec is able to distinguish very fine-grained differences among problems, as the problems in some triplets are highly similar to each other, and only humans with extensive training in the subject are able to identify their relative order of similarity. The problem embedding obtained from Prob2Vec is being used in the recommender system of an e-learning tool for an undergraduate probability course for four semesters with successful on hundreds of students, specially benefiting minorities who tend to be more isolated in the current education system. In addition, the sub-problem of concept labeling in the abstraction step is interesting in its own right. It is a multi-label problem suffering from dimensionality explosion, as there can be as many as 2 N problem types. This in two challenges: First, there are very few problems for some types, hence a direct classification on 2 N classes suffers from a severe lack of data. Second, per-concept classification suffers from imbalance of training samples and needs a very small per-concept false positive in order to achieve a reasonable per-problem false positive. We propose pre-training of the neural network with negative samples (negative pre-training) that beats a similar idea to oneshot learning BID15, where the neural network is pre-trained on classification of other concepts to have a warm start on classification of the concept of interest (transfer learning). Embedding applications: the success of simple and low-cost word embedding technique using the well-known neural network (NN) based word embedding method, Word2Vec by BID27 a), compared to expensive natural language processing methods, has motivated researchers to use embedding methods for many other areas. As examples, Doc2Vec , Paper2Vec BID41; BID16 ), Gene2Vec Cox, Graph2Vec Narayanan et al. (2017, Like2Vec, Follower2Vec and many more share the same techniques originally proposed for word embedding with modifications based on their domains, e.g. see BID16 BID23 . In this work, we propose a new application for problem embedding for personalized problem recommendation. Word/Phrase/Sentence/Paragraph embedding: the prior works on word embedding include learning a distributed representation for words by BID2, multi-task training of a convolutional neural network using weight-sharing by BID10, the continuous Skip-gram model by BID27, and the low rank representation of word-word co-occurrence matrix by BID12 ; BID33 . Previous works on phrase/sentence/paragraph embedding include vector composition that is operationalized in terms of additive and multiplicative functions by BID28 BID29 ; BID3, uniform averaging for short phrases by BID27, supervised and unsupervised recursive autoencoder defined on syntactic trees by BID38, training an encoderdecoder model to reconstruct surrounding sentences of an encoded passage by BID19, modeling sentences by long short-term memory (LSTM) neural network by BID40 and convolutional neural networks by BID4, and the weighted average of the words in addition to a modification using PCA/SVD by BID0. The simple-tough-to-beat method proposed by BID0 beats all the previous methods and is the baseline in text embedding. The rest of the paper is outlined as follows. The description of the data set for which we do problem embedding in addition to our proposed Prob2Vec method for problem embedding are presented in section 2. Negative pre-training for NN-based concept extraction is proposed in section 3. Section 4 describes the setup for similarity detection test for problem embedding, evaluates the performance of our proposed Prob2Vec method versus baselines, and presents the on the negative pre-training method. Section 5 concludes the paper with a discussion on opportunities for future work. Consider a set of M problems P = {P 1, P 2, · · ·, P M} for an undergraduate probability course, where each problem can be on a single or mixture of concepts among the set of all N concepts C = {C 1, C 2, · · ·, C N}. Note that these concepts are different from keywords of problem wordings and are not originally included in problems, but labeling problems with concepts is a contribution of this work that is proposed for achieving a proper problem representation. Instead, problems are made of words from the set DISPLAYFORM0 |Wi|. In the following subsection, we propose the Prob2Vec method for problem embedding that uses an automated rule-based concept extractor, which relieves reliance on human labeling and annotation for problem concepts. As shown in section 4, using the set of words DISPLAYFORM0 or even a subset of keywords to represent problems, text embedding baselines fail to achieve high accuracy in similarity detection task for triplets of problems. In the keyword-based approach, all redundant words of problems are ignored and the subject-related and informative words such as binomial, random variable, etc., are kept. However, since the conceptual ideas behind problems are not necessarily mapped into normal and mathematical words used in problems, even the keyword-based approach fails to work well in the similarity detection task that is explained in section 4. Alternatively, we propose a hierarchical method consisting of abstraction and embedding steps that generates a precise embedding for problems that is completely automated. The block diagram of the proposed Prob2Vec method is depicted in figure 1.(i) Abstraction step: similarity among mathematical problems is not captured by the wording of problems; instead, it is determined by the abstraction of the problems. Learners who have difficulty solving mathematical problems mostly lack the ability to do abstraction and relate problems with appropriate concepts. Instead, they try to remember procedure-based rules to fit problems in them and use their memory to solve them, which does not apply to solving hard problems on mixture of concepts. We observe the same pattern in problem representation; i.e. problem statements do not necessarily determine their identity, instead abstraction of problems by mapping them into representative concepts moves problem embedding from lexical similarity to conceptual similarity. The concepts of a problem are not mostly mentioned directly in its text, but there can be footmarks of concepts in problems. A professor and two experienced teaching assistants are asked to formulate rule-based mappings from footmarks to concepts for automation of concept extraction. As an example, the rule for labeling a problem with concept "nchoosek" is \\\\choose|\\\\binom|\\\\frac\{\s*\w+\!\s*\}\{\s*\w+\!\s*\\\\times\s*\w+\!\s*\}. By applying the rule-based concept extractor to problems, we have a new representation for problems in concept space instead of word space; i.e. (ii) Embedding step: a method similar to Skip-gram in Word2Vec is used for concept embedding. The high-level insight of Skip-gram is that a neural network with a single hidden layer is trained, where its output relates to how likely it is to have each concept co-occurred in a problem with the input concept. As an example, if concept "law-of-total-probability" of a problem is input of the neural network, we expect the neural network to state it more likely to have concept "conditional-probability" in the problem than for unrelated concepts like "Poisson-process". However, the neural network is not used for this task, but the goal is to use weights of the hidden layer for embedding. Recall the set of all concepts as {C 1, C 2, · · ·, C N}, where a problem is typically labeled with a few of them. Consider onehot coding from concepts to real-valued vectors of size N that are used for training of the neural network, where the element corresponding to a concept is one and all other elements are zero. We consider 10 neurons in hidden layer with no activation functions (so the embedded concept vectors have 10 features) and N neurons in the output that form a softmax regression classifier. In order to clarify on input-output pair of neural network, assume a problem that has a set of concepts {C 1, C 2, C 5}. The neural network is trained on all pairs DISPLAYFORM1 DISPLAYFORM2, and (C 5, C 2), where the one-hot code of the first element of a pair is the input and the one-hot code of the second element of a pair is the output of the neural network in the training phase. Hence, the neural network is trained over DISPLAYFORM3 number of training data. This way, the neural network learns the statistic from the number of times that a pair is fed into it (the neural network is probably fed with more training pairs of ("law-of-total-probability", "conditional-probability") than the pair ("law-of-total-probability", "Poisson-process")). Note that during training phase, input and output are one-hot vectors representing the input and output concepts, but after training when using the neural network, given a one-hot input vector, the output is a probability distribution on the set of all concepts. Finally, since input concepts are coded as one-hot codes, rows of hidden layer weight matrix, which is of size N by 10, are concept vectors (concept embedding) which we are really after. Denoting embedding of concept c ∈ C by E(c), problem embedding denoted by E i for problem P i is obtained as follows: DISPLAYFORM4 where f c is frequency of concept c ∈ C in our data set of M = 635 problems. Concept embedding is scaled by its corresponding frequency to penalize concepts with high frequency that are not as informative as low frequency ones. For example, concept "pmf" is less informative than concept "ML-parameter-estimation". Given problem embedding, similarity between two problems is defined as cosine of the angle between corresponding vector representations. Remark. We choose rule-based concept extractor for the abstraction step over any supervised/unsupervised classification methods for concept extraction because of two main reasons. First, there is a limited number of problems for most concepts as few as a couple of problems which makes any supervised classification method inapplicable due to lack of training data. Second, there are N concepts, so potentially there can be 2 N − 1 categories of problems which makes classification challenging for any supervised or unsupervised methods. Consider the maximum number of concepts in a problem to be k, then there are on the order of O(N k) categories of problems. Even if we consider possessing acceptable number of problems for each of the O(N k) categories, false positive needs to be on the order of O(1 N k) so that the number of false positives for each category be in the order of O. Given that there are N = 96 concepts, utilizing a supervised or unsupervised approach to achieve such a low false positive is not feasible. Exploiting the rule-based classifier for problem labeling though, we succeed to achieve as low as average 0.98% false positive and average 9.17% false negative for all concepts, where problem concepts annotated by experts are considered to be ground truth. Although 100.00% accuracy is observed in similarity detection test when utilizing problem concepts annotated by experts, those concepts are not necessarily the global optimum ground truth or the only one. Thinking of increasing accuracy in similarity detection task as an optimization problem, there is not necessarily a unique global optimum for problem concepts that lead to good performance. Hence, not having a very low false negative for rule-based does not necessarily mean that such labels are not close to a local/global optimum problem concepts. In fact, rule-based extracted concepts achieve a high 96.88% accuracy on a similarity test as is mentioned in section 4. For the purpose of problem embedding, Prob2Vec discussed in section 2.1 with a rule-based concept extractor has an acceptable performance. Here, a NN-based concept extractor is proposed that can be a complement to the rule-based version, but we mainly study it to propose our novel negative pre-training method for reducing false negative and positive ratios for concept extraction with an imbalanced training data set. Negative pre-training outperforms a similar method to one-shot learning (transfer learning) as data level classification algorithms to tackle imbalanced training data sets. The setup for concept extractor using neural networks without any snooping of human knowledge is presented first, then we propose some tricks for reducing false negative and positive ratios. The neural network used for concept extraction has an embedding layer with linear perceptrons followed by two layers of perceptrons with sigmoid activation function, and an output layer with a single perceptron with sigmoid classifier. For common words in Glove and our data set, embedding layer is initialized by Glove BID33, but for words in our data set that are not in Glove, the weights are initialized according to a uniform distribution over [−1, 1]. The embedding size is considered to be 300 and each of the other two layers have 60 perceptrons, followed by output which has a single perceptron, indicating if a concept is in the input problem or not. Note that for each concept, a separate neural network is trained. The issue with training a single neural network for all concepts is the imbalanced number of positive and negative samples for each concept. A concept is only present in a few of M = 635 problems, so having a single neural network, too many negative samples for each concept are fed into it, dramatically increasing false negatives. In the following, some tricks used for training of the above naive NN-based concept extractor are presented that reduce FN and FP ratios by at least 43.67% and up to 76.51% compared to using down sampling, which is a standard approach for training on imbalanced training data sets. The main challenge in obtaining low FN and FP is that as few as 12 problems are labeled by a concept, which makes training of neural network challenging by having 12 positive and 623 negative samples.(a) Negative pre-training: few of M = 635 problems are labeled with a specific concept, e.g. 12 problems have concept "hypothesis-MAP". Hence, few positive samples and many negative samples are provided in our data set for training of a NN-based concept extractor for a specific concept. A neural network cannot obviously be trained on an imbalanced set where all negative samples are mixed with few positive ones or FN increases dramatically. Instead, we propose two phases of training for concept C i. Consider P i = {P j : C i ∈ C j} and N i = {P j : DISPLAYFORM0 In the first phase, neural network is pre-trained on a pure set of negative samples, N i \ N i, where the trained neural network is used as a warm start for the second phase. In the second phase of training, neural network is trained on a balanced mixture of positive and negative samples, P i ∪ N i . Utilizing negative pre-training, we take advantage of negative samples in training, and not only does FN not increase, but we get an overall lower FN and FP compared to down sampling. Due to curse of dimensionality, neural network learns a good portion of the structure of negative samples in the first phase of negative pre-training that provides us with a warm start for the second phase.(b) One-shot learning (transfer learning), BID15: in the first phase of training, the neural network is first trained on classification of bags of problems with equal number of negative and positive samples of concepts that are not of interest, P j ∪ N j j =i. Then, the trained neural network is used as a warm start in the second training phase for classification of the concept of interest on a balanced set, P i ∪ N i.(c) Word selection: due to limited number of positive training samples, a neural network cannot tune any number of parameters to find important features of problems. Moreover, as a rule of thumb, the fewer parameters the neural network has, the less it is vulnerable to over-fitting and the faster it can converge to an acceptable classifier. To this end, an expert TA is asked to select informative words out of total 2242 words that are originally used for input of neural network, where this process took less than an hour. The redundant words in problems are omitted and only those among 208 selected words related to probability are kept in each problem, which reduces size of embedding matrix from 2242 × 300 to 208 × 300 and inputs more informative features to neural network. In section 4, it is shown that FP and FN ratios are reduced under this trick by at least 25.33% and up to 61.34%, which is an indication that selected words are more representative than the original ones. These selected words have been used in problem embedding for modified versions of baselines in section 4 as evidence that even keyword-based versions of embedding baselines do not capture similarity of problems. For evaluation of different problem embedding methods, a ground truth on similarity of problems is needed. To this end, four TAs are asked to select random triplets of problems, say (A, B, C) ∈ P 3 with A = B = C, and order them so that problem A is more similar to B than C; i.e. if the similarity between two problems is denoted by sim(., .), we have sim(A, B) > sim(A, C). Finally, a head TA brings into a consensus and chooses 64 triplets of problems. Note that the set of all M = 635 problems are divided into 26 modules, where each module is on a specific topic, e.g. hypothesis testing, central limit theorem, and so on. The three problems of a triplet are determined to be in the same module, so they are already on the same topic that makes similarity detection task challenging. As evidence, the similarity gap histogram for the 64 triplets of problems, sim(A, B) − sim(A, C), according to expert annotation for problem concepts and Skip-gram-based problem embedding, is shown in FIG2. It should be noted that the mentioned problem embedding is empirically proven to have the highest accuracy of 100.00% in our similarity detection test. The expert annotation for problem concepts are done by an experienced TA unaware of problem embedding project, so no bias is brought to concept labeling process. The similarity gap histogram depicts it well that similarity detection for the 64 triplets of problems is challenging due to skewedness of triplets in first bins. Prob2Vec is compared with different baseline text embedding methods in terms of the accuracy in determining the more similar problem of the second and third to the first one of a triplet. The experimental are reported in table 1. The baselines are mainly categorized into three groups as 1-Glove-based problem embedding that is derived by taking the uniform (or weighted with word frequencies) average of Glove word embedding, where the average can be taken over all words of a problem or some representative words of the problem. 2- BID0 suggest to remove the first singular vector from Glove-based problem embedding, where that singular vector corresponds to syntactic information and common words. 3-SVD-based problem embedding that has the same hierarchical approach as Prob2Vec, but concept embedding in the second step is done based on SVD decomposition of the concept co-occurrence matrix BID21. The details on baseline methods can be found in appendix A. The number of errors that the method with the best performance in each of the above categories makes in different bins of the similarity gap are shown in figure 2. For example, there are 20 triplets with similarity gap in the range [0.01, 0.21] and the best Glove-based method makes six errors out of these 20 triplets in the similarity detection test. According to table 1, the best Glove-based method is taking uniform average of embedding of selected words. Interesting patterns on concept continuity and similarity are observed from Prob2Vec concept embedding where two of them are shown in table 2. As other examples, it is observed that the most similar concept to function-RV is CDF, where function-RV refers to finding the distribution of a function of a random variable. As a TA of probability course for three years, most students do not have any clues where to start on problems for function-RV, and we always tell them to start with finding CDF of the function of random variable. It is worthy to see that NN-based concept embedding can capture such relation between concepts in seconds with small number of training samples while a human that is trained over the whole semester at university is mostly clueless where to start. We further observe the ML-parameter-E concept to be most related to the concept differentiation, where ML-parameter-E refers to maximum likelihood (ML) parameter estimation. Again, students do not get this relation for a while and they need a lot of training to get the association of MLparameter-E with differentiation of likelihood of observation to find ML parameter estimation. As another example, Bayes-formula is most similar to law-of-total-probability and the list goes on. Table 3: False negative and positive ratios of NN-based concept extraction for 1: "sample-space", 2: "Covariance", 3: "joint-Gaussian", 4: "hypothesis-MAP", 5: "hypothesis-ML".Comparison of negative pre-training, one-shot learning, word selection, down sampling, and combination of these methods applied to training process of the NN-based concept extractor is presented in table 3 for five concepts. In order to find the empirical false negative and positive ratios for each combination of methods in table 3, training and cross validation are done for 100 rounds on different random training and test samples, and false negative and positive ratios are averaged in the 100 rounds. As an example, the set N i ⊂ N i and test set are randomly selected in each of the 100 rounds for negative pre-training method, then the false negative and positive ratios of the trained neural network on the 100 instances are averaged. Employing the combination of word selection and negative pre-training reduces false negative and positive ratios by at least 43.67% and up to 76.51% compared to the naive down sampling method. For some concepts, the combination of word selection, one-shot learning, and negative pre-training in a slightly lower false negative and positive ratios than the combination of word selection and negative pre-training. However, investigating the whole table, one finds out that word selection and negative pre-training are the causes for reducing false negative and positive ratios. It is of interest that NN-based approach can reduce FN for the concept "event" to 5.11% with FP of 6.06%, where rule-based has FN of 35.71% with FP of 5.31%. A hierarchical embedding method called Prob2Vec for subject specific text is proposed in this paper. Prob2Vec is empirically proved to outperform baselines by more than 20% in a properly validated similarity detection test on triplets of problems. The Prob2Vec embedding vectors for problems are being used in the recommender system of an e-learning tool for an undergraduate probability course for four semesters. We also propose negative pre-training for training with imbalanced data sets to decrease false negatives and positives. As future work, we plan on using graphical models along with problem embedding vectors to more precisely evaluate the strengths and weaknesses of students on single and mixture of concepts to do problem recommendation in a more effective way.one of popular methods, E w: w ∈ W, where we use Glove, problem embedding for P i that is denoted by E i is computed as follows: DISPLAYFORM0 where u is the first principle component of E i: 1 ≤ i ≤ M and a is a hyper-parameter which is claimed to in best performance when a = 10 −3 to a = 10 −4. We tried different values for a inside this interval and out of it and found a = 10 −5 and a = 10 −3 to best work for our data set when using all words and a = 2 × 10 −2 to best work for when using selected words. (iii) 3-SVD: using the same hierarchical approach as Prob2Vec, concept embedding in the second step can be done with an SVD-based method instead of the Skip-gram method as follows. Recall that the concept dictionary is denoted by {C 1, C 2, · · ·, C N}, where each problem is labeled with a subset of these concepts. Let N c (C i, C j) for i = j denote number of cooccurrences of concepts C i and C j in problems of data set; i.e. there are N c (C i, C j) number of problems that are labeled with both C i and C j. The co-occurrence matrix is formed as follows: The SVD decomposition of the P P M I matrix is as P P M I = U SV, where U, S, V ∈ R N ×N, and S is a diagonal matrix. Denote embedding size of concepts by d ≤ N, and let U d be the first d columns of matrix U, S d be a diagonal matrix with the first d diagonal elements of diagonal matrix S, and V d be the first d rows of matrix V. The followings are different variants of SVD-based concept embedding BID21: DISPLAYFORM1 • eig: embedding of N concepts are given by N rows of matrix U d that are of embedding length d. • sub: N rows of U d S d are embedding of N concepts.• shifted: the P P M I matrix is defined in a slightly different way in this variant as follows: Note that the P P M I matrix is not necessarily symmetric in this case. By deriving U d and S d matrices as before, embedding of N concepts are given by N rows of U d S d.

represent the tldr for finding the corresponding scientific article

For instance, <<Bayesian optimization based online hyperparameter optimization.>>  to <<Hyperparameter tuning is arguably the most important ingredient for obtaining state of art performance in deep networks. We focus on hyperparameters that are related to the optimization algorithm, e.g. learning rates, which have a large impact on the training speed and the ing accuracy. Typically, fixed learning rate schedules are employed during training. We propose Hyperdyn a dynamic hyperparameter optimization method that selects new learning rates on the fly at the end of each epoch. Our explore-exploit framework combines Bayesian optimization (BO) with a rejection strategy, based on a simple probabilistic wait and watch test. We obtain state of art accuracy on CIFAR and Imagenet datasets, but with significantly faster training, when compared with the best manually tuned networks. Hyperparameter tuning is arguably the most important ingredient for obtaining state of art performance in deep neural networks. Currently, most networks are manually tuned after extensive trial and error, and this is derisively referred to as graduate student descent. Hyperparameter optimization (HPO), on the other hand, attempts to automate the entire process and remove the need for human intervention. Previous works on HPO propose various strategies to either adaptively select good configurations BID12 or to speed up configuration evaluations BID7.One drawback behind existing HPO frameworks is that they do not distinguish between different kinds of hyperparameters and treat them all uniformly. Broadly there are two categories of hyperparameters: those that are fixed throughout the training process and those that need to be varied during training. The former class is mostly structural (e.g. network depth and width), while the latter is mostly related to the optimization algorithm (e.g. learning rates, regularization). The two classes of hyperparameters have very different behaviors. For structural hyperparameters, online evaluation is not possible, and we need to wait for the entire training process to be completed, which is very expensive. On the other hand, for time-varying hyperparameters, it is possible to select parameters on the fly without waiting for training to finish. In this work we keep the structural hyperparameters fixed and focus on optimizing the time-varying hyperparameters to improve efficiency, stability, and accuracy of the training process. Our main contributions are as follows: We propose Hyperdyn an automated approach for dynamically tuning hyperparameters during the training process, based on the past observations. It selects new hyperparameters at the end of each epoch by combining Bayesian optimization (BO) machinery with a simple rejection strategy. It is computationally efficient since it uses Gaussian processes (GP) and simple probabilistic rejection tests. We show state of art performance on image classification benchmarks that match the accuracy of manually tuned networks while significantly improving the training speed. We demonstrate that Hyperdyn is able to automatically decide regions of start, acceleration and slow down for the training process, and can also adapt to different conditions such as batch sizes, network architecture, datasets etc. Although our framework is broadly applicable for any time-varying hyperparameter, we limit ourselves to selecting learning rates in the experiments. We now describe Hyperdyn in this context. A set of learning-rate proposals is randomly initialized along with the weights of the neural network. We choose the best learning rate based on the validation accuracy at the end of the first epoch. For Method 85% 90% 95% CIFAR 10 SGD 1.5x 2x 2x ADAM 1x 1x 2x Imagenet SGD 1.5x 4x 4x Table 1: Speed up in training (in terms of no. of iterations) over manually tuned to reach x% of the best reported top-1 validation accuracy subsequent epochs, we employ standard Bayesian optimization (BO) to obtain new proposals based on the Gaussian process framework BID11. However, we do not always accept the outcomes of BO. We design a simple probabilistic wait and watch test to decide whether to accept the BO outcome or to stick to the previously chosen learning rate, based on the improvement of the validation accuracy over the past few epochs. This rejection test is very crucial for obtaining good performance. Our experiments show that if we naively switch the learning rate to the BO output at the end of each epoch, we have training instability and bad generalization performance. This rejection framework is philosophically similar to the hyperband framework BID7 where more time is spent exploring the more promising choices. Here we require a more sophisticated framework that utilizes the temporal history to assess whether the current choice of learning rate is promising or if one should switch to a new learning rate, as proposed by BO.We investigate performance of Hyperdyn for tuning the learning rates of two most popular optimization algorithms, viz., stochastic gradient descent (SGD) and Adam BID5, on CIFAR-10 and Imagenet datasets. The are summarized in Table. 1. Our method uniformly trains faster and can quickly reach to a significant % of the best validation accuracy, which was previously obtained BID4 ) after extensive manual tuning. In Section 4, we also show that Hyperdyn outperforms other strong baselines such as epoch-based BO (i.e. no rejection) and random 5x, i.e., at every epoch we invest 5x resources more in random search than Hyperdyn. Furthermore, we find that our method is stable and trains quickly even under larger batch sizes. We used a batch size of 1000 for our Imagenet experiments, while the manually tuned baseline was on a much smaller batch size of 256. Larger batches are preferred for distributed training since they reduce the relative communication overhead. However, training on larger batches is generally challenging, and can suffer from poor generalization BID8. The adaptivity of Hyperdyn allows it to overcome this challenge. We conduct detailed empirical analysis of Hyperdyn under a variety of conditions. We find that the learning rates suggested by Hyperdyn for SGD eventually decay, but are not always monotonic in the beginning. This agrees with the previous that using more adaptive algorithms such as Adam is more beneficial in the beginning than at a later stage BID16. We find that learning rates chosen by Hyperdyn generally increase with the batch size, and this rule of thumb has been used for manual tuning BID8. We also verify that the learning rates suggested by Hyperdyn for Adam eventually converge to values that guarantee theoretical convergence. Further, we observe that SGD tuned with Hyperdyn outperforms more sophisticated algorithms (e.g. ones with momentum) that are manually tuned. This suggests the importance of tuning for good learning rates, compared to having more sophisticated optimization algorithms. Bayesian Optimization has been widely used to optimize blackbox functions. The most common frameworks use Gaussian processes (GP) for efficiently selecting good configurations BID11. Recently, BID7 introduced Hyperband, which instead focused on speeding up configuration evaluations based on a simple random search strategy. A key feature of Hyperband is that it adaptively allocates resources using a principled early stopping mechanism. In the context of tuning for learning rates, these methods have been previously employed to tune the schedule of learning rate decay, while keeping the initial learning rate fixed, BID12 ). However, this does not provide full flexibility in finding the best learning rates in each epoch. Moreover, these frameworks require for training to be completed in order to carry out their evaluations, which makes them expensive. On the other hand, Hyperdyn selects new configurations at the end of each epoch, without requiring for training to finish. Also, most previous works only compare across different HPO frameworks, but not with the state of art manually tuned networks. Previous works report around 80% validation accuracy for the hyperband algorithm on CIFAR-10 and worse for other methods such as SMAC and TPE (tree-based BO) and Spearmint (Gaussian process) BID7. None of these previous works report on Imagenet-like large datasets. For the task of finding good learning rates, other methods have been proposed that do not rely on hyperparameter optimization. One strategy is to incorporate more adaptivity into the optimization algorithm, as seen in Adam BID5, which has better performance in certain cases over SGD, but requires even more hyperparameters to be tuned. Another algorithm that automatically controls the learning rate of SGD, known as SALERA, was introduced in BID10. SALERA updates the learning rate by using an exponential moving average and deals with catastrophic events that occur in SGD (that lead to poor training accuracy) by allowing for backtracking. However, the learning rate update rule is fixed unless there is a catastrophic event, so the extent of adaptivity is limited. Additionally, the experiments were only conducted on smaller datasets such as MNIST and CIFAR10, so it is not clear how the algorithm behaves with larger datasets and larger batch sizes. An RL-based approach to learning on the fly was introduced in BID6. However, the RL framework in general requires a large amount of training data and the experiments in that work were not conducted on standard benchmark datasets such as CIFAR10 or MNIST. Recently, BID2, BID17 proposed methods for large batch training on Imagenet dataset that achieve state of art Top-1% accuracy. The work there, however, was restricted to designing a learning rate scheduler for SGD for large batch sizes. Hyperdyn on the other hand is a general framework which can be applied to a wide range of hyperparameter optimization problems and different kinds of training methods. The idea of using information learned in one task to improve on a future task, or meta-learning BID15 ), is also related to the framework of using past information to improve hyperparameter choice, considered here. LSTMs were employed to learn the gradient descent updates in BID0; BID9. However, LSTMs require a large amount training data and it is not clear if the methods can scale to standard benchmark datasets. Currently, these techniques have been shown to work for only small scale problems. The rest of paper is structured as follows -in section 2 we give a brief review of the bayesian optimization algorithm and gaussian processes that are at the center of Hyperdyn. In section 3 we describe its in detail. In section 4 we present the details of the experiments. Bayesian optimization has been widely used to find the minimize a black box function. In general, the black box functions are unknown or difficult to compute and the bayesian optimization algorithm suggests query points based on the optimization of an "easier" acquisition function. So if the blackbox function was f (·) and we had to minimize f (·). Then we would use the bayesian algorithm as described in Algorithm 1. This version of Bayesian Optimization in Algorithm 1 is a one-step sim- DISPLAYFORM0 3: Query objective function to obtain y 0 4: return η 0 plification of the one described in BID11. We next describe the acquisition function α(·; ·) and other statistical details of Algorithm 1. A Gaussian Process is a nonparamteric statistical model that is characterized by µ 0, σ 0, K(·, ·) -initial mean, initial variance and kernel function respectively. Consider the sequence of points, η 1:n (inputs) and y 1:n (noisy observations). We introduce auxillary variables f 1:n, such that f 1:n |η 1:n ∼ N (m, K) where DISPLAYFORM0 Then we have that y 1:n |f 1:n, σ 2 0 ∼ N (f 1:n, σ 2 0 I). Given this GP, we "update" µ n (·), σ n (·) given some observation {(η i, y i)} n i=1, by Algorithm 2 DISPLAYFORM1 A plethora of acquisition functions are discussed in BID11, we specifically use the expected improvement function that we describe now. Consider the improvement function I(η, v, θ) = (v − τ)I(v > τ); this captures the amount of improvement over a given threshold τ. Then expected improvement is just E[I(η, v, θ)] assuming that v is normally distributed with some mean and variance parametrized by η. Then we have DISPLAYFORM0 Typically τ = min n y n, i.e., the minimum of noisy observations. It is not necessary that the kernel function in Algorithm 1 be stationary. In fact there is vast literature on non-stationary kernels for Bayesian Optimization (See Gramacy & Lee FORMULA4). These methods are, in general, very complicated. In the following section we propose a simple compositional nonstationary kernel that works well in practice. The simplicity of Bayesian optimization makes it amenable to blackbox optimizations. However, there are a few severe limitations of Bayesian optimization that prevent it from being applied directly to neural network training. First, as stated before, the framework of Bayesian optimization is such that only one setting of learning rate parameter can be used for the entire duration of training. Second, even if we were somehow able to use multiple learning rate parameter suggestions while training, there is no stationarity, i.e., the same learning rates may produce very different at different points of training. In Hyperdyn we use different temporal estimates of the loss function change and use non stationary kernels to alleviate the two problems mentioned above. By using a simple compositional kernel we avoid the computational burden associated with general non stationary kernels. Any training algorithm can be roughly summarized as follows w t+1 = T (w t, B, Hyperparameters) where B is a batch of data, T denotes the training algorithm and w t are the parameters of some neural network at epoch t of the algorithm. T is run for some predefined number of iterations over the data. For Hyperdyn we define a new set of hyperparameters that includes the epoch number of the algorithm New Hyperparameters = (True Hyperparameters, t) Considering epoch number of the training process as a hyperparameter allows to build a composite kernel over t and the true hyperparameters; we now get kernels that are non-stationary without too much overhead from existing stationary ones. Let (η 1, t 1), (η 2, t 2) be the new hyperparameters where t i is the epoch number, then our kernel is of the form DISPLAYFORM0 Now, the K 1 (·, ·) kernel can be one of RBF kernel or Matern-x kernels over the true hyperparameters, as in standard literature, we will describe K 2 (·, ·), now referred to as the time kernel, in the following section. Unlike the true hyperparameters, we do not need to optimize over the epoch number, i.e., t i s behave in a specific way {1, 2, . . .,} upto end of training process. Such a formulation only helps in introducing non-stationarity in the kernels we use for Bayesian optimization. This is realized by changing the search space for epoch number in the Bayesian optimization to [t, t] for every epoch number t. We use a similar approach to BID14 Section 3 for training curves. The kernel K 2 (·, ·) is of the form DISPLAYFORM0 where the last equality is obtained by choosing ψ(λ) = β α Γ(α) λ α−1 exp (−λβ). This construction is motivated from the observation that SGD decays as O(1/N 1/2) in N iterations, which in our case reduced to α = 1, β = 1/2, and from work in BID14. Hyperdyn is comprised of some crucial moving parts. At the beginning we have no information and employ a purely exploratory approach as described in Algorithm 3. The function Valida- Generate p i ∈ S uniformly at random 6: DISPLAYFORM0 tion Accuracy(·, ·) takes in as input weights (first argument) and hyperparameter values (second argument) and updates weights. The output is top-1 accuracy on the validation set for the updated weights. Function Random Start gives our main algorithm, which we describe in Algorithm 4, some initial information for a more exploitive approach. In Algorithm 4, Update Weights simply updates DISPLAYFORM1 if Check Accuracy Trend(w 1:t−1, ∆) then 10: The function Update Statistical Model has been described in Section 2.1. We might be tempted to switch to the best performing hyperparameter value at every epoch (Lines 9 − 14). As we will show, that such a myopic switching (between hyperparameters) strategy is a hindrance when it comes to good generalization performance. The function Check Accuracy Trend, described in Algorithm 5, prevents us from changing our hyperparameters unnecessarily. The idea is that if a hyperparameter choice has been improving on the validation accuracy, then there is no incentive to change. DISPLAYFORM2 Algorithm 5 Check Accuracy Trend DISPLAYFORM3 return True 6: else 7:return False 8: end if An initial offset is provided so that there is a small probability of changing hyperparameter values even when current values are performing well. The experiments were conducted on Amazon AWS p2x16 EC2 machines. We only use data parallelism for gradient computation in SGD across different GPUs, i.e., if there are k GPUs and a batch size of α then each GPU calculates the gradient on a batch size of α/k and this is summed across GPUs before weight update. The datasets used were CIFAR10 and Imagenet 1K. We employed simple augmentations as described in BID4. We also consider their as the state of art since the structural setting (neural network architecture, augmentation settings etc.) there is closest to the one here. Further for batch sizes not included there we use scaling rules described in BID2. The number of initial search points for Random Start was 5. In our experiments on Hyperdyn we only use learning rate and momentum for hyperparameter optimization. The window size used is 4, temperature is 1 and offset is 0.01. The window size should not be too small ∼ 1 as it leads to very high variance while training, and for the experiments that we run older observations become obsolete fast and a large window size is unnecessary. Comparison to Manually Tuned Networks: In this section we presents when the weight update mechanism is momentum-SGD. We compare the performance of Hyperdyn -tuned momentum-SGD versus the manually tuned momentum-SGD as described in BID4 on a ResNet-20. Each epoch indicates (num points/batch size) number of iterations. As shown in FIG1, Hyperdyn tuning is always faster (at reaching superior generalization performance) than the manually tuned one. Table 1 shows the number of iterations after which a certain top-1 accuracy is reached for a ResNet-20 with batch size 128. Hyperdyn tuned SGD is substantially faster than the fixed schedule manually tuned SGD. There is no additional tuning required for Hyperdyn and as a we circumvent hours of trial and error to get the hyperparameters right. Dynamics of Hyperdyn: For any stable recursive stochastic algorithm (for details see BID1) it must be true that η k → 0 as k → ∞, where η k is the learning rate at iteration k. In FIG0 we observe that Hyperdyn tuning automatically tends to 0 as the learning process progresses. Empirically it has been found that larger learning rates are needed for larger batches in the manual setting BID8 and Hyperdyn is automatically able to achieve this. As can be seen in FIG0 the average learning rate increases as we increase the batch size from 128 to 2k. However, the same trend does not translate, when the batch size is doubled further. A possible explanation to this maybe that we are already in a very large batch regime ∼ 10% of dataset size and the learning rate dynamics behave very differently. Further, we study Hyperdyn in FIG0 when the momentum term is set to 0. We observe that it outperforms manually tuned SGD with momentum, which is a more sophisticated algorithm. This suggests that it is more important to find good learning rates for a simple algorithm, rather than attempt to design a more sophisticated algorithm. Moreover, the wait and watch strategy of Hyperdyn is incorporating past information for stability, similar in principle to momentum. This could also explain why SGD tuned with Hyperdyn can outperform manually tuned SGD with momentum. We compare Hyperdyn to a greedy version of it, where we switch to the best performing hyperparameter in every epoch. This greedy method, or epoch based Bayesian optimization, is obtained by setting the offset i.e., b = ∞ in Check Accuracy Trend sub-routine; which is nothing but the standard Bayesian optimization method with compositional kernel incorporating the temporal dependence. However, as FIG1 shows, the epoch based BO has poor generalization. We set the batch size of 1000 on ResNet-20. This observation necessitates the need for a "wait-and-watch" strategy, as realized by the Check Accuracy Trend module. Although the epoch based BO version outperforms in the initial few epochs, it plateaus quickly and is overtaken by Hyperdyn. The choice of batch size is arbitrary for FIG1 (d) as this observation is consistent across different batch sizes. We also compare Hyperdyn with a version of random search 5x, i.e., at every epoch we invest 5x resources more in random search than in Hyperdyn. We find that random search is susceptible to higher variance than Hyperdyn, especially at larger batch sizes. FIG1 shows a typical comparison of random search 5x and Hyperdyn. We employed a variant of Hyperband where we reduced the search space of learning rate by 0.1 instead of successive halving and did not find any improvement in performance over random search 5x. Using Hyperdyn with Adam: In addition to SGD, Adam is commonly used for optimization. It is adaptive, computationally and memory efficient, but also works for non-stationary objectives and noisy environments BID5. Typically, Adam is employed for the initial part of the training process while SGD is used near the end. We use Hyperdyn to optimize the hyperparameters of Adam, i.e. learning rates β 1, β 2 in BID5. Hyperdyn tuned Adam is much faster than manually tuned Adam (in terms of number of iterations) and also generalizes better on ResNet-20 with batch size 128. As our HPO algorithm proceeds, we observe convergence of learning rates DISPLAYFORM0 Figure 3: Hyperdyn tuned Adam as β 1 → 0 and β 2 is around 1, i.e. the momentum coefficient β 1 becomes very small (Fig. 3(b) ). It turns out that these limits are also needed to establish good theoretical properties for Adam; see Corollary 4.2 in BID5. This also matches the previous observation in BID13 that the momentum coefficient needs to be reduced to obtain good convergence. For Imagenet we used ResNet-50 and a batch size of 1000. This batch size is considerably larger than typical Imagenet experiments, where the batch sizes are ∼ 256. We compare in FIG3 a Hyperdyn tuned Imagenet training process to a manually tuned one. Note that the batch size for the two are different because a manual Imagenet tuning is hard for a batch size of 1000. As a we achieve the same accuracy much faster than a manually tuned training process. Additionally, in BID17 suggest that (upto 40 epochs) Hyperdyn gives the best performance. In this work we introduced a general framework for a certain class of hyperparameter optimization. The algorithm here has the advantage that it is fast, flexible -can be used on any objective function and training method, and stable on larger batch sizes. We demonstrated that our proposed optimizer is faster and at least as accurate as SGD (with momentum) or Adam. We also show that Hyperdyn is stable for larger batch sizes on Imagenet (achieves acceptable accuracy with 4x speed). We demonstrate how too much exploration can be detrimental to generalization accuracy of a training process, and propose a probabilistic "wait-and-watch" strategy. Currently we do not parallelize Hyperdyn; however, computing Validation Accuracy on the suggestion from One Step BO can be easily parallelized. At each epoch we make only suggestion and the validation accuracy on this suggestion can be computed independently of the current hyperparameter setting. In the general case, when we make multiple suggestions we can parallelize in a similar fashion to BID12. We also observe that epoch-based BO in FIG1 outperforms Hyperdyn in the initial epochs. One future direction maybe to have time varying temperature in Check Accuracy Trend based on epoch. We can also exploit the temporal gains obtained by using a backtrack algorithm at the later stages of the algorithm -at a stage when accuracy is sufficiently high but more sophisticated tuning is required to get the best error rates.>>

We propose the Prob2Vec method for problem embedding used in a personalized e-learning tool in addition to a data level classification method, called negative pre-training, for cases where the training data set is imbalanced.

We provide insightful understanding of sequence-labeling NER and propose to use two types of cross structures, both of which bring theoretical and empirical improvements.