Question: Represent the article for finding a single-sentence TLDR

Question: This work presents a method for active anomaly detection which can be built upon existing deep learning solutions for unsupervised anomaly detection. We show that a prior needs to be assumed on what the anomalies are, in order to have performance guarantees in unsupervised anomaly detection. We argue that active anomaly detection has, in practice, the same cost of unsupervised anomaly detection but with the possibility of much better . To solve this problem, we present a new layer that can be attached to any deep learning model designed for unsupervised anomaly detection to transform it into an active method, presenting on both synthetic and real anomaly detection datasets. Anomaly detection (a.k.a. outlier detection) (; ;) aims to discover rare instances that do not conform to the patterns of majority. From a business perspective, though, we are not only interested in finding rare instances, but "usefull anomalies". This problem has been amply studied recently (; ; ; ;), with solutions inspired by extreme value theory , robust statistics and graph theory .Unsupervised anomaly detection is a sub-area of outlier detection, being frequently applied since label acquisition is very expensive and time consuming. It is a specially hard task, where there is usually no information on what these rare instances are and most works use models with implicit priors or heuristics to discover these anomalies, providing an anomaly score s(x) for each instance in a dataset. Active anomaly detection is a powerful alternative approach to this problem, which has presented good in recent works such as (; ; 2017).In this work, we first show that unsupervised anomaly detection requires priors to be assumed on the anomaly distribution; we then argue in favor of approaching it with active anomaly detection, an important, but under-explored approach (Section 2). We propose a new layer, called here Universal Anomaly Inference (UAI), which can be applied on top of any unsupervised anomaly detection model based on deep learning to transform it into an active model (Section 3). This layer uses the strongest assets of deep anomaly detection models, i.e. its learned latent representations (l) and anomaly score (s), to train a classifier on the few already labeled instances. An example of such an application can be seen in FIG0, where an UAI layer is built upon a Deanoising AutoEncoder (DAE).We then present extensive experiments, analyzing the performance of our systems vs unsupervised, semi-supervised and active ones under similar budgets in both synthetic and real data, showing our algorithm improves state of the art in several datasets, with no hyperparameter tuning (Section 4). Finally, we visualize our models learned latent representations, comparing them to unsupervised models' ones and analyze our model's performance for different numbers of labels (Appendix C). defines an outlying observation, or outlier, as one that appears to deviate markedly from other members of the sample in which it occurs. states that an outlier is an observation that deviates so much from other observations as to arouse suspicion that it was generated by a different mechanism. says that normal data instances occur in high probability regions of a stochastic model, while anomalies occur in the low probability ones. Following these definitions, specially the one from , we assume there is a probability density function from which our'normal' data instances are generated: X normal ∼ p normal (x) = p (x|y = 0), where x is an instance's available information 1 and y is a label saying if the point is anomalous or not. There is also a different probability density function from which anomalous data instances are sampled: X anom ∼ p anom (x) = p (x|y = 1). A full dataset is composed of both normal and anomalous instance, being sampled from a probability distribution that follows:(X, Y) full ∼ p full (x, y) = p (y) p (x|y) X full ∼ p full (x) = p (y = 0) p normal (x) + p (y = 1) p anom (x) = (1 − λ)p normal (x) + λp anom (x)where λ is an usually small constant representing the probability of a random data point being anomalous (λ = p(y = 1)), this constant can be either known a priori or not. divides anomaly detection learning systems in three different types:• Supervised: A training and a test set are available with curated labels for non-anomalous and anomalous instances. This case is similar to an unbalanced supervised classification setting: D train/test = (X, Y) train/test ∼ p full (x, y) • Semi-Supervised: A training set is available containing only non-anomalous instances and the challenge is to identify anomalous instances in a test set. This is also called novelty detection: DISPLAYFORM0 • Unsupervised: A dataset containing both non-anomalous and anomalous instance is available and the challenge is to identify anomalous instances in it. There is no concept of a test set since anomalous instances must be sorted in the dataset itself: DISPLAYFORM1 2.1 UNSUPERVISED ANOMALY DETECTION In this work, we will focus on unsupervised anomaly detection. Here, in possession of the full set of points X ∼ p full (x), we want to find a subset X anom ⊂ X which is composed of the anomalous instances. The full distribution p full is a mixture of distributions and if these distributions overlap very closely, it may be impossible to learn the individual distributions beyond a certain accuracy threshold . It is a well-known that general mixture models are unidentifiable . In the sequence, we further show that we gain no information on p anom from p full for any small λ without a prior on the anomalies' probability distribution. This differs from the usual unidentifiability of mixture models in that we make no assumptions on the prior for p normal, showing all valid distributions of p anom are equally probable. Theorem 1. No free anomaly theorem. Consider two independent arbitrary probability distributions p normal and p anom. For a small number of anomalies λ ≈ 0, p full = p gives us no further knowledge on the distribution of p anom: DISPLAYFORM2 From Theorem 1 we can conclude that unsupervised anomaly detection requires a prior on the anomalies distribution. A more tangible example of this can be seen in FIG1, where we present a synthetic data distribution composed of three classes of data clustered in four visibly separable clusters. Anomaly detection is an undecidable problem in this setting without further information, since it is impossible to know if the low density cluster is composed of anomalies or the anomalies are the unclustered low density points (or a combination of both).If we used a high capacity model to model the data distribution in FIG1, the low density points (Right) would be detected as anomalous. If we used a low capacity model, the cluster (Center) would probably present a higher anomaly score. Our choice of algorithm implicitly imposes a prior on the detected anomalies. Theorem 1 highlights this and makes the need to consider priors explicit. In a more practicle example, assume we are working with clinical data. In this setting, some low density clusters may indicate diseases (anomalies), while other low density clusters may be caused by uncontrolled factors in the data, such as high performance athletes. At the same time, rare diseases might seem like scattered (low density) points. We want to be able to distinguish between anomalies and'uninteresting' low probability points. The usual strategy when working with unsupervised anomaly detection problems is training a parameterized model p θ (x) to capture the full data distribution p full (x) (e.g. a PCA, or AutoEncoder), and, since λ is, by definition, a small constant, assuming p full (x) ≈ p normal (x) and assuming points with low probability are anomalous . An anomaly score s(x) is then defined as s(x) = 1 p(x). There are three main problems with this strategy: if anomalous items are more common than expected, p full might be a poor approximation of p normal; if anomalous items are tightly clustered in some way, high capacity models may learn to identify that cluster as a high probability region; if anomalous items are as rare as expected, since we only have access to p full, Theorem 1 states we have no information about p anom without further assumptions on its probability distribution. Most unsupervised anomaly detection systems also already rely on further verification of the by human experts, due to their uncertain performance. Being mostly used as a ranking system to get high probability instances in the top of a'list' to be further audited by these experts. From Theorem 1, we conclude it is impossible to have an universal and reliable unsupervised anomaly detection system, while we know that most such systems already rely on the data being later audited by human experts. These arguments together argue in favor of an active learning strategy for anomaly detection, including the auditor experts in the system's training loop. Thus, anticipating feedback and benefiting from it to find further anomalous instances, which in a more robust system. Having an extremely unbalanced dataset in this problem (λ ≈ 0) is also another justification for an active learning setting, which has the potential of requiring exponentially less labeled data than supervised settings .3.1 ACTIVE ANOMALY DETECTION With these motivations, we argue in favor of active anomaly detection methods, which despite its many advantages remains an under-explored approach to this problem. Nonetheless, recent work has shown promising (; ; 2017). In unsupervised anomaly detection, we start with a dataset D = {x|x ∼ p full (x)} and want to rank elements in this dataset so that we have the highest possible recall/precision for a certain budget b, which is the number of elements selected to be audited by an expert, with no prior information on anomaly labels. In active anomaly detection, we also start with a completely unlabeled anomaly detection dataset D = {x|x ∼ p full (x)}, but instead of ranking anomalies and sending them all to be audited at once by our expert, we select them in small parts, waiting for the experts feedback before continuing. We iteratively select the most probable k b elements to be audited 2, wait for the expert to select their label, and continue training our system using this information, as shown in Algorithm 1. This requires the same budget b as an unsupervised anomaly detection system, while having the potential of achieving a much better performance. DISPLAYFORM0 labels ← labels ∪ expert.audit(top_k) DISPLAYFORM1 With this in mind, we develop the Universal Anomaly Inference (UAI) layer. This layer can be incorporated on top of any deep learning based white box anomaly detection system which provides an anomaly score for ranking anomalies. It takes as input both a latent representation layer (l(x)), created by the model, and its output anomaly score (s(x)), and passes it through a classifier to find an item's anomaly probability. DISPLAYFORM2 This is motivated by recent works stating learned representations have a simpler statistical structure , which makes the task of modeling this manifold and detecting unnatural points much simpler . In this work, we model the UAI layer using a simple logistic regression as our classifier, but any architecture could be used here: DISPLAYFORM3 where W act ∈ R 1,d+1 is a linear transformation, b act ∈ R is a bias term and σ(·) is the sigmoid function. We learn the values of W and b using back-propagation with a cross entropy loss function, while allowing the gradients to flow through l, but not through s, since s might be non-differentiable. For the rest of this document, we will refer to the networks with a UAI layer as UaiNets. An example of this architecture is shown in FIG0. In this section, we test our new UAI layer on top of two distinct architectures: a Denoising AutoEncoder (DAE, with s dae (x) = ||x −x|| 2 2 ) and a Classifier (Class, with s class (x) = cross_entropy(x y, x y)), which use standard multi layer perceptrons. Both architectures are described in details in Appendix A.1. To test our algorithm we start by analyzing its performance on synthetic data created with different properties (Section 4.1). We then present using UaiNets on real anomaly detection datasets (Section 4.2) and in a semi-supervised setting (Section 4.3). When designing experiments, we had the objective of showing that our model can work with different definitions of anomaly, while completely unsupervised models will need, by definition, to trade-off accuracy in one setting for accuracy in the other. While this may seem straight forward, these can show how robust our approach is to the choice of underlying architecture, analyzing how well they do when their underlying architecture has a bad prior for that specific "type" of anomaly. With this in mind, we used the MNIST dataset and defined four sets of experiments: 3 1. MNIST 0: For the first set of experiments, we reduced the presence of the 0 digit class to only 10% of its original number of samples, making it only 1/91 ≈ 1.1% of the dataset samples. The 0s still present in the dataset had its class randomly changed to x y ∼ Uniform([1; 9]) and were defined as anomalies. 2. MNIST 0-2: The second set of experiments follows the same dataset construction, but we reduce the number of instances of numbers 0, 1 and 2, changing the labels of the remaining items in these categories to x y ∼ Uniform([3; 9]), and again defining them as anomalous. In this dataset anomalies composed 3/73 ≈ 4.1% of the dataset. 3. MNIST hard: The third set of experiments aims to test a different type of anomaly. In order to create this dataset, we first trained a weak one hidden layer MLP classifier on MNIST and selected all misclassified instances as anomalous, keeping them in the dataset with their original properties (x x and x y). In this dataset anomalies composed ≈ 3.3% of the dataset. 4. MNIST pca: In this set of experiments, for each image class (x y), we used a PCA to reduce the dimensionality of MNIST images (x x) to 2 and selected the 5% instances with the largest reconstruction error as anomalies. We kept all 60,000 instances in the dataset with their original properties (x x and x y) and in this dataset anomalies composed 5% of the dataset. Results for these experiments are shown in FIG3 and the main taken from them is that, even though our algorithm might not get better than its underlying model for every budget-dataset pair, it is robust to different types of anomalies, which is not the case for the underlying completely unsupervised models. While Class gives really good in MNIST 0 and MNIST 0-2 datasets, it does not achieve the same performance in MNIST hard and MNIST pca, which might indicate it is better at finding clustered anomalies than low density ones. At the same time, DAE has good for MNIST pca and MNIST hard, but bad ones for MNIST 0 and MNIST 0-2, which indicates it is better at finding low density anomalies than clustered ones. Nevertheless, both UaiNets are robust in all four datasets, being able to learn even on datasets which are hard for their underlying models, although they might have a cold start to produce . Here we analyze our model's performance on public benchmarks composed of real anomaly detection datasets. We employ 11 datasets in our analysis: KDDCUP; Thyroid; Arrhythmia; KDDCUP-Rev; Yeast; Abalone; Cardiotocography (CTG); Credit Card; Covtype; Mammography (MMG); Shuttle (; ; ;). We compare our algorithm against: DAE (TAB0 presents for these real datasets. In these experiments, DAGMM (clean) was trained on a semi-supervised anomaly detection setting, using clean datasets during training, DAGMM (dirty) and DAE were trained in an unsupervised setting, while LODA-AAD, Tree-AAD and DAE uai were trained in an active anomaly detection setting. We can clearly see from these that DAE produces fairly bad for all datasets analyzed here, nevertheless, even using a simple architecture as its underlying model, DAE uai produces similar or better to the best baselines on the 11 datasets, even when the baselines were trained in completely clean training sets. DAE uai also usually presents better than LODA-AAD and Tree-AAD, which are similarly trained in an active setting. One possible criticism to our method is that the importance of the proposed approach becomes more relevant the fewer the proportion of anomalous instances, which seems self-defeating. But we see that the largest difference from the active methods to the other algorithms was in Covtype, which has less than 1% anomalies but 286,048 instances. When working with large datasets (>1M instances), even if only 0.1% of the dataset is contaminated there is still the chance to benefit from this feedback to improve performance. The active algorithms are also more robust than the others, DAGMM used different hyperparameters for each experiment, while DAE uai and AAD use the same for all (except for k which was reduced from 10 to 3 for the datasets with less than 100 anomalies). Another practical scenario where our model could be applied is a mixture of semi-supervised and unsupervised anomaly detection. In this case, we have a dataset which contains anomalies that we want to find and audit. At the same time, new data instances, which may include new types of anomalies not seen before, can be added to the dataset at any time and we would like to detect anomalies in this dataset as well. DISPLAYFORM0 With this in mind, we ran an experiment training DAE uai and LODA-AAD on KDDCUP-Rev in the same way as in Section 4.2, while evaluating it on its test set for different budgets. This test set contains 20 new types of anomalies (the train set contains 16 types of anomalies and the test set 36). The evaluation was done by selecting the most anomalous instances found by each model on the test set and calculating the recall for both seen and unseen anomalies in that group. Results for this experiment can be seen in FIG4. In this figure, the right y axis shows the number of anomalies detected in the training set for a certain budget and corresponds to the light blue lines. The left y axis present the recall for the test dataset. We see that DAGMM is not so effective on this test set, while DAE is able to detect well novelty (new classes). We also see that DAE uai is significantly better at detecting known types of anomalies, while it maintains a recall close to the best on new unseen classes, giving better than LODA-AAD for both seen and unseen classes of anomalies. Anomaly Detection This field has been amply studied and good overviews can be found in . Although many algorithms have been recently proposed, classical methods for outlier detection, like LOF Breunig et al. FORMULA6 and OC-SVM (Schölkopf et al., 2001), are still used and produce good . Recent work on anomaly detection has focused on statistical properties of "normal" data to identify these anomalies, such as , which uses Benford's Law to identify anomalies in social networks, and , which uses Extreme Value Theory to detect anomalies. Other works focus on specific types of data, focuses on spatially contextualized data, while (; ; ;) focus on graph data. Recently, energy based models and GANs have been successfully used to detect anomalies, but autoencoders are still more popular in this field. propose a method to train robust autoencoders, drawing inspiration from robust statistics and more specifically robust PCAs, focuses on clustering, and trains autoencoders that generate latent representations which are friendly for k-means. The work most similar to ours is DAGMM , where they train a deep autoencoder and use its latent representations, together with its reconstruction error, as input to a second network, which they use to predict the membership of each data instance to a mixture of gaussian models, training the whole model end-to-end in an semi-supervised manner for novelty detection. Active Anomaly Detection Despite its many advantages, active anomaly detection remains an under-explored approach to this problem, nevertheless, over the years some really interesting work has been developed in this topic. In , the authors solve the rare-category detection problem by proposing an active learning strategy to datasets with extremely skewed distributions of class sizes. reduces outlier detection to classification using artificially generated examples that play the role of potential outliers and then applies a selective sampling mechanism based on active learning to the reduced classification problem. In (Görnitz et al., 2013), the authors proposed a Semi-Supervised Anomaly Detection (SSAD) method based in Support Vector Data Description (SVDD) , which he expanded to a semi-supervised setting, where he accounts for the presence of labels for some anomalous instances, and with an active learning approach to select these instances to label. propose an active approach that combines unsupervised and supervised learning to select items to be labeled by experts, with each approach selecting k 2 instances at a time. The most similar prior works to ours in this setting are , which proposed an algorithm that can be employed on top of any ensemble methods based on random projections, and , which expands Isolation Forests to work in an active setting. Our work differs from these prior works mainly in that we prove the necessity of priors for unsupervised anomaly detection, further motivating the Active Anomaly Detection framework, and in our proposed model. UAI layers can be assembled on top of any Deep Learning based anomaly detection architecture, which is the state of the art for unsupervised anomaly detection, to make it work in an active anomaly detection setting. Besides, after each iteration with experts both LODA-AAD and Tree-AAD have a time complexity O(t), where t is the number of already labeled instances, while each iteration of UaiNets runs in constant time O with respect to t. We proposed here a new architecture, Universal Anomaly Inference (UAI), which can be applied on top of any deep learning based anomaly detection architecture. We show that, even on top of very simple architectures, like a DAE, UaiNets can produce similar/better to state-of-the-art unsupervised/semi-supervised anomaly detection methods. We also give both theoretical and practical arguments motivating active anomaly detection, arguing that, in most practical settings, there would be no detriment to using this instead of a fully unsupervised approach. We further want to make clear that we are not stating our method is better than our semi-supervised baselines (DAGMM, DCN, DSEBM-e). Our contributions are orthogonal to theirs. We propose a new approach to this hard problem which can be built on top of them, this being our main contribution in this work. To the best of our knowledge, this is the first work which applies deep learning to active anomaly detection. We use the strongest points of these deep learning algorithms (their learned representations and anomaly scores) to build an active algorithm, presenting an end-to-end architecture which learns representations by leveraging both the full dataset and the already labeled instances. Important future directions for this work are using the UAI layers confidence in its output to dynamically choose between either directly using its scores, or using the underlying unsupervised model's anomaly score to choose which instances to audit next. Another future direction would be testing new architectures for UAI layers, in this work we restricted all our analysis to simple logistic regression. A third important future work would be analyzing the robustness of UaiNets to mistakes being made by the labeling experts. Finally, making this model more interpretable, so that auditors could focus on a few "important" features when labeling anomalous instances, could increase labeling speed and make their work easier. In this section we give detailed descriptions of the experiments. Section A.1 presents the used model architectures for both DAE and Class models, as well as DAE uai and Class uai. Section A.2 presents details on the synthetic MNIST datasets and on the hyper-parameters used for the experiments. Finally, Section A.3 contains detailed descriptions on the used datasets, baselines and experimental settings for the experiments on real anomaly detection datasets. To show our algorithm can be assembled on top of any deep learning model, we tested it using two simple but very different anomaly detection models. The first model we test it on top of is a normal Denoising AutoEncoder (DAE). A DAE is a neural network mainly composed by an encoder, which transforms the input into a latent space, and a decoder, which reconstructs the input using this latent representation, typically having a loss function that minimizes the reconstruction error L 2 norm: DISPLAYFORM0 where both f enc and f dec are usually feed forward networks with the same number of layers, l ∈ R d is a d-dimensional latent representation and is a zero mean noise, sampled from a Gaussian distribution with a ϕ standard deviation. When used in anomaly detection, the reconstruction error is usually used as an approximation of the inverse of an item's probability, and as its anomaly score: DISPLAYFORM1 We then create a DAE uai network by assembling the proposed UAI layer on top of the DAE: DISPLAYFORM2 where uai(·) is the classifier chosen for the UAI layer. This architecture can be seen in FIG0. Another typical approach to unsupervised anomaly detection is, when given a dataset with labeled data X = (x x, x y), training a classifier (Class) to predict x y from x x 5 and using the cross-entropy of an item as an approximation to the inverse of its probability distribution: DISPLAYFORM3 where f class (·) is typically a feed forward neural network with p layers, from which we can use its last hidden layer (h p−1) as the data's latent representation to be used in the Class uai. DISPLAYFORM4 This architecture can be seen in FIG5. For all experiments in this work, unless otherwise stated, the DAE's encoder and decoder had independent weights and we used both the DAE and Class models with 3 hidden layers and hidden sizes. This means the latent representations provided to the UAI layers are l ∈ R 8. We implemented all experiments using TensorFlow , and used a learning rate of 0.01, batch size of 256 and the RMSprop optimizer with the default hyper-parameters. For the active learning models, we pre-train the DAE/Class model for 5000 optimization steps, select k = 10 items to be labeled at a time, and further train for 100 iterations after each labeling call. To deal with the cold start problem, for the first 10 calls of select_top, we use the base anomaly score (s) of the DAE/Class model to make this selection, using the UAI one for all later labeling decisions. Detailed statistics on the synthetic MNIST datasets can be seen in TAB2. MNIST 0 and MNIST 0-2 were mainly generated with the purpose of simulating the situation in FIG1 (Center), where anomalies were present in sparse clusters. At the same time, MNIST hard and MNIST pca were designed to present similar characteristics to the situation in FIG1 (Right), where anomalous instances are in sparse regions of the data space. For these experiments, most datasets were used as suggested in , but we processed the KDDCUP, Thyroid, Arrhythmia and KDDCUP-Rev datasets in the same manner as to be able to better compare with their :• KDDCUP : The KDDCUP99 10 percent dataset from the UCI repository. Since it contains only 20% of instances labeled as "normal" and the rest as "attacks", "normal" instances are used as anomalies, since they are in a minority group. This dataset contains 34 continuous features and 7 categorical ones. We transform these 7 categorical features into their one hot representations, and obtain a dataset with 120 features.• Thyroid : A dataset containing data from patients which can be divided in three classes: normal (not hypothyroid), hyperfunction and subnormal functioning. In this dataset, we treat the hyperfunction class as an anomaly, with the other two being treated as normal. It can be obtained from the ODDS repository. • Arrhythmia : This dataset was designed to create classification algorithms to distinguish between the presence and absence of cardiac arrhythmia. In it, we use the smallest classes (3, 4, 5, 7, 8, 9, 14, and 15) as anomalies and the others are treated as normal. This dataset can also be obtained from the ODDS repository.• KDDCUP-Rev : Since "normal" instances are a minority in the KDDCUP dataset, we keep all "normal" instances and randomly draw "attack" instances so that they compose 20% of the dataset. We compare our algorithm against: et al., 2008): Denoising Autoencoders are autoencoder architectures which are trained to reconstruct instances from noisy inputs. DISPLAYFORM0 • DAGMM : Deep Autoencoding Gaussian Mixture Model is a stateof-the-art model for semi-supervised anomaly detection which simultaneously learns a latent representation, using deep autoencoders, and uses both this latent representation and the autoencoder's reconstruction error to learn a Gaussian Mixture Model for the data distribution.• LODA-AAD : Lightweight on-line detector of anomalies (LODA) Active Anomaly Discovery (AAD) is a work which uses the active anomaly detection framework on top of LODA (Pevnỳ, 2016), which is a method based on ensembles of weak anomaly detection models.• Tree-AAD : This work learns weights for each node in an Isolation Forest anomaly detection model, by incorporating knowledge gained through active anomaly detection. Since there is no validation/test set in unsupervised anomaly detection, we cannot tune hyperparameters on a validation set. Because of this, to make the DAE baselines more competitive, we got the for several different hyper-parameter configurations and present only the best among them. This is not a realistic approach, but we only do it to our baselines, while for our proposed algorithm we keep hyper-parameters fixed for all experiments. We even keep our hidden sizes fixed to on thyroid, which only contains 6 features per instance, since our objective here is not getting the best possible , but showing the robustness of our approach. The only hyper-parameter change we make in UAI networks is that, since there are fewer anomalies in some datasets, we set our active learning approach to choose k = 3 instances at a time, instead of 10, for datasets with less than 100 anomalies. Results for DAGMM are from our implementation of this model and follow the same procedures, architectures and hyper-parameters as described in , being trained in a semisupervised setting. The for LODA-AAD and Tree-AAD were run using the code made available by the authors and with the same steps as DAE uai. 7 For all experiments, for LODA-AAD, Tree-AAD, DAE and DAE uai used the number of anomalies in the dataset as the budget b. In this section, we present more detailed for both the synthetic (Section B.1) and real (Section B.2) anomaly detection datasets, which couldn't fit on the main paper due to lack of space. We also present for synthetic anomaly detection experiments on Fashion-MNIST (Section B.3). We present here detailed for small budgets (b ≤ 5000) on the MNIST experiments, with graphs zoomed in for these budget values. Analyzing FIG6 we see that for some of these datasets UaiNets present a cold start, producing worse for small budgets. Nonetheless, after this cold start, they produce better in all MNIST experiments. An interesting future work would be to measure the confidence in the UaiNet's prediction to dynamically choose between using its anomaly score or the underlying network's one, which could solve/reduce this cold start problem. TAB3 presents a detailed comparison for experiments ran on KDDCUP, Thyroid, Arrhythmia and KDDCUP-Rev datasets with other baselines, also showing precision, recall and their standard deviations. In this table we also compare our to: • OC-SVM : One-class support vector machines are a popular kernel based anomaly detection method. In this work, we employ it with a Radial Basis Function (RBF) kernel.• DCN : Deep Clustering Network is a state-of-the-art clustering algorithm. Its architecture is designed to learn a latent representation using deep autoencoders which is easily separable when using k-means.• PAE : Denoising AutoEncoders pretrained as suggested in .• DSEBM-e : Deep Structured Energy Based Models are anomaly detection systems based on energy based models , which are a powerful tool for density estimation. We compare here against DSEBM-e, which uses a data instance's energy as the criterion to detect anomalies.• DSEBM-r : Deep Structured Energy Based Model with the same architecture and training procedures as DSEBM-e, but using an instance's reconstruction error as the criterion for anomaly detection. The presented here are averages of five runs, with standard deviations in parenthesis. In this table, for OC-SVM, PAE, DSEBM-r, DSEBM-e, DCN and DAGMM were taken from , while DAGMM * are from our implementation of DAGMM. Unfortunately, we were not able to reproduce their in the Thyroid dataset, getting a high variance in the . LODA-AAD does not scale well to large datasets, so to run it on KDDCUP and KDDCUP-Rev we needed to limit its memory about the anomalies it had already learned, forgetting the oldest ones. This reduced its runtime complexity from O(b 2) to O(b) in our tests, where b is the budget limit for the anomaly detection task. We did the same (limit memory) for Tree-AAD on KDDCUP.On this table we can see that DAE uai produces better than LODA-AAD on all analyzed datasets and than Tree-AAD on three out of four. Our proposed method also, besides presenting comparable to state-of-the-art DAGMM trained on a clean dataset, is much more stable, having a lower standard deviation than the baselines in almost all datasets. In this Section, we present for experiments on synthetic anomaly detection datasets based on Fashion-MNIST . To create these datasets we follow the same procedures as done for MNIST in Section 4.1, generating four datasets: Fashion-MNIST 0; Fashion-MNIST 0-2; Fashion-MNIST hard; Fashion-MNIST pca. Detailed statistics of these datasets can be seen in TAB4. We run experiments on these datasets following the exact same procedures as in Section 4.1. FIG8 shows the for Fashion-MNIST 0 and Fashion-MNIST 0-2, while Figure 8 show the for Fashion-MNIST hard and Fashion-MNIST pca. These figures show similar trends to the ones for MNIST, although algorithms find anomalies in these datasets harder to identify. In one run of Fashion-MNIST 0, DAE uai needed several examples to start learning and for Fashion-MNIST hard, Class uai takes a long time to start producing better than Class. Nevertheless, UaiNets are still much more robust than the underlying networks to different types of anomalies, producing good in all four datasets, even when its underlying network gives weak on that dataset. In this section we further study UaiNets, analyzing the evolution of hidden representations and anomaly scores through training (Section C.1), and the dependence of on the number of audited anomalies (Section C.2). In this section, we show visualizations of the learned representations (l dae/class) and anomaly scores (s dae/class) of UaiNets' underlying networks, presenting their evolution as more labels are fed into the network through the active learning process. With this purpose, we retrain UaiNets on both MNIST 0-2 and MNIST hard, with a hidden size of, so that its latent representation is one dimensional (l(x) ∈ R 1 ), and plot these representations vs the anomaly scores (s) of the base network (either DAE or Class) for different budgets (b). Figure 9 shows the evolution of DAE uai's underlying l dae (x) and s dae (x). In it, we can see that initially (Figures 9 (a, d) ) anomalies and normal data instances are not separable in this space. Nevertheless, with only a few labeled instances (b = 250) the space becomes much easier to separate, while for b = 2000 the space is almost perfectly linearly separable. 8 FIG0 shows the same evolution for Class uai's underlying l class (x) and s class (x). In it, we can also see the same patterns, as initially anomalies and normal data instances are not separable, but with a few labeled instances anomalies become much more identifiable. The main taken from these visualizations is how the gradient flow through l is important, since it helps the network better separate data in these spaces, allowing good performance even when the underlying networks are not good at identifying a specific type of anomaly. This experiments aim at showing how the networks choice quality evolves with the access to more labels. Here, we present the choices DAE uai network would make having access to a fixed number of expert labels. With this in mind, we train the networks in the same way as in Section 4.2, but stop after reaching a specific budget (b), showing the choices made up to that point, and after that with no further training. FIG0 shows the evolution of DAE uai anomaly choices as it is fed more expert knowledge. We can see that with only a few labels it already fairs a lot better than its underlying network. In KDDCUP with only 3,000 labeled instances, which is less than 1% of the dataset, it can correctly find 80,000 anomalies with a high precision, while the DAE with no expert knowledge does a lot worse. On Thyroid and KDDCUP-Rev, with ≈ 10% of the dataset labeled (b = 531 and b = 4000, respectively) it finds all or almost all anomalies in the dataset correctly. The Arrhythmia dataset is a lot smaller and with few anomalies, so DAE uai improves on DAE in a smaller scale here, but it still does fairly better than the underlying network. Gifs showing this choice evolution will be made available with the final publication. DISPLAYFORM0 Figure 9: (Color online) Underlying latent representations (l dae) vs anomaly score (s dae) for DAE uai network as training progresses on MNIST 0-2 and MNIST hard. DISPLAYFORM1 Figure 10: (Color online) Underlying latent representations (l class) vs anomaly score (s class) for Class uai network as training progresses on MNIST 0-2 and MNIST hard.so its probability distribution is: where P is the hyperspace containing all probability distributions, with an hyper-volume m. Now we can try to find p(p 1 |p + = p): DISPLAYFORM2 DISPLAYFORM3 where Equality and from the fact that p 1 ∈ P 1 ⇔ p 2 ∈ P 2, given a specific value of p + = p. This completes this proof. D.2 LEMMA 2. EXTREME MIXTURES LEMMA Lemma 2. Extreme mixtures lemma. Consider two independent arbitrary probability distributions p 1 and p 2. Given only a third probability distribution p + = p composed of the weighted mixture of the two, and for a small λ ≈ 0, we can find a small residual hyperplane P 1, which tends to {p}. DISPLAYFORM4 We can also find a very large residual hyperplane P 2 for p 2, which tends to: DISPLAYFORM5 where supp(·) is the support of a probability distribution. Proof. In this proof, we start with the arbitrary residual hyperplanes P r and find restrictions in the limits of λ → 0 and λ → 1. For a β ≈ 0:lim β→0 P r = lim β→0 {p r = p−β·p 1−β, ∀p ∈ P | β · p ≤ p} = lim β→0 {p r = p − β · p, ∀p ∈ P | β · p ≤ p} = {p} P r ≈ {p r = p − β · p, ∀p ∈ P | β · p ≤ p} β ≈ 0 P 1 ≈ {p r = p − λ · p, ∀p ∈ P | λ · p ≤ p} λ ≈ 0 P 2 ≈ {p r = p − (1 − λ) · p, ∀p ∈ P | (1 − λ) · p ≤ p} λ ≈ 1 ∴ β ≈ 0 For a β ≈ 1 we start with the other definition of P r: lim β→1 P r = lim β→1 {p r, ∀p r ∈ P | (1 − β) · p r ≤ p} = lim β→1 {p r, ∀p r ∈ P | supp(p r) ⊆ supp(p), (1 − β) · p r ≤ p} = {p r, ∀p r ∈ P | supp(p r) ⊆ supp(p)} P r ≈ {p r, ∀p r ∈ P | supp(p r) ⊆ supp(p)} β ≈ 1 P 1 ≈ {p r, ∀p r ∈ P | supp(p r) ⊆ supp(p)} λ ≈ 1 P 2 ≈ {p r, ∀p r ∈ P | supp(p r) ⊆ supp(p)} λ ≈ 0 ∴ β ≈ 1This finishes this proof. D.3 THEOREM 1. NO FREE ANOMALY THEOREM Theorem 1. Consider two independent arbitrary probability distributions p normal and p anom. For a small number of anomalies λ ≈ 0, the knowledge that p full = p gives us no further knowledge on the distribution of p anom: DISPLAYFORM6 Proof. Consider in Lemmas 1 and 2 that p 2 = p anom ∼ Uniform(P). We then have that, for a small value of λ ≈ 0: DISPLAYFORM7 This finishes this proof. In this section, we prove upper and lower bounds on the maximum distance a probability distribution p 1 can be from p +, based on the value of λ. This can be directly applied to p normal for small values of λ and to p anom for large ones. Theorem 2. Upper Bound on Mixture Probability Distance For two independent arbitrary probability distributions p 1 and p 2, given only a third probability distribution p + composed of the weighted mixture of the two: DISPLAYFORM0 We have an upper bound on the distance measures δ(p +, p 1) and ||p + − p 1 || given by:δ(p +, p 1) ≤ 1 2 log 1 1 − λ ||p + − p 1 || ≤ 2 log 1 1 − λ which is a tight bound for λ ≈ 0. In this equation δ(·) is the total variation distance between two probability distributions and || · || is the L 1 norm. Proof. Pinsker's inequality states that if p and q are two probability distributions on a common measurable space (A, (1−λ)·p1(x)+λ·p2(x) dx where this maximum Kullback-Leibler divergence is achieved when p 1 and p 2 are disjoint probability distributions: DISPLAYFORM1 (1−λ)·p1(x)+λ·p2(x) dx ≤ x p 1 (x) log p1(x)(1−λ)·p1(x) dx = x p 1 (x) log δ(p +, p 1) ≤ 1 2 log 1 1 − λ ||p + − p 1 || ≤ 2 log 1 1 − λ Theorem 3. Lower Bound on Maximum Mixture Probability Distance For two independent arbitrary probability distributions p 1 and p 2, given only a third probability distribution p + composed of the weighted mixture of the two: DISPLAYFORM2 We have a lower bound on the maximum possible distance measures δ(p +, p 1) and ||p + − p 1 || for a chosen maximizing p 1 given by: DISPLAYFORM3 which is a tight bound for λ ≈ 1, considering the maximum L 1 distance between two probability distributions is 2.Proof. We can prove a lower bound on the maximized distance of a probability distribution p 1 from p + by expanding the distance equations: where in (a) we lower bound based on the probability distribution that would have the smallest possible superior distance to a later maximized probability distribution p 1. This probability distribution p 1 can always maximize its superior distance to p 2 by: DISPLAYFORM4 DISPLAYFORM5 In (b) we choose the uniform distribution as the one that would reduce this superior distance and in (c) we set p 1 (a) = 1 for a random a, since p 2 is uniform. With a similar strategy we find: DISPLAYFORM6 This concludes this proof.

represent the tldr for finding the corresponding scientific article

To give you a sense - "Obfuscate code using seq2seq networks, and execute using the obfuscated code and key pair" should be close to "The paper explores a novel methodology in source code obfuscation through the application of text-based recurrent neural network network (RNN) encoder-decoder models in ciphertext generation and key generation. Sequence-to-sequence models are incorporated into the model architecture to generate obfuscated code, generate the deobfuscation key, and live execution. Quantitative benchmark comparison to existing obfuscation methods indicate significant improvement in stealth and execution cost for the proposed solution, and experiments regarding the model’s properties yield positive regarding its character variation, dissimilarity to the original codebase, and consistent length of obfuscated code. The field of code obfuscation has aimed to tackle reverse-engineering of code bases for years. The entire basis of this methodology is that if a program is constructed with logic not easily recognizable by a reader, the logic would be preserved intact and the software would be intrinsically protected. Traditional tactics include creative uses of whitespace, redundant logical operations, unnecessary conditional operations, amongst others. The common issue with obfuscation is that it can be reverseengineered, the only factor for a malicious actor would be the amount of time needed to discover the logic. DeepObfusCode is a proposed methodology to use neural networks to convert the plaintext source code into a cipher text by using the propagating architecture of neural networks to compound the randomness factor in the creation of the ciphertext. Yet at the same time, neural networks have the ability to learn statistical patterns and generate weights to convert one text to another, in our case from the ciphertext to plaintext. This would eventually permit users to simply load the ciphertext and the key to self-execute the program without foreign users viewing the inner workings of the code. From an academic standpoint, this methodology redirects obfuscation methodology towards complete obfuscation in contrary of incremental obfuscation, and suggests the usage and development of asymmetric key infrastructure in obfuscation. Beyond sequence-to-sequence network models, further obfuscation models could be built with greater degree of resilience, and other deep learning methods could be harnessed to develop alternative techniques to obfuscate code. The methodology can be adopted for more efficient, more effective obfuscation for developers protecting their proprietary codebase or cloud computing services wishing to guarantee confidentiality to customers. The algorithmic architecture could be further incorporated into larger frameworks or infrastructures to render homomorphic encryption and ensure complete anonymity of the codebase during execution from the execution provider. The objective of code obfuscation is to mask the computational processes and logic behind software code, to protect trade secrets, intellectual property, or confidential data. As such, there have been eight generalized methods behind code obfuscation, namely: (i) Name obfuscation; (ii) Data obfuscation; (iii) Code flow obfuscation; (iv) Incremental obfuscation; (v) Intermediate code optimization; (vi) Debug information obfuscation; (vi) Watermarking; (vii) Source code obfuscation. Source code obfuscation, the focus of the paper, is the process that hides the meaning behind the source code, in the case that a third party obtains the code that the code renders itself un-understandable. Under each branch of obfuscation method, there are sub-techniques to reduce understandability of code that are shared by the other obfuscation methods, including control ordering (changing the execution order of program statements), control computation (changing the control flow of the program, such as inserting dead code to increase code complexity), data aggregation (changing the access to data structures after their conversion into other types), renamed identifiers (replacing identifiers with meaningless strings as new identifiers to reduce readability and comprehension of what certain functions or methods do). While existing methods tend to require manual altering of source code with the aforementioned methods, the proposed method performs complete obfuscation with relatively randomly generated characters as the code output. Malicious attackers or readers of the code will not be able to reverse-engineer the code based on the readability of the obfuscated code, and a well-lodged key file of the model weights (e.g. kept on the execution server) would prohibit de-obfuscation. Existing methods of evaluating obfuscation techniques have tended towards qualitative surveys of difficulty and time taken to de-obfuscate by students, software engineers or computer scientists. To quantitatively compare the performance of the proposed source code obfuscation method against existing code obfuscation methods, we will modify a framework that has been used to compare obfuscation methods before, but also malleable enough for us to adapt for our specific comparison use case. The four original comparison metrics are: (i) Code potency: This metric arbitrarily computes the degree of obfuscation with traditional complexity measures, with specific focus on control flow and data obfuscation. An example of an implementation would be frequency counts of misleading statements. (ii) Resilience: This metric measures the ability of the obfuscated text to withstand attacks from automated tools. (iii) Stealth: This metric tests the difficulty in manual de-obfuscation by humans. It inherently checks how quickly adversarial de-obfuscators can detect misdirecting statements, correctly interpret supposedly-confusing identifiers, and tests their ability to uncover the logic and process behind the code without prior or minimal knowledge of the program. (iv) Execution cost: This metric measures (i) the incremental time required to perform the obfuscation, and (ii) the incremental time required to execute the obfuscated code. This would be contrasted to the time taken for the original code without obfuscation. Since the proposed obfuscation method generates a ciphertext completely different from the original source text and cannot be reverse-engineered by manual human de-obfuscation, the former two metrics (code potency and resilience) would not be used as comparison metrics for this method. The main metrics for evaluating DeepObfusCode would be stealth and execution time. Neural networks have had recent applications in the field of cryptography and stenography. There has been a recent implementation of style transfer in stenography, where secret images are encrypted within a cover image, and the secret could be obtained by passing the encrypted image through a secondary network. There is another implementation of n-dimensional convolutional neural networks used to encrypt input data. Another implementation involves the use of a multilayer perception (MLP) model to encrypt satellite images, and decrypting using a secondary MLP model. Implementations on neural networks that can execute on encrpyted data has also been a recent development. This indicates a growing interest in encrypting data through the use of neural networks. Unlike prior on data encryption through deep learning techniques, the proposed architecture opens an alternative application, which is to encrypt source code itself through the use of deep learning, then using the generated model file to decrypt and execute the code. RNN Encoder-Decoders or sequence-to-sequence models, trained to predict an output text set from provided input text, have had growing applications in statistical machine translation, grammatical error correction, and text summarization. The notion of taking input sentences and converting them to labeled output sentences with a trained model of weights for character-by-character prediction would be the basis for the proposed obfuscation method. The proposed obfuscation utilizes the advantage of text-based encoder-decoders in character generation and calculating weights to convert one string into another. The architecture behind the obfuscation is first a primary recurrent neural network (RNN) encoderdecoder model with randomly-set weights taking the original code text as an input to generate the obfuscated text. Then a secondary RNN encoder-decoder model is passed two arguments, the generated ciphertext and the original code as the corresponding label, and is trained for a number of iterations to calculate weights to generate the original text, and the weights of the network would be the key generated. This section will discuss the architecture and implementation in further detail. Figure 1: Overview of ciphertext generation. We pass the source code and a character set as inputs to initialize both character sets for the encoder-decoder model, then randomly assign weights to generate the ciphertext, an obfuscated version of the source code. To generate the obfuscated code, we first take the original legible text (source code) and a full character set (a string containing all characters, including letters, numbers and punctuation). We perform text processing for both text strings to prepare the inputs for the encoder and decoder models. First we create unique character sets for both strings, then create two dictionaries for both strings to index each character (key is index and value is character), and create two other dictionaries for both strings to return character given its index (key is character and value is index). Finally we vectorize both strings. We use the source code character set as inputs for the encoder model and both text character sets for the decoder model. We will randomly generate weights for a given number of times n (the randomness index; in our experiment we set n = 10), and set the model weights to be the generated weights array. The variation in the weights array will determine the degree of randomness in the ciphertext. Weights generated will alter the character value generated for each segment of the string, as shown in the Ciphertext Generation function C(p). c refers to the ciphertext (obfuscated code), p refers to the plaintext (source code), N refers to each weight position. f n and w rand are the n-th feature and randomized weight, respectively. Z(p) is a normalization constant that is independent of the weights. The output will be the output sequence, to which we will decode by taking reference from the index-to-character dictionary to convert the output sequence into an output character string and iteratively append characters. The ing output would be the ciphertext. After generating the ciphertext, we use it along with the original source code plaintext to generate a key. We apply the same text processing steps as before, set ciphertext as input into the encoder, and both texts as inputs into the decoder, and train the model for a certain number of iteration counts. For our experiments, we tended to use 2000 iterations, though Early Stopping mechanisms could be used to lower training time for shorter source code; the iteration count should be the number of iterations needed to ensure the output text is executable and identical to the source code text. After the training and model weight setting process is complete, export the encoder and decoder model files in HDF5 format (the key), and export the metadata (the dictionaries of index to char, and char to index) in pickle format. The Key Generation function K(p, c) accepts the arguments c ciphertext and p plaintext, and sets weights while minimizing the loss function. f n and w n are the n-th feature and weight (after training), respectively. Z(c) is a normalization constant that is independent of the weights. w n f n (p|c) + log Z(c) After the ciphertext is decoded from the output sequence, we test if the code is executable; if it fails, we retrain the model (a pre-determined loss threshold that ensures correct execution would facilitate early stopping in iterative training); if it passes, the ciphertext and key pair are retained. During live execution, we have three inputs: the obfuscated code (ciphertext), the key (model files), and the metadata files; function K(c, k) would be modified to accept c ciphertext, and k model file (along with metadata file). For our own experiments, the model and metadata files were separate; for Figure 3: Overview of live execution. To run the obfuscated code on any server or system, one would pass in the obfuscated code into an execution engine that takes the ciphertext and the lodged model files as inputs to execute the withheld code. execution in live systems, it would be possible to combine them into a single file if preferred. When we pass all three through the model container, the output value is executed as soon as it is returned, i.e. Exec(K(c, k)). The evaluate the obfuscation method compared to existing obfuscation techniques, the method will be tested on two aspects, stealth and execution cost. Other parameters such as code potency or resilience would not be applicable to this method, as those comparison metrics require some form of the original code to be preserved; but since this obfuscation method completely regenerates the code base to be indistinguishable from the original code, such testing (e.g. searching for misleading control flow) will not work. The objective of testing for the stealthiness of obfuscated code is to measure the complexity or distance in randomness from the original code. Hence the logic behind our comparison is to first obtain recognized well-obfuscated code and its deobfuscated copy (the original code), obfuscate the original code using our proposed method, then compare the distance in similarity between the original code and each of the obfuscated code. Obfuscated code samples were obtained through the International Obfuscated C Code Contest (IOCCC), and de-obfuscated code samples of the selected obfuscated code was found on forums. The obfuscated samples were winners in the IOCCC contest, signifying high quality, and were manually created by programmers. A total of five obfuscated-deobfuscated paired samples were used for the experiment, which can be found in https://github.com/dattasiddhartha-1/dsource. Obfuscated code of reputable, unbiased quality and its deobfuscated counterpart is generally difficult to obtain in large samples; the aforementioned samples would be used in bench-marking and comparison, but not for testing the inherent properties of the proposed architecture. To test the obfuscation method extensively, for each de-obfuscated sample, we performed the obfuscation to obtain the ciphertext for 100 iterations (to return 500 samples of obfuscated code samples in total, 100 per de-obfuscated code sample). Then we used Levenshtein Distance to measure the distance between two code samples; the distance is the number of deletions, insertions, or substitutions required to transform the source string into the target string. After calculating the Levenshtein Distance values between the original de-obfuscated code and the IOCCC obfuscated code, we calculated the Levenshtein Distance values between the original de-obfuscated code with each respective sample's DeepObfusCode obfuscated code and took the mean Levenshtein Distance for the sample. The are tabulated in Table 1. From the table, we can observe that in general the DeepObfusCode obfuscation has a greater degree of randomness or dissimilarity from the original source text compared to the benchmark IOCCC obfuscated text, with an average magnitude improvement of 1.2614 (the average of the ratio of proposed:benchmark across all samples) and with a standard deviation 0.7176. Set 2 of the experiment indicates the benchmark outperforming the proposed method. Inspection of set 2 reveals a greater proportion of repetitive characters as junk code insertions, which serve to confuse de-obfuscators, but also dilute the similarity (conversely inflate the Levenstein distance) since the proportion of overlapping characters to the total number of characters would be lower. While the total collection indicates a general improvement in dissimilarity, removing set 2 would indicate an average magnitude improvement of 1.5287 and a standard deviation of 0.5352. This aids in justifying that the proposed obfuscation method performs well in the aspect of stealth. The primary focus of the section would be to measure how ciphertext generation time (time to encrypt) and key generation time (time to decrypt) would vary with the length of the source code (plaintext length). The experimental design starts with the random generation of strings varying from length 1 to 4000 (to vary the plaintext length). Then we set loggers to record information regarding the six main variables we would like to test: (i) Length of string input, (ii) Randomness metric (Levenshtein distance), (iii) Execution time for encryption, (iv) Execution time for decryption, (v) Character variation, (vi) Average character length of ciphertext. As execution time depends on the device running the simulation, for reference, the simulation was run on a Python Jupyter notebook, running on Windows 10 with a Nvidia Geforce 1070 GTX graphics card (running at 10% GPU usage level). The plot of ciphertext generation time requirement against length of the source code shows no distinct pattern, while the plot of key generation time requirement against length of the source code shows a linear pattern. This infers that ciphertext generation is not length-dependent and can be executed without significant incremental cost. Since the length of the source code affects the training time required for the same number of iterations (higher length would increase training time per epoch), longer source code would require more time to generate a key, so this obfuscation method may be more suitable for smaller code bases or systems with sufficient computing resources. We can also infer the key generation takes linear time, i.e. time complexity is O(n). Beyond execution cost, this experiment yielded additional information regarding the properties of this obfuscation model. Plotting ciphertext character variation and ciphertext length against plaintext length reveals: (i) the character variation is widely distributed regardless of the length of the plaintext input, which further supports the notion of randomness of ciphertext generation, as the ciphertext is based purely on the randomness in the model weight generation; (ii) the ciphertext length is kept low (on average 72 character length) regardless of the plaintext length, which reduces obfuscated code storage requirements. The notion that the cipher generation algorithm produces short random ciphertexts further implies it would be difficult for malicious actors to reverse-engineer the ciphertext by setting random weights or training a model without a known output text. For the model used in the experiment, the model file consists of 3 layers of arrays: the main layer contains 8 arrays, of which each array has an array-length as represented in, of which each sub-array (except the third and sixth array) contains an array of 1024 values. One would have to randomly generate 32-bit floats for 975,872 values, at least to a proximate range to the actual values before they can generate readable de-obfuscated code, but even then they would need to further generate values to the 8th decimal place if they intend to obtain scrambled text without any lapse in meaning. The correlation matrix in Figure 7 summarizes the relationships between the properties tested in the experiment. This paper presents a novel obfuscation methodology using RNN encoder-decoder models to generate ciphertext from source code and generating and utilizing model weights as keys. Figure 7: Correlation matrix of properties obfuscation methods, it is at least on par in terms of stealth and is expected to outperform for larger code bases in terms of obscurity and readability, and though key generation may take a significant amount of time for larger code bases or require more computational resources, it would be less time-intensive than to manually obfuscate the source code. This would be a good use case application for services that have confidential source code in plaintext but would prefer ciphertext yet require the ability to execute."

A method for active anomaly detection. We present a new layer that can be attached to any deep learning model designed for unsupervised anomaly detection to transform it into an active method.

Represent the short summary for finding the corresponding scientific article (long)

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.