Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Spoken term detection (STD) is the task of determining whether and where a given word or phrase appears in a given segment of speech. Algorithms for STD are often aimed at maximizing the gap between the scores of positive and negative examples. As such they are focused on ensuring that utterances where the term appears are ranked higher than utterances where the term does not appear. However, they do not determine a detection threshold between the two. In this paper, we propose a new approach for setting an absolute detection threshold for all terms by introducing a new calibrated loss function. The advantage of minimizing this loss function during training is that it aims at maximizing not only the relative ranking scores, but also adjusts the system to use a fixed threshold and thus enhances system robustness and maximizes the detection accuracy rates. We use the new loss function in the structured prediction setting and extend the discriminative keyword spotting algorithm for learning the spoken term detector with a single threshold for all terms. We further demonstrate the effectiveness of the new loss function by applying it on a deep neural Siamese network in a weakly supervised setting for template-based spoken term detection, again with a single fixed threshold. Experiments with the TIMIT, WSJ and Switchboard corpora showed that our approach not only improved the accuracy rates when a fixed threshold was used but also obtained higher Area Under Curve (AUC). Spoken term detection (STD) refers to the proper detection of any occurrence of a given word or phrase in a speech signal. Typically, any such system assigns a confidence score to every term it presumably detects. A speech signal is called positive or negative, depending on whether or not it contains the desired term. Ideally, an STD system assigns a positive speech input with a score higher than the score it assigns to a negative speech input. During inference, a detection threshold is chosen to determine the point from which a score would be considered positive or negative. The choice of the threshold represents a trade-off between different operational settings, as a high value of the threshold could cause an excessive amount of false negatives (instances incorrectly classified as negative), whereas a low value of the threshold could cause additional false positives (instances incorrectly classified as positive).The performance of STD systems can be measured by the Receiver Operation Characteristics (ROC) curve, that is, a plot of the true positive (spotting a term correctly) rate as a function of the false positive (mis-spotting a term) rate. Every point on the graph corresponds to a specific threshold value. The area under the ROC curve (AUC) is the expected performance of the system for all threshold values. A common practice for finding the threshold is to empirically select the desired value using a cross validation procedure. In BID2, the threshold was selected using the ROC curve. Similarly, in BID7 BID16 and the references therein, the threshold was chosen such that the system maximized the Actual Term Weighted Value (ATWV) score BID14. Additionally, BID18 claims that a global threshold that was chosen for all terms was inferior to using a term specific threshold BID17.In this paper we propose a new method to embed an automatic adjustment of the detection threshold within a learning algorithm, so that it is fixed and known for all terms. We present two algorithmic implementations of our method: the first is a structured prediction model that is a variant of the discriminative keyword spotting algorithm proposed by BID15 BID20 BID21, and the second implementation extends the approach used for the structured prediction model on a variant of whole-word Siamese deep network models BID9 BID1 BID13. Both of these approaches in their original form aim to assign positive speech inputs with higher scores than those assigned to negative speech inputs, and were shown to have good on several datasets. However, maximizing the gap between the scores of the positive and negative examples only ensures the correct relative order between those examples, and does not fix a threshold between them; therefore it cannot guarantee a correct detection for a global threshold. Our goal is to train a system adjusted to use a global threshold valid for all terms. In this work, we set the threshold to be a fixed value, and adjust the decoding function accordingly. To do so, we propose a new loss function that trains the ranking function to separate the positive and negative instances; that is, instead of merely assigning a higher score to the positive examples, it rather fixes the threshold to be a certain constant, and assigns the positive examples with scores greater than the threshold, and the negative examples with scores less than the threshold. Additionally, this loss function is a surrogate loss function which extends the hinge loss to penalize misdetected instances, thus enhancing the system's robustness. The new loss function is an upper bound to the ranking loss function, hence minimizing the new loss function can lead to minimization of ranking errors, or equivalently to the maximization of the AUC. In the STD task, we are provided with a speech utterance and a term and the goal is to decide whether or not the term is uttered. The term can be provided as a sequence of phonemes or by an acoustic representation given by a speech segment in which the term is known to be uttered. Throughout the paper, scalars are denoted using lower case Latin letters, e.g., x, and vectors using bold face letters, e.g., x. A sequence of elements is denoted with a bar (x) and its length is written as |x|.Formally, the speech signal is represented by a sequence of acoustic feature vectorsx = (x 1, . . ., x T), where each feature vector is d dimensional x t ∈ R d for all 1 ≤ t ≤ T. Note that in our setting the number of frames T is not fixed. We denote by X = (R d) * the set of all finite length sequences over DISPLAYFORM0, where p l ∈ P for all 1 ≤ l ≤ L r and P is the set of phoneme symbols. We denote by P * the set of all finite length sequences over P.A term is a word or a short phrase and is presented to the system either as a sequence of phonemes in the strongly supervised setting or as an acoustic segment containing the term in the weakly supervised setting. We denote the abstract domain of the term representations (as either a phoneme sequence or an acoustic segment) by R. Our goal is to find a spoken term detector, which takes as input a speech segment and a term and returns a binary output indicating whether the term was pronounced in the acoustic segment or not. Most often the spoken term detector is a function that returns a real value expressing the confidence that the target term has been uttered. The confidence score outputted by this function is compared to a threshold, and if the score is above the threshold the term is declared to have been pronounced in the speech segment. Formally, the detector is a function f from X × R to R. The detection threshold is denoted by the scalar θ ∈ R. Usually there is no single threshold for all terms, and it needs to be adjusted after decoding. Our goal in this work is to propose a new method to learn the spoken term detector from a training set of examples, so that the model is adjusted to use a fixed given threshold for all terms. The function f is found from a training set of examples, where each example is composed of two speech segments and a representation of a term. Although the training set contains many different terms, the function f should be able to detect any term, not only those already seen in the training phase. In this section we describe our main idea, whereas in the next sections we propose two implementations: one with a structured prediction model where the training data is fully supervised and the term is given as a phoneme sequence, and the other with a deep learning model where the training data is weakly supervised and the term is given using a segment of speech. Recall that during inference the input to the detector is a speech segment and a term and the output is a confidence that the term was pronounced in the speech segment, which is compared to a threshold. Since the detection threshold is typically not fixed and does depend on the input term, it is often desired to learn the function f such that the confidence of a speech segment that contains the term is higher than the confidence of a speech segment that does not contain the term. Formally, let us consider two sets of speech segments. Denote by X r+ a set of speech segments in which the term r is articulated. Similarly, denote by X r− a set of speech segments in which the term r is not articulated. We assume that term r, and two instancesx + ∈ X r+ andx − ∈ X r− are drawn from a fixed but unknown probability distribution, and we denote by P{π} and E[π] the probability and the expectation of an event π under this distribution. The probability that the confidence ofx + is higher than the confidence ofx − is the area under the ROC curve (AUC) BID10 BID0: DISPLAYFORM0 Instead of keeping a threshold for each term, we adjust f so that the detection threshold will be fixed, and set to a predefined value. Assume that the predefined threshold is θ, then the accuracy in the prediction can be measured by DISPLAYFORM1 where ∧ is the logical conjunction symbol. Hence our goal is to find the parameters of the function f so as to maximize the accuracy Acc θ for a given threshold θ. Equivalently we find the parameters of function f that minimize the error defined as DISPLAYFORM2 where ∨ is the logical disjunction symbol, and I{π} is the indicator function, that equals 1 if the predicate π holds true and 0 otherwise. Unfortunately, we cannot minimize the error function directly, since it is a combinatorial quantity. A common practice is to replace the error function with a surrogate loss function which is easy to minimize. We suggest to minimize a convex upper-bound to the error function. Specifically, we replace the last term with the hinge upper bound, DISPLAYFORM3 where [π] + = max{π, 0}. The last upper bound holds true since I{π < 0} ≤ [1 − π] +. Adding the margin of 1 means that the function f faces a harder problem: not only does it need to have a confidence greater than θ for a positive speech segment and a confidence lower than θ for a negative speech segment -the confidence value must be at least θ + 1 and at most θ − 1 for positive and negative speech segments, respectively. We now turn to present two algorithmic implementations that are aimed at minimizing the loss function derived from, namely, DISPLAYFORM4 Hopefully the minimization of this loss function will lead to the minimization of Err θ in. Our first construction is based on previous work on discriminative keyword spotting and spoken term detection BID15 BID20 BID21, where the goal was to maximize the AUC. In this setting we assume that the term is expressed as a sequence of phonemes denotedp r ∈ P *.In this fully-supervised setting we define the alignment between a phoneme sequence and a speech signal. We denote by y l ∈ N the start time of phoneme p l (in frame units), and by e l = y l+1 − 1 the end time of phoneme p l, except for the phoneme p L, where the end frame is e L. The alignment sequenceȳ r corresponding to the phonemes sequencep r is a sequence of start-times and an end-time, DISPLAYFORM0, where y l is the start-time of phoneme p l and e L is the end-time of the last phoneme p L.Similar to previous work BID15 BID20 BID21, our detection function f is composed of a predefined set of n feature functions, {φ j} n j=1, each of the form φ j: X * × P * × N * → R. Each feature function takes as input an acoustic representation of a speech utterancex ∈ X *, together with the term phoneme sequencep r ∈ P *, and a candidate alignment sequenceȳ r ∈ N *, and returns a scalar in R which represents the confidence in the suggested alignment sequence given the term r. For example, one element of the feature function can sum the number of times phoneme p comes after phoneme p, while other elements of the feature function may extract properties of each acoustic feature vector x t provided that phoneme p is pronounced at time t. Our basic set of feature functions is the same as the set used in BID15.We believe that the threshold value for each term depends on the term's phonetic content and its relative duration. In order to allow f to learn these subtle differences from the data we introduced an additional set of 4 feature functions: a feature function representing a bias; a feature function that counts the number of occurrences of a phoneme in a term, i.e., |{q|q ∈p r}|; a feature function holding the number of phonemes in the term, i.e., |p r |; and a feature function holding the average length of the phonemes in a term, i.e., DISPLAYFORM1 As mentioned above, our goal is to learn a spoken term detector f, which takes as input a sequence of acoustic featuresx, a termp r, and returns a confidence value in R. The form of the function f we use is f (x,p r) = max DISPLAYFORM2 where w ∈ R n is a vector of importance weights that should be learned and φ ∈ R n is a vector function composed out of the feature functions φ j. In other words, f returns a confidence prediction about the existence of the term in the utterance by maximizing a weighted sum of the scores returned by the feature function elements over all possible alignment sequences. If the confidence of the function f is above the threshold θ then we predict that the term is pronounced in the signal and located in the time span defined by the alignment sequenceȳ that maximizes: DISPLAYFORM3 where the search for the best sequence is practically performed using the Viterbi algorithm as described in BID15. Specifically, the algorithm finds the optimal time segment for the keyword r in the speech signalx, and then aligns the phoneme sequencep r within the chosen time segment. The parameters of the model w are found by minimizing the loss function defined in. In the fully supervised case we use a slightly modified version of it, which is defined as DISPLAYFORM4 This is a convex function in the vector of the parameters w. We use the Passive-Aggressive (PA) algorithm BID5 BID15 to find the parameters w. The algorithm receives as input a set of training examples DISPLAYFORM5 and examines each of them sequentially. Initially, we set w = 0. At each iteration i, the algorithm updates w according to the current example (p ri,x DISPLAYFORM6 Denote by w i−1 the value of the weight vector before the ith iteration. We set the next weight vector w i to be the minimizer of the following optimization problem, DISPLAYFORM7 where C serves as a complexity-accuracy trade-off parameter and ξ is a non-negative slack variable, which indicates the loss of the ith example. Intuitively, we would like to minimize the loss of the current example, i.e., the slack variable ξ, while keeping the weight vector w as close as possible to our previous weight vector w i−1 . The constraint makes the projection of the utterance in which the term is uttered onto w greater than θ + 1, and the projection of the utterance in which the term is not uttered onto w less than θ − 1. The closed form solution to the above optimization problem can be derived using the Karush-Kuhn-Tucker conditions in the same lines of [6, App. A].The loss in is composed of two hinge functions and therefore introduces a more elaborate solution than the one derived for the ranking loss of BID15. We call this algorithm PA-ACC (Passive-Aggressive to maximize Accuracy). Details about the implementation of the algorithm can be seen in BID8. The PA-ACC algorithm is an online algorithm, and deals with drifting hypotheses; therefore, it is highly influenced by the recent examples. Common methods to convert an online algorithm to a batch algorithm are either by taking the average over all the parameters {w i}, or by taking the best w i over a validation set BID6 BID3. We turn to exemplify our idea in the weakly supervised setting using deep networks. This implementation is based on recent work on whole-word segmental systems BID9 BID1 BID13. These works present a Siamese network model trained with a ranking loss function. Siamese networks BID4 are neural networks with a tied set of parameters which take as input a pair of speech segments and are trained to minimize or maximize the similarity between the segments depending on whether the same term has been pronounced in the pair of segments. In this setting the term r is represented by two speech segments rather than a phoneme sequence: a speech segment in which the term r is pronounced,x +, and a speech segment, in which the term r is not pronounced,x −. Similar to those works, we assume that each example in the training set is composed of the triplet (x t,x +,x −), wherex t,x + ∈ X r+ andx − ∈ X r−. The goal in training the network is that the similarity score betweenx t andx + should be above the similarity score between DISPLAYFORM0 Denote by g u: X * → R d a deep network (the specific architecture is discussed in Section 6) with a set of parameters u, where d is the dimension of the output. Denote by ρ: R d × R d → R a measure of similarity between two output vectors of size d. The spoken term detector f u: X * × X * → R is the composition of Siamese networks g u and the similarity function. Hence an unknown speech segmentx t can be compared to a positive or negative speech segment, as follows: DISPLAYFORM1 The tied parameters u of all the models were found in BID9 BID1 BID13 using the minimization of the ranking loss function DISPLAYFORM2 for different options of the similarity function ρ. In this work we propose to minimize the loss function in FORMULA5, which is defined for the weakly supervised case as follows: DISPLAYFORM3 when the margin of γ > 0 is used. In this case, the parameter γ is not set to 1, since the function f u is not a linear function and hence is not scale invariant to the margin, as in the structured prediction case. In the next section we present our empirical comparison for all the loss functions on different speech corpora. In this section we present experimental that demonstrate the effectiveness of our proposed calibrated loss function BID6. We compared the proposed loss to the standard approach of maximizing AUC using the ranking loss as in BID11 where no fixed threshold can be set. The experiments on the structured prediction model were conducted using fully supervised training sets of read speech (TIMIT, WSJ). The experiments on the deep network model performed on a weakly supervised data of conversational speech (Switchboard). To validate the effectiveness of the proposed approach, we performed experiments with the TIMIT corpus. The training and validation sets were taken from the TIMIT training set. The training set was composed from 1,512 randomly chosen terms, corresponding to 11,139 pairs of positive and negative utterances (each term repeated more than once). Similarly, the validation set was composed from 378 different randomly chosen terms, corresponding to 2,892 pairs. The validation set was used to tune the algorithm's parameters. The test set was composed of 80 terms that were suggested as a benchmark in BID15, and are distinct from the terms used in the training and validation sets. For each term, we randomly picked at most 20 utterances in which the term was uttered and at most 20 utterances in which it was not uttered. The utterances were taken from the TIMIT test set. The number of test utterances in which the term was uttered was not always 20, since some terms were uttered less than 20 times in the whole TIMIT test set. We measure performance using the AUC defined in and using the accuracy of a fixed threshold θ denoted Acc θ. Specifically, we calculate AUC on the test set of m test examples according to DISPLAYFORM0 and the accuracy by DISPLAYFORM1 We tested the PA-ACC algorithm using two options. The first was whether the final weight vector was a of averaging or was the best to perform on the validation set. The second option was whether the new feature functions we introduced were normalized by the length of the phoneme sequence |p r | or not. The AUC and Acc θ rates found on our validation and test set are presented in Table 1. In training PA-ACC we chose arbitrarily θ = 0. Table 1: AUC and ACC θ rates of the PA-ACC algorithm. The first column indicates whether the new feature functions were normalized or not. The second column indicates whether the final weight vector was a of averaging or was the best to perform on the validation set. We can see from the table that since the TIMIT dataset is very clean the detection rates are very good and the AUC is almost always 1. The presented here are improved over the presented in BID15 due to the introduction of the new feature functions. It is interesting to note that the best Acc 0 on the validation set were obtained when the additional features were not normalized and the final weight vector was selected over the validation set, while the best Acc 0 on the test set were obtained with the opposite conditions: when the final weight vector was the average one and the additional feature functions were normalized. Further research on feature functions should be conducted and extended to a larger dataset. We now turn to compare the performance of our algorithm against two other algorithms. The first is the discriminative keyword spotting algorithm presented in BID15, which is the Passive-Aggressive algorithm trained with the ranking loss to maximize the AUC. It is denoted here as PA-AUC. We introduce two versions of this algorithm: the original version and an extended version with the additional set of feature functions described in Sec. 4. When using the extended version of PA-AUC, normalizing the features had no affect on our . Similarly, a comparison of using the final weight vector versus the best weight vector yielded similar outcomes. The second algorithm is an HMM-based spoken term detection algorithm presented in BID15. BID0 For all the algorithms we report the AUC and Acc θ in TAB1. For the two versions of PA-AUC we selected a single threshold θ that gave the best Acc θ on the validation set. Similarly we selected the best threshold for the HMM algorithm. For PA-ACC we arbitrarily selected θ = 0. It is interesting to see that the AUC of PA-ACC is the same or even higher than that of the PA-AUC. Since Acc θ is a lower bound to AUC, the AUC can be thought of as Acc θ with the best threshold selected for every term in the set. Indeed from TAB1 we see that the Acc θ was very close to the AUC but did not reach it. We evaluate the model trained on TIMIT on the Wall Street Journal (WSJ) corpus BID19. This corpus corresponds to read articles of the Wall Street Journal, and hence presents a different linguistic context compared to TIMIT. Both the discriminative system and the HMM-based system were trained on the TIMIT corpus as described above and evaluated on a different set of 80 keywords from the WSJ corpus. For each keyword, we randomly picked at most 20 utterances in which the keyword was uttered and at most 20 utterances in which it was not uttered from the si_tr_s portion of the WSJ corpus. We used the same setting as in BID15. As before we arbitrarily chose the threshold θ = 0. The are presented in TAB2. Again we see from TAB2 that the model trained with the proposed loss function led to higher accuracy rates with similar AUC rates, meaning a better separation between the positive speech utterances and the negative speech utterances. Our second set of experiments is focused on deep networks trained on weakly supervised data. Our model is based on previous work on network training using the ranking loss BID9 BID1 BID13. We used the same experimental setup as BID13 BID13. The term weak supervision BID13, refers to the fact that supervision is given in the form of known word pairs, rather than the exact location of the term and its phonetic content as in Subsection 6.1.The data was taken from the Switchboard corpus of English conversational telephone speech. Melfrequency cepstral coefficients (MFCCs) with first and second order derivatives features were extracted and cepstral mean and variance normalization (CMVN) was applied per conversation side. The training set consisted of the set of about 10k word tokens from BID11 BID12; it consisted of word segments of at least 5 characters and 0.5 seconds in duration extracted from a forced alignment of the transcriptions, and comprises about 105 minutes of speech. For the Siamese convolutional neural networks (CNNs), this set in about 100k word segment pairs. For testing, we used the 11k-token set from BID11 BID12.The architecture of each network was the same as BID13 BID13: 1-D convolution with 96 filters over 9 frames; ReLU (Rectified Linear Unit); max pooling over 3 units; 1-D convolution with 96 filters over 8 units; ReLU; max pooling over 3 units; 2048-unit fully-connected ReLU; 1024-unit fully-connected linear layer. All weights were initialized randomly. Models were trained using ADADELTA BID22.We reproduced the in BID13 by training the Siamese network using the ranking loss in with the cosine similarity as a similarity function ρ. The cosine similarity of two vectors v 1 ∈ R d and v 2 ∈ R d is defined as DISPLAYFORM0 where this function returns a number close to 1 if the two vectors are similar and a number close to -1 if the two vectors are not. We also train the network using the same similarity function using the Acc θ loss function with θ = 0 as in BID12. For the ranking loss we used γ = 0.15 while for the Acc θ loss we used γ = 0.10, because the margin γ is counted twice for Acc θ loss. These values were chosen by maximizing Acc θ on a validation set over 5 epochs. The AUC and Acc θ values for training of 5 to 30 epochs are given in TAB3. Other training parameters and settings were exactly the same as in BID13. We can see in the table that both AUC and Acc θ are higher when the system is trained with the calibrated ranking loss function. The reason that the AUC was also improved is most likely because the calibrated ranking loss function is harder to optimize than the ranking loss. In this work, we introduced a new loss function that can be used to train a spoken term detection system with a fixed desired threshold for all terms. We introduced a new discriminative structured prediction model that is based on the Passive-Aggressive algorithm. We show that the new loss can be used in training weakly supervised deep network models as well. Results suggest that our new loss function yields AUC and accuracy values that are better than previous works' .

represent the tldr for finding the corresponding scientific article

Spoken Term Detection, using structured prediction and deep networks, implementing a new loss function that both maximizes AUC and ranks according to a predefined threshold.

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.


Question: represent the article for finding a single-sentence tldr

Question: Federated learning involves jointly learning over massively distributed partitions of data generated on remote devices. Naively minimizing an aggregate loss function in such a network may disproportionately advantage or disadvantage some of the devices. In this work, we propose q-Fair Federated Learning (q-FFL), a novel optimization objective inspired by resource allocation strategies in wireless networks that encourages a more fair accuracy distribution across devices in federated networks. To solve q-FFL, we devise a scalable method, q-FedAvg, that can run in federated networks. We validate both the improved fairness and flexibility of q-FFL and the efficiency of q-FedAvg through simulations on federated datasets. With the growing prevalence of IoT-type devices, data is frequently collected and processed outside of the data center and directly on distributed devices, such as wearable devices or mobile phones. Federated learning is a promising learning paradigm in this setting that pushes statistical model training to the edge .The number of devices in federated networks is generally large-ranging from hundreds to millions. While one can naturally view federated learning as a multi-task learning problem where each device corresponds to a task , the focus is often instead to fit a single global model over these distributed devices/tasks via some empirical risk minimization objective . Naively minimizing the average loss via such an objective may disproportionately advantage or disadvantage some of the devices, which is exacerbated by the fact that the data are often heterogeneous across devices both in terms of size and distribution. In this work, we therefore ask: Can we devise an efficient optimization method to encourage a more fair distribution of the model performance across devices in federated networks?There has been tremendous recent interest in developing fair methods for machine learning. However, current methods that could help to improve the fairness of the accuracy distribution in federated networks are typically proposed for a much smaller number of devices, and may be impractical in federated settings due to the number of involved constraints BID5. Recent work that has been proposed specifically for the federated setting has also only been applied at small scales (2-3 groups/devices), and lacks flexibility by optimizing only the performance of the single worst device .In this work, we propose q-FFL, a novel optimization objective that addresses fairness issues in federated learning. Inspired by work in fair resource allocation for wireless networks, q-FFL minimizes an aggregate reweighted loss parameterized by q such that the devices with higher loss are given higher relative weight to encourage less variance in the accuracy distribution. In addition, we propose a lightweight and scalable distributed method, qFedAvg, to efficiently solve q-FFL, which carefully accounts for important characteristics of the federated setting such as communication-efficiency and low participation of devices BID3 ). We empirically demonstrate the fairness, efficiency, and flexibility of q-FFL and q-FedAvg compared with existing baselines. On average, q-FFL is able to reduce the variance of accuracies across devices by 45% while maintaining the same overall average accuracy. Fairness in Machine Learning. There are several widespread approaches in the machine learning community to address fairness, which is typically defined as the protection of some specific attribute(s) (e.g., ). In addition to preprocess the data and post-process the model BID8 ), another set of works optimize an objective under some explicit fairness constraints during training time BID2 BID5;;; BID8. Our work also enforces fairness during training, though we define fairness as the accuracy distribution across devices in federated learning, as opposed to the protection of a specific attribute (Section 3). BID5 use a notion of'minimum accuracy' as one special case of the rate constraints, which is conceptually similar to our goal. However, it requires each device to have one constraint, which is not practical in the federated setting. In federated settings, proposes a minimax optimization scheme, Agnostic Federated Learning (AFL), which optimizes for the performance of the single worst device. This method has only been applied at small scales (for a handful of groups). In addition, our objective is more flexible because q may be tuned based on the amount of fairness desired. Fairness in Resource Allocation. Fair resource allocation has been extensively studied in fields such as network management BID6;; ) and wireless communications BID7;; ). In these contexts, the problem is defined as allocating a scarce shared resource, e.g. communication time or power, among many users. In these cases directly maximizing utilities such as total throughput usually leads to unfair allocations where some users receive poor service. Several measurements have been proposed to balance between fairness and total throughput. Among them, a unified framework is captured through α-fairness , in which the emphasis on fairness can be tuned by changing a single parameter, α. If we think of the global model as a resource to serve the users (or devices), it is natural to ask similar questions about the fairness of the service that devices receive and use similar tools to promote fairness. Despite this, we are unaware of any work that uses fairness criteria from resource allocation to modify training objectives in machine learning. Inspired by the α-fairness metric, we propose a similarly modified objective function, q-Fair Federated Learning (q-FFL), to encourage a more fair accuracy distribution across devices in the context of federated training. We empirically demonstrate its benefits in Section 4.Federated and Distributed Optimization. Federated learning faces fundamental challenges such as expensive communication, variability in hardware, network connection, and power of devices, and heterogeneous local data distribution amongst devices, making it distinct from classical distributed optimization (; ; BID4 . In order to reduce communication, as well as to tolerate heterogeneity, methods that allow for local updating and low participation among devices have become de facto solvers for this setting BID4 ;). We incorporate recent advancements in this field when designing methods to solve the q-FFL objective, which we describe in Section 3.3. We first formally define the classical federated learning objective and methods, and introduce our proposed notion of fairness in Section 3.1. We then introduce q-FFL, a novel objective that encourages a more fair accuracy distribution across all devices (Section 3.2). Finally, in Section 3.3, we describe q-FedAvg, an efficient distributed method we develop to solve the objective in federated settings. Federated learning involves fitting a global model on distributed data generated on hundreds to millions of remote devices. In particular, the goal is to minimize: DISPLAYFORM0 where m is the total number of devices, p k ≥ 0, and DISPLAYFORM1 The local objective F k's can be defined by empirical risks over local data, i.e., DISPLAYFORM2 where n k is the number of samples available locally. We can set p k to be n k n, where n = k n k is the total number of samples in the entire dataset. Most prior work solves by first subsampling devices with probabilities proportional to n k at each round, and then applying an optimizer such as Stochastic Gradient Descent (SGD) locally to perform updates. These local updating methods enable flexible and efficient communication by running the optimizer for a variable number of iterations locally on each device, e.g., compared to traditional distributed (stochastic) gradient descent, which would simply calculate a subset of the gradients (; ; ;). FedAvg (Algorithm 2, Appendix A) is one of the leading methods to solve.However, solving in this manner can implicitly introduce unfairness among different devices. For instance, the learned model may be biased towards the devices with higher number of data points. Formally, we define our desired fairness criteria for federated learning below. Definition 1 (Fairness of distribution). For trained models w andw, we say that model w provides a more fair solution to Objective than modelw if the variance of the performance of model w on the m devices, {a 1, . . . a m}, is smaller than the variance of the performance of modelw on the m devices, i.e., Var(a 1, . . ., a m) ≤ Var(ã 1, . . .,ã m).In this work, we take'performance' for device k, a k, to be the testing accuracy of applying the trained model w on the test data for device k. Our goal is to reduce the variance while maintaining the same (or similar) average accuracy. A natural idea to achieve fairness as defined in would be to reweight the objective-assigning higher weight to devices with poor performance, so that the distribution of accuracies in the network reduces in variance. Note that this re-weighting must be done dynamically, as the performance of the devices depends on the model being trained, which cannot be evaluated a priori. Drawing inspiration from α-fairness, a utility function used in fair resource allocation in wireless networks, we propose the following objective q-FFL. For given local non-negative cost functions F k and parameter q > 0, we define the overall q-Fair Federated Learning (q-FFL) objective as DISPLAYFORM0 Intuitively, the larger we set q, the larger relative price we pay for devices k with high local empirical loss, DISPLAYFORM1 Here, q is a tunable parameter that depends on the amount of fairness we wish to impose in the network. Setting q = 0 does not encourage any fairness beyond the classical federated learning objective. A larger q means that we emphasize devices with higher losses (lower accuracies), thus reducing the variance between the accuracy distribution and potentially inducing more fairness in accordance with Definition 1.3.3. The solver: FedAvg-style q-Fair Federated Learning (q-FedAvg)We first propose a fair but less efficient method q-FedSGD, to illustrate the main techniques we use to solve the q-FFL objective. We then provide a more efficient counterpart q-FedAvg, by considering key properties of federated algorithms such as local updating schemes. Hyperparameter tuning: q and step-sizes. In devising a method to solve q-FFL, we begin by noting that it is important to first determine how to set q. In practice, q can be tuned based on the desired amount of fairness. It is therefore common to train a family of objectives for different q values so that a practitioner can explore the tradeoff between accuracy and fairness for the application at hand. Nevertheless, to optimize q-FFL in a scalable fashion, we rely on gradient-based methods, where the stepsize inversely depends on the Lipchitz constant of the function's gradient, which is often unknown and selected via grid search BID10 ). As we intend to optimize q-FFL for various values of q, the Lipchitz constant will change as we change q-requiring step-size tuning for all values of q. This can quickly cause the search space to explode. To overcome this issue, we propose estimating the local Lipchitz constant of the gradient for the family of q-FFL by using the Lipchitz constant we infer on q = 0. This allows us to dynamically adjust the step-size for the q-FFL objective, avoiding the manual tuning for each q. In Lemma 2 we formalize the relation between the Lipschitz constant, L, for q = 0 and q > 0.Lemma 2. If the non-negative function f (·) has a Lipchitz gradient with constant L, then for any q ≥ 0 and at any point w, DISPLAYFORM2 is an upper-bound for the local Lipchitz constant of the gradient of DISPLAYFORM3 Furthermore, the gradient DISPLAYFORM4 See proof in Appendix B.A first approach: q-FedSGD. In our first fair algorithm qFedSGD, we solve Objective using mini-batch SGD on a subset of devices at each round, and apply the above to each selected device to obtain local Lipchitz constants for gradients of local functions F k. By averaging those estimates, we obtain an estimate for the Lipchitz constant for the gradient of q-FFL. Then, the step-size (inverse of this estimate) is applied, like other gradient based algorithms; see Algorithm 3 in Appendix A for more details. Algorithm 1 q-FedAvg (proposed method) DISPLAYFORM5 Server chooses a subset S t of K devices at random (each device k is chosen with prob. p k) Server sends w t to all chosen devices 5:Each device k updating w t for E epochs of SGD with step size η to obtainw to run some number of local updates and then apply the updates in the gradient computation of q-FFL. The details of our method (q-FedAvg) are given in Algorithm 1. Note that when q=0, q-FFL corresponds to the normal objective in federated learning (Equation FORMULA0), and q-FedAvg is also reduced to FedAvg where no fairness modification is introduced. We first describe our experimental setup, then demonstrate the improved fairness of the q-FFL objective by comparing q-FFL with several baselines, and finally show the efficiency of q-FedAvg compared with q-FedSGD.Experiment setups. We explore both convex and nonconvex models on four federated datasets curated from prior work in federated learning (; BID4 BID4 . Full details of the datasets and models are given in Appendix D. Throughout the experiments, we show on the Vehicle dataset. Similar on all datasets are provided in Appendix C. Fairness of q-FFL. We verify that the proposed objective q-FFL leads to more fair solutions (according to Definition 1) for federated data, compared with FedAvg and two other baselines that are likely to impose fairness in federated networks. Compare with FedAvg. In Figure 1 (left), we compare the final testing accuracy distributions of two objectives (q=0 and a tuned value of q=5) averaged across 5 random shuffles of Vehicle. We observe that the objective with q=5 in more centered (i.e., fair) testing accuracy distributions with lower variance. We further report the worst and best 10% testing accuracies and the variances of accuracies in Table 1. We see that the average testing accuracy remains almost unchanged with the proposed objective despite significant reductions in variance. See similar on other datasets in Figure 2 and TAB2 in Appendix C. Compare with weighing each device equally. We compare q-FFL with a heuristic that samples devices uniformly and report testing accuracy in Figure 1 (middle). A table with the statistics of accuracy distribution on all datasets is given in the appendix in Table 3. While the'weighing each device equally' heuristic tends to outperform our method in training accuracy distributions (Figure 5 and TAB8 in Appendix D.3), our method produces more fair solutions in terms of testing accuracies. One explanation for this is that uniform sampling is a static method and can easily overfit to devices with very few data points, whereas q-FFL has better generalization properties due to its dynamic nature. FORMULA0 ). Middle: Fairness of q-FFL q>0 compared with the uniform sampling baseline. Right: q-FedAvg converges faster than q-FedSGD. Table 1. Statistics of the testing accuracy distribution for q-FFL on Vehicle. By setting q > 0, the variance of the final accuracy distribution decreases, and the worst 10% accuracy increases, while the overall accuracy remains fairly constant. Avg. Worst 10% Best 10% Var. q = 0 87.3% 43.0%95.7% 291 q = 5 87.7% 69.9%94.0% 48performance of the device with the highest loss. This is the only work we are aware of that aims to address fairness issues in federated learning. See Appendix D.2 for details of our AFL implementation. We also observe that q-FFL outperforms AFL when q is set appropriately TAB4, Appendix D). We note that q is also tunable depending on the amount of fairness desired. Interestingly, we observe q-FFL converges faster compared with AFL (see Figure 7 in Appendix D.3) in terms of communication rounds. Choosing q. A natural question is determine how q should be tuned in the q-FFL objective. The framework is flexible in that it allows one to choose q to tradeoff between reduced variance of the accuracy distribution and a high average accuracy. In particular, a reasonable approach in practice would be to run Algorithm 1 with multiple q's in parallel to obtain multiple final global models, and then let each device select amongst these based on performance on the validation data. We show benefits of this device-specific strategy in TAB9 in Appendix D.3.Efficiency of q-FedAvg. Finally, we show the efficiency of q-FedAvg by comparing Algorithm 1 with its nonlocal-updating baseline q-FedSGD (Algorithm 3) with the same objective (q > 0). At each communication round, q-FedAvg runs one epoch of local updates on each selected device, while q-FedSGD runs gradient descent using all local training data on that device. In Figure 1 (We summarize the FedAvg algorithm proposed in below. Server chooses a subset S t of K devices at random (each device k is chosen with probability p k) Server sends w t to all chosen devices Each device k updates w t for E epochs of SGD on F k with step-size η to obtain w t+1 kEach chosen device k sends w t+1 k back to the server Server aggregates the w's as w DISPLAYFORM0 We summarize our proposed method q-FedSGD below. Algorithm 3 q-FedSGD DISPLAYFORM1 Server chooses a subset S t of K devices at random (each device k is chosen with prob. p k) Server sends w t to all chosen devices 5:Each device computes: DISPLAYFORM0 Each chosen device k sends ∆ Server aggregates the computes w t+1 as: DISPLAYFORM1 Proof. At any point w, we can compute ∇ 2 f (w) DISPLAYFORM0 As a , DISPLAYFORM1 Fairness of q-FFL. We demonstrate the improved fairness of q-FFL on all the four datasets in Figure 2 and TAB2.Comparison with uniform sampling. We compare q-FFL with uniform sampling schemes and report testing accuracy on all datasets in Figure 3. A table with the final accuracies and variances is given in Table 3. While the'weighing each device equally' heuristic tends to outperform our method in training accuracy distributions (Figure 5 and TAB8), our Figure 2. q-FFL leads to fairer test accuracy distributions. With q > 0, the distributions shift towards the center as low accuracies increase at the cost of decreasing high accuracies on some devices. Setting q=0 corresponds to the original objective (Equation FORMULA0). The selected q values for q > 0 on the four datasets, as well as distribution statistics, are shown in TAB2. Figure 3. q-FFL (q > 0) compared with uniform sampling. In terms of testing accuracy, our objective produces more fair solutions than uniform sampling. Distribution statistics are provided in Table 3. Table 3. More statistics indicating the ing fairness of q-FFL compared with the uniform sampling baseline. Again, we observe that the testing accuracy of the worst 10% devices tends to increase, and the variance of the final testing accuracies is smaller. Figure 4. Fix an objective (i.e., using the same q) for each dataset, q-FedAvg (Algorithm 1) compared with q-FedSGD (Algorithm 3). We can see that our method adopting local updating schemes converges faster in terms of communication rounds on most datasets.method produces more fair solutions in terms of testing accuracies. One explanation for this is that uniform sampling is a static method and can easily overfit to devices with very few data points, whereas q-FFL has better generalization properties due to its dynamic nature. Comparison with weighing each device adversarially. We show the of comparing q-FFL with AFL on the two datasets in TAB4. q-FFL outperforms AFL in terms of increasing the lowest accuracies. In addition, q-FFL is more flexible as the parameter q enables the trade-off between increasing the worst accuracies and decreasing the best accuracies. Efficiency of q-FedAvg. In Figure 4, we show that on most datasets, q-FedAvg converges faster than q-FedSGD in terms of communication rounds due to its local updating scheme. We note here that number of rounds is a reasonable metric for comparison between these methods as they process the same amount of data and perform equivalent amount of communication at each round. Our method is also lightweight, and can be easily integrated into existing implementations of federated learning algorithms such as TensorFlow Federated (TFF). We provide full details on the datasets and models used in our experiments. The statistics of four federated datasets are summarized in TAB6. We report total number of devices, total number of samples, and mean and deviation in the sizes of total data points on each device. Additional details on the datasets and models are described below.• Synthetic: We follow a similar set up as that in and impose additional heterogeneity. The model is y = argmax(softmax(W x + b)), x ∈ R 60, W ∈ R 10×60, b ∈ R 10, and the goal is to learn a global W and b. Samples (X k, Y k) and local models on each device k satisfies DISPLAYFORM0, where the covariance matrix Σ is diagonal with Σ j,j = j −1.2. Each element in v k is drawn from TAB2 Fair Resource Allocation in Federated Learning N (B k, 1), B k ∼ N. There are 100 devices in total and the number of samples on each devices follows a power law.• Vehicle 1: We use the same Vehicle Sensor (Vehicle) dataset as , modelling each sensor as a device. Each sample has a 100-dimension feature and a binary label indicating whether this sample is on an AAV-type or DWtype vehicle. We train a linear SVM. We tune the hyperparameters in SVM and report the best configuration.• Sent140: It is a collection of tweets from Sentiment140 BID10 ) (Sent140). The task is text sentiment analysis which we model as a binary classification problem. The model takes as input a 25-word sequence, embeds each word into a 300-dimensional space using pretrained Glove , and outputs a binary label after two LSTM layers and one densely-connected layer.• We implement a non-stochastic version of AFL where all devices are selected and updated each round and do a grid search on the AFL hyperparameters, γ w and γ λ. In order to draw a fair comparison, we modify Algorithm 1 by sampling all devices and letting each of them run gradient descent at each round. We use the same public datasets (Adult and Fashion MNIST) as in . We randomly split data on each local device into 80% training set, 10% testing set, and 10% validation set. We tune q from {0.001, 0.01, 0.1, 1, 2, 5, 10, 15} on the validation set and report accuracy distributions on the testing set. For each dataset TAB2 Fair Resource Allocation in Federated Learning Table 6. Average testing accuracy under q-FFL objectives. We show that the ing solutions of q=0 and q¿0 objectives have approximately the same accuracies both with respect to all data points and with respect to all devices. Objective Accuracy w.r. Figure 5. q-FFL (q > 0) compared with uniform sampling in training accuracy. We see that in most cases uniform sampling has higher (and more fair) training accuracies due to the fact that it is overfitting to devices with few samples.we repeat this process for five randomly selected train/test/validation splits, and report the mean and standard deviation across these five runs where applicable. For Synthetic, Vehicle, Sent140, and Shakespeare, optimal 2 q values are 1, 5, 1, and 0.001 respectively. We randomly sample 10 devices each round. We tune the learning rate on FedAvg and use the same learning rate for all experiments of that dataset. The learning rates for Synthetic, Vehicle, Sent140, and Shakespeare are 0.1, 0.01, 0.03, and 0.8 respectively. When running AFL methods, we search for a best γ w and γ λ such that AFL achieves the highest testing accuracy on the device with the highest loss within a fixed number of rounds. For Adult, we use γ w = 0.1 and γ λ = 0.1; for Fashion MNIST, we use γ w = 0.001 and γ λ = 0.01. We use the same γ w as step sizes for q-FedAvg on Adult and Fashion MNIST. In TAB4, q 1 = 0.01, q 2 = 2 for q-FFL on Adult and q 1 = 5, q 2 = 15 for q-FFL on Fashion MNIST. The number of local epochs is fixed to 1 whenever we do local updates. Average testing accuracy with respect to devices. We have shown that q-FFL leads to more fair accuracy distributions while maintaining approximately the same testing accuracies in Section 4. Note that we report average testing accuracy with respect to all data points in Table 1 and 2. We observe similar on average accuracy with respect to all devices between q = 0 and q > 0 objectives, as shown in Table 6.Efficiency of q-FFL compared with AFL. One added benefit of q-FFL is that it leads to faster convergence than AFL even when we use non-local-updating methods for both objectives. In Figure 7, we show that when fixing the final testing accuracy for the single worst device, q-FFL converges faster than AFL. As the number of devices increases (from Fashion MNIST to Vehicle), the performance gap between AFL and q-FFL becomes larger because AFL introduces larger variance TAB2 Fair Resource Allocation in Federated Learning Figure 6. The convergence speed of q-FFL compared with FedAvg. We plot the distance to the highest accuracy achieved versus communication rounds. Although q-FFL with q > 0 is a more difficult optimization problem, for the q values we choose that could lead to more fair , the convergence speed is comparable to that of q = 0.Choosing q. We solve q-FFL with q ∈ {0, 0.001, 0.01, 0.1, 1, 2, 5, 10} in parallel. After training, each device selects the best ing model based on the validation data and tests the performance of the model using testing set. We report the in terms of testing accuracy in TAB9. Using this strategy, accuracy variance is reduced and average accuracy is increased. However, this will induce more local computation and additional communication load in each round. But this does not increase the number of communication rounds. Convergence speed of q-FFL. In Section 4, we show that our solver q-FedAvg using local updating schemes converges significantly faster than q-FedSGD. A natural question one might ask is: will the q-FFL (q > 0) objective slows the convergence compared with FedAvg? We empirically investigate this on real datasets. We use q-FedAvg to solve q-FFL, and compare it with FedAvg. As demonstrated in Figure 6, the q values we are choosing that in more fair solutions do not significantly slowdown convergence. 52.1% ±.3% 42.1% ± 2.1% 69.0% ± 4.4% 54 ± 27 multiple q's 52.0 ± 1.5% % 41.0% ± 4.3% 72.0% ± 4.8% 72 ± 32

We propose a novel optimization objective that encourages fairness in heterogeneous federated networks, and develop a scalable method to solve it.

Represent the short summary for finding the corresponding scientific article (long).

The paper designs two algorithms for the stochastic AUC maximization problem with state-of-the-art complexities when using deep neural network as predictive model, which are also verified by empirical studies.


Question: Represent the scientific article to find a short TLDR

Question: We propose a novel autoencoding model called Pairwise Augmented GANs. We train a generator and an encoder jointly and in an adversarial manner. The generator network learns to sample realistic objects. In turn, the encoder network at the same time is trained to map the true data distribution to the prior in latent space. To ensure good reconstructions, we introduce an augmented adversarial reconstruction loss. Here we train a discriminator to distinguish two types of pairs: an object with its augmentation and the one with its reconstruction. We show that such adversarial loss compares objects based on the content rather than on the exact match. We experimentally demonstrate that our model generates samples and reconstructions of quality competitive with state-of-the-art on datasets MNIST, CIFAR10, CelebA and achieves good quantitative on CIFAR10. Deep generative models are a powerful tool to sample complex high dimensional objects from a low dimensional manifold. The dominant approaches for learning such generative models are variational autoencoders (VAEs) and generative adversarial networks (GANs) BID12. VAEs allow not only to generate samples from the data distribution, but also to encode the objects into the latent space. However, VAE-like models require a careful likelihood choice. Misspecifying one may lead to undesirable effects in samples and reconstructions (e.g., blurry images). On the contrary, GANs do not rely on an explicit likelihood and utilize more complex loss function provided by a discriminator. As a , they produce higher quality images. However, the original formulation of GANs BID12 lacks an important encoding property that allows many practical applications. For example, it is used in semi-supervised learning, in a manipulation of object properties using low dimensional manifold BID7 and in an optimization utilizing the known structure of embeddings BID11.VAE-GAN hybrids are of great interest due to their potential ability to learn latent representations like VAEs, while generating high-quality objects like GANs. In such generative models with a bidirectional mapping between the data space and the latent space one of the desired properties is to have good reconstructions (x ≈ G(E(x))). In many hybrid approaches BID30 BID34 BID36 BID3 BID33 as well as in VAE-like methods it is achieved by minimizing L 1 or L 2 pixel-wise norm between x and G(E(x)). However, the main drawback of using these standard reconstruction losses is that they enforce the generative model to recover too many unnecessary details of the source object x. For example, to reconstruct a bird picture we do not need an exact position of the bird on an image, but the pixel-wise loss penalizes a lot for shifted reconstructions. Recently, improved ALI model BID8 by introducing a reconstruction loss in the form of a discriminator which classifies pairs (x, x) and (x, G(E(x))). However, in such approach, the discriminator tends to detect the fake pair (x, G(E(x))) just by checking the identity of x and G(E(x)) which leads to vanishing gradients. In this paper, we propose a novel autoencoding model which matches the distributions in the data space and in the latent space independently as in BID36. To ensure good reconstructions, we introduce an augmented adversarial reconstruction loss as a discriminator which classifies pairs (x, a(x)) and (x, G(E(x))) where a(·) is a stochastic augmentation function. This enforces the DISPLAYFORM0 discriminator to take into account content invariant to the augmentation, thus making training more robust. We call this approach Pairwise Augmented Generative Adversarial Networks (PAGANs).Measuring a reconstruction quality of autoencoding models is challenging. A standard reconstruction metric RMSE does not perform the content-based comparison. To deal with this problem we propose a novel metric Reconstruction Inception Dissimilarity (RID) which is robust to content-preserving transformations (e.g., small shifts of an image). We show qualitative on common datasets such as MNIST BID19, CIFAR10 BID17 and CelebA BID21. PAGANs outperform existing VAE-GAN hybrids in Inception Score BID31 and Fréchet Inception Distance BID14 except for the recently announced method PD-WGAN BID10 on CIFAR10 dataset. Let us consider an adversarial learning framework where our goal is to match the true distribution p * (x) to the model distribution p θ (x). As it was proposed in the original paper BID12, the model distribution p θ (x) is induced by the generator G θ: z − → x where z is sampled from a prior p(z). To match the distributions p * (x) and p θ (x) in an adversarial manner, we introduce a discriminator DISPLAYFORM0 It takes an object x and predicts the probability that this object is sampled from the true distribution p * (x). The training procedure of GANs BID12 ) is based on the minimax game of two players: the generator G θ and the discriminator D ψ. This game is defined as follows DISPLAYFORM1 where V (θ, ψ) is a value function for this game. The optimal discriminator D ψ * given fixed generator G θ is DISPLAYFORM2 DISPLAYFORM3 The solution of the reconstruction matching problem ensures that reconstructions G θ (E ϕ (x)) correspond to the source object x up to defined random augmentations a(x). In PAGANs model we introduce the minimax game for training the adversarial distance between the reconstructions and augmentations of the source object x. We consider the discriminator D ψ which takes a pair (x, y) and classifies it into one of the following classes:• the real class: pairs (x, y) from the distribution p * (x)r(y|x), i.e. the object x is taken from the true distribution p * (x) and the second y is obtained from the x by the random augmentation a(x); • the fake class: pairs (x, y) from the distribution DISPLAYFORM0 i.e. x is sampled from p * (x) then z is generated from the conditional distribution q ϕ (z|x) by the encoder E ϕ (x) and y is produced by the generator G ϕ (z) from the conditional model distribution p θ (y|z).Then the minimax problem is DISPLAYFORM1 where DISPLAYFORM2 Let us prove that such minimax game will match the distributions r(y|x) and p θ,ϕ (y|x). At first, we find the optimal discriminator: Proposition 1. Given a fixed generator G θ and a fixed encoder E ϕ, the optimal discriminator D ψ * is DISPLAYFORM3 Proof. Given in Appendix A.1.Then we can prove that given an optimal discriminator the value function V (θ, ϕ, ψ) is equivalent to the expected Jensen-Shanon divergence between the distributions r(y|x) and p θ,ϕ (y|x). Proposition 2. The minimization of the value function V under an optimal discriminator D ψ * is equivalent to the minimization of the expected Jensen-Shanon divergence between r(y|x) and p θ,ϕ (y|x), i.e. DISPLAYFORM4 Proof. Given in Appendix A.2.If r(y|x) = δ x (y) then the optimal discriminator D ψ * (x, y) will learn an indicator I{x = y} as was proved in. As a consequence, the objectives of the generator and the encoder are very unstable and have vanishing gradients in practice. On the contrary, if the distribution r(y|x) is non-degenerate as in our model then the value function V (θ, ϕ, ψ) will be well-behaved and much more stable which we observed in practice. We obtain that for the generator and the encoder we should optimize the sum of two value functions:• the generator's objective: DISPLAYFORM0 • the encoder's objective: DISPLAYFORM1 Draw N samples from the dataset and the prior DISPLAYFORM2 Compute generator loss DISPLAYFORM3 Gradient update on discriminator networks DISPLAYFORM4 Gradient update on generator-encoder networks until convergenceIn practice in order to speed up the training we follow BID12 and use more stable objectives replacing log(1 − D(·)) with − log(D(·)). See Figure 1 for the description of our model and Algorithm 1 for an algorithmic illustration of the training procedure. We can straightforwardly extend the definition of PAGANs model to f -PAGANs which minimize the f -divergence and to WPAGANs which optimize the Wasserstein-1 distance. More detailed analysis of these models is placed in Appendix C. Recent papers on VAE-GAN hybrids explore different ways to build a generative model with an encoder part. One direction is to apply adversarial training in the VAE framework to match the variational posterior distribution q(z|x) and the prior distribution p(z) BID23 or to match the marginal q(z) and p(z) BID22 BID33. Another way within the VAE model is to introduce the discriminator as a part of a data likelihood BID18 BID3. Within the GANs framework, a common technique is to regularize the model with the reconstruction loss term BID5 BID30 BID34.Another principal approach is to train the generator and the encoder BID8 simultaneously in a fully adversarial way. These methods match the joint distributions p * (x)q(z|x) and p θ (x|z)p(z) by training the discriminator which classifies the pairs (x, z). ALICE model introduces an additional entropy loss for dealing with the non-identifiability issues in ALI model. approximated the entropy loss with the cycle-consistency term which is equivalent to the adversarial reconstruction loss. The model of BID27 puts ALI to the VAE framework where the same joint distributions are matched in an adversarial manner. As an alternative, BID34 train generator and encoder by optimizing the minimax game without the discriminator. Optimal transport approach is also explored, BID10 introduce an algorithm based on primal and dual formulations of an optimal transport problem. In PAGANs model the marginal distributions in the data space p * (x) and p θ (x) and in the latent space p(z) and q(z) are matched independently as in BID36. Additionally, the augmented adversarial reconstruction loss is minimized by fooling the discriminator which classifies the pairs (x, a(x)) and (x, G(E(x))). In this section, we validate our model experimentally. At first, we compare PAGAN with other similar methods that allow performing both inference and generation using Inception Score and Fréchet Inception Distance. Secondly, to measure reconstruction quality, we introduce Reconstruction Inception Dissimilarity (RID) and prove its usability. In the last two experiments we show the importance of the adversarial loss and augmentations. For the architecture choice we used deterministic DCGAN 1 generator and discriminator networks provided by pfnet-research 2, the encoder network has the same architecture as the discriminator except for the output dimension. The encoder's output is a factorized normal distribution. Thus DISPLAYFORM0 ϕ (x)I) where µ ϕ, σ ϕ are outputs of the encoder network. The discriminator D(z) architecture is chosen to be a 2 layer MLP with 512, 256 hidden units. We also used the same default hyperparameters as provided in the repository and applied a spectral normalization following BID24. For the augmentation a(x) defined in Section 3 we used a combination of reflecting 10% pad and the random crop to the same image size. The prior distribution p(z) is chosen to be a standard distribution N (0, I). To evaluate Inception Score and Fréchet Inception Distance we used the official implementation provided in tensorflow 1.10.1 BID0.To optimize objectives FORMULA2, FORMULA2, we need to have a discriminator working on pairs (x, y). This can be done using special network architectures like siam networks BID4 or via an image concatenation. The latter approach can be implemented in two concurrent ways: concatenating channel or widthwise. Empirically we found that the siam architecture does not lead to significant improvement and concatenating width wise to be the most stable. We use this configuration in all the experiments. To see whether our method provides good quality samples from the prior, we compared our model to related works that allow an inverse mapping. We performed our evaluations on CIFAR10 dataset since quantitative metrics are available there. Considering Fréchet Inception Distance (FID), our model outperforms all other methods. Inception Score shows that PAGANs significantly better than others except for recently announced PD-WGAN. Quantitative are given in TAB0. For S-VAE we report IS that is reproduced using officially provided code and hyperparameters 3. Plots with samples and reconstructions for CIFAR10 dataset are provided in Figure 2. Additional visual for more datasets can be found in Appendix E.3. BID33 87.7 4.18 ± 0.04 ALI 5.34 ± 0.04 AGE BID34 39.51 5.9 ± 0.04 ALICE 6.02 ± 0.03 S-VAE BID6 6.055 α-GANs BID30 6.2 AS-VAE BID28 6.3 PD-WGAN, λ mix = 0 BID10 33.0 6.70 ± 0.09 PAGAN (ours) 32.84 6.56 ± 0.06 The traditional approach to estimate the reconstruction quality is to compute RMSE distance from source images to reconstructed ones. However, this metric suffers from focusing on exact reconstruction and is not content aware. RMSE penalizes content-preserving transformations while allows such undesirable effect as blurriness which degrades visual quality significantly. We propose a novel metric Reconstruction Inception Dissimilarity (RID) which is based on a pre-trained classification network and is defined as follows: DISPLAYFORM0 where p(y|x) is a pre-trained classifier that estimates the label distribution given an image. Similar to BID31 we use a pre-trained Inception Network BID32 to calculate softmax outputs. Low RID indicates that the content did not change after reconstruction. To calculate standard deviations, we use the same approach as for IS and split test set on 10 equal parts 4. Moreover RID is robust to augmentations that do not change the visual content and in this sense is much better than RMSE. To compare new metric with RMSE, we train a vanilla VAE with resnet-like architecture on CI-FAR10. We compute RID for its reconstructions and real images with the augmentation (mirror 10% pad + random crop). In TAB1 we show that RMSE for VAE is better in comparison to augmented images (AUG), but we are not satisfied with its reconstructions (see Figure 12 in Appendix E.4), Figure 3 provides even more convincing . RID allows a fair comparison, for VAE it is dramatically higher (44.33) than for AUG (1.57). Value 1.57 for AUG says that KL divergence is close to zero and thus content is almost not changed. We also provide estimated RID and RMSE for AGE that was publicly available 5. From TAB1 we see that PAGANs outperform AGE which reflects that our model has better reconstruction quality. To prove the importance of an adversarial loss, we experiment replacing adversarial loss with the 4 Split is done sequentially without shuffling 5 Pretrained AGE: https://github.com/DmitryUlyanov/AGE TAB2. IS and FID suggest that our model without adversarial loss performed worse in generation. Reconstruction quality significantly dropped considering RID. Visual in Appendix E.1 confirm our quantitative findings. In ALICE model an adversarial reconstruction loss was implemented without an augmentation. As we discussed in Section 1 its absence leads to undesirable effects. Here we run an experiment to show that our model without augmentation performs worse. Quantitative provided in TAB2 illustrate that our model without an augmentation fails to recover both good reconstruction and generation properties. Visual comparisons can be found in Appendix E.2. Using the obtained from the last two experiments we conclude that adversarial reconstruction loss works significantly better with augmentation. Experiments checking augmentation effects (see Appendix B for details) conclude the following. A good augmentation: 1) is required to be non-deterministic, 2) should preserve the content of source image, 3) should be hard to use pixel-wise comparison for discriminator. In this paper, we proposed a novel framework with an augmented adversarial reconstruction loss. We introduced RID to estimate reconstructions quality for images. It was empirically shown that this metric could perform content-based comparison of reconstructed images. Using RID, we proved the value of augmentation in our experiments. We showed that the augmented adversarial loss in this framework plays a key role in getting not only good reconstructions but good generated images. Some open questions are still left for future work. More complex architectures may be used to achieve better IS and RID. The random shift augmentation may not the only possible choice, and other smart choices are also possible. A.1 PROOF OF PROPOSITION 1 (OPTIMAL DISCRIMINATOR) Proposition 1. Given a fixed generator G θ and a fixed encoder E ϕ, the optimal discriminator D ψ * is DISPLAYFORM0 Proof. For fixed generator and encoder, the value function V (ψ) with respect to the discriminator is DISPLAYFORM1 Let us introduce new variables and notations DISPLAYFORM2 Then DISPLAYFORM3 Using the of the paper BID12 we obtain DISPLAYFORM4 A.2 PROOF OF PROPOSITION 2Proposition 2. The minimization of the value function V under an optimal discriminator D ψ * is equivalent to the minimization of the expected Jensen-Shanon divergence between r(y|x) and p θ,ϕ (y|x), i.e. DISPLAYFORM5 Proof. As in the paper BID12 we rewrite the value function V (θ, ϕ) for the optimal discriminator D ψ * as follows DISPLAYFORM6 APPENDIX B CHOICE OF AUGMENTATION Augmentation choice might be problem specific. Thereby we additionally study different augmentations and provide an intuition how to choose the right transformation. Theory suggests to pick up a stochastic augmentation. The practical choice should take into account the desired properties of reconstructions. A random shift of an image by a small margin is sufficient to create good quality reconstructions. However, this shift should not be large because it may inherit augmentation artifacts. This can be spotted beforehand just looking at pairs (x, a(x)). Once these pairs are not satisfactory, model reconstructions would be bad as well. In this experiment, we investigate the effects caused by the padding and random crop augmentation. We choose different padding size (comparatively to the original image size) and plot FID, RID and IS metrics. The provide the intuition to choose padding size (see Figure 4). Padding should be chosen to maintain visual content while making impossible to compare augmented and original images by nearly element-wise comparison. Larger padding cause undesirable effects in reconstructions that are captured by RID (see Figure 5). Visual quality of samples, on the other hand is slightly better with more aggressive augmentation considering FID metric, what is explained by more robust training due to less mode collapse problem. We also checked two different augmentation types: Gaussian blur and random contrast (see Figures 6, 7). Both augmentations led to highly unstable training and did not yield satisfactory (IS was 2.15 and 4.18 respectively). Therefore we conclude that a good augmentation is better to change spatial image structure preserving content (as padding does) what will force the discriminator to take content into account. • is required to be non-deterministic • should preserve the content of source image • should be hard to use pixel-wise comparison for discriminator APPENDIX C EXTENDING PAGANS C.1 f -DIVERGENCE PAGANS f -GANs BID26 are the generalization of GAN approach. BID26 introduces the model which minimizes the f -divergence D f BID1 between the true distribution p * (x) and the model distibution p θ (x), i.e. it solves the optimization problem DISPLAYFORM0 where f: R + − → R is a convex, lower-semicontinuous function satisfying f = 0.The minimax game for f -GANs is defined as DISPLAYFORM1 where V (θ, ψ) is a value function and f * is a Fenchel conjugate of f BID25. For fixed parameters θ, the optimal T ψ * (x) is f p * (x) DISPLAYFORM2. Then the value function V (θ, ψ *) for optimal parameters ψ * equals to f -divergence between the distributions p * and p θ BID25, i.e. DISPLAYFORM3

Represent the TLDR for finding the corresponding scientific article

We propose a novel autoencoding model with augmented adversarial reconstruction loss. We intoduce new metric for content-based assessment of reconstructions.

represent this text to find a much much longer article about it with say hundreds-thousands of words:

A backward model of previous (state, action) given the next state, i.e. P(s_t, a_t | s_{t+1}), can be used to simulate additional trajectories terminating at states of interest! Improves RL learning efficiency.