Learning with label proportions (LLP), which is a learning task that only provides unlabeled data in bags and each bag's label proportion, has widespread successful applications in practice. However, most of the existing LLP methods don't consider the knowledge transfer for uncertain data. This paper presents a transfer learning-based approach for the problem of learning with label proportions(TL-LLP) to transfer knowledge from source task to target task where both the source and target tasks contain uncertain data. Our approach first formulates objective model for the uncertain data and deals with transfer learning at the same time, and then proposes an iterative framework to build an accurate classifier for the target task. Extensive experiments have shown that the proposed TL-LLP method can obtain the better accuracies and is less sensitive to noise compared with the existing LLP methods.
@cite_36: Ecological inference (EI) is a classical problem from political science to model voting behavior of individuals given only aggregate election results. recently formulated EI as machine learning problem using distribution regression, and applied it to analyze US presidential elections. However, distribution regression unnecessarily aggregates individual-level covariates available from census microdata, and ignores known structure of the aggregation mechanism. We instead formulate the problem as learning with label proportions (LLP), and develop a new, probabilistic, LLP method to solve it. Our model is the straightforward one where individual votes are latent variables. We use cardinality potentials to efficiently perform exact inference over latent variables during learning, and introduce a novel message-passing algorithm to extend cardinality potentials to multivariate probability models for use within multiclass LLP problems. We show experimentally that LLP outperforms distribution regression for predicting individual-level attributes, and that our method is as good as or better than existing state-of-the-art LLP methods.
@cite_19: This paper deals with the problem of multi-instance learning when label proportions are provided. In this classification problem, the instances of the dataset are divided into disjoint groups, where there is no certainty about the labels associated with individual samples. However, in each group the number of instances that belong to each class is known. We propose several versions of an EM-algorithm that learns naive Bayes models to deal with the exposed problem. The proposed algorithms are evaluated on synthetic and real datasets, and compared with state-of-the-art approaches. The obtained results show a competitive behaviour of our proposals.
@cite_55: 
@cite_26: Opinion mining and demographic attribute inference have many applications in social science. In this paper, we propose models to infer daily joint probabilities of multiple latent attributes from Twitter data, such as political sentiment and demographic attributes. Since it is costly and time-consuming to annotate data for traditional supervised classification, we instead propose scalable Learning from Label Proportions (LLP) models for demographic and opinion inference using U.S. Census, national and state political polls, and Cook partisan voting index as population level data. In LLP classification settings, the training data is divided into a set of unlabeled bags, where only the label distribution in of each bag is known, removing the requirement of instance-level annotations. Our proposed LLP model, Weighted Label Regularization (WLR), provides a scalable generalization of prior work on label regularization to support weights for samples inside bags, which is applicable in this setting where bags are arranged hierarchically (e.g., county-level bags are nested inside of state-level bags). We apply our model to Twitter data collected in the year leading up to the 2016 U.S. presidential election, producing estimates of the relationships among political sentiment and demographics over time and place. We find that our approach closely tracks traditional polling data stratified by demographic category, resulting in error reductions of 28-44 over baseline approaches. We also provide descriptive evaluations showing how the model may be used to estimate interactions among many variables and to identify linguistic temporal variation, capabilities which are typically not feasible using traditional polling methods.
Some methods are proposed to deal with LLP problem based on probabilistic models. Hern 'a @cite_19 adopt several versions of an Expectation-Maximization algorithm to learn a naive Bayes model which assumes conditional independence between the predictive variables. @cite_55 propose a new learning framework from the Bayesian perspective by estimating the conditional class density to estimate the posterior probability. Meanwhile, with the deep belief networks model for estimating the log-probability, they rebuild the posterior probability for classification problem. @cite_36 build a probabilistic approach applied to the US presidential election, which uses cardinality potentials to perform exact inference over latent variables during learning, and introduces a novel message-passing algorithm to extend cardinality potentials to multivariate probability models. @cite_26 develop a models to estimate the relationship between political sentiment and demographics during the U.S. presidential election.