Multi-Output Dependence (MOD) learning is a generalization of standard classification problems that allows for multiple outputs that are dependent on each other. A primary issue that arises in the context of MOD learning is that for any given input pattern there can be multiple correct output patterns. This changes the learning task from function approximation to relation approximation. Previous algorithms do not consider this problem, and thus cannot be readily applied to MOD problems. To perform MOD learning, we introduce the Hierarchical Multi-Output Nearest Neighbor model (HMONN) that employs a basic learning model for each output and a modified nearest neighbor approach to refine the initial results.
@cite_5: In this paper we present methods of enhancing existing discriminative classifiers for multi-labeled predictions. Discriminative methods like support vector machines perform very well for uni-labeled text classification tasks. Multi-labeled classification is a harder task subject to relatively less attention. In the multi-labeled setting, classes are often related to each other or part of a is-a hierarchy. We present a new technique for combining text features and features indicating relationships between classes, which can be used with any discriminative algorithm. We also present two enhancements to the margin of SVMs for building better models in the presence of overlapping classes. We present results of experiments on real world text benchmark datasets. Our new methods beat accuracy of existing methods with statistically significant improvements.
@cite_1: Machine learning develops intelligent computer systems that are able to generalize from previously seen examples. A new domain of machine learning, in which the prediction must satisfy the additional constraints found in structured data, poses one of machine learning's greatest challenges: learning functional dependencies between arbitrary input and output domains. This volume presents and analyzes the state of the art in machine learning algorithms and theory in this novel field. The contributors discuss applications as diverse as machine translation, document markup, computational biology, and information extraction, among others, providing a timely overview of an exciting field. Contributors Yasemin Altun, Gokhan Bakir [no dot over i], Olivier Bousquet, Sumit Chopra, Corinna Cortes, Hal Daume III, Ofer Dekel, Zoubin Ghahramani, Raia Hadsell, Thomas Hofmann, Fu Jie Huang, Yann LeCun, Tobias Mann, Daniel Marcu, David McAllester, Mehryar Mohri, William Stafford Noble, Fernando Perez-Cruz, Massimiliano Pontil, Marc'Aurelio Ranzato, Juho Rousu, Craig Saunders, Bernhard Scholkopf, Matthias W. Seeger, Shai Shalev-Shwartz, John Shawe-Taylor, Yoram Singer, Alexander J. Smola, Sandor Szedmak, Ben Taskar, Ioannis Tsochantaridis, S.V.N Vishwanathan, Jason Weston Gokhan Bakir [no dot over i] is Research Scientist at the Max Planck Institute for Biological Cybernetics in Tubingen, Germany. Thomas Hofmann is a Director of Engineering at Google's Engineering Center in Zurich and Adjunct Associate Professor of Computer Science at Brown University. Bernhard Scholkopf is Director of the Max Planck Institute for Biological Cybernetics and Professor at the Technical University Berlin. Alexander J. Smola is Senior Principal Researcher and Machine Learning Program Leader at National ICT Australia Australian National University, Canberra. Ben Taskar is Assistant Professor in the Computer and Information Science Department at the University of Pennsylvania. S. V. N. Vishwanathan is Senior Researcher in the Statistical Machine Learning Program, National ICT Australia with an adjunct appointment at the Research School for Information Sciences and Engineering, Australian National University.
@cite_6: The widely known binary relevance method for multi-label classification, which considers each label as an independent binary problem, has often been overlooked in the literature due to the perceived inadequacy of not directly modelling label correlations. Most current methods invest considerable complexity to model interdependencies between labels. This paper shows that binary relevance-based methods have much to offer, and that high predictive performance can be obtained without impeding scalability to large datasets. We exemplify this with a novel classifier chains method that can model label correlations while maintaining acceptable computational complexity. We extend this approach further in an ensemble framework. An extensive empirical evaluation covers a broad range of multi-label datasets with a variety of evaluation metrics. The results illustrate the competitiveness of the chaining method against related and state-of-the-art methods, both in terms of predictive performance and time complexity.
@cite_2: 
Other work has examined classification with multiple labels, although the labels are generally not considered to be dependent on each other and multiple correct labels are not considered. Multi-label classification considers problems with multiple outputs, but no dependency between the outputs is modeled. Tsoumakas al @cite_1 give an overview of multi-label classification. They define two main approaches for multi-label classification. The first approach is problem transformation, where the given data is transformed into a single problem that already has a well defined solution. The second approach is algorithm adaptation, where current algorithms are modified to solve the multi-label classification problem. Recent work has looked at correlations between labels in multi-label classification to improve accuracy @cite_5 . Read @cite_2 @cite_6 introduced chain classifiers for supporting these correlations which could be viable for supporting MOD problems.