Multi-class Classification Model Inspired by Quantum Detection Theory

Multi-class Classification Model Inspired by Quantum Detection Theory

Abstract

Machine Learning has become very famous in today’s world which assist to identifying the patterns from the raw data. Technological advancement has led to substantial improvement in Machine Learning which, thus helping to improve prediction. Current Machine Learning models are based on Classical Theory, which can be replaced by Quantum Theory to improve the effectiveness of the model.

In the previous work, we developed binary classifier inspired by Quantum Detection Theory. In this extended abstract, our main goal is to develop multi class classifier. We generally use the terminology multinomial classification or multi-class classification when we have a classification problem for classifying observations or instances into one of three or more classes.

Binary Classifier, Multi-class Classifier, Quantum Mechanics, Signal Detection

P.Tiwari, M. MelucciMulti-class Classification Model Inspired by Quantum Detection Theory

\conference

Proceedings of the 8th Symposium on
Future Directions in Information Access 2018

\authorone

Prayag Tiwari, Massimo Melucci
Department of Information Engineering

1 Introduction

Quantum Mechanics has already shown its effectiveness in many fields so there is a good possibility that it will prove to be useful in Machine Learning (ML) as well. Quantum theory can open a new way towards quantum inspired ML which might outperform traditional machine learning if used properly. The theory of Quantum Mechanics (QM) has been implemented in several domain of IR recently by Li et al. (2018); Zhang, Song, Zhang, Wang, Li, Li and Wang (2018); Zhang et al. (2016); Zhang, Niu, Su, Wang, Ma and Song (2018); Wang et al. (2016); Zhang, Su, Zhang, Wang and Song (2018); Li et al. (2018, 2015) . Quantum Probability theory is the quantum generalization of classical probability theory, which was developed by von Neumann (1955). Classical probability theory provides that a system can have either state 0 or 1 and quantum probability comes into existence to go beyond classical theory and describe states in between 0 and 1 with classical states.

The effectiveness of the state-of-the-art classification algorithms relies on logical theory of sets, theory of probability and the algebra of vector spaces. For example, the most straightforward technique is called Naive Bayes, which considers objects (e.g. documents) as elements of sets and applies basic probability measures to these sets for selecting classes. Another effective classification technique called Support Vector Machines considers objects as points of a multi-dimensional space and aims to select subspaces as classes. However, an effective combination of techniques stemming from different theories is still missing, although it has been investigated in IR since the book on the Geometry of IR by van Rijsbergen (1979).

Despite its effectiveness in some domains, classification effectiveness is still unsatisfactory in a number of application domains due to a variety of reasons, such as the number of categories and the nature of data. The number of categories may be so large that a classification technique that is effective for a few categories may be ineffective when thousands of categories are required; moreover, the nature of data may be so complex that the techniques that are effective for simple objects may prove to be ineffective for complex objects. A sensible approach to addressing the problems caused by unconventional categorical systems or complex data is to adapt well-known and effective techniques to these new contexts. Another approach, which is indeed the focus of this paper, is to radically change paradigm and to investigate whether a new theoretical framework may be beneficial and be a new research direction.

Quantum Theory may provide a theoretical model for classification. To our knowledge no work has been done on quantum based classification so this paper is a first step to enter into quantum inspired machine learning and prove the effectiveness of quantum theory in classification.

2 Proposed Methodology and Discussion

In the previous work, we developed a binary classifier inspired by Quantum Detection Theory by Di Buccio et al. (2018). The main task was to identify whether a document belongs to a given topic or not. We used Reuters215781 in order to check the performance of our model. Our proposed binary classifier model inspired from quantum detection theory performed very well in terms of recall and F-measures for most of the topics. An experiment was done on the small dataset so work is still in progress. Our algorithm works as follows: it starts by computing the density operators and from positive and negative samples, respectively. In order to achieve this, for a particular feature, we first compute the number of documents with non-zero values in the feature. In this way, one vector is generated from each class, thus obtaining two vectors which are respectively denoted as and . These vectors can be regarded as a representation of the feature statistics among a class. We normalize the vectors and compute the outer spaces in order to obtain the density operators and :

 ρ1=|v1⟩⟨v1|tr(|v1⟩⟨v1|)ρ0=|v0⟩⟨v0|tr(|v0⟩⟨v0|) (1)

We computed the projection operator according to the eigen decomposition described in (Melucci, 2015), that is,

 ρ1−λρ0=ηP+βP⊥η>0β<0PP⊥=0 (2)

where is the prior probability of the negative class and . Moreover, is the positive eigenvalue corresponding to which represents the subspaces of the vectors representing the documents to be accepted in the target class.

In this extended abstract, our main goal is to develop multi class classifier. We generally use the terminology ”multinomial classification or multi-class classification” when we have a classification problem for classifying observations or instances into one of three or more classes.

The main theory behind quantum inspired multi-class classification is as follows: The choice among the hypotheses, which the asserts ”The system has the density operator ,” in which can be based on the result of the measurement of commuting operators , making a resolution of identity operator :

 P1+P2+....+PN=1 (3)

Our problem is getting the set of projectors so that the choice among the hypothesis can be made with the minimum average cost. It will assist in the event of Quantum Detection Theory, in constructing and estimating the best receiver for the communications system. In this, messages are coded into an alphabets of 3 or more symbols, and a distinct signal is transmitted for each.

Assume is the prior probability of the hypothesis , and is the cost for choosing when is correct. So the average cost per decision can be described as,

 ¯K=N∑i=1N∑j=1ξjKijTr(ρjPi), (4)

which has to be minimized by the set of given commuting projection operators . In particular, , , , can be approximate to the average probability of error.

In each hypotheses, the state of the system is in pure state , so the projection operator will have such form, . Here is the linear combination of the the given state . The main problem is to find the set of projectors minimizing the average cost when more than two categories or hypotheses are available; the solution can be a generalization of the solution of the problem of finding the set of projectors in the event of two categories.

3 Conclusion and Future Works

Although research work is still in progress we are testing a multi class classifier based on Quantum Detection Theory and we expect that it is possible to develop such a model. In order to learn about Quantum Detection Theory (QDT) and classification tasks in more detail, these works may be beneficial and a basis for developing this model. (Helstrom, 1971; Yuen et al., 1975; Helstrom, 1969, 1972; Eldar and Forney, 2001; Helstrom and Kennedy, 1974; Helstrom, 1968; Holevo, 1998; Vilnrotter and Lau, 2001; Di Buccio and Melucci, n.d.; Melucci, n.d.; Melucci et al., 2012; Melucci and van Rijsbergen, 2011; Melucci and Baeza-Yates, 2011; Melucci, 2018, 2017; Nanni and Melucci, 2016)

Acknowledgement

This work is supported by the Quantum Access and Retrieval Theory (QUARTZ) project, which has received funding from the European Unionâs Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 721321.

Footnotes

1. http://www.daviddlewis.com/resources/testcollections/reuters21578/

References

1. Di Buccio, E., Li, Q., Melucci, M. and Tiwari, P. (2018), Binary classification model inspired from quantum detection theory, in ‘Proceedings of the 2018 ACM SIGIR International Conference on Theory of Information Retrieval’, ACM, pp. 187–190.
2. Di Buccio, E. and Melucci, M. (n.d.), Evaluation of a feedback algorithm inspired by quantum detection for dynamic search tasks.
3. Eldar, Y. C. and Forney, G. D. (2001), ‘On quantum detection and the square-root measurement’, IEEE Transactions on Information Theory 47(3), 858–872.
4. Helstrom, C. and Kennedy, R. (1974), ‘Noncommuting observables in quantum detection and estimation theory’, IEEE Transactions on Information Theory 20(1), 16–24.
5. Helstrom, C. W. (1968), ‘Detection theory and quantum mechanics (ii)’, Information and Control 13(2), 156–171.
6. Helstrom, C. W. (1969), ‘Quantum detection and estimation theory’, Journal of Statistical Physics 1(2), 231–252.
7. Helstrom, C. W. (1971), ‘Quantum detection theory’.
8. Helstrom, C. W. (1972), Vii quantum detection theory, in ‘Progress in optics’, Vol. 10, Elsevier, pp. 289–369.
9. Holevo, A. S. (1998), ‘The capacity of the quantum channel with general signal states’, IEEE Transactions on Information Theory 44(1), 269–273.
10. Li, Q., Li, J., Zhang, P. and Song, D. (2015), Modeling multi-query retrieval tasks using density matrix transformation, in ‘Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval’, ACM, pp. 871–874.
11. Li, Q., Uprety, S., Wang, B. and Song, D. (2018), ‘Quantum-inspired complex word embedding’, arXiv preprint arXiv:1805.11351 .
12. Melucci, M. (2015), ‘Relevance feedback algorithms inspired by quantum detection’, IEEE Transactions on Knowledge and Data Engineering 28(4), 1022–1034.
13. Melucci, M. (2017), ‘An algorithm to calculate a quantum probability space’, arXiv preprint arXiv:1710.10158 .
14. Melucci, M. (2018), ‘An efficient algorithm to compute a quantum probability space’, IEEE Transactions on Knowledge and Data Engineering .
15. Melucci, M. (n.d.), Introduction to information retrieval and quantum mechanics, Springer.
16. Melucci, M. and Baeza-Yates, R. (2011), Advanced topics in information retrieval, Vol. 33, Springer Science & Business Media.
17. Melucci, M. and van Rijsbergen, K. (2011), Quantum mechanics and information retrieval, in ‘Advanced topics in information retrieval’, Springer, pp. 125–155.
18. Melucci, M. et al. (2012), ‘Contextual search: A computational framework’, Foundations and Trends® in Information Retrieval 6(4–5), 257–405.
19. Nanni, L. and Melucci, M. (2016), ‘Combination of projectors, standard texture descriptors and bag of features for classifying images’, Neurocomputing 173, 1602–1614.
20. van Rijsbergen, C. J. (1979), Information Retrieval, II edn, Butterworths, London, chapter Automatic Classification, pp. 36–65.
21. Vilnrotter, V. and Lau, C. (2001), ‘Quantum detection theory for the free-space channel’, The InterPlanetary Network Progress Report 42-146, April–June 2001 pp. 1–34.
22. von Neumann, J. (1955), Mathematical Foundations of Quantum Mechanics, Princeton University Press, Princeton, New Jersey, USA.
23. Wang, B., Zhang, P., Li, J., Song, D., Hou, Y. and Shang, Z. (2016), ‘Exploration of quantum interference in document relevance judgement discrepancy’, Entropy 18(4), 144.
24. Yuen, H., Kennedy, R. and Lax, M. (1975), ‘Optimum testing of multiple hypotheses in quantum detection theory’, IEEE Transactions on Information Theory 21(2), 125–134.
25. Zhang, P., Li, J., Wang, B., Zhao, X., Song, D., Hou, Y. and Melucci, M. (2016), ‘A quantum query expansion approach for session search’, Entropy 18(4), 146.
26. Zhang, P., Niu, J., Su, Z., Wang, B., Ma, L. and Song, D. (2018), ‘End-to-end quantum-like language models with application to question answering’.
27. Zhang, P., Su, Z., Zhang, L., Wang, B. and Song, D. (2018), ‘A quantum many-body wave function inspired language modeling approach’, arXiv preprint arXiv:1808.09891 .
28. Zhang, Y., Song, D., Zhang, P., Wang, P., Li, J., Li, X. and Wang, B. (2018), ‘A quantum-inspired multimodal sentiment analysis framework’, Theoretical Computer Science .
You are adding the first comment!
How to quickly get a good reply:
• Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
• Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
• Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
{"ft_lang_label":"__label__en","ft_lang_prob":0.8413523,"math_prob":0.8179676,"size":11502,"snap":"2019-26-2019-30","text_gpt3_token_len":2688,"char_repetition_ratio":0.14158984,"word_repetition_ratio":0.052213393,"special_character_ratio":0.25091287,"punctuation_ratio":0.2052655,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9543442,"pos_list":[0,1,2,3,4,5,6],"im_url_duplicate_count":[null,null,null,null,null,null,null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2019-06-24T15:50:44Z\",\"WARC-Record-ID\":\"<urn:uuid:e6f6834d-f612-4f69-8ea1-c08e250e512a>\",\"Content-Length\":\"184866\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:82746589-6963-4e7b-bd6f-3cc1305cc47c>\",\"WARC-Concurrent-To\":\"<urn:uuid:09f8c23f-fa62-4173-9fa9-dc3700d57c36>\",\"WARC-IP-Address\":\"35.186.203.76\",\"WARC-Target-URI\":\"https://www.groundai.com/project/multi-class-classification-model-inspired-by-quantum-detection-theory/\",\"WARC-Payload-Digest\":\"sha1:N2N3KK7HF2YUBSGMQZRVSBTOC3TRB7FG\",\"WARC-Block-Digest\":\"sha1:3N3JNNSOKYWMHWGLNPM7VA4QBB7D62G3\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2019/CC-MAIN-2019-26/CC-MAIN-2019-26_segments_1560627999615.68_warc_CC-MAIN-20190624150939-20190624172939-00071.warc.gz\"}"}