4. A method for document representation by measuring informative content of words, comprising: associating a probability distribution P(w|C) for a plurality of words in a document collection C; forming a weighting model of word importance to the collection from the probability distribution therein; determining a distribution t wd of a selected word w in a particular document d in the collection; computing a deviation of the distribution of the selected word w in the document d from the probability distribution P(w|C) of the word w in the collection C in accordance with a concavity function −ln η P(T w ≧t wd |λ w ) comprising a selectively variable concavity control parameter η for setting a curvature of a logarithm of the concavity function −ln η P(T w ≧t wd −λ w ) in which the concavity control parameter η is effected relative to a probable frequency of the word w in the document d; and assigning a greater importance of the selected word w to the informative content of the document d by the greater than computed deviation in accordance with a concavity function −ln η P(T w ≧t wd −λ w ) whereby the document d can be represented relative to the collection by the assigned importance of a plurality of selected words.
4. A method for document representation by measuring informative content of words, comprising: associating a probability distribution P(w|C) for a plurality of words in a document collection C; forming a weighting model of word importance to the collection from the probability distribution therein; determining a distribution t wd of a selected word w in a particular document d in the collection; computing a deviation of the distribution of the selected word w in the document d from the probability distribution P(w|C) of the word w in the collection C in accordance with a concavity function −ln η P(T w ≧t wd |λ w ) comprising a selectively variable concavity control parameter η for setting a curvature of a logarithm of the concavity function −ln η P(T w ≧t wd −λ w ) in which the concavity control parameter η is effected relative to a probable frequency of the word w in the document d; and assigning a greater importance of the selected word w to the informative content of the document d by the greater than computed deviation in accordance with a concavity function −ln η P(T w ≧t wd −λ w ) whereby the document d can be represented relative to the collection by the assigned importance of a plurality of selected words. 6. The method of claim 4 wherein the computing is effected in accordance with: R ⁢ ⁢ S ⁢ ⁢ V ⁡ ( q , d ) = ∑ w ∈ q ⋂ d ⁢ - q w ⁢ ln η ⁢ P ⁡ ( T w ≥ t w ⁢ ⁢ d | λ w ) wherein t w ⁢ ⁢ d = x w ⁢ ⁢ d ⁢ log ⁡ ( 1 + c ⁢ avg ⁢ ⁢ l ⁢ ld ) parameter c normalizes word frequencies for determining the importance of the word, and parameter η sets curvature of the logarithm and, q represents a query; RSV(q, d) represents a ranking function representing a retrieval status value of document d for query q; x wd represents number of occurrences of word w in document d; T w represents Continuous Random Variable for normalized occurrences; q w represents a query for a word; and λ w represents a set of parameters of a preselected probability distribution.