1. A processing system for representing documents relative to importance of words in the document, including: a processor comprising a weighting model of word importance of a word w in a document d in a collection relative to an importance of the word w in other documents in the collection by computing a deviation of distribution of the word w from a probability distribution of the word w in other documents in the collection, wherein the deviation distribution is weighted in accordance with a concavity function comprising a selectively variable concavity control parameter η for setting a curvature of a logarithm of the concavity function and wherein the concavity control parameter η is effected relative to a probable frequency of the word w in the document d according to −ln η P(T w ≧t wd |λ w ) where T w is a continuous random variable modeling normalized term frequencies for a probability distribution parameterized by a set of parameters λ w and t wd represents frequency of the word w in the document d.
1. A processing system for representing documents relative to importance of words in the document, including: a processor comprising a weighting model of word importance of a word w in a document d in a collection relative to an importance of the word w in other documents in the collection by computing a deviation of distribution of the word w from a probability distribution of the word w in other documents in the collection, wherein the deviation distribution is weighted in accordance with a concavity function comprising a selectively variable concavity control parameter η for setting a curvature of a logarithm of the concavity function and wherein the concavity control parameter η is effected relative to a probable frequency of the word w in the document d according to −ln η P(T w ≧t wd |λ w ) where T w is a continuous random variable modeling normalized term frequencies for a probability distribution parameterized by a set of parameters λ w and t wd represents frequency of the word w in the document d. 2. The system of claim 1 wherein the weighting model is represented as: R ⁢ ⁢ S ⁢ ⁢ V ⁡ ( q , d ) = ∑ w ∈ q ⋂ d ⁢ - q w ⁢ ln η ⁢ P ⁡ ( T w ≥ t w ⁢ ⁢ d | λ w ) wherein parameter c normalizes word frequencies for determining the importance of the word wherein t w ⁢ ⁢ d = x w ⁢ ⁢ d ⁢ log ⁡ ( 1 + c ⁢ avg ⁢ ⁢ l ⁢ ld ) , and parameter η sets curvature of the logarithm; and, q represents a query; RSV(q, d) represents a ranking function representing retrieval status value of document d for query q; x wd represents number of occurrences of word w in document d; P(w|C) represents Probability of the word in the collection; and q w represents a query for a word.