1. A method of automatic text segmentation, the method comprising: estimating, for each token in a set of tokens in a description, through use of a machine having one or more processors, a probability that the token is irrelevant; associating, with each token in the set of tokens in the description, one of a first value, a second value, or a third value, based on whether, respectively, the token occurs in a header of the description, a lexical association exists between the token and a token in the header, or the lexical association is absent and the token does not occur in the header; iterating, through use of the machine, over a plurality of groups of sequential tokens in the description, in each iteration, selecting a group, computing a relevance value of the selected group based, at least in part, on at least one estimated probability of one or more tokens outside the selected group and on values associated with one or more tokens in the selected group; and indicating, through use of the machine, one of the plurality of groups as having a greatest relevance value.
1. A method of automatic text segmentation, the method comprising: estimating, for each token in a set of tokens in a description, through use of a machine having one or more processors, a probability that the token is irrelevant; associating, with each token in the set of tokens in the description, one of a first value, a second value, or a third value, based on whether, respectively, the token occurs in a header of the description, a lexical association exists between the token and a token in the header, or the lexical association is absent and the token does not occur in the header; iterating, through use of the machine, over a plurality of groups of sequential tokens in the description, in each iteration, selecting a group, computing a relevance value of the selected group based, at least in part, on at least one estimated probability of one or more tokens outside the selected group and on values associated with one or more tokens in the selected group; and indicating, through use of the machine, one of the plurality of groups as having a greatest relevance value. 2. The method of claim 1 , wherein estimating the probability that the token is irrelevant includes examining a plurality of descriptions and determining a frequency of occurrence of the token throughout the plurality of descriptions.