From java-commitsgonzalezjerry@example.com Thu Mar 01 10:21:38 2007 Return-Path: <java-commitsgonzalezjerry@example.com> Delivered-To: gonzalezjerry@example.com Received: (qmail 20778 invoked from network); 1 Mar 2007 10:21:38 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Mar 2007 10:21:38 -0000 Received: (qmail 5138 invoked by uid 500); 1 Mar 2007 10:21:47 -0000 Delivered-To: gonzalezjerry@example.com Received: (qmail 5119 invoked by uid 500); 1 Mar 2007 10:21:46 -0000 Mailing-List: contact gonzalezjerry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:gonzalezjerry@example.com> List-Unsubscribe: <mailto:gonzalezjerry@example.com> List-Post: <mailto:gonzalezjerry@example.com> List-Id: <java-commits.lucene.apache.org> Reply-To: gonzalezjerry@example.com Delivered-To: mailing list gonzalezjerry@example.com Received: (qmail 5108 invoked by uid 99); 1 Mar 2007 10:21:46 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 02:21:46 -0800 X-ASF-Spam-Status: No, hits=-99.5 required=10.0 tests=ALL_TRUSTED,NO_REAL_NAME X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 02:21:37 -0800 Received: by eris.apache.org (Postfix, from userid 65534) id 234F81A981D; Thu, 1 Mar 2007 02:21:17 -0800 (PST) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r513258 - /lucene/java/trunk/src/java/org/apache/lucene/search/ScoreDocComparator.java Date: Thu, 01 Mar 2007 10:21:17 -0000 To: gonzalezjerry@example.com From: gonzalezjerry@example.com X-Mailer: svnmailer-1.1.0 Message-Id: <gonzalezjerry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org Author: otis Date: Thu Mar 1 02:21:16 2007 New Revision: 513258 URL: http://svn.apache.org/viewvc?view=rev&rev=513258 Log: - Wrapped looong liiines Modified: lucene/java/trunk/src/java/org/apache/lucene/search/ScoreDocComparator.java Modified: lucene/java/trunk/src/java/org/apache/lucene/search/ScoreDocComparator.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/java/org/apache/lucene/search/ScoreDocComparator.java?view=diff&rev=513258&r1=513257&r2=513258 ============================================================================== --- lucene/java/trunk/src/java/org/apache/lucene/search/ScoreDocComparator.java (original) +++ lucene/java/trunk/src/java/org/apache/lucene/search/ScoreDocComparator.java Thu Mar 1 02:21:16 2007 @@ -44,7 +44,6 @@ } }; - /** Special comparator for sorting hits according to index order (document number). */ static final ScoreDocComparator INDEXORDER = new ScoreDocComparator() { public int compare (ScoreDoc i, ScoreDoc j) { @@ -60,35 +59,39 @@ } }; - /** * Compares two ScoreDoc objects and returns a result indicating their * sort order. * @param i First ScoreDoc * @param j Second ScoreDoc - * @return <code>-1</code> if <code>i</code> should come before <code>j</code><br><code>1</code> if <code>i</code> should come after <code>j</code><br><code>0</code> if they are equal + * @return <code>-1</code> if <code>i</code> should come before <code>j</code><br> + * <code>1</code> if <code>i</code> should come after <code>j</code><br> + * <code>0</code> if they are equal * @see java.util.Comparator */ int compare (ScoreDoc i, ScoreDoc j); - /** * Returns the value used to sort the given document. The * object returned must implement the java.io.Serializable - * interface. This is used by multisearchers to determine how to collate results from their searchers. + * interface. This is used by multisearchers to determine how + * to collate results from their searchers. * @see FieldDoc * @param i Document * @return Serializable object */ Comparable sortValue (ScoreDoc i); - /** - * Returns the type of sort. Should return <code>SortField.SCORE</code>, <code>SortField.DOC</code>, <code>SortField.STRING</code>, <code>SortField.INTEGER</code>, - * <code>SortField.FLOAT</code> or <code>SortField.CUSTOM</code>. It is not valid to return <code>SortField.AUTO</code>. - * This is used by multisearchers to determine how to collate results from their searchers. + * Returns the type of sort. Should return <code>SortField.SCORE</code>, + * <code>SortField.DOC</code>, <code>SortField.STRING</code>, + * <code>SortField.INTEGER</code>, <code>SortField.FLOAT</code> or + * <code>SortField.CUSTOM</code>. It is not valid to return + * <code>SortField.AUTO</code>. + * This is used by multisearchers to determine how to collate results + * from their searchers. * @return One of the constants in SortField. * @see SortField */ int sortType(); -} \ No newline at end of file +} From java-commitsgonzalezjerry@example.com Thu Mar 01 14:23:21 2007 Return-Path: <java-commitsgonzalezjerry@example.com> Delivered-To: gonzalezjerry@example.com Received: (qmail 11304 invoked from network); 1 Mar 2007 14:23:20 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Mar 2007 14:23:20 -0000 Received: (qmail 93054 invoked by uid 500); 1 Mar 2007 14:23:29 -0000 Delivered-To: gonzalezjerry@example.com Received: (qmail 93030 invoked by uid 500); 1 Mar 2007 14:23:28 -0000 Mailing-List: contact gonzalezjerry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:gonzalezjerry@example.com> List-Unsubscribe: <mailto:gonzalezjerry@example.com> List-Post: <mailto:gonzalezjerry@example.com> List-Id: <java-commits.lucene.apache.org> Reply-To: gonzalezjerry@example.com Delivered-To: mailing list gonzalezjerry@example.com Received: (qmail 93016 invoked by uid 99); 1 Mar 2007 14:23:28 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 06:23:28 -0800 X-ASF-Spam-Status: No, hits=-99.5 required=10.0 tests=ALL_TRUSTED,NO_REAL_NAME X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 06:23:18 -0800 Received: by eris.apache.org (Postfix, from userid 65534) id 3E9351A981A; Thu, 1 Mar 2007 06:22:58 -0800 (PST) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r513344 - in /lucene/java/trunk/contrib/analyzers/src: java/org/apache/lucene/analysis/ngram/EdgeNGramTokenizer.java test/org/apache/lucene/analysis/ngram/EdgeNGramTokenizerTest.java Date: Thu, 01 Mar 2007 14:22:58 -0000 To: gonzalezjerry@example.com From: gonzalezjerry@example.com X-Mailer: svnmailer-1.1.0 Message-Id: <gonzalezjerry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org Author: otis Date: Thu Mar 1 06:22:57 2007 New Revision: 513344 URL: http://svn.apache.org/viewvc?view=rev&rev=513344 Log: - LUCENE-759: Made the tokenizer capable of creating n-grams of a varying sizes - from min to max characters per n-gram. Patch from Adam Hiatt. Modified: lucene/java/trunk/contrib/analyzers/src/java/org/apache/lucene/analysis/ngram/EdgeNGramTokenizer.java lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/EdgeNGramTokenizerTest.java Modified: lucene/java/trunk/contrib/analyzers/src/java/org/apache/lucene/analysis/ngram/EdgeNGramTokenizer.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/analyzers/src/java/org/apache/lucene/analysis/ngram/EdgeNGramTokenizer.java?view=diff&rev=513344&r1=513343&r2=513344 ============================================================================== --- lucene/java/trunk/contrib/analyzers/src/java/org/apache/lucene/analysis/ngram/EdgeNGramTokenizer.java (original) +++ lucene/java/trunk/contrib/analyzers/src/java/org/apache/lucene/analysis/ngram/EdgeNGramTokenizer.java Thu Mar 1 06:22:57 2007 @@ -24,75 +24,123 @@ import java.io.Reader; /** - * Tokenizes the input into n-grams of the given size. + * Tokenizes the input from an edge into n-grams of given size(s). * @author Otis Gospodnetic + * @author Adam Hiatt */ public class EdgeNGramTokenizer extends Tokenizer { - // which side to get the n-gram from - // TODO: switch to using this enum when we move to 1.5+ -// public enum Side { -// FRONT (), -// BACK (); -// } + public static final Side DEFAULT_SIDE = Side.FRONT; + public static final int DEFAULT_MAX_GRAM_SIZE = 1; + public static final int DEFAULT_MIN_GRAM_SIZE = 1; + + // Replace this with an enum when the Java 1.5 upgrade is made, the impl will be simplified /** Specifies which side of the input the n-gram should be generated from */ public static class Side { + private String label; + /** Get the n-gram from the front of the input */ public static Side FRONT = new Side("front"); + /** Get the n-gram from the end of the input */ public static Side BACK = new Side("back"); - private Side(String label) {} + + // Private ctor + private Side(String label) { this.label = label; } + + + public String getLabel() { return label; } + + // Get the appropriate Side from a string + public static Side getSide(String sideName) { + if (FRONT.getLabel().equals(sideName)) { + return FRONT; + } + else if (BACK.getLabel().equals(sideName)) { + return BACK; + } + return null; + } } + + private int minGram; + private int maxGram; private int gramSize; private Side side; + private boolean started = false; private int inLen; private String inStr; - private boolean started = false; + /** - * Creates EdgeNGramTokenizer that can generate an n-gram of the given size. + * Creates EdgeNGramTokenizer that can generate n-grams in the sizes of the given range + * * @param input Reader holding the input to be tokenized - * @param side the {@link Side} from which to chop off an n-gram - * @param gramSize the size of the n-gram to generate + * @param side the {@link Side} from which to chop off an n-gram + * @param minGram the smallest n-gram to generate + * @param maxGram the largest n-gram to generate */ - public EdgeNGramTokenizer(Reader input, Side side, int gramSize) { + public EdgeNGramTokenizer(Reader input, Side side, int minGram, int maxGram) { super(input); - if (gramSize < 1) { - throw new IllegalArgumentException("gramSize must be greater than zero"); + + if (side == null) { + throw new IllegalArgumentException("sideLabel must be either front or back"); + } + + if (minGram < 1) { + throw new IllegalArgumentException("minGram must be greater than zero"); } - this.gramSize = gramSize; + + if (minGram > maxGram) { + throw new IllegalArgumentException("minGram must not be greater than maxGram"); + } + + this.minGram = minGram; + this.maxGram = maxGram; this.side = side; } - public EdgeNGramTokenizer(Reader input, String side, int gramSize) { - + /** + * Creates EdgeNGramTokenizer that can generate n-grams in the sizes of the given range + * + * @param input Reader holding the input to be tokenized + * @param sideLabel the name of the {@link Side} from which to chop off an n-gram + * @param minGram the smallest n-gram to generate + * @param maxGram the largest n-gram to generate + */ + public EdgeNGramTokenizer(Reader input, String sideLabel, int minGram, int maxGram) { + this(input, Side.getSide(sideLabel), minGram, maxGram); } /** Returns the next token in the stream, or gonzalezjerry@example.com. */ public final Token next() throws IOException { - // if we already returned the edge n-gram, we are done - if (started) - return null; + // if we are just starting, read the whole input if (!started) { started = true; char[] chars = new char[1024]; input.read(chars); - inStr = new String(chars).trim(); // remove any trailing empty strings + inStr = new String(chars).trim(); // remove any trailing empty strings inLen = inStr.length(); + gramSize = minGram; + } + + // if the remaining input is too short, we can't generate any n-grams + if (gramSize > inLen) { + return null; } - // if the input is too short, we can't generate any n-grams - if (gramSize > inLen) + + // if we have hit the end of our n-gram size range, quit + if (gramSize > maxGram) { return null; - if (side == Side.FRONT) - return new Token(inStr.substring(0, gramSize), 0, gramSize); - else - return new Token(inStr.substring(inLen-gramSize), inLen-gramSize, inLen); - } + } + + Token tok; + if (side == Side.FRONT) { + tok = new Token(inStr.substring(0, gramSize), 0, gramSize); + } + else { + tok = new Token(inStr.substring(inLen-gramSize), inLen-gramSize, inLen); + } - static Side side(String label) { - if (label == null || label.trim().length() == 0) - throw new IllegalArgumentException("Label must be either 'front' or 'back'"); - if (label.equals("front")) - return Side.FRONT; - else - return Side.BACK; + gramSize++; + return tok; } } Modified: lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/EdgeNGramTokenizerTest.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/EdgeNGramTokenizerTest.java?view=diff&rev=513344&r1=513343&r2=513344 ============================================================================== --- lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/EdgeNGramTokenizerTest.java (original) +++ lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/EdgeNGramTokenizerTest.java Thu Mar 1 06:22:57 2007 @@ -28,54 +28,90 @@ * @author Otis Gospodnetic */ public class EdgeNGramTokenizerTest extends TestCase { - private StringReader input; - - public void setUp() { - input = new StringReader("abcde"); - } + private StringReader input; - public void testInvalidInput() throws Exception { - boolean gotException = false; - try { - new EdgeNGramTokenizer(input, EdgeNGramTokenizer.Side.FRONT, 0); - } catch (IllegalArgumentException e) { - gotException = true; - } - assertTrue(gotException); - } + public void setUp() { + input = new StringReader("abcde"); + } - public void testInvalidInput2() throws Exception { - boolean gotException = false; - try { - new EdgeNGramTokenizer(input, EdgeNGramTokenizer.Side.FRONT, -1); - } catch (IllegalArgumentException e) { - gotException = true; - } - assertTrue(gotException); + public void testInvalidInput() throws Exception { + boolean gotException = false; + try { + new EdgeNGramTokenizer(input, EdgeNGramTokenizer.Side.FRONT, 0, 0); + } catch (IllegalArgumentException e) { + gotException = true; } + assertTrue(gotException); + } - public void testFrontUnigram() throws Exception { - EdgeNGramTokenizer tokenizer = new EdgeNGramTokenizer(input, EdgeNGramTokenizer.Side.FRONT, 1); - Token token = null; - token = tokenizer.next(); - assertEquals("(a,0,1)", token.toString()); - token = tokenizer.next(); - assertNull(token); + public void testInvalidInput2() throws Exception { + boolean gotException = false; + try { + new EdgeNGramTokenizer(input, EdgeNGramTokenizer.Side.FRONT, 2, 1); + } catch (IllegalArgumentException e) { + gotException = true; } + assertTrue(gotException); + } - public void testBackUnigram() throws Exception { - EdgeNGramTokenizer tokenizer = new EdgeNGramTokenizer(input, EdgeNGramTokenizer.Side.BACK, 1); - Token token = null; - token = tokenizer.next(); - assertEquals("(e,4,5)", token.toString()); - token = tokenizer.next(); - assertNull(token); + public void testInvalidInput3() throws Exception { + boolean gotException = false; + try { + new EdgeNGramTokenizer(input, EdgeNGramTokenizer.Side.FRONT, -1, 2); + } catch (IllegalArgumentException e) { + gotException = true; } + assertTrue(gotException); + } - public void testOversizedNgrams() throws Exception { - EdgeNGramTokenizer tokenizer = new EdgeNGramTokenizer(input, EdgeNGramTokenizer.Side.FRONT, 6); - Token token = null; - token = tokenizer.next(); - assertNull(token); - } + public void testFrontUnigram() throws Exception { + EdgeNGramTokenizer tokenizer = new EdgeNGramTokenizer(input, EdgeNGramTokenizer.Side.FRONT, 1, 1); + Token token = null; + token = tokenizer.next(); + assertEquals("(a,0,1)", token.toString()); + token = tokenizer.next(); + assertNull(token); + } + + public void testBackUnigram() throws Exception { + EdgeNGramTokenizer tokenizer = new EdgeNGramTokenizer(input, EdgeNGramTokenizer.Side.BACK, 1, 1); + Token token = null; + token = tokenizer.next(); + assertEquals("(e,4,5)", token.toString()); + token = tokenizer.next(); + assertNull(token); + } + + public void testOversizedNgrams() throws Exception { + EdgeNGramTokenizer tokenizer = new EdgeNGramTokenizer(input, EdgeNGramTokenizer.Side.FRONT, 6, 6); + Token token = null; + token = tokenizer.next(); + assertNull(token); + } + + public void testFrontRangeOfNgrams() throws Exception { + EdgeNGramTokenizer tokenizer = new EdgeNGramTokenizer(input, EdgeNGramTokenizer.Side.FRONT, 1, 3); + Token token = null; + token = tokenizer.next(); + assertEquals("(a,0,1)", token.toString()); + token = tokenizer.next(); + assertEquals("(ab,0,2)", token.toString()); + token = tokenizer.next(); + assertEquals("(abc,0,3)", token.toString()); + token = tokenizer.next(); + assertNull(token); + } + + public void testBackRangeOfNgrams() throws Exception { + EdgeNGramTokenizer tokenizer = new EdgeNGramTokenizer(input, EdgeNGramTokenizer.Side.BACK, 1, 3); + Token token = null; + token = tokenizer.next(); + assertEquals("(e,4,5)", token.toString()); + token = tokenizer.next(); + assertEquals("(de,3,5)", token.toString()); + token = tokenizer.next(); + assertEquals("(cde,2,5)", token.toString()); + token = tokenizer.next(); + assertNull(token); + } } From java-commitsgonzalezjerry@example.com Thu Mar 01 23:11:14 2007 Return-Path: <java-commitsgonzalezjerry@example.com> Delivered-To: gonzalezjerry@example.com Received: (qmail 30062 invoked from network); 1 Mar 2007 23:11:13 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Mar 2007 23:11:13 -0000 Received: (qmail 71533 invoked by uid 500); 1 Mar 2007 23:11:22 -0000 Delivered-To: gonzalezjerry@example.com Received: (qmail 71517 invoked by uid 500); 1 Mar 2007 23:11:22 -0000 Mailing-List: contact gonzalezjerry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:gonzalezjerry@example.com> List-Unsubscribe: <mailto:gonzalezjerry@example.com> List-Post: <mailto:gonzalezjerry@example.com> List-Id: <java-commits.lucene.apache.org> Reply-To: gonzalezjerry@example.com Delivered-To: mailing list gonzalezjerry@example.com Received: (qmail 71506 invoked by uid 99); 1 Mar 2007 23:11:21 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 15:11:21 -0800 X-ASF-Spam-Status: No, hits=-99.5 required=10.0 tests=ALL_TRUSTED,NO_REAL_NAME X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Mar 2007 15:11:12 -0800 Received: by eris.apache.org (Postfix, from userid 65534) id 643EB1A981A; Thu, 1 Mar 2007 15:10:52 -0800 (PST) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r513547 - /lucene/java/trunk/src/java/org/apache/lucene/index/Term.java Date: Thu, 01 Mar 2007 23:10:52 -0000 To: gonzalezjerry@example.com From: gonzalezjerry@example.com X-Mailer: svnmailer-1.1.0 Message-Id: <gonzalezjerry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org Author: yonik Date: Thu Mar 1 15:10:51 2007 New Revision: 513547 URL: http://svn.apache.org/viewvc?view=rev&rev=513547 Log: clarify Term javadoc: field shouldn't be null either Modified: lucene/java/trunk/src/java/org/apache/lucene/index/Term.java Modified: lucene/java/trunk/src/java/org/apache/lucene/index/Term.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/java/org/apache/lucene/index/Term.java?view=diff&rev=513547&r1=513546&r2=513547 ============================================================================== --- lucene/java/trunk/src/java/org/apache/lucene/index/Term.java (original) +++ lucene/java/trunk/src/java/org/apache/lucene/index/Term.java Thu Mar 1 15:10:51 2007 @@ -30,7 +30,7 @@ String text; /** Constructs a Term with the given field and text. - * <p>Note that a text value of null results in undefined + * <p>Note that a null field or null text value results in undefined * behavior for most Lucene APIs that accept a Term parameter. */ public Term(String fld, String txt) { this(fld, txt, true); From java-commitsgonzalezjerry@example.com Fri Mar 02 17:54:50 2007 Return-Path: <java-commitsgonzalezjerry@example.com> Delivered-To: gonzalezjerry@example.com Received: (qmail 99856 invoked from network); 2 Mar 2007 17:54:49 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2007 17:54:49 -0000 Received: (qmail 38160 invoked by uid 500); 2 Mar 2007 17:54:58 -0000 Delivered-To: gonzalezjerry@example.com Received: (qmail 38142 invoked by uid 500); 2 Mar 2007 17:54:57 -0000 Mailing-List: contact gonzalezjerry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:gonzalezjerry@example.com> List-Unsubscribe: <mailto:gonzalezjerry@example.com> List-Post: <mailto:gonzalezjerry@example.com> List-Id: <java-commits.lucene.apache.org> Reply-To: gonzalezjerry@example.com Delivered-To: mailing list gonzalezjerry@example.com Received: (qmail 38131 invoked by uid 99); 2 Mar 2007 17:54:57 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Mar 2007 09:54:57 -0800 X-ASF-Spam-Status: No, hits=-99.5 required=10.0 tests=ALL_TRUSTED,NO_REAL_NAME X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Mar 2007 09:54:48 -0800 Received: by eris.apache.org (Postfix, from userid 65534) id 26C9A1A981A; Fri, 2 Mar 2007 09:54:28 -0800 (PST) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r513866 - /lucene/java/trunk/contrib/analyzers/src/java/org/apache/lucene/analysis/ngram/NGramTokenizer.java Date: Fri, 02 Mar 2007 17:54:28 -0000 To: gonzalezjerry@example.com From: gonzalezjerry@example.com X-Mailer: svnmailer-1.1.0 Message-Id: <gonzalezjerry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org Author: otis Date: Fri Mar 2 09:54:27 2007 New Revision: 513866 URL: http://svn.apache.org/viewvc?view=rev&rev=513866 Log: - 2-char indentation Modified: lucene/java/trunk/contrib/analyzers/src/java/org/apache/lucene/analysis/ngram/NGramTokenizer.java Modified: lucene/java/trunk/contrib/analyzers/src/java/org/apache/lucene/analysis/ngram/NGramTokenizer.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/analyzers/src/java/org/apache/lucene/analysis/ngram/NGramTokenizer.java?view=diff&rev=513866&r1=513865&r2=513866 ============================================================================== --- lucene/java/trunk/contrib/analyzers/src/java/org/apache/lucene/analysis/ngram/NGramTokenizer.java (original) +++ lucene/java/trunk/contrib/analyzers/src/java/org/apache/lucene/analysis/ngram/NGramTokenizer.java Fri Mar 2 09:54:27 2007 @@ -28,63 +28,63 @@ * @author Otis Gospodnetic */ public class NGramTokenizer extends Tokenizer { - public static final int DEFAULT_MIN_NGRAM_SIZE = 1; - public static final int DEFAULT_MAX_NGRAM_SIZE = 2; + public static final int DEFAULT_MIN_NGRAM_SIZE = 1; + public static final int DEFAULT_MAX_NGRAM_SIZE = 2; - private int minGram, maxGram; - private int gramSize; - private int pos = 0; - private int inLen; - private String inStr; - private boolean started = false; + private int minGram, maxGram; + private int gramSize; + private int pos = 0; + private int inLen; + private String inStr; + private boolean started = false; - /** - * Creates NGramTokenizer with given min and max n-grams. - * @param input Reader holding the input to be tokenized - * @param minGram the smallest n-gram to generate - * @param maxGram the largest n-gram to generate - */ - public NGramTokenizer(Reader input, int minGram, int maxGram) { - super(input); - if (minGram < 1) { - throw new IllegalArgumentException("minGram must be greater than zero"); - } - if (minGram > maxGram) { - throw new IllegalArgumentException("minGram must not be greater than maxGram"); - } - this.minGram = minGram; - this.maxGram = maxGram; + /** + * Creates NGramTokenizer with given min and max n-grams. + * @param input Reader holding the input to be tokenized + * @param minGram the smallest n-gram to generate + * @param maxGram the largest n-gram to generate + */ + public NGramTokenizer(Reader input, int minGram, int maxGram) { + super(input); + if (minGram < 1) { + throw new IllegalArgumentException("minGram must be greater than zero"); } - /** - * Creates NGramTokenizer with default min and max n-grams. - * @param input Reader holding the input to be tokenized - */ - public NGramTokenizer(Reader input) { - this(input, DEFAULT_MIN_NGRAM_SIZE, DEFAULT_MAX_NGRAM_SIZE); + if (minGram > maxGram) { + throw new IllegalArgumentException("minGram must not be greater than maxGram"); } + this.minGram = minGram; + this.maxGram = maxGram; + } + /** + * Creates NGramTokenizer with default min and max n-grams. + * @param input Reader holding the input to be tokenized + */ + public NGramTokenizer(Reader input) { + this(input, DEFAULT_MIN_NGRAM_SIZE, DEFAULT_MAX_NGRAM_SIZE); + } - /** Returns the next token in the stream, or gonzalezjerry@example.com. */ - public final Token next() throws IOException { - if (!started) { - started = true; - gramSize = minGram; - char[] chars = new char[1024]; - input.read(chars); - inStr = new String(chars).trim(); // remove any trailing empty strings - inLen = inStr.length(); - } + /** Returns the next token in the stream, or gonzalezjerry@example.com. */ + public final Token next() throws IOException { + if (!started) { + started = true; + gramSize = minGram; + char[] chars = new char[1024]; + input.read(chars); + inStr = new String(chars).trim(); // remove any trailing empty strings + inLen = inStr.length(); + } - if (pos+gramSize > inLen) { // if we hit the end of the string - pos = 0; // reset to beginning of string - gramSize++; // increase n-gram size - if (gramSize > maxGram) // we are done - return null; - if (pos+gramSize > inLen) - return null; - } - String gram = inStr.substring(pos, pos+gramSize); - int oldPos = pos; - pos++; - return new Token(gram, oldPos, oldPos+gramSize); + if (pos+gramSize > inLen) { // if we hit the end of the string + pos = 0; // reset to beginning of string + gramSize++; // increase n-gram size + if (gramSize > maxGram) // we are done + return null; + if (pos+gramSize > inLen) + return null; } + String gram = inStr.substring(pos, pos+gramSize); + int oldPos = pos; + pos++; + return new Token(gram, oldPos, oldPos+gramSize); + } } From java-commitsgonzalezjerry@example.com Fri Mar 02 18:20:17 2007 Return-Path: <java-commitsgonzalezjerry@example.com> Delivered-To: gonzalezjerry@example.com Received: (qmail 5990 invoked from network); 2 Mar 2007 18:20:16 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2007 18:20:16 -0000 Received: (qmail 84590 invoked by uid 500); 2 Mar 2007 18:20:24 -0000 Delivered-To: gonzalezjerry@example.com Received: (qmail 84578 invoked by uid 500); 2 Mar 2007 18:20:24 -0000 Mailing-List: contact gonzalezjerry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:gonzalezjerry@example.com> List-Unsubscribe: <mailto:gonzalezjerry@example.com> List-Post: <mailto:gonzalezjerry@example.com> List-Id: <java-commits.lucene.apache.org> Reply-To: gonzalezjerry@example.com Delivered-To: mailing list gonzalezjerry@example.com Received: (qmail 84567 invoked by uid 99); 2 Mar 2007 18:20:24 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Mar 2007 10:20:24 -0800 X-ASF-Spam-Status: No, hits=-99.5 required=10.0 tests=ALL_TRUSTED,NO_REAL_NAME X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Mar 2007 10:20:14 -0800 Received: by eris.apache.org (Postfix, from userid 65534) id 94F3E1A981A; Fri, 2 Mar 2007 10:19:54 -0800 (PST) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r513876 - in /lucene/java/trunk: CHANGES.txt contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/EdgeNGramTokenFilterTest.java contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/NGramTokenFilterTest.java Date: Fri, 02 Mar 2007 18:19:54 -0000 To: gonzalezjerry@example.com From: gonzalezjerry@example.com X-Mailer: svnmailer-1.1.0 Message-Id: <gonzalezjerry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org Author: otis Date: Fri Mar 2 10:19:53 2007 New Revision: 513876 URL: http://svn.apache.org/viewvc?view=rev&rev=513876 Log: - LUCENE-759: Two n-gram producting TokenFilters (using them for the spellchecker in SOLR-81) Added: lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/EdgeNGramTokenFilterTest.java lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/NGramTokenFilterTest.java Modified: lucene/java/trunk/CHANGES.txt Modified: lucene/java/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/lucene/java/trunk/CHANGES.txt?view=diff&rev=513876&r1=513875&r2=513876 ============================================================================== --- lucene/java/trunk/CHANGES.txt (original) +++ lucene/java/trunk/CHANGES.txt Fri Mar 2 10:19:53 2007 @@ -34,6 +34,11 @@ implementations to be specified via the System property org.apache.lucene.store.FSDirectoryLockFactoryClass. (Mike McCandless) +New features + + 1. LUCENE-759: Added two n-gram-producing TokenFilters. + (Otis Gospodnetic) + Optimizations ======================= Release 2.1.0 2007-02-14 ======================= Added: lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/EdgeNGramTokenFilterTest.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/EdgeNGramTokenFilterTest.java?view=auto&rev=513876 ============================================================================== --- lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/EdgeNGramTokenFilterTest.java (added) +++ lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/EdgeNGramTokenFilterTest.java Fri Mar 2 10:19:53 2007 @@ -0,0 +1,119 @@ +package org.apache.lucene.analysis.ngram; + +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import org.apache.lucene.analysis.Token; +import org.apache.lucene.analysis.TokenStream; +import org.apache.lucene.analysis.WhitespaceTokenizer; + +import java.io.StringReader; + +import junit.framework.TestCase; + +/** + * Tests {@link EdgeNGramTokenFilter} for correctness. + * @author Otis Gospodnetic + */ +public class EdgeNGramTokenFilterTest extends TestCase { + private TokenStream input; + + public void setUp() { + input = new WhitespaceTokenizer(new StringReader("abcde")); + } + + public void testInvalidInput() throws Exception { + boolean gotException = false; + try { + new EdgeNGramTokenFilter(input, EdgeNGramTokenFilter.Side.FRONT, 0, 0); + } catch (IllegalArgumentException e) { + gotException = true; + } + assertTrue(gotException); + } + + public void testInvalidInput2() throws Exception { + boolean gotException = false; + try { + new EdgeNGramTokenFilter(input, EdgeNGramTokenFilter.Side.FRONT, 2, 1); + } catch (IllegalArgumentException e) { + gotException = true; + } + assertTrue(gotException); + } + + public void testInvalidInput3() throws Exception { + boolean gotException = false; + try { + new EdgeNGramTokenFilter(input, EdgeNGramTokenFilter.Side.FRONT, -1, 2); + } catch (IllegalArgumentException e) { + gotException = true; + } + assertTrue(gotException); + } + + public void testFrontUnigram() throws Exception { + EdgeNGramTokenFilter tokenizer = new EdgeNGramTokenFilter(input, EdgeNGramTokenFilter.Side.FRONT, 1, 1); + Token token = null; + token = tokenizer.next(); + assertEquals("(a,0,1)", token.toString()); + token = tokenizer.next(); + assertNull(token); + } + + public void testBackUnigram() throws Exception { + EdgeNGramTokenFilter tokenizer = new EdgeNGramTokenFilter(input, EdgeNGramTokenFilter.Side.BACK, 1, 1); + Token token = null; + token = tokenizer.next(); + assertEquals("(e,4,5)", token.toString()); + token = tokenizer.next(); + assertNull(token); + } + + public void testOversizedNgrams() throws Exception { + EdgeNGramTokenFilter tokenizer = new EdgeNGramTokenFilter(input, EdgeNGramTokenFilter.Side.FRONT, 6, 6); + Token token = null; + token = tokenizer.next(); + assertNull(token); + } + + public void testFrontRangeOfNgrams() throws Exception { + EdgeNGramTokenFilter tokenizer = new EdgeNGramTokenFilter(input, EdgeNGramTokenFilter.Side.FRONT, 1, 3); + Token token = null; + token = tokenizer.next(); + assertEquals("(a,0,1)", token.toString()); + token = tokenizer.next(); + assertEquals("(ab,0,2)", token.toString()); + token = tokenizer.next(); + assertEquals("(abc,0,3)", token.toString()); + token = tokenizer.next(); + assertNull(token); + } + + public void testBackRangeOfNgrams() throws Exception { + EdgeNGramTokenFilter tokenizer = new EdgeNGramTokenFilter(input, EdgeNGramTokenFilter.Side.BACK, 1, 3); + Token token = null; + token = tokenizer.next(); + assertEquals("(e,4,5)", token.toString()); + token = tokenizer.next(); + assertEquals("(de,3,5)", token.toString()); + token = tokenizer.next(); + assertEquals("(cde,2,5)", token.toString()); + token = tokenizer.next(); + assertNull(token); + } +} Added: lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/NGramTokenFilterTest.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/NGramTokenFilterTest.java?view=auto&rev=513876 ============================================================================== --- lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/NGramTokenFilterTest.java (added) +++ lucene/java/trunk/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/NGramTokenFilterTest.java Fri Mar 2 10:19:53 2007 @@ -0,0 +1,139 @@ +package org.apache.lucene.analysis.ngram; + +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import org.apache.lucene.analysis.Token; +import org.apache.lucene.analysis.TokenStream; +import org.apache.lucene.analysis.WhitespaceTokenizer; + +import java.io.StringReader; +import java.util.ArrayList; + +import junit.framework.TestCase; + +/** + * Tests {@link NGramTokenFilter} for correctness. + * @author Otis Gospodnetic + */ +public class NGramTokenFilterTest extends TestCase { + private TokenStream input; + private ArrayList tokens = new ArrayList(); + + public void setUp() { + input = new WhitespaceTokenizer(new StringReader("abcde")); + } + + public void testInvalidInput() throws Exception { + boolean gotException = false; + try { + new NGramTokenFilter(input, 2, 1); + } catch (IllegalArgumentException e) { + gotException = true; + } + assertTrue(gotException); + } + + public void testInvalidInput2() throws Exception { + boolean gotException = false; + try { + new NGramTokenFilter(input, 0, 1); + } catch (IllegalArgumentException e) { + gotException = true; + } + assertTrue(gotException); + } + + public void testUnigrams() throws Exception { + NGramTokenFilter filter = new NGramTokenFilter(input, 1, 1); + + Token token = null; + do { + token = filter.next(); + if (token != null) { + tokens.add(token.toString()); +// System.out.println(token.termText()); +// System.out.println(token); +// Thread.sleep(1000); + } + } while (token != null); + + assertEquals(5, tokens.size()); + ArrayList exp = new ArrayList(); + exp.add("(a,0,1)"); exp.add("(b,1,2)"); exp.add("(c,2,3)"); exp.add("(d,3,4)"); exp.add("(e,4,5)"); + assertEquals(exp, tokens); + } + + public void testBigrams() throws Exception { + NGramTokenFilter filter = new NGramTokenFilter(input, 2, 2); + + Token token = null; + do { + token = filter.next(); + if (token != null) { + tokens.add(token.toString()); +// System.out.println(token.termText()); +// System.out.println(token); +// Thread.sleep(1000); + } + } while (token != null); + + assertEquals(4, tokens.size()); + ArrayList exp = new ArrayList(); + exp.add("(ab,0,2)"); exp.add("(bc,1,3)"); exp.add("(cd,2,4)"); exp.add("(de,3,5)"); + assertEquals(exp, tokens); + } + + public void testNgrams() throws Exception { + NGramTokenFilter filter = new NGramTokenFilter(input, 1, 3); + + Token token = null; + do { + token = filter.next(); + if (token != null) { + tokens.add(token.toString()); +// System.out.println(token.termText()); +// System.out.println(token); +// Thread.sleep(1000); + } + } while (token != null); + + assertEquals(12, tokens.size()); + ArrayList exp = new ArrayList(); + exp.add("(a,0,1)"); exp.add("(b,1,2)"); exp.add("(c,2,3)"); exp.add("(d,3,4)"); exp.add("(e,4,5)"); + exp.add("(ab,0,2)"); exp.add("(bc,1,3)"); exp.add("(cd,2,4)"); exp.add("(de,3,5)"); + exp.add("(abc,0,3)"); exp.add("(bcd,1,4)"); exp.add("(cde,2,5)"); + assertEquals(exp, tokens); + } + + public void testOversizedNgrams() throws Exception { + NGramTokenFilter filter = new NGramTokenFilter(input, 6, 7); + + Token token = null; + do { + token = filter.next(); + if (token != null) { + tokens.add(token.toString()); +// System.out.println(token.termText()); +// System.out.println(token); +// Thread.sleep(1000); + } + } while (token != null); + + assertTrue(tokens.isEmpty()); + } +} From java-commitsgonzalezjerry@example.com Fri Mar 02 18:29:17 2007 Return-Path: <java-commitsgonzalezjerry@example.com> Delivered-To: gonzalezjerry@example.com Received: (qmail 7642 invoked from network); 2 Mar 2007 18:29:17 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2007 18:29:16 -0000 Received: (qmail 1139 invoked by uid 500); 2 Mar 2007 18:29:25 -0000 Delivered-To: gonzalezjerry@example.com Received: (qmail 1117 invoked by uid 500); 2 Mar 2007 18:29:25 -0000 Mailing-List: contact gonzalezjerry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:gonzalezjerry@example.com> List-Unsubscribe: <mailto:gonzalezjerry@example.com> List-Post: <mailto:gonzalezjerry@example.com> List-Id: <java-commits.lucene.apache.org> Reply-To: gonzalezjerry@example.com Delivered-To: mailing list gonzalezjerry@example.com Received: (qmail 1106 invoked by uid 99); 2 Mar 2007 18:29:25 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Mar 2007 10:29:25 -0800 X-ASF-Spam-Status: No, hits=-99.5 required=10.0 tests=ALL_TRUSTED,NO_REAL_NAME X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Mar 2007 10:29:16 -0800 Received: by eris.apache.org (Postfix, from userid 65534) id E01521A981A; Fri, 2 Mar 2007 10:28:55 -0800 (PST) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r513878 - /lucene/java/trunk/contrib/spellchecker/src/java/org/apache/lucene/search/spell/SpellChecker.java Date: Fri, 02 Mar 2007 18:28:55 -0000 To: gonzalezjerry@example.com From: gonzalezjerry@example.com X-Mailer: svnmailer-1.1.0 Message-Id: <gonzalezjerry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org Author: otis Date: Fri Mar 2 10:28:55 2007 New Revision: 513878 URL: http://svn.apache.org/viewvc?view=rev&rev=513878 Log: - LUCENE-786: Javadoc improvements from Karl Wettin and an extra searcher.close() from me Modified: lucene/java/trunk/contrib/spellchecker/src/java/org/apache/lucene/search/spell/SpellChecker.java Modified: lucene/java/trunk/contrib/spellchecker/src/java/org/apache/lucene/search/spell/SpellChecker.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/spellchecker/src/java/org/apache/lucene/search/spell/SpellChecker.java?view=diff&rev=513878&r1=513877&r2=513878 ============================================================================== --- lucene/java/trunk/contrib/spellchecker/src/java/org/apache/lucene/search/spell/SpellChecker.java (original) +++ lucene/java/trunk/contrib/spellchecker/src/java/org/apache/lucene/search/spell/SpellChecker.java Fri Mar 2 10:28:55 2007 @@ -88,6 +88,10 @@ IndexWriter writer = new IndexWriter(spellIndex, null, true); writer.close(); } + // close the old searcher, if there was one + if (searcher != null) { + searcher.close(); + } searcher = new IndexSearcher(this.spellIndex); } @@ -99,6 +103,14 @@ } /** + * As the Lucene similarity that is used to fetch the most relevant n-grammed terms + * is not the same as the edit distance strategy used to calculate the best + * matching spell-checked word from the hits that Lucene found, one usually has + * to retrieve a couple of numSug's in order to get the true best match. + * + * I.e. if numSug == 1, don't count on that suggestion being the best one. + * Thus, you should set this value to <b>at least</b> 5 for a good suggestion. + * * Suggest similar words * @param word String the word you want a spell check done on * @param numSug int the number of suggest words @@ -110,6 +122,14 @@ } /** + * As the Lucene similarity that is used to fetch the most relevant n-grammed terms + * is not the same as the edit distance strategy used to calculate the best + * matching spell-checked word from the hits that Lucene found, one usually has + * to retrieve a couple of numSug's in order to get the true best match. + * + * I.e. if numSug == 1, don't count on that suggestion being the best one. + * Thus, you should set this value to <b>at least</b> 5 for a good suggestion. + * * Suggest similar words (restricted or not to a field of a user index) * @param word String the word you want a spell check done on * @param numSug int the number of suggest words From java-commitsgonzalezjerry@example.com Sun Mar 04 03:23:23 2007 Return-Path: <java-commitsgonzalezjerry@example.com> Delivered-To: gonzalezjerry@example.com Received: (qmail 19091 invoked from network); 4 Mar 2007 03:23:23 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 4 Mar 2007 03:23:23 -0000 Received: (qmail 52330 invoked by uid 500); 4 Mar 2007 03:23:31 -0000 Delivered-To: gonzalezjerry@example.com Received: (qmail 52314 invoked by uid 500); 4 Mar 2007 03:23:31 -0000 Mailing-List: contact gonzalezjerry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:gonzalezjerry@example.com> List-Unsubscribe: <mailto:gonzalezjerry@example.com> List-Post: <mailto:gonzalezjerry@example.com> List-Id: <java-commits.lucene.apache.org> Reply-To: gonzalezjerry@example.com Delivered-To: mailing list gonzalezjerry@example.com Received: (qmail 52302 invoked by uid 99); 4 Mar 2007 03:23:31 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 03 Mar 2007 19:23:31 -0800 X-ASF-Spam-Status: No, hits=-99.5 required=10.0 tests=ALL_TRUSTED,NO_REAL_NAME X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 03 Mar 2007 19:23:21 -0800 Received: by eris.apache.org (Postfix, from userid 65534) id D4A671A981A; Sat, 3 Mar 2007 19:23:01 -0800 (PST) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r514344 - in /lucene/java/trunk: CHANGES.txt src/java/org/apache/lucene/index/SegmentReader.java src/test/org/apache/lucene/search/TestMultiSearcher.java Date: Sun, 04 Mar 2007 03:23:01 -0000 To: gonzalezjerry@example.com From: gonzalezjerry@example.com X-Mailer: svnmailer-1.1.0 Message-Id: <gonzalezjerry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org Author: yonik Date: Sat Mar 3 19:23:00 2007 New Revision: 514344 URL: http://svn.apache.org/viewvc?view=rev&rev=514344 Log: only open .nrm file once: LUCENE-821 Modified: lucene/java/trunk/CHANGES.txt lucene/java/trunk/src/java/org/apache/lucene/index/SegmentReader.java lucene/java/trunk/src/test/org/apache/lucene/search/TestMultiSearcher.java Modified: lucene/java/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/lucene/java/trunk/CHANGES.txt?view=diff&rev=514344&r1=514343&r2=514344 ============================================================================== --- lucene/java/trunk/CHANGES.txt (original) +++ lucene/java/trunk/CHANGES.txt Sat Mar 3 19:23:00 2007 @@ -34,6 +34,10 @@ implementations to be specified via the System property org.apache.lucene.store.FSDirectoryLockFactoryClass. (Mike McCandless) + 4. LUCENE-821: The new single-norm-file introduded by LUCENE-756 + failed to reduce the number of open descriptors since it was still + opened once per field with norms. (yonik) + New features 1. LUCENE-759: Added two n-gram-producing TokenFilters. Modified: lucene/java/trunk/src/java/org/apache/lucene/index/SegmentReader.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/java/org/apache/lucene/index/SegmentReader.java?view=diff&rev=514344&r1=514343&r2=514344 ============================================================================== --- lucene/java/trunk/src/java/org/apache/lucene/index/SegmentReader.java (original) +++ lucene/java/trunk/src/java/org/apache/lucene/index/SegmentReader.java Sat Mar 3 19:23:00 2007 @@ -54,6 +54,9 @@ IndexInput freqStream; IndexInput proxStream; + // optionally used for the .nrm file shared by multiple norms + private IndexInput singleNormStream; + // Compound File Reader when based on a compound file segment CompoundFileReader cfsReader = null; @@ -92,6 +95,17 @@ } this.dirty = false; } + + /** Closes the underlying IndexInput for this norm. + * It is still valid to access all other norm properties after close is called. + * @throws IOException + */ + public void close() throws IOException { + if (in != null && in != singleNormStream) { + in.close(); + } + in = null; + } } private Hashtable norms = new Hashtable(); @@ -457,6 +471,9 @@ byte[] bytes = new byte[maxDoc()]; norms(field, bytes, 0); norm.bytes = bytes; // cache it + // it's OK to close the underlying IndexInput as we have cached the + // norms and will never read them again. + norm.close(); } return norm.bytes; } @@ -473,6 +490,7 @@ Norm norm = (Norm) norms.get(field); if (norm == null) // not an indexed field return; + norm.dirty = true; // mark it dirty normsDirty = true; @@ -494,13 +512,10 @@ return; } - IndexInput normStream = (IndexInput) norm.in.clone(); - try { // read from disk - normStream.seek(norm.normSeek); - normStream.readBytes(bytes, offset, maxDoc()); - } finally { - normStream.close(); - } + // Read from disk. norm.in may be shared across multiple norms and + // should only be used in a synchronized context. + norm.in.seek(norm.normSeek); + norm.in.readBytes(bytes, offset, maxDoc()); } @@ -515,8 +530,27 @@ if (!si.hasSeparateNorms(fi.number)) { d = cfsDir; } - long normSeek = (fileName.endsWith("." + IndexFileNames.NORMS_EXTENSION) ? nextNormSeek : 0); - norms.put(fi.name, new Norm(d.openInput(fileName), fi.number, normSeek)); + + // singleNormFile means multiple norms share this file + boolean singleNormFile = fileName.endsWith("." + IndexFileNames.NORMS_EXTENSION); + IndexInput normInput = null; + long normSeek; + + if (singleNormFile) { + normSeek = nextNormSeek; + if (singleNormStream==null) { + singleNormStream = d.openInput(fileName); + } + // All norms in the .nrm file can share a single IndexInput since + // they are only used in a synchronized context. + // If this were to change in the future, a clone could be done here. + normInput = singleNormStream; + } else { + normSeek = 0; + normInput = d.openInput(fileName); + } + + norms.put(fi.name, new Norm(normInput, fi.number, normSeek)); nextNormSeek += maxDoc; // increment also if some norms are separate } } @@ -527,7 +561,11 @@ Enumeration enumerator = norms.elements(); while (enumerator.hasMoreElements()) { Norm norm = (Norm) enumerator.nextElement(); - norm.in.close(); + norm.close(); + } + if (singleNormStream != null) { + singleNormStream.close(); + singleNormStream = null; } } } Modified: lucene/java/trunk/src/test/org/apache/lucene/search/TestMultiSearcher.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/test/org/apache/lucene/search/TestMultiSearcher.java?view=diff&rev=514344&r1=514343&r2=514344 ============================================================================== --- lucene/java/trunk/src/test/org/apache/lucene/search/TestMultiSearcher.java (original) +++ lucene/java/trunk/src/test/org/apache/lucene/search/TestMultiSearcher.java Sat Mar 3 19:23:00 2007 @@ -149,7 +149,6 @@ // no exception should happen at this point Document d = hits2.doc(i); } - mSearcher2.close(); // test the subSearcher() method: Query subSearcherQuery = parser.parse("id:doc1"); @@ -161,6 +160,7 @@ hits2 = mSearcher2.search(subSearcherQuery); assertEquals(1, hits2.length()); assertEquals(1, mSearcher2.subSearcher(hits2.id(0))); // hit from searchers2[1] + mSearcher2.close(); //-------------------------------------------------------------------- // scenario 3 From java-commitsgonzalezjerry@example.com Sun Mar 04 08:25:54 2007 Return-Path: <java-commitsgonzalezjerry@example.com> Delivered-To: gonzalezjerry@example.com Received: (qmail 68092 invoked from network); 4 Mar 2007 08:25:53 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 4 Mar 2007 08:25:53 -0000 Received: (qmail 31516 invoked by uid 500); 4 Mar 2007 08:26:02 -0000 Delivered-To: gonzalezjerry@example.com Received: (qmail 31503 invoked by uid 500); 4 Mar 2007 08:26:01 -0000 Mailing-List: contact gonzalezjerry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:gonzalezjerry@example.com> List-Unsubscribe: <mailto:gonzalezjerry@example.com> List-Post: <mailto:gonzalezjerry@example.com> List-Id: <java-commits.lucene.apache.org> Reply-To: gonzalezjerry@example.com Delivered-To: mailing list gonzalezjerry@example.com Received: (qmail 31492 invoked by uid 99); 4 Mar 2007 08:26:01 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 04 Mar 2007 00:26:01 -0800 X-ASF-Spam-Status: No, hits=-99.5 required=10.0 tests=ALL_TRUSTED,NO_REAL_NAME X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 04 Mar 2007 00:25:52 -0800 Received: by eris.apache.org (Postfix, from userid 65534) id 3E2841A981A; Sun, 4 Mar 2007 00:25:32 -0800 (PST) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r514370 - in /lucene/java/trunk/src: java/org/apache/lucene/store/RAMDirectory.java test/org/apache/lucene/index/TestIndexReader.java test/org/apache/lucene/store/MockRAMDirectory.java test/org/apache/lucene/store/MockRAMInputStream.java Date: Sun, 04 Mar 2007 08:25:32 -0000 To: gonzalezjerry@example.com From: gonzalezjerry@example.com X-Mailer: svnmailer-1.1.0 Message-Id: <gonzalezjerry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org Author: mikemccand Date: Sun Mar 4 00:25:31 2007 New Revision: 514370 URL: http://svn.apache.org/viewvc?view=rev&rev=514370 Log: LUCENE-820: improve MockRAMDirectory to mimic Windows (can't delete open files); added test case for this bug (this root cause of this bug was already fixed by the fix to LUCENE-821) Added: lucene/java/trunk/src/test/org/apache/lucene/store/MockRAMInputStream.java (with props) Modified: lucene/java/trunk/src/java/org/apache/lucene/store/RAMDirectory.java lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexReader.java lucene/java/trunk/src/test/org/apache/lucene/store/MockRAMDirectory.java Modified: lucene/java/trunk/src/java/org/apache/lucene/store/RAMDirectory.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/java/org/apache/lucene/store/RAMDirectory.java?view=diff&rev=514370&r1=514369&r2=514370 ============================================================================== --- lucene/java/trunk/src/java/org/apache/lucene/store/RAMDirectory.java (original) +++ lucene/java/trunk/src/java/org/apache/lucene/store/RAMDirectory.java Sun Mar 4 00:25:31 2007 @@ -173,7 +173,7 @@ /** Removes an existing file in the directory. * @throws IOException if the file does not exist */ - public synchronized final void deleteFile(String name) throws IOException { + public synchronized void deleteFile(String name) throws IOException { RAMFile file = (RAMFile)fileMap.get(name); if (file!=null) { fileMap.remove(name); @@ -215,7 +215,7 @@ } /** Returns a stream reading an existing file. */ - public final IndexInput openInput(String name) throws IOException { + public IndexInput openInput(String name) throws IOException { RAMFile file; synchronized (this) { file = (RAMFile)fileMap.get(name); Modified: lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexReader.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexReader.java?view=diff&rev=514370&r1=514369&r2=514370 ============================================================================== --- lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexReader.java (original) +++ lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexReader.java Sun Mar 4 00:25:31 2007 @@ -386,6 +386,38 @@ } + // Make sure you can set norms & commit, and there are + // no extra norms files left: + public void testWritingNormsNoReader() throws IOException + { + Directory dir = new MockRAMDirectory(); + IndexWriter writer = null; + IndexReader reader = null; + Term searchTerm = new Term("content", "aaa"); + + // add 1 documents with term : aaa + writer = new IndexWriter(dir, new WhitespaceAnalyzer(), true); + writer.setUseCompoundFile(false); + addDoc(writer, searchTerm.text()); + writer.close(); + + // now open reader & set norm for doc 0 (writes to + // _0_1.s0) + reader = IndexReader.open(dir); + reader.setNorm(0, "content", (float) 2.0); + reader.close(); + + // now open reader again & set norm for doc 0 (writes to _0_2.s0) + reader = IndexReader.open(dir); + reader.setNorm(0, "content", (float) 2.0); + reader.close(); + assertFalse("failed to remove first generation norms file on writing second generation", + dir.fileExists("_0_1.s0")); + + dir.close(); + } + + public void testDeleteReaderWriterConflictUnoptimized() throws IOException{ deleteReaderWriterConflict(false); } Modified: lucene/java/trunk/src/test/org/apache/lucene/store/MockRAMDirectory.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/test/org/apache/lucene/store/MockRAMDirectory.java?view=diff&rev=514370&r1=514369&r2=514370 ============================================================================== --- lucene/java/trunk/src/test/org/apache/lucene/store/MockRAMDirectory.java (original) +++ lucene/java/trunk/src/test/org/apache/lucene/store/MockRAMDirectory.java Sun Mar 4 00:25:31 2007 @@ -19,8 +19,11 @@ import java.io.IOException; import java.io.File; +import java.io.FileNotFoundException; import java.util.Iterator; import java.util.Random; +import java.util.Map; +import java.util.HashMap; /** * This is a subclass of RAMDirectory that adds methods @@ -35,18 +38,37 @@ long maxUsedSize; double randomIOExceptionRate; Random randomState; + boolean noDeleteOpenFile = true; + + // NOTE: we cannot initialize the Map here due to the + // order in which our constructor actually does this + // member initialization vs when it calls super. It seems + // like super is called, then our members are initialized: + Map openFiles; public MockRAMDirectory() throws IOException { super(); + if (openFiles == null) { + openFiles = new HashMap(); + } } public MockRAMDirectory(String dir) throws IOException { super(dir); + if (openFiles == null) { + openFiles = new HashMap(); + } } public MockRAMDirectory(Directory dir) throws IOException { super(dir); + if (openFiles == null) { + openFiles = new HashMap(); + } } public MockRAMDirectory(File dir) throws IOException { super(dir); + if (openFiles == null) { + openFiles = new HashMap(); + } } public void setMaxSizeInBytes(long maxSize) { @@ -68,6 +90,17 @@ } /** + * Emulate windows whereby deleting an open file is not + * allowed (raise IOException). + */ + public void setNoDeleteOpenFile(boolean value) { + this.noDeleteOpenFile = value; + } + public boolean getNoDeleteOpenFile() { + return noDeleteOpenFile; + } + + /** * If 0.0, no exceptions will be thrown. Else this should * be a double 0.0 - 1.0. We will randomly throw an * IOException on the first write to an OutputStream based @@ -91,7 +124,26 @@ } } + public synchronized void deleteFile(String name) throws IOException { + synchronized(openFiles) { + if (noDeleteOpenFile && openFiles.containsKey(name)) { + throw new IOException("MockRAMDirectory: file \"" + name + "\" is still open: cannot delete"); + } + } + super.deleteFile(name); + } + public IndexOutput createOutput(String name) { + if (openFiles == null) { + openFiles = new HashMap(); + } + synchronized(openFiles) { + if (noDeleteOpenFile && openFiles.containsKey(name)) { + // RuntimeException instead of IOException because + // super() does not throw IOException currently: + throw new RuntimeException("MockRAMDirectory: file \"" + name + "\" is still open: cannot overwrite"); + } + } RAMFile file = new RAMFile(this); synchronized (this) { RAMFile existing = (RAMFile)fileMap.get(name); @@ -103,6 +155,27 @@ } return new MockRAMOutputStream(this, file); + } + + public IndexInput openInput(String name) throws IOException { + RAMFile file; + synchronized (this) { + file = (RAMFile)fileMap.get(name); + } + if (file == null) + throw new FileNotFoundException(name); + else { + synchronized(openFiles) { + if (openFiles.containsKey(name)) { + Integer v = (Integer) openFiles.get(name); + v = new Integer(v.intValue()+1); + openFiles.put(name, v); + } else { + openFiles.put(name, new Integer(1)); + } + } + } + return new MockRAMInputStream(this, name, file); } /** Provided for testing purposes. Use sizeInBytes() instead. */ Added: lucene/java/trunk/src/test/org/apache/lucene/store/MockRAMInputStream.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/test/org/apache/lucene/store/MockRAMInputStream.java?view=auto&rev=514370 ============================================================================== --- lucene/java/trunk/src/test/org/apache/lucene/store/MockRAMInputStream.java (added) +++ lucene/java/trunk/src/test/org/apache/lucene/store/MockRAMInputStream.java Sun Mar 4 00:25:31 2007 @@ -0,0 +1,62 @@ +package org.apache.lucene.store; + +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Used by MockRAMDirectory to create an input stream that + * keeps track of when it's been closed. + */ + +public class MockRAMInputStream extends RAMInputStream { + private MockRAMDirectory dir; + private String name; + + /** Construct an empty output buffer. */ + public MockRAMInputStream(MockRAMDirectory dir, String name, RAMFile f) { + super(f); + this.name = name; + this.dir = dir; + } + + public void close() { + super.close(); + synchronized(dir.openFiles) { + Integer v = (Integer) dir.openFiles.get(name); + if (v.intValue() == 1) { + dir.openFiles.remove(name); + } else { + v = new Integer(v.intValue()-1); + dir.openFiles.put(name, v); + } + } + } + + public Object clone() { + MockRAMInputStream clone = (MockRAMInputStream) super.clone(); + synchronized(dir.openFiles) { + if (dir.openFiles.containsKey(name)) { + Integer v = (Integer) dir.openFiles.get(name); + v = new Integer(v.intValue()+1); + dir.openFiles.put(name, v); + } else { + throw new RuntimeException("BUG: cloned file was not open?"); + } + } + return clone; + } +} Propchange: lucene/java/trunk/src/test/org/apache/lucene/store/MockRAMInputStream.java ------------------------------------------------------------------------------ svn:eol-style = native From java-commitsgonzalezjerry@example.com Mon Mar 05 14:28:25 2007 Return-Path: <java-commitsgonzalezjerry@example.com> Delivered-To: gonzalezjerry@example.com Received: (qmail 32650 invoked from network); 5 Mar 2007 14:28:25 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 5 Mar 2007 14:28:25 -0000 Received: (qmail 7556 invoked by uid 500); 5 Mar 2007 14:28:33 -0000 Delivered-To: gonzalezjerry@example.com Received: (qmail 7542 invoked by uid 500); 5 Mar 2007 14:28:33 -0000 Mailing-List: contact gonzalezjerry@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:gonzalezjerry@example.com> List-Unsubscribe: <mailto:gonzalezjerry@example.com> List-Post: <mailto:gonzalezjerry@example.com> List-Id: <java-commits.lucene.apache.org> Reply-To: gonzalezjerry@example.com Delivered-To: mailing list gonzalezjerry@example.com Received: (qmail 7531 invoked by uid 99); 5 Mar 2007 14:28:33 -0000 Received: from herse.apache.org (HELO herse.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Mar 2007 06:28:33 -0800 X-ASF-Spam-Status: No, hits=-99.5 required=10.0 tests=ALL_TRUSTED,NO_REAL_NAME X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO eris.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Mar 2007 06:28:23 -0800 Received: by eris.apache.org (Postfix, from userid 65534) id 0CD991A981A; Mon, 5 Mar 2007 06:28:03 -0800 (PST) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r514675 - in /lucene/java/trunk: ./ src/java/org/apache/lucene/document/ src/java/org/apache/lucene/search/ src/test/org/apache/lucene/search/ Date: Mon, 05 Mar 2007 14:28:02 -0000 To: gonzalezjerry@example.com From: gonzalezjerry@example.com X-Mailer: svnmailer-1.1.0 Message-Id: <gonzalezjerry@example.com> X-Virus-Checked: Checked by ClamAV on apache.org Author: gsingers Date: Mon Mar 5 06:28:01 2007 New Revision: 514675 URL: http://svn.apache.org/viewvc?view=rev&rev=514675 Log: https://issues.apache.org/jira/browse/LUCENE-822 Applied Mark Miller's patch. Added Unit tests to TestRemoteSearchable and TestMultiSearcher. To get RemoteSearchable to work, had to make FieldSelector and FieldSelectorResult serializable. All tests pass. Modified: lucene/java/trunk/CHANGES.txt lucene/java/trunk/src/java/org/apache/lucene/document/FieldSelector.java lucene/java/trunk/src/java/org/apache/lucene/document/FieldSelectorResult.java lucene/java/trunk/src/java/org/apache/lucene/document/MapFieldSelector.java lucene/java/trunk/src/java/org/apache/lucene/search/IndexSearcher.java lucene/java/trunk/src/java/org/apache/lucene/search/MultiSearcher.java lucene/java/trunk/src/java/org/apache/lucene/search/RemoteSearchable.java lucene/java/trunk/src/java/org/apache/lucene/search/Searchable.java lucene/java/trunk/src/test/org/apache/lucene/search/TestMultiSearcher.java lucene/java/trunk/src/test/org/apache/lucene/search/TestRemoteSearchable.java Modified: lucene/java/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/lucene/java/trunk/CHANGES.txt?view=diff&rev=514675&r1=514674&r2=514675 ============================================================================== --- lucene/java/trunk/CHANGES.txt (original) +++ lucene/java/trunk/CHANGES.txt Mon Mar 5 06:28:01 2007 @@ -43,6 +43,8 @@ 1. LUCENE-759: Added two n-gram-producing TokenFilters. (Otis Gospodnetic) + 2. LUCENE-822: Added FieldSelector capabilities to Searchable for use with RemoteSearcher, and other Searchable implementations. (Mark Miller, Grant Ingersoll) + Optimizations ======================= Release 2.1.0 2007-02-14 ======================= Modified: lucene/java/trunk/src/java/org/apache/lucene/document/FieldSelector.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/java/org/apache/lucene/document/FieldSelector.java?view=diff&rev=514675&r1=514674&r2=514675 ============================================================================== --- lucene/java/trunk/src/java/org/apache/lucene/document/FieldSelector.java (original) +++ lucene/java/trunk/src/java/org/apache/lucene/document/FieldSelector.java Mon Mar 5 06:28:01 2007 @@ -1,4 +1,6 @@ package org.apache.lucene.document; + +import java.io.Serializable; /** * Copyright 2004 The Apache Software Foundation * @@ -20,7 +22,7 @@ * what Fields get loaded on a {@link Document} by {@link org.apache.lucene.index.IndexReader#document(int,org.apache.lucene.document.FieldSelector)} * **/ -public interface FieldSelector { +public interface FieldSelector extends Serializable { /** * Modified: lucene/java/trunk/src/java/org/apache/lucene/document/FieldSelectorResult.java URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/java/org/apache/lucene/document/FieldSelectorResult.java?view=diff&rev=514675&r1=514674&r2=514675 ============================================================================== --- lucene/java/trunk/src/java/org/apache/lucene/document/FieldSelectorResult.java (original) +++ lucene/java/trunk/src/java/org/apache/lucene/document/FieldSelectorResult.java Mon Mar 5 06:28:01 2007 @@ -1,4 +1,6 @@ package org.apache.lucene.document; + +import java.io.Serializable; /** * Copyright 2004 The Apache Software Foundation * @@ -20,7 +22,7 @@ * **/ //Replace with an enumerated type in 1.5 -public final class FieldSelectorResult { +public final class FieldSelectorResult implements Serializable { /** * Load this {@link Field} every time the {@link Document} is loaded, reading in the data as it is encounterd. @@ -28,7 +30,7 @@ *<p/> * {@link Document#add(Fieldable)} should be called by the Reader. */ - public static final FieldSelectorResult LOAD = new FieldSelectorResult(0); + public transient static final FieldSelectorResult LOAD = new FieldSelectorResult(0); /** * Lazily load this {@link Field}. This means the {@link Field} is valid, but it may not actually contain its data until * invoked. {@link Document#getField(String)} SHOULD NOT BE USED. {@link Document#getFieldable(String)} is safe to use and should @@ -36,14 +38,14 @@ *<p/> * {@link Document#add(Fieldable)} should be called by the Reader. */ - public static final FieldSelectorResult LAZY_LOAD = new FieldSelectorResult(1); + public transient static final FieldSelectorResult LAZY_LOAD = new FieldSelectorResult(1); /** * Do not load the {@link Field}. {@link Document#getField(String)} and {@link Document#getFieldable(String)} should return null. * {@link Document#add(Fieldable)} is not called. * <p/> * {@link Document#add(Fieldable)} should not be called by the Reader. */ - public static final FieldSelectorResult NO_LOAD = new FieldSelectorResult(2); + public transient static final FieldSelectorResult NO_LOAD = new FieldSelectorResult(2); /** * Load this field as in the {@link #LOAD} case, but immediately return from {@link Field} loading for the {@link Document}. Thus, the * Document may not have its complete set of Fields. {@link Document#getField(String)} and {@link Document#getFieldable(String)} should @@ -51,23 +53,23 @@ * <p/> * {@link Document#add(Fieldable)} should be called by the Reader. */ - public static final FieldSelectorResult LOAD_AND_BREAK = new FieldSelectorResult(3); + public transient static final FieldSelectorResult LOAD_AND_BREAK = new FieldSelectorResult(3); /** * Behaves much like {@link #LOAD} but does not uncompress any compressed data. This is used for internal purposes. * {@link Document#getField(String)} and {@link Document#getFieldable(String)} should not return null. * <p/> * {@link Document#add(Fieldable)} should be called by the Reader. */ - public static final FieldSelectorResult LOAD_FOR_MERGE = new FieldSelectorResult(4); + public transient static final FieldSelectorResult LOAD_FOR_MERGE = new FieldSelectorResult(4); /** Expert: Load the size of this {@link Field} rather than its value. * Size is measured as number of bytes required to store the field == bytes for a binary or any compressed value,
{'source': 'Apache Software Foundation Public Mail Archives'}