From java-usereric91@example.com Sun Jun 01 06:02:43 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 84471 invoked from network); 1 Jun 2008 06:02:43 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jun 2008 06:02:43 -0000 Received: (qmail 29604 invoked by uid 500); 1 Jun 2008 06:02:38 -0000 Delivered-To: eric91@example.com Received: (qmail 29572 invoked by uid 500); 1 Jun 2008 06:02:38 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 29559 invoked by uid 99); 1 Jun 2008 06:02:38 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 31 May 2008 23:02:38 -0700 X-ASF-Spam-Status: No, hits=2.6 required=10.0 tests=DNS_FROM_OPENWHOIS,SPF_HELO_PASS,SPF_PASS,WHOIS_MYPRIVREG X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of eric91@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO kuber.nabble.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 06:01:42 +0000 Received: from [83.161.120.155] (helo=isper.nabble.com) by kuber.nabble.com with esmtp (Exim 4.63) (envelope-from <eric91@example.com>) id 1K2gdn-00042Q-WC for eric91@example.com; Sat, 31 May 2008 23:02:04 -0700 Message-ID: <eric91@example.com> Date: Sat, 31 May 2008 23:01:10 -0700 (PDT) From: syedfa <eric91@example.com> To: eric91@example.com Subject: Displaying and highlighting results from a Wild Card and Fuzzy search using Lucene in Java MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: eric91@example.com X-Virus-Checked: Checked by ClamAV on apache.org Dear Fellow Java and Lucene developers: I am trying to display my results from doing a search of an xml document (some quotes from shakespeare's "Hamlet") using a WildCard and Fuzzy search, and then I'm trying to highlight the keyword(s) in the results, but unfortunately I am having problems. In the code that I have listed below, I am null as my out put for any potential hits I should get, however, the search term I am using (?rr*) should match the word "arrows" that appears in the xml document. Can you see what I am doing wrong? I get the feeling that the problem is with the Analyzer object, but not sure how to highlight my results without it. With a QueryParser constructor, it requires an Analyzer object, but not the WildCard nor Fuzzy constructors. Does anyone have a solution? Thanks in advance to all those who reply. Sincerely; Fayyaz public List wildSearch(File indexDir) throws Exception { List searchResult = new ArrayList(); Directory fsDir=FSDirectory.getDirectory(indexDir); IndexSearcher is=new IndexSearcher(fsDir); Analyzer analyser = new StandardAnalyzer(); Query parser=new WildcardQuery(new Term("LINES", "?rr*")); long start=new Date().getTime(); Hits hits=is.search(parser); long end=new Date().getTime(); QueryScorer scorer = new QueryScorer(parser); Highlighter highlighter = new Highlighter(scorer); Highlighter high = new Highlighter(scorer); Fragmenter fragmenter = new NullFragmenter(); Fragmenter fragment = new SimpleFragmenter(250); highlighter.setTextFragmenter(fragmenter); high.setTextFragmenter(fragment); for(int i=0; i<hits.length(); i++){ Document doc=hits.doc(i); String lns = doc.get("LINES"); TokenStream lines = analyser.tokenStream("LINES", new StringReader(lns)); CachingTokenFilter filter = new CachingTokenFilter(lines); String highlightedLines = highlighter.getBestFragment(filter, lns); filter.reset(); String highlight = high.getBestFragment(filter, lns); SearchResult resultBean = new SearchResult(); resultBean.setNarrator(hits.doc(i).get("SPEAKER")); resultBean.setHitResult(highlight); resultBean.setQuote(highlightedLines); searchResult.add(resultBean); System.out.println(resultBean.getNarrator()); System.out.println(resultBean.getHitResult()); System.out.println(""); System.out.println(resultBean.getQuote()); System.out.println(""); } System.err.println("Found " + hits.length() + " document(s)(in " + (end-start) + " milliseconds) that matched query '" + "':"); return searchResult; } The xml document that I am searching through is listed below: <PLAY> <TITLE>The Tragedy of Hamlet, Prince of Denmark</TITLE> <SPEECH> <SPEAKER>LORD POLONIUS</SPEAKER> <LINES>Yet here, Laertes! aboard, aboard, for shame! The wind sits in the shoulder of your sail, And you are stay'd for. There; my blessing with thee! And these few precepts in thy memory See thou character. Give thy thoughts no tongue, Nor any unproportioned thought his act. Be thou familiar, but by no means vulgar. Those friends thou hast, and their adoption tried, Grapple them to thy soul with hoops of steel; But do not dull thy palm with entertainment Of each new-hatch'd, unfledged comrade. Beware Of entrance to a quarrel, but being in, Bear't that the opposed may beware of thee. Give every man thy ear, but few thy voice; Take each man's censure, but reserve thy judgment. Costly thy habit as thy purse can buy, But not express'd in fancy; rich, not gaudy; For the apparel oft proclaims the man, And they in France of the best rank and station Are of a most select and generous chief in that. Neither a borrower nor a lender be; For loan oft loses both itself and friend, And borrowing dulls the edge of husbandry. This above all: to thine ownself be true, And it must follow, as the night the day, Thou canst not then be false to any man. Farewell: my blessing season this in thee!</LINES> </SPEECH> <SPEECH> <SPEAKER>HAMLET</SPEAKER> <LINES>To be, or not to be: that is the question: Whether 'tis nobler in the mind to suffer The slings and arrows of outrageous fortune, Or to take arms against a sea of troubles, And by opposing end them? To die: to sleep; No more; and by a sleep to say we end The heart-ache and the thousand natural shocks That flesh is heir to, 'tis a consummation Devoutly to be wish'd. To die, to sleep; To sleep: perchance to dream: ay, there's the rub; For in that sleep of death what dreams may come When we have shuffled off this mortal coil, Must give us pause: there's the respect That makes calamity of so long life; For who would bear the whips and scorns of time, The oppressor's wrong, the proud man's contumely, The pangs of despised love, the law's delay, The insolence of office and the spurns That patient merit of the unworthy takes, When he himself might his quietus make With a bare bodkin? who would fardels bear, To grunt and sweat under a weary life, But that the dread of something after death, The undiscover'd country from whose bourn No traveller returns, puzzles the will And makes us rather bear those ills we have Than fly to others that we know not of? Thus conscience does make cowards of us all; And thus the native hue of resolution Is sicklied o'er with the pale cast of thought, And enterprises of great pith and moment With this regard their currents turn awry, And lose the name of action.--Soft you now! The fair Ophelia! Nymph, in thy orisons Be all my sins remember'd.</LINES> </SPEECH> <SPEECH> <SPEAKER>HAMLET</SPEAKER> <LINES>To be, or not to be: that is the question: Whether 'tis nobler in the mind to suffer The slings and arrows of outrageous fortune, Or to take arms against a sea of troubles, And by opposing end them? To die: to sleep; No more; and by a sleep to say we end The heart-ache and the thousand natural shocks That flesh is heir to, 'tis a consummation Devoutly to be wish'd. To die, to sleep; To sleep: perchance to dream: ay, there's the rub; For in that sleep of death what dreams may come When we have shuffled off this mortal coil, Must give us pause: there's the respect That makes calamity of so long life; For who would bear the whips and scorns of time, The oppressor's wrong, the proud man's contumely, The pangs of despised love, the law's delay, The insolence of office and the spurns That patient merit of the unworthy takes, When he himself might his quietus make With a bare bodkin? who would fardels bear, To grunt and sweat under a weary life, But that the dread of something after death, The undiscover'd country from whose bourn No traveller returns, puzzles the will And makes us rather bear those ills we have Than fly to others that we know not of? Thus conscience does make cowards of us all; And thus the native hue of resolution Is sicklied o'er with the pale cast of thought, And enterprises of great pith and moment With this regard their currents turn awry, And lose the name of action.--Soft you now! The fair Ophelia! Nymph, in thy orisons Be all my sins remember'd.</LINES> </SPEECH> <SPEECH> <SPEAKER>HAMLET</SPEAKER> <LINES>To be, or not to be: that is the question: Whether 'tis nobler in the mind to suffer The slings and arrows of outrageous fortune, Or to take arms against a sea of troubles, And by opposing end them? To die: to sleep; No more; and by a sleep to say we end The heart-ache and the thousand natural shocks That flesh is heir to, 'tis a consummation Devoutly to be wish'd. To die, to sleep; To sleep: perchance to dream: ay, there's the rub; For in that sleep of death what dreams may come When we have shuffled off this mortal coil, Must give us pause: there's the respect That makes calamity of so long life; For who would bear the whips and scorns of time, The oppressor's wrong, the proud man's contumely, The pangs of despised love, the law's delay, The insolence of office and the spurns That patient merit of the unworthy takes, When he himself might his quietus make With a bare bodkin? who would fardels bear, To grunt and sweat under a weary life, But that the dread of something after death, The undiscover'd country from whose bourn No traveller returns, puzzles the will And makes us rather bear those ills we have Than fly to others that we know not of? Thus conscience does make cowards of us all; And thus the native hue of resolution Is sicklied o'er with the pale cast of thought, And enterprises of great pith and moment With this regard their currents turn awry, And lose the name of action.--Soft you now! The fair Ophelia! Nymph, in thy orisons Be all my sins remember'd.</LINES> </SPEECH> </PLAY> -- View this message in context: http://www.nabble.com/Displaying-and-highlighting-results-from-a-Wild-Card-and-Fuzzy-search-using-Lucene-in-Java-tp17582678p17582678.html Sent from the Lucene - Java Users mailing list eric91@example.com. --------------------------------------------------------------------- To unsubscribe, e-mail: eric91@example.com For additional commands, e-mail: eric91@example.com From java-usereric91@example.com Sun Jun 01 13:30:54 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 73932 invoked from network); 1 Jun 2008 13:30:54 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jun 2008 13:30:54 -0000 Received: (qmail 32535 invoked by uid 500); 1 Jun 2008 13:30:49 -0000 Delivered-To: eric91@example.com Received: (qmail 32503 invoked by uid 500); 1 Jun 2008 13:30:48 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 32492 invoked by uid 99); 1 Jun 2008 13:30:48 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 06:30:48 -0700 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of eric91@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO rv-out-0506.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 13:29:56 +0000 Received: by rv-out-0506.google.com with SMTP id f6so632760rvb.5 for <eric91@example.com>; Sun, 01 Jun 2008 06:30:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; bh=RuCJ7gmLb/+ns8LJSF5nUfJFzfM486VmODvCHR2jjF4=; b=OFWz4O9yQ6mn+VlpGx+KBKN8rlELDJU4Vs8RLtVIbrQ/YdEuszo2Lbjbg9TZ58wN1cwuoY6KoAIJksyJ1KTs0CN+f9I9SApXNnw3pcLmkWGqao3wtHtRkBnkBb2tTBIdvqEkOF3PqZp9RW16TAACWLymaVJYL6a4fIMqYNLKgw8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=Rh7VxiHoEJdD/2zv1pdWea4J0pNuG9EDlH4DjgT1z3uhPzlQ/ayzHWCd0fgXFVv/39pfLDHfQ4tQwyYFNBzYk9QTcoNsZSYETHvyrJs8doCRcrQhXBzD8YfKd2tGyHWO0oCD8cNDEDWe31a/VbgAGui3nTmc9zBDh4c+QjPfJUo= Received: by 83.161.120.155 with SMTP id h10mr1150176wfj.7.1212327012583; Sun, 01 Jun 2008 06:30:12 -0700 (PDT) Received: by 83.161.120.155 with HTTP; Sun, 1 Jun 2008 06:30:12 -0700 (PDT) Message-ID: <eric91@example.com> Date: Sun, 1 Jun 2008 16:30:12 +0300 From: "Doron Cohen" <eric91@example.com> To: eric91@example.com Subject: Re: IndexReader.reopen memory leak In-Reply-To: <eric91@example.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_17012_14700017.1212327012587" References: <eric91@example.com> <eric91@example.com> <eric91@example.com> <eric91@example.com> <eric91@example.com> <eric91@example.com> <eric91@example.com> <eric91@example.com> <eric91@example.com> X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_17012_14700017.1212327012587 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi John, IndexReader newInner=in.reopen(); > if (in!=newInner) > { > in.close(); > this.in=newInner; > > // code to clean up my data > _cache.clear(); > _indexData.load(this, true); > init(_fieldConfig); > } > Just to be sure on this, could you confirm the two appearances above: - in - this.in refer to exactly the same variable? Assuming they are, could you provide some more code: - entire method containing the above code - method reopen() of your FilteredIndexReader. - method newReader() - constructor of FilteredIndexReader if it is invoked from newReader() Regards, Doron ------=_Part_17012_14700017.1212327012587-- From java-usereric91@example.com Sun Jun 01 14:07:46 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 79929 invoked from network); 1 Jun 2008 14:07:46 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jun 2008 14:07:46 -0000 Received: (qmail 61251 invoked by uid 500); 1 Jun 2008 14:07:42 -0000 Delivered-To: eric91@example.com Received: (qmail 61211 invoked by uid 500); 1 Jun 2008 14:07:42 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 61200 invoked by uid 99); 1 Jun 2008 14:07:42 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 07:07:42 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of eric91@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO an-out-0708.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 14:06:53 +0000 Received: by an-out-0708.google.com with SMTP id c37so191149anc.49 for <eric91@example.com>; Sun, 01 Jun 2008 07:07:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:user-agent:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding; bh=+sc7stU4fS5cdjGpoazqoKU2BwmBlav3KQAzOu3hufM=; b=ltr589CsgPUTdD8lOTrebTWgGK1r4KSyoQXyl8EriZckXeDVy2XL+9PTCQalZhMN8lpozNQDyqIrdUfLUe/vsSZcq6fUbyaKhrwefBQknwos7GNtiGxUpTZGxY7dIR3+Ov6pKj6adPUyCTC5yJkCDFlOOKm2UcDH6meDhKSyYYU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding; b=ZgTsmwmH+bk4Fr2lMMFb8hAD+wMrwwIY4KSi68JvXhnc8VmimAqWuw9PE3A9KK2Az1+apm8NTmQq6/x3RFDxU7zaCU+T7v3FzFJ51LCVKcBUkSkVo0WsFBtfz9mXoXL/8JTeTvQRTm1Eya8IScB3Nr6UQDeTUZ8vB/spJ5QK14w= Received: by 83.161.120.155 with SMTP id l8mr12001664ane.121.1212329230234; Sun, 01 Jun 2008 07:07:10 -0700 (PDT) Received: from ?83.161.120.155? ( [83.161.120.155]) by mx.google.com with ESMTPS id z21sm2105904qbc.27.283.161.120.155.07.08 (version=SSLv3 cipher=RC4-MD5); Sun, 01 Jun 2008 07:07:09 -0700 (PDT) Message-ID: <eric91@example.com> Date: Sun, 01 Jun 2008 10:07:07 -0400 From: Mark Miller <eric91@example.com> User-Agent: Thunderbird 83.161.120.155 (X11/20080505) MIME-Version: 1.0 To: eric91@example.com Subject: Re: IndexReader.reopen memory leak References: <eric91@example.com> <eric91@example.com> <eric91@example.com> <eric91@example.com> <eric91@example.com> <eric91@example.com> <eric91@example.com> <eric91@example.com> <eric91@example.com> In-Reply-To: <eric91@example.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org >> > Yes...I constantly index with 8 threads on one writer while searching > with many more threads. Then I let it run for like an hour and watch. > The index is tiny to start and then grows to a moderate size...nothing > crazy. > > I am also reopening a lot on a real index of 3.5 million + docs > though...and I see no leak evidence there either. A couple interesting limitations with these results: In the reopen test I was only using one field. I'll try a lot more. On the 3.5+ million index there are loads of fields, but field norms are off. --------------------------------------------------------------------- To unsubscribe, e-mail: eric91@example.com For additional commands, e-mail: eric91@example.com From java-usereric91@example.com Sun Jun 01 17:10:51 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 43420 invoked from network); 1 Jun 2008 17:10:50 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jun 2008 17:10:50 -0000 Received: (qmail 56457 invoked by uid 500); 1 Jun 2008 17:10:45 -0000 Delivered-To: eric91@example.com Received: (qmail 56428 invoked by uid 500); 1 Jun 2008 17:10:45 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 56417 invoked by uid 99); 1 Jun 2008 17:10:45 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 10:10:45 -0700 X-ASF-Spam-Status: No, hits=0.2 required=10.0 tests=SPF_PASS,URIBL_GREY X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of eric91@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO yw-out-2324.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 17:09:54 +0000 Received: by yw-out-2324.google.com with SMTP id 3so260098ywj.5 for <eric91@example.com>; Sun, 01 Jun 2008 10:10:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; bh=y6F2UiUrtRJyK8c90rWtgNLulWCsCyJoImXG0T388TU=; b=bIyolgIUdGhJh2pN41zUSGd9R3bN0Wo/0lM1ceBMPc/RsuLN72CN/6a5DuSpidCdVeLKo/t3QtTz7rksTfwiqOPoWzISFDovn4+5Raokqu8OGn6fflhVZNyi/jEbiuILp/hpY+zHO7+tCYiX6TywIxHuOQOKmSCKOxBqlYqmuo4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=OM4G8nJjFT0V17mbo1Lus2cgNJa0226Qif/dw3IvJkKe8KMSA0pRFL5SYARik8TUom44nj/B7nBSWqXeFZ+0Pe3Xq6dpRKMDQBdotiL4aVIgBTgoRTleLq/H77rR2kmiu6qN8FWj3GdtMxsdZjxqljEu3w832tRJ0gEVWnVKT+M= Received: by 83.161.120.155 with SMTP id a41mr8788420ybb.181.1212340210540; Sun, 01 Jun 2008 10:10:10 -0700 (PDT) Received: by 83.161.120.155 with HTTP; Sun, 1 Jun 2008 10:10:10 -0700 (PDT) Message-ID: <eric91@example.com> Date: Sun, 1 Jun 2008 14:10:10 -0300 From: "Marcelo Ochoa" <eric91@example.com> To: eric91@example.com Subject: ANN: New release Lucene-Oracle integration MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Virus-Checked: Checked by ClamAV on apache.org Hi All: I am just releasing a new binary distribution of Oracle-Lucene integration by using Lucene-OJVM Data Catridge. Here the change log: * Compiled against Lucene 2.3.2 production release * Used latest API for merging based on RAM usage * Use Writer for deleting during Sync * Confirm 4x improvement during indexing reported by Lucene dev. group * Fix workaround which changes order of the rowids in ODCRIDList * Added an Spanish WikiPedia Analyzer for testing * Reports IOException instead of RunTimeException to signal EOF or File Not Found * Decouple Flush functionality from TableIndexer I would like to say thanks a lot to Michael McCandless for helping to solve nice glitch with Oracle JIT compiler which causes that DocumentsWriter class do not work on 83.161.120.155 release. 11g binary version have a workaround for this problem. Oracle OJVM dev. team told me that this problems its not reproducible on 11.2 and 83.161.120.155 versions. Latest binary dist. can be downloaded at: http://sourceforge.net/project/showfiles.php?group_id=56183&package_id=255524&release_id=603580 Also I have posted a new entry at my blog with some performance experience against Wikipedia Spanish dump uploaded to XMLDB: http://marceloochoa.blogspot.com/2008/06/new-binary-release-of-lucene-oracle.html Latest documentation is at: http://docs.google.com/Doc?id=ddgw7sjp_54fgj9kg Best regards, Marcelo. -- Marcelo F. Ochoa http://marceloochoa.blogspot.com/ http://marcelo.ochoa.googlepages.com/home ______________ Do you Know DBPrism? Look @ DB Prism's Web Site http://www.dbprism.com.ar/index.html More info? Chapter 17 of the book "Programming the Oracle Database using Java & Web Services" http://www.amazon.com/gp/product/1555583296/ Chapter 21 of the book "Professional XML Databases" - Wrox Press http://www.amazon.com/gp/product/1861003587/ Chapter 8 of the book "Oracle & Open Source" - O'Reilly http://www.oreilly.com/catalog/oracleopen/ --------------------------------------------------------------------- To unsubscribe, e-mail: eric91@example.com For additional commands, e-mail: eric91@example.com From java-usereric91@example.com Sun Jun 01 17:16:21 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 44099 invoked from network); 1 Jun 2008 17:16:21 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jun 2008 17:16:21 -0000 Received: (qmail 62120 invoked by uid 500); 1 Jun 2008 17:16:18 -0000 Delivered-To: eric91@example.com Received: (qmail 62083 invoked by uid 500); 1 Jun 2008 17:16:17 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 62072 invoked by uid 99); 1 Jun 2008 17:16:17 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 10:16:17 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of eric91@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO smtprelay10.ispgateway.de) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 17:15:21 +0000 Received: from [83.161.120.155] (helo=p5b25c4ca.dip.t-dialin.net) by smtprelay10.ispgateway.de with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.68) (envelope-from <eric91@example.com>) id 1K2r9k-0004eS-03 for eric91@example.com; Sun, 01 Jun 2008 19:15:44 +0200 From: Daniel Naber <eric91@example.com> To: eric91@example.com Subject: Re: Displaying and highlighting results from a Wild Card and Fuzzy search using Lucene in Java Date: Sun, 1 Jun 2008 19:15:38 +0200 User-Agent: KMail/1.9.6 (enterprise 20070904.708012) References: <eric91@example.com> In-Reply-To: <eric91@example.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <eric91@example.com> X-Df-Sender: 695637 X-Virus-Checked: Checked by ClamAV on apache.org On Sonntag, 1. Juni 2008, syedfa wrote: > I am trying to display my results from doing a search of an xml document > (some quotes from shakespeare's "Hamlet") using a WildCard and Fuzzy > search, and then I'm trying to highlight the keyword(s) in the results, > but unfortunately I am having problems. Please see http://wiki.apache.org/lucene-java/LuceneFAQ#head-75566820ee94a425c7e2950ac61d24e405fbd914 regards Daniel -- http://www.danielnaber.de --------------------------------------------------------------------- To unsubscribe, e-mail: eric91@example.com For additional commands, e-mail: eric91@example.com From java-usereric91@example.com Sun Jun 01 19:16:48 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 76410 invoked from network); 1 Jun 2008 19:16:48 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jun 2008 19:16:48 -0000 Received: (qmail 19030 invoked by uid 500); 1 Jun 2008 19:16:43 -0000 Delivered-To: eric91@example.com Received: (qmail 18992 invoked by uid 500); 1 Jun 2008 19:16:43 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 18981 invoked by uid 99); 1 Jun 2008 19:16:43 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 12:16:43 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO radix.cryptio.net) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 19:15:48 +0000 Received: by radix.cryptio.net (Postfix, from userid 1007) id 7EFC071C0BE; Sun, 1 Jun 2008 12:16:10 -0700 (PDT) Received: from localhost (localhost [83.161.120.155]) by radix.cryptio.net (Postfix) with ESMTP id 7BE5471C07A; Sun, 1 Jun 2008 12:16:10 -0700 (PDT) Date: Sun, 1 Jun 2008 12:16:10 -0700 (PDT) From: Chris Hostetter <eric91@example.com> To: Lucene Users <eric91@example.com> cc: Daniel Freudenberger <eric91@example.com> Subject: RE: how to unsubscribe? In-Reply-To: <000001c8c24a$88ae1e60$9a0a5b20$@eric91@example.com> Message-ID: <eric91@example.com> References: <001b01c8c245$d2ab1ae0$780150a0$@eric91@example.com> <eric91@example.com> <000001c8c24a$88ae1e60$9a0a5b20$@eric91@example.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-ID: <eric91@example.com> X-Virus-Checked: Checked by ClamAV on apache.org : I've already tried this but the subject line is fixed and I wrote a roman to : convince the mail daemon that I'm not interested in spamming.. but it didn't : care :) Silly question, but you were sending your email to "eric91@example.com..." and not "java-user@..." correct? Are you still having a problem, or were you able to unsubscribe? If it is still a problem, my suggestion is to file an "INFRA" bug in Jira using the "Mailing List" component, and attach copies of the email you sent (with full headers) and the response you got (with full headers)... https://issues.apache.org/jira/browse/INFRA -Hoss --------------------------------------------------------------------- To unsubscribe, e-mail: eric91@example.com For additional commands, e-mail: eric91@example.com From java-usereric91@example.com Sun Jun 01 19:24:36 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 77330 invoked from network); 1 Jun 2008 19:24:35 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jun 2008 19:24:35 -0000 Received: (qmail 25678 invoked by uid 500); 1 Jun 2008 19:24:32 -0000 Delivered-To: eric91@example.com Received: (qmail 25642 invoked by uid 500); 1 Jun 2008 19:24:32 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 25631 invoked by uid 99); 1 Jun 2008 19:24:32 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 12:24:32 -0700 X-ASF-Spam-Status: No, hits=0.2 required=10.0 tests=SPF_PASS,WHOIS_MYPRIVREG X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO radix.cryptio.net) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 19:23:42 +0000 Received: by radix.cryptio.net (Postfix, from userid 1007) id 9DCF871C39D; Sun, 1 Jun 2008 12:23:59 -0700 (PDT) Received: from localhost (localhost [83.161.120.155]) by radix.cryptio.net (Postfix) with ESMTP id 9A27771C393 for <eric91@example.com>; Sun, 1 Jun 2008 12:23:59 -0700 (PDT) Date: Sun, 1 Jun 2008 12:23:59 -0700 (PDT) From: Chris Hostetter <eric91@example.com> To: eric91@example.com Subject: Re: Opening an index directory inside a jar In-Reply-To: <eric91@example.com> Message-ID: <eric91@example.com> References: <eric91@example.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Checked: Checked by ClamAV on apache.org : The crux of the issue seems to be that lucene cannot open segments file that : is inside the jar (under luceneFiles/index directory) i'm not entirely sure why it would have problems finding the segments file, but a larger problem is that Lucene needs random access which (last time i checked) isn't available when accessing files in jars... http://www.nabble.com/Accessing-Lucene-Index-stored-in-a-jar-file-to3009604.html ...you cn always include the index in a jar, and then extract it before using it. : unit/integration/functional tests depend on index files to be created. The : manual step of creating the index files breaks the automated CI builds or : some reliance on building the index in some tmp directory. Unfortunately : that approach has issues if we run tests concurrently. Also, building the : index takes a couple of minutes, so generating them on the fly for tests is : expensive and increases the build time. there's no inherent reason why concurrent tests need to collide if you use temp directories -- just have each test create it's own private tmp directory and copy the index (or exactract the index from the jar) to that private directory in the "setUp" method. -Hoss --------------------------------------------------------------------- To unsubscribe, e-mail: eric91@example.com For additional commands, e-mail: eric91@example.com From java-usereric91@example.com Sun Jun 01 19:43:49 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 81750 invoked from network); 1 Jun 2008 19:43:49 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jun 2008 19:43:49 -0000 Received: (qmail 34321 invoked by uid 500); 1 Jun 2008 19:43:44 -0000 Delivered-To: eric91@example.com Received: (qmail 34287 invoked by uid 500); 1 Jun 2008 19:43:44 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 34276 invoked by uid 99); 1 Jun 2008 19:43:44 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 12:43:44 -0700 X-ASF-Spam-Status: No, hits=0.2 required=10.0 tests=SPF_PASS,WHOIS_MYPRIVREG X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO radix.cryptio.net) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 19:42:49 +0000 Received: by radix.cryptio.net (Postfix, from userid 1007) id 28D5571C346; Sun, 1 Jun 2008 12:43:11 -0700 (PDT) Received: from localhost (localhost [83.161.120.155]) by radix.cryptio.net (Postfix) with ESMTP id 24CDF71C331; Sun, 1 Jun 2008 12:43:11 -0700 (PDT) Date: Sun, 1 Jun 2008 12:43:11 -0700 (PDT) From: Chris Hostetter <eric91@example.com> To: eric91@example.com, eric91@example.com Subject: Re: =?UTF-8?Q?date=20filter=20filtering=20out=20non-dated=20items=3F?= In-Reply-To: <{{EMAIL+SOCIAL_SECURITY_NUMBER}}> Message-ID: <eric91@example.com> References: <{{EMAIL+SOCIAL_SECURITY_NUMBER}}> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Checked: Checked by ClamAV on apache.org : While I could add a future date to these documents, this kind of feels : hackish and I would be interested in other ideas on how to filter out : expired documents. this just came up on the solr list, the answer is equally applicable but note that you'll need to combine it with some other query class (MatchAllDocs perhaps)... >> you have to invert your logic. docs that "have not yet expired, or will >> never expire" match the negacted query for "docs expired in the past" http://www.nabble.com/expression-in-an-fq-parameter-fails-to17353677.html#a17375261 -Hoss --------------------------------------------------------------------- To unsubscribe, e-mail: eric91@example.com For additional commands, e-mail: eric91@example.com From java-usereric91@example.com Sun Jun 01 20:17:27 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 86530 invoked from network); 1 Jun 2008 20:17:27 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jun 2008 20:17:27 -0000 Received: (qmail 50636 invoked by uid 500); 1 Jun 2008 20:17:22 -0000 Delivered-To: eric91@example.com Received: (qmail 50596 invoked by uid 500); 1 Jun 2008 20:17:22 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 50585 invoked by uid 99); 1 Jun 2008 20:17:22 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 13:17:22 -0700 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO spunkymail-a12.g.dreamhost.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 20:16:32 +0000 Received: from [83.161.120.155] (adsl-074-229-189-244.sip.rmo.bellsouth.net [83.161.120.155]) by spunkymail-a12.g.dreamhost.com (Postfix) with ESMTP id 648757FAA for <eric91@example.com>; Sun, 1 Jun 2008 13:16:48 -0700 (PDT) Message-Id: <eric91@example.com> From: Grant Ingersoll <eric91@example.com> To: eric91@example.com In-Reply-To: <eric91@example.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v924) Subject: Re: Lucene search time in real production use? Date: Sun, 1 Jun 2008 16:16:46 -0400 References: <eric91@example.com> <eric91@example.com> X-Mailer: Apple Mail (2.924) X-Virus-Checked: Checked by ClamAV on apache.org Those benchmarks are pretty old, I think. -Grant On May 31, 2008, at 12:28 PM, Karl Wettin wrote: > > 31 maj 2008 kl. 14.25 skrev lucene user: >> What are some average search and retrieval times for Lucene queries >> in real >> production use? Would people include relevant stuff like the number >> of >> documents in your index, etc.? >> >> Thanks for your help! > > http://lucene.apache.org/java/docs/benchmarks.html > > How well it works depends on many factors. What your corpus looks > like, load on index, what sort of queries are executed, hardware, et > c. You can estimate how your application will work by using and > extending the benchmarker contrib tool. > > > karl > > --------------------------------------------------------------------- > To unsubscribe, e-mail: eric91@example.com > For additional commands, e-mail: eric91@example.com > -------------------------- Grant Ingersoll http://www.lucidimagination.com Lucene Helpful Hints: http://wiki.apache.org/lucene-java/BasicsOfPerformance http://wiki.apache.org/lucene-java/LuceneFAQ --------------------------------------------------------------------- To unsubscribe, e-mail: eric91@example.com For additional commands, e-mail: eric91@example.com From java-usereric91@example.com Sun Jun 01 20:30:17 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 90894 invoked from network); 1 Jun 2008 20:30:17 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jun 2008 20:30:17 -0000 Received: (qmail 58927 invoked by uid 500); 1 Jun 2008 20:30:14 -0000 Delivered-To: eric91@example.com Received: (qmail 58888 invoked by uid 500); 1 Jun 2008 20:30:14 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 58877 invoked by uid 99); 1 Jun 2008 20:30:14 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 13:30:14 -0700 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO spunkymail-a12.g.dreamhost.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 20:29:09 +0000 Received: from [83.161.120.155] (adsl-074-229-189-244.sip.rmo.bellsouth.net [83.161.120.155]) by spunkymail-a12.g.dreamhost.com (Postfix) with ESMTP id 20B487FAA for <eric91@example.com>; Sun, 1 Jun 2008 13:29:31 -0700 (PDT) Message-Id: <eric91@example.com> From: Grant Ingersoll <eric91@example.com> To: eric91@example.com In-Reply-To: <eric91@example.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v924) Subject: Re: Frequencies sorted by frequencies Date: Sun, 1 Jun 2008 16:29:29 -0400 References: <eric91@example.com> <eric91@example.com> <eric91@example.com> X-Mailer: Apple Mail (2.924) X-Virus-Checked: Checked by ClamAV on apache.org I don't know of a way, sorry. Most of the Similarity methods do not take a field name. On May 29, 2008, at 9:20 AM, Hider, Sandy wrote: > Thanks for taking the time to answer. I see what you mean. The thing > is I also plan on using the standard score. Would there be a way to > use > the both the standard score and the TF-only Score in a single index? > > Sandy > > > -----Original Message----- > From: Grant Ingersoll [mailto:eric91@example.com] > Sent: Wednesday, May 28, 2008 2:34 PM > To: eric91@example.com > Subject: Re: Frequencies sorted by frequencies > > I think you could override all the Similarity factors except tf() with > 1, such that the term frequency is the only factor in the scoring. > Then you just submit the term as a query. Note, I think you will need > to override the similarity during indexing, too, so that norm length > is > turned off, too. Note, I haven't tried it :-). Use the explain() > functionality to double check. At any rate, it should be quick to > test. > > See > http://lucene.apache.org/java/2_3_2/api/org/apache/lucene/search/Similar > ity.html > > -Grant > > > On May 28, 2008, at 10:48 AM, Hider, Sandy wrote: > >> Hi All, >> I am trying to figure out a quick way to find the top N documents >> sorted by frequency of a term. >> >> I found: >> >> IndexRead.termDocs() >> >> which provides an enumeration of doc() and freq() but it returns an >> enumeration sorted by doc number. Is there a way to get the results >> sorted by freq? Or is there another query I can run the find these >> results? >> >> Thanks in advance, >> >> Sandy >> > > -------------------------- > Grant Ingersoll > http://www.lucidimagination.com > > Lucene Helpful Hints: > http://wiki.apache.org/lucene-java/BasicsOfPerformance > http://wiki.apache.org/lucene-java/LuceneFAQ > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: eric91@example.com > For additional commands, e-mail: eric91@example.com > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: eric91@example.com > For additional commands, e-mail: eric91@example.com > -------------------------- Grant Ingersoll http://www.lucidimagination.com Lucene Helpful Hints: http://wiki.apache.org/lucene-java/BasicsOfPerformance http://wiki.apache.org/lucene-java/LuceneFAQ --------------------------------------------------------------------- To unsubscribe, e-mail: eric91@example.com For additional commands, e-mail: eric91@example.com From java-usereric91@example.com Sun Jun 01 20:45:14 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 94113 invoked from network); 1 Jun 2008 20:45:14 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jun 2008 20:45:14 -0000 Received: (qmail 66094 invoked by uid 500); 1 Jun 2008 20:45:09 -0000 Delivered-To: eric91@example.com Received: (qmail 66054 invoked by uid 500); 1 Jun 2008 20:45:09 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 66043 invoked by uid 99); 1 Jun 2008 20:45:09 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 13:45:09 -0700 X-ASF-Spam-Status: No, hits=2.4 required=10.0 tests=MSGID_MULTIPLE_AT,SPF_HELO_PASS,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO moutng.kundenserver.de) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 20:44:20 +0000 Received: from DanielPC (p4FC75F16.dip.t-dialin.net [83.161.120.155]) by mrelayeu.kundenserver.de (node=mrelayeu5) with ESMTP (Nemesis) id 0ML25U-1K2uPo1QRB-0007Ip; Sun, 01 Jun 2008 22:44:33 +0200 From: "Daniel Freudenberger" <eric91@example.com> To: "'Chris Hostetter'" <eric91@example.com>, "'Lucene Users'" <eric91@example.com> References: <001b01c8c245$d2ab1ae0$780150a0$@eric91@example.com> <eric91@example.com> <000001c8c24a$88ae1e60$9a0a5b20$@eric91@example.com> <eric91@example.com> In-Reply-To: <eric91@example.com> Subject: RE: how to unsubscribe? Date: Sun, 1 Jun 2008 22:44:30 +0200 Message-ID: <000001c8c428$4b3b90f0$e1b2b2d0$@eric91@example.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcjEG/vvX+t4jfNwSii+rPAlEWbuygAC/p7g Content-Language: de X-Provags-ID: V01U2FsdGVkX18NRnYTQtqg9jMGm/V+7tldHdQTCosVoxwaIwd r/ZEhmPqIPNIcCUWFpYO62r19bd4hk8PDfZxYrvSY1DHxZje8J 91Q/OsAz/s2JmKTdxojwQYmQS9HnQLQP9z58jog8KjgEmLwEZ3 kfg== X-Virus-Checked: Checked by ClamAV on apache.org As you can see I'm still part of this list. I'll submit a bug report. Thanks in advance, Daniel -----Original Message----- From: Chris Hostetter [mailto:eric91@example.com] Sent: Sunday, June 01, 2008 9:16 PM To: Lucene Users Cc: Daniel Freudenberger Subject: RE: how to unsubscribe? : I've already tried this but the subject line is fixed and I wrote a roman to : convince the mail daemon that I'm not interested in spamming.. but it didn't : care :) Silly question, but you were sending your email to "eric91@example.com..." and not "java-user@..." correct? Are you still having a problem, or were you able to unsubscribe? If it is still a problem, my suggestion is to file an "INFRA" bug in Jira using the "Mailing List" component, and attach copies of the email you sent (with full headers) and the response you got (with full headers)... https://issues.apache.org/jira/browse/INFRA -Hoss --------------------------------------------------------------------- To unsubscribe, e-mail: eric91@example.com For additional commands, e-mail: eric91@example.com From java-usereric91@example.com Mon Jun 02 04:37:09 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 78649 invoked from network); 2 Jun 2008 04:37:07 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Jun 2008 04:37:07 -0000 Received: (qmail 31722 invoked by uid 500); 2 Jun 2008 04:37:03 -0000 Delivered-To: eric91@example.com Received: (qmail 31692 invoked by uid 500); 2 Jun 2008 04:37:03 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 31678 invoked by uid 99); 2 Jun 2008 04:37:03 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 21:37:02 -0700 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS,WHOIS_MYPRIVREG X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of eric91@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO wf-out-1314.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Jun 2008 04:36:13 +0000 Received: by wf-out-1314.google.com with SMTP id 28so726937wfc.20 for <eric91@example.com>; Sun, 01 Jun 2008 21:36:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; bh=vT9w1tvpVpT3hua0apQ4pXvEQJe55UiCTsdNn5Mjf/8=; b=pWwl8Y7dEN8wh9+G7CfQquJ+y7LPz+NKBjZSa42IAS5fA3feek0KY2xVZo4fmJqWLjE3ubSA7/bLa81S8ynt6eBsY1fsFW0I2AFqOM/L+y7jzghLv47haewmr1ZFJetoYBWd/tS1aa5HgY3QY/cIoi1eq15uIOHQuqqdZXZpLdE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=kNVQLnVkeDwZJwmd0AkLfgxkaVP6lWT4gcw4zxztTQFi0+qovG0V8nEfKFaQEF2HnXOy9Y5h4pj4ahMr3pNgojGAY+5eA1rjbNAJhfLERBqrvgDv1nS11JmamSbW1KSMXV7UL9V5EQZnr7lS+kSfk1HpL1Z6AVRpw9ETw9RUbn4= Received: by 83.161.120.155 with SMTP id a13mr3359054wfg.224.1212381390129; Sun, 01 Jun 2008 21:36:30 -0700 (PDT) Received: by 83.161.120.155 with HTTP; Sun, 1 Jun 2008 21:36:29 -0700 (PDT) Message-ID: <eric91@example.com> Date: Mon, 2 Jun 2008 07:36:29 +0300 From: "Doron Cohen" <eric91@example.com> To: eric91@example.com Subject: Re: Opening an index directory inside a jar In-Reply-To: <eric91@example.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_18552_30817861.1212381390121" References: <eric91@example.com> <eric91@example.com> X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_18552_30817861.1212381390121 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline > > : The crux of the issue seems to be that lucene cannot open segments file > that > : is inside the jar (under luceneFiles/index directory) > > i'm not entirely sure why it would have problems finding the segments > file, but a larger problem is that Lucene needs random access which (last > time i checked) isn't available when accessing files in jars... > > > http://www.nabble.com/Accessing-Lucene-Index-stored-in-a-jar-file-to3009604.html > I bumped into (though never used) http://commons.apache.org/vfs/apidocs/org/apache/commons/vfs/provider/jar/JarFileSystem.html There, FileContent has this method getRandomAccessContent(org.apache.commons.vfs.util.RandomAccessMode) so it seems worth exploring. HTH, Doron ------=_Part_18552_30817861.1212381390121-- From java-usereric91@example.com Mon Jun 02 05:20:20 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 86720 invoked from network); 2 Jun 2008 05:20:20 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Jun 2008 05:20:20 -0000 Received: (qmail 47719 invoked by uid 500); 2 Jun 2008 05:20:16 -0000 Delivered-To: eric91@example.com Received: (qmail 47691 invoked by uid 500); 2 Jun 2008 05:20:16 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 47680 invoked by uid 99); 2 Jun 2008 05:20:16 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 22:20:16 -0700 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of eric91@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO rv-out-0506.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Jun 2008 05:19:28 +0000 Received: by rv-out-0506.google.com with SMTP id f6so921170rvb.5 for <eric91@example.com>; Sun, 01 Jun 2008 22:19:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; bh=cODnMXYlBEkxPJBcUiKUIKP59fId1m5X2tOLdgHEbIw=; b=Kq+xWtUn1mYsGFI8Q7fV7+Xxz9u+TkGwAZpTY+7FcnAjoe+AbdlJOgcHbnmWP00kfpHeE1aiHLGvclbtKbTY3jYOi6NP17ZHWq3Xpgsa5mfbSXOsQ/5RiVJYovuuFZ52KdbbgQdy5LfrKaVMslTd80O0nNev3ST7m/lS+0znYZs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=CZXWt3XQXlPoUXZEWXwG4sH8UFWUlxxHzVG6g81LbdW7LaxfsPOHDOfGe+9Mtw1tbkiQmM6l/jlxWvJjPN9dXWYfCKsOEvLfDvp4RnBrFQR7+Bq+mMor6DrWdBA4z5b02gLPM29jP21GtxzHH1Bu8Jxv28UromSOigw87ypords= Received: by 83.161.120.155 with SMTP id n13mr4677048rvq.184.1212383984964; Sun, 01 Jun 2008 22:19:44 -0700 (PDT) Received: by 83.161.120.155 with HTTP; Sun, 1 Jun 2008 22:19:44 -0700 (PDT) Message-ID: <eric91@example.com> Date: Mon, 2 Jun 2008 10:49:44 +0530 From: "Sonu Sudhakar" <eric91@example.com> To: eric91@example.com Subject: Re: Boolean Query Issue In-Reply-To: <eric91@example.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_11430_31902300.1212383984898" References: <eric91@example.com> <eric91@example.com> <eric91@example.com> X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_11430_31902300.1212383984898 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi, I have done some more analysis on this issue. I think it is related to lucene's default operator. I am getting excat results, when I sets the default operator as 'OR', but facing problem when setting the default operator as 'AND'. The following are the lucene QueryParser outputs for both cases. Query :* TTL:store AND TTL:data OR TTL:variable *1. When lucene default operator is '*OR' *QueryParser output using toString method: * +TTL:store +TTL:data TTL:variable *2. When lucene default operator is '*AND' *QueryParser output using toString method: *+TTL:store TTL:data TTL:variable *The output of second case is confusing me. Could anybody please give me an explanation for this behavior? Thanks, Sonu On Thu, May 29, 2008 at 3:49 PM, Sonu Sudhakar <eric91@example.com> wrote: > Erick, > > Thanks for your reply. > > I am working with approximately 1 million documents. They are indexed in 4 > servers. Each document has multiple fields. I am using ParallelMultiSearcher > for searching purpose. > > I tried a few queries in the title(TTL) field. > > i started with a simple query without boolean operators. > > *1. TTL:data => 3733 results (all matches had "data" in title)* > > Then I tried a second one with AND operator. > > *2. TTL:data AND TTL:store => 19 results* > > I analyzed the results. the results had both "data" and "store" in the > title. > > *I then tried OR operator* > > *3. TTL:data AND TTL:store OR TTL:variable* > > I got 3,733 results., same as the query TTL:data. > > I even tried giving a meaningless query > > TTL:data AND TTL:storettttt OR TTL:variablettttt => 3,733 results (The > results were same as that of TTL:data.) > > TTL:data AND TTL:computer OR TTL:device => 3,733 results (this also showed > the same results as above) > > The same thing repeats for other cases too. The queries below also behaved > the same way. > i.e. - > > 1. TTL:store AND TTL:data OR TTL:variable => 76 results > 2. TTL:store AND TTL:datatttt OR TTL:variabletttt => 76 results > 3. TTL:store AND TTL:computer OR TTL:device => 76 results > > > 1. TTL:variable AND TTL:data OR TTL:store => 1,496 results > 2. TTL:variable AND TTL:datatttt OR TTL:storetttt => 1,496 results > 3. TTL:variable AND TTL:computer OR TTL:device => 1,496 results > > I hope you have a clearer picture of my issue now. > > Thanks, > Sonu > > > On Wed, May 28, 2008 at 7:09 PM, Erick Erickson <eric91@example.com> > wrote: > >> It's unclear what you *should* expect. How is your data >> distributed? >> >> In other words, how many documents do you have? In this example, >> for instance, >> 1. TTL:data AND TTL:store OR TTL:variable => 3,733 results >> it considered the TTL:data part only. >> >> it's perfecily reasonable if every document that had "variable" in the >> field *also* has "data" and "store" in the field. So your numbers >> don't give us much to work with..... >> >> Remember, though, that Lucene syntax isn't a pure boolean syntax. See >> >> http://wiki.apache.org/lucene-java/BooleanQuerySyntax >> >> And when in doubt parenthesize <G>... >> >> Best >> Erick >> >> On Wed, May 28, 2008 at 7:44 AM, Sonu Sudhakar <eric91@example.com> wrote: >> >> > Hi, >> > >> > I have some issue with boolean queries. >> > >> > I am using Lucene-core-2.3.1. >> > >> > I have done test on boolean query with 3 terms (data, store, variable) >> in >> > my >> > TTL field. The TTL field is indexed and searched using StandardAnalyzer. >> > >> > The three terms when searched individually gave the following result >> > >> > 1. TTL:data => 3,733 results >> > 2. TTL:store => 76 results >> > 3. TTL:variable => 1,496 results >> > >> > But found issue when combining these terms with boolean operators. >> > >> > e.g. >> > 1. TTL:data AND TTL:store OR TTL:variable => 3,733 results >> > it considered the TTL:data part only. >> > >> > 2. TTL:store AND TTL:data OR TTL:variable => 76 results >> > it considered the TTL:store part only. >> > >> > 3. TTL:variable AND TTL:data OR TTL:store => 1,496 results >> > it considered the TTL:variable part only. >> > >> > But I am getting correct result when combining terms with 'AND' >> operator. I >> > think the issue is with 'OR' operator. >> > >> > >> > Could anybody give an explanation for this behavior of lucene? >> > Could you give suggestions to rectify this? >> > >> > Thanks, >> > Sonu >> > >> > > ------=_Part_11430_31902300.1212383984898-- From java-usereric91@example.com Mon Jun 02 05:24:44 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 87296 invoked from network); 2 Jun 2008 05:24:44 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Jun 2008 05:24:44 -0000 Received: (qmail 50488 invoked by uid 500); 2 Jun 2008 05:24:40 -0000 Delivered-To: eric91@example.com Received: (qmail 50444 invoked by uid 500); 2 Jun 2008 05:24:40 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 50433 invoked by uid 99); 2 Jun 2008 05:24:40 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2008 22:24:40 -0700 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of eric91@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO wf-out-1314.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Jun 2008 05:23:52 +0000 Received: by wf-out-1314.google.com with SMTP id 28so739449wfc.20 for <eric91@example.com>; Sun, 01 Jun 2008 22:24:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; bh=Eosqr+hbLM7sHuIE2bn/PSDYDTns/vv1F3Oamti8BV8=; b=YQlqaoFgNqd5mTAhdfycF7VLqLeMSC109QGrXsRx5pM7ZFXdGzpvY1r5Zm/W+LLtfaiPWENvD0CoJYsNKfvfRgMSRCZ/K2K9Lv5D910JM1UKfGCAabmJ1aTq6kJ5UxscLsCbPZNilIPpQ0Tc3zdTO8lLapX0NJDs/o4EJyhi8Gk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=BTf80hR2zlLJJY78UeX4sqVOqYAj1oW+Me+ade6moyBnFbjHEhqhaEtTljOiLvexs23E3CVLyYypM1M4r0R6O2iDO4ef3A8QurAts8i+pY3IV7T8tuO3ig457TDnvDYD00qZrFyNKUwcjVvYSEpwPBfDu+36fYfy70dLmE5wkeg= Received: by 83.161.120.155 with SMTP id l4mr154860wff.183.1212384248749; Sun, 01 Jun 2008 22:24:08 -0700 (PDT) Received: by 83.161.120.155 with HTTP; Sun, 1 Jun 2008 22:24:08 -0700 (PDT) Message-ID: <eric91@example.com> Date: Mon, 2 Jun 2008 08:24:08 +0300 From: "Doron Cohen" <eric91@example.com> To: eric91@example.com Subject: Re: How to add PageRank score with lucene's relevant score in sorting In-Reply-To: <eric91@example.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_18608_3757963.1212384248735" References: <eric91@example.com> X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_18608_3757963.1212384248735 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi Jarvis, > I have a problem that how to "combine" two score to sort the search > result documents. > for example I have 10 million pages in lucene index , and i know their > pagerank scores. i give a query to it , every docs returned have a > lucene-score, mark it as R (relevant score), and i also have its > pagerank score, mark it as P, what i need is i want to sort the search > result base on the value "P+R". You know if i store the pagerank score in > index and get it every search time , then compute P+R , then sort it , this > way is too slow. in my system , when the search hits 500000 result , the > sort may cost about 20s. > Check CustomScoreQuery in http://lucene.apache.org/java/2_3_2/api/core/org/apache/lucene/search/function/package-summary.html Probably something like this: - implement ValueSource on top of the pagerank values, - create a valueSourceQuery on top of it, - create a customScoreQuery on top of the original query and the valueSourceQuery. Note that by default, customScoreQuery multiplies the scores, but you can override this. Doron ------=_Part_18608_3757963.1212384248735-- From java-usereric91@example.com Mon Jun 02 12:50:14 2008 Return-Path: <java-usereric91@example.com> Delivered-To: eric91@example.com Received: (qmail 28316 invoked from network); 2 Jun 2008 12:50:14 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Jun 2008 12:50:14 -0000 Received: (qmail 46342 invoked by uid 500); 2 Jun 2008 12:50:09 -0000 Delivered-To: eric91@example.com Received: (qmail 46320 invoked by uid 500); 2 Jun 2008 12:50:09 -0000 Mailing-List: contact eric91@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:eric91@example.com> List-Unsubscribe: <mailto:eric91@example.com> List-Post: <mailto:eric91@example.com> List-Id: <java-user.lucene.apache.org> Reply-To: eric91@example.com Delivered-To: mailing list eric91@example.com Received: (qmail 46304 invoked by uid 99); 2 Jun 2008 12:50:09 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Jun 2008 05:50:09 -0700 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of eric91@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO fg-out-1718.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Jun 2008 12:49:20 +0000 Received: by fg-out-1718.google.com with SMTP id l27so906555fgb.27 for <eric91@example.com>; Mon, 02 Jun 2008 05:49:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; bh=AuqghmB7nwyOCJ/8INCY+OMUsX1O4Co8VBiMc3bwFyQ=; b=vGh+2kizWLp2eI2qKYE8rgFPSDXjaljFxZgELxhOk7NwhDYJZ33Cs302Hz8jQkHHybYUNB3xpNI3oVknDWGTx6oscFkKG7eBX90oYMpM49yXqFIaj5Q1FNR+k8SpOOXVUKatd6kw/PiojqDVc49UsFq0stedINbJXBP1FtKZgZ4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=FoMS6hbr/gbPa0A9dge0n9UFPSHxD5qyvkU6OhF1NPAv1GErjPZ1/JiFuGN+UoxMGx63sVgI9wn4LNRwcPAMpj7FnssyhpP7Ug6YUhCoryfkTXhds6boMximsttkrpg6LXrTwM6K1/U5BKhPgOnm240quYCwih7riPKRgXDIi5Y= Received: by 83.161.120.155 with SMTP id v10mr6316123fgb.39.1212410976135; Mon, 02 Jun 2008 05:49:36 -0700 (PDT) Received: by 83.161.120.155 with HTTP; Mon, 2 Jun 2008 05:49:36 -0700 (PDT) Message-ID: <eric91@example.com> Date: Mon, 2 Jun 2008 08:49:36 -0400 From: "Erick Erickson" <eric91@example.com> To: eric91@example.com Subject: Re: Boolean Query Issue In-Reply-To: <eric91@example.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_8517_12106446.1212410976143" References: <eric91@example.com> <eric91@example.com> <eric91@example.com> <eric91@example.com> X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_8517_12106446.1212410976143 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline You need to include your code I think. This makes no sense on a quick look, so unless we see some code it'll be hard to know whether we're looking at anything relevant. Best Erick On Mon, Jun 2, 2008 at 1:19 AM, Sonu Sudhakar <eric91@example.com> wrote: > Hi, > > I have done some more analysis on this issue. I think it is related to > lucene's default operator. > I am getting excat results, when I sets the default operator as 'OR', but > facing problem when setting the default operator as 'AND'. > > The following are the lucene QueryParser outputs for both cases. > > Query :* TTL:store AND TTL:data OR TTL:variable > > *1. When lucene default operator is '*OR' > > *QueryParser output using toString method: * > +TTL:store +TTL:data TTL:variable > > > *2. When lucene default operator is '*AND' > > *QueryParser output using toString method: > *+TTL:store TTL:data TTL:variable > > *The output of second case is confusing me. > > Could anybody please give me an explanation for this behavior? > > Thanks, > Sonu > > On Thu, May 29, 2008 at 3:49 PM, Sonu Sudhakar <eric91@example.com> wrote: > > > Erick, > > > > Thanks for your reply. > > > > I am working with approximately 1 million documents. They are indexed in > 4 > > servers. Each document has multiple fields. I am using > ParallelMultiSearcher > > for searching purpose. > > > > I tried a few queries in the title(TTL) field. > > > > i started with a simple query without boolean operators. > > > > *1. TTL:data => 3733 results (all matches had "data" in title)* > > > > Then I tried a second one with AND operator. > > > > *2. TTL:data AND TTL:store => 19 results* > > > > I analyzed the results. the results had both "data" and "store" in the > > title. > > > > *I then tried OR operator* > > > > *3. TTL:data AND TTL:store OR TTL:variable* > > > > I got 3,733 results., same as the query TTL:data. > > > > I even tried giving a meaningless query > > > > TTL:data AND TTL:storettttt OR TTL:variablettttt => 3,733 results (The > > results were same as that of TTL:data.) > > > > TTL:data AND TTL:computer OR TTL:device => 3,733 results (this also > showed > > the same results as above) > > > > The same thing repeats for other cases too. The queries below also > behaved > > the same way. > > i.e. - > > > > 1. TTL:store AND TTL:data OR TTL:variable => 76 results > > 2. TTL:store AND TTL:datatttt OR TTL:variabletttt => 76 results > > 3. TTL:store AND TTL:computer OR TTL:device => 76 results > > > > > > 1. TTL:variable AND TTL:data OR TTL:store => 1,496 results > > 2. TTL:variable AND TTL:datatttt OR TTL:storetttt => 1,496 results > > 3. TTL:variable AND TTL:computer OR TTL:device => 1,496 results > > > > I hope you have a clearer picture of my issue now. > > > > Thanks, > > Sonu > > > >
{'source': 'Apache Software Foundation Public Mail Archives'}