From lucene-net-user-returnchristophersmith@example.com Tue Feb 01 23:58:09 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 5847 invoked from network); 1 Feb 2011 23:58:08 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Feb 2011 23:58:08 -0000 Received: (qmail 61868 invoked by uid 500); 1 Feb 2011 23:58:08 -0000 Delivered-To: christophersmith@example.com Received: (qmail 61795 invoked by uid 500); 1 Feb 2011 23:58:07 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 61787 invoked by uid 99); 1 Feb 2011 23:58:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Feb 2011 23:58:07 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,RFC_ABUSE_POST,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of christophersmith@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO bay0-omc4-s14.bay0.hotmail.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Feb 2011 23:57:57 +0000 Received: from BAY138-W7 ([83.161.120.155]) by bay0-omc4-s14.bay0.hotmail.com with Microsoft SMTPSVC231.721.8345); Tue, 1 Feb 2011 15:57:36 -0800 Message-ID: <christophersmith@example.com> Content-Type: multipart/alternative; boundary="_9ae59f8d-9ad9-4d3f-ae70-d3e4d7ae09aa_" X-Originating-IP: [83.161.120.155] From: Lucas E Wall <christophersmith@example.com> To: <christophersmith@example.com> Subject: Question Date: Tue, 1 Feb 2011 18:57:35 -0500 Importance: Normal MIME-Version: 1.0 X-OriginalArrivalTime: 01 Feb 2011 23:57:36.0518 (UTC) FILETIME=[CCE72260:01CBC26B] X-Virus-Checked: Checked by ClamAV on apache.org --_9ae59f8d-9ad9-4d3f-ae70-d3e4d7ae09aa_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I am new to Lucene and have the following questions. What is the best way = to understand what is required to install Lucene in a server? Also=2C can = i make Lucene run searches on links to xml data on the web?Thanks = = --_9ae59f8d-9ad9-4d3f-ae70-d3e4d7ae09aa_-- From lucene-net-user-returnchristophersmith@example.com Wed Feb 02 00:05:36 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 16427 invoked from network); 2 Feb 2011 00:05:36 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Feb 2011 00:05:36 -0000 Received: (qmail 72882 invoked by uid 500); 2 Feb 2011 00:05:35 -0000 Delivered-To: christophersmith@example.com Received: (qmail 72830 invoked by uid 500); 2 Feb 2011 00:05:35 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 72822 invoked by uid 99); 2 Feb 2011 00:05:35 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Feb 2011 00:05:35 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO mail-qy0-f169.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Feb 2011 00:05:29 +0000 Received: by qyk7 with SMTP id 7so4914078qyk.14 for <christophersmith@example.com>; Tue, 01 Feb 2011 16:05:06 -0800 (PST) Received: by 83.161.120.155 with SMTP id jz4mr7756721qcb.135.1296605106471; Tue, 01 Feb 2011 16:05:06 -0800 (PST) MIME-Version: 1.0 Received: by 83.161.120.155 with HTTP; Tue, 1 Feb 2011 16:04:45 -0800 (PST) X-Originating-IP: [83.161.120.155] In-Reply-To: <christophersmith@example.com> References: <christophersmith@example.com> From: Aaron Powell <christophersmith@example.com> Date: Wed, 2 Feb 2011 11:04:45 +1100 Message-ID: <christophersmith@example.com> Subject: Re: Question To: christophersmith@example.com Content-Type: multipart/alternative; boundary=001485e9a89ec903cf049b4166d5 X-Virus-Checked: Checked by ClamAV on apache.org --001485e9a89ec903cf049b4166d5 Content-Type: text/plain; charset=ISO-8859-1 You don't actually install Lucene.Net, it's just a library which you reference into your application. Solr is an installable Lucene service, which essentially provides RESTful endpoints to Lucene (java), or so goes my understanding. With regards to what you can search with Lucene, well that really comes down to anything you can push into the index. Keep in mind that Lucene is just a indexer and searcher, it's not a crawler or anything. You have to push the data to the indexer, and you have to write queries to get it back out. I've got some blogs on my site about getting started with Lucene.Net - http://www.aaron-powell.com/lucene-net-overview Aaron Powell Umbraco Core Team Member <http://umbraco.codeplex.com> | FunnelWeb Team Member <http://funnelweblog.com> http://www.aaron-powell.com | http://twitter.com/slace | Skype: aaron.l.powell | MSN: christophersmith@example.com On Wed, Feb 2, 2011 at 10:57 AM, Lucas E Wall <christophersmith@example.com> wrote: > > I am new to Lucene and have the following questions. What is the best way > to understand what is required to install Lucene in a server? Also, can i > make Lucene run searches on links to xml data on the web?Thanks > --001485e9a89ec903cf049b4166d5-- From lucene-net-user-returnchristophersmith@example.com Wed Feb 02 01:03:56 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 41116 invoked from network); 2 Feb 2011 01:03:56 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Feb 2011 01:03:56 -0000 Received: (qmail 38014 invoked by uid 500); 2 Feb 2011 01:03:56 -0000 Delivered-To: christophersmith@example.com Received: (qmail 37950 invoked by uid 500); 2 Feb 2011 01:03:55 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 37937 invoked by uid 99); 2 Feb 2011 01:03:55 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Feb 2011 01:03:55 +0000 X-ASF-Spam-Status: No, hits=4.7 required=5.0 tests=FREEMAIL_FROM,FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,RFC_ABUSE_POST,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of christophersmith@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO bay0-omc4-s20.bay0.hotmail.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Feb 2011 01:03:50 +0000 Received: from BAY138-W11 ([83.161.120.155]) by bay0-omc4-s20.bay0.hotmail.com with Microsoft SMTPSVC231.721.8345); Tue, 1 Feb 2011 17:03:29 -0800 Message-ID: <christophersmith@example.com> Content-Type: multipart/alternative; boundary="_b2710208-230a-43ea-a999-5bd23ad03f31_" X-Originating-IP: [83.161.120.155] From: Lucas E Wall <christophersmith@example.com> To: <christophersmith@example.com> Subject: RE: Question Date: Tue, 1 Feb 2011 20:03:28 -0500 Importance: Normal In-Reply-To: <christophersmith@example.com> References: <christophersmith@example.com>,<christophersmith@example.com> MIME-Version: 1.0 X-OriginalArrivalTime: 02 Feb 2011 01:03:29.0046 (UTC) FILETIME=[00CB0B60:01CBC275] --_b2710208-230a-43ea-a999-5bd23ad03f31_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Thanks=2C Aaron. I went through your blog and it makes a lot of sense. Gi= ven that Lucen is asp friendly=2C can I call the library from mssql? Where= does the indexing gets stored? Do I need to provide a database for do fil= es I need indexed=2C and for the index as well? May be my questions are a = little bit too entry level. > From: christophersmith@example.com > Date: Wed=2C 2 Feb 2011 11:04:45 +1100 > Subject: Re: Question > To: christophersmith@example.com >=20 > You don't actually install Lucene.Net=2C it's just a library which you > reference into your application. Solr is an installable Lucene service=2C > which essentially provides RESTful endpoints to Lucene (java)=2C or so go= es my > understanding. >=20 > With regards to what you can search with Lucene=2C well that really comes= down > to anything you can push into the index. Keep in mind that Lucene is just= a > indexer and searcher=2C it's not a crawler or anything. You have to push = the > data to the indexer=2C and you have to write queries to get it back out. > I've got some blogs on my site about getting started with Lucene.Net - > http://www.aaron-powell.com/lucene-net-overview > Aaron Powell > Umbraco Core Team Member <http://umbraco.codeplex.com> | FunnelWeb Team > Member <http://funnelweblog.com> >=20 > http://www.aaron-powell.com | http://twitter.com/slace | Skype: > aaron.l.powell | MSN: christophersmith@example.com >=20 >=20 > On Wed=2C Feb 2=2C 2011 at 10:57 AM=2C Lucas E Wall <christophersmith@example.com= m> wrote: >=20 > > > > I am new to Lucene and have the following questions. What is the best = way > > to understand what is required to install Lucene in a server? Also=2C = can i > > make Lucene run searches on links to xml data on the web?Thanks > > = --_b2710208-230a-43ea-a999-5bd23ad03f31_-- From lucene-net-user-returnchristophersmith@example.com Wed Feb 02 01:16:16 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 51759 invoked from network); 2 Feb 2011 01:16:16 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Feb 2011 01:16:16 -0000 Received: (qmail 49103 invoked by uid 500); 2 Feb 2011 01:16:16 -0000 Delivered-To: christophersmith@example.com Received: (qmail 49049 invoked by uid 500); 2 Feb 2011 01:16:15 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 49040 invoked by uid 99); 2 Feb 2011 01:16:15 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Feb 2011 01:16:15 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO mail-qy0-f169.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Feb 2011 01:16:09 +0000 Received: by qyk7 with SMTP id 7so4965800qyk.14 for <christophersmith@example.com>; Tue, 01 Feb 2011 17:15:48 -0800 (PST) Received: by 83.161.120.155 with SMTP id hy6mr5846234qcb.287.1296608961767; Tue, 01 Feb 2011 17:09:21 -0800 (PST) MIME-Version: 1.0 Received: by 83.161.120.155 with HTTP; Tue, 1 Feb 2011 17:09:01 -0800 (PST) X-Originating-IP: [83.161.120.155] In-Reply-To: <christophersmith@example.com> References: <christophersmith@example.com> <christophersmith@example.com> <christophersmith@example.com> From: Aaron Powell <christophersmith@example.com> Date: Wed, 2 Feb 2011 12:09:01 +1100 Message-ID: <christophersmith@example.com> Subject: Re: Question To: christophersmith@example.com Content-Type: multipart/alternative; boundary=00163628430a942ada049b424c2b X-Virus-Checked: Checked by ClamAV on apache.org --00163628430a942ada049b424c2b Content-Type: text/plain; charset=ISO-8859-1 Lucene.Net uses the same binary data store that Lucene uses which is stored on the file system (generally, it depends on what Directory instance you provide to the indexer & searcher). Some projects, such as NHibernate.Search and RavenDB use Lucene.Net internally and handle syncronizing the data stores (DB & Lucene). If you're trying to index things such as HTML/ XML/ PDF/ etc you have to write your own way to read the data into Lucene though. Aaron Powell Umbraco Core Team Member <http://umbraco.codeplex.com> | FunnelWeb Team Member <http://funnelweblog.com> http://www.aaron-powell.com | http://twitter.com/slace | Skype: aaron.l.powell | MSN: christophersmith@example.com On Wed, Feb 2, 2011 at 12:03 PM, Lucas E Wall <christophersmith@example.com> wrote: > > Thanks, Aaron. I went through your blog and it makes a lot of sense. > Given that Lucen is asp friendly, can I call the library from mssql? Where > does the indexing gets stored? Do I need to provide a database for do files > I need indexed, and for the index as well? May be my questions are a little > bit too entry level. > > > From: christophersmith@example.com > > Date: Wed, 2 Feb 2011 11:04:45 +1100 > > Subject: Re: Question > > To: christophersmith@example.com > > > > You don't actually install Lucene.Net, it's just a library which you > > reference into your application. Solr is an installable Lucene service, > > which essentially provides RESTful endpoints to Lucene (java), or so goes > my > > understanding. > > > > With regards to what you can search with Lucene, well that really comes > down > > to anything you can push into the index. Keep in mind that Lucene is just > a > > indexer and searcher, it's not a crawler or anything. You have to push > the > > data to the indexer, and you have to write queries to get it back out. > > I've got some blogs on my site about getting started with Lucene.Net - > > http://www.aaron-powell.com/lucene-net-overview > > Aaron Powell > > Umbraco Core Team Member <http://umbraco.codeplex.com> | FunnelWeb Team > > Member <http://funnelweblog.com> > > > > http://www.aaron-powell.com | http://twitter.com/slace | Skype: > > aaron.l.powell | MSN: christophersmith@example.com > > > > > > On Wed, Feb 2, 2011 at 10:57 AM, Lucas E Wall <christophersmith@example.com> > wrote: > > > > > > > > I am new to Lucene and have the following questions. What is the best > way > > > to understand what is required to install Lucene in a server? Also, > can i > > > make Lucene run searches on links to xml data on the web?Thanks > > > > > --00163628430a942ada049b424c2b-- From lucene-net-user-returnchristophersmith@example.com Wed Feb 02 07:11:28 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 12180 invoked from network); 2 Feb 2011 07:11:28 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Feb 2011 07:11:28 -0000 Received: (qmail 99535 invoked by uid 500); 2 Feb 2011 07:11:27 -0000 Delivered-To: christophersmith@example.com Received: (qmail 99290 invoked by uid 500); 2 Feb 2011 07:11:24 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 99282 invoked by uid 99); 2 Feb 2011 07:11:22 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Feb 2011 07:11:22 +0000 X-ASF-Spam-Status: No, hits=4.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,FREEMAIL_REPLY,RCVD_IN_DNSWL_NONE,RFC_ABUSE_POST,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of christophersmith@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO col0-omc4-s12.col0.hotmail.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Feb 2011 07:11:18 +0000 Received: from COL109-W46 ([83.161.120.155]) by col0-omc4-s12.col0.hotmail.com with Microsoft SMTPSVC231.721.8345); Tue, 1 Feb 2011 23:10:57 -0800 Message-ID: <christophersmith@example.com> X-Originating-IP: [83.161.120.155] From: Prescott Nasser <christophersmith@example.com> To: <christophersmith@example.com> Subject: RE: Question Date: Tue, 1 Feb 2011 23:10:57 -0800 Importance: Normal In-Reply-To: <christophersmith@example.com> References: <christophersmith@example.com> <christophersmith@example.com>,<christophersmith@example.com>,<christophersmith@example.com> Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginalArrivalTime: 02 Feb 2011 07:10:57.0177 (UTC) FILETIME=[5680D890:01CBC2A8] Just to add since you're likely on a windows platform=2C check out Ifilters= and how to use them- they are probably the easiest way you have to extract= data from pdf/html/xml. =20 Check out this for getting started with using the Ifilter interface: http:/= /www.codeproject.com/KB/cs/IFilter.aspx?msg=3D2428047 =20 Once you extract the plain text - that is where Lucene comes in to parse th= at plain text and create an index. =20 ~P > From: christophersmith@example.com > Date: Wed=2C 2 Feb 2011 12:09:01 +1100 > Subject: Re: Question > To: christophersmith@example.com >=20 > Lucene.Net uses the same binary data store that Lucene uses which is stor= ed > on the file system (generally=2C it depends on what Directory instance yo= u > provide to the indexer & searcher). >=20 > Some projects=2C such as NHibernate.Search and RavenDB use Lucene.Net > internally and handle syncronizing the data stores (DB & Lucene). > If you're trying to index things such as HTML/ XML/ PDF/ etc you have to > write your own way to read the data into Lucene though. > Aaron Powell > Umbraco Core Team Member <http://umbraco.codeplex.com> | FunnelWeb Team > Member <http://funnelweblog.com> >=20 > http://www.aaron-powell.com | http://twitter.com/slace | Skype: > aaron.l.powell | MSN: christophersmith@example.com >=20 >=20 > On Wed=2C Feb 2=2C 2011 at 12:03 PM=2C Lucas E Wall <christophersmith@example.com= m> wrote: >=20 > > > > Thanks=2C Aaron. I went through your blog and it makes a lot of sense. > > Given that Lucen is asp friendly=2C can I call the library from mssql? = Where > > does the indexing gets stored? Do I need to provide a database for do f= iles > > I need indexed=2C and for the index as well? May be my questions are a = little > > bit too entry level. > > > > > From: christophersmith@example.com > > > Date: Wed=2C 2 Feb 2011 11:04:45 +1100 > > > Subject: Re: Question > > > To: christophersmith@example.com > > > > > > You don't actually install Lucene.Net=2C it's just a library which yo= u > > > reference into your application. Solr is an installable Lucene servic= e=2C > > > which essentially provides RESTful endpoints to Lucene (java)=2C or s= o goes > > my > > > understanding. > > > > > > With regards to what you can search with Lucene=2C well that really c= omes > > down > > > to anything you can push into the index. Keep in mind that Lucene is = just > > a > > > indexer and searcher=2C it's not a crawler or anything. You have to p= ush > > the > > > data to the indexer=2C and you have to write queries to get it back o= ut. > > > I've got some blogs on my site about getting started with Lucene.Net = - > > > http://www.aaron-powell.com/lucene-net-overview > > > Aaron Powell > > > Umbraco Core Team Member <http://umbraco.codeplex.com> | FunnelWeb Te= am > > > Member <http://funnelweblog.com> > > > > > > http://www.aaron-powell.com | http://twitter.com/slace | Skype: > > > aaron.l.powell | MSN: christophersmith@example.com > > > > > > > > > On Wed=2C Feb 2=2C 2011 at 10:57 AM=2C Lucas E Wall <walllucas@hotmai= l.com> > > wrote: > > > > > > > > > > > I am new to Lucene and have the following questions. What is the be= st > > way > > > > to understand what is required to install Lucene in a server? Also= =2C > > can i > > > > make Lucene run searches on links to xml data on the web?Thanks > > > > > > > > = From lucene-net-user-returnchristophersmith@example.com Wed Feb 02 19:23:13 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 22344 invoked from network); 2 Feb 2011 19:23:13 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Feb 2011 19:23:13 -0000 Received: (qmail 96418 invoked by uid 500); 2 Feb 2011 19:23:13 -0000 Delivered-To: christophersmith@example.com Received: (qmail 96270 invoked by uid 500); 2 Feb 2011 19:23:12 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 96262 invoked by uid 99); 2 Feb 2011 19:23:11 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Feb 2011 19:23:11 +0000 X-ASF-Spam-Status: No, hits=4.0 required=5.0 tests=FREEMAIL_FROM,FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of christophersmith@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-bw0-f48.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Feb 2011 19:23:06 +0000 Received: by bwz8 with SMTP id 8so924350bwz.35 for <christophersmith@example.com>; Wed, 02 Feb 2011 11:22:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=+WVKHGfQSFvMXCBpw9/Uufvy5lkCsBGIxU897uvHyHc=; b=gsEAXkXog8ValnzwwVMVxT6xmlx+OxU7Uy0rnzVQU4qVXO9urlBwBtMTf/7sNO4QF0 DoJg5KjbRCqmA4w/NwbTnVQAOVxFdRAvRKYAiuuzutnLLYwu0O7HrmFa474Yv7LBiC7A XL0LJweeLAOD2Bb3AKzKLiaEEMy+3cE33YSeE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=DW4BwTYNjMb+ohf14LB0FfEVTIorkgp7FYn32WXHf8U9TPSg6hjJoWih/05xcSPK+B WvssxxmeuzGxQ0WJ9AeTD5ZaS2fMvGE09eRzNntVOvlARzaQMubXQZ/j4M+B2D9Zo5Xc 2MZgX+JWG3sR57pfrYElxXXCA6OFWUGSKzVLo= Received: by 83.161.120.155 with SMTP id c5mr3824915bko.124.1296674564847; Wed, 02 Feb 2011 11:22:44 -0800 (PST) MIME-Version: 1.0 Received: by 83.161.120.155 with HTTP; Wed, 2 Feb 2011 11:22:23 -0800 (PST) In-Reply-To: <christophersmith@example.com> References: <christophersmith@example.com> <christophersmith@example.com> <christophersmith@example.com> <christophersmith@example.com> <christophersmith@example.com> From: Kevin Miller <christophersmith@example.com> Date: Wed, 2 Feb 2011 13:22:23 -0600 Message-ID: <christophersmith@example.com> Subject: Re: Question To: christophersmith@example.com Content-Type: multipart/alternative; boundary=0016e6d99b7dd3ba00049b519247 X-Virus-Checked: Checked by ClamAV on apache.org --0016e6d99b7dd3ba00049b519247 Content-Type: text/plain; charset=ISO-8859-1 I've also had success using Tika to do text extraction (via IKVM) This GitHub repo has example code and tests for pulling contents out of PDF, word documents etc. https://github.com/KevM/tikaondotnet Works great for me in a product I helped create. Once you have the text of the document you index it as you would normal content. Kevin Miller <https://github.com/KevM/tikaondotnet> On Wed, Feb 2, 2011 at 1:10 AM, Prescott Nasser <christophersmith@example.com>wrote: > > Just to add since you're likely on a windows platform, check out Ifilters > and how to use them- they are probably the easiest way you have to extract > data from pdf/html/xml. > > Check out this for getting started with using the Ifilter interface: > http://www.codeproject.com/KB/cs/IFilter.aspx?msg=2428047 > > Once you extract the plain text - that is where Lucene comes in to parse > that plain text and create an index. > > ~P > > > > > > From: christophersmith@example.com > > Date: Wed, 2 Feb 2011 12:09:01 +1100 > > Subject: Re: Question > > To: christophersmith@example.com > > > > Lucene.Net uses the same binary data store that Lucene uses which is > stored > > on the file system (generally, it depends on what Directory instance you > > provide to the indexer & searcher). > > > > Some projects, such as NHibernate.Search and RavenDB use Lucene.Net > > internally and handle syncronizing the data stores (DB & Lucene). > > If you're trying to index things such as HTML/ XML/ PDF/ etc you have to > > write your own way to read the data into Lucene though. > > Aaron Powell > > Umbraco Core Team Member <http://umbraco.codeplex.com> | FunnelWeb Team > > Member <http://funnelweblog.com> > > > > http://www.aaron-powell.com | http://twitter.com/slace | Skype: > > aaron.l.powell | MSN: christophersmith@example.com > > > > > > On Wed, Feb 2, 2011 at 12:03 PM, Lucas E Wall <christophersmith@example.com> > wrote: > > > > > > > > Thanks, Aaron. I went through your blog and it makes a lot of sense. > > > Given that Lucen is asp friendly, can I call the library from mssql? > Where > > > does the indexing gets stored? Do I need to provide a database for do > files > > > I need indexed, and for the index as well? May be my questions are a > little > > > bit too entry level. > > > > > > > From: christophersmith@example.com > > > > Date: Wed, 2 Feb 2011 11:04:45 +1100 > > > > Subject: Re: Question > > > > To: christophersmith@example.com > > > > > > > > You don't actually install Lucene.Net, it's just a library which you > > > > reference into your application. Solr is an installable Lucene > service, > > > > which essentially provides RESTful endpoints to Lucene (java), or so > goes > > > my > > > > understanding. > > > > > > > > With regards to what you can search with Lucene, well that really > comes > > > down > > > > to anything you can push into the index. Keep in mind that Lucene is > just > > > a > > > > indexer and searcher, it's not a crawler or anything. You have to > push > > > the > > > > data to the indexer, and you have to write queries to get it back > out. > > > > I've got some blogs on my site about getting started with Lucene.Net > - > > > > http://www.aaron-powell.com/lucene-net-overview > > > > Aaron Powell > > > > Umbraco Core Team Member <http://umbraco.codeplex.com> | FunnelWeb > Team > > > > Member <http://funnelweblog.com> > > > > > > > > http://www.aaron-powell.com | http://twitter.com/slace | Skype: > > > > aaron.l.powell | MSN: christophersmith@example.com > > > > > > > > > > > > On Wed, Feb 2, 2011 at 10:57 AM, Lucas E Wall <christophersmith@example.com > > > > > wrote: > > > > > > > > > > > > > > I am new to Lucene and have the following questions. What is the > best > > > way > > > > > to understand what is required to install Lucene in a server? Also, > > > can i > > > > > make Lucene run searches on links to xml data on the web?Thanks > > > > > > > > > > > > --0016e6d99b7dd3ba00049b519247-- From lucene-net-user-returnchristophersmith@example.com Wed Feb 02 22:35:54 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 51702 invoked from network); 2 Feb 2011 22:35:54 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Feb 2011 22:35:54 -0000 Received: (qmail 56232 invoked by uid 500); 2 Feb 2011 22:35:53 -0000 Delivered-To: christophersmith@example.com Received: (qmail 56172 invoked by uid 500); 2 Feb 2011 22:35:53 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 56164 invoked by uid 99); 2 Feb 2011 22:35:53 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Feb 2011 22:35:53 +0000 X-ASF-Spam-Status: No, hits=0.7 required=5.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: 83.161.120.155 is neither permitted nor denied by domain of christophersmith@example.com) Received: from [83.161.120.155] (HELO DC03.howardshome.nl) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Feb 2011 22:35:47 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Subject: RE: Question Date: Wed, 2 Feb 2011 23:35:22 +0100 Message-ID: <christophersmith@example.com> In-Reply-To: <christophersmith@example.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Question Thread-Index: AcvCbPOvBubLkiigQT+X8EwLFPWXeAAuwrPw References: <christophersmith@example.com> <christophersmith@example.com> From: =?iso-8859-1?Q?Ren=E9_de_Vries?= <christophersmith@example.com> To: <christophersmith@example.com> X-Virus-Checked: Checked by ClamAV on apache.org If you're looking for a quick, manageable solution for Full-text search = on SQL Server data, I recommend taking a christophersmith@example.com. Solr is like a = huge management layer around Lucene and hides a lot of the details, even = though you can still get at them through config files. It has a GREAT Data Import Handler which does all the heavy lifting for = you to connect to Sql Server. I was able to set up indexing 15 million = news articles in one afternoon. For indexing PDF's and such, Solr has = dedicated handlers, which I haven't used but seem pretty easy to set up. = Solr is a full java solution, but don't let that scare you as it is easy = to set up and lot of help is available. There is a SolrNet library which = allows easy access to all the solr functions from a .net app. This = tutorial is particular handy: = http://crazorsharp.blogspot.com/2010/01/full-text-search-using-solr-lucen= e-and.html=20 Other than that, I recommend reading the book Lucene in Action, or the = Solr book. There are also LOTS of good tutorials available Ren=E9 -----Original Message----- From: Aaron Powell [mailto:christophersmith@example.com]=20 Sent: woensdag 2 februari 2011 1:05 To: christophersmith@example.com Subject: Re: Question You don't actually install Lucene.Net, it's just a library which you reference into your application. Solr is an installable Lucene service, which essentially provides RESTful endpoints to Lucene (java), or so = goes my understanding. With regards to what you can search with Lucene, well that really comes = down to anything you can push into the index. Keep in mind that Lucene is = just a indexer and searcher, it's not a crawler or anything. You have to push = the data to the indexer, and you have to write queries to get it back out. I've got some blogs on my site about getting started with Lucene.Net - http://www.aaron-powell.com/lucene-net-overview Aaron Powell Umbraco Core Team Member <http://umbraco.codeplex.com> | FunnelWeb Team Member <http://funnelweblog.com> http://www.aaron-powell.com | http://twitter.com/slace | Skype: aaron.l.powell | MSN: christophersmith@example.com On Wed, Feb 2, 2011 at 10:57 AM, Lucas E Wall <christophersmith@example.com> = wrote: > > I am new to Lucene and have the following questions. What is the best = way > to understand what is required to install Lucene in a server? Also, = can i > make Lucene run searches on links to xml data on the web?Thanks > From lucene-net-user-returnchristophersmith@example.com Thu Feb 03 02:50:24 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 73270 invoked from network); 3 Feb 2011 02:50:24 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Feb 2011 02:50:24 -0000 Received: (qmail 56626 invoked by uid 500); 3 Feb 2011 02:50:23 -0000 Delivered-To: christophersmith@example.com Received: (qmail 56390 invoked by uid 500); 3 Feb 2011 02:50:22 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 56379 invoked by uid 99); 3 Feb 2011 02:50:21 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 02:50:21 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,RFC_ABUSE_POST,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of christophersmith@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO blu0-omc4-s13.blu0.hotmail.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 02:50:15 +0000 Received: from BLU158-W2 ([83.161.120.155]) by blu0-omc4-s13.blu0.hotmail.com with Microsoft SMTPSVC231.721.8345); Wed, 2 Feb 2011 18:49:54 -0800 Message-ID: <christophersmith@example.com> Content-Type: multipart/alternative; boundary="_d4175409-3168-400f-b852-98decd66ed89_" X-Originating-IP: [83.161.120.155] From: Igor Chirokov <christophersmith@example.com> To: <christophersmith@example.com> Subject: RE: Question Date: Wed, 2 Feb 2011 21:49:53 -0500 Importance: Normal In-Reply-To: <christophersmith@example.com> References: <christophersmith@example.com> <christophersmith@example.com>,<christophersmith@example.com> MIME-Version: 1.0 X-OriginalArrivalTime: 03 Feb 2011 02:49:54.0021 (UTC) FILETIME=[08F2A950:01CBC34D] X-Virus-Checked: Checked by ClamAV on apache.org --_d4175409-3168-400f-b852-98decd66ed89_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable You can llok: http://www.walnutilsoft.com/ =20 In this solution you have integrated Lucine solutiion for SQL Server and Or= acle databases as well. Here is my product=2C it's free. Also you can see simple code for .net and java Lucine. =20 Thanks=2C Igor =20 > Subject: RE: Question > Date: Wed=2C 2 Feb 2011 23:35:22 +0100 > From: christophersmith@example.com > To: christophersmith@example.com >=20 > If you're looking for a quick=2C manageable solution for Full-text search= on SQL Server data=2C I recommend taking a christophersmith@example.com. Solr is like a hu= ge management layer around Lucene and hides a lot of the details=2C even th= ough you can still get at them through config files. >=20 > It has a GREAT Data Import Handler which does all the heavy lifting for y= ou to connect to Sql Server. I was able to set up indexing 15 million news = articles in one afternoon. For indexing PDF's and such=2C Solr has dedicate= d handlers=2C which I haven't used but seem pretty easy to set up.=20 >=20 > Solr is a full java solution=2C but don't let that scare you as it is eas= y to set up and lot of help is available. There is a SolrNet library which = allows easy access to all the solr functions from a .net app. This tutorial= is particular handy: http://crazorsharp.blogspot.com/2010/01/full-text-sea= rch-using-solr-lucene-and.html=20 >=20 > Other than that=2C I recommend reading the book Lucene in Action=2C or th= e Solr book. There are also LOTS of good tutorials available >=20 > Ren=E9 >=20 > -----Original Message----- > From: Aaron Powell [mailto:christophersmith@example.com]=20 > Sent: woensdag 2 februari 2011 1:05 > To: christophersmith@example.com > Subject: Re: Question >=20 > You don't actually install Lucene.Net=2C it's just a library which you > reference into your application. Solr is an installable Lucene service=2C > which essentially provides RESTful endpoints to Lucene (java)=2C or so go= es my > understanding. >=20 > With regards to what you can search with Lucene=2C well that really comes= down > to anything you can push into the index. Keep in mind that Lucene is just= a > indexer and searcher=2C it's not a crawler or anything. You have to push = the > data to the indexer=2C and you have to write queries to get it back out. > I've got some blogs on my site about getting started with Lucene.Net - > http://www.aaron-powell.com/lucene-net-overview > Aaron Powell > Umbraco Core Team Member <http://umbraco.codeplex.com> | FunnelWeb Team > Member <http://funnelweblog.com> >=20 > http://www.aaron-powell.com | http://twitter.com/slace | Skype: > aaron.l.powell | MSN: christophersmith@example.com >=20 >=20 > On Wed=2C Feb 2=2C 2011 at 10:57 AM=2C Lucas E Wall <christophersmith@example.com= m> wrote: >=20 > > > > I am new to Lucene and have the following questions. What is the best w= ay > > to understand what is required to install Lucene in a server? Also=2C c= an i > > make Lucene run searches on links to xml data on the web?Thanks > > = --_d4175409-3168-400f-b852-98decd66ed89_-- From lucene-net-user-returnchristophersmith@example.com Thu Feb 03 11:37:20 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 98175 invoked from network); 3 Feb 2011 11:37:20 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Feb 2011 11:37:20 -0000 Received: (qmail 81255 invoked by uid 500); 3 Feb 2011 11:37:20 -0000 Delivered-To: christophersmith@example.com Received: (qmail 81013 invoked by uid 500); 3 Feb 2011 11:37:18 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 81004 invoked by uid 99); 3 Feb 2011 11:37:17 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 11:37:17 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of christophersmith@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-yx0-f176.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 11:37:11 +0000 Received: by yxm8 with SMTP id 8so458791yxm.35 for <christophersmith@example.com>; Thu, 03 Feb 2011 03:36:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:date:x-google-sender-auth :message-id:subject:from:to:content-type; bh=Ck1seW436TLMjpqBKzQKiIWJceiSoso2Eqom+U4+Pl8=; b=q6knIsl8M+OjYeFugmw7V5CS+6esCJUtsguyMDBRVuVOz1asGai2Jme9ULoRY9rXZG Bui02jKAPDcol2yK0ZfnsdfVb+QEJeoUWAFgkFMooI1bzBdfn61rxOEjKNOYxjzTKhUu Q324xBTN0R2oPt2AWghgJen3XLd2VRwFxegR4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=l6CcCrtq1CjMF6wcAb8tT6CwCmwzAAPGO0hGGY6kyHo9EbLzqniX7cX/hGB4KSiEIf 62A+D5MHwsrk/nUs/tKUF3vhMpqoj5+GA/GDbnQCmylDnJPyGT38o7bFyUkBNeLGdpCM zbbfokdzFJFIpBMLwgcs6TFjUz9QUjUk/ddWA= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id q27mr12368413yba.420.1296733010503; Thu, 03 Feb 2011 03:36:50 -0800 (PST) Sender: christophersmith@example.com Received: by 83.161.120.155 with HTTP; Thu, 3 Feb 2011 03:36:50 -0800 (PST) Date: Thu, 3 Feb 2011 12:36:50 +0100 X-Google-Sender-Auth: _ASqD24GRLqBh4nbI60criN7F_g Message-ID: <christophersmith@example.com> Subject: How to use a profile based search in Lucene From: Francois Blochaou <christophersmith@example.com> To: christophersmith@example.com Content-Type: multipart/alternative; boundary=000e0cd593e875aa8d049b5f2eef --000e0cd593e875aa8d049b5f2eef Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi I use lucene in my web application. I would like to add some feature as a profile based search. How can i do that in Lucene or any other open source library. Thanks --=20 BLOCHAOU Fran=E7ois Software Developer BEF Technology SARL Web : http://www.bef-technology.com Mail: christophersmith@example.com Tel: 0033381401363 Mobil : 0033628424213 / 0041797945635 --000e0cd593e875aa8d049b5f2eef-- From lucene-net-user-returnchristophersmith@example.com Thu Feb 03 12:16:31 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 13993 invoked from network); 3 Feb 2011 12:16:31 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Feb 2011 12:16:31 -0000 Received: (qmail 11374 invoked by uid 500); 3 Feb 2011 12:16:30 -0000 Delivered-To: christophersmith@example.com Received: (qmail 11165 invoked by uid 500); 3 Feb 2011 12:16:28 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 11156 invoked by uid 99); 3 Feb 2011 12:16:27 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 12:16:27 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of christophersmith@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-iy0-f176.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 12:16:20 +0000 Received: by iyb26 with SMTP id 26so1076160iyb.35 for <christophersmith@example.com>; Thu, 03 Feb 2011 04:15:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to :content-type; bh=dieNJ1MvKodRtnKBedWDa7NtCybC0vELICkzLvSglE8=; b=jaesoDg6EMUN+OhplTaN4+ucyhvJyg++KNq2RcaTxK3IDRzPkI8zuWvt40BHekkd0n vu9Xsr3NT47Lc6KOlBuitS1/TC9p/XpQvXnwdLoSP3rka6uB2Hebw3pNbYJaalvzO4+j Q/vr1mHjuWHTPIqEsCDDvX/aYf4+yfUsc3egw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=Z76TU1TcIBe12xtGXlPySg0wkhyQPd7OsiCYGvrEAMxaOAMawP7QfVI6w+CbnkReIg nqedFwCeQ36dZd+su8dKrNOydu9J8u0m+zxCzjZoXVXXdNmH6EdnAZT8653vADsLLDX/ TUbKETeWyL04aBIkLuZMpagxIGaROGfD0uCAo= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id j9mr11349383ibd.137.1296735358998; Thu, 03 Feb 2011 04:15:58 -0800 (PST) Received: by 83.161.120.155 with HTTP; Thu, 3 Feb 2011 04:15:58 -0800 (PST) Date: Thu, 3 Feb 2011 17:45:58 +0530 Message-ID: <christophersmith@example.com> Subject: partial update over lucene index From: K a r n a v <christophersmith@example.com> To: christophersmith@example.com Content-Type: multipart/alternative; boundary=00221534c92770da49049b5fbaec X-Virus-Checked: Checked by ClamAV on apache.org --00221534c92770da49049b5fbaec Content-Type: text/plain; charset=ISO-8859-1 I want to do partial update over lucene index (rather than doing complete indexing everytime i want to do only partial update) how can i do that... please give me idea or atleast give me the supporting links to guide me. -- *Thanks & Regards*, *Karunaker Reddy V * --00221534c92770da49049b5fbaec-- From lucene-net-user-returnchristophersmith@example.com Thu Feb 03 13:01:40 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 32305 invoked from network); 3 Feb 2011 13:01:40 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Feb 2011 13:01:40 -0000 Received: (qmail 68114 invoked by uid 500); 3 Feb 2011 13:01:40 -0000 Delivered-To: christophersmith@example.com Received: (qmail 67800 invoked by uid 500); 3 Feb 2011 13:01:37 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 67792 invoked by uid 99); 3 Feb 2011 13:01:37 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 13:01:37 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of christophersmith@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-iw0-f176.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 13:01:30 +0000 Received: by iwn2 with SMTP id 2so1118459iwn.35 for <christophersmith@example.com>; Thu, 03 Feb 2011 05:01:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=J3m8lSZl74KlU7Et1avaAfjuwk/4c1cy58eveoJs3UA=; b=o7s7JgNYc79s6O9b6wyZzextXrZ4PWLdNQbQLSCpW5Ei/vYVfM5cEPG1NweMPEeCNy zyM5GkXOHWf7OI+6VLH7JtOVj7v4ZIiMlxHsVTAbrEvYWwSeOvyYMiyajtUVxgdlnwUU LkIpF6B6uOnorPQlB20RWoFiJhDE5W6fvwyxo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=RP6T4EvSmCfc/ju+4EZ/80caCFwnzOvMl4AphQgvH2FMpvN1egf5cOdeCa57jN1phg zvXe+3p9RLxzRNg77sx10mwLcwvAujXhwBqaPwyi87iC4Q1bTvJyW+yNlLOkSn/W+Lzc 22Vhul0Nj3n9t5GrNcdW7eoou7/Zb63zXRRI8= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id gw8mr12690066icb.359.1296738068158; Thu, 03 Feb 2011 05:01:08 -0800 (PST) Received: by 83.161.120.155 with HTTP; Thu, 3 Feb 2011 05:01:08 -0800 (PST) In-Reply-To: <christophersmith@example.com> References: <christophersmith@example.com> Date: Thu, 3 Feb 2011 15:01:08 +0200 Message-ID: <christophersmith@example.com> Subject: Re: partial update over lucene index From: digy digy <christophersmith@example.com> To: christophersmith@example.com Content-Type: multipart/alternative; boundary=20cf30549e8feb5a13049b605b12 X-Virus-Checked: Checked by ClamAV on apache.org --20cf30549e8feb5a13049b605b12 Content-Type: text/plain; charset=ISO-8859-1 What do you mean by "partially updating" the index. This seems to be a XY-problem (http://www.perlmonks.org/index.pl?node_id=542341) DIGY On Thu, Feb 3, 2011 at 2:15 PM, K a r n a v <christophersmith@example.com>wrote: > I want to do partial update over lucene index (rather than doing complete > indexing everytime i want to do only partial update) > how can i do that... > please give me idea or atleast give me the supporting links to guide me. > > -- > *Thanks & Regards*, > *Karunaker Reddy V > * > --20cf30549e8feb5a13049b605b12-- From lucene-net-user-returnchristophersmith@example.com Thu Feb 03 14:17:45 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 73411 invoked from network); 3 Feb 2011 14:17:45 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Feb 2011 14:17:45 -0000 Received: (qmail 78384 invoked by uid 500); 3 Feb 2011 14:17:45 -0000 Delivered-To: christophersmith@example.com Received: (qmail 78019 invoked by uid 500); 3 Feb 2011 14:17:41 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 78005 invoked by uid 99); 3 Feb 2011 14:17:40 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 14:17:40 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of christophersmith@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail.catapultsystems.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 14:17:31 +0000 Received: from CATINEXC04.catapultsystems.com ([83.161.120.155]) by CATINEXC04.catapultsystems.com ([83.161.120.155]) with mapi id 14.01.0270.001; Thu, 3 Feb 2011 08:17:08 -0600 From: Josh Handel <christophersmith@example.com> To: "christophersmith@example.com" <christophersmith@example.com> Subject: RE: partial update over lucene index Thread-Topic: partial update over lucene index Thread-Index: AQHLw5wzYSXu7Ra2dEmcJZt1/gB8jJPv0UFQ Date: Thu, 3 Feb 2011 14:17:08 +0000 Message-ID: <christophersmith@example.com> References: <christophersmith@example.com> In-Reply-To: <christophersmith@example.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [83.161.120.155] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org How I handle this is that I look up a document that I want to update (ya th= at means bloating my index with some kind of PK that is meaningful to me) t= hen I delete that document, then add the new one.. Its not a super fast app= roach.. But my index is only feed new and updated articles on a schedule.. = And if the document is new I just add it. And to keep my readers in step, = I actually wrap my lucene indexes in a WCF service I lovingly call Rikene s= o my readers know when to refresh based on index writes. (my WCF service = also does nightly merges as so the deleted data does get pruned out rather = than bloating the index files un-necessarily) I know that's pretty high level, but I hope that gets you pointed in the ri= ght direction. Josh -----Original Message----- From: K a r n a v [mailto:christophersmith@example.com]=20 Sent: Thursday, February 03, 2011 6:16 AM To: christophersmith@example.com Subject: partial update over lucene index I want to do partial update over lucene index (rather than doing complete i= ndexing everytime i want to do only partial update) how can i do that... please give me idea or atleast give me the supporting links to guide me. -- *Thanks & Regards*, *Karunaker Reddy V * From lucene-net-user-returnchristophersmith@example.com Thu Feb 03 14:34:32 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 77763 invoked from network); 3 Feb 2011 14:34:32 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Feb 2011 14:34:32 -0000 Received: (qmail 96628 invoked by uid 500); 3 Feb 2011 14:34:32 -0000 Delivered-To: christophersmith@example.com Received: (qmail 96295 invoked by uid 500); 3 Feb 2011 14:34:30 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 96287 invoked by uid 99); 3 Feb 2011 14:34:29 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 14:34:29 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of christophersmith@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO service20.mimecast.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with SMTP; Thu, 03 Feb 2011 14:34:23 +0000 Received: from MAILER2.alfredst.oxford-analytica.local (116-142-12-86.static.virginmedia.com [83.161.120.155]) by service20.mimecast.com; Thu, 03 Feb 2011 14:34:00 +0000 Received: from BEDIVERE.alfredst.oxford-analytica.local ([83.161.120.155]) by MAILER2.alfredst.oxford-analytica.local with Microsoft SMTPSVC231.721.8345); Thu, 3 Feb 2011 14:33:58 +0000 Content-class: urn:content-classes:message MIME-Version: 1.0 X-MimeOLE: Produced By Microsoft Exchange V6.5 Subject: RE: How to use a profile based search in Lucene Date: Thu, 3 Feb 2011 14:33:57 -0000 Message-ID: <christophersmith@example.com> In-Reply-To: <christophersmith@example.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: How to use a profile based search in Lucene Thread-Index: AcvDl7JmljmhDE44QOitRuO6NfAZBQAFZ/7A References: <christophersmith@example.com> From: "Moray McConnachie" <christophersmith@example.com> To: <christophersmith@example.com> X-OriginalArrivalTime: 03 Feb 2011 14:33:58.0151 (UTC) FILETIME=[64695970:01CBC3AF] X-MC-Unique: 111020314340001602 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Francois, I'm not sure I understand your query?=20 Do you mean how do you personalise searches to individual users? If so, I think it is the job of your web app, not the Lucene engine, to do = this. We do this by writing a wrapper to the Lucene search as a cut-down class wi= th the query fields we think are of interest to the user, based on their ow= n use of the search and their behaviour.=20 We store instances of the wrapper object in a serialized form linked to the= user Ids of the user. On user login we instantiate our wrapper object and call its method Generat= eLuceneQuery(), which provides us with a Lucene Query object we can then ru= n on the search data and present a profile-based results from search. Is that the kind of thing you mean? Moray -------------------------------------=20 Moray McConnachie Director of IT 231.721.8345 Oxford Analytica http://www.oxan.com -----Original Message----- From: christophersmith@example.com [mailto:christophersmith@example.com] On B= ehalf Of Francois Blochaou Sent: 03 February 2011 11:37 To: christophersmith@example.com Subject: How to use a profile based search in Lucene Hi I use lucene in my web application. I would like to add some feature as a p= rofile based search. How can i do that in Lucene or any other open source library. Thanks -- BLOCHAOU Fran=E7ois Software Developer BEF Technology SARL Web : http://www.bef-technology.com Mail: christophersmith@example.com Tel: 0033381401363 Mobil : 0033628424213 / 0041797945635 --------------------------------------------------------- Disclaimer=20 This message and any attachments are confidential and/or privileged. If thi= s has been sent to you in error, please do not use, retain or disclose them= , and contact the sender as soon as possible. Oxford Analytica Ltd Registered in England: No. 1196703 5 Alfred Street, Oxford United Kingdom, OX1 4EH --------------------------------------------------------- From lucene-net-user-returnchristophersmith@example.com Thu Feb 03 15:27:30 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 3439 invoked from network); 3 Feb 2011 15:27:29 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Feb 2011 15:27:29 -0000 Received: (qmail 92764 invoked by uid 500); 3 Feb 2011 15:27:29 -0000 Delivered-To: christophersmith@example.com Received: (qmail 92453 invoked by uid 500); 3 Feb 2011 15:27:26 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 92444 invoked by uid 99); 3 Feb 2011 15:27:25 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 15:27:25 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of christophersmith@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-iy0-f176.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 15:27:21 +0000 Received: by iyb26 with SMTP id 26so1264352iyb.35 for <christophersmith@example.com>; Thu, 03 Feb 2011 07:27:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=Q7VxOFymHzw6uW8ZqwtVOBxtVlSU1r8EUxA1PmUQuHc=; b=VmRiY8w0qIaWjvDukxgzrOcsrgsJV+H8ftj4HZzp24idHi2EloQNHU1RN7XbifppLw 1wYU+YehAXNVzSXdA9fTkcPoEm/UmMJlwhErATs0+RNRYDg7gRm7mIEdJGtiO6FpbwYy huj+YQi8GVBWpD/o9fQ0Ysmgkcd03XctAQAbw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=Qidk70oxlGteoIPci8o0Yl58YrQwUnCKbLyM14ZvFYuI3WZpaU+h1yY7QzBRybZeJi +kIuN+j/aGeNED/mq3t9ujx6QTChumnpDhrxFj3grJzVHeVCdY440sxm5tIrE/NC/jO4 BgVSnjzgCdJBqJR/jaGAFV4/5fQ122q9+4MHI= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id j9mr11580932ibd.137.1296746820693; Thu, 03 Feb 2011 07:27:00 -0800 (PST) Received: by 83.161.120.155 with HTTP; Thu, 3 Feb 2011 07:27:00 -0800 (PST) In-Reply-To: <christophersmith@example.com> References: <christophersmith@example.com> <christophersmith@example.com> Date: Thu, 3 Feb 2011 20:57:00 +0530 Message-ID: <christophersmith@example.com> Subject: Re: partial update over lucene index From: K a r n a v <christophersmith@example.com> To: christophersmith@example.com Content-Type: multipart/alternative; boundary=00221534c9279c7381049b626536 --00221534c9279c7381049b626536 Content-Type: text/plain; charset=ISO-8859-1 ok let me given an example for partial updating.. lets say i have 1 lakh records in my database table T1..and i'm trying to index these records using lucene.... the query which retirives 1lakh records itself takes more than 10mins and once i retrieve these records ..the lucene logic will do the indexing part ..assume it may take 5 mins...so total 15 mins... ..if the T1 records are growing very fast like...50k records will be added daily...and database query wil take much time day by day....in the mean while the first 1lak records will be updated i need to fetch all the creords( the preior 1lkh records + new records) and need to index.... repeatedly i need to do this. the whole process is taking more than 4o mins...and the newly added records are not listing in site becoz the index is updating after every 40mins are 1hr. so this is not good... So what i want is....when a record is added/updated....at that time only i want to update the index.......so my querying time is reduced and indexing time also..... for this one....im asking how can i update/add new/delete the existing docs ... for the existing index...how can i add new records... give me an example for this....i mean example code..or idea...or atleast websites links to guide me.... thx in advance... On Thu, Feb 3, 2011 at 6:31 PM, digy digy <christophersmith@example.com> wrote: > What do you mean by "partially updating" the index. This seems to be a > XY-problem (http://www.perlmonks.org/index.pl?node_id=542341) > > DIGY > > On Thu, Feb 3, 2011 at 2:15 PM, K a r n a v <christophersmith@example.com > >wrote: > > > I want to do partial update over lucene index (rather than doing complete > > indexing everytime i want to do only partial update) > > how can i do that... > > please give me idea or atleast give me the supporting links to guide me. > > > > -- > > *Thanks & Regards*, > > *Karunaker Reddy V > > * > > > -- *Thanks & Regards*, *Karunaker Reddy V *http://www.flickr.com/photos/karnav/ *Ooh!!*, and one more thing: *no matter who you are, you were built to be brilliant and designed to make a difference in this world*.* PLEASE DOT IT*! --00221534c9279c7381049b626536-- From lucene-net-user-returnchristophersmith@example.com Thu Feb 03 15:31:09 2011 Return-Path: <lucene-net-user-returnchristophersmith@example.com> Delivered-To: christophersmith@example.com Received: (qmail 4653 invoked from network); 3 Feb 2011 15:31:08 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Feb 2011 15:31:08 -0000 Received: (qmail 2963 invoked by uid 500); 3 Feb 2011 15:31:08 -0000 Delivered-To: christophersmith@example.com Received: (qmail 2939 invoked by uid 500); 3 Feb 2011 15:31:06 -0000 Mailing-List: contact christophersmith@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:christophersmith@example.com> List-Unsubscribe: <mailto:christophersmith@example.com> List-Post: <mailto:christophersmith@example.com> List-Id: <lucene-net-user.lucene.apache.org> Reply-To: christophersmith@example.com Delivered-To: mailing list christophersmith@example.com Received: (qmail 2931 invoked by uid 99); 3 Feb 2011 15:31:05 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 15:31:05 +0000 X-ASF-Spam-Status: No, hits=1.8 required=5.0 tests=FREEMAIL_FROM,FREEMAIL_REPLY,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of christophersmith@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-ey0-f176.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 15:31:01 +0000 Received: by eyz10 with SMTP id 10so661122eyz.35 for <christophersmith@example.com>; Thu, 03 Feb 2011 07:30:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=S/xlcWL7H0+eWWLqe6GqR/0B0OhSTOR6DYaWYOHt4xs=; b=PR71+eiPi9fcloIrBIHjgUuoahdL979vx3wPsEzsPlesv80mTLPZ3WiUbnshWQ7dY5 57rZ/FSp0rZo+gMQb8Cus2FAVERuSObqmJVTktwB7NYyDeoPofku/XHqgJAI/21iPOHJ bp1x0uaLk+XGQG9OVyrGuFhGIbuvkjbxAWsuo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=TVKscvcKFYLJ38DAA954+L0B2r5hSCzccWq3MYuz+L6YICXWjr8c5RJsBuWkV1Jtca PTINpXVAuSFSaMGCwlkvklxgNxTpY6FOKgtDalxLKzCoWWUVijb2xXXIijM4Jdqi9C/6 TZfk7Y6mmmrozCNwfC51U/LE8Gae4hdgzzjYo= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id h12mr13554446ebh.21.1296747039724; Thu, 03 Feb 2011 07:30:39 -0800 (PST) Received: by 83.161.120.155 with HTTP; Thu, 3 Feb 2011 07:30:39 -0800 (PST) In-Reply-To: <christophersmith@example.com> References: <christophersmith@example.com> <christophersmith@example.com> <christophersmith@example.com> Date: Thu, 3 Feb 2011 10:30:39 -0500 Message-ID: <christophersmith@example.com> Subject: Re: partial update over lucene index From: Wyatt Barnett <christophersmith@example.com> To: christophersmith@example.com Content-Type: text/plain; charset=ISO-8859-1 I have no idea what 1lkh is, but let me take a stab at answering your question: the Trick here is to make sure you have some key in the record then, when the update happens, you: 1) lookup the record in the lucene index by key 2) remove the old record 3) add the new record Index should take it into account pretty much immediately and all is right with the world. On Thu, Feb 3, 2011 at 10:27 AM, K a r n a v <christophersmith@example.com> wrote: > ok let me given an example for partial updating.. > > lets say i have 1 lakh records in my database table T1..and i'm trying to > index these records using lucene.... > the query which retirives 1lakh records itself takes more than 10mins and > once i retrieve these records ..the lucene logic will do the indexing part > ..assume it may take 5 mins...so total 15 mins... ..if the T1 records are > growing very fast like...50k records will be added daily...and database > query wil take much time day by day....in the mean while the first 1lak > records will be updated i need to fetch all the creords( the preior 1lkh > records + new records) and need to index.... repeatedly i need to do this. > > the whole process is taking more than 4o mins...and the newly added records > are not listing in site becoz the index is updating after every 40mins are > 1hr. so this is not good... > > So what i want is....when a record is added/updated....at that time only i > want to update the index.......so my querying time is reduced and indexing > time also..... for this one....im asking how can i update/add new/delete the > existing docs ... > for the existing index...how can i add new records... > > give me an example for this....i mean example code..or idea...or atleast > websites links
{'source': 'Apache Software Foundation Public Mail Archives'}