From nutch-devdfranklin@example.org Fri Jul 01 09:00:22 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 68956 invoked from network); 1 Jul 2005 09:00:22 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jul 2005 09:00:22 -0000 Received: (qmail 13277 invoked by uid 500); 1 Jul 2005 09:00:21 -0000 Delivered-To: dfranklin@example.org Received: (qmail 13218 invoked by uid 500); 1 Jul 2005 09:00:20 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 13203 invoked by uid 99); 1 Jul 2005 09:00:19 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jul 2005 02:00:19 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: neutral (asf.osuosl.org: local policy) Received: from [83.161.120.155] (HELO mx3.telekom.hu) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jul 2005 02:00:22 -0700 Message-ID: <dfranklin@example.org> Date: Fri, 01 Jul 2005 10:59:05 +0200 From: "dfranklin@example.org" <dfranklin@example.org> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; hu-HU; rv:1.7.8) Gecko/20050511 X-Accept-Language: hu, en MIME-Version: 1.0 To: dfranklin@example.org Subject: Bug in index-more plugin? References: <dfranklin@example.org> <dfranklin@example.org> In-Reply-To: <dfranklin@example.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Antivirus: avast! (VPS 0526-3, 2005.06.30), Outbound message X-Antivirus-Status: Clean X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Dear List, I found a problem in MoreIndexingFilter.java. When I indexing segments, I get large list of error messages: can't parse errorenous date: Wed, 10 Sep 2003 11:59:14 or can't parse errorenous date: Wed, 10 Sep 2003 11:59:14GMT I modifiing source code (I don't make 'patch'): Original (lines 137-138): DateFormat df = new SimpleDateFormat("EEE MMM dd HH:mm:ss yyyy zzz"); Date d = df.parse(date); New: DateFormat df = new SimpleDateFormat("EEE, MMM dd HH:mm:ss yyyy", Locale.US); Date d = df.parse(date.substring(0,25)); The modified code works fine. Best Regards, Ferenc From nutch-devdfranklin@example.org Fri Jul 01 09:06:38 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 71462 invoked from network); 1 Jul 2005 09:06:38 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jul 2005 09:06:38 -0000 Received: (qmail 24204 invoked by uid 500); 1 Jul 2005 09:06:37 -0000 Delivered-To: dfranklin@example.org Received: (qmail 24081 invoked by uid 500); 1 Jul 2005 09:06:36 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 24068 invoked by uid 99); 1 Jul 2005 09:06:36 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jul 2005 02:06:36 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: local policy) Received: from [83.161.120.155] (HELO mslinux.media-style.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jul 2005 02:06:39 -0700 Received: from localhost (localhost [83.161.120.155]) by mslinux.media-style.com (Postfix) with ESMTP id A8DEE1440A2 for <dfranklin@example.org>; Fri, 1 Jul 2005 11:02:44 +0200 (CEST) Received: from mslinux.media-style.com ([83.161.120.155]) by localhost (mslinux.media-style.com [83.161.120.155]) (amavisd-new, port 10024) with ESMTP id 02965-05 for <dfranklin@example.org>; Fri, 1 Jul 2005 11:02:44 +0200 (CEST) Received: from [83.161.120.155] (unknown [83.161.120.155]) by mslinux.media-style.com (Postfix) with ESMTP id 4DE3D14409E for <dfranklin@example.org>; Fri, 1 Jul 2005 11:02:44 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v730) In-Reply-To: <dfranklin@example.org> References: <dfranklin@example.org> <dfranklin@example.org> <dfranklin@example.org> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <dfranklin@example.org> Content-Transfer-Encoding: 7bit From: Stefan Groschupf <dfranklin@example.org> Subject: Re: Bug in index-more plugin? Date: Fri, 1 Jul 2005 11:07:08 +0200 To: dfranklin@example.org X-Mailer: Apple Mail (2.730) X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at media-style.com X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Hi, this was already mentioned under the topic 'index-more: can't parse erroneous date'. I suggest we open a bug and vote for this bug. Can you open this bug? I will vote for it. :-) Stefan Am 01.07.2005 um 10:59 schrieb dfranklin@example.org: > Dear List, > > I found a problem in MoreIndexingFilter.java. > When I indexing segments, I get large list of error messages: > can't parse errorenous date: Wed, 10 Sep 2003 11:59:14 or > can't parse errorenous date: Wed, 10 Sep 2003 11:59:14GMT > > I modifiing source code (I don't make 'patch'): > Original (lines 137-138): > DateFormat df = new SimpleDateFormat("EEE MMM dd HH:mm:ss yyyy zzz"); > Date d = df.parse(date); > New: > DateFormat df = new SimpleDateFormat("EEE, MMM dd HH:mm:ss yyyy", > Locale.US); > Date d = df.parse(date.substring(0,25)); > > The modified code works fine. > > Best Regards, > Ferenc > > From nutch-devdfranklin@example.org Fri Jul 01 09:43:33 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 89483 invoked from network); 1 Jul 2005 09:43:32 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jul 2005 09:43:32 -0000 Received: (qmail 92709 invoked by uid 500); 1 Jul 2005 09:43:29 -0000 Delivered-To: dfranklin@example.org Received: (qmail 92559 invoked by uid 500); 1 Jul 2005 09:43:27 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 92519 invoked by uid 99); 1 Jul 2005 09:43:26 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jul 2005 02:43:26 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: neutral (asf.osuosl.org: local policy) Received: from [83.161.120.155] (HELO mx3.telekom.hu) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jul 2005 02:43:28 -0700 Message-ID: <dfranklin@example.org> Date: Fri, 01 Jul 2005 11:42:19 +0200 From: "dfranklin@example.org" <dfranklin@example.org> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; hu-HU; rv:1.7.8) Gecko/20050511 X-Accept-Language: hu, en MIME-Version: 1.0 To: dfranklin@example.org Subject: Re: Bug in index-more plugin? References: <dfranklin@example.org> <dfranklin@example.org> <dfranklin@example.org> <dfranklin@example.org> In-Reply-To: <dfranklin@example.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Antivirus: avast! (VPS 0526-3, 2005.06.30), Outbound message X-Antivirus-Status: Clean X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Dear Stephan, I think this is a bug. Where I can open a bug? In wiki? Regards, Ferenc Stefan Groschupf wrotte: > Hi, > this was already mentioned under the topic 'index-more: can't parse > erroneous date'. > I suggest we open a bug and vote for this bug. > Can you open this bug? I will vote for it. :-) > Stefan > > Am 01.07.2005 um 10:59 schrieb dfranklin@example.org: > >> Dear List, >> >> I found a problem in MoreIndexingFilter.java. >> When I indexing segments, I get large list of error messages: >> can't parse errorenous date: Wed, 10 Sep 2003 11:59:14 or >> can't parse errorenous date: Wed, 10 Sep 2003 11:59:14GMT >> >> I modifiing source code (I don't make 'patch'): >> Original (lines 137-138): >> DateFormat df = new SimpleDateFormat("EEE MMM dd HH:mm:ss yyyy zzz"); >> Date d = df.parse(date); >> New: >> DateFormat df = new SimpleDateFormat("EEE, MMM dd HH:mm:ss yyyy", >> Locale.US); >> Date d = df.parse(date.substring(0,25)); >> >> The modified code works fine. >> >> Best Regards, >> Ferenc >> >> > > > From nutch-devdfranklin@example.org Fri Jul 01 09:47:39 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 90967 invoked from network); 1 Jul 2005 09:47:39 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jul 2005 09:47:39 -0000 Received: (qmail 2782 invoked by uid 500); 1 Jul 2005 09:47:31 -0000 Delivered-To: dfranklin@example.org Received: (qmail 2737 invoked by uid 500); 1 Jul 2005 09:47:30 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 2563 invoked by uid 99); 1 Jul 2005 09:47:29 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jul 2005 02:47:29 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: local policy) Received: from [83.161.120.155] (HELO mslinux.media-style.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jul 2005 02:47:30 -0700 Received: from localhost (localhost [83.161.120.155]) by mslinux.media-style.com (Postfix) with ESMTP id F0B251440A2 for <dfranklin@example.org>; Fri, 1 Jul 2005 11:43:37 +0200 (CEST) Received: from mslinux.media-style.com ([83.161.120.155]) by localhost (mslinux.media-style.com [83.161.120.155]) (amavisd-new, port 10024) with ESMTP id 03197-02 for <dfranklin@example.org>; Fri, 1 Jul 2005 11:43:37 +0200 (CEST) Received: from [83.161.120.155] (unknown [83.161.120.155]) by mslinux.media-style.com (Postfix) with ESMTP id 8E16514409E for <dfranklin@example.org>; Fri, 1 Jul 2005 11:43:37 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v730) In-Reply-To: <dfranklin@example.org> References: <dfranklin@example.org> <dfranklin@example.org> <dfranklin@example.org> <dfranklin@example.org> <dfranklin@example.org> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <dfranklin@example.org> Content-Transfer-Encoding: 7bit From: Stefan Groschupf <dfranklin@example.org> Subject: Re: Bug in index-more plugin? Date: Fri, 1 Jul 2005 11:48:01 +0200 To: dfranklin@example.org X-Mailer: Apple Mail (2.730) X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at media-style.com X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N You can find the nutch bug tracking here: http://issues.apache.org/jira/secure/BrowseProject.jspa?id=10680 Cheers, Stefan Am 01.07.2005 um 11:42 schrieb dfranklin@example.org: > Dear Stephan, > > I think this is a bug. Where I can open a bug? In wiki? > > Regards, > Ferenc > > Stefan Groschupf wrotte: > > >> Hi, >> this was already mentioned under the topic 'index-more: can't >> parse erroneous date'. >> I suggest we open a bug and vote for this bug. >> Can you open this bug? I will vote for it. :-) >> Stefan >> >> Am 01.07.2005 um 10:59 schrieb dfranklin@example.org: >> >> >>> Dear List, >>> >>> I found a problem in MoreIndexingFilter.java. >>> When I indexing segments, I get large list of error messages: >>> can't parse errorenous date: Wed, 10 Sep 2003 11:59:14 or >>> can't parse errorenous date: Wed, 10 Sep 2003 11:59:14GMT >>> >>> I modifiing source code (I don't make 'patch'): >>> Original (lines 137-138): >>> DateFormat df = new SimpleDateFormat("EEE MMM dd HH:mm:ss yyyy >>> zzz"); >>> Date d = df.parse(date); >>> New: >>> DateFormat df = new SimpleDateFormat("EEE, MMM dd HH:mm:ss >>> yyyy", Locale.US); >>> Date d = df.parse(date.substring(0,25)); >>> >>> The modified code works fine. >>> >>> Best Regards, >>> Ferenc >>> >>> >>> >> >> >> >> > > > From nutch-devdfranklin@example.org Fri Jul 01 09:56:15 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 93895 invoked from network); 1 Jul 2005 09:56:14 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jul 2005 09:56:14 -0000 Received: (qmail 20838 invoked by uid 500); 1 Jul 2005 09:56:04 -0000 Delivered-To: dfranklin@example.org Received: (qmail 20703 invoked by uid 500); 1 Jul 2005 09:56:03 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 20663 invoked by uid 500); 1 Jul 2005 09:56:03 -0000 Delivered-To: dfranklin@example.org Received: (qmail 20646 invoked by uid 99); 1 Jul 2005 09:56:02 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO ajax.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jul 2005 02:56:02 -0700 Received: from ajax.apache.org (ajax.apache.org [83.161.120.155]) by ajax.apache.org (Postfix) with ESMTP id C47A5D2 for <dfranklin@example.org>; Fri, 1 Jul 2005 11:55:59 +0200 (CEST) Message-ID: <dfranklin@example.org> Date: Fri, 1 Jul 2005 11:55:59 +0200 (CEST) From: =?UTF-8?Q?Lutisch=C3=A1n_Ferenc_=28JIRA=29?= <dfranklin@example.org> To: dfranklin@example.org Subject: [jira] Created: (NUTCH-65) index-more plugin can't parse large set of modification-date Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N index-more plugin can't parse large set of modification-date ------------------------------------------------------------- Key: NUTCH-65 URL: http://issues.apache.org/jira/browse/NUTCH-65 Project: Nutch Type: Bug Components: indexer =20 Environment: nutch 0.7, java 1.5, linux Reporter: Lutisch=C3=A1n Ferenc I found a problem in MoreIndexingFilter.java. When I indexing segments, I get large list of error messages: can't parse errorenous date: Wed, 10 Sep 2003 11:59:14 or can't parse errorenous date: Wed, 10 Sep 2003 11:59:14GMT I modifiing source code (I don't make a 'patch'): Original (lines 137-138): DateFormat df =3D new SimpleDateFormat("EEE MMM dd HH:mm:ss yyyy zzz"); Date d =3D df.parse(date); New: DateFormat df =3D new SimpleDateFormat("EEE, MMM dd HH:mm:ss yyyy", Locale.= US); Date d =3D df.parse(date.substring(0,25)); The modified code works fine. --=20 This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira From nutch-devdfranklin@example.org Fri Jul 01 09:56:29 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 94169 invoked from network); 1 Jul 2005 09:56:29 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jul 2005 09:56:29 -0000 Received: (qmail 22877 invoked by uid 500); 1 Jul 2005 09:56:27 -0000 Delivered-To: dfranklin@example.org Received: (qmail 22862 invoked by uid 500); 1 Jul 2005 09:56:27 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 22849 invoked by uid 99); 1 Jul 2005 09:56:27 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jul 2005 02:56:27 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: neutral (asf.osuosl.org: local policy) Received: from [83.161.120.155] (HELO mx3.telekom.hu) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jul 2005 02:56:29 -0700 Message-ID: <dfranklin@example.org> Date: Fri, 01 Jul 2005 11:55:20 +0200 From: "dfranklin@example.org" <dfranklin@example.org> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; hu-HU; rv:1.7.8) Gecko/20050511 X-Accept-Language: hu, en MIME-Version: 1.0 To: dfranklin@example.org Subject: Re: Bug in index-more plugin? References: <dfranklin@example.org> <dfranklin@example.org> <dfranklin@example.org> <dfranklin@example.org> <dfranklin@example.org> <dfranklin@example.org> In-Reply-To: <dfranklin@example.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Antivirus: avast! (VPS 0526-3, 2005.06.30), Outbound message X-Antivirus-Status: Clean X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Dear Stephan, Thanks for url. Stefan Groschupf wotte: > You can find the nutch bug tracking here: > http://issues.apache.org/jira/secure/BrowseProject.jspa?id=10680 > Cheers, > Stefan > Am 01.07.2005 um 11:42 schrieb dfranklin@example.org: From nutch-devdfranklin@example.org Fri Jul 01 10:51:03 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 10720 invoked from network); 1 Jul 2005 10:51:03 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Jul 2005 10:51:03 -0000 Received: (qmail 23005 invoked by uid 500); 1 Jul 2005 10:51:01 -0000 Delivered-To: dfranklin@example.org Received: (qmail 22982 invoked by uid 500); 1 Jul 2005 10:51:00 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 22969 invoked by uid 500); 1 Jul 2005 10:51:00 -0000 Delivered-To: dfranklin@example.org Received: (qmail 22966 invoked by uid 99); 1 Jul 2005 10:51:00 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO ajax.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jul 2005 03:51:00 -0700 Received: from ajax.apache.org (ajax.apache.org [83.161.120.155]) by ajax.apache.org (Postfix) with ESMTP id 2B6E113 for <dfranklin@example.org>; Fri, 1 Jul 2005 12:50:57 +0200 (CEST) Message-ID: <dfranklin@example.org> Date: Fri, 1 Jul 2005 12:50:57 +0200 (CEST) From: "Jerome Charron (JIRA)" <dfranklin@example.org> To: dfranklin@example.org Subject: [jira] Commented: (NUTCH-65) index-more plugin can't parse large set of modification-date In-Reply-To: <dfranklin@example.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/NUTCH-65?page=3Dcomments#action_= 12314886 ]=20 Jerome Charron commented on NUTCH-65: ------------------------------------- It could be a good idea to declare the DataFormat as a final static constan= t ... no? > index-more plugin can't parse large set of modification-date > ------------------------------------------------------------- > > Key: NUTCH-65 > URL: http://issues.apache.org/jira/browse/NUTCH-65 > Project: Nutch > Type: Bug > Components: indexer > Environment: nutch 0.7, java 1.5, linux > Reporter: Lutisch=C3=A1n Ferenc > > I found a problem in MoreIndexingFilter.java. > When I indexing segments, I get large list of error messages: > can't parse errorenous date: Wed, 10 Sep 2003 11:59:14 or > can't parse errorenous date: Wed, 10 Sep 2003 11:59:14GMT > I modifiing source code (I don't make a 'patch'): > Original (lines 137-138): > DateFormat df =3D new SimpleDateFormat("EEE MMM dd HH:mm:ss yyyy zzz"); > Date d =3D df.parse(date); > New: > DateFormat df =3D new SimpleDateFormat("EEE, MMM dd HH:mm:ss yyyy", Local= e.US); > Date d =3D df.parse(date.substring(0,25)); > The modified code works fine. --=20 This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira From nutch-devdfranklin@example.org Sat Jul 02 19:32:19 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 37687 invoked from network); 2 Jul 2005 19:32:19 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Jul 2005 19:32:19 -0000 Received: (qmail 86108 invoked by uid 500); 2 Jul 2005 19:32:14 -0000 Delivered-To: dfranklin@example.org Received: (qmail 86086 invoked by uid 500); 2 Jul 2005 19:32:13 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 86073 invoked by uid 500); 2 Jul 2005 19:32:13 -0000 Delivered-To: dfranklin@example.org Received: (qmail 86059 invoked by uid 99); 2 Jul 2005 19:32:13 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO ajax.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Jul 2005 12:32:13 -0700 Received: from ajax.apache.org (ajax.apache.org [83.161.120.155]) by ajax.apache.org (Postfix) with ESMTP id 22CFA13 for <dfranklin@example.org>; Sat, 2 Jul 2005 21:32:10 +0200 (CEST) Message-ID: <dfranklin@example.org> Date: Sat, 2 Jul 2005 21:32:10 +0200 (CEST) From: "Andrzej Bialecki (JIRA)" <dfranklin@example.org> To: dfranklin@example.org Subject: [jira] Closed: (NUTCH-60) Bad language identifier plugin performances In-Reply-To: <{{EMAIL+PHONE}}> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/NUTCH-60?page=all ] Andrzej Bialecki closed NUTCH-60: ---------------------------------- Resolution: Fixed Patches have been applied. Thanks! > Bad language identifier plugin performances > ------------------------------------------- > > Key: NUTCH-60 > URL: http://issues.apache.org/jira/browse/NUTCH-60 > Project: Nutch > Type: Improvement > Components: indexer > Reporter: Jerome Charron > Priority: Minor > Attachments: NUTCH-60-050526.patch, NUTCH-60-050605.patch, NUTCH-60-050607.patch, NUTCH-60-050627.patch > > As reported by Stefan Groschupf (http://dfranklin@example.org/msg04090.html) the language identifier plugin consumes a lot of processing time. > Some optimizations and/or configuration options are required. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira From nutch-devdfranklin@example.org Sat Jul 02 19:43:14 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 39788 invoked from network); 2 Jul 2005 19:43:14 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Jul 2005 19:43:14 -0000 Received: (qmail 93127 invoked by uid 500); 2 Jul 2005 19:43:13 -0000 Delivered-To: dfranklin@example.org Received: (qmail 92926 invoked by uid 500); 2 Jul 2005 19:43:13 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 92909 invoked by uid 500); 2 Jul 2005 19:43:13 -0000 Delivered-To: dfranklin@example.org Received: (qmail 92905 invoked by uid 99); 2 Jul 2005 19:43:13 -0000 X-ASF-Spam-Status: No, hits=0.6 required=10.0 tests=HOT_NASTY X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO ajax.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Jul 2005 12:43:12 -0700 Received: from ajax.apache.org (ajax.apache.org [83.161.120.155]) by ajax.apache.org (Postfix) with ESMTP id 07D8B15 for <dfranklin@example.org>; Sat, 2 Jul 2005 21:43:10 +0200 (CEST) Message-ID: <dfranklin@example.org> Date: Sat, 2 Jul 2005 21:43:10 +0200 (CEST) From: "Andrzej Bialecki (JIRA)" <dfranklin@example.org> To: dfranklin@example.org Subject: [jira] Closed: (NUTCH-57) text and html files unrecognized In-Reply-To: <{{EMAIL+PHONE}}> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/NUTCH-57?page=all ] Andrzej Bialecki closed NUTCH-57: ---------------------------------- Resolution: Fixed Applied. > text and html files unrecognized > -------------------------------- > > Key: NUTCH-57 > URL: http://issues.apache.org/jira/browse/NUTCH-57 > Project: Nutch > Type: Bug > Components: indexer > Environment: Nutch 0.7Dev > Reporter: Marc Delerue > Attachments: NUTCH-57-050509.patch > > While crawling : > http://XXX.XXX.XXX.XXX/yyyyy.txtorg.apache.nutch.util.mime.MimeTypeException : invalid Sub Type plain > and > http://XXX.XXX.XXX.XXX/yyyyy.htmlorg.apache.nutch.util.mime.MimeTypeException : invalid Sub Type html > The html and text files are fetched but not indexed. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira From nutch-devdfranklin@example.org Sat Jul 02 19:54:15 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 41639 invoked from network); 2 Jul 2005 19:54:14 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Jul 2005 19:54:14 -0000 Received: (qmail 2889 invoked by uid 500); 2 Jul 2005 19:54:13 -0000 Delivered-To: dfranklin@example.org Received: (qmail 2865 invoked by uid 500); 2 Jul 2005 19:54:13 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 2851 invoked by uid 500); 2 Jul 2005 19:54:13 -0000 Delivered-To: dfranklin@example.org Received: (qmail 2842 invoked by uid 99); 2 Jul 2005 19:54:12 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO ajax.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Jul 2005 12:54:12 -0700 Received: from ajax.apache.org (ajax.apache.org [83.161.120.155]) by ajax.apache.org (Postfix) with ESMTP id B0A9312 for <dfranklin@example.org>; Sat, 2 Jul 2005 21:54:09 +0200 (CEST) Message-ID: <dfranklin@example.org> Date: Sat, 2 Jul 2005 21:54:09 +0200 (CEST) From: "Andrzej Bialecki (JIRA)" <dfranklin@example.org> To: dfranklin@example.org Subject: [jira] Closed: (NUTCH-27) Patch to get a status of running Fetcher In-Reply-To: <dfranklin@example.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/NUTCH-27?page=all ] Andrzej Bialecki closed NUTCH-27: ---------------------------------- Resolution: Fixed Applied. > Patch to get a status of running Fetcher > ---------------------------------------- > > Key: NUTCH-27 > URL: http://issues.apache.org/jira/browse/NUTCH-27 > Project: Nutch > Type: Improvement > Components: fetcher > Reporter: Stefan Grroschupf > > transferred from: > http://sourceforge.net/tracker/index.php?func=detail&aid=987298&group_id=59548&atid=491356 > submitted by: > Andrzej Bialecki > When running a Fetcher it helps to know how many pages > have been processed so far. Currently this information > is printed to a log, but it would be useful to be able > to get it programmatically whenever the managing > application needs it. > The attached patch provides this functionality. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira From nutch-devdfranklin@example.org Sat Jul 02 20:26:15 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 48508 invoked from network); 2 Jul 2005 20:26:15 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Jul 2005 20:26:15 -0000 Received: (qmail 37413 invoked by uid 500); 2 Jul 2005 20:26:14 -0000 Delivered-To: dfranklin@example.org Received: (qmail 37212 invoked by uid 500); 2 Jul 2005 20:26:13 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 37198 invoked by uid 500); 2 Jul 2005 20:26:13 -0000 Delivered-To: dfranklin@example.org Received: (qmail 37194 invoked by uid 99); 2 Jul 2005 20:26:13 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO ajax.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Jul 2005 13:26:13 -0700 Received: from ajax.apache.org (ajax.apache.org [83.161.120.155]) by ajax.apache.org (Postfix) with ESMTP id 2D47A13 for <dfranklin@example.org>; Sat, 2 Jul 2005 22:26:10 +0200 (CEST) Message-ID: <dfranklin@example.org> Date: Sat, 2 Jul 2005 22:26:10 +0200 (CEST) From: "Andrzej Bialecki (JIRA)" <dfranklin@example.org> To: dfranklin@example.org Subject: [jira] Closed: (NUTCH-32) Nutch Webapp could only be deployed on root namespace In-Reply-To: <{{EMAIL+PHONE}}> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/NUTCH-32?page=all ] Andrzej Bialecki closed NUTCH-32: ---------------------------------- > Nutch Webapp could only be deployed on root namespace > ----------------------------------------------------- > > Key: NUTCH-32 > URL: http://issues.apache.org/jira/browse/NUTCH-32 > Project: Nutch > Type: Bug > Components: web gui > Reporter: Jerome Charron > Priority: Minor > Attachments: NUTCH-32.new.patch, NUTCH-32.patch > > Some links in the JSP pages are using absolute paths instead of relative ones. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira From nutch-devdfranklin@example.org Sat Jul 02 20:37:14 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 56023 invoked from network); 2 Jul 2005 20:37:14 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Jul 2005 20:37:14 -0000 Received: (qmail 44265 invoked by uid 500); 2 Jul 2005 20:37:13 -0000 Delivered-To: dfranklin@example.org Received: (qmail 44081 invoked by uid 500); 2 Jul 2005 20:37:12 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 44068 invoked by uid 500); 2 Jul 2005 20:37:12 -0000 Delivered-To: dfranklin@example.org Received: (qmail 44065 invoked by uid 99); 2 Jul 2005 20:37:12 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO ajax.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Jul 2005 13:37:12 -0700 Received: from ajax.apache.org (ajax.apache.org [83.161.120.155]) by ajax.apache.org (Postfix) with ESMTP id B406213 for <dfranklin@example.org>; Sat, 2 Jul 2005 22:37:09 +0200 (CEST) Message-ID: <dfranklin@example.org> Date: Sat, 2 Jul 2005 22:37:09 +0200 (CEST) From: "CC Chaman (JIRA)" <dfranklin@example.org> To: dfranklin@example.org Subject: [jira] Created: (NUTCH-66) Cookies are not being read properly Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Cookies are not being read properly ----------------------------------- Key: NUTCH-66 URL: http://issues.apache.org/jira/browse/NUTCH-66 Project: Nutch Type: Improvement Components: fetcher Reporter: CC Chaman Priority: Minor Cookies that do not begin with a period are not being accepted. For example "cnn.com" instead of the RFC ".cnn.com". But A LOT of sites seem to not know the standard. It would be nice if the plugin accepted those cookies as well. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira From nutch-devdfranklin@example.org Sat Jul 02 20:48:15 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 61212 invoked from network); 2 Jul 2005 20:48:14 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Jul 2005 20:48:14 -0000 Received: (qmail 49519 invoked by uid 500); 2 Jul 2005 20:48:14 -0000 Delivered-To: dfranklin@example.org Received: (qmail 49499 invoked by uid 500); 2 Jul 2005 20:48:13 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 49486 invoked by uid 500); 2 Jul 2005 20:48:13 -0000 Delivered-To: dfranklin@example.org Received: (qmail 49483 invoked by uid 99); 2 Jul 2005 20:48:13 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO ajax.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Jul 2005 13:48:13 -0700 Received: from ajax.apache.org (ajax.apache.org [83.161.120.155]) by ajax.apache.org (Postfix) with ESMTP id 9C4B712 for <dfranklin@example.org>; Sat, 2 Jul 2005 22:48:10 +0200 (CEST) Message-ID: <dfranklin@example.org> Date: Sat, 2 Jul 2005 22:48:10 +0200 (CEST) From: "Andrzej Bialecki (JIRA)" <dfranklin@example.org> To: dfranklin@example.org Subject: [jira] Closed: (NUTCH-56) Crawling sites with 403 Forbidden robots.txt In-Reply-To: <{{EMAIL+PHONE}}> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/NUTCH-56?page=all ] Andrzej Bialecki closed NUTCH-56: ---------------------------------- Resolution: Fixed Applied. I changed the name of the property to follow an already existing "http.robots.*" hierarchy. Thanks! > Crawling sites with 403 Forbidden robots.txt > -------------------------------------------- > > Key: NUTCH-56 > URL: http://issues.apache.org/jira/browse/NUTCH-56 > Project: Nutch > Type: Improvement > Components: fetcher > Reporter: Andy Liu > Priority: Minor > Attachments: robots_403.patch > > If a 403 error is encountered when trying to access the robots.txt file, Nutch does not crawl any pages from that site. This behavior is consistent with the RFC recommendation for the robot exclusion protocol. > However, Google does crawl sites that exhibit this type of behavior, because most webmasters of these sites are unaware of robots.txt conventions and do want their site to be crawled. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira From nutch-devdfranklin@example.org Sun Jul 03 15:06:14 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 40242 invoked from network); 3 Jul 2005 15:06:14 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Jul 2005 15:06:14 -0000 Received: (qmail 83778 invoked by uid 500); 3 Jul 2005 15:06:13 -0000 Delivered-To: dfranklin@example.org Received: (qmail 83548 invoked by uid 500); 3 Jul 2005 15:06:11 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 83534 invoked by uid 99); 3 Jul 2005 15:06:11 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 08:06:11 -0700 X-ASF-Spam-Status: No, hits=0.2 required=10.0 tests=HTML_90_100,HTML_MESSAGE X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: domain of dfranklin@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mx14.yandex.ru) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 08:06:12 -0700 Received: from e-0.ar3.brd.zp.ukrtel.net ([83.161.120.155]:4593 "EHLO me" smtp-auth: "shortname" TLS-CIPHER: <none> TLS-PEER-CN1: <none>) by mail.yandex.ru with ESMTP id S1782708AbVGCPGE (ORCPT <rfc822;dfranklin@example.org>); Sun, 3 Jul 2005 19:06:04 +0400 X-Comment: RFC 2476 MSA dfranklin@example.org logged sender identity as: shortname From: "Ilia S. Yatsenko" <dfranklin@example.org> To: <dfranklin@example.org> Subject: both html parser have bug with javascript Date: Sun, 3 Jul 2005 18:05:57 +0300 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0000_01C57FF9.DD983050" X-Mailer: Microsoft Office Outlook, Build 11.0.5510 Thread-Index: AcV/4LdLa6j2EkYLRQOOn/FkYLFA5A== X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Message-Id: <dfranklin@example.org> X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N ------=_NextPart_000_0000_01C57FF9.DD983050 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Hello :) Sorry my little English I have issue with both html parsers. I see in summaries next text: 2JavaScript1.3JavaScriptJavaScriptjavascriptjavascript1.1javascript1.2javasc ript1.3javascript my text description. Or 2javascript my text description. Or javascriptjavascript1.2javascript my text description. But summary not should have it Respectfully ------=_NextPart_000_0000_01C57FF9.DD983050-- From nutch-devdfranklin@example.org Sun Jul 03 16:09:28 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 55495 invoked from network); 3 Jul 2005 16:09:28 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Jul 2005 16:09:28 -0000 Received: (qmail 25819 invoked by uid 500); 3 Jul 2005 16:09:27 -0000 Delivered-To: dfranklin@example.org Received: (qmail 25617 invoked by uid 500); 3 Jul 2005 16:09:26 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 25604 invoked by uid 99); 3 Jul 2005 16:09:26 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 09:09:26 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: domain of dfranklin@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mx14.yandex.ru) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 09:09:27 -0700 Received: from e-0.ar3.brd.zp.ukrtel.net ([83.161.120.155]:51188 "EHLO me" smtp-auth: "shortname" TLS-CIPHER: <none> TLS-PEER-CN1: <none>) by mail.yandex.ru with ESMTP id S1782396AbVGCQJU (ORCPT <rfc822;dfranklin@example.org>); Sun, 3 Jul 2005 20:09:20 +0400 X-Comment: RFC 2476 MSA dfranklin@example.org logged sender identity as: shortname From: "Ilia S. Yatsenko" <dfranklin@example.org> To: <dfranklin@example.org> Subject: RE: both html parser have bug with javascript Date: Sun, 3 Jul 2005 19:09:15 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 Thread-Index: AcV/4LdLa6j2EkYLRQOOn/FkYLFA5AACMxhA X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 In-Reply-To: <dfranklin@example.org> Message-Id: <dfranklin@example.org> X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Opps, I see my mistake O-) -----Original Message----- From: Ilia S. Yatsenko [mailto:dfranklin@example.org] Sent: Sunday, July 03, 2005 6:06 PM To: dfranklin@example.org Subject: both html parser have bug with javascript Hello :) Sorry my little English I have issue with both html parsers. I see in summaries next text: 2JavaScript1.3JavaScriptJavaScriptjavascriptjavascript1.1javascript1.2javasc ript1.3javascript my text description. Or 2javascript my text description. Or javascriptjavascript1.2javascript my text description. But summary not should have it Respectfully From nutch-devdfranklin@example.org Mon Jul 04 00:19:25 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 46389 invoked from network); 4 Jul 2005 00:19:24 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 4 Jul 2005 00:19:24 -0000 Received: (qmail 90180 invoked by uid 500); 4 Jul 2005 00:19:23 -0000 Delivered-To: dfranklin@example.org Received: (qmail 90136 invoked by uid 500); 4 Jul 2005 00:19:22 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 90123 invoked by uid 99); 4 Jul 2005 00:19:22 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 17:19:22 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: local policy) Received: from [83.161.120.155] (HELO mail.mxinteractive.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 17:19:23 -0700 Received: from localhost ([83.161.120.155]) by mail.mxinteractive.com for dfranklin@example.org; Sun, 3 Jul 2005 20:16:53 -0400 From: "Chirag Chaman" <dfranklin@example.org> To: <dfranklin@example.org> Subject: RE: both html parser have bug with javascript Date: Sun, 3 Jul 2005 20:17:09 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.6353 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Thread-Index: AcV/4LdLa6j2EkYLRQOOn/FkYLFA5AACMxhAABD93lA= In-Reply-To: <dfranklin@example.org> X-Virus-Checked: Checked by ClamAV on apache.org Message-Id: <dfranklin@example.org> X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Actually, I think the JavaScript is there as it's part of the HTML page -- but it should not be part of the summaries. Has anyone found a solution to not showing the "JavaScript" or "text/css" -- that shows up from time to time? CC- -----Original Message----- From: Ilia S. Yatsenko [mailto:dfranklin@example.org] Sent: Sunday, July 03, 2005 12:09 PM To: dfranklin@example.org Subject: RE: both html parser have bug with javascript Opps, I see my mistake O-) -----Original Message----- From: Ilia S. Yatsenko [mailto:dfranklin@example.org] Sent: Sunday, July 03, 2005 6:06 PM To: dfranklin@example.org Subject: both html parser have bug with javascript Hello :) Sorry my little English I have issue with both html parsers. I see in summaries next text: 2JavaScript1.3JavaScriptJavaScriptjavascriptjavascript1.1javascript1.2javasc ript1.3javascript my text description. Or 2javascript my text description. Or javascriptjavascript1.2javascript my text description. But summary not should have it Respectfully From nutch-devdfranklin@example.org Mon Jul 04 01:57:18 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 80453 invoked from network); 4 Jul 2005 01:57:18 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 4 Jul 2005 01:57:18 -0000 Received: (qmail 56178 invoked by uid 500); 4 Jul 2005 01:57:17 -0000 Delivered-To: dfranklin@example.org Received: (qmail 56137 invoked by uid 500); 4 Jul 2005 01:57:16 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 56119 invoked by uid 500); 4 Jul 2005 01:57:16 -0000 Delivered-To: dfranklin@example.org Received: (qmail 56112 invoked by uid 99); 4 Jul 2005 01:57:15 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO ajax.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 18:57:15 -0700 Received: from ajax.apache.org (ajax.apache.org [83.161.120.155]) by ajax.apache.org (Postfix) with ESMTP id AB52212 for <dfranklin@example.org>; Mon, 4 Jul 2005 03:57:12 +0200 (CEST) Message-ID: <dfranklin@example.org> Date: Mon, 4 Jul 2005 03:57:12 +0200 (CEST) From: "Nick Lothian (JIRA)" <dfranklin@example.org> To: dfranklin@example.org Subject: [jira] Commented: (NUTCH-65) index-more plugin can't parse large set of modification-date In-Reply-To: <dfranklin@example.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/NUTCH-65?page=3Dcomments#action_= 12314976 ]=20 Nick Lothian commented on NUTCH-65: ----------------------------------- Re: It could be a good idea to declare the DataFormat as a final static co= nstant ... no? NO! SimpleDateFormat is not thread safe. Using it as a final static field i= s a bug. See (for instance): http://www-106.ibm.com/developerworks/java/lib= rary/j-jtp09263.html > index-more plugin can't parse large set of modification-date > ------------------------------------------------------------- > > Key: NUTCH-65 > URL: http://issues.apache.org/jira/browse/NUTCH-65 > Project: Nutch > Type: Bug > Components: indexer > Environment: nutch 0.7, java 1.5, linux > Reporter: Lutisch=C3=A1n Ferenc > > I found a problem in MoreIndexingFilter.java. > When I indexing segments, I get large list of error messages: > can't parse errorenous date: Wed, 10 Sep 2003 11:59:14 or > can't parse errorenous date: Wed, 10 Sep 2003 11:59:14GMT > I modifiing source code (I don't make a 'patch'): > Original (lines 137-138): > DateFormat df =3D new SimpleDateFormat("EEE MMM dd HH:mm:ss yyyy zzz"); > Date d =3D df.parse(date); > New: > DateFormat df =3D new SimpleDateFormat("EEE, MMM dd HH:mm:ss yyyy", Local= e.US); > Date d =3D df.parse(date.substring(0,25)); > The modified code works fine. --=20 This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira From nutch-devdfranklin@example.org Mon Jul 04 03:18:09 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 97069 invoked from network); 4 Jul 2005 03:18:08 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 4 Jul 2005 03:18:08 -0000 Received: (qmail 22013 invoked by uid 500); 4 Jul 2005 03:18:07 -0000 Delivered-To: dfranklin@example.org Received: (qmail 21981 invoked by uid 500); 4 Jul 2005 03:18:07 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 21963 invoked by uid 500); 4 Jul 2005 03:18:06 -0000 Delivered-To: dfranklin@example.org Received: (qmail 21958 invoked by uid 99); 4 Jul 2005 03:18:06 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 20:18:06 -0700 X-ASF-Spam-Status: No, hits=0.1 required=10.0 tests=HTML_40_50,HTML_MESSAGE,RCVD_BY_IP X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: domain of dfranklin@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO wproxy.gmail.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 20:18:07 -0700 Received: by wproxy.gmail.com with SMTP id 50so1429036wri for <dfranklin@example.org>; Sun, 03 Jul 2005 20:18:03 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:references; b=YSDuJ7kctIKIrNBKiGNlMo0IiviAv8AkufOhHjBW+MeXvYVnToeh/QIuEPP30DQAkhfMjxo8npyhJmtKD+43g+O+0bsGGxavCaKMZExVps2RE38QEz3uH65ZLM/4sPHCqGsq5+6ioP32M8lLSzwd8JywrgRlIdQKNP81x/iz2hk= Received: by 83.161.120.155 with SMTP id 22mr3415905wrr; Sun, 03 Jul 2005 20:18:03 -0700 (PDT) Received: by 83.161.120.155 with HTTP; Sun, 3 Jul 2005 20:18:03 -0700 (PDT) Message-ID: <dfranklin@example.org> Date: Mon, 4 Jul 2005 11:18:03 +0800 From: =?GB2312?Q?Nutch=BF=AA=B7=A2=D3=CA=BC=FE?= <dfranklin@example.org> Reply-To: =?GB2312?Q?Nutch=BF=AA=B7=A2=D3=CA=BC=FE?= <dfranklin@example.org> To: dfranklin@example.org, Andy Liu <dfranklin@example.org> Subject: Re: Why Crawl failed to fetch so many pages? In-Reply-To: <dfranklin@example.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_10050_4253473.1120447083094" References: <dfranklin@example.org> <dfranklin@example.org> <dfranklin@example.org> <dfranklin@example.org> X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N ------=_Part_10050_4253473.1120447083094 Content-Type: text/plain; charset=GB2312 Content-Transfer-Encoding: base64 Content-Disposition: inline cGxlYXNlIG1vZGlmeSBiZWxvdyAKo6gjIHNraXAgVVJMcyBjb250YWluaW5nIGNlcnRhaW4gY2hh cmFjdGVycyBhcyBwcm9iYWJsZSBxdWVyaWVzLCBldGMuCiMgLVs/KiFAPV2jqQpiZWNhdXNlIHRo ZSBsaW5rIApodHRwOi8vbmV3cy5idWFhLmVkdS5jbi9kaXNwbmV3cy5waHA/dHlwZT0xJm5pZD0y NTAwJnNfdGFibGU9bmV3c190eHQKaW5jbHVkZXMgdGhlID89JiB3aGljaCB3aWxsIGJlIGlnbm9y ZWQgCml0IHdpbGwgYmUgCigjIHNraXAgVVJMcyBjb250YWluaW5nIGNlcnRhaW4gY2hhcmFjdGVy cyBhcyBwcm9iYWJsZSBxdWVyaWVzLCBldGMuCiMgLVtAXSkKCgoKMjAwNS80LzE0LCBBbmR5IExp dSA8YW5keWxpdTEyMjdAZ21haWwuY29tPjoKPiAKPiBCeSBkZWZhdWx0LCBOdXRjaCBvbmx5IGNy YXdscyB0aGUgZmlyc3QgMTAwIG91dGxpbmtzIG9uIGEgcGFnZS4gTWF5YmUKPiB0aGF0J3MgeW91 ciBwcm9ibGVtPwo+IAo+IE9uIDQvMTQvMDUsIE1hdHRoaWFzIEphZWtsZSA8amFla2xlQGV2ZW50 YXguZGU+IHdyb3RlOgo+ID4gPiB0cnkKPiA+ID4gK15odHRwOi8vbmV3cy5idWFhLmVkdS5jbi8q Cj4gPiBUaGlzIHNob3VsZCBub3QgYmUgdGhlIHJlYXNvbi4KPiA+IFlvdXIgcmVnZXggZml0cyBv biB1cmxzIHN0YXJ0aW5nIHdpdGg6Cj4gPiBodHRwOi8vbmV3cy5idWFhLmVkdS5jbgo+ID4gaHR0 cDovL25ld3MuYnVhYS5lZHUuY24vCj4gPiBodHRwOi8vbmV3cy5idWFhLmVkdS5jbi8vCj4gPiBo dHRwOi8vbmV3cy5idWFhLmVkdS5jbi8vLyAuLi4KPiA+Cj4gPiBUaGUgb25seSB0aGluZyBJIHdv dWxkIHRyeSBpcyB0byBlc2NhcGUgc29tZSBjYXJhY3RlcnMgdG8gbWFrZSBpdCBtb3JlCj4gPiBj b3JyZWN0LiBBIGRvdCBtZWFucyBldmVyeSBwb3NzaWJsZSBzaWduLiBCZXR0ZXI6Cj4gPiArXmh0 dHA6XC9cL25ld3NcLmJ1YWFcLmVkdVwuY24KPiA+Cj4gPiBEaWQgeW91IG1ha2UgZW5vdWdoIHJv dW5kcywgdG8gZ2V0IHRoZSB3YW50ZWQgZGVwdGg/Cj4gPiBXaXRoIGV2ZXJ5IGNyYXdsIHlvdSBv bmx5IGZldGNoIHRoZSBhbHJlYWR5IGtub3duIGxpbmtzLgo+ID4KPiA+IE1hdHRoaWFzCj4gPgo+ ID4gLS0KPiA+IGh0dHA6Ly93d3cuZXZlbnRheC5jb20gLSBldmVudGF4IEdtYkgKPiA+IGh0dHA6 Ly93d3cudW1rcmVpc2ZpbmRlci5kZSAtIERpZSBTdWNobWFzY2hpbmUgZqi5ciBMb2thbGVzIHVu ZCBFdmVudHMKPiA+Cj4gCgoKCi0tIApURUwgMDUxMi02ODI1MTIzMy02OTY2Ck1TTjpwcmV0dHlz aW5vQGhvdG1haWwuY29tCk1haWw6amltaWppbnpoYW5nQEJlblEuY29tClFROjU4NjI0OTUxCkJl blEuY29tIDxodHRwOi8vQmVuUS5jb20+CjI2OCBTaGlzaGFuIFJvYWQsIE5ldyBEaXN0cmljdCwg ClN1emhvdSwgQ2hpbmEK ------=_Part_10050_4253473.1120447083094-- From nutch-devdfranklin@example.org Mon Jul 04 03:33:47 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 99796 invoked from network); 4 Jul 2005 03:33:47 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 4 Jul 2005 03:33:47 -0000 Received: (qmail 32095 invoked by uid 500); 4 Jul 2005 03:33:44 -0000 Delivered-To: dfranklin@example.org Received: (qmail 32065 invoked by uid 500); 4 Jul 2005 03:33:44 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 32028 invoked by uid 99); 4 Jul 2005 03:33:44 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 20:33:44 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: domain of dfranklin@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mx14.yandex.ru) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 20:33:45 -0700 Received: from e-0.ar3.brd.zp.ukrtel.net ([83.161.120.155]:1022 "EHLO me" smtp-auth: "shortname" TLS-CIPHER: <none> TLS-PEER-CN1: <none>) by mail.yandex.ru with ESMTP id S1782577AbVGDDd2 (ORCPT <rfc822;dfranklin@example.org>); Mon, 4 Jul 2005 07:33:28 +0400 X-Comment: RFC 2476 MSA dfranklin@example.org logged sender identity as: shortname From: "Ilia S. Yatsenko" <dfranklin@example.org> To: <dfranklin@example.org> Subject: RE: both html parser have bug with javascript Date: Mon, 4 Jul 2005 06:33:24 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 In-Reply-To: <dfranklin@example.org> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Thread-Index: AcV/4LdLa6j2EkYLRQOOn/FkYLFA5AACMxhAABfaEiA= Message-Id: <dfranklin@example.org> X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N I thought "javascript" shown in summaries because I enable parse-js plug-in. I have disabled it, made new database but got the same result :( -----Original Message----- From: Ilia S. Yatsenko [mailto:dfranklin@example.org] Sent: Sunday, July 03, 2005 7:09 PM To: dfranklin@example.org Subject: RE: both html parser have bug with javascript Opps, I see my mistake O-) -----Original Message----- From: Ilia S. Yatsenko [mailto:dfranklin@example.org] Sent: Sunday, July 03, 2005 6:06 PM To: dfranklin@example.org Subject: both html parser have bug with javascript Hello :) Sorry my little English I have issue with both html parsers. I see in summaries next text: 2JavaScript1.3JavaScriptJavaScriptjavascriptjavascript1.1javascript1.2javasc ript1.3javascript my text description. Or 2javascript my text description. Or javascriptjavascript1.2javascript my text description. But summary not should have it Respectfully From nutch-devdfranklin@example.org Mon Jul 04 03:42:16 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 2228 invoked from network); 4 Jul 2005 03:42:16 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 4 Jul 2005 03:42:16 -0000 Received: (qmail 38335 invoked by uid 500); 4 Jul 2005 03:42:15 -0000 Delivered-To: dfranklin@example.org Received: (qmail 38126 invoked by uid 500); 4 Jul 2005 03:42:14 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 38104 invoked by uid 500); 4 Jul 2005 03:42:14 -0000 Delivered-To: dfranklin@example.org Received: (qmail 38093 invoked by uid 99); 4 Jul 2005 03:42:14 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO ajax.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 20:42:13 -0700 Received: from ajax.apache.org (ajax.apache.org [83.161.120.155]) by ajax.apache.org (Postfix) with ESMTP id DC0AA17 for <dfranklin@example.org>; Mon, 4 Jul 2005 05:42:10 +0200 (CEST) Message-ID: <dfranklin@example.org> Date: Mon, 4 Jul 2005 05:42:10 +0200 (CEST) From: "zhangjin (JIRA)" <dfranklin@example.org> To: dfranklin@example.org Subject: [jira] Created: (NUTCH-67) I want crawl the websites including news.yahoo.com,game.yahoo.com,blog.yahoo.com,etc! Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N I want crawl the websites including news.yahoo.com,game.yahoo.com,blog.yahoo.com,etc! --------------------------------------------------------------------------------------- Key: NUTCH-67 URL: http://issues.apache.org/jira/browse/NUTCH-67 Project: Nutch Type: Wish Components: fetcher Environment: Windows 2000,weblogic Reporter: zhangjin how do I config them in the crawl-urlfilter.txt? I config them below,but it is not successful. # The url filter file used by the crawl command. # Better for intranet crawling. # Be sure to change MY.DOMAIN.NAME to your domain name. # Each non-comment, non-blank line contains a regular expression # prefixed by '+' or '-'. The first matching pattern in the file # determines whether a URL is included or ignored. If no pattern # matches, the URL is ignored. # skip file:, ftp:, & mailto: urls -^(file|ftp|mailto): # skip image and other suffixes we can't yet parse -\.(gif|GIF|jpg|JPG|ico|ICO|css|sit|eps|wmf|zip|ppt|mpg|xls|gz|rpm|tgz|mov|MOV|exe)$ # skip URLs containing certain characters as probable queries, etc. -[?*!@=] # accept hosts in MY.DOMAIN.NAME #+^http://([a-z0-9]*\.)*MY.DOMAIN.NAME/ +^http://([a-z0-9]*\.)*yahoo.com/ # skip everything else #-. but It can not work, and can not crawl the domain name (DOMAIN.NAME) inluding news.yahoo.com,game.yahoo.com,blog.yahoo.com why? -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira From nutch-devdfranklin@example.org Mon Jul 04 03:43:39 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 2660 invoked from network); 4 Jul 2005 03:43:39 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 4 Jul 2005 03:43:39 -0000 Received: (qmail 39455 invoked by uid 500); 4 Jul 2005 03:43:39 -0000 Delivered-To: dfranklin@example.org Received: (qmail 39253 invoked by uid 500); 4 Jul 2005 03:43:38 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 39240 invoked by uid 99); 4 Jul 2005 03:43:38 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 20:43:38 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: domain of dfranklin@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mx14.yandex.ru) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 20:43:39 -0700 Received: from e-0.ar3.brd.zp.ukrtel.net ([83.161.120.155]:11774 "EHLO me" smtp-auth: "shortname" TLS-CIPHER: <none> TLS-PEER-CN1: <none>) by mail.yandex.ru with ESMTP id S1782437AbVGDDna (ORCPT <rfc822;dfranklin@example.org>); Mon, 4 Jul 2005 07:43:30 +0400 X-Comment: RFC 2476 MSA dfranklin@example.org logged sender identity as: shortname From: "Ilia S. Yatsenko" <dfranklin@example.org> To: <dfranklin@example.org> Subject: RE: both html parser have bug with javascript Date: Mon, 4 Jul 2005 06:43:26 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 In-Reply-To: <dfranklin@example.org> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Thread-Index: AcV/4LdLa6j2EkYLRQOOn/FkYLFA5AACMxhAABfaEiAAAEv5sA== Message-Id: <dfranklin@example.org> X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N And this <%@ Language=VBScript %> shown in summaries I thought ANY text between < and > should be always ignored and unknown tags too. :) -----Original Message----- From: Ilia S. Yatsenko [mailto:dfranklin@example.org] Sent: Monday, July 04, 2005 6:33 AM To: dfranklin@example.org Subject: RE: both html parser have bug with javascript I thought "javascript" shown in summaries because I enable parse-js plug-in. I have disabled it, made new database but got the same result :( -----Original Message----- From: Ilia S. Yatsenko [mailto:dfranklin@example.org] Sent: Sunday, July 03, 2005 7:09 PM To: dfranklin@example.org Subject: RE: both html parser have bug with javascript Opps, I see my mistake O-) -----Original Message----- From: Ilia S. Yatsenko [mailto:dfranklin@example.org] Sent: Sunday, July 03, 2005 6:06 PM To: dfranklin@example.org Subject: both html parser have bug with javascript Hello :) Sorry my little English I have issue with both html parsers. I see in summaries next text: 2JavaScript1.3JavaScriptJavaScriptjavascriptjavascript1.1javascript1.2javasc ript1.3javascript my text description. Or 2javascript my text description. Or javascriptjavascript1.2javascript my text description. But summary not should have it Respectfully From nutch-devdfranklin@example.org Mon Jul 04 03:55:23 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 5907 invoked from network); 4 Jul 2005 03:55:23 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 4 Jul 2005 03:55:23 -0000 Received: (qmail 49845 invoked by uid 500); 4 Jul 2005 03:55:22 -0000 Delivered-To: dfranklin@example.org Received: (qmail 49667 invoked by uid 500); 4 Jul 2005 03:55:21 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 49653 invoked by uid 99); 4 Jul 2005 03:55:21 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 20:55:21 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: domain of dfranklin@example.org designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mx14.yandex.ru) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jul 2005 20:55:22 -0700 Received: from e-0.ar3.brd.zp.ukrtel.net ([83.161.120.155]:11759 "EHLO me" smtp-auth: "shortname" TLS-CIPHER: <none> TLS-PEER-CN1: <none>) by mail.yandex.ru with ESMTP id S1782611AbVGDDzO (ORCPT <rfc822;dfranklin@example.org>); Mon, 4 Jul 2005 07:55:14 +0400 X-Comment: RFC 2476 MSA dfranklin@example.org logged sender identity as: shortname From: "Ilia S. Yatsenko" <dfranklin@example.org> To: <dfranklin@example.org> Subject: RE: [jira] Created: (NUTCH-67) I want crawl the websites including news.yahoo.com,game.yahoo.com,blog.yahoo.com,etc! Date: Mon, 4 Jul 2005 06:55:10 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 In-Reply-To: <dfranklin@example.org> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Thread-Index: AcWAS2vVIyrFOTm/QPWaWNpD8EjE8wAAJLGA Message-Id: <dfranklin@example.org> X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Try this +^http://([a-z0-9\.\-]*)\.yahoo\.com/ I hope it help you :) -----Original Message----- From: zhangjin (JIRA) [mailto:dfranklin@example.org] Sent: Monday, July 04, 2005 6:42 AM To: dfranklin@example.org Subject: [jira] Created: (NUTCH-67) I want crawl the websites including news.yahoo.com,game.yahoo.com,blog.yahoo.com,etc! I want crawl the websites including news.yahoo.com,game.yahoo.com,blog.yahoo.com,etc! ---------------------------------------------------------------------------- ----------- Key: NUTCH-67 URL: http://issues.apache.org/jira/browse/NUTCH-67 Project: Nutch Type: Wish Components: fetcher Environment: Windows 2000,weblogic Reporter: zhangjin how do I config them in the crawl-urlfilter.txt? I config them below,but it is not successful. # The url filter file used by the crawl command. # Better for intranet crawling. # Be sure to change MY.DOMAIN.NAME to your domain name. # Each non-comment, non-blank line contains a regular expression # prefixed by '+' or '-'. The first matching pattern in the file # determines whether a URL is included or ignored. If no pattern # matches, the URL is ignored. # skip file:, ftp:, & mailto: urls -^(file|ftp|mailto): # skip image and other suffixes we can't yet parse -\.(gif|GIF|jpg|JPG|ico|ICO|css|sit|eps|wmf|zip|ppt|mpg|xls|gz|rpm|tgz|mov|M OV|exe)$ # skip URLs containing certain characters as probable queries, etc. -[?*!@=] # accept hosts in MY.DOMAIN.NAME #+^http://([a-z0-9]*\.)*MY.DOMAIN.NAME/ +^http://([a-z0-9]*\.)*yahoo.com/ # skip everything else #-. but It can not work, and can not crawl the domain name (DOMAIN.NAME) inluding news.yahoo.com,game.yahoo.com,blog.yahoo.com why? -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira From nutch-devdfranklin@example.org Mon Jul 04 09:22:20 2005 Return-Path: <nutch-devdfranklin@example.org> Delivered-To: dfranklin@example.org Received: (qmail 35682 invoked from network); 4 Jul 2005 09:22:20 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 4 Jul 2005 09:22:20 -0000 Received: (qmail 29065 invoked by uid 500); 4 Jul 2005 09:22:15 -0000 Delivered-To: dfranklin@example.org Received: (qmail 28948 invoked by uid 500); 4 Jul 2005 09:22:14 -0000 Mailing-List: contact dfranklin@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:dfranklin@example.org> List-Unsubscribe: <mailto:dfranklin@example.org> List-Post: <mailto:dfranklin@example.org> List-Id: <nutch-dev.lucene.apache.org> Reply-To: dfranklin@example.org Delivered-To: mailing list dfranklin@example.org Received: (qmail 28898 invoked by uid 500); 4 Jul 2005 09:22:14 -0000 Delivered-To: dfranklin@example.org Received: (qmail 28861 invoked by uid 99); 4 Jul 2005 09:22:13 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO ajax.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Jul 2005 02:22:13 -0700 Received: from ajax.apache.org (ajax.apache.org [83.161.120.155]) by ajax.apache.org (Postfix) with ESMTP id B36AE15 for <dfranklin@example.org>; Mon, 4 Jul 2005 11:22:10 +0200 (CEST) Message-ID: <dfranklin@example.org> Date: Mon, 4 Jul 2005 11:22:10 +0200 (CEST) From: "Jerome Charron (JIRA)" <dfranklin@example.org> To: dfranklin@example.org Subject: [jira] Commented: (NUTCH-65) index-more plugin can't parse large set of modification-date In-Reply-To: <dfranklin@example.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/NUTCH-65?page=3Dcomments#action_= 12314991 ]=20 Jerome Charron commented on NUTCH-65: ------------------------------------- Thta's a VERY good reason. Thanks for info. > index-more plugin can't parse large set of modification-date > ------------------------------------------------------------- > > Key: NUTCH-65 > URL: http://issues.apache.org/jira/browse/NUTCH-65 > Project: Nutch > Type: Bug > Components: indexer > Environment: nutch 0.7, java 1.5, linux > Reporter: Lutisch=C3=A1n Ferenc > > I found a problem in MoreIndexingFilter.java. > When I indexing segments, I get large list of error messages: > can't parse errorenous date: Wed, 10 Sep 2003 11:59:14 or > can't parse errorenous date: Wed, 10 Sep 2003 11:59:14GMT > I modifiing source code (I don't make a 'patch'): > Original (lines 137-138): > DateFormat df =3D new SimpleDateFormat("EEE MMM dd HH:mm:ss yyyy zzz"); > Date d =3D df.parse(date); > New: > DateFormat df =3D new SimpleDateFormat("EEE, MMM dd HH:mm:ss yyyy", Local= e.US); > Date d =3D df.parse(date.substring(0,25)); > The modified code works fine. --=20 This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
{'source': 'Apache Software Foundation Public Mail Archives'}