From schneiderjacob@example.net Tue Jul 06 12:54:48 2010
Return-Path: <schneiderjacob@example.net>
Delivered-To: schneiderjacob@example.net
Received: (qmail 85415 invoked from network); 6 Jul 2010 12:54:47 -0000
Received: from unknown (HELO mail.apache.org) (83.161.120.155)
  by 83.161.120.155 with SMTP; 6 Jul 2010 12:54:47 -0000
Received: (qmail 25927 invoked by uid 500); 6 Jul 2010 12:54:47 -0000
Delivered-To: schneiderjacob@example.net
Received: (qmail 25779 invoked by uid 500); 6 Jul 2010 12:54:45 -0000
Mailing-List: contact schneiderjacob@example.net; run by ezmlm
Precedence: bulk
List-Help: <mailto:schneiderjacob@example.net>
List-Unsubscribe: <mailto:schneiderjacob@example.net>
List-Post: <mailto:schneiderjacob@example.net>
List-Id: <user.tika.apache.org>
Reply-To: schneiderjacob@example.net
Delivered-To: mailing list schneiderjacob@example.net
Received: (qmail 25771 invoked by uid 99); 6 Jul 2010 12:54:44 -0000
Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Jul 2010 12:54:44 +0000
X-ASF-Spam-Status: No, hits=0.0 required=10.0
	tests=FREEMAIL_FROM,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of schneiderjacob@example.net designates 83.161.120.155 as permitted sender)
Received: from [83.161.120.155] (HELO mail-iw0-f182.google.com) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Jul 2010 12:54:36 +0000
Received: by iwn35 with SMTP id 35so5127108iwn.13
        for <schneiderjacob@example.net>; Tue, 06 Jul 2010 05:54:15 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:received:mime-version:received:in-reply-to
         :references:from:date:message-id:subject:to:content-type;
        bh=WjqCjRneALdFrPmWE9sW9T/l2+spa9DHUo+hiBaq9Qc=;
        b=b8ECx8OVClZAkotKaHAXH5k8krCg6g1rfWIwk3+mn6x+Y+j9beRDyyTGXWHxgfsLJO
         4JriDD6Zq/MMSoT+tHtIzSVvT929li5Ts1TQRip2z1kJamabqwGznACfvWY0N2r1EiKf
         IYJgGzXHBdlHY/MdQJ5fMgL71b1JH8BqmtwkU=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type;
        b=Cifu5B2+B/eoJfydoiJ04CEAuAQZT+o76b7QgorAmkwZ0RxGoS0R7UEfsSDUFC8Bhd
         tXOa/Ku/WgF7iuvG/MVlKNUC6TTGQ0W54FuqhKSALjNgWvbHcaJCwWYGaOg7OBGl0OK5
         5EQf6pbqJ9Cu8yp1ZjHxi0q5Hrg0v5uORjrtg=
Received: by 83.161.120.155 with SMTP id cg3mr4586776ibb.51.1278420855145; 
	Tue, 06 Jul 2010 05:54:15 -0700 (PDT)
MIME-Version: 1.0
Received: by 83.161.120.155 with HTTP; Tue, 6 Jul 2010 05:53:54 -0700 (PDT)
In-Reply-To: <schneiderjacob@example.net>
References: <schneiderjacob@example.net> 
	<schneiderjacob@example.net> 
	<schneiderjacob@example.net>
From: Jukka Zitting <schneiderjacob@example.net>
Date: Tue, 6 Jul 2010 15:53:54 +0300
Message-ID: <schneiderjacob@example.net>
Subject: Re: Building Tika 8.0
To: schneiderjacob@example.net
Content-Type: text/plain; charset=ISO-8859-1
X-Virus-Checked: Checked by ClamAV on apache.org

Hi,

On Wed, Jun 30, 2010 at 6:14 PM, Jan <schneiderjacob@example.net> wrote:
> I indeed did a SVN update. After your clue that the tika-core seems to be
> out of date, I investigated a bit further and it looks like the Snapshot
> repository of Apache is not up to date. When building the Tika parsers it
> tries to fetch the tike-core jar from
> http://repository.apache.org/snapshots/org/apache/tika/tika-core/0.8-SNAPSHOT/
> and this one is not up to date.

Our CI build that deploys the snapshots was out of date. I updated the
build configuration and now new snapshots should again get
automatically deployed after each svn commit.

BR,

Jukka Zitting

From schneiderjacob@example.net Tue Jul 06 13:03:40 2010
Return-Path: <schneiderjacob@example.net>
Delivered-To: schneiderjacob@example.net
Received: (qmail 88128 invoked from network); 6 Jul 2010 13:03:40 -0000
Received: from unknown (HELO mail.apache.org) (83.161.120.155)
  by 83.161.120.155 with SMTP; 6 Jul 2010 13:03:40 -0000
Received: (qmail 40756 invoked by uid 500); 6 Jul 2010 13:03:40 -0000
Delivered-To: schneiderjacob@example.net
Received: (qmail 40608 invoked by uid 500); 6 Jul 2010 13:03:37 -0000
Mailing-List: contact schneiderjacob@example.net; run by ezmlm
Precedence: bulk
List-Help: <mailto:schneiderjacob@example.net>
List-Unsubscribe: <mailto:schneiderjacob@example.net>
List-Post: <mailto:schneiderjacob@example.net>
List-Id: <user.tika.apache.org>
Reply-To: schneiderjacob@example.net
Delivered-To: mailing list schneiderjacob@example.net
Received: (qmail 40600 invoked by uid 99); 6 Jul 2010 13:03:37 -0000
Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Jul 2010 13:03:37 +0000
X-ASF-Spam-Status: No, hits=0.0 required=10.0
	tests=FREEMAIL_FROM,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of schneiderjacob@example.net designates 83.161.120.155 as permitted sender)
Received: from [83.161.120.155] (HELO mail-gy0-f182.google.com) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Jul 2010 13:03:28 +0000
Received: by gyd8 with SMTP id 8so2018715gyd.13
        for <schneiderjacob@example.net>; Tue, 06 Jul 2010 06:02:07 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:received:mime-version:received:in-reply-to
         :references:from:date:message-id:subject:to:content-type;
        bh=lQR1+WsSgwnx1lZ+Xr34Zk9OWd/BGxpIMqdmdVuSLRE=;
        b=wMYVQUrICOXfBuj6oQXpMKCTjSPu/rgdy1X6Djmd9PLuYpyGMur2O78Ul9SwzDKgly
         w19vUmpLxuINotLwwPsZ4vYaCbQi1S1pSGoXhlf2U9ajPUzf3xSoX/ZqDlbB+e1gP3HD
         rPR/G9uVN3cLt01brsZc3ajn4ZNiB2y2QJPyU=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type;
        b=AQ8k9HrxdBC9Elbq2pGlWHdfSwT+Juo9PAkSMNxZRPg/DVhx5acTKWFlD4M8B1fDCN
         1am9QENyajWdm/QMukVKvtW6NlVfgerm4Ke875r5v9H9sgTX8fT8UpiINgqZC5/K4OeH
         wFH1Hg8tbOu9S3rkNvhunsFZYVvy3SIL22Cg4=
Received: by 83.161.120.155 with SMTP id e12mr590123muk.7.1278421324327; Tue, 06 
	Jul 2010 06:02:04 -0700 (PDT)
MIME-Version: 1.0
Received: by 83.161.120.155 with HTTP; Tue, 6 Jul 2010 06:01:42 -0700 (PDT)
In-Reply-To: <schneiderjacob@example.net>
References: <schneiderjacob@example.net>
From: Jukka Zitting <schneiderjacob@example.net>
Date: Tue, 6 Jul 2010 16:01:42 +0300
Message-ID: <schneiderjacob@example.net>
Subject: Re: How can I help Tika to choose the right extractor for my data?
To: schneiderjacob@example.net
Content-Type: text/plain; charset=ISO-8859-1
X-Virus-Checked: Checked by ClamAV on apache.org

Hi,

Sorry for the late response.

On Tue, Jun 29, 2010 at 2:53 AM, zabrane Mikael <schneiderjacob@example.net> wrote:
> I learned that's possible to help (or advice) Tika to choose the right
> extractor for a document if I have for example its MimeType.
> Am exactly in this case. For each document in my collection, I know its
> MimeType.
> How one can apply this idea guys (code snippet please)?

You'll want to pass the media type as a part of the input metadata you
pass to the parsing process, like this:

    Metadata metadata = new Metadata():
    metadata.set(Metadata.CONTENT_TYPE, knownType);

    Parser parser = new AutoDetectParser();
    parser.parse(..., metadata, ...);

> Finally, does someone know when Tika-0.8 will be released?

At current pace I expect it to be out sometime in this quarter.

BR,

Jukka Zitting

From schneiderjacob@example.net Tue Jul 06 18:28:58 2010
Return-Path: <schneiderjacob@example.net>
Delivered-To: schneiderjacob@example.net
Received: (qmail 90894 invoked from network); 6 Jul 2010 18:28:58 -0000
Received: from unknown (HELO mail.apache.org) (83.161.120.155)
  by 83.161.120.155 with SMTP; 6 Jul 2010 18:28:58 -0000
Received: (qmail 41716 invoked by uid 500); 6 Jul 2010 18:28:58 -0000
Delivered-To: schneiderjacob@example.net
Received: (qmail 41650 invoked by uid 500); 6 Jul 2010 18:28:57 -0000
Mailing-List: contact schneiderjacob@example.net; run by ezmlm
Precedence: bulk
List-Help: <mailto:schneiderjacob@example.net>
List-Unsubscribe: <mailto:schneiderjacob@example.net>
List-Post: <mailto:schneiderjacob@example.net>
List-Id: <user.tika.apache.org>
Reply-To: schneiderjacob@example.net
Delivered-To: mailing list schneiderjacob@example.net
Received: (qmail 41642 invoked by uid 99); 6 Jul 2010 18:28:57 -0000
Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Jul 2010 18:28:57 +0000
X-ASF-Spam-Status: No, hits=4.4 required=10.0
	tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of schneiderjacob@example.net designates 83.161.120.155 as permitted sender)
Received: from [83.161.120.155] (HELO mail-fx0-f54.google.com) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Jul 2010 18:28:49 +0000
Received: by fxm13 with SMTP id 13so5510593fxm.13
        for <schneiderjacob@example.net>; Tue, 06 Jul 2010 11:27:27 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:mime-version:received:received:in-reply-to
         :references:date:message-id:subject:from:to:content-type;
        bh=plsJ2jyFxtm9AYe7g1XhYt52JHs9znklBHUR+cknU4g=;
        b=JAC51jVZPm7xZhlZcZVILw3kJYUV10FiLV4kFU/OjHCs/E2C0q9DCBTWezpSDkB27m
         EfJ4PYnhoZxn61hp+p6hz0DtIit69+NcDFYdN0ZHkhQ85B6jgw8Z8+ZR9d8EumKJ349Y
         +Ujx+587rivBBKFxnGK6vyICSmz1K96Ql3q3Q=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=B+EA6Kcyn6v30xpIbdgAs4Fjq+a+RJIHn24mhEdE/by6d+dreezwhHwcVOV3tqb3yW
         oqU41VU1wawVQRq4O52ATc6JuRFw2FT2bC2F0ag3DXtmchvrhyBCle6FBwAWT7daLT5V
         eZe/nfGWRu60Vm3MEJPMkA7OCsWGOSTpqQkR8=
MIME-Version: 1.0
Received: by 83.161.120.155 with SMTP id 14mr288855hbm.177.1278440847344; Tue, 
	06 Jul 2010 11:27:27 -0700 (PDT)
Received: by 83.161.120.155 with HTTP; Tue, 6 Jul 2010 11:27:27 -0700 (PDT)
In-Reply-To: <schneiderjacob@example.net>
References: <schneiderjacob@example.net>
	<schneiderjacob@example.net>
Date: Tue, 6 Jul 2010 20:27:27 +0200
Message-ID: <schneiderjacob@example.net>
Subject: Re: How can I help Tika to choose the right extractor for my data?
From: zabrane Mikael <schneiderjacob@example.net>
To: schneiderjacob@example.net
Content-Type: multipart/alternative; boundary=001485f44d449284a9048abc34ae
X-Virus-Checked: Checked by ClamAV on apache.org

--001485f44d449284a9048abc34ae
Content-Type: text/plain; charset=ISO-8859-1

Thanks Jukka !

2010/7/6 Jukka Zitting <schneiderjacob@example.net>

> Hi,
>
> Sorry for the late response.
>
> On Tue, Jun 29, 2010 at 2:53 AM, zabrane Mikael <schneiderjacob@example.net>
> wrote:
> > I learned that's possible to help (or advice) Tika to choose the right
> > extractor for a document if I have for example its MimeType.
> > Am exactly in this case. For each document in my collection, I know its
> > MimeType.
> > How one can apply this idea guys (code snippet please)?
>
> You'll want to pass the media type as a part of the input metadata you
> pass to the parsing process, like this:
>
>    Metadata metadata = new Metadata():
>    metadata.set(Metadata.CONTENT_TYPE, knownType);
>
>    Parser parser = new AutoDetectParser();
>    parser.parse(..., metadata, ...);
>
> > Finally, does someone know when Tika-0.8 will be released?
>
> At current pace I expect it to be out sometime in this quarter.
>
> BR,
>
> Jukka Zitting
>



-- 
Regards
Zabrane

--001485f44d449284a9048abc34ae
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Thanks Jukka !<br><br><div class=3D"gmail_quote">2010/7/6 Jukka Zitting <sp=
an dir=3D"ltr">&lt;<a href=3D"mailto:schneiderjacob@example.net">jukka.zitting=
@gmail.com</a>&gt;</span><br><blockquote class=3D"gmail_quote" style=3D"mar=
gin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi,<br>
<br>
Sorry for the late response.<br>
<div class=3D"im"><br>
On Tue, Jun 29, 2010 at 2:53 AM, zabrane Mikael &lt;<a href=3D"mailto:zabra=
schneiderjacob@example.net">schneiderjacob@example.net</a>&gt; wrote:<br>
&gt; I learned that&#39;s possible to help (or advice) Tika to choose the r=
ight<br>
&gt; extractor for a document if I have for example its MimeType.<br>
&gt; Am exactly in this case. For each document in my collection, I know it=
s<br>
&gt; MimeType.<br>
&gt; How one can apply this idea guys (code snippet please)?<br>
<br>
</div>You&#39;ll want to pass the media type as a part of the input metadat=
a you<br>
pass to the parsing process, like this:<br>
<br>
 =A0 =A0Metadata metadata =3D new Metadata():<br>
 =A0 =A0metadata.set(Metadata.CONTENT_TYPE, knownType);<br>
<br>
 =A0 =A0Parser parser =3D new AutoDetectParser();<br>
 =A0 =A0parser.parse(..., metadata, ...);<br>
<div class=3D"im"><br>
&gt; Finally, does someone know when Tika-0.8 will be released?<br>
<br>
</div>At current pace I expect it to be out sometime in this quarter.<br>
<br>
BR,<br>
<font color=3D"#888888"><br>
Jukka Zitting<br>
</font></blockquote></div><br><br clear=3D"all"><br>-- <br>Regards<br>Zabra=
ne<br>

--001485f44d449284a9048abc34ae--

From schneiderjacob@example.net Sat Jul 10 03:33:48 2010
Return-Path: <schneiderjacob@example.net>
Delivered-To: schneiderjacob@example.net
Received: (qmail 1651 invoked from network); 10 Jul 2010 03:33:48 -0000
Received: from unknown (HELO mail.apache.org) (83.161.120.155)
  by 83.161.120.155 with SMTP; 10 Jul 2010 03:33:48 -0000
Received: (qmail 13166 invoked by uid 500); 10 Jul 2010 03:33:48 -0000
Delivered-To: schneiderjacob@example.net
Received: (qmail 13010 invoked by uid 500); 10 Jul 2010 03:33:45 -0000
Mailing-List: contact schneiderjacob@example.net; run by ezmlm
Precedence: bulk
List-Help: <mailto:schneiderjacob@example.net>
List-Unsubscribe: <mailto:schneiderjacob@example.net>
List-Post: <mailto:schneiderjacob@example.net>
List-Id: <user.tika.apache.org>
Reply-To: schneiderjacob@example.net
Delivered-To: mailing list schneiderjacob@example.net
Received: (qmail 13002 invoked by uid 99); 10 Jul 2010 03:33:44 -0000
Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 10 Jul 2010 03:33:44 +0000
X-ASF-Spam-Status: No, hits=0.0 required=10.0
	tests=FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of schneiderjacob@example.net designates 83.161.120.155 as permitted sender)
Received: from [83.161.120.155] (HELO mail-gw0-f54.google.com) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 10 Jul 2010 03:33:35 +0000
Received: by gwb15 with SMTP id 15so1760843gwb.13
        for <schneiderjacob@example.net>; Fri, 09 Jul 2010 20:32:14 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:received:received:from:content-type
         :content-transfer-encoding:subject:date:message-id:to:mime-version
         :x-mailer;
        bh=qHKGjOZxbh4FTTQpYTC8myuKw+6nWPDnfQvLi41It8U=;
        b=pfjBW1CEZBeai35ZwUtOAA9i3al3rM/oYMyopLZqsj8Q+KHqnVvo8tC5VcXH5nkENY
         2tPSTcC5ZJWW0jeTx8Lk2eEXuguNzY5QnfXeOnIdZt+CCJzr88iDQBdubPSIP/EFjoXs
         2t+U+Krjn17CbkI4KJpqm1bkTqqZ8o52tQJMA=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=from:content-type:content-transfer-encoding:subject:date:message-id
         :to:mime-version:x-mailer;
        b=rRV6oLC8RLFUJK+BB4B8by3zUv8l4qUktPH+oqVqTV9Mn3ScbtZzaGxt1aLUDbA3QU
         qO/z7P5PG/jhAP+IOeYUXFyXQa5SAUZWVpYLimbZhuv3SD7opbk9rUKRmKawoCWBFUk9
         5XegSPG4feJfUrcPKZsZ0pLGG/WECwEOopj74=
Received: by 83.161.120.155 with SMTP id u21mr2751984ybb.77.1278732732263;
        Fri, 09 Jul 2010 20:32:12 -0700 (PDT)
Received: from [83.161.120.155] (pool-71-113-170-209.frstil.dsl-w.verizon.net [83.161.120.155])
        by mx.google.com with ESMTPS id v6sm908529ybm.23.283.161.120.155.32.09
        (version=TLSv1/SSLv3 cipher=RC4-MD5);
        Fri, 09 Jul 2010 20:32:10 -0700 (PDT)
From: David Kovar <schneiderjacob@example.net>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Subject: Test suite for Tika?
Date: Fri, 9 Jul 2010 22:32:07 -0500
Message-Id: <schneiderjacob@example.net>
To: schneiderjacob@example.net
Mime-Version: 1.0 (Apple Message framework v1081)
X-Mailer: Apple Mail (2.1081)
X-Virus-Checked: Checked by ClamAV on apache.org

Good evening,

Is there an available set of documents that is used to validate Tika's =
performance? I am working on validating the performance of some =
ediscovery tools and such a test set would be very useful.

Thank you.

-David





From schneiderjacob@example.net Sat Jul 10 04:09:29 2010
Return-Path: <schneiderjacob@example.net>
Delivered-To: schneiderjacob@example.net
Received: (qmail 11749 invoked from network); 10 Jul 2010 04:09:29 -0000
Received: from unknown (HELO mail.apache.org) (83.161.120.155)
  by 83.161.120.155 with SMTP; 10 Jul 2010 04:09:29 -0000
Received: (qmail 26510 invoked by uid 500); 10 Jul 2010 04:09:29 -0000
Delivered-To: schneiderjacob@example.net
Received: (qmail 26360 invoked by uid 500); 10 Jul 2010 04:09:26 -0000
Mailing-List: contact schneiderjacob@example.net; run by ezmlm
Precedence: bulk
List-Help: <mailto:schneiderjacob@example.net>
List-Unsubscribe: <mailto:schneiderjacob@example.net>
List-Post: <mailto:schneiderjacob@example.net>
List-Id: <user.tika.apache.org>
Reply-To: schneiderjacob@example.net
Delivered-To: mailing list schneiderjacob@example.net
Received: (qmail 26352 invoked by uid 99); 10 Jul 2010 04:09:26 -0000
Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 10 Jul 2010 04:09:26 +0000
X-ASF-Spam-Status: No, hits=-0.1 required=10.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: local policy)
Received: from [83.161.120.155] (HELO mail.jpl.nasa.gov) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 10 Jul 2010 04:09:19 +0000
Received: from mail.jpl.nasa.gov (altvirehtstap02.jpl.nasa.gov [83.161.120.155])
	by smtp.jpl.nasa.gov (Switch-3.4.3/Switch-3.4.3) with ESMTP id o6A47uXD004826
	(using TLSv1/SSLv3 with cipher RC4-MD5 (128 bits) verified FAIL)
	for <schneiderjacob@example.net>; Fri, 9 Jul 2010 21:07:56 -0700
Received: from ALTPHYEMBEVSP20.RES.AD.JPL ([83.161.120.155]) by
 ALTVIREHTSTAP02.RES.AD.JPL ([83.161.120.155]) with mapi; Fri, 9 Jul 2010
 21:07:56 -0700
From: "Mattmann, Chris A (388J)" <schneiderjacob@example.net>
To: "schneiderjacob@example.net" <schneiderjacob@example.net>
Date: Fri, 9 Jul 2010 21:07:54 -0700
Subject: Re: Test suite for Tika?
Thread-Topic: Test suite for Tika?
Thread-Index: Acsf4LWqZ+vfDXvBQxGbbj2u+iVWLQABMLb8
Message-ID: <schneiderjacob@example.net>
In-Reply-To: <schneiderjacob@example.net>
Accept-Language: en-US
Content-Language: en
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: multipart/alternative;
	boundary="_000_C85D402A17FF2ChrisAMattmannjplnasagov_"
MIME-Version: 1.0
X-Source-IP: altvirehtstap02.jpl.nasa.gov [83.161.120.155]
X-Source-Sender: schneiderjacob@example.net
X-AUTH: Authorized
X-Virus-Checked: Checked by ClamAV on apache.org

--_000_C85D402A17FF2ChrisAMattmannjplnasagov_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi David,

The unit tests for the tika-parsers modules contains the test documents in =
the directory here:

http://svn.apache.org/repos/asf/tika/trunk/tika-parsers/src/test/resources/=
test-documents/

HTH,
Chris



On 7/9/10 8:32 PM, "David Kovar" <schneiderjacob@example.net> wrote:

Good evening,

Is there an available set of documents that is used to validate Tika's perf=
ormance? I am working on validating the performance of some ediscovery tool=
s and such a test set would be very useful.

Thank you.

-David







++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: schneiderjacob@example.net
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


--_000_C85D402A17FF2ChrisAMattmannjplnasagov_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<HTML>
<HEAD>
<TITLE>Re: Test suite for Tika?</TITLE>
</HEAD>
<BODY>
<FONT FACE=3D"Consolas, Courier New, Courier"><SPAN STYLE=3D'font-size:11pt=
'>Hi David,<BR>
<BR>
The unit tests for the tika-parsers modules contains the test documents in =
the directory here:<BR>
<BR>
<a href=3D"http://svn.apache.org/repos/asf/tika/trunk/tika-parsers/src/test=
/resources/test-documents/">http://svn.apache.org/repos/asf/tika/trunk/tika=
-parsers/src/test/resources/test-documents/</a><BR>
<BR>
HTH,<BR>
Chris<BR>
<BR>
<BR>
<BR>
On 7/9/10 8:32 PM, &quot;David Kovar&quot; &lt;<a href=3D"schneiderjacob@example.net"=
>schneiderjacob@example.net</a>&gt; wrote:<BR>
<BR>
</SPAN></FONT><BLOCKQUOTE><FONT FACE=3D"Consolas, Courier New, Courier"><SP=
AN STYLE=3D'font-size:11pt'>Good evening,<BR>
<BR>
Is there an available set of documents that is used to validate Tika's perf=
ormance? I am working on validating the performance of some ediscovery tool=
s and such a test set would be very useful.<BR>
<BR>
Thank you.<BR>
<BR>
-David<BR>
<BR>
<BR>
<BR>
<BR>
<BR>
</SPAN></FONT></BLOCKQUOTE><FONT FACE=3D"Consolas, Courier New, Courier"><S=
PAN STYLE=3D'font-size:11pt'><BR>
</SPAN><FONT SIZE=3D"2"><SPAN STYLE=3D'font-size:10pt'><BR>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++<BR>
Chris Mattmann, Ph.D.<BR>
Senior Computer Scientist<BR>
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA<BR>
Office: 171-266B, Mailstop: 171-246<BR>
Email: <FONT COLOR=3D"#0000FF"><U><a href=3D"schneiderjacob@example.net">C=
schneiderjacob@example.net</a><BR>
</U></FONT>WWW: &nbsp;&nbsp;<FONT COLOR=3D"#0000FF"><U><a href=3D"http://su=
nset.usc.edu/~mattmann/">http://sunset.usc.edu/~mattmann/</a><BR>
</U></FONT>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
++<BR>
Adjunct Assistant Professor, Computer Science Department <BR>
University of Southern California, Los Angeles, CA 90089 USA<BR>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++<BR>
</SPAN></FONT><SPAN STYLE=3D'font-size:11pt'><BR>
</SPAN></FONT>
</BODY>
</HTML>


--_000_C85D402A17FF2ChrisAMattmannjplnasagov_--

From schneiderjacob@example.net Sun Jul 11 14:31:25 2010
Return-Path: <schneiderjacob@example.net>
Delivered-To: schneiderjacob@example.net
Received: (qmail 68239 invoked from network); 11 Jul 2010 14:31:25 -0000
Received: from unknown (HELO mail.apache.org) (83.161.120.155)
  by 83.161.120.155 with SMTP; 11 Jul 2010 14:31:25 -0000
Received: (qmail 21575 invoked by uid 500); 11 Jul 2010 14:31:25 -0000
Delivered-To: schneiderjacob@example.net
Received: (qmail 21478 invoked by uid 500); 11 Jul 2010 14:31:24 -0000
Mailing-List: contact schneiderjacob@example.net; run by ezmlm
Precedence: bulk
List-Help: <mailto:schneiderjacob@example.net>
List-Unsubscribe: <mailto:schneiderjacob@example.net>
List-Post: <mailto:schneiderjacob@example.net>
List-Id: <user.tika.apache.org>
Reply-To: schneiderjacob@example.net
Delivered-To: mailing list schneiderjacob@example.net
Received: (qmail 21470 invoked by uid 99); 11 Jul 2010 14:31:23 -0000
Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 11 Jul 2010 14:31:23 +0000
X-ASF-Spam-Status: No, hits=2.2 required=10.0
	tests=FREEMAIL_FROM,HTML_MESSAGE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of schneiderjacob@example.net designates 83.161.120.155 as permitted sender)
Received: from [83.161.120.155] (HELO mail-iw0-f182.google.com) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 11 Jul 2010 14:31:16 +0000
Received: by iwn35 with SMTP id 35so4742141iwn.13
        for <schneiderjacob@example.net>; Sun, 11 Jul 2010 07:29:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:received:received:from:mime-version
         :content-type:subject:date:in-reply-to:to:references:message-id
         :x-mailer;
        bh=jYUnHQFrNISAInCrhPHR1YJLrCyh8vKb3KHbRXFgWQQ=;
        b=EPd+OyTz0N4OCb66LRigpGr9K0CXHHDsxkB5G1zq06rlz4a6V8R/Dfnru1notYVIdF
         ahezC1uxyuAc20M7xEQ0sAqsuGxQ/JYDDRayH45bSN8VK2EAeS9L8Vuk3/GvU5H65Hw7
         ehF5llaokTsiJu7+QQ9uZuz+pLHfBiv1pMTDg=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=from:mime-version:content-type:subject:date:in-reply-to:to
         :references:message-id:x-mailer;
        b=TL5pcBL5bIE5ZQip56CjjgB8kUqxwilw1eji2JN1da+JtuymNRMTDlAco0kBEg2s+Y
         P/JoVBY+fwP71S6OmYZ0rNx7OJ8ZLPOLlRqLwDc3VG7B3Jwxho7bQTG4brKJXT/2gCj4
         z+hWJKeLY4My5I0Q5OhVOQiS8DCD6D9Z6DeKA=
Received: by 83.161.120.155 with SMTP id c13mr13592103ibx.71.1278858595248;
        Sun, 11 Jul 2010 07:29:55 -0700 (PDT)
Received: from [83.161.120.155] (pool-71-113-170-209.frstil.dsl-w.verizon.net [83.161.120.155])
        by mx.google.com with ESMTPS id g31sm14655808ibh.4.283.161.120.155.29.52
        (version=TLSv1/SSLv3 cipher=RC4-MD5);
        Sun, 11 Jul 2010 07:29:52 -0700 (PDT)
From: David Kovar <schneiderjacob@example.net>
Mime-Version: 1.0 (Apple Message framework v1081)
Content-Type: multipart/alternative; boundary=Apple-Mail-7--96320058
Subject: Re: Test suite for Tika?
Date: Sun, 11 Jul 2010 09:29:50 -0500
In-Reply-To: <schneiderjacob@example.net>
To: schneiderjacob@example.net
References: <schneiderjacob@example.net>
Message-Id: <schneiderjacob@example.net>
X-Mailer: Apple Mail (2.1081)
X-Virus-Checked: Checked by ClamAV on apache.org


--Apple-Mail-7--96320058
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Chris,

Thank you very much.

-David



On Jul 9, 2010, at 11:07 PM, Mattmann, Chris A (388J) wrote:

> Hi David,
>=20
> The unit tests for the tika-parsers modules contains the test =
documents in the directory here:
>=20
> =
http://svn.apache.org/repos/asf/tika/trunk/tika-parsers/src/test/resources=
/test-documents/
>=20
> HTH,
> Chris
>=20
>=20
>=20
> On 7/9/10 8:32 PM, "David Kovar" <schneiderjacob@example.net> wrote:
>=20
> Good evening,
>=20
> Is there an available set of documents that is used to validate Tika's =
performance? I am working on validating the performance of some =
ediscovery tools and such a test set would be very useful.
>=20
> Thank you.
>=20
> -David
>=20
>=20
>=20
>=20
>=20
>=20
>=20
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: schneiderjacob@example.net
> WWW:   http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department=20
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>=20


--Apple-Mail-7--96320058
Content-Transfer-Encoding: 7bit
Content-Type: text/html;
	charset=us-ascii

<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Chris,<div><br></div><div>Thank you very much.</div><div><br></div><div>-David</div><div><div><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><br class="Apple-interchange-newline"></span><br class="Apple-interchange-newline">
</div>
<br><div><div>On Jul 9, 2010, at 11:07 PM, Mattmann, Chris A (388J) wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>
<font face="Consolas, Courier New, Courier"><span style="font-size:11pt">Hi David,<br>
<br>
The unit tests for the tika-parsers modules contains the test documents in the directory here:<br>
<br>
<a href="http://svn.apache.org/repos/asf/tika/trunk/tika-parsers/src/test/resources/test-documents/">http://svn.apache.org/repos/asf/tika/trunk/tika-parsers/src/test/resources/test-documents/</a><br>
<br>
HTH,<br>
Chris<br>
<br>
<br>
<br>
On 7/9/10 8:32 PM, "David Kovar" &lt;<a href="x-msg://schneiderjacob@example.net">schneiderjacob@example.net</a>&gt; wrote:<br>
<br>
</span></font><blockquote><font face="Consolas, Courier New, Courier"><span style="font-size:11pt">Good evening,<br>
<br>
Is there an available set of documents that is used to validate Tika's performance? I am working on validating the performance of some ediscovery tools and such a test set would be very useful.<br>
<br>
Thank you.<br>
<br>
-David<br>
<br>
<br>
<br>
<br>
<br>
</span></font></blockquote><font face="Consolas, Courier New, Courier"><span style="font-size:11pt"><br>
</span><font size="2"><span style="font-size:10pt"><br>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++<br>
Chris Mattmann, Ph.D.<br>
Senior Computer Scientist<br>
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA<br>
Office: 171-266B, Mailstop: 171-246<br>
Email: <font color="#0000FF"><u><a href="x-msg://schneiderjacob@example.net">schneiderjacob@example.net</a><br>
</u></font>WWW: &nbsp;&nbsp;<font color="#0000FF"><u><a href="http://sunset.usc.edu/~mattmann/">http://sunset.usc.edu/~mattmann/</a><br>
</u></font>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++<br>
Adjunct Assistant Professor, Computer Science Department <br>
University of Southern California, Los Angeles, CA 90089 USA<br>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++<br>
</span></font><span style="font-size:11pt"><br>
</span></font>
</div>


</blockquote></div><br></div></body></html>
--Apple-Mail-7--96320058--

From schneiderjacob@example.net Mon Jul 12 01:11:19 2010
Return-Path: <schneiderjacob@example.net>
Delivered-To: schneiderjacob@example.net
Received: (qmail 36785 invoked from network); 12 Jul 2010 01:11:19 -0000
Received: from unknown (HELO mail.apache.org) (83.161.120.155)
  by 83.161.120.155 with SMTP; 12 Jul 2010 01:11:19 -0000
Received: (qmail 32038 invoked by uid 500); 12 Jul 2010 01:11:19 -0000
Delivered-To: schneiderjacob@example.net
Received: (qmail 32000 invoked by uid 500); 12 Jul 2010 01:11:18 -0000
Mailing-List: contact schneiderjacob@example.net; run by ezmlm
Precedence: bulk
List-Help: <mailto:schneiderjacob@example.net>
List-Unsubscribe: <mailto:schneiderjacob@example.net>
List-Post: <mailto:schneiderjacob@example.net>
List-Id: <user.tika.apache.org>
Reply-To: schneiderjacob@example.net
Delivered-To: mailing list schneiderjacob@example.net
Received: (qmail 31992 invoked by uid 99); 12 Jul 2010 01:11:18 -0000
Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Jul 2010 01:11:18 +0000
X-ASF-Spam-Status: No, hits=2.2 required=10.0
	tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of schneiderjacob@example.net designates 83.161.120.155 as permitted sender)
Received: from [83.161.120.155] (HELO mail-qy0-f182.google.com) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Jul 2010 01:11:10 +0000
Received: by qyk7 with SMTP id 7so5345842qyk.13
        for <schneiderjacob@example.net>; Sun, 11 Jul 2010 18:09:48 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:mime-version:received:sender:received:date
         :x-google-sender-auth:message-id:subject:from:to:content-type;
        bh=lV8TRMmR3JbFDnAQOS5Oh0ClQHdlVpdU3qTI0YT223A=;
        b=oxid3Q3DSdMnOnVY2GErIkL15mPIFcUL+tB69X1R5cV0Xo5QCeCm7n1b+SQcoXmqVP
         ZhU+reMnGTa0gb49r/oF41ovQlej/arOUlgIy00tx5fP/8RzZhMU7Gmbcmn8DM9cDb8Y
         pZ9UNjY2Rs3kPOpfgHXSReIzO7J7RvCCQC5Nk=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:date:x-google-sender-auth:message-id:subject
         :from:to:content-type;
        b=Tj+IJyFqNHzt0rlzrv+BWPYqiOS4RAHAdFvhujgNuXQsDnBeUYpX3/WRN2gNnL6j7g
         8SdA6LYRKoUDrQaFIJae7IjbHj3r8WdLdo7zXM0QL0FhwbRlDu9L8hCoxuWFitgnZrF6
         fmwC7oO3Lx+gnDZtpOsYSTP5jCNkeBF5oWAPQ=
MIME-Version: 1.0
Received: by 83.161.120.155 with SMTP id u20mr1666012qal.101.1278896988402; Sun, 
	11 Jul 2010 18:09:48 -0700 (PDT)
Sender: schneiderjacob@example.net
Received: by 83.161.120.155 with HTTP; Sun, 11 Jul 2010 18:09:48 -0700 (PDT)
Date: Sun, 11 Jul 2010 20:09:48 -0500
X-Google-Sender-Auth: DTn94bJl8decw3NQlnH7_mmtJes
Message-ID: <schneiderjacob@example.net>
Subject: I want to capture metadata from individual files in a package
From: Paul Jakubik <schneiderjacob@example.net>
To: schneiderjacob@example.net
Content-Type: multipart/alternative; boundary=00c09f899274b2b6f7048b2668ce
X-Virus-Checked: Checked by ClamAV on apache.org

--00c09f899274b2b6f7048b2668ce
Content-Type: text/plain; charset=UTF-8

Hi,

I want to be able to parse zip, tar.gz, etc. files and extract metadata from
each file in the package. When I looked through the code, it looks like the
package parser creates a separate metadata object for each file in the
package, and I don't see a way to get to that object.

- Are there any plans for adding the ability to extract metadata from each
file in the package?

Here are the first two ways I thought of that this could be implemented:
- Add metadata to the context, and add a clear method so the user can clear
the metadata after each file in the package is parsed (in the ContentHandler
when a "div" element is closed).
- Write all metadata to the header section of the generated XHTML for each
document.

Will there be a way to get this metadata anytime soon?

Paul

--00c09f899274b2b6f7048b2668ce
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi,<div><br></div><div>I want to be able to parse zip, tar.gz, etc. files a=
nd extract metadata from each file in the package. When I looked through th=
e code, it looks like the package parser creates a separate metadata object=
 for each file in the package, and I don&#39;t see a way to get to that obj=
ect.</div>
<div><br></div><div>- Are there any plans for adding the ability to extract=
 metadata from each file in the package?</div><div><br></div><div>Here are =
the first two ways I thought of that this could be implemented:</div><div>
- Add metadata to the context, and add a clear method so the user can clear=
 the metadata after each file in the package is parsed (in the ContentHandl=
er when a &quot;div&quot; element is closed).</div><div>- Write all metadat=
a to the header section of the generated XHTML for each document.</div>
<div><br></div><div>Will there be a way to get this metadata anytime soon?<=
/div><div><br></div><div>Paul</div>

--00c09f899274b2b6f7048b2668ce--

From schneiderjacob@example.net Mon Jul 12 02:36:08 2010
Return-Path: <schneiderjacob@example.net>
Delivered-To: schneiderjacob@example.net
Received: (qmail 46854 invoked from network); 12 Jul 2010 02:36:08 -0000
Received: from unknown (HELO mail.apache.org) (83.161.120.155)
  by 83.161.120.155 with SMTP; 12 Jul 2010 02:36:08 -0000
Received: (qmail 46716 invoked by uid 500); 12 Jul 2010 02:36:08 -0000
Delivered-To: schneiderjacob@example.net
Received: (qmail 46644 invoked by uid 500); 12 Jul 2010 02:36:06 -0000
Mailing-List: contact schneiderjacob@example.net; run by ezmlm
Precedence: bulk
List-Help: <mailto:schneiderjacob@example.net>
List-Unsubscribe: <mailto:schneiderjacob@example.net>
List-Post: <mailto:schneiderjacob@example.net>
List-Id: <user.tika.apache.org>
Reply-To: schneiderjacob@example.net
Delivered-To: mailing list schneiderjacob@example.net
Received: (qmail 46636 invoked by uid 99); 12 Jul 2010 02:36:06 -0000
Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Jul 2010 02:36:06 +0000
X-ASF-Spam-Status: No, hits=-0.0 required=10.0
	tests=SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of schneiderjacob@example.net designates 83.161.120.155 as permitted sender)
Received: from [83.161.120.155] (HELO mail-01.cse.ucsc.edu) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Jul 2010 02:35:58 +0000
Received: from [83.161.120.155] (c-67-169-32-112.hsd1.ca.comcast.net [83.161.120.155])
	by mail-01.cse.ucsc.edu (Postfix) with ESMTP id EE7FC10080BB
	for <schneiderjacob@example.net>; Sun, 11 Jul 2010 19:35:06 -0700 (PDT)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1081)
Subject: Re: I want to capture metadata from individual files in a package
From: Jonathan Koren <schneiderjacob@example.net>
In-Reply-To: <schneiderjacob@example.net>
Date: Sun, 11 Jul 2010 19:35:06 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <schneiderjacob@example.net>
References: <schneiderjacob@example.net>
To: schneiderjacob@example.net
X-Mailer: Apple Mail (2.1081)
X-Virus-Checked: Checked by ClamAV on apache.org

On Jul 11, 2010, at 6:09 PM, Paul Jakubik wrote:
> Hi,
>=20
> I want to be able to parse zip, tar.gz, etc. files and extract =
metadata from each file in the package. When I looked through the code, =
it looks like the package parser creates a separate metadata object for =
each file in the package, and I don't see a way to get to that object.
>=20
> - Are there any plans for adding the ability to extract metadata from =
each file in the package?
>=20
> Here are the first two ways I thought of that this could be =
implemented:
> - Add metadata to the context, and add a clear method so the user can =
clear the metadata after each file in the package is parsed (in the =
ContentHandler when a "div" element is closed).
> - Write all metadata to the header section of the generated XHTML for =
each document.
>=20
> Will there be a way to get this metadata anytime soon?


I asked about this same thing almost exactly a  year ago.
=
http://mail-archives.apache.org/mod_mbox/lucene-tika-dev/200906.mbox/%3c39=
schneiderjacob@example.net%3e

and got unceremoniously shot down.
=
http://mail-archives.apache.org/mod_mbox/lucene-tika-dev/200907.mbox/%3c51=
schneiderjacob@example.net%3e

	That would confuse the distinction between metadata and normal
	document text, especially when just the character stream is =
extracted.
	If you need access to the entry metadata, it would probably be =
better
	to expose it as attributes of the package-entry div.

Still really really want this feature. =20

I dont like the idea of writing all the metadata in the HEAD, because =
now you don't know which file had which metadata.  I guess you could =
always make the ID attribute of XHTML's META tag correspond to the =
filename of the contained file, but it doesn't feels kind of sloppy to =
do that.

Mostly, I think the problem comes from trying to shove everything into =
an HTML instead of just fully embracing XML isn't necessarily the best =
choice, especially for package files.  If XML was used, there's no =
reason why you couldn't have something that looked like:
	<FILE>
		<META key=3D"" value=3D"" />
		<CONTENT>
			<FILE>
				<META key=3D"" value=3D"" />
				<CONTENT>
				</CONTENT>
			</FILE>
			<FILE>
				<META key=3D"" value=3D"" />
				<CONTENT>
				</CONTENT>
			</FILE>
	</FILE>

But the XHTML-vs-XML ship has sailed, so there's no point in =
re-litigating that.  Perhaps it's something to consider for version 2.0.

An alternative way of handling this would be to create a nonrecursive =
version of AutoDetectParser.  That way, when the parser returned the =
metadata on the package, a metadata key could be set like, =
isPackage=3DTRUE, and then the user could get an iterator to each =
contained file contained package, and then manually call =
AutoDetectParserNonRecursive on each of the contained files, thus =
getting the metadata as needed.

--
Jonathan Koren
schneiderjacob@example.net
http://www.soe.ucsc.edu/~jonathan/



From schneiderjacob@example.net Mon Jul 19 13:27:38 2010
Return-Path: <schneiderjacob@example.net>
Delivered-To: schneiderjacob@example.net
Received: (qmail 51110 invoked from network); 19 Jul 2010 13:27:38 -0000
Received: from unknown (HELO mail.apache.org) (83.161.120.155)
  by 83.161.120.155 with SMTP; 19 Jul 2010 13:27:38 -0000
Received: (qmail 31458 invoked by uid 500); 19 Jul 2010 13:27:38 -0000
Delivered-To: schneiderjacob@example.net
Received: (qmail 31308 invoked by uid 500); 19 Jul 2010 13:27:36 -0000
Mailing-List: contact schneiderjacob@example.net; run by ezmlm
Precedence: bulk
List-Help: <mailto:schneiderjacob@example.net>
List-Unsubscribe: <mailto:schneiderjacob@example.net>
List-Post: <mailto:schneiderjacob@example.net>
List-Id: <user.tika.apache.org>
Reply-To: schneiderjacob@example.net
Delivered-To: mailing list schneiderjacob@example.net
Received: (qmail 31300 invoked by uid 99); 19 Jul 2010 13:27:35 -0000
Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Jul 2010 13:27:35 +0000
X-ASF-Spam-Status: No, hits=2.2 required=10.0
	tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of schneiderjacob@example.net designates 83.161.120.155 as permitted sender)
Received: from [83.161.120.155] (HELO mail-fx0-f54.google.com) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Jul 2010 13:27:27 +0000
Received: by fxm13 with SMTP id 13so2368728fxm.13
        for <schneiderjacob@example.net>; Mon, 19 Jul 2010 06:27:06 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:mime-version:received:sender:received:date
         :x-google-sender-auth:message-id:subject:from:to:content-type;
        bh=ArTURUSSuyH40Nwjk2KlfbGLcu0VdBGiAqCUr+lKjHo=;
        b=uyCz4pqOMlXqt1ssLf1GZCSaQiP6PgE4eIrQvF94RhRylZpcGX+LJS6g0s31jMPtVQ
         9iUfi+PxVT6bECNyz4abgAueYFBmtFRDEckfafKckbhscrNrJZ5Q8bfUqaclqCEiJHKf
         nhUGq7375AgGN75113MZfQgQnE1d2Ume0HfOA=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:date:x-google-sender-auth:message-id:subject
         :from:to:content-type;
        b=YUTGeSHf8cnd9gb5b6UIilI6MCl8qDyYMZHCFj1JLNMUWmyITgCiAS2TeS1595mRv3
         5q+qkhiPD9aStPg93MlKGGNO4seD3Fa76ySytS0By6wueBOSXbD9NoZYD5ZMeOJCdXXN
         d7Q4TkbQB8M+lnoUzy0KXylm53iqN3mgGz0fo=
MIME-Version: 1.0
Received: by 83.161.120.155 with SMTP id g12mr3722696fap.21.1279546026628; 
	Mon, 19 Jul 2010 06:27:06 -0700 (PDT)
Sender: schneiderjacob@example.net
Received: by 83.161.120.155 with HTTP; Mon, 19 Jul 2010 06:27:06 -0700 (PDT)
Date: Mon, 19 Jul 2010 16:27:06 +0300
X-Google-Sender-Auth: QsEMy5Alw212uEPaowLOpPpRR-Y
Message-ID: <schneiderjacob@example.net>
Subject: Problem with Tika configuration
From: Sergiy Karpenko <schneiderjacob@example.net>
To: schneiderjacob@example.net
Content-Type: multipart/alternative; boundary=001636c5ace5641955048bbd8611
X-Virus-Checked: Checked by ClamAV on apache.org

--001636c5ace5641955048bbd8611
Content-Type: text/plain; charset=ISO-8859-1

Hello, friends

I want configure tika to use only PDFParser

So I make  tika-config.xml with exact content:

<parser name="parse-pdf" class="org.apache.tika.parser.pdf.PDFParser">
  <mime>application/pdf</mime>
</parser>

And I have test

      File file = getResourceAsFile("/test-documents/testPDF.pdf");
      TikaConfig myTC = new
TikaConfig(getResourceAsFile("/test-documents/tika-config.xml"));
      String s1 = ParseUtils.getStringContent(file, myTC);

It fails on last line
java.lang.NullPointerException
    at
org.apache.tika.utils.ParseUtils.getStringContent(ParseUtils.java:111)
    at
org.apache.tika.utils.ParseUtils.getStringContent(ParseUtils.java:170)
    at
org.apache.tika.utils.ParseUtils.getStringContent(ParseUtils.java:188)
    at org.apache.tika.TestParsers.testOwnPDFParser(TestParsers.java:60)

Debug shows that tika-config.xml contain incorrect configuration

Next one works fine:
<blabla>
<parser name="parse-pdf" class="org.apache.tika.parser.pdf.PDFParser">
  <mime>application/pdf</mime>
</parser>
</blabla>

Is there any documentation about Tika configuration, or at least a link to
correct and well formed tika-config.xml?

Thanks

--001636c5ace5641955048bbd8611
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hello, friends<br><br>I want configure tika to use only PDFParser <br><br>S=
o I make=A0 tika-config.xml with exact content:<br><br>&lt;parser name=3D&q=
uot;parse-pdf&quot; class=3D&quot;org.apache.tika.parser.pdf.PDFParser&quot=
;&gt;<br>
=A0 &lt;mime&gt;application/pdf&lt;/mime&gt;<br>&lt;/parser&gt;<br><br>And =
I have test<br><br>=A0=A0=A0=A0=A0 File file =3D getResourceAsFile(&quot;/t=
est-documents/testPDF.pdf&quot;);<br>=A0=A0=A0=A0=A0 TikaConfig myTC =3D ne=
w TikaConfig(getResourceAsFile(&quot;/test-documents/tika-config.xml&quot;)=
);<br>
=A0=A0=A0=A0=A0 String s1 =3D ParseUtils.getStringContent(file, myTC);<br><=
br>It fails on last line<br>java.lang.NullPointerException<br>=A0=A0=A0 at =
org.apache.tika.utils.ParseUtils.getStringContent(ParseUtils.java:111)<br>=
=schneiderjacob@example.net(ParseUtils.j=
ava:170)<br>
=schneiderjacob@example.net(ParseUtils.j=
ava:188)<br>=schneiderjacob@example.net(TestP=
arsers.java:60)<br><br>Debug shows that tika-config.xml contain incorrect c=
onfiguration<br>
<br>Next one works fine:<br>&lt;blabla&gt;<br>&lt;parser name=3D&quot;parse=
-pdf&quot; class=3D&quot;org.apache.tika.parser.pdf.PDFParser&quot;&gt;<br>=
=A0 &lt;mime&gt;application/pdf&lt;/mime&gt;<br>&lt;/parser&gt;<br>&lt;/bla=
bla&gt;<br>
<br>Is there any documentation about Tika configuration, or at least a link=
 to correct and well formed tika-config.xml?<br><br>Thanks<br><br><br>

--001636c5ace5641955048bbd8611--

From schneiderjacob@example.net Wed Jul 21 05:46:11 2010
Return-Path: <schneiderjacob@example.net>
Delivered-To: schneiderjacob@example.net
Received: (qmail 34929 invoked from network); 21 Jul 2010 05:46:11 -0000
Received: from unknown (HELO mail.apache.org) (83.161.120.155)
  by 83.161.120.155 with SMTP; 21 Jul 2010 05:46:11 -0000
Received: (qmail 74271 invoked by uid 500); 21 Jul 2010 05:46:11 -0000
Delivered-To: schneiderjacob@example.net
Received: (qmail 74121 invoked by uid 500); 21 Jul 2010 05:46:08 -0000
Mailing-List: contact schneiderjacob@example.net; run by ezmlm
Precedence: bulk
List-Help: <mailto:schneiderjacob@example.net>
List-Unsubscribe: <mailto:schneiderjacob@example.net>
List-Post: <mailto:schneiderjacob@example.net>
List-Id: <user.tika.apache.org>
Reply-To: schneiderjacob@example.net
Delivered-To: mailing list schneiderjacob@example.net
Received: (qmail 74113 invoked by uid 99); 21 Jul 2010 05:46:07 -0000
Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Jul 2010 05:46:07 +0000
X-ASF-Spam-Status: No, hits=0.0 required=10.0
	tests=FREEMAIL_FROM,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of schneiderjacob@example.net designates 83.161.120.155 as permitted sender)
Received: from [83.161.120.155] (HELO mail-iw0-f182.google.com) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Jul 2010 05:45:59 +0000
Received: by iwn35 with SMTP id 35so8215943iwn.13
        for <schneiderjacob@example.net>; Tue, 20 Jul 2010 22:44:38 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:received:mime-version:received:in-reply-to
         :references:from:date:message-id:subject:to:content-type
         :content-transfer-encoding;
        bh=eattXcpfb9Ial8bM3OC1VJLZd36zU09Hf4HCUU8171s=;
        b=L2vUjBcFhD668oCuWiUMg7f/MkMuV4vDpRkrTS2wZYGfaJ0pgoqC5wTZFpPPGC8ev/
         D2jeVgVQk1aJ2doheRkhF/0ggShrsm0w3Qo/3TQ3uolUFmRzfY5gxpntZKNMN+Yh5oOW
         1EVRHtFtPFde+Gd2OzygEGDJKblalDNgSyVYg=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=OmzACfV3o43LCH4Xz+MA1lg5ULv3z1tH8RTt4HlCZkoGQUYDmJMnh7eG3ZO90Cdy7V
         mMxxMkOSSfIoFM+3nb7iGWdkLgBgHlVWQa0G5QOH5pXolU2y3soeqsym0/xPVi3kZFY9
         tn46HcWf0pMwwm5ZbGpjx/z5ULSu9Kb+bVn1A=
Received: by 83.161.120.155 with SMTP id h13mr9045296ibv.1.1279691078735; Tue, 
	20 Jul 2010 22:44:38 -0700 (PDT)
MIME-Version: 1.0
Received: by 83.161.120.155 with HTTP; Tue, 20 Jul 2010 22:44:18 -0700 (PDT)
In-Reply-To: <schneiderjacob@example.net>
References: <schneiderjacob@example.net>
From: Jukka Zitting <schneiderjacob@example.net>
Date: Wed, 21 Jul 2010 08:44:18 +0300
Message-ID: <schneiderjacob@example.net>
Subject: Re: Problem with Tika configuration
To: schneiderjacob@example.net
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Virus-Checked: Checked by ClamAV on apache.org

Hi,

On Mon, Jul 19, 2010 at 4:27 PM, Sergiy Karpenko
<schneiderjacob@example.net> wrote:
> I want configure tika to use only PDFParser

The easiest way to achieve this is to directly use the PDFParser class
instead of working through the configuration.

> =A0=A0=A0=A0=A0 File file =3D getResourceAsFile("/test-documents/testPDF.=
pdf");
> =A0=A0=A0=A0=A0 TikaConfig myTC =3D new
> TikaConfig(getResourceAsFile("/test-documents/tika-config.xml"));
> =A0=A0=A0=A0=A0 String s1 =3D ParseUtils.getStringContent(file, myTC);

Use something like this instead:

    Parser parser =3D new PDFParser();
    ContentHandler handler =3D new BodyContentHandler();
    Metadata metadata =3D new Metadata();
    ParseContext context =3D new ParseContext();

    InputStream stream =3D TikaInputStream.get(new File("document.pdf"));
    try {
        parser.parse(stream, handler, metadata, context);
    } finally {
        stream.close();
    }

    String content =3D handler.toString();

BR,

Jukka Zitting

From schneiderjacob@example.net Thu Jul 22 15:24:00 2010
Return-Path: <schneiderjacob@example.net>
Delivered-To: schneiderjacob@example.net
Received: (qmail 30320 invoked from network); 22 Jul 2010 15:24:00 -0000
Received: from unknown (HELO mail.apache.org) (83.161.120.155)
  by 83.161.120.155 with SMTP; 22 Jul 2010 15:24:00 -0000
Received: (qmail 83922 invoked by uid 500); 22 Jul 2010 15:24:00 -0000
Delivered-To: schneiderjacob@example.net
Received: (qmail 83829 invoked by uid 500); 22 Jul 2010 15:23:59 -0000
Mailing-List: contact schneiderjacob@example.net; run by ezmlm
Precedence: bulk
List-Help: <mailto:schneiderjacob@example.net>
List-Unsubscribe: <mailto:schneiderjacob@example.net>
List-Post: <mailto:schneiderjacob@example.net>
List-Id: <user.tika.apache.org>
Reply-To: schneiderjacob@example.net
Delivered-To: mailing list schneiderjacob@example.net
Received: (qmail 83820 invoked by uid 99); 22 Jul 2010 15:23:58 -0000
Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 22 Jul 2010 15:23:58 +0000
X-ASF-Spam-Status: No, hits=-2.3 required=10.0
	tests=RCVD_IN_DNSWL_MED,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: local policy)
Received: from [83.161.120.155] (HELO ostiiron1.osti.gov) (83.161.120.155)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 22 Jul 2010 15:23:52 +0000
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Ah4FAIn+R0ysEIFu/2dsb2JhbACfHYFABsFkhTYE
X-IronPort-AV: E=Sophos;i="4.55,243,1278270000"; 
   d="scan'208";a="3654567"
Received: from ostiex01.osti.gov ([83.161.120.155])
  by ostiiron1.osti.gov with ESMTP; 22 Jul 2010 10:20:40 +0500
Received: from OSTIEX01.osti.gov ([888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4]) by OSTIEX01.osti.gov ([888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4]) with
 mapi; Thu, 22 Jul 2010 11:22:31 -0400
From: "Ensor, Neal" <schneiderjacob@example.net>
To: "schneiderjacob@example.net" <schneiderjacob@example.net>
Date: Thu, 22 Jul 2010 11:22:30 -0400
Subject: PDF and MS Word Metadata question: page counts
Thread-Topic: PDF and MS Word Metadata question: page counts
Thread-Index: AQHLKbGza2VbKXmYsEaE8uAS2WZmYA==
Message-ID: <schneiderjacob@example.net>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Virus-Checked: Checked by ClamAV on apache.org

Just a curiousity:  I'm currently using tika 0.7 for some simple text extra=
ction, and noticed that for some reason I can't access page counts for eith=
er PDF or Word documents.

I know the information is available via underlying library calls (e.g., PDF=
 box) and appears it should be available via extended information in the MS=
 Office parser, but I don't see it in the metadata of any documents I tried=
.  My question is, was there some reason why page counts are omitted?  I ha=
cked my local copy of PDFParser to provide such via the PDDocument.getNumbe=
rOfPages() call,  but was wondering if I missed something somewhere or ther=
e might be a reason to not provide such information.  For the Word document=
s, I guess since it should be provided, guess I'm out of luck there, but fo=
r my purposes, I'd like at least parsed PDF metadata to provide that inform=
ation if possible...  Thanks!

Neal Ensor
schneiderjacob@example.net=


{'source': 'Apache Software Foundation Public Mail Archives'}