From pdfbox-dev-returntranthomas@example.com Sun Mar 01 01:54:54 2009 Return-Path: <pdfbox-dev-returntranthomas@example.com> Delivered-To: tranthomas@example.com Received: (qmail 75724 invoked from network); 1 Mar 2009 01:54:54 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Mar 2009 01:54:54 -0000 Received: (qmail 84362 invoked by uid 500); 1 Mar 2009 01:54:54 -0000 Delivered-To: tranthomas@example.com Received: (qmail 84344 invoked by uid 500); 1 Mar 2009 01:54:54 -0000 Mailing-List: contact tranthomas@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:tranthomas@example.com> List-Unsubscribe: <mailto:tranthomas@example.com> List-Post: <mailto:tranthomas@example.com> List-Id: <pdfbox-dev.incubator.apache.org> Reply-To: tranthomas@example.com Delivered-To: mailing list tranthomas@example.com Received: (qmail 84331 invoked by uid 99); 1 Mar 2009 01:54:54 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 28 Feb 2009 17:54:54 -0800 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of tranthomas@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO qw-out-1920.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Mar 2009 01:54:46 +0000 Received: by qw-out-1920.google.com with SMTP id 14so1595929qwa.54 for <tranthomas@example.com>; Sat, 28 Feb 2009 17:54:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=RaNVdft3d1na4gtCNK0vavyUrFGjmbWWQ8vbPwlEov0=; b=Lfbx8Hnq3MoOnHzMwSp7LH6ehp+JxE0SdFi7x45hb5xPU0bxB+iwnnDtRAB32Ysf+9 x1JdYLqX7C5PM/FsnD108WP8L8dsD18PfKfT2akqDPJshpcWELbtgd3r1J0m9NaVjY87 IWjhPMWLdfUFo2BrQUleXJynAdSlfOvVGEq7I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=j9sSu8Lwb+Of/h9XBFBFRmnxCehixr81enNJx6xy9scTjgnXwYX4csVlAJVh9UBeSI if4BbDOX7ExEIzemc4yaDOcVxLaVQapA8cZnEOjG36DE+P4JmGgY4LhtvOsVIfcjTjgK TohiqytDa48eFKmR+h4DtaOFS+WfEifeX17ZE= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id l20mr2566210qcp.60.1235872465471; Sat, 28 Feb 2009 17:54:25 -0800 (PST) In-Reply-To: <tranthomas@example.com> References: <tranthomas@example.com> <tranthomas@example.com> Date: Sat, 28 Feb 2009 20:54:25 -0500 Message-ID: <tranthomas@example.com> Subject: Re: Where's the pdfbox/trunk/test/input/rendering folder? From: Daniel Wilson <tranthomas@example.com> To: tranthomas@example.com Content-Type: multipart/alternative; boundary=0016364eca9e4ac819046404fbf4 X-Virus-Checked: Checked by ClamAV on apache.org --0016364eca9e4ac819046404fbf4 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Will do! thanks. dwilson On Sat, Feb 28, 2009 at 11:41 AM, Jeremias Maerki <tranthomas@example.com>wrote: > Hmm, looking into the SVN logs it doesn't look like that part made it > into the ASF tranthomas@example.com. I think I can explain part of it: > > https://issues.apache.org/jira/browse/PDFBOX-2 was the original import. > The file was made in April but you added the "rendering" directory in > May in the SourceForge CVS repository: > > http://pdfbox.cvs.sourceforge.net/viewvc/pdfbox/pdfbox/test/input/rendering/ > > Jukka then moved all changes since the dump from CVS over here: > http://svn.apache.org/viewvc?view=rev&revision=682654 > > This included your TestPDFToImage but not the reference files. Maybe > they were not automatically picked up because they were binary. So > probably just an oversight as you suspect. Anyway, I would suggest you > just add them yourself. The test is certainly a useful one. > > On 28.02.2009 03:06:47 Daniel Wilson wrote: > > As of last May, our testimage target looked for files in that folder to > > render as PNG files for the automation of image-production testing. And > ... > > I just verified it still looks for that folder -- see line 279 of > > > http://svn.apache.org/repos/asf/incubator/pdfbox/trunk/src/test/java/org/apache/pdfbox/util/TestPDFToImage.java > > > > Is this just an oversight? > > > > Thanks. > > > > dwilson > > > > > Jeremias Maerki > > --0016364eca9e4ac819046404fbf4-- From pdfbox-dev-returntranthomas@example.com Sun Mar 01 18:31:51 2009 Return-Path: <pdfbox-dev-returntranthomas@example.com> Delivered-To: tranthomas@example.com Received: (qmail 10478 invoked from network); 1 Mar 2009 18:31:51 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Mar 2009 18:31:51 -0000 Received: (qmail 37360 invoked by uid 500); 1 Mar 2009 18:31:51 -0000 Delivered-To: tranthomas@example.com Received: (qmail 37345 invoked by uid 500); 1 Mar 2009 18:31:50 -0000 Mailing-List: contact tranthomas@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:tranthomas@example.com> List-Unsubscribe: <mailto:tranthomas@example.com> List-Post: <mailto:tranthomas@example.com> List-Id: <pdfbox-dev.incubator.apache.org> Reply-To: tranthomas@example.com Delivered-To: mailing list tranthomas@example.com Received: (qmail 37334 invoked by uid 99); 1 Mar 2009 18:31:50 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Mar 2009 10:31:50 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of tranthomas@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail.basistech.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Mar 2009 18:31:44 +0000 Received: from MSG-BOX.basistech.net ([83.161.120.155]) by mail.basistech.com with Microsoft SMTPSVC+1-512-754-7244); Sun, 1 Mar 2009 13:31:22 -0500 Received: from MSG-BOX.basistech.net ([888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4]) by MSG-BOX.basistech.net ([888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4]) with mapi; Sun, 1 Mar 2009 13:31:22 -0500 From: Ken Glidden <tranthomas@example.com> To: "tranthomas@example.com" <tranthomas@example.com>, "tranthomas@example.com" <tranthomas@example.com> Date: Sun, 1 Mar 2009 13:31:14 -0500 Subject: RE: [jira] Resolved: (PDFBOX-430) Incorrect diacritic placement in text extraction Thread-Topic: [jira] Resolved: (PDFBOX-430) Incorrect diacritic placement in text extraction Thread-Index: AcmZybOhQguWWyLNQEykXL9JpupTuwA0fjiQ Message-ID: <tranthomas@example.com> References: <1837444066.1234992241888.JavaMail.jira@brutus> <716415661.1234992961875.JavaMail.jira@brutus> <tranthomas@example.com> In-Reply-To: <tranthomas@example.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginalArrivalTime: 01 Mar 2009 18:31:22.0603 (UTC) FILETIME=[EBF43BB0:01C99A9B] X-Virus-Checked: Checked by ClamAV on apache.org SSBhbSBzYWlkIEtlbiBHbGlkZGVuLg0KSSdtIFZQIG9mIEVuZ2luZWVyaW5nIGF0IEJhc2lzIFRl Y2hub2xvZ3kgYW5kIGFtIHdvcmtpbmcgZGlyZWN0bHkgd2l0aCBCcmlhbiBvbiB0aGlzLg0KTm8g bGVnYWwgaXNzdWVzIHRvIHdvcnJ5IGFib3V0Lg0KQ2hlZXJzLg0KDQotLS0tLU9yaWdpbmFsIE1l c3NhZ2UtLS0tLQ0KRnJvbTogSmVyZW1pYXMgTWFlcmtpIFttYWlsdG86ZGV2QGplcmVtaWFzLW1h ZXJraS5jaF0gDQpTZW50OiBTYXR1cmRheSwgRmVicnVhcnkgMjgsIDIwMDkgMTI6MjYgUE0NClRv OiBwZGZib3gtZGV2QGluY3ViYXRvci5hcGFjaGUub3JnDQpTdWJqZWN0OiBSZTogW2ppcmFdIFJl c29sdmVkOiAoUERGQk9YLTQzMCkgSW5jb3JyZWN0IGRpYWNyaXRpYyBwbGFjZW1lbnQgaW4gdGV4 dCBleHRyYWN0aW9uDQoNCkJyaWFuLA0KDQp5b3Ugc3RhdGUgaGVyZSB0aGF0IHlvdSd2ZSBhcHBs aWVkIGEgcGF0Y2ggYnkgb25lIEtlbiBHbGlkZGVuLiBJIGNhbm5vdA0KZmluZCBhbnkgcG9zdCBv ciBzdWJtaXNzaW9uIGZyb20gYSBwZXJzb24gd2l0aCB0aGF0IG5hbWUgb24gdGhlIFBERkJveA0K bWFpbGluZyBsaXN0cy4gU28gSSdtIGNvbmNlcm5lZCBhYm91dCB0aGUgbGVnYWwgdHJhaWwgaGVy ZS4gQ2FuIHlvdQ0KZXhwbGFpbiB0aGF0LCBwbGVhc2U/IFRoYW5rIHlvdS4NCg0KT24gMTguMDIu MjAwOSAyMjozNjowMSBCcmlhbiBDYXJyaWVyIChKSVJBKSB3cm90ZToNCj4gDQo+ICAgICAgWyBo dHRwczovL2lzc3Vlcy5hcGFjaGUub3JnL2ppcmEvYnJvd3NlL1BERkJPWC00MzA/cGFnZT1jb20u YXRsYXNzaWFuLmppcmEucGx1Z2luLnN5c3RlbS5pc3N1ZXRhYnBhbmVsczphbGwtdGFicGFuZWwg XQ0KPiANCj4gQnJpYW4gQ2FycmllciByZXNvbHZlZCBQREZCT1gtNDMwLg0KPiAtLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQo+IA0KPiAgICAgUmVzb2x1dGlvbjogRml4ZWQNCj4g DQo+IEZpeGVkIHdpdGggcGF0Y2ggYnkgS2VuIEdsaWRkZW4gdGhhdCBtZXJnZXMgYSBzaW5nbGUg ZGlhY3JpdGljIHRleHQgY2h1bmsgaW50byB0aGUgcHJldmlvdXMgdGV4dCBjaHVuayBpZiB0aGV5 IG92ZXJsYXAuICBOb3RlIHRoYXQgdGhpcyB3aWxsIG5vdCBzb2x2ZSBwcm9ibGVtcyB3aGVyZSB0 aGUgZGlhY3JpdGljIGNvbWVzIG11Y2ggYWZ0ZXIgdGhlIHRleHQgY2h1bmsgaXQgb3ZlcmxheXMs IGJ1dCB3ZSBoYXZlIG5vdCBvYnNlcnZlZCBQREYgZmlsZXMgbGlrZSB0aGF0Lg0KPiANCj4gU2Vu ZGluZyAgICAgICAgdHJ1bmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3BkZmJveC91dGlsL1BE RlRleHRTdHJpcHBlci5qYXZhDQo+IFNlbmRpbmcgICAgICAgIHRydW5rL3NyYy9tYWluL2phdmEv b3JnL2FwYWNoZS9wZGZib3gvdXRpbC9UZXh0UG9zaXRpb24uamF2YQ0KPiBTZW5kaW5nICAgICAg ICB0cnVuay90ZXN0L2lucHV0L0Fjcm9iYXQ5LnBkZi1zb3J0ZWQudHh0DQo+IFNlbmRpbmcgICAg ICAgIHRydW5rL3Rlc3QvaW5wdXQvQWNyb2JhdDkucGRmLnR4dA0KPiBUcmFuc21pdHRpbmcgZmls ZSBkYXRhIC4uLi5Db21taXR0ZWQgcmV2aXNpb24gNzQ1NjY1Lg0KPiANCj4gDQo+IA0KPiA+IElu Y29ycmVjdCBkaWFjcml0aWMgcGxhY2VtZW50IGluIHRleHQgZXh0cmFjdGlvbg0KPiA+IC0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KPiA+DQo+ID4gICAg ICAgICAgICAgICAgIEtleTogUERGQk9YLTQzMA0KPiA+ICAgICAgICAgICAgICAgICBVUkw6IGh0 dHBzOi8vaXNzdWVzLmFwYWNoZS5vcmcvamlyYS9icm93c2UvUERGQk9YLTQzMA0KPiA+ICAgICAg ICAgICAgIFByb2plY3Q6IFBERkJveA0KPiA+ICAgICAgICAgIElzc3VlIFR5cGU6IEJ1Zw0KPiA+ ICAgICAgICAgICAgUmVwb3J0ZXI6IEJyaWFuIENhcnJpZXINCj4gPg0KPiA+IFNvbWUgUERGIGZp bGVzIHN0b3JlIGRpYWNyaXRpY3MgKGFjY2VudHMgb3ZlciBjaGFyYWN0ZXJzKSBhcyBzZXBhcmF0 ZSB0ZXh0IGVsZW1lbnRzLiBUaGUgUERGIGZpbGVzIGVzc2VudGlhbGx5IGhhdmUgYSBjaHVuayBv ZiB0ZXh0IGFuZCB0aGVuIGJhY2t1cCBhbmQgcGxhY2UgdGhlIGRpYWNyaXRpYyBvdmVyIG9uZSBv ZiB0aGUgY2hhcmFjdGVycyBpbiB0aGUgY2h1bmsgb2YgdGV4dC4gV2l0aCB0ZXh0IGV4dHJhY3Rp b24sIHRoZSBjdXJyZW50IGRlc2lnbiBkb2VzIG5vdCBhbGxvdyB0aGUgZGlhY3JpdGljIHRvIGJl IHBsYWNlZCBvdmVyIGEgY2hhcmFjdGVyIGluIHRoZSBjaHVuayBhbmQgaW5zdGVhZCBpdCBpcyBw bGFjZWQgYWZ0ZXIgdGhlIGNodW5rLiANCj4gPiBUaGUgZGVidWctZGlhYzIucGRmIGZpbGUgaW4g UERGQk9YLTQyOSBzaG93cyB0aGlzIHByb2JsZW0uIA0KPiANCj4gLS0gDQo+IFRoaXMgbWVzc2Fn ZSBpcyBhdXRvbWF0aWNhbGx5IGdlbmVyYXRlZCBieSBKSVJBLg0KPiAtDQo+IFlvdSBjYW4gcmVw bHkgdG8gdGhpcyBlbWFpbCB0byBhZGQgYSBjb21tZW50IHRvIHRoZSBpc3N1ZSBvbmxpbmUuDQoN Cg0KDQoNCkplcmVtaWFzIE1hZXJraQ0KDQo= From pdfbox-dev-returntranthomas@example.com Sun Mar 01 20:06:17 2009 Return-Path: <pdfbox-dev-returntranthomas@example.com> Delivered-To: tranthomas@example.com Received: (qmail 38318 invoked from network); 1 Mar 2009 20:06:17 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 1 Mar 2009 20:06:17 -0000 Received: (qmail 88793 invoked by uid 500); 1 Mar 2009 20:06:17 -0000 Delivered-To: tranthomas@example.com Received: (qmail 88779 invoked by uid 500); 1 Mar 2009 20:06:17 -0000 Mailing-List: contact tranthomas@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:tranthomas@example.com> List-Unsubscribe: <mailto:tranthomas@example.com> List-Post: <mailto:tranthomas@example.com> List-Id: <pdfbox-dev.incubator.apache.org> Reply-To: tranthomas@example.com Delivered-To: mailing list tranthomas@example.com Delivered-To: moderator for tranthomas@example.com Received: (qmail 65675 invoked by uid 99); 27 Feb 2009 09:30:13 -0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) From: Kai Dietrich <tranthomas@example.com> To: tranthomas@example.com Subject: pdfextracttextbatch Date: Fri, 27 Feb 2009 10:29:31 +0100 User-Agent: KMail/1.9.9 MIME-Version: 1.0 Content-Type: Multipart/Mixed; boundary="Boundary-00=_8J7pJC5XC9KTsL1" Message-Id: <tranthomas@example.com> X-Virus-Checked: Checked by ClamAV on apache.org --Boundary-00=_8J7pJC5XC9KTsL1 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Hello list, first of all, thank you all for pdfbox, I'm currently using it to extract the text from a huge (40.000+) collection of PDF files and it works pretty good (besides failing on some strange encodings, broken headers and the like). Then again, here comes the problem: My poor little box is running at 100% CPU all night with a "find . -name '*.pdf' | xargs pdfextracttext" and I'm not even at 1 file/second. Most of the CPU load probably comes from starting and stopping the Java VM - which is a huge waste of time and energy. So what would be quite helpful is a tool which has the "for file in files"-loop inside the VM. So, here comes the tool :) It's an addon to org.apache.pdfbox.ExtractText -- it just removes the output-file parameter and handles all given files as input pdf files. Problem is, I can't get pdfbox 0.8.0 to work well, because of some incompatability with the fontbox package from my distro (Gentoo). And I think I'll just torture my little box a bit longer until the find-xargs-extracttext job is done. But I don't want to waste the idea and the code, so maybe someone who has a working developer setup could test and improve on exception handling. Greetings Kai --Boundary-00=_8J7pJC5XC9KTsL1 Content-Type: text/x-java; charset="utf-8"; name="ExtractTextBatch.java" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="ExtractTextBatch.java" /* * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with * this work for additional information regarding copyright ownership. * The ASF licenses this file to You under the Apache License, Version 2.0 * (the "License"); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ package org.apache.pdfbox; import org.apache.pdfbox.ExtractText; /** * This is the main program that simply parses the pdf documents and transforms them * into text (batch version of ExtractText). * * @author <a href="mailto:tranthomas@example.com">Ben Litchfield</a> * @author <a href="mailto:tranthomas@example.com">Kai Dietrich</a> * @version $Revision$ */ public class ExtractTextBatch { private static final String PASSWORD = "-password"; private static final String ENCODING = "-encoding"; private static final String CONSOLE = "-console"; private static final String START_PAGE = "-startPage"; private static final String END_PAGE = "-endPage"; private static final String SORT = "-sort"; private static final String HTML = "-html"; /** * private constructor. */ private ExtractTextBatch() { //static class } /** * Infamous main method. * * @param args Command line arguments, should be one and a reference to a file. * * @throws Exception If there is an error parsing the document. */ public static void main( String[] args ) throws Exception { int startPDFs = 0; for( int i=0; i<args.length; i++ ) { if( args[i].equals( PASSWORD ) || args[i].equals( ENCODING ) || args[i].equals( START_PAGE ) || args[i].equals( END_PAGE ) ) { i++; } else if ( args[i].equals( CONSOLE ) || args[i].equals( SORT ) || args[i].equals( HTML ) ) { } else { startPDFs = i; break; } } String params[] = new String[startPDFs+1]; for( int i=0; i<startPDFs; i++) { params[i] = args[i]; } for( int i=startPDFs; i<args.length; i++) { params[startPDFs] = args[i]; try { org.apache.pdfbox.ExtractText.main(params); } catch(Exception e) { System.out.println("Oooops. Exception occured with file " + args[i] + ", continuing..."); } } } /** * This will print the usage requirements and exit. */ private static void usage() { System.err.println( "Usage: java org.apache.pdfbox.ExtractTextBatch [OPTIONS] <PDF file>...\n" + " -password <password> Password to decrypt documents\n" + " -encoding <output encoding> (ISO-8859-1,UTF-16BE,UTF-16LE,...)\n" + " -console Send text to console instead of file\n" + " -html Output in HTML format instead of raw text\n" + " -sort Sort the text before writing\n" + " -startPage <number> The first page to start extraction(1 based)\n" + " -endPage <number> The last page to extract(inclusive)\n" + " <PDF file>... The PDF document(s) to use\n" ); System.exit( 1 ); } } --Boundary-00=_8J7pJC5XC9KTsL1-- From pdfbox-dev-returntranthomas@example.com Mon Mar 02 08:31:25 2009 Return-Path: <pdfbox-dev-returntranthomas@example.com> Delivered-To: tranthomas@example.com Received: (qmail 39659 invoked from network); 2 Mar 2009 08:31:25 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2009 08:31:25 -0000 Received: (qmail 64945 invoked by uid 500); 2 Mar 2009 08:31:25 -0000 Delivered-To: tranthomas@example.com Received: (qmail 64900 invoked by uid 500); 2 Mar 2009 08:31:25 -0000 Mailing-List: contact tranthomas@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:tranthomas@example.com> List-Unsubscribe: <mailto:tranthomas@example.com> List-Post: <mailto:tranthomas@example.com> List-Id: <pdfbox-dev.incubator.apache.org> Reply-To: tranthomas@example.com Delivered-To: mailing list tranthomas@example.com Received: (qmail 64889 invoked by uid 99); 2 Mar 2009 08:31:25 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2009 00:31:25 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO tux17.hoststar.ch) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2009 08:31:15 +0000 Received: from [83.161.120.155] (adsl-89-217-30-251.adslplus.ch [83.161.120.155]) (authenticated bits=0) by tux17.hoststar.ch (8.13.6/8.12.11) with ESMTP id n228Urdf001504 for <tranthomas@example.com>; Mon, 2 Mar 2009 09:30:54 +0100 Date: Mon, 02 Mar 2009 09:30:52 +0100 From: Jeremias Maerki <tranthomas@example.com> To: tranthomas@example.com Subject: Re: contributing to PDFBox (was: [jira] Resolved: (PDFBOX-430) Incorrect diacritic placement in text extraction) In-Reply-To: <tranthomas@example.com> References: <tranthomas@example.com> <tranthomas@example.com> Message-Id: <tranthomas@example.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-Mailer: Becky! ver. 2.47.01 [en] X-Antivirus: avast! (VPS 090301-0, 01.03.2009), Outbound message X-Antivirus-Status: Clean X-Virus-Checked: Checked by ClamAV on apache.org Thanks for speaking up, Ken. It's a great thing you're contributing to PDFBox. But we actually do have legal issues to worry about here. The way this happened, we don't have a legal trail to make sure that your contributions are actually intended for inclusion and under what license. Only Brian (hopefully) knows your intentions. When you attach a patch to a Jira issue, you have to tick a checkbox indicating that you intend this for inclusion: "[ ] Grant license to ASF for inclusion in ASF works (as per the Apache License §5) Contributions intended for inclusion in ASF products (eg. patches, code) must be licensed to ASF under the terms of the Apache License. Other attachments (eg. log dumps, test cases) need not be." With §5 of the ALv2 you explicitely give the ASF the same license for your changes as the ASF gives to its users. That is enough for smaller patches (bugfixes, small improvements). As soon as you contribute considerable new functionality or new files which have a certain "artistic" aspect, the §5 is considered insufficient at which point committers are expected to ask for an Contributor License Agreement to be filed with the ASF. Also, regular contributors should send in a CLA as it is also a precondition to becoming a committer. For even larger contributions (like whole new subsystems), a contribution may even have to go through IP clearance with an explicit separate license grant on the code submitted. So there are various levels. The lines are probably not always very clearly drawn. But the intent is to protect the users and the contributors (i.e. you) from legal harm [1]. That can only happen if we have a clean legal trail. [1] http://apache.org/foundation/how-it-works.html#what (see especially the third point in the list) I only notice after this started that you and Justin LeFebvre are from the same company. Both of you have written more than one patch. So I would like to suggest that both of you send in an ICLA [2]. Please also check if the work contracts in your company make it necessary to send in a CCLA [2] in addition to the ICLAs. [2] http://apache.org/licenses/#clas A committer can always ask the PMC chair or an ASF member to check if a particular ICLA has been recorded, yet. Ken, can I ask you to attach the two (original) patches, that were processed via Brian, to the JIRA issues associated with them so the gaps are filled, even if that happens after the two patches were processed. I think that should be enough to correct the situation. In the future, please attach your patches to a new JIRA issues and take it from there. There are other points also: by directly working with Brian, there is no discussion (if necessary) around this if anyone has any issues. Other committers can only react after everything has already happened. You're also not taking part in the community whose building is the most important task of PDFBox being in the Apache Incubator. And you're not getting the same visibility you'd get if you take part in discussions here. Only that way does the existing team have a chance to get to know you and to eventually vote you in as a committer if you turn out to be a regular contributor. Given that two employees of your company contribute to PDFBox means that it is important to you. Then it is all the more important that you participate in the project and jointly help evolve the project in directions that help you. Everybody (especially Brian), don't feel bad about this! The Incubation phase is here for everybody to learn who we do things inside the Apache Software Foundation. There are a few rules that makes the ASF so different from the ordinary SourceForge project. I know it's a lot of new stuff especially new committers have to learn. Hopefully, we mentors can help clear things up if there are questions or problems. Thank you for your understanding! On 01.03.2009 19:31:14 Ken Glidden wrote: > I am said Ken Glidden. > I'm VP of Engineering at Basis Technology and am working directly with Brian on this. > No legal issues to worry about. > Cheers. > > -----Original Message----- > From: Jeremias Maerki [mailto:tranthomas@example.com] > Sent: Saturday, February 28, 2009 12:26 PM > To: tranthomas@example.com > Subject: Re: [jira] Resolved: (PDFBOX-430) Incorrect diacritic placement in text extraction > > Brian, > > you state here that you've applied a patch by one Ken Glidden. I cannot > find any post or submission from a person with that name on the PDFBox > mailing lists. So I'm concerned about the legal trail here. Can you > explain that, please? Thank you. > > On 18.02.2009 22:36:01 Brian Carrier (JIRA) wrote: > > > > [ https://issues.apache.org/jira/browse/PDFBOX-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] > > > > Brian Carrier resolved PDFBOX-430. > > ---------------------------------- > > > > Resolution: Fixed > > > > Fixed with patch by Ken Glidden that merges a single diacritic text chunk into the previous text chunk if they overlap. Note that this will not solve problems where the diacritic comes much after the text chunk it overlays, but we have not observed PDF files like that. > > > > Sending trunk/src/main/java/org/apache/pdfbox/util/PDFTextStripper.java > > Sending trunk/src/main/java/org/apache/pdfbox/util/TextPosition.java > > Sending trunk/test/input/Acrobat9.pdf-sorted.txt > > Sending trunk/test/input/Acrobat9.pdf.txt > > Transmitting file data ....Committed revision 745665. > > > > > > > > > Incorrect diacritic placement in text extraction > > > ------------------------------------------------ > > > > > > Key: PDFBOX-430 > > > URL: https://issues.apache.org/jira/browse/PDFBOX-430 > > > Project: PDFBox > > > Issue Type: Bug > > > Reporter: Brian Carrier > > > > > > Some PDF files store diacritics (accents over characters) as separate text elements. The PDF files essentially have a chunk of text and then backup and place the diacritic over one of the characters in the chunk of text. With text extraction, the current design does not allow the diacritic to be placed over a character in the chunk and instead it is placed after the chunk. > > > The debug-diac2.pdf file in PDFBOX-429 shows this problem. > > > > -- > > This message is automatically generated by JIRA. > > - > > You can reply to this email to add a comment to the issue online. > > > > > Jeremias Maerki > Jeremias Maerki From pdfbox-dev-returntranthomas@example.com Mon Mar 02 14:12:20 2009 Return-Path: <pdfbox-dev-returntranthomas@example.com> Delivered-To: tranthomas@example.com Received: (qmail 24115 invoked from network); 2 Mar 2009 14:12:20 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2009 14:12:20 -0000 Received: (qmail 14785 invoked by uid 500); 2 Mar 2009 14:12:20 -0000 Delivered-To: tranthomas@example.com Received: (qmail 14772 invoked by uid 500); 2 Mar 2009 14:12:20 -0000 Mailing-List: contact tranthomas@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:tranthomas@example.com> List-Unsubscribe: <mailto:tranthomas@example.com> List-Post: <mailto:tranthomas@example.com> List-Id: <pdfbox-dev.incubator.apache.org> Reply-To: tranthomas@example.com Delivered-To: mailing list tranthomas@example.com Received: (qmail 14761 invoked by uid 99); 2 Mar 2009 14:12:20 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2009 06:12:20 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO spunkymail-a6.g.dreamhost.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2009 14:12:12 +0000 Received: from [83.161.120.155] (65-78-24-168.c3-0.smr-ubr1.sbo-smr.ma.cable.rcn.com [83.161.120.155]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by spunkymail-a6.g.dreamhost.com (Postfix) with ESMTP id 9B033109F2B for <tranthomas@example.com>; Mon, 2 Mar 2009 06:11:49 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v753.1) In-Reply-To: <tranthomas@example.com> References: <tranthomas@example.com> <tranthomas@example.com> <tranthomas@example.com> Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed Message-Id: <tranthomas@example.com> Content-Transfer-Encoding: quoted-printable From: Brian Carrier <tranthomas@example.com> Subject: Re: contributing to PDFBox (was: [jira] Resolved: (PDFBOX-430) Incorrect diacritic placement in text extraction) Date: Mon, 2 Mar 2009 09:11:47 -0500 To: tranthomas@example.com X-Mailer: Apple Mail (2.753.1) X-Virus-Checked: Checked by ClamAV on apache.org Hi Jeremias, Sorry for the confusion. Ken did most of the patch and then had to =20 work on some other projects, so I did some final touches and =20 submitted it. We already have a CCLA on file. thanks, brian On Mar 2, 2009, at 3:30 AM, Jeremias Maerki wrote: > Thanks for speaking up, Ken. It's a great thing you're contributing to > PDFBox. But we actually do have legal issues to worry about here. > > The way this happened, we don't have a legal trail to make sure that > your contributions are actually intended for inclusion and under what > license. Only Brian (hopefully) knows your intentions. When you =20 > attach a > patch to a Jira issue, you have to tick a checkbox indicating that you > intend this for inclusion: > > "[ ] Grant license to ASF for inclusion in ASF works (as per the =20 > Apache > License =A75) > Contributions intended for inclusion in ASF products (eg. patches, =20 > code) > must be licensed to ASF under the terms of the Apache License. Other > attachments (eg. log dumps, test cases) need not be." > > With =A75 of the ALv2 you explicitely give the ASF the same license = for > your changes as the ASF gives to its users. That is enough for smaller > patches (bugfixes, small improvements). As soon as you contribute > considerable new functionality or new files which have a certain > "artistic" aspect, the =A75 is considered insufficient at which point > committers are expected to ask for an Contributor License Agreement to > be filed with the ASF. Also, regular contributors should send in a CLA > as it is also a precondition to becoming a committer. For even larger > contributions (like whole new subsystems), a contribution may even =20 > have > to go through IP clearance with an explicit separate license grant on > the code submitted. So there are various levels. The lines are =20 > probably > not always very clearly drawn. But the intent is to protect the users > and the contributors (i.e. you) from legal harm [1]. That can only > happen if we have a clean legal trail. > > [1] http://apache.org/foundation/how-it-works.html#what > (see especially the third point in the list) > > I only notice after this started that you and Justin LeFebvre are from > the same company. Both of you have written more than one patch. So I > would like to suggest that both of you send in an ICLA [2]. Please =20 > also > check if the work contracts in your company make it necessary to =20 > send in > a CCLA [2] in addition to the ICLAs. > > [2] http://apache.org/licenses/#clas > > A committer can always ask the PMC chair or an ASF member to check =20 > if a > particular ICLA has been recorded, yet. > > Ken, can I ask you to attach the two (original) patches, that were > processed via Brian, to the JIRA issues associated with them so the =20= > gaps > are filled, even if that happens after the two patches were =20 > processed. I > think that should be enough to correct the situation. In the future, > please attach your patches to a new JIRA issues and take it from =20 > there. > > There are other points also: by directly working with Brian, there =20 > is no > discussion (if necessary) around this if anyone has any issues. Other > committers can only react after everything has already happened. =20 > You're > also not taking part in the community whose building is the most > important task of PDFBox being in the Apache Incubator. And you're not > getting the same visibility you'd get if you take part in discussions > here. Only that way does the existing team have a chance to get to =20 > know > you and to eventually vote you in as a committer if you turn out to =20= > be a > regular contributor. Given that two employees of your company =20 > contribute > to PDFBox means that it is important to you. Then it is all the more > important that you participate in the project and jointly help evolve > the project in directions that help you. > > Everybody (especially Brian), don't feel bad about this! The =20 > Incubation > phase is here for everybody to learn who we do things inside the =20 > Apache > Software Foundation. There are a few rules that makes the ASF so > different from the ordinary SourceForge project. I know it's a lot of > new stuff especially new committers have to learn. Hopefully, we =20 > mentors > can help clear things up if there are questions or problems. > > Thank you for your understanding! > > On 01.03.2009 19:31:14 Ken Glidden wrote: >> I am said Ken Glidden. >> I'm VP of Engineering at Basis Technology and am working directly =20 >> with Brian on this. >> No legal issues to worry about. >> Cheers. >> >> -----Original Message----- >> From: Jeremias Maerki [mailto:tranthomas@example.com] >> Sent: Saturday, February 28, 2009 12:26 PM >> To: tranthomas@example.com >> Subject: Re: [jira] Resolved: (PDFBOX-430) Incorrect diacritic =20 >> placement in text extraction >> >> Brian, >> >> you state here that you've applied a patch by one Ken Glidden. I =20 >> cannot >> find any post or submission from a person with that name on the =20 >> PDFBox >> mailing lists. So I'm concerned about the legal trail here. Can you >> explain that, please? Thank you. >> >> On 18.02.2009 22:36:01 Brian Carrier (JIRA) wrote: >>> >>> [ https://issues.apache.org/jira/browse/PDFBOX-430?=20 >>> page=3Dcom.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel = ] >>> >>> Brian Carrier resolved PDFBOX-430. >>> ---------------------------------- >>> >>> Resolution: Fixed >>> >>> Fixed with patch by Ken Glidden that merges a single diacritic =20 >>> text chunk into the previous text chunk if they overlap. Note =20 >>> that this will not solve problems where the diacritic comes much =20 >>> after the text chunk it overlays, but we have not observed PDF =20 >>> files like that. >>> >>> Sending trunk/src/main/java/org/apache/pdfbox/util/=20 >>> PDFTextStripper.java >>> Sending trunk/src/main/java/org/apache/pdfbox/util/=20 >>> TextPosition.java >>> Sending trunk/test/input/Acrobat9.pdf-sorted.txt >>> Sending trunk/test/input/Acrobat9.pdf.txt >>> Transmitting file data ....Committed revision 745665. >>> >>> >>> >>>> Incorrect diacritic placement in text extraction >>>> ------------------------------------------------ >>>> >>>> Key: PDFBOX-430 >>>> URL: https://issues.apache.org/jira/browse/=20 >>>> PDFBOX-430 >>>> Project: PDFBox >>>> Issue Type: Bug >>>> Reporter: Brian Carrier >>>> >>>> Some PDF files store diacritics (accents over characters) as =20 >>>> separate text elements. The PDF files essentially have a chunk =20 >>>> of text and then backup and place the diacritic over one of the =20 >>>> characters in the chunk of text. With text extraction, the =20 >>>> current design does not allow the diacritic to be placed over a =20 >>>> character in the chunk and instead it is placed after the chunk. >>>> The debug-diac2.pdf file in PDFBOX-429 shows this problem. >>> >>> --=20 >>> This message is automatically generated by JIRA. >>> - >>> You can reply to this email to add a comment to the issue online. >> >> >> >> >> Jeremias Maerki >> > > > > > Jeremias Maerki > From pdfbox-dev-returntranthomas@example.com Mon Mar 02 14:31:07 2009 Return-Path: <pdfbox-dev-returntranthomas@example.com> Delivered-To: tranthomas@example.com Received: (qmail 39058 invoked from network); 2 Mar 2009 14:31:07 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2009 14:31:07 -0000 Received: (qmail 46766 invoked by uid 500); 2 Mar 2009 14:31:04 -0000 Delivered-To: tranthomas@example.com Received: (qmail 46756 invoked by uid 500); 2 Mar 2009 14:31:04 -0000 Mailing-List: contact tranthomas@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:tranthomas@example.com> List-Unsubscribe: <mailto:tranthomas@example.com> List-Post: <mailto:tranthomas@example.com> List-Id: <pdfbox-dev.incubator.apache.org> Reply-To: tranthomas@example.com Delivered-To: mailing list tranthomas@example.com Received: (qmail 46740 invoked by uid 99); 2 Mar 2009 14:31:04 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2009 06:31:04 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO tux17.hoststar.ch) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2009 14:30:55 +0000 Received: from [83.161.120.155] (adsl-89-217-30-251.adslplus.ch [83.161.120.155]) (authenticated bits=0) by tux17.hoststar.ch (8.13.6/8.12.11) with ESMTP id n22EUa9d001191 for <tranthomas@example.com>; Mon, 2 Mar 2009 15:30:37 +0100 Date: Mon, 02 Mar 2009 15:30:35 +0100 From: Jeremias Maerki <tranthomas@example.com> To: tranthomas@example.com Subject: Re: contributing to PDFBox (was: [jira] Resolved: (PDFBOX-430) Incorrect diacritic placement in text extraction) In-Reply-To: <tranthomas@example.com> References: <tranthomas@example.com> <tranthomas@example.com> Message-Id: <tranthomas@example.com> MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable X-Mailer: Becky! ver. 2.47.01 [en] X-Antivirus: avast! (VPS 090301-0, 01.03.2009), Outbound message X-Antivirus-Status: Clean X-Virus-Checked: Checked by ClamAV on apache.org I can see that there is a CCLA from Basis Technology Corporation but it doesn't list Ken Glidden nor Justin LeFebvre in Schedule A. Are you also an employee of that company? Anyway, the CCLA is always in addition to an ICLA if a CCLA is necessary. Contributions to ASF projects are done alwa= ys by an individual, never by a company. That's why the ICLA is the most important of the two. A company may have to officially sanction that some of their employees contribute code to the ASF depending on the work contracts. That's what the CCLA is for. I'm sorry that I have to be difficult here but we have to make sure this is all done right. On 02.03.2009 15:11:47 Brian Carrier wrote: > Hi Jeremias, >=20 > Sorry for the confusion. Ken did most of the patch and then had to =20 > work on some other projects, so I did some final touches and =20 > submitted it. We already have a CCLA on file. >=20 > thanks, > brian >=20 >=20 > On Mar 2, 2009, at 3:30 AM, Jeremias Maerki wrote: >=20 > > Thanks for speaking up, Ken. It's a great thing you're contributing to > > PDFBox. But we actually do have legal issues to worry about here. > > > > The way this happened, we don't have a legal trail to make sure that > > your contributions are actually intended for inclusion and under what > > license. Only Brian (hopefully) knows your intentions. When you =20 > > attach a > > patch to a Jira issue, you have to tick a checkbox indicating that you > > intend this for inclusion: > > > > "[ ] Grant license to ASF for inclusion in ASF works (as per the =20 > > Apache > > License =A75) > > Contributions intended for inclusion in ASF products (eg. patches, =20 > > code) > > must be licensed to ASF under the terms of the Apache License. Other > > attachments (eg. log dumps, test cases) need not be." > > > > With =A75 of the ALv2 you explicitely give the ASF the same license for > > your changes as the ASF gives to its users. That is enough for smaller > > patches (bugfixes, small improvements). As soon as you contribute > > considerable new functionality or new files which have a certain > > "artistic" aspect, the =A75 is considered insufficient at which point > > committers are expected to ask for an Contributor License Agreement to > > be filed with the ASF. Also, regular contributors should send in a CLA > > as it is also a precondition to becoming a committer. For even larger > > contributions (like whole new subsystems), a contribution may even =20 > > have > > to go through IP clearance with an explicit separate license grant on > > the code submitted. So there are various levels. The lines are =20 > > probably > > not always very clearly drawn. But the intent is to protect the users > > and the contributors (i.e. you) from legal harm [1]. That can only > > happen if we have a clean legal trail. > > > > [1] http://apache.org/foundation/how-it-works.html#what > > (see especially the third point in the list) > > > > I only notice after this started that you and Justin LeFebvre are from > > the same company. Both of you have written more than one patch. So I > > would like to suggest that both of you send in an ICLA [2]. Please =20 > > also > > check if the work contracts in your company make it necessary to =20 > > send in > > a CCLA [2] in addition to the ICLAs. > > > > [2] http://apache.org/licenses/#clas > > > > A committer can always ask the PMC chair or an ASF member to check =20 > > if a > > particular ICLA has been recorded, yet. > > > > Ken, can I ask you to attach the two (original) patches, that were > > processed via Brian, to the JIRA issues associated with them so the =20 > > gaps > > are filled, even if that happens after the two patches were =20 > > processed. I > > think that should be enough to correct the situation. In the future, > > please attach your patches to a new JIRA issues and take it from =20 > > there. > > > > There are other points also: by directly working with Brian, there =20 > > is no > > discussion (if necessary) around this if anyone has any issues. Other > > committers can only react after everything has already happened. =20 > > You're > > also not taking part in the community whose building is the most > > important task of PDFBox being in the Apache Incubator. And you're not > > getting the same visibility you'd get if you take part in discussions > > here. Only that way does the existing team have a chance to get to =20 > > know > > you and to eventually vote you in as a committer if you turn out to =20 > > be a > > regular contributor. Given that two employees of your company =20 > > contribute > > to PDFBox means that it is important to you. Then it is all the more > > important that you participate in the project and jointly help evolve > > the project in directions that help you. > > > > Everybody (especially Brian), don't feel bad about this! The =20 > > Incubation > > phase is here for everybody to learn who we do things inside the =20 > > Apache > > Software Foundation. There are a few rules that makes the ASF so > > different from the ordinary SourceForge project. I know it's a lot of > > new stuff especially new committers have to learn. Hopefully, we =20 > > mentors > > can help clear things up if there are questions or problems. > > > > Thank you for your understanding! > > > > On 01.03.2009 19:31:14 Ken Glidden wrote: > >> I am said Ken Glidden. > >> I'm VP of Engineering at Basis Technology and am working directly =20 > >> with Brian on this. > >> No legal issues to worry about. > >> Cheers. > >> > >> -----Original Message----- > >> From: Jeremias Maerki [mailto:tranthomas@example.com] > >> Sent: Saturday, February 28, 2009 12:26 PM > >> To: tranthomas@example.com > >> Subject: Re: [jira] Resolved: (PDFBOX-430) Incorrect diacritic =20 > >> placement in text extraction > >> > >> Brian, > >> > >> you state here that you've applied a patch by one Ken Glidden. I =20 > >> cannot > >> find any post or submission from a person with that name on the =20 > >> PDFBox > >> mailing lists. So I'm concerned about the legal trail here. Can you > >> explain that, please? Thank you. > >> > >> On 18.02.2009 22:36:01 Brian Carrier (JIRA) wrote: > >>> > >>> [ https://issues.apache.org/jira/browse/PDFBOX-430?=20 > >>> page=3Dcom.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] > >>> > >>> Brian Carrier resolved PDFBOX-430. > >>> ---------------------------------- > >>> > >>> Resolution: Fixed > >>> > >>> Fixed with patch by Ken Glidden that merges a single diacritic =20 > >>> text chunk into the previous text chunk if they overlap. Note =20 > >>> that this will not solve problems where the diacritic comes much =20 > >>> after the text chunk it overlays, but we have not observed PDF =20 > >>> files like that. > >>> > >>> Sending trunk/src/main/java/org/apache/pdfbox/util/=20 > >>> PDFTextStripper.java > >>> Sending trunk/src/main/java/org/apache/pdfbox/util/=20 > >>> TextPosition.java > >>> Sending trunk/test/input/Acrobat9.pdf-sorted.txt > >>> Sending trunk/test/input/Acrobat9.pdf.txt > >>> Transmitting file data ....Committed revision 745665. > >>> > >>> > >>> > >>>> Incorrect diacritic placement in text extraction > >>>> ------------------------------------------------ > >>>> > >>>> Key: PDFBOX-430 > >>>> URL: https://issues.apache.org/jira/browse/=20 > >>>> PDFBOX-430 > >>>> Project: PDFBox > >>>> Issue Type: Bug > >>>> Reporter: Brian Carrier > >>>> > >>>> Some PDF files store diacritics (accents over characters) as =20 > >>>> separate text elements. The PDF files essentially have a chunk =20 > >>>> of text and then backup and place the diacritic over one of the =20 > >>>> characters in the chunk of text. With text extraction, the =20 > >>>> current design does not allow the diacritic to be placed over a =20 > >>>> character in the chunk and instead it is placed after the chunk. > >>>> The debug-diac2.pdf file in PDFBOX-429 shows this problem. > >>> > >>> --=20 > >>> This message is automatically generated by JIRA. > >>> - > >>> You can reply to this email to add a comment to the issue online. > >> > >> > >> > >> > >> Jeremias Maerki > >> > > > > > > > > > > Jeremias Maerki > > >=20 Jeremias Maerki From pdfbox-dev-returntranthomas@example.com Mon Mar 02 15:22:12 2009 Return-Path: <pdfbox-dev-returntranthomas@example.com> Delivered-To: tranthomas@example.com Received: (qmail 3896 invoked from network); 2 Mar 2009 15:22:12 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2009 15:22:12 -0000 Received: (qmail 32029 invoked by uid 500); 2 Mar 2009 15:22:12 -0000 Delivered-To: tranthomas@example.com Received: (qmail 32008 invoked by uid 500); 2 Mar 2009 15:22:12 -0000 Mailing-List: contact tranthomas@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:tranthomas@example.com> List-Unsubscribe: <mailto:tranthomas@example.com> List-Post: <mailto:tranthomas@example.com> List-Id: <pdfbox-dev.incubator.apache.org> Reply-To: tranthomas@example.com Delivered-To: mailing list tranthomas@example.com Received: (qmail 31981 invoked by uid 99); 2 Mar 2009 15:22:12 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2009 07:22:12 -0800 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: unknown ip:83.161.120.155 (nike.apache.org: encountered unrecognized mechanism during SPF processing of domain of tranthomas@example.com) Received: from [83.161.120.155] (HELO ladle.sendmail.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2009 15:22:01 +0000 Received: from spork.sendmail.com (tls.sendmail.com [83.161.120.155]) by ladle.sendmail.com (Switch-3.3.1/Sentrion-3.0.0) with ESMTP id n22FLmia017446 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <tranthomas@example.com>; Mon, 2 Mar 2009 07:21:48 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=sendmail.com; s=ladle.dkim; t=1236007308; bh=XYSVEAepJTwSJ/hdbaVogF+eQpRy/6EXcsZh Z9b2x10=; h=Received:X-DKIM:DKIM-Signature:Reply-To:From:To: References:In-Reply-To:Subject:Date:Organization:Message-ID: MIME-Version:Content-Type:Content-Transfer-Encoding:X-Mailer: Thread-Index:Content-Language:X-MM-Ex-RefId; b=rdscmZVOPQYqsMDA87E RvyHvgASRn4JrqeDtxiEWL6ShDDF0BSfXm+WNUOyard+7R6QqotlG1Q9CBgWGTB07Hj zpbaH/VztQxuHm8WKenWN2qxTsMfzMGi/TKjmiT34goKvXrey7aFaAHNdeigE1aUJrV 0xmxnTNHwaOsSsgWAE= Received: from takashismi (125-15-38-251.rev.home.ne.jp [83.161.120.155]) (authenticated bits=0) by spork.sendmail.com (Switch-3.3.3mp/Switch-3.3.3mp) with ESMTP id n22FLajR013599 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for <tranthomas@example.com>; Mon, 2 Mar 2009 07:21:38 -0800 X-DKIM: Sendmail DKIM Filter v2.5.6 spork.sendmail.com n22FLajR013599 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=sendmail.com; s=spork.dkim; t=1236007299; bh=XYSVEAepJTwSJ/hdbaVogF+eQpRy/6EXcsZh Z9b2x10=; h=Reply-To:From:To:References:In-Reply-To:Subject:Date: Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding; b=E+IG1OJl3IFO2p0kxGZro1qStF0lr9CRxvvFNT9LxNI0BkT8ZOvFAnFZ4diX3FvSw aVoPcaStNf7N8zxGN4o79VkvKCezSUwkDemJ+v5POs9szBsATIWqHPvZ31a8XzPt++N +ySSy54CpIsTR4AQCH0nPGNa7DpniU5wo0zickI= Reply-To: <tranthomas@example.com> From: "Takashi Komatsubara" <tranthomas@example.com> To: <tranthomas@example.com> References: <607873083.1234177019902.JavaMail.jira@brutus> <519624715.1234866179682.JavaMail.jira@brutus> In-Reply-To: <519624715.1234866179682.JavaMail.jira@brutus> Subject: RE: [jira] Commented: (PDFBOX-420) Japanese Characters are garbled. Date: Tue, 3 Mar 2009 00:21:14 +0900 Organization: Sendmail, Inc Message-ID: <000301c99b4a$88716db0$99544910$@com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcmQ6iaI0c3RsKtMTBO3f+NAlJKVmgKX9ptg Content-Language: ja X-MM-Ex-RefId: 14888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4090302072138-0FFDDB90-0049B01E/0-0/0-1 X-Virus-Checked: Checked by ClamAV on apache.org Jukka, I have tried to contact with Pin Xue, have never got any response from = him. The patches he provided was at the sourceforge, and I have no = relationship with him. My work was collect cjk patches and fixes some tiny issues. There is no possibilities to get his permission. So, please ignore this patches and forget to support CJK by the Apache = PDFBox. If I have sometime to create my original CJK patches or someone = contribute that kind of patches, that't the time we start to think to = support CJK for Korea/China/Japanese people. Thank you, Takashi. -----Original Message----- From: Jukka Zitting (JIRA) [mailto:tranthomas@example.com]=20 Sent: Tuesday, February 17, 2009 7:23 PM To: tranthomas@example.com Subject: [jira] Commented: (PDFBOX-420) Japanese Characters are garbled. [ = https://issues.apache.org/jira/browse/PDFBOX-420?page=3Dcom.atlassian.jir= a.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D126741= 67#action_12674167 ]=20 Jukka Zitting commented on PDFBOX-420: -------------------------------------- If you can, please try contacting Pin Xue again to get his confirmation = on contributing this under the Apache license. How/why did you work together on this? If you share an employer and = wrote this patch at work, then an OK from your manager is typically = enough and often necessary, depending on the copyright terms of the = employment contract. How much of the code is written by Pin Xue? We can include the code = under the BSD license, but that requires special mentioning in the = LICENSE and NOTICE files that we and all the downstream projects need to = worry about. So if it's just a few lines of code then we're probably = better off by just rewriting them if we can't get an OK for the Apache = license. > Japanese Characters are garbled. > -------------------------------- > > Key: PDFBOX-420 > URL: https://issues.apache.org/jira/browse/PDFBOX-420 > Project: PDFBox > Issue Type: Bug > Components: Text extraction > Affects Versions: 0.8.0-incubator > Reporter: Takashi Komatsubara > Priority: Critical > Attachments: supportJapanese-fontbox.patch, = supportJapanese.patch, TestFilesForJapaneseGarbledIssue.zip > > > The extracted Japanese characters are completely garbled. > This issue is very critical for Japanese users. --=20 This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pdfbox-dev-returntranthomas@example.com Mon Mar 02 15:24:19 2009 Return-Path: <pdfbox-dev-returntranthomas@example.com> Delivered-To: tranthomas@example.com Received: (qmail 4742 invoked from network); 2 Mar 2009 15:24:19 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2009 15:24:19 -0000 Received: (qmail 35803 invoked by uid 500); 2 Mar 2009 15:24:19 -0000 Delivered-To: tranthomas@example.com Received: (qmail 35790 invoked by uid 500); 2 Mar 2009 15:24:19 -0000 Mailing-List: contact tranthomas@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:tranthomas@example.com> List-Unsubscribe: <mailto:tranthomas@example.com> List-Post: <mailto:tranthomas@example.com> List-Id: <pdfbox-dev.incubator.apache.org> Reply-To: tranthomas@example.com Delivered-To: mailing list tranthomas@example.com Received: (qmail 35778 invoked by uid 99); 2 Mar 2009 15:24:19 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2009 07:24:19 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2009 15:24:17 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 33EC3234C4A8 for <tranthomas@example.com>; Mon, 2 Mar 2009 07:23:56 -0800 (PST) Message-ID: <1400323113.1236007436211.JavaMail.jira@brutus> Date: Mon, 2 Mar 2009 07:23:56 -0800 (PST) From: "Michael Gesmann (JIRA)" <tranthomas@example.com> To: tranthomas@example.com Subject: [jira] Created: (PDFBOX-435) Handling of trailers MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Handling of trailers -------------------- Key: PDFBOX-435 URL: https://issues.apache.org/jira/browse/PDFBOX-435 Project: PDFBox Issue Type: Improvement Components: Parsing Affects Versions: 0.8.0-incubator Reporter: Michael Gesmann Priority: Minor Some PDF generating tools seem to produce invalid trailers but can nevertheless be displayed e.g. by Acrobat Reader. Therefore, it would be nice if PDFBox could also process these documents. Example 1 (no EOL behind "trailer", as generated by "ScanSoft PDF Create! 4", attached you will find an example): trailer<</Root 4 0 R/Info 1 0 R/Size 10/Prev 2979/ID[<00000000000000000000000000000000><215eab4c095713feb4cdbb15a9eba968>]>> Example 2 (not EOL but just a blank behind "trailer", cannot publish my example): trailer << /Size 26 /Root 24 0 R /Info 25 0 R /ID[<98fc28410100000042090000d1d1a606><98fc28410100000042090000d1d1a606>] >>: Here is a fix proposal: private boolean parseTrailer() throws IOException { if(pdfSource.peek() != 't'){ return false; } //read "trailer" String nextLine = readLine(); if( !nextLine.equals( "trailer" ) ) { // fix for example no 1 and no 2 // in some cases the EOL is missing and the trailer immediately continues with "<<" or with a blank character // even if this does not comply with PDF reference we want to support as many PDFs as possible // Acrobat reader can also deal with this. if (nextLine.startsWith("trailer")) { byte[] b = nextLine.getBytes(); int len = "trailer".length(); pdfSource.unread(b, len, b.length-len); } else { return false; } } // fix for example no2 // in some cases the EOL is missing and the trailer continues with " <<" // even if this does not comply with PDF reference we want to support as many PDFs as possible // Acrobat reader can also deal with this. skipSpaces(); COSDictionary parsedTrailer = parseCOSDictionary(); COSDictionary docTrailer = document.getTrailer(); if( docTrailer == null ) { document.setTrailer( parsedTrailer ); } else { docTrailer.addAll( parsedTrailer ); } skipSpaces(); return true; } -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pdfbox-dev-returntranthomas@example.com Mon Mar 02 15:29:22 2009 Return-Path: <pdfbox-dev-returntranthomas@example.com> Delivered-To: tranthomas@example.com Received: (qmail 8432 invoked from network); 2 Mar 2009 15:29:19 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2009 15:29:19 -0000 Received: (qmail 49324 invoked by uid 500); 2 Mar 2009 15:29:19 -0000 Delivered-To: tranthomas@example.com Received: (qmail 49314 invoked by uid 500); 2 Mar 2009 15:29:19 -0000 Mailing-List: contact tranthomas@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:tranthomas@example.com> List-Unsubscribe: <mailto:tranthomas@example.com> List-Post: <mailto:tranthomas@example.com> List-Id: <pdfbox-dev.incubator.apache.org> Reply-To: tranthomas@example.com Delivered-To: mailing list tranthomas@example.com Received: (qmail 49303 invoked by uid 99); 2 Mar 2009 15:29:19 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2009 07:29:19 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [83.161.120.155] (HELO brutus.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2009 15:29:17 +0000 Received: from brutus (localhost [83.161.120.155]) by brutus.apache.org (Postfix) with ESMTP id 2F8CB234C4A8 for <tranthomas@example.com>; Mon, 2 Mar 2009 07:28:56 -0800 (PST) Message-ID: <769283122.1236007736193.JavaMail.jira@brutus> Date: Mon, 2 Mar 2009 07:28:56 -0800 (PST) From: "Michael Gesmann (JIRA)" <tranthomas@example.com> To: tranthomas@example.com Subject: [jira] Updated: (PDFBOX-435) Handling of trailers In-Reply-To: <1400323113.1236007436211.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PDFBOX-435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gesmann updated PDFBOX-435: ----------------------------------- Attachment: trailerNoEOL.pdf example for case no. 1 > Handling of trailers > -------------------- > > Key: PDFBOX-435 > URL: https://issues.apache.org/jira/browse/PDFBOX-435 > Project: PDFBox > Issue Type: Improvement > Components: Parsing > Affects Versions: 0.8.0-incubator > Reporter: Michael Gesmann > Priority: Minor > Attachments: trailerNoEOL.pdf > > > Some PDF generating tools seem to produce invalid trailers but can nevertheless be displayed e.g. by Acrobat Reader. > Therefore, it would be nice if PDFBox could also process these documents. > Example 1 (no EOL behind "trailer", as generated by "ScanSoft PDF Create! 4", attached you will find an example): > trailer<</Root 4 0 R/Info 1 0 R/Size 10/Prev 2979/ID[<00000000000000000000000000000000><215eab4c095713feb4cdbb15a9eba968>]>> > Example 2 (not EOL but just a blank behind "trailer", cannot publish my example): > trailer << > /Size 26 > /Root 24 0 R > /Info 25 0 R > /ID[<98fc28410100000042090000d1d1a606><98fc28410100000042090000d1d1a606>] > >>: > Here is a fix proposal: > private boolean parseTrailer() throws IOException > { > if(pdfSource.peek() != 't'){ > return false; > } > //read "trailer" > String nextLine = readLine(); > if( !nextLine.equals( "trailer" ) ) { > // fix for example no 1 and no 2 > // in some cases the EOL is missing and the trailer immediately continues with "<<" or with a blank character > // even if this does not comply with PDF reference we want to support as many PDFs as possible > // Acrobat reader can also deal with this. > if (nextLine.startsWith("trailer")) > { > byte[] b = nextLine.getBytes(); > int len = "trailer".length(); > pdfSource.unread(b, len, b.length-len); > } > else > { > return false; > } > } > // fix for example no2 > // in some cases the EOL is missing and the trailer continues with " <<" > // even if this does not comply with PDF reference we want to support as many PDFs as possible > // Acrobat reader can also deal with this. > skipSpaces(); > > COSDictionary parsedTrailer = parseCOSDictionary(); > COSDictionary docTrailer = document.getTrailer(); > if( docTrailer == null ) > { > document.setTrailer( parsedTrailer ); > } > else > { > docTrailer.addAll( parsedTrailer ); > } > skipSpaces(); > return true; > } -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. From pdfbox-dev-returntranthomas@example.com Mon Mar 02 20:52:55 2009 Return-Path: <pdfbox-dev-returntranthomas@example.com> Delivered-To: tranthomas@example.com Received: (qmail 77401 invoked from network); 2 Mar 2009 20:52:55 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 2 Mar 2009 20:52:55 -0000 Received: (qmail 53665 invoked by uid 500); 2 Mar 2009 20:52:55 -0000 Delivered-To: tranthomas@example.com Received: (qmail 53653 invoked by uid 500); 2 Mar 2009 20:52:55 -0000 Mailing-List: contact tranthomas@example.com; run by ezmlm Precedence: bulk List-Help: <mailto:tranthomas@example.com> List-Unsubscribe: <mailto:tranthomas@example.com> List-Post: <mailto:tranthomas@example.com> List-Id: <pdfbox-dev.incubator.apache.org> Reply-To: tranthomas@example.com Delivered-To: mailing list tranthomas@example.com Received: (qmail 53640 invoked by uid 99); 2 Mar 2009 20:52:55 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2009 12:52:55 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of tranthomas@example.com designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail-fx0-f167.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2009 20:52:49 +0000 Received: by fxm11 with SMTP id 11so2109669fxm.12 for <tranthomas@example.com>; Mon, 02 Mar 2009 12:52:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=1qBgHx3/w0ZF8vKMgXII22imHfi+qKFbQEVXUvN9IE8=; b=Mp8vQmHmup3O+N2vqiplPsQ0l2pm56TKKYsfAD2ozh/Di863xXXFPYu1+B4dRH2LHe pJGXwdi9tunk7P+snHeMXCxx/eY+tz0m9/vXp2cEhxOlnYpblBds8ZM391dkJG9IH+GM r2CsTVuxLhVKYE5gGP4rwfYmxLOZtppozWeKo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=aW59oqtGFHReundrkEVY+wSILozl3ZApdS6Bpb/mEeXR168BVMTLjy3PXLHo7SRx3b SZgO/Hyb04dKGfRZ1LkeJ2ubvOP2c9+PPTNGOEp68G2ulG6B4vU8Zc34BO6zQqSATkQx rNKsi0/VCl6thwjys4O99tO9NyMDB8WR6wlFY= MIME-Version: 1.0 Received: by 83.161.120.155 with SMTP id p18mr2246866bkh.173.1236027147574; Mon, 02 Mar 2009 12:52:27 -0800 (PST) In-Reply-To: <000301c99b4a$88716db0$99544910$@com> References: <607873083.1234177019902.JavaMail.jira@brutus> <519624715.1234866179682.JavaMail.jira@brutus> <000301c99b4a$88716db0$99544910$@com> Date: Mon, 2 Mar 2009 21:52:27 +0100 Message-ID: <tranthomas@example.com> Subject: Re: [jira] Commented: (PDFBOX-420) Japanese Characters are garbled. From: Jukka Zitting <tranthomas@example.com> To: tranthomas@example.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Hi,
{'source': 'Apache Software Foundation Public Mail Archives'}