From pylucene-devchoijoshua@example.net Tue Feb 03 14:47:11 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 14538 invoked from network); 3 Feb 2009 14:47:11 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Feb 2009 14:47:11 -0000 Received: (qmail 11142 invoked by uid 500); 3 Feb 2009 14:47:11 -0000 Delivered-To: choijoshua@example.net Received: (qmail 11125 invoked by uid 500); 3 Feb 2009 14:47:11 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Received: (qmail 11092 invoked by uid 99); 3 Feb 2009 14:47:11 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Feb 2009 06:47:10 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO mail-qy0-f20.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Feb 2009 14:47:01 +0000 Received: by qyk13 with SMTP id 13so2963503qyk.5 for <choijoshua@example.net>; Tue, 03 Feb 2009 06:46:40 -0800 (PST) Received: by 83.161.120.155 with SMTP id r5mr8458812qaj.37.1233672400471; Tue, 03 Feb 2009 06:46:40 -0800 (PST) Received: from ?83.161.120.155? (pool-173-48-164-75.bstnma.fios.verizon.net [83.161.120.155]) by mx.google.com with ESMTPS id k44sm12636131rnd.16.283.161.120.155.46.39 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 03 Feb 2009 06:46:40 -0800 (PST) Message-Id: <choijoshua@example.net> From: Michael McCandless <choijoshua@example.net> To: choijoshua@example.net Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Subject: using PyLucene from Java Date: Tue, 3 Feb 2009 09:46:38 -0500 X-Mailer: Apple Mail (2.930.3) X-Virus-Checked: Checked by ClamAV on apache.org One quick question about PyLucene/JCC: is it "symmetric" at runtime, ie, my "main" could be in Java and I can init access to Python and then invoke Python sources? EG maybe I created a Python function for indexing a bunch of files, but the rest of my app in in Java, so I want to create my writer in Java, pass it to the Python function, and then continue using the writer in Java. (I realize JCC allows subclassing of Java classes to in Python, eg the Analyzers examples). Mike From pylucene-devchoijoshua@example.net Tue Feb 03 15:01:13 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 22578 invoked from network); 3 Feb 2009 15:01:13 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Feb 2009 15:01:13 -0000 Received: (qmail 36482 invoked by uid 500); 3 Feb 2009 15:01:13 -0000 Delivered-To: choijoshua@example.net Received: (qmail 36469 invoked by uid 500); 3 Feb 2009 15:01:13 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Delivered-To: moderator for choijoshua@example.net Received: (qmail 13205 invoked by uid 99); 3 Feb 2009 14:48:59 -0000 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Message-Id: <choijoshua@example.net> From: Michael McCandless <choijoshua@example.net> To: choijoshua@example.net Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Subject: trying to build PyLucene off Lucene's trunk Date: Tue, 3 Feb 2009 09:48:24 -0500 X-Mailer: Apple Mail (2.930.3) X-Virus-Checked: Checked by ClamAV on apache.org I'm trying build PyLucene off of Lucene's current trunk, on OS X 10.5.6. So I first did "ant package-tgz" in a clean Lucene checkout, to get a tgz with all interesting JARS (side note: it'd be nice if JCC could get the JARs out of the tgz or zip file). Then I tried to run the example at http://lucene.apache.org/pylucene/jcc/documentation/readme.html but hit a few initial challenges: * Where does extensions.jar come from? (I'm trying to run jcc, and apparently I need to point it to extensions.jar). * I'm getting a cryptic error from JCC when it tries to init the JVM (I'm on OS X 10.5.6): _NSJVMLoadLibrary: NSAddLibrary failed for /System/Library/Frameworks/ JavaVM.framework/Versions/CurrentJDK/Libraries/libjvm.dylib JavaVM FATAL: Failed to load the jvm library. Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/runpy.py", line 95, in run_module filename, loader, alter_sys) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/runpy.py", line 52, in _run_module_code mod_name, mod_fname, mod_loader) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/runpy.py", line 32, in _run_code exec code in run_globals File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/site-packages/JCC-2.1-py2.5-macosx-10.3-i386.egg/jcc/ __init__.py", line 29, in <module> import jcc.__main__ File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/site-packages/JCC-2.1-py2.5-macosx-10.3-i386.egg/jcc/ __main__.py", line 5, in <module> cpp.jcc(sys.argv) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/site-packages/JCC-2.1-py2.5-macosx-10.3-i386.egg/jcc/ cpp.py", line 343, in jcc vmargs='-Djava.awt.headless=true') ValueError: An error occurred while creating Java VM I'm using 1.6.0 Java VM. Maybe I'm supposed to use 1.5? Mike From pylucene-devchoijoshua@example.net Tue Feb 03 19:15:06 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 96361 invoked from network); 3 Feb 2009 19:15:06 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Feb 2009 19:15:06 -0000 Received: (qmail 89139 invoked by uid 500); 3 Feb 2009 19:15:05 -0000 Delivered-To: choijoshua@example.net Received: (qmail 89109 invoked by uid 500); 3 Feb 2009 19:15:05 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Received: (qmail 89098 invoked by uid 99); 3 Feb 2009 19:15:05 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Feb 2009 11:15:05 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO mail10.dslextreme.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with SMTP; Tue, 03 Feb 2009 19:14:57 +0000 Received: (qmail 10524 invoked from network); 3 Feb 2009 19:14:34 -0000 Received: from unknown (HELO ovaltofu.org) (83.161.120.155) by mail10.dslextreme.com with (DHE-RSA-AES256-SHA encrypted) SMTP; Tue, 03 Feb 2009 11:14:34 -0800 Received: from [83.161.120.155] ([83.161.120.155]) (authenticated bits=0) by ovaltofu.org (8.13.8/8.13.5) with ESMTP id n13JG2n3027999 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for <choijoshua@example.net>; Tue, 3 Feb 2009 11:16:02 -0800 (PST) Date: Tue, 3 Feb 2009 11:15:15 -0800 (PST) From: Andi Vajda <choijoshua@example.net> X-X-Sender: choijoshua@example.net Reply-To: Andi Vajda <choijoshua@example.net> To: choijoshua@example.net Subject: Re: using PyLucene from Java In-Reply-To: <choijoshua@example.net> Message-ID: <choijoshua@example.net> References: <choijoshua@example.net> User-Agent: Alpine 2.00 (OSX 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Checked: Checked by ClamAV on apache.org On Tue, 3 Feb 2009, Michael McCandless wrote: > One quick question about PyLucene/JCC: is it "symmetric" at runtime, > ie, my "main" could be in Java and I can init access to Python and > then invoke Python sources? > > EG maybe I created a Python function for indexing a bunch of files, but > the rest of my app in in Java, so I want to create my writer in Java, pass > it to the Python function, and then continue using the writer in Java. > > (I realize JCC allows subclassing of Java classes to in Python, eg the > Analyzers examples). Yes, you can do this (embed a Python VM in a Java VM) if your JCC-built extension is built with shared mode, using --shared. I use this reverse embedding when running Python code inside Tomcat, for example. In order to access Python code from Java you have to: 1. write a Java class that declares the 'native' methods that JCC is going to hook into (and generate code for) to invoke your Python code 2. write a Python class that 'extends' this Java class. (see http://lucene.apache.org/pylucene/jcc/documentation/readme.html#extensions) 3. be sure to have the path to the JCC's libjcc.so on -D java.library.path and the resulting egg on your CLASSPATH 4. initialize the Python VM from Java (this part is not yet documented as this is a rather new feature, actually): import org.apache.jcc.PythonVM; import your.fancy.extension; .... PythonVM.start(programName /* a String, decoration really */, args /* a String[] passed to Python's init); .... vm = PythonVM.get(); /* here, in instantiate, we're referring to your python class as in: from your.fancy import extension */ ext = (your.fancy.extension) vm.instantiate("your.fancy", "extension"); ... use the ext instance as any other Java object ... Andi.. From pylucene-devchoijoshua@example.net Tue Feb 03 19:17:21 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 96722 invoked from network); 3 Feb 2009 19:17:21 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Feb 2009 19:17:21 -0000 Received: (qmail 91985 invoked by uid 500); 3 Feb 2009 19:17:21 -0000 Delivered-To: choijoshua@example.net Received: (qmail 91958 invoked by uid 500); 3 Feb 2009 19:17:21 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Received: (qmail 91947 invoked by uid 99); 3 Feb 2009 19:17:20 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Feb 2009 11:17:20 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO mail10.dslextreme.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with SMTP; Tue, 03 Feb 2009 19:17:13 +0000 Received: (qmail 10209 invoked from network); 3 Feb 2009 19:16:51 -0000 Received: from unknown (HELO ovaltofu.org) (83.161.120.155) by mail10.dslextreme.com with (DHE-RSA-AES256-SHA encrypted) SMTP; Tue, 03 Feb 2009 11:16:51 -0800 Received: from [83.161.120.155] ([83.161.120.155]) (authenticated bits=0) by ovaltofu.org (8.13.8/8.13.5) with ESMTP id n13JIIJA011318 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for <choijoshua@example.net>; Tue, 3 Feb 2009 11:18:19 -0800 (PST) Date: Tue, 3 Feb 2009 11:17:31 -0800 (PST) From: Andi Vajda <choijoshua@example.net> X-X-Sender: choijoshua@example.net Reply-To: Andi Vajda <choijoshua@example.net> To: choijoshua@example.net Subject: Re: trying to build PyLucene off Lucene's trunk In-Reply-To: <choijoshua@example.net> Message-ID: <choijoshua@example.net> References: <choijoshua@example.net> User-Agent: Alpine 2.00 (OSX 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Checked: Checked by ClamAV on apache.org On Tue, 3 Feb 2009, Michael McCandless wrote: > > I'm trying build PyLucene off of Lucene's current trunk, on OS X > 10.5.6. > > So I first did "ant package-tgz" in a clean Lucene checkout, to get a > tgz with all interesting JARS (side note: it'd be nice if JCC could > get the JARs out of the tgz or zip file). PyLucene's Makefile does all of that for you. As described here: http://lucene.apache.org/pylucene/documentation/install.html 1. build JCC 2. use the Makefile to build PyLucene > Then I tried to run the example at > http://lucene.apache.org/pylucene/jcc/documentation/readme.html but > hit a few initial challenges: > > * Where does extensions.jar come from? (I'm trying to run jcc, and > apparently I need to point it to extensions.jar). PyLucene's Makefile builds it for you. > * I'm getting a cryptic error from JCC when it tries to init the > JVM (I'm on OS X 10.5.6): > > _NSJVMLoadLibrary: NSAddLibrary failed for > /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Libraries/libjvm.dylib > JavaVM FATAL: Failed to load the jvm library. Could it be a 64-bit vs 32-bit VM issue. On Mac OS X, Python is 32-bit unless you build your own. Be sure to use the corresponding Java VM. Andi.. > > Traceback (most recent call last): > File > "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/runpy.py", > line 95, in run_module > filename, loader, alter_sys) > File > "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/runpy.py", > line 52, in _run_module_code > mod_name, mod_fname, mod_loader) > File > "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/runpy.py", > line 32, in _run_code > exec code in run_globals > File > "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/JCC-2.1-py2.5-macosx-10.3-i386.egg/jcc/__init__.py", > line 29, in <module> > import jcc.__main__ > File > "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/JCC-2.1-py2.5-macosx-10.3-i386.egg/jcc/__main__.py", > line 5, in <module> > cpp.jcc(sys.argv) > File > "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/JCC-2.1-py2.5-macosx-10.3-i386.egg/jcc/cpp.py", > line 343, in jcc > vmargs='-Djava.awt.headless=true') > ValueError: An error occurred while creating Java VM > > I'm using 1.6.0 Java VM. Maybe I'm supposed to use 1.5? > > Mike From pylucene-devchoijoshua@example.net Tue Feb 03 19:35:55 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 5730 invoked from network); 3 Feb 2009 19:35:55 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Feb 2009 19:35:55 -0000 Received: (qmail 24449 invoked by uid 500); 3 Feb 2009 19:35:55 -0000 Delivered-To: choijoshua@example.net Received: (qmail 24431 invoked by uid 500); 3 Feb 2009 19:35:55 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Received: (qmail 24420 invoked by uid 99); 3 Feb 2009 19:35:55 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Feb 2009 11:35:55 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO yw-out-2324.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Feb 2009 19:35:46 +0000 Received: by yw-out-2324.google.com with SMTP id 3so691646ywj.5 for <choijoshua@example.net>; Tue, 03 Feb 2009 11:35:24 -0800 (PST) Received: by 83.161.120.155 with SMTP id l16mr1483020agb.3.1233689723511; Tue, 03 Feb 2009 11:35:23 -0800 (PST) Received: from ?83.161.120.155? (pool-173-48-164-75.bstnma.fios.verizon.net [83.161.120.155]) by mx.google.com with ESMTPS id 7sm10317253agb.0.283.161.120.155.35.22 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 03 Feb 2009 11:35:22 -0800 (PST) Message-Id: <choijoshua@example.net> From: Michael McCandless <choijoshua@example.net> To: choijoshua@example.net In-Reply-To: <choijoshua@example.net> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Subject: Re: using PyLucene from Java Date: Tue, 3 Feb 2009 14:35:21 -0500 References: <choijoshua@example.net> <choijoshua@example.net> X-Mailer: Apple Mail (2.930.3) X-Virus-Checked: Checked by ClamAV on apache.org Excellent, thanks Andi. Mike Andi Vajda wrote: > > On Tue, 3 Feb 2009, Michael McCandless wrote: > >> One quick question about PyLucene/JCC: is it "symmetric" at runtime, >> ie, my "main" could be in Java and I can init access to Python and >> then invoke Python sources? >> >> EG maybe I created a Python function for indexing a bunch of files, >> but >> the rest of my app in in Java, so I want to create my writer in >> Java, pass >> it to the Python function, and then continue using the writer in >> Java. >> >> (I realize JCC allows subclassing of Java classes to in Python, eg >> the >> Analyzers examples). > > Yes, you can do this (embed a Python VM in a Java VM) if your JCC- > built extension is built with shared mode, using --shared. > > I use this reverse embedding when running Python code inside Tomcat, > for example. > > In order to access Python code from Java you have to: > 1. write a Java class that declares the 'native' methods that JCC > is going > to hook into (and generate code for) to invoke your Python code > 2. write a Python class that 'extends' this Java class. > (see http://lucene.apache.org/pylucene/jcc/documentation/readme.html#extensions) > 3. be sure to have the path to the JCC's libjcc.so on -D > java.library.path > and the resulting egg on your CLASSPATH > 4. initialize the Python VM from Java (this part is not yet > documented as > this is a rather new feature, actually): > > import org.apache.jcc.PythonVM; > import your.fancy.extension; > > .... > PythonVM.start(programName /* a String, decoration really */, > args /* a String[] passed to Python's init); > .... > > vm = PythonVM.get(); > /* here, in instantiate, we're referring to your python class as > in: from your.fancy import extension > */ > ext = (your.fancy.extension) > vm.instantiate("your.fancy", "extension"); > > ... use the ext instance as any other Java object ... > > Andi.. From pylucene-devchoijoshua@example.net Tue Feb 03 19:39:01 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 6325 invoked from network); 3 Feb 2009 19:38:59 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 3 Feb 2009 19:38:59 -0000 Received: (qmail 26701 invoked by uid 500); 3 Feb 2009 19:38:59 -0000 Delivered-To: choijoshua@example.net Received: (qmail 26667 invoked by uid 500); 3 Feb 2009 19:38:59 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Received: (qmail 26656 invoked by uid 99); 3 Feb 2009 19:38:59 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Feb 2009 11:38:59 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO rn-out-0910.google.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Feb 2009 19:38:51 +0000 Received: by rn-out-0910.google.com with SMTP id 56so1583292rnw.4 for <choijoshua@example.net>; Tue, 03 Feb 2009 11:38:27 -0800 (PST) Received: by 83.161.120.155 with SMTP id e3mr112817agc.53.1233689907355; Tue, 03 Feb 2009 11:38:27 -0800 (PST) Received: from ?83.161.120.155? (pool-173-48-164-75.bstnma.fios.verizon.net [83.161.120.155]) by mx.google.com with ESMTPS id 7sm10210766agd.39.283.161.120.155.38.23 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 03 Feb 2009 11:38:27 -0800 (PST) Message-Id: <choijoshua@example.net> From: Michael McCandless <choijoshua@example.net> To: choijoshua@example.net In-Reply-To: <choijoshua@example.net> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Subject: Re: trying to build PyLucene off Lucene's trunk Date: Tue, 3 Feb 2009 14:38:21 -0500 References: <choijoshua@example.net> <choijoshua@example.net> X-Mailer: Apple Mail (2.930.3) X-Virus-Checked: Checked by ClamAV on apache.org On Feb 3, 2009, at 2:17 PM, Andi Vajda wrote: > > On Tue, 3 Feb 2009, Michael McCandless wrote: > >> >> I'm trying build PyLucene off of Lucene's current trunk, on OS X >> 10.5.6. >> >> So I first did "ant package-tgz" in a clean Lucene checkout, to get a >> tgz with all interesting JARS (side note: it'd be nice if JCC could >> get the JARs out of the tgz or zip file). > > PyLucene's Makefile does all of that for you. > As described here: > http://lucene.apache.org/pylucene/documentation/install.html > > 1. build JCC > 2. use the Makefile to build PyLucene Duh, I had missed the Makefile! Gotta read instructions better... >> Then I tried to run the example at >> http://lucene.apache.org/pylucene/jcc/documentation/readme.html but >> hit a few initial challenges: >> >> * Where does extensions.jar come from? (I'm trying to run jcc, and >> apparently I need to point it to extensions.jar). > > PyLucene's Makefile builds it for you. OK. >> * I'm getting a cryptic error from JCC when it tries to init the >> JVM (I'm on OS X 10.5.6): >> >> _NSJVMLoadLibrary: NSAddLibrary failed for /System/Library/ >> Frameworks/JavaVM.framework/Versions/CurrentJDK/Libraries/ >> libjvm.dylib >> JavaVM FATAL: Failed to load the jvm library. > > Could it be a 64-bit vs 32-bit VM issue. On Mac OS X, Python is 32- > bit unless you build your own. Be sure to use the corresponding Java > VM. Good point, looks like this is my problem. Thanks! Mike From pylucene-devchoijoshua@example.net Thu Feb 05 19:49:45 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 85367 invoked from network); 5 Feb 2009 19:49:45 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 5 Feb 2009 19:49:45 -0000 Received: (qmail 55140 invoked by uid 500); 5 Feb 2009 19:49:45 -0000 Delivered-To: choijoshua@example.net Received: (qmail 55113 invoked by uid 500); 5 Feb 2009 19:49:45 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Delivered-To: moderator for choijoshua@example.net Received: (qmail 46612 invoked by uid 99); 5 Feb 2009 19:45:52 -0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of choijoshua@example.net designates 83.161.120.155 as permitted sender) Date: Thu, 5 Feb 2009 14:45:21 -0500 From: Aaron Lav <choijoshua@example.net> To: choijoshua@example.net Cc: choijoshua@example.net Subject: Occasional SIGSEGV passing lists of ints to Java int[]s Message-ID: <choijoshua@example.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="C7zPtVaVf+AK4Oqc" Content-Disposition: inline User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Checked: Checked by ClamAV on apache.org --C7zPtVaVf+AK4Oqc Content-Type: text/plain; charset=us-ascii Content-Disposition: inline It seems like there's a bug which sometimes causes passing lists of ints to Java int[]s to generate a SIGSEGV. The crash occurs in Python's listitem.c while iterating over a list (verified by comparing the RIP reported in the hs_err_pid<pid> file against a disassembly). It tries to access a->ob_item[i], which turns out to be NULL, and thus faults when trying to increment the reference count with y_INCREF(a->ob_item[i]); I've verified this by looking through the list with gdb on a core dump: the list pointer is available in a register, the object header of the list has the right reference count, length, and type, and the other elements in the list seem to have the right refcount (1), type (PyInt_Type), and plausible integer values. This occurs for me under ubuntu/x86_64 with the Hotspot JVM (reproduced on both gutsy/sun-java6-jdk/python2.4 and intrepid/openjdk-6-jdk/python2.5.2), probably more often on multicore/multiprocessor machines. Building JCC with or without NO_SHARED doesn't seem to make any difference. It seems to happen both with -Xcheck:jni as a VM option, and without, and with JCC versions as late as r740966. I'm attaching a test case (python and java files) to this email: I've also put a tarball at http://www.pobox.com/~asl2/jcc-jarray.tar.bz2 which has a prebuilt jar, a build.sh script to build both the jar and the JCC wrapper, a run.sh script to repeatedly run the test code (it doesn't always fault, but I'd expect that if it doesn't do so after 50-100 tries, it's not going to in that environment), and a sample hs_err_pid logfile. Does this reproduce for anyone, or sound familiar? Thanks, Aaron Lav (choijoshua@example.net) --C7zPtVaVf+AK4Oqc Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="test_array.py" import _testjcc import random def setupEnv(): if not _testjcc.getVMEnv(): _testjcc.initVM(classpath='.', vmargs="-Xcheck:jni") def test(): count = 2000000 a = [int(random.randrange(-1, 70000)) for _ in range(count)] b = [int(random.randrange(-1, 32)) for _ in range(count)] factory = _testjcc.TestJcc(a,b) if __name__ == '__main__': setupEnv() test() --C7zPtVaVf+AK4Oqc Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="TestJcc.java" package org.dyndns.asl2.TestJcc; public class TestJcc { public TestJcc(int a[], int b[]) { } } --C7zPtVaVf+AK4Oqc-- From pylucene-devchoijoshua@example.net Fri Feb 06 01:39:26 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 35344 invoked from network); 6 Feb 2009 01:39:25 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 6 Feb 2009 01:39:25 -0000 Received: (qmail 24977 invoked by uid 500); 6 Feb 2009 01:39:25 -0000 Delivered-To: choijoshua@example.net Received: (qmail 24953 invoked by uid 500); 6 Feb 2009 01:39:25 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Delivered-To: moderator for choijoshua@example.net Received: (qmail 76654 invoked by uid 99); 6 Feb 2009 00:44:40 -0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of choijoshua@example.net designates 83.161.120.155 as permitted sender) Date: Thu, 5 Feb 2009 19:44:10 -0500 From: Aaron Lav <choijoshua@example.net> To: choijoshua@example.net Subject: Re: Occasional SIGSEGV passing lists of ints to Java int[]s Message-ID: <choijoshua@example.net> References: <choijoshua@example.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="0OAP2g/MAC+5xKAE" Content-Disposition: inline In-Reply-To: <choijoshua@example.net> User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Checked: Checked by ClamAV on apache.org --0OAP2g/MAC+5xKAE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thu, Feb 05, 2009 at 02:45:21PM -0500, Aaron Lav wrote: > It seems like there's a bug which sometimes causes passing lists of > ints to Java int[]s to generate a SIGSEGV. > ... I've noticed that a call to the wrapped functions doesn't seem to be necessary to generate the exception: all that's required is that the module be imported and the JVM initalized. The list then seems to have an element set to NULL, and any access, whether from Python or from jcc trying to convert it to a JArray<int>, will fault. (I realized this while trying to figure out why my hardware watchpoints weren't triggering.) I'm attaching a revised test_array.py which still generates a SIGSEGV, and a gdb session. (The NULL tends to show up at one or two offsets, although the what offsets those are may vary with the code and environment running, which is how I knew to look at 0x1a92a4 in the gdb session.) Aaron Lav (choijoshua@example.net) --0OAP2g/MAC+5xKAE Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=gdb_trace This GDB was configured as "x86_64-linux-gnu"... (no debugging symbols found) (gdb) set args test_array.py (gdb) break org888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4yndns888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4sl888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4TestJ888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4t_TestJcc_init_(org888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4yndns888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4sl888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4TestJ888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4t_TestJcc*, _object*, _object*) Function "org888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4yndns888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4sl888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4TestJ888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4t_TestJcc_init_(org888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4yndns888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4sl888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4TestJ888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4t_TestJcc*, _object*, _object*)" not defined. Make breakpoint pending on future shared library load? (y or [n]) y Breakpoint 1 (org888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4yndns888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4sl888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4TestJ888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4t_TestJcc_init_(org888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4yndns888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4sl888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4TestJ888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4t_TestJcc*, _object*, _object*)) pending. (gdb) run Starting program: /usr/bin/python test_array.py (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) [Thread debugging using libthread_db enabled] (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) [New Thread 0x7fc22d7fc6e0 (LWP 11766)] (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) [New Thread 0x417ae950 (LWP 11769)] [New Thread 0x418af950 (LWP 11770)] [New Thread 0x41630950 (LWP 11771)] [New Thread 0x41347950 (LWP 11772)] [New Thread 0x41448950 (LWP 11773)] [New Thread 0x41cca950 (LWP 11774)] [New Thread 0x41dcb950 (LWP 11775)] [New Thread 0x41ecc950 (LWP 11776)] [New Thread 0x41fcd950 (LWP 11777)] [New Thread 0x40f8f950 (LWP 11778)] 0x7fc22d714d88 0x7fc22d714f80 [Switching to Thread 0x7fc22d7fc6e0 (LWP 11766)] ---Type <return> to continue, or q <return> to quit--- Breakpoint 1, 0x00007fc22c7c6f10 in org888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4yndns888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4sl888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4TestJ888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4t_TestJcc_init_ () from /home/asl2/repos/people/alav/ticket_4583/build/lib.linux-x86_64-2.4/testjcc/_testjcc.so (gdb) x/20g 0x7fc22d714f80 0x7fc22d714f80: 0x0000000000000002 0x0000000000700280 0x7fc22d714f90: 0x00000000001e847e 0x00007fc1e4139010 0x7fc22d714fa0: 0x000000000021a1a7 0x0000000000000000 0x7fc22d714fb0: 0x00007fc22d714f60 0x00000000fffffffe 0x7fc22d714fc0: 0x0000000000000000 0x0000000000000000 0x7fc22d714fd0: 0x0000000000700280 0x00000000001e847e 0x7fc22d714fe0: 0x00007fc1e284e7a0 0x00000000001e847e 0x7fc22d714ff0: 0x0000000000000000 0x0000000000000000 0x7fc22d715000: 0x000000000000003f 0x0000000000000000 0x7fc22d715010: 0x0000000000702920 0x0000000000702920 (gdb) x/10g 0x00007fc1e4139010 + (8*0x1a92a4) 0x7fc1e4e82530: 0x0000000000000000 0x00007fc1da064a58 0x7fc1e4e82540: 0x00007fc1da064a40 0x00007fc1da064a28 0x7fc1e4e82550: 0x00007fc1da0651d8 0x00007fc1da0651c0 0x7fc1e4e82560: 0x00007fc1da0651a8 0x00007fc1da065190 0x7fc1e4e82570: 0x00007fc1da065178 0x00007fc1da065160 --0OAP2g/MAC+5xKAE Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="test_array.py" import _testjcc def test(): count = 999999 # crashes count = 950000 # doesn't count = 975000 # maybe 1 in 4? sometimes count = 999999 * 2 a = [i for i in range(count)] b = [i for i in range(count)] print hex(id(a)), hex(id(b)) for j in a: assert type(j) == int for j in b: assert type(j) == int # factory = _testjcc.TestJcc(a,b) if __name__ == '__main__': # initalizing the VM is required to generate the error. _testjcc.initVM(classpath='.', vmargs="-Xcheck:jni") test() --0OAP2g/MAC+5xKAE-- From pylucene-devchoijoshua@example.net Fri Feb 06 06:42:24 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 48614 invoked from network); 6 Feb 2009 06:42:23 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 6 Feb 2009 06:42:23 -0000 Received: (qmail 14958 invoked by uid 500); 6 Feb 2009 06:42:22 -0000 Delivered-To: choijoshua@example.net Received: (qmail 14939 invoked by uid 500); 6 Feb 2009 06:42:22 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Received: (qmail 14893 invoked by uid 99); 6 Feb 2009 06:42:22 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 05 Feb 2009 22:42:22 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO mail10.dslextreme.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with SMTP; Fri, 06 Feb 2009 06:42:14 +0000 Received: (qmail 17851 invoked from network); 6 Feb 2009 06:41:52 -0000 Received: from unknown (HELO ovaltofu.org) (83.161.120.155) by mail10.dslextreme.com with (DHE-RSA-AES256-SHA encrypted) SMTP; Thu, 05 Feb 2009 22:41:52 -0800 Received: from [83.161.120.155] ([83.161.120.155]) (authenticated bits=0) by ovaltofu.org (8.13.8/8.13.5) with ESMTP id n166gDo9000145 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for <choijoshua@example.net>; Thu, 5 Feb 2009 22:42:13 -0800 (PST) Date: Thu, 5 Feb 2009 22:42:32 -0800 (PST) From: Andi Vajda <choijoshua@example.net> X-X-Sender: choijoshua@example.net Reply-To: Andi Vajda <choijoshua@example.net> To: choijoshua@example.net Subject: Re: Occasional SIGSEGV passing lists of ints to Java int[]s In-Reply-To: <choijoshua@example.net> Message-ID: <choijoshua@example.net> References: <choijoshua@example.net> <choijoshua@example.net> User-Agent: Alpine 2.00 (OSX 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Checked: Checked by ClamAV on apache.org On Thu, 5 Feb 2009, Aaron Lav wrote: > On Thu, Feb 05, 2009 at 02:45:21PM -0500, Aaron Lav wrote: >> It seems like there's a bug which sometimes causes passing lists of >> ints to Java int[]s to generate a SIGSEGV. >> ... > > I've noticed that a call to the wrapped functions doesn't seem to be > necessary to generate the exception: all that's required is that the > module be imported and the JVM initalized. The list then seems to > have an element set to NULL, and any access, whether from Python or > from jcc trying to convert it to a JArray<int>, will fault. > > (I realized this while trying to figure out why my hardware > watchpoints weren't triggering.) > > I'm attaching a revised test_array.py which still generates a SIGSEGV, > and a gdb session. > > (The NULL tends to show up at one or two offsets, although the > what offsets those are may vary with the code and environment running, which > is how I knew to look at 0x1a92a4 in the gdb session.) Strange indeed. I looked over this a bit but didn't try to reproduce it yet. I didn't see anything odd. If the mere _presence_ of JCC and calling initVM() causes the arrays to get corrupted, this could be hard to debug. Have you tried moving things around, like creating the arrays differently ? For example: a = list(xrange(count)) instead of the list comprehension ? Did you try a more recent version of Python ? Have you tried this on 32-bit ? When you don't actually use the arrays after creating them, how are you getting the crash ? inside the assert loops ? Andi.. From pylucene-devchoijoshua@example.net Fri Feb 06 18:07:40 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 747 invoked from network); 6 Feb 2009 18:07:40 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 6 Feb 2009 18:07:40 -0000 Received: (qmail 93430 invoked by uid 500); 6 Feb 2009 18:07:40 -0000 Delivered-To: choijoshua@example.net Received: (qmail 93410 invoked by uid 500); 6 Feb 2009 18:07:39 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Received: (qmail 93399 invoked by uid 99); 6 Feb 2009 18:07:39 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Feb 2009 10:07:39 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of choijoshua@example.net designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail1.panix.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Feb 2009 18:07:30 +0000 Received: from panix5.panix.com (panix5.panix.com [83.161.120.155]) by mail1.panix.com (Postfix) with ESMTP id EA77A29412 for <choijoshua@example.net>; Fri, 6 Feb 2009 13:07:09 -0500 (EST) Received: by panix5.panix.com (Postfix, from userid 19385) id C98A824255; Fri, 6 Feb 2009 13:07:09 -0500 (EST) Date: Fri, 6 Feb 2009 13:07:09 -0500 From: Aaron Lav <choijoshua@example.net> To: choijoshua@example.net Subject: Re: Occasional SIGSEGV passing lists of ints to Java int[]s Message-ID: <choijoshua@example.net> References: <choijoshua@example.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <choijoshua@example.net> User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Checked: Checked by ClamAV on apache.org On Thu, Feb 05, 2009 at 02:45:21PM -0500, Aaron Lav wrote: > > (apologies for the broken threading. I don't seem to be getting email from this list: I've tried resubscribing.) > Have you tried moving things around, like creating the arrays differently ? For example: a = list(xrange(count)) instead of the list comprehension ? I've tried list(xrange(count)) and range(count), and those don't fail. a = [] for i in range(count): a.append(i) b = [] for i in range(count): b.append(i) does fail. > Did you try a more recent version of Python ? I've just tried locally building 2.6.1 on my gutsy laptop, and it still fails with that. > Have you tried this on 32-bit ? I've built an i386-arch KVM VM, and it doesn't crash there. (It does crash on a very similar x86-64 VM.) > When you don't actually use the arrays after creating them, how are you getting the crash ? inside the assert loops ? Yes. (Some print statements would have made that clearer.) Other ideas I'm planning: * try a python --with-pydebug build * stub out parts of jcc initialization. (Obviously this could just move the bug around.) * fiddle with the JVM's GC parameters * see if setting a hardware watchpoint earlier catches where the data is being changed, or if it seems to be being put in the list wrong. Aaron Lav (choijoshua@example.net) From pylucene-devchoijoshua@example.net Fri Feb 06 18:44:50 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 19827 invoked from network); 6 Feb 2009 18:44:50 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 6 Feb 2009 18:44:50 -0000 Received: (qmail 45827 invoked by uid 500); 6 Feb 2009 18:44:50 -0000 Delivered-To: choijoshua@example.net Received: (qmail 45802 invoked by uid 500); 6 Feb 2009 18:44:50 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Received: (qmail 45791 invoked by uid 99); 6 Feb 2009 18:44:50 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Feb 2009 10:44:50 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mail10.dslextreme.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with SMTP; Fri, 06 Feb 2009 18:44:42 +0000 Received: (qmail 5726 invoked from network); 6 Feb 2009 18:44:20 -0000 Received: from unknown (HELO ovaltofu.org) (83.161.120.155) by mail10.dslextreme.com with (DHE-RSA-AES256-SHA encrypted) SMTP; Fri, 06 Feb 2009 10:44:20 -0800 Received: from [83.161.120.155] ([83.161.120.155]) (authenticated bits=0) by ovaltofu.org (8.13.8/8.13.5) with ESMTP id n16IiQYT015436 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for <choijoshua@example.net>; Fri, 6 Feb 2009 10:44:28 -0800 (PST) References: <choijoshua@example.net> <choijoshua@example.net> Message-Id: <choijoshua@example.net> From: Andi Vajda <choijoshua@example.net> To: "choijoshua@example.net" <choijoshua@example.net> In-Reply-To: <choijoshua@example.net> Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit X-Mailer: iPod Mail (5G77) Mime-Version: 1.0 (iPod Mail 5G77) Subject: Re: Occasional SIGSEGV passing lists of ints to Java int[]s Date: Fri, 6 Feb 2009 10:44:56 -0800 Cc: "choijoshua@example.net" <choijoshua@example.net> X-Virus-Checked: Checked by ClamAV on apache.org On Feb 6, 2009, at 10:07, Aaron Lav <choijoshua@example.net> wrote: > On Thu, Feb 05, 2009 at 02:45:21PM -0500, Aaron Lav wrote: >> >> > (apologies for the broken threading. I don't seem to be > getting email from this list: I've tried resubscribing.) > > >> Have you tried moving things around, like creating the arrays >> differently ? > For example: a = list(xrange(count)) instead of the list > comprehension ? > > I've tried list(xrange(count)) and range(count), and those don't fail. > > a = [] > for i in range(count): > a.append(i) > b = [] > for i in range(count): > b.append(i) > > does fail. > >> Did you try a more recent version of Python ? > > I've just tried locally building 2.6.1 on my gutsy laptop, and > it still fails with that. > >> Have you tried this on 32-bit ? > > I've built an i386-arch KVM VM, and it doesn't crash there. (It > does crash on a very similar x86-64 VM.) > >> When you don't actually use the arrays after creating them, how are >> you > getting the crash ? inside the assert loops ? > > Yes. (Some print statements would have made that clearer.) > > Other ideas I'm planning: > * try a python --with-pydebug build > * stub out parts of jcc initialization. (Obviously this could just > move the bug around.) > * fiddle with the JVM's GC parameters > * see if setting a hardware watchpoint earlier catches where the > data is being changed, or if it seems to be being put in the list > wrong. Does it crash if you don't call initVM() ? Andi.. > > > Aaron Lav (choijoshua@example.net) > From pylucene-devchoijoshua@example.net Fri Feb 06 18:49:34 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 21773 invoked from network); 6 Feb 2009 18:49:34 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 6 Feb 2009 18:49:34 -0000 Received: (qmail 53189 invoked by uid 500); 6 Feb 2009 18:49:34 -0000 Delivered-To: choijoshua@example.net Received: (qmail 53172 invoked by uid 500); 6 Feb 2009 18:49:34 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Received: (qmail 53159 invoked by uid 99); 6 Feb 2009 18:49:34 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Feb 2009 10:49:34 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of choijoshua@example.net designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail2.panix.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Feb 2009 18:49:26 +0000 Received: from panix5.panix.com (panix5.panix.com [83.161.120.155]) by mail2.panix.com (Postfix) with ESMTP id 46F5E34802 for <choijoshua@example.net>; Fri, 6 Feb 2009 13:49:05 -0500 (EST) Received: by panix5.panix.com (Postfix, from userid 19385) id 2BE0124255; Fri, 6 Feb 2009 13:49:05 -0500 (EST) Date: Fri, 6 Feb 2009 13:49:05 -0500 From: Aaron Lav <choijoshua@example.net> To: choijoshua@example.net Subject: Re: Occasional SIGSEGV passing lists of ints to Java int[] Message-ID: <choijoshua@example.net> References: <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <choijoshua@example.net> User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Checked: Checked by ClamAV on apache.org On Fri, Feb 06, 2009 at 10:44:56AM -0800, Andi Vajda wrote: > > On Feb 6, 2009, at 10:07, Aaron Lav <choijoshua@example.net> wrote: > >> On Thu, Feb 05, 2009 at 02:45:21PM -0500, Aaron Lav wrote: >>> >>> >> (apologies for the broken threading. I don't seem to be >> getting email from this list: I've tried resubscribing.) >> >> >>> Have you tried moving things around, like creating the arrays >>> differently ? >> For example: a = list(xrange(count)) instead of the list comprehension >> ? >> >> I've tried list(xrange(count)) and range(count), and those don't fail. >> >> a = [] >> for i in range(count): >> a.append(i) >> b = [] >> for i in range(count): >> b.append(i) >> >> does fail. >> >>> Did you try a more recent version of Python ? >> >> I've just tried locally building 2.6.1 on my gutsy laptop, and >> it still fails with that. >> >>> Have you tried this on 32-bit ? >> >> I've built an i386-arch KVM VM, and it doesn't crash there. (It >> does crash on a very similar x86-64 VM.) >> >>> When you don't actually use the arrays after creating them, how are >>> you >> getting the crash ? inside the assert loops ? >> >> Yes. (Some print statements would have made that clearer.) >> >> Other ideas I'm planning: >> * try a python --with-pydebug build >> * stub out parts of jcc initialization. (Obviously this could just >> move the bug around.) >> * fiddle with the JVM's GC parameters >> * see if setting a hardware watchpoint earlier catches where the data >> is being changed, or if it seems to be being put in the list wrong. > > Does it crash if you don't call initVM() ? No, the call to _testjcc.initVM(...) seems to be required to make it crash. Aaron Lav (choijoshua@example.net) From pylucene-devchoijoshua@example.net Fri Feb 06 19:07:11 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 30622 invoked from network); 6 Feb 2009 19:07:11 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 6 Feb 2009 19:07:11 -0000 Received: (qmail 84600 invoked by uid 500); 6 Feb 2009 19:07:11 -0000 Delivered-To: choijoshua@example.net Received: (qmail 84572 invoked by uid 500); 6 Feb 2009 19:07:11 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Received: (qmail 84559 invoked by uid 99); 6 Feb 2009 19:07:11 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Feb 2009 11:07:11 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mail10.dslextreme.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with SMTP; Fri, 06 Feb 2009 19:07:03 +0000 Received: (qmail 8653 invoked from network); 6 Feb 2009 19:06:42 -0000 Received: from unknown (HELO ovaltofu.org) (83.161.120.155) by mail10.dslextreme.com with (DHE-RSA-AES256-SHA encrypted) SMTP; Fri, 06 Feb 2009 11:06:42 -0800 Received: from [83.161.120.155] ([83.161.120.155]) (authenticated bits=0) by ovaltofu.org (8.13.8/8.13.5) with ESMTP id n16J6nuX002321 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for <choijoshua@example.net>; Fri, 6 Feb 2009 11:06:50 -0800 (PST) Date: Fri, 6 Feb 2009 11:07:24 -0800 (PST) From: Andi Vajda <choijoshua@example.net> X-X-Sender: choijoshua@example.net Reply-To: Andi Vajda <choijoshua@example.net> To: choijoshua@example.net Subject: Re: Occasional SIGSEGV passing lists of ints to Java int[] In-Reply-To: <choijoshua@example.net> Message-ID: <choijoshua@example.net> References: <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> User-Agent: Alpine 2.00 (OSX 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Checked: Checked by ClamAV on apache.org On Fri, 6 Feb 2009, Aaron Lav wrote: >> Does it crash if you don't call initVM() ? > > No, the call to _testjcc.initVM(...) seems to be required to > make it crash. There are two pieces to initVM(): - initVM() proper (defined in jcc.cpp) - initializing your classes The initVM() that is called from Python is a function called __initialize__() that is generated by JCC. It's defined in a file called __init__.cpp. It first calls the actual initVM() and then calls the __initialize__() on each top level package JCC generates wrappers for. For example, PyLucene's __initialize__() looks like: PyObject *__initialize__(PyObject *module, PyObject *args, PyObject *kwds) { PyObject *env = initVM(module, args, kwds); if (env == NULL) return NULL; jav888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4__initialize__(module); org888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4__initialize__(module); return env; } Does it still crash if you comment out the calls to __initialize__(module) that the top level __initialize__() makes ? Andi.. From pylucene-devchoijoshua@example.net Fri Feb 06 19:20:27 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 36370 invoked from network); 6 Feb 2009 19:20:27 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 6 Feb 2009 19:20:27 -0000 Received: (qmail 4453 invoked by uid 500); 6 Feb 2009 19:20:27 -0000 Delivered-To: choijoshua@example.net Received: (qmail 4437 invoked by uid 500); 6 Feb 2009 19:20:27 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Received: (qmail 4426 invoked by uid 99); 6 Feb 2009 19:20:27 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Feb 2009 11:20:27 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of choijoshua@example.net designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail1.panix.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Feb 2009 19:20:20 +0000 Received: from panix5.panix.com (panix5.panix.com [83.161.120.155]) by mail1.panix.com (Postfix) with ESMTP id 569AB29415; Fri, 6 Feb 2009 14:19:57 -0500 (EST) Received: by panix5.panix.com (Postfix, from userid 19385) id 3EF7D24255; Fri, 6 Feb 2009 14:19:57 -0500 (EST) Date: Fri, 6 Feb 2009 14:19:57 -0500 From: Aaron Lav <choijoshua@example.net> To: choijoshua@example.net Cc: choijoshua@example.net Subject: Re: Occasional SIGSEGV passing lists of ints to Java int[] Message-ID: <choijoshua@example.net> References: <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <choijoshua@example.net> User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Checked: Checked by ClamAV on apache.org On Fri, Feb 06, 2009 at 11:07:24AM -0800, Andi Vajda wrote: > > On Fri, 6 Feb 2009, Aaron Lav wrote: > >>> Does it crash if you don't call initVM() ? >> >> No, the call to _testjcc.initVM(...) seems to be required to >> make it crash. > > There are two pieces to initVM(): > - initVM() proper (defined in jcc.cpp) > - initializing your classes > > The initVM() that is called from Python is a function called > __initialize__() that is generated by JCC. It's defined in a file called > __init__.cpp. It first calls the actual initVM() and then calls the > __initialize__() on each top level package JCC generates wrappers for. > > For example, PyLucene's __initialize__() looks like: > > PyObject *__initialize__(PyObject *module, PyObject *args, PyObject *kwds) > { > PyObject *env = initVM(module, args, kwds); > > if (env == NULL) > return NULL; > > jav888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4__initialize__(module); > org888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4__initialize__(module); > > return env; > } > > Does it still crash if you comment out the calls to > __initialize__(module) that the top level __initialize__() makes ? Yes. I commented out the lines #for name, entries in packages: # line(out, indent + 1, '%s888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4__initialize__(module);', name) from python.py, removed the build directory and rebuilt (checking that the 'jav888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4__initialize__' and 'org888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4__initialize__' calls were gone from __init__.cpp), and reran, and it still faults. Aaron (choijoshua@example.net) From pylucene-devchoijoshua@example.net Fri Feb 06 19:53:27 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 47055 invoked from network); 6 Feb 2009 19:53:27 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 6 Feb 2009 19:53:27 -0000 Received: (qmail 52365 invoked by uid 500); 6 Feb 2009 19:53:27 -0000 Delivered-To: choijoshua@example.net Received: (qmail 52344 invoked by uid 500); 6 Feb 2009 19:53:27 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Received: (qmail 52333 invoked by uid 99); 6 Feb 2009 19:53:27 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Feb 2009 11:53:27 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.161.120.155] (HELO mail10.dslextreme.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with SMTP; Fri, 06 Feb 2009 19:53:20 +0000 Received: (qmail 10028 invoked from network); 6 Feb 2009 19:52:58 -0000 Received: from unknown (HELO ovaltofu.org) (83.161.120.155) by mail10.dslextreme.com with (DHE-RSA-AES256-SHA encrypted) SMTP; Fri, 06 Feb 2009 11:52:58 -0800 Received: from [83.161.120.155] (h-67-103-33-25.snfccasy.dynamic.covad.net [83.161.120.155]) (authenticated bits=0) by ovaltofu.org (8.13.8/8.13.5) with ESMTP id n16Jr52n016765 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Fri, 6 Feb 2009 11:53:07 -0800 (PST) References: <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> Message-Id: <choijoshua@example.net> From: Andi Vajda <choijoshua@example.net> To: "choijoshua@example.net" <choijoshua@example.net> In-Reply-To: <choijoshua@example.net> Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (iPod Mail 5G77) Subject: Re: Occasional SIGSEGV passing lists of ints to Java int[] Date: Fri, 6 Feb 2009 11:50:14 -0800 Cc: "choijoshua@example.net" <choijoshua@example.net>, "choijoshua@example.net" <choijoshua@example.net> X-Mailer: iPod Mail (5G77) X-Virus-Checked: Checked by ClamAV on apache.org On Feb 6, 2009, at 11:19, Aaron Lav <choijoshua@example.net> wrote: > On Fri, Feb 06, 2009 at 11:07:24AM -0800, Andi Vajda wrote: >> >> On Fri, 6 Feb 2009, Aaron Lav wrote: >> >>>> Does it crash if you don't call initVM() ? >>> >>> No, the call to _testjcc.initVM(...) seems to be required to >>> make it crash. >> >> There are two pieces to initVM(): >> - initVM() proper (defined in jcc.cpp) >> - initializing your classes >> >> The initVM() that is called from Python is a function called >> __initialize__() that is generated by JCC. It's defined in a file >> called >> __init__.cpp. It first calls the actual initVM() and then calls the >> __initialize__() on each top level package JCC generates wrappers >> for. >> >> For example, PyLucene's __initialize__() looks like: >> >> PyObject *__initialize__(PyObject *module, PyObject *args, PyObject >> *kwds) >> { >> PyObject *env = initVM(module, args, kwds); >> >> if (env == NULL) >> return NULL; >> >> jav888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4__initialize__(module); >> org888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4__initialize__(module); >> >> return env; >> } >> >> Does it still crash if you comment out the calls to >> __initialize__(module) that the top level __initialize__() makes ? > > Yes. I commented out the lines > #for name, entries in packages: > # line(out, indent + 1, '%s888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4__initialize__(module);', name) > from python.py, removed the build directory and rebuilt > (checking that the 'jav888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4__initialize__' and > 'org888a:dbb7:2d49:b3f6:8605:bf4e:caed:e5a4__initialize__' calls were gone from __init__.cpp), and reran, > and it still faults. Ok, so keeping these commented out, how much can you comment out of the actual initVM() defined in jcc.cpp until it no longer crashes ? To make rebuilds fast, use shared mode and just rebuild jcc so that you get a new libjcc.so (be sure to call install). With non shared mode, you can edit the copy of jcc.cpp JCC copied into your extension's build tree and re-build with --install. Does turning --debug on have an effect on the crash ? (I expect that to also turn compiler optimizations off -O0) Andi.. > > > Aaron (choijoshua@example.net) From pylucene-devchoijoshua@example.net Fri Feb 06 21:23:24 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 73011 invoked from network); 6 Feb 2009 21:23:24 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 6 Feb 2009 21:23:24 -0000 Received: (qmail 86476 invoked by uid 500); 6 Feb 2009 21:23:23 -0000 Delivered-To: choijoshua@example.net Received: (qmail 86450 invoked by uid 500); 6 Feb 2009 21:23:23 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Received: (qmail 86439 invoked by uid 99); 6 Feb 2009 21:23:23 -0000 Received: from athena.apache.org (HELO athena.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Feb 2009 13:23:23 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of choijoshua@example.net designates 83.161.120.155 as permitted sender) Received: from [83.161.120.155] (HELO mail2.panix.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Feb 2009 21:23:15 +0000 Received: from panix5.panix.com (panix5.panix.com [83.161.120.155]) by mail2.panix.com (Postfix) with ESMTP id 23D073480E; Fri, 6 Feb 2009 16:22:55 -0500 (EST) Received: by panix5.panix.com (Postfix, from userid 19385) id 0CBA524255; Fri, 6 Feb 2009 16:22:54 -0500 (EST) Date: Fri, 6 Feb 2009 16:22:54 -0500 From: Aaron Lav <choijoshua@example.net> To: choijoshua@example.net Cc: choijoshua@example.net Subject: Re: Occasional SIGSEGV passing lists of ints to Java int[] Message-ID: <choijoshua@example.net> References: <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> <choijoshua@example.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <choijoshua@example.net> User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Checked: Checked by ClamAV on apache.org On Fri, Feb 06, 2009 at 11:50:14AM -0800, Andi Vajda wrote: > > Ok, so keeping these commented out, how much can you comment out of the > actual initVM() defined in jcc.cpp until it no longer crashes ? Unfortunately, if I add Py_INCREF(Py_None); return Py_None; just before if (JNI_CreateJavaVM(&vm, (void **) &vm_env, &vm_args) < 0) it doesn't crash: if I put it just after, then it does. So either initializing the JVM perturbs the environment enough to cause the bug to manifest, or the code responsible is somewhere in Hotspot. > --debug on have an effect on the crash ? (I expect that to also turn > compiler optimizations off -O0) No effect I noticed for --debug. I checked the compiler command output, and it did include -O0. Aaron (choijoshua@example.net) From pylucene-devchoijoshua@example.net Fri Feb 06 21:44:42 2009 Return-Path: <pylucene-devchoijoshua@example.net> Delivered-To: choijoshua@example.net Received: (qmail 92977 invoked from network); 6 Feb 2009 21:44:42 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur.apache.org with SMTP; 6 Feb 2009 21:44:42 -0000 Received: (qmail 26888 invoked by uid 500); 6 Feb 2009 21:44:42 -0000 Delivered-To: choijoshua@example.net Received: (qmail 26871 invoked by uid 500); 6 Feb 2009 21:44:42 -0000 Mailing-List: contact choijoshua@example.net; run by ezmlm Precedence: bulk List-Help: <mailto:choijoshua@example.net> List-Unsubscribe: <mailto:choijoshua@example.net> List-Post: <mailto:choijoshua@example.net> List-Id: <pylucene-dev.lucene.apache.org> Reply-To: choijoshua@example.net Delivered-To: mailing list choijoshua@example.net Received: (qmail 26860 invoked by uid 99); 6 Feb 2009 21:44:42 -0000 Received: from nike.apache.org (HELO nike.apache.org) (83.161.120.155) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Feb 2009 13:44:42 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [83.161.120.155] (HELO mail10.dslextreme.com) (83.161.120.155) by apache.org (qpsmtpd/0.29) with SMTP; Fri, 06 Feb 2009 21:44:33 +0000 Received: (qmail 32300 invoked from network); 6 Feb 2009 21:44:11 -0000 Received: from unknown (HELO ovaltofu.org) (83.161.120.155) by mail10.dslextreme.com with
{'source': 'Apache Software Foundation Public Mail Archives'}