From httpclient-user-returnmelendeztammy@example.org Tue Feb 01 16:18:51 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 39773 invoked from network); 1 Feb 2005 16:18:51 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 1 Feb 2005 16:18:51 -0000 Received: (qmail 15776 invoked by uid 500); 1 Feb 2005 16:18:51 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 15760 invoked by uid 500); 1 Feb 2005 16:18:50 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Delivered-To: moderator for melendeztammy@example.org Received: (qmail 11055 invoked by uid 99); 1 Feb 2005 16:16:56 -0000 X-ASF-Spam-Status: No, hits=0.5 required=10.0 tests=DNS_FROM_RFC_WHOIS,FORGED_RCVD_HELO X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: local policy) Message-ID: <melendeztammy@example.org> Date: Tue, 01 Feb 2005 17:18:43 +0100 From: Frederic Houbie <melendeztammy@example.org> User-Agent: Mozilla Thunderbird 0.8 (Windows/20040913) X-Accept-Language: en-us, en MIME-Version: 1.0 To: melendeztammy@example.org Subject: Session management Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Hi all, I have a special thing to do. I explain myself. I have an application that I made which is a catalog of product to sell, with full of information. A part of it is the search catalog part, which allow you to fill your shopping basket. This application is made on a database. On the same database, I have a Ecommerce web application with payment and so on. I would like to transfer my shopping basket to this application (I know how, it is a struts application and I've got the sources), then redirect to the e commerce application final checkout page. My problem is to know how I could share my session between these two webapps. Or, can I initiate the session on the ecommerce application, fill the basket then open a web browser that will use the session initiated by httpclient ? I need help..... thanks Frederic --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org From httpclient-user-returnmelendeztammy@example.org Tue Feb 01 22:29:11 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 61244 invoked from network); 1 Feb 2005 22:29:11 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 1 Feb 2005 22:29:11 -0000 Received: (qmail 63205 invoked by uid 500); 1 Feb 2005 22:29:06 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 63171 invoked by uid 500); 1 Feb 2005 22:29:05 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Received: (qmail 63148 invoked by uid 99); 1 Feb 2005 22:29:05 -0000 X-ASF-Spam-Status: No, hits=0.2 required=10.0 tests=NO_REAL_NAME,RCVD_BY_IP,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: domain of melendeztammy@example.org designates 83.161.120.155 as permitted sender) Received: from wproxy.gmail.com (HELO wproxy.gmail.com) (83.161.120.155) by apache.org (qpsmtpd/0.28) with ESMTP; Tue, 01 Feb 2005 14:29:02 -0800 Received: by wproxy.gmail.com with SMTP id 71so1196810wri for <melendeztammy@example.org>; Tue, 01 Feb 2005 14:28:56 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding; b=Cv9R6Xaa8ykq/Tt/ISJSbT6ATfh1te6wLAkpaNKX/S6yyi4Hiat5/iQrkohXEIeilte1CLu6T5P4+9r402n9Hii6C3OjEAadL0ge4dewevslnC0Ith8l5cloyClxOtgP+C+h9gs3Wz0YZp71jW+oE06Hn1hOOs0CtQlGFyofM7c= Received: by 83.161.120.155 with SMTP id o50mr72128wro; Tue, 01 Feb 2005 14:28:56 -0800 (PST) Received: by 83.161.120.155 with HTTP; Tue, 1 Feb 2005 14:28:56 -0800 (PST) Message-ID: <melendeztammy@example.org> Date: Tue, 1 Feb 2005 23:28:56 +0100 From: <melendeztammy@example.org> Reply-To: <melendeztammy@example.org> To: melendeztammy@example.org Subject: interrupt a httpclient connection Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Hi, I'm trying to write some kind of web browser using httpclient. And I found one problem with implementing STOP button. I cannot find any way, how to stop outgoing/incoming data. I found some conversation about same problem http://issues.apache.org/bugzilla/show_bug.cgi?id=20288 and it looks that this problem was solved somehow, but in current version of http client (2.0.2, 3.0), I cannot find these changes. Anyone has idea, how can I implement browser stop button (stop all data transfer immediately). Thanks for any help or hint. Petr --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org From httpclient-user-returnmelendeztammy@example.org Tue Feb 01 22:48:08 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 70384 invoked from network); 1 Feb 2005 22:48:08 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 1 Feb 2005 22:48:08 -0000 Received: (qmail 93339 invoked by uid 500); 1 Feb 2005 22:48:04 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 93279 invoked by uid 500); 1 Feb 2005 22:48:04 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Received: (qmail 93206 invoked by uid 99); 1 Feb 2005 22:48:04 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: neutral (hermes.apache.org: local policy) Received: from mail21.bluewin.ch (HELO mail21.bluewin.ch) (83.161.120.155) by apache.org (qpsmtpd/0.28) with ESMTP; Tue, 01 Feb 2005 14:48:02 -0800 Received: from xbox.localdomain (83.161.120.155) by mail21.bluewin.ch (Bluewin AG 7.0.035) id 41DEABCE00269A44 for melendeztammy@example.org; Tue, 1 Feb 2005 22:47:55 +0000 Received: from 83.161.120.155 (unknown [83.161.120.155]) by xbox.localdomain (Postfix) with ESMTP id C9893B6ECB for <melendeztammy@example.org>; Tue, 1 Feb 2005 23:47:57 +0100 (CET) Subject: Re: interrupt a httpclient connection From: Oleg Kalnichevski <melendeztammy@example.org> To: melendeztammy@example.org In-Reply-To: <melendeztammy@example.org> References: <melendeztammy@example.org> Content-Type: text/plain Date: Tue, 01 Feb 2005 23:47:54 +0100 Message-Id: <melendeztammy@example.org> Mime-Version: 1.0 X-Mailer: Evolution 2.0.2 (2.0.2-3) Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Petr, Use HttpMethod#abort method of the 3.0 API. Please note this method is not available in HttpClient 2.0.x Hope this helps, Oleg On Tue, 2005-02-01 at 23:28 +0100, melendeztammy@example.org wrote: > Hi, > > I'm trying to write some kind of web browser using httpclient. And I > found one problem with implementing STOP button. I cannot find any > way, how to stop outgoing/incoming data. > > I found some conversation about same problem > http://issues.apache.org/bugzilla/show_bug.cgi?id=20288 and it looks > that this problem was solved somehow, but in current version of http > client (2.0.2, 3.0), I cannot find these changes. > > Anyone has idea, how can I implement browser stop button (stop all > data transfer immediately). Thanks for any help or hint. > > Petr > > --------------------------------------------------------------------- > To unsubscribe, e-mail: melendeztammy@example.org > For additional commands, e-mail: melendeztammy@example.org > --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org From httpclient-user-returnmelendeztammy@example.org Wed Feb 02 04:15:43 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 99225 invoked from network); 2 Feb 2005 04:15:43 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 2 Feb 2005 04:15:43 -0000 Received: (qmail 74042 invoked by uid 500); 2 Feb 2005 04:15:43 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 74023 invoked by uid 500); 2 Feb 2005 04:15:42 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Received: (qmail 74009 invoked by uid 99); 2 Feb 2005 04:15:42 -0000 X-ASF-Spam-Status: No, hits=0.1 required=10.0 tests=FORGED_RCVD_HELO X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: local policy) Received: from savgw1.sandcherry.com (HELO savgw1.sccorp.com) (83.161.120.155) by apache.org (qpsmtpd/0.28) with SMTP; Tue, 01 Feb 2005 20:15:42 -0800 Received: from exchange01.sccorp.com ([83.161.120.155]) by savgw1.sccorp.com (SAVSMTP 83.161.120.155) with SMTP id M2005020121153922295 for <melendeztammy@example.org>; Tue, 01 Feb 2005 21:15:39 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5.6944.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: HttpClient under load Date: Tue, 1 Feb 2005 21:14:51 -0700 Message-ID: <melendeztammy@example.org> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: HttpClient under load Thread-Index: AcUIr52zIHQur+4rSEm5WGHMNX8MHAAK9arw From: "Prasad Sethumadhavan" <melendeztammy@example.org> To: "HttpClient User Discussion" <melendeztammy@example.org> X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Hi, I need some help or pointers regarding using the HttpClient under heavy load. I am currently using the HttpClient under moderate load (50 concurrent threads). Each thread instatiates a new HttpClient client, uses it and does the method.releaseConnection() before exiting. SimpleHttpConnectionManager manager =3D new SimpleHttpConnectionManager(); manager.getParams().setStaleCheckingEnabled(true); manager.closeIdleConnections(1000L); manager.getParams().setConnectionTimeout( (int) DEFAULT_CONNECTION_TIMEOUT); =20 httpclient =3D new HttpClient(manager); httpclient.getParams().setCookiePolicy( CookiePolicy.BROWSER_COMPATIBILITY); However when I run this setup under load, I keep getting socket exceptions on the client side. I am using tomcat web server and I have increased the max-processor & access-count. I don't see any errors or exception in the tomcat log. However I keep getting a lot of these client socket bind exceptions after sometime. I guess it is happening due to the unavailability of ports. I would like to know some best practices in using the HttpClient under load. I am sure tha it is my bad, but wanted to get some input from the user/dev community. Should I have a pool of HttpClient objects and reuse them? Should I use the multi-threaded connection manager? I need to run this under heavier load and need to fix this soon. Any help, code snippets, or guidance would be greatly appreciated. Please let me know if you need any other information. Thanks in advance Prasad Client StackTrace: [ERROR] 01 Feb 2005 20:43:56,177 NDC[] T[Thread-20] HTTP HTTP I/O transport exception occurred=20 java.net.BindException: Address already in use: melendeztammy@example.org(Native Method) at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:364) at java.net.Socket.connect(Socket.java:507) at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor Impl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSoc ket(ReflectionSocketFactory.java:128) at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.crea teSocket(DefaultProtocolSocketFactory.java:124) at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:70 5) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMe thodDirector.java:359) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMetho dDirector.java:168) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:3 93) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:3 24) --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org From httpclient-user-returnmelendeztammy@example.org Wed Feb 02 11:12:09 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 3665 invoked from network); 2 Feb 2005 11:12:08 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 2 Feb 2005 11:12:08 -0000 Received: (qmail 59910 invoked by uid 500); 2 Feb 2005 11:12:07 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 59778 invoked by uid 500); 2 Feb 2005 11:12:06 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Received: (qmail 59676 invoked by uid 99); 2 Feb 2005 11:12:05 -0000 X-ASF-Spam-Status: No, hits=0.2 required=10.0 tests=NO_REAL_NAME,RCVD_BY_IP,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: domain of melendeztammy@example.org designates 83.161.120.155 as permitted sender) Received: from wproxy.gmail.com (HELO wproxy.gmail.com) (83.161.120.155) by apache.org (qpsmtpd/0.28) with ESMTP; Wed, 02 Feb 2005 03:12:04 -0800 Received: by wproxy.gmail.com with SMTP id 71so56909wri for <melendeztammy@example.org>; Wed, 02 Feb 2005 03:12:01 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=fogoEo2tvYOWu5+tJ22qjUMZcL3kAXLoULVFd2twZHwSZ/xSwxc8nttNnfGa4HDknOHUL8QHO05+m9gPFaE2/gm3/fAaaFtuZHsgKTVdVI0w8tS+OBbe3frAv2bahSOu79ZXUSh1DgEClqo+UPrgiaXyAmxGWTk2kqxzaiAzMjA= Received: by 83.161.120.155 with SMTP id 9mr245101wrp; Wed, 02 Feb 2005 03:12:00 -0800 (PST) Received: by 83.161.120.155 with HTTP; Wed, 2 Feb 2005 03:12:00 -0800 (PST) Message-ID: <melendeztammy@example.org> Date: Wed, 2 Feb 2005 12:12:00 +0100 From: <melendeztammy@example.org> Reply-To: <melendeztammy@example.org> To: HttpClient User Discussion <melendeztammy@example.org> Subject: Re: interrupt a httpclient connection In-Reply-To: <melendeztammy@example.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <melendeztammy@example.org> <melendeztammy@example.org> X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Thanks a lot, client.abort() works great! When searching documentation and source codes of new version, I missed this method somehow.. Petr --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org From httpclient-user-returnmelendeztammy@example.org Wed Feb 02 13:38:51 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 75711 invoked from network); 2 Feb 2005 13:38:51 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 2 Feb 2005 13:38:51 -0000 Received: (qmail 22105 invoked by uid 500); 2 Feb 2005 13:38:51 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 22083 invoked by uid 500); 2 Feb 2005 13:38:50 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Received: (qmail 22069 invoked by uid 99); 2 Feb 2005 13:38:50 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: local policy) Received: from mxout6.cac.washington.edu (HELO mxout6.cac.washington.edu) (83.161.120.155) by apache.org (qpsmtpd/0.28) with ESMTP; Wed, 02 Feb 2005 05:38:49 -0800 Received: from smtp.washington.edu (smtp.washington.edu [83.161.120.155]) by mxout6.cac.washington.edu (8.13.1+UW04.08/8.13.3+UW05.01) with ESMTP id j12DclY0008596 for <melendeztammy@example.org>; Wed, 2 Feb 2005 05:38:47 -0800 Received: from [83.161.120.155] (pool-+1-561-916-4685.bos.east.verizon.net [83.161.120.155]) (authenticated bits=0) by smtp.washington.edu (8.13.1+UW04.08/8.13.3+UW05.01) with ESMTP id j12DcjVg028572 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NOT) for <melendeztammy@example.org>; Wed, 2 Feb 2005 05:38:46 -0800 Mime-Version: 1.0 (Apple Message framework v619) In-Reply-To: <melendeztammy@example.org> References: <melendeztammy@example.org> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <melendeztammy@example.org> Content-Transfer-Encoding: 7bit From: Michael Becke <melendeztammy@example.org> Subject: Re: HttpClient under load Date: Wed, 2 Feb 2005 08:38:40 -0500 To: "HttpClient User Discussion" <melendeztammy@example.org> X-Mailer: Apple Mail (2.619) X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Hi Prasad, Given the scenario you describe below you are more likely leaking one connection/socket per request. You definitely want to reuse instances of HttpClient and especially instances of a ConnectionManager. You can either use the multi-threaded connection manager from all threads or have a dedicated SimpleHttpConnectionManager per thread. Please have a look at the performance guide for some more ideas <http://jakarta.apache.org/commons/httpclient/3.0/performance.html>. Also please note that calling manager.closeIdleConnections(1000L) only closes connections that have already been idle for 1000ms. It does not continuously poll for idle connections, closing them when they've been idle for 1000ms. Mike On Feb 1, 2005, at 11:14 PM, Prasad Sethumadhavan wrote: > Hi, > I need some help or pointers regarding using the HttpClient under heavy > load. I am currently using the HttpClient under moderate load (50 > concurrent threads). Each thread instatiates a new HttpClient client, > uses it and does the method.releaseConnection() before exiting. > > SimpleHttpConnectionManager manager = new > SimpleHttpConnectionManager(); > manager.getParams().setStaleCheckingEnabled(true); > manager.closeIdleConnections(1000L); > manager.getParams().setConnectionTimeout( > (int) DEFAULT_CONNECTION_TIMEOUT); > > httpclient = new HttpClient(manager); > httpclient.getParams().setCookiePolicy( > CookiePolicy.BROWSER_COMPATIBILITY); > > However when I run this setup under load, I keep getting socket > exceptions on the client side. I am using tomcat web server and I have > increased the max-processor & access-count. I don't see any errors or > exception in the tomcat log. > > However I keep getting a lot of these client socket bind exceptions > after sometime. I guess it is happening due to the unavailability of > ports. I would like to know some best practices in using the HttpClient > under load. I am sure tha it is my bad, but wanted to get some input > from the user/dev community. Should I have a pool of HttpClient objects > and reuse them? Should I use the multi-threaded connection manager? I > need to run this under heavier load and need to fix this soon. Any > help, > code snippets, or guidance would be greatly appreciated. Please let me > know if you need any other information. > > Thanks in advance > Prasad > > > Client StackTrace: > > [ERROR] 01 Feb 2005 20:43:56,177 NDC[] T[Thread-20] HTTP HTTP > I/O transport exception occurred > java.net.BindException: Address already in use: connect > at java.net.PlainSocketImpl.socketConnect(Native Method) > at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) > at > java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) > at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:364) > at java.net.Socket.connect(Socket.java:507) > at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso > r > Impl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at > org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSo > c > ket(ReflectionSocketFactory.java:128) > at > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.cre > a > teSocket(DefaultProtocolSocketFactory.java:124) > at > org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java: > 70 > 5) > at > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpM > e > thodDirector.java:359) > at > org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMeth > o > dDirector.java:168) > at > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java: > 3 > 93) > at > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java: > 3 > 24) > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: melendeztammy@example.org > For additional commands, e-mail: > melendeztammy@example.org > --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org From httpclient-user-returnmelendeztammy@example.org Wed Feb 02 14:32:26 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 22568 invoked from network); 2 Feb 2005 14:32:26 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 2 Feb 2005 14:32:26 -0000 Received: (qmail 32984 invoked by uid 500); 2 Feb 2005 14:32:25 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 32967 invoked by uid 500); 2 Feb 2005 14:32:25 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Received: (qmail 32953 invoked by uid 99); 2 Feb 2005 14:32:25 -0000 X-ASF-Spam-Status: No, hits=0.4 required=10.0 tests=DNS_FROM_RFC_ABUSE X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: local policy) Received: from web53209.mail.yahoo.com (HELO web53209.mail.yahoo.com) (83.161.120.155) by apache.org (qpsmtpd/0.28) with SMTP; Wed, 02 Feb 2005 06:32:23 -0800 Received: (qmail 1614 invoked by uid 60001); 2 Feb 2005 14:32:21 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=49FQ68r61XN6buV9LYmrE5j4FBHoTRAR7nJM6wZX0pKiUeUr4bY4p3I8kih53/40k7AnTXAJH94u7K/w5lDJGxbo3KvnD004g77P8wnnG70wlrvldBHlApn4A/Xdxz2woWYr5HUF+9XRByaKE5tkFCw4Qq7TV/umrH6kKYip5R0= ; Message-ID: <melendeztammy@example.org> Received: from [83.161.120.155] by web53209.mail.yahoo.com via HTTP; Wed, 02 Feb 2005 06:32:21 PST Date: Wed, 2 Feb 2005 06:32:21 -0800 (PST) From: ergodic 7 <melendeztammy@example.org> Subject: re:Downloading files with http client To: melendeztammy@example.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Is there any way of finding out the size of a file before downloading it via the get method? GetMethod method = new GetMethod(url); method.getResponseBody(); Regards __________________________________ Do you Yahoo!? Read only the mail you want - Yahoo! Mail SpamGuard. http://promotions.yahoo.com/new_mail --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org From httpclient-user-returnmelendeztammy@example.org Wed Feb 02 14:40:39 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 28105 invoked from network); 2 Feb 2005 14:40:39 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 2 Feb 2005 14:40:39 -0000 Received: (qmail 49655 invoked by uid 500); 2 Feb 2005 14:40:39 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 49632 invoked by uid 500); 2 Feb 2005 14:40:39 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Received: (qmail 49615 invoked by uid 99); 2 Feb 2005 14:40:39 -0000 X-ASF-Spam-Status: No, hits=0.1 required=10.0 tests=FORGED_RCVD_HELO X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: local policy) Received: from fw1-inet.semantico.net (HELO semantico.com) (83.161.120.155) by apache.org (qpsmtpd/0.28) with ESMTP; Wed, 02 Feb 2005 06:40:37 -0800 Received: from spampd.localdomain (localhost [83.161.120.155]) by localhost.semantico.com (Postfix) with ESMTP id 7E26F320210 for <melendeztammy@example.org>; Wed, 2 Feb 2005 14:40:34 +0000 (GMT) Received: from [83.161.120.155] (coriander.semantico.net [83.161.120.155]) by semantico.com (Postfix) with ESMTP id 29E1E320210 for <melendeztammy@example.org>; Wed, 2 Feb 2005 14:40:34 +0000 (GMT) Message-ID: <melendeztammy@example.org> Date: Wed, 02 Feb 2005 14:40:39 +0000 From: Dominic Mitchell <melendeztammy@example.org> Organization: Semantico User-Agent: Mozilla Thunderbird 0.9 (Windows/20041103) X-Accept-Language: en-us, en MIME-Version: 1.0 To: HttpClient User Discussion <melendeztammy@example.org> Subject: Re: Downloading files with http client References: <melendeztammy@example.org> In-Reply-To: <melendeztammy@example.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on paprika.rp.lan X-Spam-Status: No, hits=-100.0 required=5.0 tests=USER_IN_WHITELIST autolearn=no version=2.63 X-Spam-Level: X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N ergodic 7 wrote: > Is there any way of finding out the size of a file > before downloading it via the get method? > > GetMethod method = new GetMethod(url); > method.getResponseBody(); You could try using a HeadMethod and querying the Content-Length, but that's not guaranteed to be present... -Dom -- | Semantico: creators of major online resources | | URL: http://www.semantico.com/ | | Tel: +1-561-916-4685 / Fax: +1-561-916-4685 | | Address: 33 Bond St., Brighton, Sussex, BN1 1RD, UK. | --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org From httpclient-user-returnmelendeztammy@example.org Wed Feb 02 15:12:23 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 46455 invoked from network); 2 Feb 2005 15:12:23 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 2 Feb 2005 15:12:23 -0000 Received: (qmail 32178 invoked by uid 500); 2 Feb 2005 15:12:23 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 32142 invoked by uid 500); 2 Feb 2005 15:12:22 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Received: (qmail 32077 invoked by uid 99); 2 Feb 2005 15:12:22 -0000 X-ASF-Spam-Status: No, hits=0.4 required=10.0 tests=DNS_FROM_RFC_ABUSE,HTML_30_40,HTML_MESSAGE X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: local policy) Received: from mtagate2.de.ibm.com (HELO mtagate2.de.ibm.com) (83.161.120.155) by apache.org (qpsmtpd/0.28) with ESMTP; Wed, 02 Feb 2005 07:12:21 -0800 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [83.161.120.155]) by mtagate2.de.ibm.com (8.12.10/8.12.10) with ESMTP id j12FCEbP124208 for <melendeztammy@example.org>; Wed, 2 Feb 2005 15:12:14 GMT Received: from d12av04.megacenter.de.ibm.com (d12av04.megacenter.de.ibm.com [83.161.120.155]) by d12nrmr1607.megacenter.de.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id j12FCEM2118222 for <melendeztammy@example.org>; Wed, 2 Feb 2005 16:12:14 +0100 Received: from d12av04.megacenter.de.ibm.com (loopback [83.161.120.155]) by d12av04.megacenter.de.ibm.com (8.12.11/8.12.11) with ESMTP id j12FCEsl024102 for <melendeztammy@example.org>; Wed, 2 Feb 2005 16:12:14 +0100 Received: from d12ml067.megacenter.de.ibm.com (d12ml067.megacenter.de.ibm.com [83.161.120.155]) by d12av04.megacenter.de.ibm.com (8.12.11/8.12.11) with ESMTP id j12FCEv6024099 for <melendeztammy@example.org>; Wed, 2 Feb 2005 16:12:14 +0100 In-Reply-To: <melendeztammy@example.org> To: "HttpClient User Discussion" <melendeztammy@example.org> MIME-Version: 1.0 Subject: re:Downloading files with http client X-Mailer: Lotus Notes Release 6.0 September 26, 2002 From: Roland Weber <melendeztammy@example.org> X-MIMETrack: S/MIME Sign by Notes Client on Roland Weber/Germany/IBM(Release 6.0|September 26, 2002) at 02.02.2005 16:08:01, Serialize by Notes Client on Roland Weber/Germany/IBM(Release 6.0|September 26, 2002) at 02.02.2005 16:08:01, Serialize melendeztammy@example.org 16:08:01, S/MIME Sign melendeztammy@example.org 16:08:01: The cryptographic key was not found, Serialize by Router on D12ML067/12/M/IBM(Release 6.51HF338 | June 21, 2004) at 02/02/2005 16:12:13, Serialize complete at 02/02/2005 16:12:13 Message-ID: <melendeztammy@example.org> Date: Wed, 2 Feb 2005 16:12:13 +0100 Content-Type: multipart/alternative; boundary="=_alternative 005321ACC1256F9C_=" X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N --=_alternative 005321ACC1256F9C_= Content-Type: text/plain; charset="US-ASCII" Hello, ergodic 7 <melendeztammy@example.org> wrote on 02.02.2005 15:32:21: > Is there any way of finding out the size of a file > before downloading it via the get method? > > GetMethod method = new GetMethod(url); > method.getResponseBody(); > If method.getResponseContentLength() returns something different than -1, you have the length. But if the server doesn't send a "Content-Length" header, the length information is just not available. Either way, if you expect large files to be sent, you should *never* use method.getResponseBody(), because the response body will then be read into a byte array in memory. Instead, use method.getResponseBodyAsStream() to access the method body. This will also allow you to read the body in digestible chunks (like 16K at a time or so) and write it to a file on disk without clogging the JVM heap. That's if "downloading" refers to the act of reading the response body. If you want to know the size before even executing the GET request, you have to try the HEAD method as suggested by Dominic. hope that helps, Roland --=_alternative 005321ACC1256F9C_=-- From httpclient-user-returnmelendeztammy@example.org Thu Feb 03 00:06:34 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 76005 invoked from network); 3 Feb 2005 00:06:34 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 3 Feb 2005 00:06:34 -0000 Received: (qmail 10929 invoked by uid 500); 3 Feb 2005 00:06:34 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 10911 invoked by uid 500); 3 Feb 2005 00:06:33 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Received: (qmail 10895 invoked by uid 99); 3 Feb 2005 00:06:33 -0000 X-ASF-Spam-Status: No, hits=0.1 required=10.0 tests=FORGED_RCVD_HELO X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: local policy) Received: from savgw1.sandcherry.com (HELO savgw1.sccorp.com) (83.161.120.155) by apache.org (qpsmtpd/0.28) with SMTP; Wed, 02 Feb 2005 16:06:32 -0800 Received: from exchange01.sccorp.com ([83.161.120.155]) by savgw1.sccorp.com (SAVSMTP 83.161.120.155) with SMTP id M2005020217062926831 for <melendeztammy@example.org>; Wed, 02 Feb 2005 17:06:29 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5.6944.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: HttpClient under load Date: Wed, 2 Feb 2005 17:05:34 -0700 Message-ID: <melendeztammy@example.org> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: HttpClient under load Thread-Index: AcUJLAtNFGf0sKk9RBaLE74j6MhZsAARUTCg From: "Prasad Sethumadhavan" <melendeztammy@example.org> To: "HttpClient User Discussion" <melendeztammy@example.org> X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Hi Mike, Thanks for your input. I am now using a single HttpClient instance and a multi-threaded connection manager that is shared by all threads. It is not failing any more. I am trying to implement a "fetch timeout" feature using the http client. Basically it means that if I have not received a complete response within "n" seconds then I need to return a fetch timeout.=20 Is there any built in mechanism in HttpClient for this or do I need to have a monitor thread that calls HttpMethod.abort() after the timeout period (if the method has not finished execution). What would be the result of calling the abort method on the worker thread? (what exceptions etc..). Please advise if there are any better ways to implement the same. Thanks in advance Prasad -----Original Message----- From: Michael Becke [mailto:melendeztammy@example.org]=20 Sent: Wednesday, February 02, 2005 6:39 AM To: HttpClient User Discussion Subject: Re: HttpClient under load Hi Prasad, Given the scenario you describe below you are more likely leaking one connection/socket per request. You definitely want to reuse instances of HttpClient and especially instances of a ConnectionManager. You can either use the multi-threaded connection manager from all threads or have a dedicated SimpleHttpConnectionManager per thread. Please have a look at the performance guide for some more ideas <http://jakarta.apache.org/commons/httpclient/3.0/performance.html>. Also please note that calling manager.closeIdleConnections(1000L) only closes connections that have already been idle for 1000ms. It does not continuously poll for idle connections, closing them when they've been idle for 1000ms. Mike On Feb 1, 2005, at 11:14 PM, Prasad Sethumadhavan wrote: > Hi, > I need some help or pointers regarding using the HttpClient under=20 > heavy load. I am currently using the HttpClient under moderate load=20 > (50 concurrent threads). Each thread instatiates a new HttpClient=20 > client, uses it and does the method.releaseConnection() before exiting. > > SimpleHttpConnectionManager manager =3D new=20 > SimpleHttpConnectionManager(); > manager.getParams().setStaleCheckingEnabled(true); > manager.closeIdleConnections(1000L); > manager.getParams().setConnectionTimeout( > (int) DEFAULT_CONNECTION_TIMEOUT); > > httpclient =3D new HttpClient(manager); > httpclient.getParams().setCookiePolicy( > CookiePolicy.BROWSER_COMPATIBILITY); > > However when I run this setup under load, I keep getting socket=20 > exceptions on the client side. I am using tomcat web server and I have > increased the max-processor & access-count. I don't see any errors or=20 > exception in the tomcat log. > > However I keep getting a lot of these client socket bind exceptions=20 > after sometime. I guess it is happening due to the unavailability of=20 > ports. I would like to know some best practices in using the=20 > HttpClient under load. I am sure tha it is my bad, but wanted to get=20 > some input from the user/dev community. Should I have a pool of=20 > HttpClient objects and reuse them? Should I use the multi-threaded=20 > connection manager? I need to run this under heavier load and need to=20 > fix this soon. Any help, code snippets, or guidance would be greatly=20 > appreciated. Please let me know if you need any other information. > > Thanks in advance > Prasad > > > Client StackTrace: > > [ERROR] 01 Feb 2005 20:43:56,177 NDC[] T[Thread-20] HTTP HTTP > I/O transport exception occurred > java.net.BindException: Address already in use: connect > at java.net.PlainSocketImpl.socketConnect(Native Method) > at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) > at > java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) > at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:364) > at java.net.Socket.connect(Socket.java:507) > at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess > o > r > Impl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at > org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createS > o > c > ket(ReflectionSocketFactory.java:128) > at > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.cr > e > a > teSocket(DefaultProtocolSocketFactory.java:124) > at > org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java: > 70 > 5) > at > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(Http > M > e > thodDirector.java:359) > at > org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMet > h > o > dDirector.java:168) > at > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:=20 > 3 > 93) > at > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:=20 > 3 > 24) > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: melendeztammy@example.org > For additional commands, e-mail: =20 > melendeztammy@example.org > --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org From httpclient-user-returnmelendeztammy@example.org Thu Feb 03 00:22:15 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 81927 invoked from network); 3 Feb 2005 00:22:15 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 3 Feb 2005 00:22:15 -0000 Received: (qmail 34967 invoked by uid 500); 3 Feb 2005 00:22:14 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 34942 invoked by uid 500); 3 Feb 2005 00:22:14 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Received: (qmail 34922 invoked by uid 99); 3 Feb 2005 00:22:14 -0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: domain of melendeztammy@example.org designates 83.161.120.155 as permitted sender) Received: from c2ksmtpg.cucbc.com (HELO c2ksmtpg.cucbc.com) (83.161.120.155) by apache.org (qpsmtpd/0.28) with ESMTP; Wed, 02 Feb 2005 16:22:12 -0800 Received: from c2kmail1.cucbc.com ([83.161.120.155]) by c2ksmtpg.cucbc.com with Microsoft SMTPSVC(5.0.2195.6713); Wed, 2 Feb 2005 16:22:11 -0800 Received: c2kmail1.cucbc.com 83.161.120.155 from 83.161.120.155 83.161.120.155 via HTTP with MS-WebStorage 6.0.6249 Received: from juliusmasutanilx by c2kmail1.cucbc.com; 02 Feb 2005 16:22:10 -0800 Subject: RE: HttpClient under load From: Julius Davies <melendeztammy@example.org> To: HttpClient User Discussion <melendeztammy@example.org> In-Reply-To: <melendeztammy@example.org> References: <melendeztammy@example.org> Content-Type: text/plain Content-Transfer-Encoding: 7bit Date: Wed, 02 Feb 2005 16:22:10 -0800 Message-Id: <melendeztammy@example.org> Mime-Version: 1.0 X-Mailer: Evolution 2.0.3 X-OriginalArrivalTime: 03 Feb 2005 00:22:11.0260 (UTC) FILETIME=[674267C0:01C50986] X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Hi, Prasad, You can use this to catch sockets that stop writing back to your stream: ========================== HttpClient client = new HttpClient(); client.setTimeout( 60000 ); // socket timeout of 60 seconds ========================== I don't think it will catch sockets that write back slowly (e.g. 1 character per second). For that you probably need to implement your own timeout. yours, Julius On Wed, 2005-02-02 at 17:05 -0700, Prasad Sethumadhavan wrote: > Hi Mike, > Thanks for your input. I am now using a single HttpClient instance and a > multi-threaded connection manager that is shared by all threads. It is > not failing any more. > > I am trying to implement a "fetch timeout" feature using the http > client. Basically it means that if I have not received a complete > response within "n" seconds then I need to return a fetch timeout. > Is there any built in mechanism in HttpClient for this or do I need to > have a monitor thread that calls HttpMethod.abort() after the timeout > period (if the method has not finished execution). What would be the > result of calling the abort method on the worker thread? (what > exceptions etc..). Please advise if there are any better ways to > implement the same. > > Thanks in advance > Prasad > > -----Original Message----- > From: Michael Becke [mailto:melendeztammy@example.org] > Sent: Wednesday, February 02, 2005 6:39 AM > To: HttpClient User Discussion > Subject: Re: HttpClient under load > > Hi Prasad, > > Given the scenario you describe below you are more likely leaking one > connection/socket per request. You definitely want to reuse instances > of HttpClient and especially instances of a ConnectionManager. You can > either use the multi-threaded connection manager from all threads or > have a dedicated SimpleHttpConnectionManager per thread. Please have a > look at the performance guide for some more ideas > <http://jakarta.apache.org/commons/httpclient/3.0/performance.html>. > > Also please note that calling manager.closeIdleConnections(1000L) only > closes connections that have already been idle for 1000ms. It does not > continuously poll for idle connections, closing them when they've been > idle for 1000ms. > > Mike > > On Feb 1, 2005, at 11:14 PM, Prasad Sethumadhavan wrote: > > > Hi, > > I need some help or pointers regarding using the HttpClient under > > heavy load. I am currently using the HttpClient under moderate load > > (50 concurrent threads). Each thread instatiates a new HttpClient > > client, uses it and does the method.releaseConnection() before > exiting. > > > > SimpleHttpConnectionManager manager = new > > SimpleHttpConnectionManager(); > > manager.getParams().setStaleCheckingEnabled(true); > > manager.closeIdleConnections(1000L); > > manager.getParams().setConnectionTimeout( > > (int) DEFAULT_CONNECTION_TIMEOUT); > > > > httpclient = new HttpClient(manager); > > httpclient.getParams().setCookiePolicy( > > CookiePolicy.BROWSER_COMPATIBILITY); > > > > However when I run this setup under load, I keep getting socket > > exceptions on the client side. I am using tomcat web server and I have > > > increased the max-processor & access-count. I don't see any errors or > > exception in the tomcat log. > > > > However I keep getting a lot of these client socket bind exceptions > > after sometime. I guess it is happening due to the unavailability of > > ports. I would like to know some best practices in using the > > HttpClient under load. I am sure tha it is my bad, but wanted to get > > some input from the user/dev community. Should I have a pool of > > HttpClient objects and reuse them? Should I use the multi-threaded > > connection manager? I need to run this under heavier load and need to > > fix this soon. Any help, code snippets, or guidance would be greatly > > appreciated. Please let me know if you need any other information. > > > > Thanks in advance > > Prasad > > > > > > Client StackTrace: > > > > [ERROR] 01 Feb 2005 20:43:56,177 NDC[] T[Thread-20] HTTP HTTP > > I/O transport exception occurred > > java.net.BindException: Address already in use: connect > > at java.net.PlainSocketImpl.socketConnect(Native Method) > > at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) > > at > > java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) > > at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) > > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:364) > > at java.net.Socket.connect(Socket.java:507) > > at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess > > o > > r > > Impl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:585) > > at > > org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createS > > o > > c > > ket(ReflectionSocketFactory.java:128) > > at > > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.cr > > e > > a > > teSocket(DefaultProtocolSocketFactory.java:124) > > at > > org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java: > > > 70 > > 5) > > at > > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(Http > > M > > e > > thodDirector.java:359) > > at > > org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMet > > h > > o > > dDirector.java:168) > > at > > > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java: > > 3 > > 93) > > at > > > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java: > > 3 > > 24) > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: melendeztammy@example.org > > For additional commands, e-mail: > > melendeztammy@example.org > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: melendeztammy@example.org > For additional commands, e-mail: melendeztammy@example.org > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: melendeztammy@example.org > For additional commands, e-mail: melendeztammy@example.org > --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org From httpclient-user-returnmelendeztammy@example.org Thu Feb 03 00:32:59 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 85568 invoked from network); 3 Feb 2005 00:32:59 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 3 Feb 2005 00:32:59 -0000 Received: (qmail 52047 invoked by uid 500); 3 Feb 2005 00:32:59 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 52032 invoked by uid 500); 3 Feb 2005 00:32:58 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Received: (qmail 52019 invoked by uid 99); 3 Feb 2005 00:32:58 -0000 X-ASF-Spam-Status: No, hits=0.1 required=10.0 tests=FORGED_RCVD_HELO X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: local policy) Received: from savgw1.sandcherry.com (HELO savgw1.sccorp.com) (83.161.120.155) by apache.org (qpsmtpd/0.28) with SMTP; Wed, 02 Feb 2005 16:32:57 -0800 Received: from exchange01.sccorp.com ([83.161.120.155]) by savgw1.sccorp.com (SAVSMTP 83.161.120.155) with SMTP id M2005020217325426875 for <melendeztammy@example.org>; Wed, 02 Feb 2005 17:32:54 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5.6944.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: HttpClient under load Date: Wed, 2 Feb 2005 17:32:00 -0700 Message-ID: <melendeztammy@example.org> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: HttpClient under load Thread-Index: AcUJhexbHYm45yKdSVekBZNAugwjyQAAOjjw From: "Prasad Sethumadhavan" <melendeztammy@example.org> To: "HttpClient User Discussion" <melendeztammy@example.org> X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Thanks Julius. I need to handle slow servers too. =20 Actually I tried using the methodParams.setSoTimeout(5000)//5 seconds with 3.0beta1 and the servlet sleeps for 30 seconds before writing some response. However no exception was thrown. Any ideas?? -----Original Message----- From: Julius Davies [mailto:melendeztammy@example.org]=20 Sent: Wednesday, February 02, 2005 5:22 PM To: HttpClient User Discussion Subject: RE: HttpClient under load Hi, Prasad, You can use this to catch sockets that stop writing back to your stream: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D HttpClient client =3D new HttpClient(); client.setTimeout( 60000 ); // socket timeout of 60 seconds =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D I don't think it will catch sockets that write back slowly (e.g. 1 character per second). For that you probably need to implement your own timeout. yours, Julius On Wed, 2005-02-02 at 17:05 -0700, Prasad Sethumadhavan wrote: > Hi Mike, > Thanks for your input. I am now using a single HttpClient instance and > a multi-threaded connection manager that is shared by all threads. It=20 > is not failing any more. >=20 > I am trying to implement a "fetch timeout" feature using the http=20 > client. Basically it means that if I have not received a complete=20 > response within "n" seconds then I need to return a fetch timeout. > Is there any built in mechanism in HttpClient for this or do I need to > have a monitor thread that calls HttpMethod.abort() after the timeout=20 > period (if the method has not finished execution). What would be the=20 > result of calling the abort method on the worker thread? (what=20 > exceptions etc..). Please advise if there are any better ways to=20 > implement the same. >=20 > Thanks in advance > Prasad >=20 > -----Original Message----- > From: Michael Becke [mailto:melendeztammy@example.org] > Sent: Wednesday, February 02, 2005 6:39 AM > To: HttpClient User Discussion > Subject: Re: HttpClient under load >=20 > Hi Prasad, >=20 > Given the scenario you describe below you are more likely leaking one=20 > connection/socket per request. You definitely want to reuse instances > of HttpClient and especially instances of a ConnectionManager. You=20 > can either use the multi-threaded connection manager from all threads=20 > or have a dedicated SimpleHttpConnectionManager per thread. Please=20 > have a look at the performance guide for some more ideas=20 > <http://jakarta.apache.org/commons/httpclient/3.0/performance.html>. >=20 > Also please note that calling manager.closeIdleConnections(1000L) only > closes connections that have already been idle for 1000ms. It does=20 > not continuously poll for idle connections, closing them when they've=20 > been idle for 1000ms. >=20 > Mike >=20 > On Feb 1, 2005, at 11:14 PM, Prasad Sethumadhavan wrote: >=20 > > Hi, > > I need some help or pointers regarding using the HttpClient under=20 > > heavy load. I am currently using the HttpClient under moderate load=20 > > (50 concurrent threads). Each thread instatiates a new HttpClient=20 > > client, uses it and does the method.releaseConnection() before > exiting. > > > > SimpleHttpConnectionManager manager =3D new=20 > > SimpleHttpConnectionManager(); > > manager.getParams().setStaleCheckingEnabled(true); > > manager.closeIdleConnections(1000L); > > manager.getParams().setConnectionTimeout( > > (int) DEFAULT_CONNECTION_TIMEOUT); > > > > httpclient =3D new HttpClient(manager); > > httpclient.getParams().setCookiePolicy( > > CookiePolicy.BROWSER_COMPATIBILITY); > > > > However when I run this setup under load, I keep getting socket=20 > > exceptions on the client side. I am using tomcat web server and I=20 > > have >=20 > > increased the max-processor & access-count. I don't see any errors=20 > > or exception in the tomcat log. > > > > However I keep getting a lot of these client socket bind exceptions=20 > > after sometime. I guess it is happening due to the unavailability of > > ports. I would like to know some best practices in using the=20 > > HttpClient under load. I am sure tha it is my bad, but wanted to get > > some input from the user/dev community. Should I have a pool of=20 > > HttpClient objects and reuse them? Should I use the multi-threaded=20 > > connection manager? I need to run this under heavier load and need=20 > > to fix this soon. Any help, code snippets, or guidance would be=20 > > greatly appreciated. Please let me know if you need any other information. > > > > Thanks in advance > > Prasad > > > > > > Client StackTrace: > > > > [ERROR] 01 Feb 2005 20:43:56,177 NDC[] T[Thread-20] HTTP HTTP > > I/O transport exception occurred > > java.net.BindException: Address already in use: connect > > at java.net.PlainSocketImpl.socketConnect(Native Method) > > at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) > > at > > java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) > > at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) > > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:364) > > at java.net.Socket.connect(Socket.java:507) > > at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcce > > ss > > o > > r > > Impl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:585) > > at > > org.apache.commons.httpclient.protocol.ReflectionSocketFactory.creat > > eS > > o > > c > > ket(ReflectionSocketFactory.java:128) > > at > > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory. > > cr > > e > > a > > teSocket(DefaultProtocolSocketFactory.java:124) > > at > > org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java: >=20 > > 70 > > 5) > > at > > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(Ht > > tp > > M > > e > > thodDirector.java:359) > > at > > org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpM > > et > > h > > o > > dDirector.java:168) > > at > > > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:=20 > > 3 > > 93) > > at > > > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:=20 > > 3 > > 24) > > > > > > > > > > -------------------------------------------------------------------- > > - To unsubscribe, e-mail:=20 > > melendeztammy@example.org > > For additional commands, e-mail: =20 > > melendeztammy@example.org > > >=20 >=20 > --------------------------------------------------------------------- > To unsubscribe, e-mail: melendeztammy@example.org > For additional commands, e-mail:=20 > melendeztammy@example.org >=20 >=20 >=20 >=20 > --------------------------------------------------------------------- > To unsubscribe, e-mail: melendeztammy@example.org > For additional commands, e-mail:=20 > melendeztammy@example.org >=20 --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org From httpclient-user-returnmelendeztammy@example.org Thu Feb 03 13:33:04 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 32394 invoked from network); 3 Feb 2005 13:33:04 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 3 Feb 2005 13:33:04 -0000 Received: (qmail 98661 invoked by uid 500); 3 Feb 2005 13:33:04 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 98645 invoked by uid 500); 3 Feb 2005 13:33:03 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Received: (qmail 98627 invoked by uid 99); 3 Feb 2005 13:33:03 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: local policy) Received: from mxout3.cac.washington.edu (HELO mxout3.cac.washington.edu) (83.161.120.155) by apache.org (qpsmtpd/0.28) with ESMTP; Thu, 03 Feb 2005 05:33:02 -0800 Received: from smtp.washington.edu (smtp.washington.edu [83.161.120.155]) by mxout3.cac.washington.edu (8.13.1+UW04.08/8.13.3+UW05.01) with ESMTP id j13DX0IM026216 for <melendeztammy@example.org>; Thu, 3 Feb 2005 05:33:00 -0800 Received: from [83.161.120.155] (pool-+1-561-916-4685.bos.east.verizon.net [83.161.120.155]) (authenticated bits=0) by smtp.washington.edu (8.13.1+UW04.08/8.13.3+UW05.01) with ESMTP id j13DWwqn020971 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NOT) for <melendeztammy@example.org>; Thu, 3 Feb 2005 05:33:00 -0800 Mime-Version: 1.0 (Apple Message framework v619) In-Reply-To: <melendeztammy@example.org> References: <melendeztammy@example.org> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <melendeztammy@example.org> Content-Transfer-Encoding: 7bit From: Michael Becke <melendeztammy@example.org> Subject: Re: HttpClient under load Date: Thu, 3 Feb 2005 08:32:52 -0500 To: "HttpClient User Discussion" <melendeztammy@example.org> X-Mailer: Apple Mail (2.619) X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Hi Prasad, > I am trying to implement a "fetch timeout" feature using the http > client. Basically it means that if I have not received a complete > response within "n" seconds then I need to return a fetch timeout. > Is there any built in mechanism in HttpClient for this or do I need to > have a monitor thread that calls HttpMethod.abort() after the timeout > period (if the method has not finished execution). What would be the > result of calling the abort method on the worker thread? (what > exceptions etc..). Please advise if there are any better ways to > implement the same. HttpMethod.abort() is the way to go. You will need to keep track of the execution time and abort the methods from a separate thread. Calling abort will most likely cause an IOException but it depends where exactly the method is in the process of execution. Mike --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org From httpclient-user-returnmelendeztammy@example.org Thu Feb 03 13:56:04 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 42270 invoked from network); 3 Feb 2005 13:56:03 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 3 Feb 2005 13:56:03 -0000 Received: (qmail 43796 invoked by uid 500); 3 Feb 2005 13:56:03 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 43784 invoked by uid 500); 3 Feb 2005 13:56:03 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Delivered-To: moderator for melendeztammy@example.org Received: (qmail 75272 invoked by uid 99); 3 Feb 2005 12:11:23 -0000 X-ASF-Spam-Status: No, hits=0.4 required=10.0 tests=DNS_FROM_RFC_ABUSE X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: local policy) Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=TM6bNoRcHihnBzbC09Ap/hdqG5tnQ3gs4gQ3Ey22n861aF1fH0hFYL2ENIH5qyhjwCI11OeOzYmf0mVWtcU9ORB+aXa0R8NovIaUbnmhIF2vt9PpDef3hM1pgZrur6Dsq7NIjf/bF1hoTkEmQgdJGZ8dgviKyU34gNG2ot42aHc= ; Message-ID: <melendeztammy@example.org> Date: Thu, 3 Feb 2005 04:11:20 -0800 (PST) From: Nittala <melendeztammy@example.org> Subject: Connection Reset by server To: melendeztammy@example.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Hi, Am using the Httpclient 3.0 with the MultiThreadedConnectionManager. My server is Weblogic 8.1 and the time out for idle connections is set to 60 seconds in weblogic. However, when i try to communicate from my client (Java Web Start using JDK1.3.1 Runtime), i often get the Connection reset by peer _JVM exception. The code that i am using looks something like this. mthcm = new MultiThreadedHttpConnectionManager(); client = new HttpClient(mthcm); client.getState().setCookiePolicy(CookiePolicy.COMPATIBILITY); client.getParams().setParameter(HttpMethodParams.RETRY_HANDLER, new DefaultHttpMethodRetryHandler(4,false)); PostMethod method =null; try{ method = new PostMethod(url); NameValuePair nvp = new NameValuePair("query", input); method.addParameter(nvp); int status = client.executeMethod(method); if (status == HttpStatus.SC_OK) { InputStream ins = method.getResponseBodyAsStream(); ObjectInputStream objectinputstream = new ObjectInputStream(ins); obj = objectinputstream.readObject(); objectinputstream.close(); ins.close(); }else{ method.getResponseBody(); } }finally { if (method != null) method.releaseConnection(); return obj; } Any pointer on where i am doing wrong will be of great help. thanks all, Nit Tala __________________________________ Do you Yahoo!? Yahoo! Mail - Find what you need with new enhanced search. http://info.mail.yahoo.com/mail_250 --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org From httpclient-user-returnmelendeztammy@example.org Sat Feb 05 01:30:30 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 56015 invoked from network); 5 Feb 2005 01:30:30 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 5 Feb 2005 01:30:30 -0000 Received: (qmail 75855 invoked by uid 500); 5 Feb 2005 01:30:30 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 75821 invoked by uid 500); 5 Feb 2005 01:30:29 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Received: (qmail 75805 invoked by uid 99); 5 Feb 2005 01:30:29 -0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: domain of melendeztammy@example.org designates 83.161.120.155 as permitted sender) Received: from sun1rly3.tmcs.net (HELO sun1rly3.tmcs.net) (83.161.120.155) by apache.org (qpsmtpd/0.28) with ESMTP; Fri, 04 Feb 2005 17:30:28 -0800 Received: from corpmail.office.tmcs ([83.161.120.155]) by sun1rly3.tmcs.net (8.12.10/8.12.9/200406301403) with ESMTP id j151UQDn006720 for <melendeztammy@example.org>; Fri, 4 Feb 2005 17:30:26 -0800 Received: from [83.161.120.155] (evt-sun1-18-40-22-172-dhcp [83.161.120.155]) by corpmail.office.tmcs with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2657.72) id DY0Q0HQM; Fri, 4 Feb 2005 17:37:03 -0800 Message-ID: <melendeztammy@example.org> Date: Fri, 04 Feb 2005 17:27:34 -0800 From: Gamas Sugiarto <melendeztammy@example.org> User-Agent: Mozilla Thunderbird 1.0 (X11/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: melendeztammy@example.org Subject: Importing Hotmail Addressbook from Java Client using commons-httpclient APIs Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Hi guys, I am trying to get Hotmail Addressbook Page from my Java App, but haven't been succesfull. I am using Jakarta commons-httpclient APIs, JDK1.4.2 to do so. Any help would be greatly appreciated. Thank you very much. Below is my code: public void doImport(String sLogin, String sPassword) throws Exception { InputStream result = null; if(client == null) { client = new HttpClient(); } client.getHostConfiguration().setHost("login.passport.com", 443, "https"); client.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY); PostMethod authpost = new PostMethod("/ppsecure/post.srf"); NameValuePair action = new NameValuePair("sec", "rem"); NameValuePair userid = new NameValuePair("login", sLogin); NameValuePair password = new NameValuePair("passwd", sPassword); authpost.setRequestBody(new NameValuePair[] {action, userid, password}); int status = client.executeMethod(authpost); System.out.println("\nLogin form post: " + authpost.getStatusLine().toString()); System.out.println("\nbody=\n"+authpost.getResponseBodyAsString()); CookieSpec cookiespec = CookiePolicy.getCookieSpec(CookiePolicy.BROWSER_COMPATIBILITY); Cookie[] logoncookies = cookiespec.match( "login.passport.com", 443, "/ppsecure/post.srf", true, client.getState().getCookies()); System.out.println("Logon cookies:"); if (logoncookies.length == 0) { vErrorMessages.add("login invalid"); System.out.println("None"); } else { for (int i = 0; i < logoncookies.length; i++) { System.out.println("- " + logoncookies[i].toString()); } } // release any connection resources used by the method authpost.releaseConnection(); // Usually a successful form-based login results in a redicrect to // another url statuscode = authpost.getStatusCode(); if ((statuscode == HttpStatus.SC_MOVED_TEMPORARILY) || (statuscode == HttpStatus.SC_MOVED_PERMANENTLY) || (statuscode == HttpStatus.SC_SEE_OTHER) || (statuscode == HttpStatus.SC_TEMPORARY_REDIRECT)) { Header header = authpost.getResponseHeader("location"); if (header != null) { String newuri = header.getValue(); if ((newuri == null) || (newuri.equals(""))) { newuri = "/"; } System.out.println("Redirect target: " + newuri); client.getHostConfiguration().setHost("hotmail.msn.com", 80, "http"); GetMethod redirect = new GetMethod("/cgi-bin/addresses"); redirect.setFollowRedirects(true); client.executeMethod(redirect); System.out.println("Redirect: " + redirect.getStatusLine().toString()); result = redirect.getResponseBodyAsStream(); BufferedReader reader = new BufferedReader(new InputStreamReader(result)); StringBuffer sbResult = new StringBuffer(); int r = 0; while((r = reader.read()) != -1) { sbResult.append((char)r); } reader.close(); addressBook = sbResult.toString(); redirect.releaseConnection(); } else { vErrorMessages.add("invalid redirect"); System.out.println("Invalid redirect"); } } } This is what I get in my console: Login form post: HTTP/1.1 200 OK body= <HTML><HEAD><meta HTTP-EQUIV="Content-Type" content="text/html; charset=iso-8859-1"><META HTTP-EQUIV="REFRESH" CONTENT="0; URL=http://login.passport.net/UICookiesDisabled.srf?rollrs=11"><script>function OnBack(){}</script></HEAD></HTML> Logon cookies: None --------------------------------------------------------------------- To unsubscribe, e-mail: melendeztammy@example.org For additional commands, e-mail: melendeztammy@example.org From httpclient-user-returnmelendeztammy@example.org Sat Feb 05 01:43:10 2005 Return-Path: <httpclient-user-returnmelendeztammy@example.org> Delivered-To: melendeztammy@example.org Received: (qmail 58948 invoked from network); 5 Feb 2005 01:43:10 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (83.161.120.155) by minotaur-2.apache.org with SMTP; 5 Feb 2005 01:43:10 -0000 Received: (qmail 81968 invoked by uid 500); 5 Feb 2005 01:43:10 -0000 Delivered-To: melendeztammy@example.org Received: (qmail 81947 invoked by uid 500); 5 Feb 2005 01:43:10 -0000 Mailing-List: contact melendeztammy@example.org; run by ezmlm Precedence: bulk List-Help: <mailto:melendeztammy@example.org> List-Unsubscribe: <mailto:melendeztammy@example.org> List-Post: <mailto:melendeztammy@example.org> List-Id: "HttpClient User Discussion" <httpclient-user.jakarta.apache.org> Reply-To: "HttpClient User Discussion" <melendeztammy@example.org> Delivered-To: mailing list melendeztammy@example.org Received: (qmail 81933 invoked by uid 99); 5 Feb 2005 01:43:09 -0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: domain of melendeztammy@example.org designates 83.161.120.155 as permitted sender) Received: from c2ksmtpg.cucbc.com (HELO c2ksmtpg.cucbc.com) (83.161.120.155) by apache.org (qpsmtpd/0.28) with ESMTP; Fri, 04 Feb
{'source': 'Apache Software Foundation Public Mail Archives'}