httpclient exception "org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection"

Question

I am trying to send request to my server, with following code. it failed at 3rd request, always.

import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.nio.charset.Charset;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.HttpStatus;
import org.apache.http.HttpVersion;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager;
import org.apache.http.params.BasicHttpParams;
import org.apache.http.params.CoreConnectionPNames;
import org.apache.http.params.HttpParams;
import org.apache.http.params.HttpProtocolParams;
import org.json.JSONException;
import org.json.JSONObject;
import org.json.JSONTokener;

public class HttpClientTest {
    private HttpClient client;

    public HttpClientTest() {
        HttpParams params = new BasicHttpParams();
        params.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 15000);  
        params.setParameter(CoreConnectionPNames.SO_TIMEOUT, 15000);

        HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
        HttpProtocolParams.setContentCharset(params, "utf-8");
        HttpProtocolParams.setUseExpectContinue(params, true);
        ThreadSafeClientConnManager cm = new ThreadSafeClientConnManager();
        cm.setMaxTotal(100);
        client = new DefaultHttpClient(cm, params);

        while (true) {
            HttpPost mPost = new HttpPost("http://myip/myservice");

            JSONObject json = new JSONObject();
            try {
                json.put("serialNumber", "abcd");
            } catch (JSONException e1) {
                e1.printStackTrace();
            }
            StringEntity s = null;
            try {
                s = new StringEntity(json.toString());
            } catch (UnsupportedEncodingException e) {
                e.printStackTrace();
            }               
            s.setContentEncoding("UTF-8");
            s.setContentType("application/json");
            mPost.setEntity(s);

            JSONObject response = null;

            System.out.println("HttpClientTest ---> send post");
            HttpResponse mHttpResponse;
            try {
                mHttpResponse = client.execute(mPost);
                System.out.println("HttpClientTest  ---> get response");
                if(mHttpResponse.getStatusLine().getStatusCode() == HttpStatus.SC_OK){
                    HttpEntity entity = mHttpResponse.getEntity(); 
                    ContentType contentType = ContentType.getOrDefault(entity);
                    Charset charset = contentType.getCharset();
                    response = new JSONObject(new JSONTokener(new InputStreamReader(entity.getContent(), charset)));

                    System.out.println("HttpClientTest ---> get result:" + response.toString());
                } else {
                    mPost.abort();
                    break;
                }
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }

    public static void main(String[] args) {
        HttpClientTest t = new HttpClientTest();
    }
}

the exception as following:

org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection
    at org.apache.http.impl.conn.tsccm.ConnPoolByRoute.getEntryBlocking(ConnPoolByRoute.java:417)
    at org.apache.http.impl.conn.tsccm.ConnPoolByRoute$1.getPoolEntry(ConnPoolByRoute.java:300)
    at org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager$1.getConnection(ThreadSafeClientConnManager.java:224)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:401)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
    at com.i360r.client.takeaway.network.HttpClientTest.<init>(HttpClientTest.java:68)
    at com.i360r.client.takeaway.network.HttpClientTest.main(HttpClientTest.java:88)

Bilesh Ganguly · Accepted Answer · 2019-12-18 19:29:06Z

89

I had the same problem and I found the fix. This timeout is because of a connection leak. In my case, I'm using httpDelete method and not consuming the response. Instead, I'm checking the status of the response.

The fix is, the response entity needs to be consumed. In order to ensure the proper release of system resources, one must close the content stream associated with the entity.

So I used EntityUtils.consumeQuietly(response.getEntity()); which ensures that the entity content is fully consumed and the content stream, if exists, is closed.

edited Dec 18, 2019 at 19:29

Bilesh Ganguly

4,1814 gold badges42 silver badges65 bronze badges

answered Apr 2, 2014 at 20:17

Krishna Gollapudi

1,67916 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

rrhartjr Over a year ago

Had a similar issue for with the Fluent HTTP Client, when ignoring the response. Still have to call execute().discardContent() to avoid connection leaks.

Nicholas Terry Over a year ago

So VERY annoying.... The library should discard if you dont consume it... Or at least give you a warning about it...

Karussell Over a year ago

What about response.close()?

Jaraws Over a year ago

What if there is an exception and in that case response object will be null? How do we release the connection is such a scenario?

Gaël J Over a year ago

See other answers below, using a CloseableHttpClient and CloseableHttpResponse is the preferred approach.

coderz · Accepted Answer · 2016-08-29 14:35:30Z

39

I have fixed it! add mPost.releaseConnection() in finally blocks.

 try {
 } catch (Exception e) {
 } finally {
  mPost.releaseConnection();
 }

DO update package org.apache.httpcomponents to 4.2.1

edited Aug 29, 2016 at 14:35

coderz

5,01912 gold badges53 silver badges80 bronze badges

answered Aug 9, 2012 at 23:13

user1447011

1,2433 gold badges12 silver badges14 bronze badges

Comments

Suyog Tilak · Accepted Answer · 2018-04-05 15:49:16Z

13

Just put the line where you are getting your response inside try-with-resources and use CloseableHttpResponse instead of HttpResponse like this:

try(final CloseableHttpResponse mHttpResponse = client.execute(mPost);)
{
 System.out.println("HttpClientTest  ---> get response");
 ....remainder code

The mHttpResponse object will be auto consumed and closed for you

Hope this helps!

edited Apr 5, 2018 at 15:49

answered Apr 3, 2018 at 15:27

Suyog Tilak

1311 silver badge3 bronze badges

1 Comment

J. Allen Over a year ago

The client variable must be an instance of CloseableHttpClient for this to work

Emma Marcier · Accepted Answer · 2020-07-06 23:10:54Z

8

Use a CloseableHttpClient instead of HttpClient. You'll get a CloseableHttpResponse instead of a HttpResponse which support close(). So, when you are done with the response, just close it, no need to close the connection.

CloseableHttpResponse response = closableHttpClient.execute(httpPost, context);

...do what you need to do with it:

response.close();

edited Jul 6, 2020 at 23:10

Emma Marcier

27.8k12 gold badges49 silver badges71 bronze badges

answered Jan 17, 2019 at 10:30

Johan Andersson

1092 silver badges4 bronze badges

1 Comment

DaveyDaveDave Over a year ago

While this is probably true, it seems like maybe it's intended as a comment on one or more other answers, rather than an answer in its own right, as it doesn't really address the question directly. I'd suggest either editing it to phrase it as an answer to the question, making it a comment on a different answer or creating your own self-answered question, and linking to that in a comment here.

Brad Parks · Accepted Answer · 2016-11-17 19:47:28Z

4

This can also happen if you're using ApacheHttpClient with DropWizard 0.6.2, which behind the scenes creates an MultiThreadedHttpConnectionManager with a default configuration - and that default configuration only allowed 2 concurrent http connections at a time more info here.

So with this configuration, if your server is getting swamped and making requests to the same host all the time, you'll have max 2 connections allowed running at a time!

answered Nov 17, 2016 at 19:47

Brad Parks

73.3k71 gold badges297 silver badges374 bronze badges

Comments

HariJustForFun · Accepted Answer · 2019-10-07 10:17:54Z

4

We faced the same issue, and we were limited because the connection from pool has initial value of DefaultMaxPerRoute as 2. Ours was an API which would fire different calls with same URI but, different body. Once we set it explicitly to higher value like this

PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
    cm.setMaxTotal(40);
    cm.setDefaultMaxPerRoute(20);

we were able to resolve it.

answered Oct 7, 2019 at 10:17

HariJustForFun

5512 gold badges9 silver badges21 bronze badges

Comments

Sanghyun Lee · Accepted Answer · 2018-10-15 12:46:23Z

1

As Rama said, this could be a sign of a connection leak.

releaseConnection() could help resolve this issue but you might need to find out the root cause.

Also, this could happen because there's no available connection you can lease which could happen when you don't have timeout config for the HttpClient. In this case, releaseConnection() wouldn't help.

answered Oct 15, 2018 at 12:46

Sanghyun Lee

23.4k19 gold badges106 silver badges127 bronze badges

1 Comment

Jaraws Over a year ago

What if there is an exception and in that case response object will be null? How do we release the connection is such a scenario if I am not using try with resources?

Tinkerbell · Accepted Answer · 2020-01-20 06:42:27Z

1

We were hitting this error when writing and reading parquets via spark. The exact error was:
com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout waiting for connection from pool

We finally figured out that this was related to the upgrade of Hadoop to version 3.1.3

Seems that in this new version the Hadoop, the property name been changed from:
fs.s3a.connection.maximum
to:
spark.hadoop.fs.s3a.connection.maximum

Renaming the property in our spark configuration, set us back on track.

answered Jan 20, 2020 at 6:42

Tinkerbell

1881 silver badge12 bronze badges

Comments

Aarathi · Accepted Answer · 2020-10-13 03:48:42Z

1

I hit the same error. It turns out that I needed to close the response.

CloseableHttpResponse response = null;
try {
       response = ##some code
} catch(Exception e) {
  if (response != null) {
      response.close();
  }
}

answered Oct 13, 2020 at 3:48

Aarathi

111 bronze badge

1 Comment

Nitish Bhardwaj Over a year ago

I guess it's better to use try with resource here.

Troy Nechanicky · Accepted Answer · 2020-04-09 19:21:48Z

0

In my case the bufferedReader that was consuming the entity content stream was not being closed. Closing it resolved the error.

answered Apr 9, 2020 at 19:21

Troy Nechanicky

931 silver badge9 bronze badges

Collectives™ on Stack Overflow

httpclient exception "org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection"

10 Answers 10

5 Comments

Comments

1 Comment

1 Comment

Comments

Comments

1 Comment

Comments

1 Comment

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

10 Answers 10

5 Comments

Comments

1 Comment

1 Comment

Comments

Comments

1 Comment

Comments

1 Comment

Comments

Linked

Related