Java - Fastest Stream for TCP/HTTP Communication?

Question

I'm trying to write a Java HTTP Proxy Tunnelling program, and I need an experts advice about the best and fastest stream to use for the communication.

I've implemented the basic functionality and everything works fine. The only matter is communication speed or performance. My HTTP proxy system consists of a server program, running on a remote server and a client program running on the local machine. So far, the program looks like this:

Listener.java :

/**
 * Listens and accepts connection requests from the browser
 */
ServerSocket listener = null;
try {
    listener = new ServerSocket(port, 128);
} catch (IOException ex) {
    ex.printStackTrace(System.err);
}

ExecutorService executor = Executors.newCachedThreadPool();

Socket connection;
while (!shutdown) {
    try {
        connection = listener.accept();
        executor.execute(new ProxyTunnel(connection));
    } catch (IOException ex) {
        ex.printStackTrace(System.err);
    }
}

ProxyTunnel.java :

try {
    byte[] buffer = new byte[8192];  // 8-KB buffer
    InputStream browserInput = browser.getInputStream();
    OutputStream browserOutput = browser.getOutputStream();

    // Reading browser request ...
    StringBuilder request = new StringBuilder(2048);
    do {
        int read = browserInput.read(buffer);
        logger.log(read + " bytes read from browser.");
        if (read > 0) {
            request.append(new String(buffer, 0, read));
        }
    } while (browserInput.available() > 0 && read > 0);

    // Connecting to proxy server ...
    Socket server = new Socket(SERVER_IP, SERVER_PORT);
    server.setSoTimeout(5000);  // Setting 5 sec read timeout
    OutputStream serverOutput = server.getOutputStream();
    InputStream serverInput = server.getInputStream();

    // Sending request to server ...
    serverOutput.write(request.toString().getBytes());
    serverOutput.flush();

    // Waiting for server response ...
    StringBuilder response = new StringBuilder(16384);
    do {
        try {
            read = serverInput.read(buffer);
        } catch (SocketTimeoutException ex) {
            break; // Timeout!
        }
        if (read > 0) {
            // Send response to browser.");
            response.append(new String(buffer, 0, read));
            browserOutput.write(buffer, 0, read);
            browserOutput.flush();
        }
    } while (read > 0);

    // Closing connections ...
    server.close();

} catch (IOException ex) {
    ex.printStackTrace(System.err);
} finally {
    try {
        browser.close();
    } catch (IOException ex) {
        ex.printStackTrace(System.err);
    }
}

The server program uses a similar fashion and sends the HTTP request to the destination server (e.g. www.stackoverflow.com) and forwards the response to the client program, where the client program forwards the response to the local browser.

How can I improve the performance of these TCP/HTTP communications?
Does using buffered streams such as BufferedInputSream and BufferedOutputStream improve communication performance?
Will I gain any performance improvements if I use java.nio Channels and Buffers, instead of java.net Sockets and java.io Stream?

@Michael-O: For learning and educational purposes, of course! Better think twice before insulting people! — Seyed Mohammad
– Seyed Mohammad, Commented Mar 9, 2014 at 8:20

Community · Accepted Answer · 2017-05-23 10:34:39Z

14

Don't do it yourself

Advice 0: there are plenty of proxy servers out there, much more scalable, stable and mature. Do you really need to write your own?

Don't use `StringBuilder`/`String` to buffer request

byte[] buffer = new byte[8192];  // 8-KB buffer
//...
browserInput.read(buffer);
//...
request.append(new String(buffer, 0, read));
//...
serverOutput.write(request.toString().getBytes());

This is flawed for several reasons:

you are assuming your HTTP calls are text (ASCII) only, binary data will be malformed after transforming to String and back to byte[], see: String, byte[] and compression
even if the protocol is text-based, you are using system's default encoding. I bet this is not what you want
finally, the most important part: do not buffer the whole request. Read chunk of data from incoming request and forward it immediately to target server in one iteration. There is absolutely no need for the extra memory overhead and latency. Immediately after receiving few bytes dispatch them and forget about them.

Don't use `Executors.newCachedThreadPool()`

This pool can grow inifinitely, creating thousands of threads during peak. Essentially you create one thread per connection (except that the pool reuses free threads, but creates new if none available). Consider Executors.newFixedThreadPool(100) - 100-200 threads should be enough in most cases. Above that you'll most likely burn your CPU barely in context switching, without doing much work. Don't be afraid of latency, scale out.

Use non-blocking netty stack

Which brings us to the final advice. Drop blocking sockets altogether. They are handy, but don't scale well due to thread-per-connection requirement. Too much memory is spent to hold stack, too much CPU is wasted for context switching. netty is great and it builds powerful abstraction over NIO.

Check out the examples, they include HTTP client/server code. There is a bit of a learning curve, but you can expect performance growth by several order of magnitued.

edited May 23, 2017 at 10:34

CommunityBot

11 silver badge

answered Aug 19, 2012 at 8:39

Tomasz Nurkiewicz

342k72 gold badges713 silver badges680 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Seyed Mohammad Over a year ago

Nice points. I will definitely consider these in my program. But you didn't answer my other questions. I mean question #2 and #3 at the end.

Tomasz Nurkiewicz Over a year ago

@SeyedMohammad: right. For #2: generally, yes, as long as you are not buffering already buffered stream. But in your case you already read data in chunks, buffering will only increase memory consumption slightly and possibly make your code slower. BTW consider IOUtils - very useful! Ad. #3 - netty is built on top of NIO, I think I was pretty clear that it's the right way to go. Bare NIO channels will generally be faster, but you have to get them right.

Collectives™ on Stack Overflow

Java - Fastest Stream for TCP/HTTP Communication?

1 Answer 1

Don't do it yourself

Don't use `StringBuilder`/`String` to buffer request

Don't use `Executors.newCachedThreadPool()`

Use non-blocking netty stack

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Don't do it yourself

Don't use StringBuilder/String to buffer request

Don't use Executors.newCachedThreadPool()

Use non-blocking netty stack

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related

Don't use `StringBuilder`/`String` to buffer request

Don't use `Executors.newCachedThreadPool()`