Why is node.js only processing six requests at a time?


Question

We have a node.js server which implements a REST API as a proxy to a central server which has a slightly different, and unfortunately asymmetric REST API.

Our client, which runs in various browsers, asks the node server to get the tasks from the central server. The node server gets a list of all the task ids from the central one and returns them to the client. The client then makes two REST API calls per id through the proxy.

As far as I can tell, this stuff is all done asynchronously. In the console log, it looks like this when I start the client:

Requested GET URL under /api/v1/tasks/*: /api/v1/tasks/

This takes a couple seconds to get the list from the central server. As soon as it gets the response, the server barfs this out very quickly:

Requested GET URL under /api/v1/tasks/id/:id :/api/v1/tasks/id/438
Requested GET URL under /api/v1/workflow/id/:id :/api/v1/workflow/id/438
Requested GET URL under /api/v1/tasks/id/:id :/api/v1/tasks/id/439
Requested GET URL under /api/v1/workflow/id/:id :/api/v1/workflow/id/439
Requested GET URL under /api/v1/tasks/id/:id :/api/v1/tasks/id/441
Requested GET URL under /api/v1/workflow/id/:id :/api/v1/workflow/id/441

Then, each time a pair of these requests gets a result from the central server, another two lines is barfed out very quickly.

So it seems our node.js server is only willing to have six requests out at a time.

1
64
8/22/2012 11:57:59 AM

Accepted Answer

It's the limit on number of concurrent connections in the browser:

How many concurrent AJAX (XmlHttpRequest) requests are allowed in popular browsers?

I have upvoted the other answers, as they helped me diagnose the problem. The clue was that node's socket limit was 5, and I was getting 6 at a time. 6 is the limit in Chrome, which is what I was using to test the server.

12
5/23/2017 11:46:55 AM

There are no TCP connection limits imposed by Node itself. (The whole point is that it's highly concurrent and can handle thousands of simultaneous connections.) Your OS may limit TCP connections.

It's more likely that you're either hitting some kind of limitation of your backend server, or you're hitting the builtin HTTP library's connection limit, but it's hard to say without more details about that server or your Node implementation.

Node's built-in HTTP library (and obviously any libraries built on top of it, which are most) maintains a connection pool (via the Agent class) so that it can utilize HTTP keep-alives. This helps increase performance when you're running many requests to the same server: rather than opening a TCP connection, making a HTTP request, getting a response, closing the TCP connection, and repeating; new requests can be issued on reused TCP connections.

In node 0.10 and earlier, the HTTP Agent will only open 5 simultaneous connections to a single host by default. You can change this easily: (assuming you've required the HTTP module as http)

http.globalAgent.maxSockets = 20; // or whatever

node 0.12 sets the default maxSockets to Infinity.

You may want to keep some kind of connection limit in place. You don't want to completely overwhelm your backend server with hundreds of HTTP requests under a second – performance will most likely be worse than if you just let the Agent's connection pool do its thing, throttling requests so as to not overload your server. Your best bet will be to run some experiments to see what the optimal number of concurrent requests is in your situation.

However, if you really don't want connection pooling, you can simply bypass the pool entirely – sent agent to false in the request options:

http.get({host:'localhost', port:80, path:'/', agent:false}, callback);

In this case, there will be absolutely no limit on concurrent HTTP requests.


Licensed under: CC-BY-SA with attribution
Not affiliated with: Stack Overflow
Icon