You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
269 lines
6.4 KiB
269 lines
6.4 KiB
# hyperquest |
|
|
|
treat http requests as a streaming transport |
|
|
|
[![build status](https://secure.travis-ci.org/substack/hyperquest.png)](http://travis-ci.org/substack/hyperquest) |
|
|
|
The hyperquest api is a subset of [request](https://github.com/mikeal/request). |
|
|
|
This module works in the browser with [browserify](http://browserify.org). |
|
|
|
# rant |
|
|
|
![animated gif rant](http://substack.net/images/substack.gif) |
|
|
|
This module disables a lot of infuriating things about core http that WILL cause |
|
bugs in your application if you think of http as just another kind of stream: |
|
|
|
* http requests have a default idle timeout of 2 minutes. This is terrible if |
|
you just want to pipe together a bunch of persistent backend processes over |
|
http. |
|
|
|
* There is a default connection pool of 5 requests. If you have 5 or more extant |
|
http requests, any additional requests will HANG for NO GOOD REASON. |
|
|
|
hyperquest turns these annoyances off so you can just pretend that core http is |
|
just a fancier version of tcp and not the horrible monstrosity that it actually |
|
is. |
|
|
|
I have it on good authority that these annoyances will be fixed in node 0.12. |
|
|
|
# example |
|
|
|
# simple streaming GET |
|
|
|
``` js |
|
var hyperquest = require('hyperquest'); |
|
hyperquest('http://localhost:8000').pipe(process.stdout); |
|
``` |
|
|
|
``` |
|
$ node example/req.js |
|
beep boop |
|
``` |
|
|
|
# pooling is evil |
|
|
|
Now to drive the point home about pooling being evil and almost always never |
|
what you want ever. |
|
|
|
[request](https://github.com/mikeal/request) |
|
has its own forever agent thing that works pretty much the same as node core |
|
http.request: the wrong, horrible, broken way. |
|
|
|
For instance, the following request code takes 12+ seconds to finish: |
|
|
|
``` js |
|
var http = require('http'); |
|
var request = require('request'); |
|
|
|
var server = http.createServer(function (req, res) { |
|
res.write(req.url.slice(1) + '\n'); |
|
setTimeout(res.end.bind(res), 3000); |
|
}); |
|
|
|
server.listen(5000, function () { |
|
var pending = 20; |
|
for (var i = 0; i < 20; i++) { |
|
var r = request('http://localhost:5000/' + i); |
|
r.pipe(process.stdout, { end: false }); |
|
r.on('end', function () { |
|
if (--pending === 0) server.close(); |
|
}); |
|
} |
|
}); |
|
|
|
process.stdout.setMaxListeners(0); // turn off annoying warnings |
|
``` |
|
|
|
``` |
|
substack : example $ time node many_request.js |
|
0 |
|
1 |
|
2 |
|
3 |
|
4 |
|
5 |
|
6 |
|
7 |
|
8 |
|
9 |
|
10 |
|
11 |
|
12 |
|
13 |
|
14 |
|
15 |
|
16 |
|
17 |
|
18 |
|
19 |
|
|
|
real 0m12.423s |
|
user 0m0.424s |
|
sys 0m0.048s |
|
``` |
|
|
|
Surprising? YES. This is pretty much never what you want, particularly if you |
|
have a lot of streaming http API endpoints. Your code will just *HANG* once the |
|
connection pool fills up and it won't start working again until some connections |
|
die for whatever reason. I have encountered this so many times in production |
|
instances and it is SO hard to track down reliably. |
|
|
|
Compare to using hyperquest, which is exactly the same code but it takes 3 |
|
seconds instead of 12 to finish because it's not completely self-crippled like |
|
request and core http.request. |
|
|
|
``` js |
|
var http = require('http'); |
|
var hyperquest = require('hyperquest'); |
|
|
|
var server = http.createServer(function (req, res) { |
|
res.write(req.url.slice(1) + '\n'); |
|
setTimeout(res.end.bind(res), 3000); |
|
}); |
|
|
|
server.listen(5000, function () { |
|
var pending = 20; |
|
for (var i = 0; i < 20; i++) { |
|
var r = hyperquest('http://localhost:5000/' + i); |
|
r.pipe(process.stdout, { end: false }); |
|
r.on('end', function () { |
|
if (--pending === 0) server.close(); |
|
}); |
|
} |
|
}); |
|
|
|
process.stdout.setMaxListeners(0); // turn off annoying warnings |
|
``` |
|
``` |
|
$ time node many_hyperquest.js |
|
0 |
|
1 |
|
2 |
|
3 |
|
4 |
|
5 |
|
6 |
|
8 |
|
9 |
|
7 |
|
10 |
|
11 |
|
12 |
|
13 |
|
14 |
|
15 |
|
16 |
|
17 |
|
18 |
|
19 |
|
|
|
real 0m3.284s |
|
user 0m0.288s |
|
sys 0m0.060s |
|
``` |
|
|
|
So the other thing is, the justification I've heard supporting this horrible |
|
limit-of-5 pooling behavior is "performance". The first example which has been |
|
tuned for "performance" takes 12 seconds. The second example that removes these |
|
"performance" enhancements takes 3. Some performance improvement INDEED! |
|
|
|
# methods |
|
|
|
``` js |
|
var hyperquest = require('hyperquest'); |
|
``` |
|
|
|
## var req = hyperquest(uri, opts={}, cb) |
|
|
|
Create an outgoing http request to `uri` or `opts.uri`. |
|
You need not pass any arguments here since there are setter methods documented |
|
below. |
|
|
|
Return a readable or duplex stream depending on the `opts.method`. |
|
|
|
Default option values: |
|
|
|
* opts.method - `"GET"` |
|
* opts.headers - `{}` |
|
* opts.auth - undefined, but is set automatically when the `uri` has an auth |
|
string in it such as `"http://user:passwd@host"`. `opts.auth` is of the form |
|
`"user:pass"`, just like `http.request()`. |
|
* opts.agent - `false` |
|
* opts.timeout - `Math.pow(2, 32) * 1000` The value is passed as an argument |
|
to the underlying `req.setTimeout()` function. |
|
* opts.localAddress - local interface to bind for network connections |
|
(Node.js only) |
|
|
|
In https mode, you can specify options to the underlying `tls.connect()` call: |
|
|
|
* opts.pfx |
|
* opts.key |
|
* opts.cert |
|
* opts.ca |
|
* opts.ciphers |
|
* opts.rejectUnauthorized |
|
* opts.secureProtocol |
|
|
|
The request does not go through until the `nextTick` so you can set values |
|
outside of the `opts` so long as they are called on the same tick. |
|
|
|
Optionally you can pass a `cb(err, res)` to set up listeners for `'error'` and |
|
`'response'` events in one place. |
|
|
|
Note that the optional `cb` is NOT like |
|
[request](https://github.com/mikeal/request) |
|
in that hyperquest will not buffer content for you or decode to json or any such |
|
magical thing. |
|
|
|
## req.setHeader(key, value); |
|
|
|
Set an outgoing header `key` to `value`. |
|
|
|
## req.setLocation(uri); |
|
|
|
Set the location if you didn't specify it in the `hyperquest()` call. |
|
|
|
## var req = hyperquest.get(uri, opts, cb) |
|
|
|
Return a readable stream from `hyperquest(..., { method: 'GET' })`. |
|
|
|
## var req = hyperquest.put(uri, opts, cb) |
|
|
|
Return a duplex stream from `hyperquest(..., { method: 'PUT' })`. |
|
|
|
## var req = hyperquest.post(uri, opts, cb) |
|
|
|
Return a duplex stream from `hyperquest(..., { method: 'POST' })`. |
|
|
|
## var req = hyperquest.delete(uri, opts, cb) |
|
|
|
Return a readable stream from `hyperquest(..., { method: 'DELETE' })`. |
|
|
|
# events |
|
|
|
## req.on('request', function (req) {}) |
|
|
|
The `'request'` event is fired with the *ClientRequest* object created |
|
as a result of the underlying `http.request()` call. |
|
|
|
## req.on('response', function (res) {}) |
|
|
|
The `'response'` event is forwarded from the underlying `http.request()`. |
|
|
|
## req.on('error', function (res) {}) |
|
|
|
The `'error'` event is forwarded from the underlying `http.request()`. |
|
|
|
# install |
|
|
|
With [npm](https://npmjs.org) do: |
|
|
|
``` |
|
npm install hyperquest |
|
``` |
|
|
|
# license |
|
|
|
MIT
|
|
|