Skip to content

Commit 0895460

Browse files
brianccharmander
andauthored
pg-query-stream@3.0 release (brianc#2059)
* Fix one unintentional possible breaking change & update readme * Update changelog more * Update CHANGELOG.md Co-Authored-By: Charmander <~@charmander.me> Co-authored-by: Charmander <~@charmander.me>
1 parent 19308f9 commit 0895460

File tree

3 files changed

+30
-6
lines changed

3 files changed

+30
-6
lines changed

CHANGELOG.md

+6
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,12 @@ For richer information consult the commit log on github with referenced pull req
44

55
We do not include break-fix version release in this file.
66

7+
### pg-query-stream@3.0.0
8+
9+
- [Rewrote stream internals](https://github.com/brianc/node-postgres/pull/2051) to better conform to node stream semantics. This should make pg-query-stream much better at respecting [highWaterMark](https://nodejs.org/api/stream.html#stream_new_stream_readable_options) and getting rid of some edge case bugs when using pg-query-stream as an async iterator. Due to the size and nature of this change (effectively a full re-write) it's safest to bump the semver major here, though almost all tests remain untouched and still passing, which brings us to a breaking change to the API....
10+
- Changed `stream.close` to `stream.destroy` which is the [official](https://nodejs.org/api/stream.html#stream_readable_destroy_error) way to terminate a readable stream. This is a __breaking change__ if you rely on the `stream.close` method on pg-query-stream...though should be just a find/replace type operation to upgrade as the semantics remain very similar (not exactly the same, since internals are rewritten, but more in line with how streams are "supposed" to behave).
11+
- Unified the `config.batchSize` and `config.highWaterMark` to both do the same thing: control how many rows are buffered in memory. The `ReadableStream` will manage exactly how many rows are requested from the cursor at a time. This should give better out of the box performance and help with efficient async interation.
12+
713
### pg@7.17.0
814

915
- Add support for `idle_in_transaction_session_timeout` [option](https://github.com/brianc/node-postgres/pull/2049).

packages/pg-query-stream/index.js

+2-2
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@ const Cursor = require('pg-cursor')
33

44
class PgQueryStream extends Readable {
55
constructor(text, values, config = {}) {
6-
const { batchSize = 100 } = config;
6+
const { batchSize, highWaterMark = 100 } = config;
77
// https://nodejs.org/api/stream.html#stream_new_stream_readable_options
8-
super({ objectMode: true, emitClose: true, autoDestroy: true, highWaterMark: batchSize })
8+
super({ objectMode: true, emitClose: true, autoDestroy: true, highWaterMark: batchSize || highWaterMark })
99
this.cursor = new Cursor(text, values, config)
1010

1111
// delegate Submittable callbacks to cursor
+22-4
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,26 @@
11
var assert = require('assert')
22
var QueryStream = require('../')
33

4-
var stream = new QueryStream('SELECT NOW()', [], {
5-
batchSize: 88
6-
})
4+
describe('stream config options', () => {
5+
// this is mostly for backwards compatability.
6+
it('sets readable.highWaterMark based on batch size', () => {
7+
var stream = new QueryStream('SELECT NOW()', [], {
8+
batchSize: 88
9+
})
10+
assert.equal(stream._readableState.highWaterMark, 88)
11+
})
12+
13+
it('sets readable.highWaterMark based on highWaterMark config', () => {
14+
var stream = new QueryStream('SELECT NOW()', [], {
15+
highWaterMark: 88
16+
})
17+
18+
assert.equal(stream._readableState.highWaterMark, 88)
19+
})
720

8-
assert.equal(stream._readableState.highWaterMark, 88)
21+
it('defaults to 100 for highWaterMark', () => {
22+
var stream = new QueryStream('SELECT NOW()', [])
23+
24+
assert.equal(stream._readableState.highWaterMark, 100)
25+
})
26+
})

0 commit comments

Comments
 (0)