Skip to content

Commit 060116a

Browse files
jraymakersjraymakers
authored
Merge pull request #36 from duckdb/jray/update-all-readmes
update all READMEs
2 parents 0ab9842 + aa628a0 commit 060116a

File tree

6 files changed

+393
-324
lines changed

6 files changed

+393
-324
lines changed

‎README.md

Lines changed: 53 additions & 313 deletions
Original file line numberDiff line numberDiff line change
@@ -1,337 +1,77 @@
11
# DuckDB Node Bindings & API
22

3-
[Node](https://nodejs.org/) bindings to the [DuckDB](https://duckdb.org/) C API, plus a friendly API for using DuckDB in Node applications.
3+
[Node](https://nodejs.org/) bindings to the [DuckDB C API](https://duckdb.org/docs/api/c/overview), plus a friendly API for using DuckDB in Node applications.
44

5-
This repository also contains a [GitHub Packages NPM Registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-npm-registry) that serves as a temporary location for the NPM packages until they are ready to be published to the main NPM registry.
5+
## Packages
66

7-
The following platforms are supported:
8-
- Linux x64
9-
- Mac OS ARM64 (Apple Silicon)
10-
- Windows x64
7+
### Documentation
118

12-
## To install the packages hosted here:
9+
- [@duckdb/node-api](api/pkgs/@duckdb/node-api/README.md)
10+
- [@duckdb/node-bindings](bindings/pkgs/@duckdb/node-bindings/README.md)
11+
- [@duckdb/node-bindings-darwin-arm64](bindings/pkgs/@duckdb/node-bindings-darwin-arm64/README.md)
12+
- [@duckdb/node-bindings-linux-x64](bindings/pkgs/@duckdb/node-bindings-linux-x64/README.md)
13+
- [@duckdb/node-bindings-win32-x64](bindings/pkgs/@duckdb/node-bindings-win32-x64/README.md)
1314

14-
### 1. Create a [GitHub Personal Access Token (classic)](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic) (PAT) with at least `read:packages` scope.
15+
### Published
1516

16-
Remember to copy the token value and save it somewhere safe.
17+
- [@duckdb/node-api](https://www.npmjs.com/package/@duckdb/node-api)
18+
- [@duckdb/node-bindings](https://www.npmjs.com/package/@duckdb/node-bindings)
19+
- [@duckdb/node-bindings-darwin-arm64](https://www.npmjs.com/package/@duckdb/node-bindings-darwin-arm64)
20+
- [@duckdb/node-bindings-linux-x64](https://www.npmjs.com/package/@duckdb/node-bindings-linux-x64)
21+
- [@duckdb/node-bindings-win32-x64](https://www.npmjs.com/package/@duckdb/node-bindings-win32-x64)
1722

18-
### 2. Create a local directory with a `package.json` containing:
23+
## Development
1924

20-
```
21-
{
22-
"dependencies": {
23-
"@duckdb/node-api": "*"
24-
}
25-
}
26-
```
25+
### Setup
26+
- [Install pnpm](https://pnpm.io/installation)
27+
- `pnpm install`
2728

28-
### 3. In the same directory, create an `.npmrc` containing:
29+
### Build & Test Bindings
30+
- `cd bindings`
31+
- `pnpm run build`
32+
- `pnpm test`
2933

30-
```
31-
@duckdb:registry=https://npm.pkg.github.com
32-
```
34+
### Build & Test API
35+
- `cd api`
36+
- `pnpm run build`
37+
- `pnpm test`
3338

34-
### 4. In the same directory, log in to `npm` using your PAT:
39+
### Run API Benchmarks
40+
- `cd api`
41+
- `pnpm bench`
3542

36-
```
37-
npm login --scope=@duckdb
38-
```
43+
### Update Package Versions
3944

40-
When prompted, enter your GitHub username, and, for your password, your GitHub PAT.
45+
Change version in:
46+
- `api/pkgs/@duckdb/node-api/package.json`
47+
- `bindings/pkgs/@duckdb/node-bindings/package.json`
48+
- `bindings/pkgs/@duckdb/node-bindings-darwin-arm64/package.json`
49+
- `bindings/pkgs/@duckdb/node-bindings-linux-x64/package.json`
50+
- `bindings/pkgs/@duckdb/node-bindings-win32-x64/package.json`
4151

42-
### 5. In the same directory, run `npm install`.
52+
### Upgrade DuckDB Version
4353

44-
This should create a `node_modules` directory containing three packages:
45-
- `@duckdb/node-api`
46-
- `@duckdb/node-bindings`
47-
- `@duckdb/node-bindings-darwin-arm64` OR `@duckdb/node-bindings-linux-x64` OR `@duckdb/node-bindings-win32-x64`
54+
Change version in:
55+
- `bindings/scripts/fetch_libduckdb_linux.py`
56+
- `bindings/scripts/fetch_libduckdb_mac.py`
57+
- `bindings/scripts/fetch_libduckdb_win.py`
58+
- `bindings/test/constants.test.ts`
4859

49-
## To test the installation:
60+
Also change DuckDB version in package versions.
5061

51-
### 1. Create a test file.
62+
### Check Function Signatures
5263

53-
Example (`test.mjs`):
54-
```
55-
import duckdb, { DuckDBInstance, DuckDBTypeId } from '@duckdb/node-api';
64+
- `node scripts/checkFunctionSignatures.mjs [writeFiles]`
5665

57-
console.log(duckdb.version());
66+
Checks for differences between the function signatures in `duckdb.h` and those declared in `duckdb.d.ts` and implemented in `duckdb_node_bindings.cpp`.
5867

59-
const instance = await DuckDBInstance.create();
60-
const connection = await instance.connect();
61-
const result = await connection.run('from test_all_types()');
62-
const chunk = await result.fetchChunk();
63-
for (let c = 0; c < chunk.columnCount; c++) {
64-
console.log(`[col ${c}] ${result.columnName(c)}::${DuckDBTypeId[result.columnTypeId(c)]}`);
65-
const type = result.columnType(c);
66-
const column = chunk.getColumn(c);
67-
for (let r = 0; r < chunk.rowCount; r++) {
68-
const value = column.getItem(r);
69-
console.log(` [row ${r}] ${valueToString(value, type)}`);
70-
}
71-
}
68+
Optionally outputs JSON files that can be diff'd.
7269

73-
function replacer(key, value) {
74-
return typeof value === "bigint" ? { $bigint: value.toString() } : value;
75-
}
70+
Useful when upgrading the DuckDB version to detect changes to the C API.
7671

77-
function valueToString(value, type) {
78-
switch (type.typeId) {
79-
case DuckDBTypeId.ARRAY:
80-
return value
81-
? `ARRAY [${Array.from({ length: type.length }).map((_, i) => valueToString(value.getItem(i), type.valueType)).join(', ')}]`
82-
: 'null';
83-
case DuckDBTypeId.DECIMAL:
84-
return JSON.stringify(value, replacer);
85-
case DuckDBTypeId.INTERVAL:
86-
return JSON.stringify(value, replacer);
87-
case DuckDBTypeId.LIST:
88-
return value
89-
? `LIST [${Array.from({ length: value.itemCount }).map((_, i) => valueToString(value.getItem(i), type.valueType)).join(', ')}]`
90-
: 'null';
91-
case DuckDBTypeId.MAP:
92-
return value
93-
? `MAP { ${value.map((entry) => `${valueToString(entry.key, type.keyType)}: ${valueToString(entry.value, type.valueType)}`).join(', ')} }`
94-
: 'null';
95-
case DuckDBTypeId.STRUCT:
96-
return value
97-
? `STRUCT { ${value.map((entry, i) => `"${entry.name}": ${valueToString(entry.value, type.entries[i].valueType)}`).join(', ')} }`
98-
: 'null';
99-
case DuckDBTypeId.TIME_TZ:
100-
return JSON.stringify(value, replacer);
101-
case DuckDBTypeId.UNION:
102-
return value
103-
? valueToString(value.value, type.alternatives.find((alt) => alt.tag === value.tag).valueType)
104-
: 'null';
105-
default:
106-
return String(value);
107-
}
108-
}
109-
```
72+
### Publish Packages
11073

111-
### 2. Run the test file using:
112-
113-
```
114-
node test.mjs
115-
```
116-
117-
Expected output for above example:
118-
119-
```
120-
v1.1.2
121-
[col 0] bool::BOOLEAN
122-
[row 0] false
123-
[row 1] true
124-
[row 2] null
125-
[col 1] tinyint::TINYINT
126-
[row 0] -128
127-
[row 1] 127
128-
[row 2] null
129-
[col 2] smallint::SMALLINT
130-
[row 0] -32768
131-
[row 1] 32767
132-
[row 2] null
133-
[col 3] int::INTEGER
134-
[row 0] -2147483648
135-
[row 1] 2147483647
136-
[row 2] null
137-
[col 4] bigint::BIGINT
138-
[row 0] -9223372036854775808
139-
[row 1] 9223372036854775807
140-
[row 2] null
141-
[col 5] hugeint::HUGEINT
142-
[row 0] -170141183460469231731687303715884105728
143-
[row 1] 170141183460469231731687303715884105727
144-
[row 2] null
145-
[col 6] uhugeint::UHUGEINT
146-
[row 0] 0
147-
[row 1] 340282366920938463463374607431768211455
148-
[row 2] null
149-
[col 7] utinyint::UTINYINT
150-
[row 0] 0
151-
[row 1] 255
152-
[row 2] null
153-
[col 8] usmallint::USMALLINT
154-
[row 0] 0
155-
[row 1] 65535
156-
[row 2] null
157-
[col 9] uint::UINTEGER
158-
[row 0] 0
159-
[row 1] 4294967295
160-
[row 2] null
161-
[col 10] ubigint::UBIGINT
162-
[row 0] 0
163-
[row 1] 18446744073709551615
164-
[row 2] null
165-
[col 11] varint::VARINT
166-
[row 0] -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368
167-
[row 1] 179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368
168-
[row 2] null
169-
[col 12] date::DATE
170-
[row 0] -2147483646
171-
[row 1] 2147483646
172-
[row 2] null
173-
[col 13] time::TIME
174-
[row 0] 0
175-
[row 1] 86400000000
176-
[row 2] null
177-
[col 14] timestamp::TIMESTAMP
178-
[row 0] -9223372022400000000
179-
[row 1] 9223372036854775806
180-
[row 2] null
181-
[col 15] timestamp_s::TIMESTAMP_S
182-
[row 0] -9223372022400
183-
[row 1] 9223372036854
184-
[row 2] null
185-
[col 16] timestamp_ms::TIMESTAMP_MS
186-
[row 0] -9223372022400000
187-
[row 1] 9223372036854775
188-
[row 2] null
189-
[col 17] timestamp_ns::TIMESTAMP_NS
190-
[row 0] -9223286400000000000
191-
[row 1] 9223372036854775806
192-
[row 2] null
193-
[col 18] time_tz::TIME_TZ
194-
[row 0] {"microseconds":0,"offset":57599}
195-
[row 1] {"microseconds":86400000000,"offset":-57599}
196-
[row 2] null
197-
[col 19] timestamp_tz::TIMESTAMP_TZ
198-
[row 0] -9223372022400000000
199-
[row 1] 9223372036854775806
200-
[row 2] null
201-
[col 20] float::FLOAT
202-
[row 0] -3.4028234663852886e+38
203-
[row 1] 3.4028234663852886e+38
204-
[row 2] null
205-
[col 21] double::DOUBLE
206-
[row 0] -1.7976931348623157e+308
207-
[row 1] 1.7976931348623157e+308
208-
[row 2] null
209-
[col 22] dec_4_1::DECIMAL
210-
[row 0] {"scaledValue":-9999,"type":{"typeId":19,"width":4,"scale":1}}
211-
[row 1] {"scaledValue":9999,"type":{"typeId":19,"width":4,"scale":1}}
212-
[row 2] null
213-
[col 23] dec_9_4::DECIMAL
214-
[row 0] {"scaledValue":-999999999,"type":{"typeId":19,"width":9,"scale":4}}
215-
[row 1] {"scaledValue":999999999,"type":{"typeId":19,"width":9,"scale":4}}
216-
[row 2] null
217-
[col 24] dec_18_6::DECIMAL
218-
[row 0] {"scaledValue":{"$bigint":"-999999999999999999"},"type":{"typeId":19,"width":18,"scale":6}}
219-
[row 1] {"scaledValue":{"$bigint":"999999999999999999"},"type":{"typeId":19,"width":18,"scale":6}}
220-
[row 2] null
221-
[col 25] dec38_10::DECIMAL
222-
[row 0] {"scaledValue":{"$bigint":"-99999999999999999999999999999999999999"},"type":{"typeId":19,"width":38,"scale":10}}
223-
[row 1] {"scaledValue":{"$bigint":"99999999999999999999999999999999999999"},"type":{"typeId":19,"width":38,"scale":10}}
224-
[row 2] null
225-
[col 26] uuid::UUID
226-
[row 0] -170141183460469231731687303715884105728
227-
[row 1] 170141183460469231731687303715884105727
228-
[row 2] null
229-
[col 27] interval::INTERVAL
230-
[row 0] {"months":0,"days":0,"micros":{"$bigint":"0"}}
231-
[row 1] {"months":999,"days":999,"micros":{"$bigint":"999999999"}}
232-
[row 2] null
233-
[col 28] varchar::VARCHAR
234-
[row 0] 🦆🦆🦆🦆🦆🦆
235-
[row 1] goose
236-
[row 2] null
237-
[col 29] blob::BLOB
238-
[row 0] thisisalongblobwithnullbytes
239-
[row 1] a
240-
[row 2] null
241-
[col 30] bit::BIT
242-
[row 0] 1000100101110100010101001110101
243-
[row 1] 10101
244-
[row 2] null
245-
[col 31] small_enum::ENUM
246-
[row 0] DUCK_DUCK_ENUM
247-
[row 1] GOOSE
248-
[row 2] null
249-
[col 32] medium_enum::ENUM
250-
[row 0] enum_0
251-
[row 1] enum_299
252-
[row 2] null
253-
[col 33] large_enum::ENUM
254-
[row 0] enum_0
255-
[row 1] enum_69999
256-
[row 2] null
257-
[col 34] int_array::LIST
258-
[row 0] LIST []
259-
[row 1] LIST [42, 999, null, null, -42]
260-
[row 2] null
261-
[col 35] double_array::LIST
262-
[row 0] LIST []
263-
[row 1] LIST [42, NaN, Infinity, -Infinity, null, -42]
264-
[row 2] null
265-
[col 36] date_array::LIST
266-
[row 0] LIST []
267-
[row 1] LIST [0, 2147483647, -2147483647, null, 19124]
268-
[row 2] null
269-
[col 37] timestamp_array::LIST
270-
[row 0] LIST []
271-
[row 1] LIST [0, 9223372036854775807, -9223372036854775807, null, 1652372625000000]
272-
[row 2] null
273-
[col 38] timestamptz_array::LIST
274-
[row 0] LIST []
275-
[row 1] LIST [0, 9223372036854775807, -9223372036854775807, null, 1652397825000000]
276-
[row 2] null
277-
[col 39] varchar_array::LIST
278-
[row 0] LIST []
279-
[row 1] LIST [🦆🦆🦆🦆🦆🦆, goose, null, ]
280-
[row 2] null
281-
[col 40] nested_int_array::LIST
282-
[row 0] LIST []
283-
[row 1] LIST [LIST [], LIST [42, 999, null, null, -42], null, LIST [], LIST [42, 999, null, null, -42]]
284-
[row 2] null
285-
[col 41] struct::STRUCT
286-
[row 0] STRUCT { "a": null, "b": null }
287-
[row 1] STRUCT { "a": 42, "b": 🦆🦆🦆🦆🦆🦆 }
288-
[row 2] null
289-
[col 42] struct_of_arrays::STRUCT
290-
[row 0] STRUCT { "a": null, "b": null }
291-
[row 1] STRUCT { "a": LIST [42, 999, null, null, -42], "b": LIST [🦆🦆🦆🦆🦆🦆, goose, null, ] }
292-
[row 2] null
293-
[col 43] array_of_structs::LIST
294-
[row 0] LIST []
295-
[row 1] LIST [STRUCT { "a": null, "b": null }, STRUCT { "a": 42, "b": 🦆🦆🦆🦆🦆🦆 }, null]
296-
[row 2] null
297-
[col 44] map::MAP
298-
[row 0] MAP { }
299-
[row 1] MAP { key1: 🦆🦆🦆🦆🦆🦆, key2: goose }
300-
[row 2] null
301-
[col 45] union::UNION
302-
[row 0] Frank
303-
[row 1] 5
304-
[row 2] null
305-
[col 46] fixed_int_array::ARRAY
306-
[row 0] ARRAY [null, 2, 3]
307-
[row 1] ARRAY [4, 5, 6]
308-
[row 2] null
309-
[col 47] fixed_varchar_array::ARRAY
310-
[row 0] ARRAY [a, null, c]
311-
[row 1] ARRAY [d, e, f]
312-
[row 2] null
313-
[col 48] fixed_nested_int_array::ARRAY
314-
[row 0] ARRAY [ARRAY [null, 2, 3], null, ARRAY [null, 2, 3]]
315-
[row 1] ARRAY [ARRAY [4, 5, 6], ARRAY [null, 2, 3], ARRAY [4, 5, 6]]
316-
[row 2] null
317-
[col 49] fixed_nested_varchar_array::ARRAY
318-
[row 0] ARRAY [ARRAY [a, null, c], null, ARRAY [a, null, c]]
319-
[row 1] ARRAY [ARRAY [d, e, f], ARRAY [a, null, c], ARRAY [d, e, f]]
320-
[row 2] null
321-
[col 50] fixed_struct_array::ARRAY
322-
[row 0] ARRAY [STRUCT { "a": null, "b": null }, STRUCT { "a": 42, "b": 🦆🦆🦆🦆🦆🦆 }, STRUCT { "a": null, "b": null }]
323-
[row 1] ARRAY [STRUCT { "a": 42, "b": 🦆🦆🦆🦆🦆🦆 }, STRUCT { "a": null, "b": null }, STRUCT { "a": 42, "b": 🦆🦆🦆🦆🦆🦆 }]
324-
[row 2] null
325-
[col 51] struct_of_fixed_array::STRUCT
326-
[row 0] STRUCT { "a": ARRAY [null, 2, 3], "b": ARRAY [a, null, c] }
327-
[row 1] STRUCT { "a": ARRAY [4, 5, 6], "b": ARRAY [d, e, f] }
328-
[row 2] null
329-
[col 52] fixed_array_of_int_list::ARRAY
330-
[row 0] ARRAY [LIST [], LIST [42, 999, null, null, -42], LIST []]
331-
[row 1] ARRAY [LIST [42, 999, null, null, -42], LIST [], LIST [42, 999, null, null, -42]]
332-
[row 2] null
333-
[col 53] list_of_fixed_int_array::LIST
334-
[row 0] LIST [ARRAY [null, 2, 3], ARRAY [4, 5, 6], ARRAY [null, 2, 3]]
335-
[row 1] LIST [ARRAY [4, 5, 6], ARRAY [null, 2, 3], ARRAY [4, 5, 6]]
336-
[row 2] null
337-
```
74+
- Update package versions (as above).
75+
- Use the workflow dispatch for the [DuckDB Node Bindings & API GitHub action](https://github.com/duckdb/duckdb-node-neo/actions/workflows/DuckDBNodeBindingsAndAPI.yml).
76+
- Select all initially-unchecked checkboxes to build on all platforms and publish all packages.
77+
- Uncheck "Publish Dry Run" to actually publish.

0 commit comments

Comments
 (0)