Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
3fe8915
chore: dep update
willfarrell Mar 19, 2026
5fc1e4c
chore: dep update
willfarrell Mar 19, 2026
0d9200e
chore: dep update
willfarrell Mar 20, 2026
4473d06
chore: dep update
willfarrell Mar 20, 2026
44e3119
chore: dep update
willfarrell Mar 20, 2026
7dbc7c2
chore: update @willfarrell-ds to 0.0.0-alpha.5
willfarrell Mar 20, 2026
a632d35
chore: dep update
willfarrell Mar 21, 2026
3a9d347
ci: add in missing configs
willfarrell Mar 21, 2026
057e756
chore: dep update
willfarrell Mar 21, 2026
07b3266
chore: dep update
willfarrell Mar 22, 2026
0b90097
chore: dep update
willfarrell Mar 25, 2026
0daf2f3
chore: dep update
willfarrell Apr 11, 2026
1167424
chore: dep update
willfarrell Apr 12, 2026
bf6e4b5
chore: dep update
willfarrell Apr 12, 2026
131c1bc
chore: lint
willfarrell Apr 12, 2026
b972646
fix: perf optimizations
willfarrell Apr 12, 2026
9a9aca0
fix: add in retrys and otehr small improvements
willfarrell Apr 12, 2026
5d03583
chore: dep update
willfarrell Apr 13, 2026
cb7611d
chore: dep update
willfarrell Apr 13, 2026
c1b59ff
chore(deps-dev): bump vite in the npm_and_yarn group across 1 directory
dependabot[bot] Apr 6, 2026
bf5e0af
chore: dep update
willfarrell Apr 13, 2026
a1b0ea6
chore: dep update
willfarrell Apr 14, 2026
6546fb3
chore: dep update
willfarrell Apr 14, 2026
1d3af0e
chore: dep update
willfarrell Apr 14, 2026
31c12cc
feat: add in new qws services
willfarrell Apr 14, 2026
e53a112
fix: type updates
willfarrell Apr 14, 2026
a4ce719
chore: dep update
willfarrell Apr 15, 2026
2bc9a57
fix: abort signals
willfarrell Apr 15, 2026
6ad0d3b
docs: add in new streams
willfarrell Apr 15, 2026
3b59622
docs: add in new example
willfarrell Apr 16, 2026
53e3c56
test: add in missing tests
willfarrell Apr 16, 2026
db6e806
chore: version bump
willfarrell Apr 16, 2026
e3133f2
feat: add in json support
willfarrell Apr 16, 2026
88807c2
feat: add in encrypt pkg
willfarrell Apr 16, 2026
124c392
ci: fix tests
willfarrell Apr 16, 2026
1be8d7d
Merge branch 'main' into develop
willfarrell Apr 16, 2026
e108752
fix: sec issues
willfarrell Apr 16, 2026
16c13a4
fix: sec issues
willfarrell Apr 16, 2026
33fe9ad
ci: fix codeql issues
willfarrell Apr 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ representative at an online or offline event.

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
[INSERT CONTACT METHOD].
willfarrell@proton.me.
All complaints will be reviewed and investigated promptly and fairly.

All community leaders are obligated to respect the privacy and security of the
Expand Down
4 changes: 2 additions & 2 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Contributing

In the spirit of Open Source Software, everyone is very welcome to contribute to this repository. Feel free to [raise issues](https://github.com/willfarrell/csv-rex/issues) or to [submit Pull Requests](https://github.com/willfarrell/csv-rex/pulls).
In the spirit of Open Source Software, everyone is very welcome to contribute to this repository. Feel free to [raise issues](https://github.com/willfarrell/datastream/issues) or to [submit Pull Requests](https://github.com/willfarrell/datastream/pulls).

Before contributing to the project, make sure to have a look at our [Code of Conduct](/.github/CODE_OF_CONDUCT.md).


## Licence

Licensed under [MIT Licence](LICENSE). Copyright (c) 2022 [will Farrell](https://github.com/willfarrell), and the [csv-rex team](https://github.com/willfarrell/csv-rex/graphs/contributors).
Licensed under [MIT Licence](LICENSE). Copyright (c) 2026 [will Farrell](https://github.com/willfarrell), and the [datastream team](https://github.com/willfarrell/datastream/graphs/contributors).
2 changes: 1 addition & 1 deletion .github/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@datastream/github-workflows",
"version": "0.1.6",
"version": "0.2.0",
"private": true,
"engines": {
"node": ">=24.0"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ permissions:
jobs:
build:
name: Build
if: ${{ github.event.pull_request.merged }}
if: ${{ github.event.pull_request.merged && github.event.pull_request.head.repo.full_name == github.repository }}
runs-on: ubuntu-latest

permissions:
Expand Down
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
*.cjs
*.cjs.map
!*.config.cjs
*.mjs
*.mjs.map
!*.config.mjs
coverage

# Logs
Expand Down Expand Up @@ -115,4 +117,4 @@ dist
.nova

# OS
.DS_Store
.DS_Store
6 changes: 2 additions & 4 deletions bin/esbuild
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,11 @@ for package in packages/*; do
module="${module#./}"
fi
if [ -f ${package}/${module}.js ]; then
#node_modules/.bin/esbuild --platform=node --target=node18 --format=cjs ${package}/${module}.js --sourcemap=external --allow-overwrite --outfile=${package}/${module}.node.cjs
node_modules/.bin/esbuild --platform=node --target=node18 --format=esm ${package}/${module}.js --sourcemap=external --allow-overwrite --outfile=${package}/${module}.node.mjs
node_modules/.bin/esbuild --platform=node --target=node24 --format=esm ${package}/${module}.js --sourcemap=external --allow-overwrite --outfile=${package}/${module}.node.mjs
node_modules/.bin/esbuild --platform=browser --format=esm ${package}/${module}.js --sourcemap=external --allow-overwrite --outfile=${package}/${module}.web.mjs
fi
if [ -f ${package}/${module}.node.js ]; then
#node_modules/.bin/esbuild --platform=node --target=node18 --format=cjs ${package}/${module}.node.js --sourcemap=external --allow-overwrite --outfile=${package}/${module}.node.cjs
node_modules/.bin/esbuild --platform=node --target=node18 --format=esm ${package}/${module}.node.js --sourcemap=external --allow-overwrite --outfile=${package}/${module}.node.mjs
node_modules/.bin/esbuild --platform=node --target=node24 --format=esm ${package}/${module}.node.js --sourcemap=external --allow-overwrite --outfile=${package}/${module}.node.mjs
fi
if [ -f ${package}/${module}.web.js ]; then
node_modules/.bin/esbuild --platform=browser --format=esm ${package}/${module}.web.js --sourcemap=external --allow-overwrite --outfile=${package}/${module}.web.mjs
Expand Down
6 changes: 3 additions & 3 deletions biome.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"$schema": "https://biomejs.dev/schemas/2.4.8/schema.json",
"$schema": "https://biomejs.dev/schemas/2.4.12/schema.json",
"vcs": {
"enabled": true,
"clientKind": "git",
Expand All @@ -20,10 +20,10 @@
"noBannedTypes": "off"
},
"correctness": {
"noUnusedFunctionParameters": "off"
"noUnusedFunctionParameters": "warn"
},
"suspicious": {
"noExplicitAny": "off"
"noExplicitAny": "warn"
}
}
},
Expand Down
3 changes: 3 additions & 0 deletions commitlint.config.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
module.exports = {
extends: ["@commitlint/config-conventional"],
};
17 changes: 17 additions & 0 deletions docs/packages/core.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,23 @@
- `chunkSize`
- `signal`

## Null handling

Node.js streams use `push(null)` to signal end-of-stream (EOF). To allow `null` values to flow through object-mode pipelines without terminating the stream, datastream wraps them with a sentinel Symbol (`Symbol.for("@datastream/null")`).

This is handled automatically when using datastream's built-in functions (`createReadableStream`, `createTransformStream`, `createPassThroughStream`, `createWritableStream`, `streamToArray`). You only need to be aware of it when reading chunks directly from a stream, such as listening to `data` events or using `for await...of` on a mid-pipeline stream.

```javascript
// Handled automatically - null values round-trip correctly
const output = await streamToArray(createReadableStream([1, null, 3]))
// [1, null, 3]

// Direct access - you may see the sentinel Symbol
stream.on('data', (chunk) => {
// chunk may be Symbol.for("@datastream/null") instead of null
})
```

## Examples

```javascript
Expand Down
Loading
Loading