Does backpressure is handled in split2 as its is resulting for me in heap memory limit error #55
Description
Below code snippet is resulting into heap memory exhaustion.
try {
const downloadedFile = this._s3client.getObject(params)
const userStream: Readable = await downloadedFile.createReadStream()
for await (const userId of pipeline(userStream, split2(), piplineCallback)) {
console.log(userId)
}
} catch (error) {
console.log('Error should be caught here.')
throw error
}
const piplineCallback = (err) => {
if (err) {
logger.error({ exception: err }, 'Error occured in splitted userstream pipeline.')
throw new InternalServerError(err)
} else {
logger.debug('Userstream pipeline succeded.')
}
}
This code results into following error:
"FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory\n"
I am using node.js v12.x
Do I need to handle backpressure explicity here somehow?