[Bug] [Node.js] skywalking s3 upload error

[someone-1]

Search before asking

• I had searched in the issues and found no similar issues.

Apache SkyWalking Component

NodeJS Server Side Agent (apache/skywalking-nodejs)

What happened

Hi
I am using the skywalking skywalking-backend-js library, when uploading file (size> 1Mb) to s3 using AWS nodeJs SDK.
the upload is failing with an error. I see skywalking files in the stack trace.

nom page versions
"skywalking-backend-js": "^0.4.1"
"aws-sdk": "^2.418.0",

agent config:

agent.start({
  serviceName: 's3 upload',
  collectorAddress: 'localhost:11800',
});

complete stack trace:

{
  error: Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
      at ClientRequest.setHeader (_http_outgoing.js:518:11)
      at /Users/demo-apps/node_modules/skywalking-backend-js/lib/plugins/HttpPlugin.js:101:76
      at Array.forEach (<anonymous>)
      at Object.module.request (/Users/demo-apps/node_modules/skywalking-backend-js/lib/plugins/HttpPlugin.js:101:37)
      at features.constructor.handleRequest (/Users/demo-apps/node_modules/aws-sdk/lib/http/node.js:42:23)
      at executeSend (/Users/demo-apps/node_modules/aws-sdk/lib/event_listeners.js:334:29)
      at Request.SEND (/Users/demo-apps/node_modules/aws-sdk/lib/event_listeners.js:348:9)
      at Request.callListeners (/Users/demo-apps/node_modules/aws-sdk/lib/sequential_executor.js:102:18)
      at Request.emit (/Users/demo-apps/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
      at Request.emit (/Users/demo-apps/node_modules/aws-sdk/lib/request.js:683:14) {
    code: 'NetworkingError',
    region: 'us-east-1',
    hostname: 'localhost',
    retryable: true,
    time: 2022-04-06T09:17:22.133Z
  }
}

What you expected to happen

the application should upload file without any failures.

How to reproduce

use simple setup to reproduce the bug using Localstack

index.js

const agent = require('skywalking-backend-js').default;
const express = require('express');
const uploadFile = require('./upload')

agent.start({
  serviceName: 's3 upload',
  collectorAddress: 'localhost:11800',
});

const app = express();

app.get('/upload', (req, res) => {
    uploadFile('file-greater-than-1mb.pdf', 'hello1').then(val => {
        console.log({val})
        res.send("uploaded")
     }).catch((error) => {
        console.error({error})
        res.send("error")
     })
});

app.listen(8081, () => {
    console.log('Started on port', 8081);
});

upload.js

const fs = require('fs')
const AWS = require('aws-sdk')
require('dotenv').config()

const credentials = {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_KEY,
}
const bucketName = process.env.AWS_BUCKET_NAME
AWS.config.update({ region: 'us-east-1' });

const s3 = new AWS.S3({
    credentials,
    endpoint:  'http://localhost:4566' ,
    s3ForcePathStyle: true,
 })

const uploadFile = async (sourceFilePath, fileName) => {
    // read file from file system
    const fileContent = fs.readFileSync(sourceFilePath);
    console.log({fileContent})

    const params = {
        Bucket: bucketName,
        Key: fileName,
        Body: fileContent,
    };

    try {
        await s3.putObject(params).promise();
    } catch (error1) {
        console.error({error1})
        throw error1
    }
}

module.exports = uploadFile

Anything else

docker-compose.yml to setup Localstack (local s3)

version: '3.2'
services:
  localstack:
    image: localstack/localstack:latest
    container_name: localstack_demo
    ports:
      - '4563-4599:4563-4599'
      - '8055:8080'
    environment:
      - SERVICES=s3
      - DEBUG=1
      - DATA_DIR=/tmp/localstack/data
    volumes:
      - './.localstack:/tmp/localstack'

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[wu-sheng]

This issue title is not well writen. Please fix.

[wu-sheng]

Are you going to submit a pull request to fix?

[someone-1]

not familiar with the repository, if anyone can help, I can submit a PR

[kezhenxu94]

@someone-1 did you confirm that it works if you remove skywalking agent from your codes?

[someone-1]

@kezhenxu94 application works (successful uploads file) if I remove the skywalking agent code

[wuwen5]

I tried to fixed this problem, please review. apache/skywalking-nodejs#82

This problem seems to be related to the request header Expect.

In aws-sdk/lib/http/node.js

var stream = http.request(options, function (httpResp) {...}
...
httpRequest.stream = stream; // attach stream to httpRequest

var expect = httpRequest.headers.Expect || httpRequest.headers.expect;
if (expect === '100-continue') {
  stream.once('continue', function() {
    self.writeBody(stream, httpRequest);
  });
} else {
  this.writeBody(stream, httpRequest);
}

[kezhenxu94]

@someone-1 thank you for reporting this and providing the complete reproduce steps and code snippets, and thanks @wuwen5 for fixing this 🙇