unable to sync: Error: Non-file stream objects are not supported with SigV4 in AWS.S3

[vmarimuthu]

Can you upgrade aws-sdk version?

[woudsma]

Does anyone have a workaround for this issue?
I'm able to download files using client.downloadFile, but uploading using client.uploadFile or client.uploadDir gives an error.

Either: NoSuchBucket: The specified bucket does not exist (which is weird, because downloadFile works with the same params)
Or: Error: Non-file stream objects are not supported with SigV4

[bradgreens]

Since my app was also using serverless I switched to using the serverless s3 client package and it works outside of us-east-1 (e.g. the much newer us-east-2).

[DanielApt]

This is solved by updating the aws-sdk package. It works for me (eu-west-2 region) when I updated the aws-sdk package to v2.100.0.

[LukasBombach]

Here's a workaround: Use the S3 Client from the latest SDK and pass it on to the s3 client

const AWS = require('aws-sdk');
const s3 = require('s3');

const awsS3Client = new AWS.S3({
    region: 'eu-central-1',
    signatureVersion: 'v4'
});

const client = s3.createClient({
    s3Client: awsS3Client
});

[asharma93]

@LukasBombach still works 👍

Ran into the SigV4 issue using the webpack-s3-plugin and resorted to adding an upload script with this workaround.

[anon-r-7]

Other folks stop just short of a full answer here but not including how to specify credentials in the AWS.S3 client, here's the full version that you'll need:

const AWS = require('aws-sdk');
const s3 = require('s3');

...
  const awsS3Client = new AWS.S3({
    region: process.env.AWS_ZONE,
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    signatureVersion: 'v4',
  });

  const client = s3.createClient({
    s3Client: awsS3Client
  });

[slavafomin]

@LukasBombach Thank you sir, you've saved my day.

[manoj-compro]

@LukasBombach , @ryanostrom
I was also facing the same issue (Error: Non-file stream objects are not supported with SigV4 in AWS.S3) but after updating as per your suggestion I m getting an timeout error somehow. After 100% its end event does not get fired. Thanks in advance. As I m new to this so may be I am missing something here.

Here is the details :
"aws-sdk": "^2.478.0",
"s3": "^4.4.0",

const awsS3Client = new AWS.S3({
accessKeyId: argv.accessKeyId,
secretAccessKey: argv.secretAccessKey,
region: 'us-east-2',
signatureVersion: 'v4'
});

const client = s3.createClient({
s3Client: awsS3Client
});

let uploaderObj;

  uploaderObj = client.uploadDir(params);


let prevProgress = 0;
uploaderObj.on('progress', function () {
  if (prevProgress !== uploaderObj.progressAmount) {
    console.log(`Progress: ${Math.round(uploaderObj.progressAmount / uploaderObj.progressTotal * 100)}%`);
    prevProgress = uploaderObj.progressAmount;
  }
});

uploaderObj.on('error', (error) => {
  reject(error);
});

uploaderObj.on('end', () => {
  if (prevProgress === 0)
    console.log('Nothing to upload. Already up to date.');
  resolve('Done uploading.');
})

[BlackHole1]

@manoj-compro me too

[omarandrade]

@manoj-compro me too, same version of sdk and s3.

[BlackHole1]

@omarandrade @manoj-compro #223

[ManojKumarHC]

@manoj-compro
Even i am facing same issue , after 100% it is getting timeed out . the folder is not uploading to S3 . Any fix for this

[manoj-compro]

require('@auth0/s3') worked for me with same API

[ManojKumarHC]

Bro I did not understood , where i need to use this require('@auth0/s3') , can u share me the complete code . it will be helpfull

[manoj-compro]

uninstall S3 and install https://www.npmjs.com/package/@auth0/s3 this module as this is an extension of s3 and issue is fixed in @auth0/s3. All other functions/API are same.

in Js file replace require('s3') by require('@auth0/s3')

[ManojKumarHC]

Thanks bro , Now there is no error thrown . now i am getting "done uploading" but the folder is not getting synced , or uploaded to S3 , here is my code . please check once . help me out this .

const AWS = require('aws-sdk');
const s3 = require('@auth0/s3');

const awsS3Client = new AWS.S3( {
accessKeyId: accessKeyId,
secretAccessKey: secretAccessKey,
region: 'ap-south-1',
signatureVersion: 'v4'
});

const client = s3.createClient({
s3Client: awsS3Client
} );

console.log( 'my', mybucket1);
    var params = {
        localDir: './' + local1 ,
        deleteRemoved: false, 
        s3Params: {
          Bucket: mybucket1,
          Prefix: "/prefix",
          ACL: 'public-read'

        },
};

      var uploader = client.uploadDir(params);
      uploader.on('error', function(err) {
        console.error("unable to sync:", err.stack);
      });
      uploader.on('progress', function() {
        console.log("progress", uploader.progressAmount, uploader.progressTotal);
      });
      uploader.on('end', function() {
        console.log( "done uploading" );
  
      })

[bradgreens]

Nice work Bro

[ManojKumarHC]

@manoj-compro help me on the above code shared .. now i am getting error as "unable to sync: KMS.NotFoundException: Invalid arn"

[manoj-compro]

Below is the working code. if still not fix then try by changing region of bucket.

var s3 = require('@auth0/s3');
var s3Client = s3.createClient({
  s3Options: {
    accessKeyId: argv.accessKeyId
    secretAccessKey: argv.secretAccessKey 
    region: 'us-east-2'
  }
});

const params = {
      localDir: './dist',
      deleteRemoved: true,
      s3Params: {
        Bucket: config[argv.env].s3Bucket,
        Prefix: "",
      },
    };

 uploaderObj = s3Client.uploadDir(params);

[emilgeo]

uninstall S3 and install https://www.npmjs.com/package/@auth0/s3 this module as this is an extension of s3 and issue is fixed in @auth0/s3. All other functions/API are same.

in Js file replace require('s3') by require('@auth0/s3')

This solved the issue for us.