SDK errors with latest Bedrock clients import_client.setCredentialFeature and util_endpoints_2.EndpointCache is not a constructor

[abhirpat]

Checkboxes for prior research

• I've gone through Developer Guide and API reference
• I've checked AWS Forums and StackOverflow.
• I've searched for previous similar issues and didn't find any solution.

Describe the bug

The npm package documentation shows RetrieveAndGenerateStream is available but it throws Error: TypeError: RetrieveAndGenerateStreamCommand is not a constructor with latest version.

Regression Issue

• Select this option if this issue appears to be a regression.

SDK version number

@aws-sdk/client-bedrock-agent-runtime": "3.706.0"

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v20.18.1

Reproduction Steps

To reproduce, try importing RetrieveAndGenerateStreamCommand and it's not available

Observed Behavior

Throws error

Error: TypeError: RetrieveAndGenerateStreamCommand is not a constructor
    at rag (/Users/Documents/Scripts/Node/bedrock-kb-dual.js:43:19)
    at Object.<anonymous> (/Users/Documents/Scripts/Node/bedrock-kb-dual.js:50:1)
    at Module._compile (node:internal/modules/cjs/loader:1469:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1548:10)
    at Module.load (node:internal/modules/cjs/loader:1288:32)
    at Module._load (node:internal/modules/cjs/loader:1104:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:173:12)
    at node:internal/main/run_main_module:28:49

Expected Behavior

The command should be made available since it is documented in the package.

Possible Solution

No response

Additional Information/Context

No response

[RanVaknin]

Hi @abhirpat ,

I'm able to import it just fine:

Are you sure you are using the latest version? What happens if you run this command?

npm ls -a @aws-sdk/client-bedrock-agent-runtime

Thanks,
Ran~

[anandakumarpalanisamy]

I get a similar error in AWS SDK for JavaScript V3 @aws-sdk/[email protected]

{
  "errorType": "TypeError",
  "errorMessage": "import_client_transfer.ListFileTransferResultsCommand is not a constructor",
  "trace": [
    "TypeError: import_client_transfer.ListFileTransferResultsCommand is not a constructor",
    "    at checkFileTransfer (/var/task/index.js:1068:42)",
    "    at Runtime.handler (/var/task/index.js:2511:28)",
    "    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"
  ]
}

[RanVaknin]

Hi @anandakumarpalanisamy ,

I can import this command as well. Same question to you:

What happens if you run this command?

npm ls -a @aws-sdk/client-transfer

Thanks,
Ran~

[abhirpat]

@RanVaknin While I resolved the initial dependency issue, upgrading to "@aws-sdk/client-bedrock-runtime": "^3.706.0" from 3.616.0 has introduced new compatibility problems. These errors could be due to the newer version conflicts with other AWS SDK clients (version 3.5XX.0 or 3.6XX.0) that are present in the Lambda function or layers. After reverting 3.706.0 to 3.616.0 fixes it.

Could you please help investigate the source code to identify what's causing these issues? These version conflicts between AWS SDK packages are making maintenance difficult, as even minor version upgrade on one single package can lead to breaking changes and require upgrade of other packages.

2024-12-07T04:49:40.519Z	d0aae169-c7ff-4903-a855-f8dfeef9ef28	ERROR	Invoke Error 	{
    "errorType": "Error",
    "errorMessage": "{\"message\":\"Bedrock amazon.titan-embed-text-v1 returned TypeError: (0 , import_client.setCredentialFeature) is not a function\",\"type\":\"Error\"}",
    "stack": [
        "Error: {\"message\":\"Bedrock amazon.titan-embed-text-v1 returned TypeError: (0 , import_client.setCredentialFeature) is not a function\",\"type\":\"Error\"}",
        "    at bedrockClient (/opt/lib/bedrock/bedrockClient.js:88:15)",
        "    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)",
        "    at async invokeBedrockModel (/opt/lib/bedrock/bedrockModels.js:55:22)",
        "    at async getEmbeddingsBedrock (/opt/lib/embeddings.js:36:24)",
        "    at async /opt/lib/esbodybuilder.js:128:45",
        "    at async Object.run_query_es (/opt/lib/es_query.js:71:22)",
        "    at async runQuery (/opt/lib/fulfillment-event/getHit.js:30:20)",
        "    at async runFirstQuery (/opt/lib/fulfillment-event/getHit.js:178:22)",
        "    at async getHit (/opt/lib/fulfillment-event/getHit.js:388:23)",
        "    at async getInitialHit (/opt/lib/fulfillment-event/processFulfillmentEvent.js:110:37)"
    ]
}

2024-12-07T05:05:30.296Z	undefined	ERROR	Uncaught Exception 	{
    "errorType": "TypeError",
    "errorMessage": "util_endpoints_2.EndpointCache is not a constructor",
    "stack": [
        "TypeError: util_endpoints_2.EndpointCache is not a constructor",
        "    at Object.<anonymous> (/opt/nodejs/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/endpoint/endpointResolver.js:7:15)",
        "    at Module._compile (node:internal/modules/cjs/loader:1469:14)",
        "    at Module._extensions..js (node:internal/modules/cjs/loader:1548:10)",
        "    at Module.load (node:internal/modules/cjs/loader:1288:32)",
        "    at Module._load (node:internal/modules/cjs/loader:1104:12)",
        "    at Module.require (node:internal/modules/cjs/loader:1311:19)",
        "    at require (node:internal/modules/helpers:179:18)",
        "    at Object.<anonymous> (/opt/nodejs/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/runtimeConfig.shared.js:10:28)",
        "    at Module._compile (node:internal/modules/cjs/loader:1469:14)",
        "    at Module._extensions..js (node:internal/modules/cjs/loader:1548:10)"
    ]
}

[RanVaknin]

Hi @abhirpat,

This seems to be related to general issue of dependency management rather than an SDK related issue.
If you have different SDK clients pinned to a different minor versions, these will rely on different versions of the same utility dependencies like middleware-stack and endpoint-resolver and will lead to issues.

You need to make sure to update all of your @aws-sdk/* to the same minor version to avoid the transitive dependency issue.

Thanks,
Ran~

[abhirpat]

@RanVaknin Thank you for your response. As I mentioned earlier, this has been my observation as well. I wanted to ask if it would be possible to make this is a feature request to the SDK team to minimize transitive dependencies that sdk client use. This would help reduce dependency conflicts or innovate other way to improve isolation between different clients.

The main challenge we’re facing is that adding a new command to one client currently requires upgrading all SDK clients in the same code package for Lambda/layers. Increasing isolation could make these sdk dependencies more manageable.

Would this be something the team might consider? Also, is there a documentation you recommend on best practices with sdk v3 dependency management?

[RanVaknin]

Hi @abhirpat,

What you are suggesting is not feasible and would require rebuilding the SDK from the ground up. There are core dependencies that we have to supply to each SDK client because each client is designed to be its own standalone module.

We have this existing Feature request.

The existing feature request proposes using caret ranges for versioning to help manage minor updates and reduce conflicts. However, it doesn't fully eliminate the need for managing dependencies. While this method can help manage some conflicts by ensuring more consistent updates, developers must still monitor and adjust their setups to avoid issues from these shared dependencies.

If you think that feature request is helpful, feel free to give it a thumbs up as it helps with prioritization.

Since what you are asking for is not possible, I suggest we focus on unblocking you for now. Are you able to mitigate the errors you are getting by making sure all the minor versions aligned to the @latest?

Thanks,
Ran~

[abhirpat]

Hi @RanVaknin I upgraded all other SDK dependences to latest but now they are throwing error. Here is error from client-comprehend @aws-sdk/client-comprehend": "3.699.0"

{
    "errorType": "TypeError",
    "errorMessage": "util_endpoints_2.EndpointCache is not a constructor",
    "stack": [
        "TypeError: util_endpoints_2.EndpointCache is not a constructor",
        "    at Object.<anonymous> (/opt/nodejs/node_modules/@aws-sdk/client-comprehend/dist-cjs/endpoint/endpointResolver.js:7:15)",
        "    at Module._compile (node:internal/modules/cjs/loader:1469:14)",
        "    at Module._extensions..js (node:internal/modules/cjs/loader:1548:10)",
        "    at Module.load (node:internal/modules/cjs/loader:1288:32)",
        "    at Module._load (node:internal/modules/cjs/loader:1104:12)",
        "    at Module.require (node:internal/modules/cjs/loader:1311:19)",
        "    at require (node:internal/modules/helpers:179:18)",
        "    at Object.<anonymous> (/opt/nodejs/node_modules/@aws-sdk/client-comprehend/dist-cjs/runtimeConfig.shared.js:10:28)",
        "    at Module._compile (node:internal/modules/cjs/loader:1469:14)",
        "    at Module._extensions..js (node:internal/modules/cjs/loader:1548:10)"
    ]
}

[RanVaknin]

Hi @abhirpat,

Seems like there is still a dependency issue. How are you running your code? Are you running it locally from a node js env? or deploying it somewhere like Lambda or docker?

You have a few options to identify where the discrepancies come from.

npm ls to see all the minor versions of dependencies that are @aws-sdk/* You can supplement these with npm ls -a @aws-sdk/client-* to identify if you have multiple copies of the same dependency set to different minor versions.

Another option is to let NPM dedupe those dependencies for you by running npm dedupe. Or, you can delete node_modules and package-lock.json and then running npm install to generate a new lock file.

I'm not sure why you are running into this issue consistently, it might be due to a outdated lock file, or something in your infrastructure that is impacting how dependencies are resolved. Without knowing your setup it will be difficult to diagnose.

Thanks,
Ran~

[abhirpat]

Hi @RanVaknin it is deployed in the lambda and different layers. It is open source here. Before every build deployed, the node modules are removed and re-packaged. Could you please elucidate on your previous initial line of thought "there is still a dependency issue"?

[RanVaknin]

Hi @abhirpat ,

What I meant by "there is still a dependency issue" is that the SDK is working fine, I'm able to import the aforementioned packages without seeing this error which leads me to believe that the lambda layers is where this issue is manifesting.

From provided repo code:

        "@aws-sdk/client-bedrock-agent-runtime": "^3.616.0",
        "@aws-sdk/client-bedrock-runtime": "^3.616.0",
        "@aws-sdk/client-firehose": "^3.511.0",
        "@aws-sdk/client-sagemaker-runtime": "^3.511.0",
		"@aws-sdk/s3-request-presigner": "^3.511.0",

You clearly have multiple SDK clients set to different versions. Can you please update them all to the SAME latest version (today we are on 3.705.0) and see if it solves your issue?

Thanks,
Ran~

[abhirpat]

@RanVaknin Yes, I already have upgraded to all the versions. You can ignore that file. That's not showing you development I was talking about because it is a previous release. It was for your reference which dependencies are involved.

[RanVaknin]

Hi @abhirpat ,

There must be an issue with the packaging step then. I'm able to create a NodeJS lambda with a layer bundling Bedrock runtime client and comprehend and run a comprehend command and I'm not seeing the error you were running into:

import { readdir, readFile } from 'fs/promises';
import { join } from 'path';
import { ComprehendClient, ListEndpointsCommand } from '@aws-sdk/client-comprehend';

export const handler = async () => {
    try {
        const sdkPath = '/opt/nodejs/node_modules/@aws-sdk';
        const modules = await readdir(sdkPath);
        const versions = {};
        for (const mod of modules) {
            const pkg = await readFile(join(sdkPath, mod, 'package.json'), 'utf8');
            versions[mod] = JSON.parse(pkg).version;
        }

        const client = new ComprehendClient();
        const response = await client.send(new ListEndpointsCommand({}));
        
        return {
            statusCode: 200,
            body: { versions, response }
        };
    } catch (err) {
        console.log('Error:', err);
        return { error: err.message };
    }
};

results in:

{
  "statusCode": 200,
  "body": {
    "versions": {
      "client-bedrock-runtime": "3.709.0",
      "client-sso": "3.709.0",
      "client-sso-oidc": "3.709.0",
      "client-sts": "3.709.0",
      "core": "3.709.0",
      "credential-provider-env": "3.709.0",
      "credential-provider-http": "3.709.0",
      "credential-provider-ini": "3.709.0",
      "credential-provider-node": "3.709.0",
      "credential-provider-process": "3.709.0",
      "credential-provider-sso": "3.709.0",
      "credential-provider-web-identity": "3.709.0",
      "middleware-host-header": "3.709.0",
      "middleware-logger": "3.709.0",
      "middleware-recursion-detection": "3.709.0",
      "middleware-user-agent": "3.709.0",
      "region-config-resolver": "3.709.0",
      "token-providers": "3.709.0",
      "types": "3.709.0",
      "util-endpoints": "3.709.0",
      "util-locate-window": "3.693.0",
      "util-user-agent-browser": "3.709.0",
      "util-user-agent-node": "3.709.0"
    },
    "response": {
      "$metadata": {
        "httpStatusCode": 200,
        "requestId": "REDACTED",
        "attempts": 1,
        "totalRetryDelay": 0
      },
      "EndpointPropertiesList": []
    }
  }
}

You can use this snippet to print out the various versions your layer is using. If you see a mismatch, its probably an issue with how you bundle your layer.

You can try:

$(DST): $(RESOURCES) 
	echo "Building $(NAME)"; 
	rm -r ./nodejs || true
	rm -r ./node_modules || true
	npm install  -production
+	npm dedupe
	mkdir ./nodejs 
	mv node_modules ./nodejs/node_modules 
	rm -r $(DST) || true
	zip -r $(DST) .

Thanks,
Ran~

[abhirpat]

Thanks @RanVaknin. With recent release of sdk, these issues resolved itself. We are good to close.