Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

invalid configuration option `:xray_recorder' on Kinesis #51

Open
mt-kelvintaywl opened this issue Jul 21, 2020 · 7 comments
Open

invalid configuration option `:xray_recorder' on Kinesis #51

mt-kelvintaywl opened this issue Jul 21, 2020 · 7 comments
Assignees

Comments

@mt-kelvintaywl
Copy link

mt-kelvintaywl commented Jul 21, 2020

Hi guys, are there better documentations on what dependencies i'll need to update for working with instrumenting this SDK with Ruby Lambda functions?

I followed the example, and have tried to instrument X-Ray with a Lambda function (with Kinesis data stream as its event source).

require 'aws-sdk-lambda'
require 'aws-xray-sdk/lambda'

require './lib/logger.rb'

xray_config = {
  context_missing: 'LOG_ERROR',
  logger: LOGGER
}
XRay.recorder.configure(xray_config)

def my_function(event:, context:)
  client = Aws::Lambda::Client.new

  # business logic

  client.get_account_settings.account_usage.to_h
end

However, I seem to hit a possible bug(?) :

invalid configuration option `:xray_recorder'

The redacted error trace:


"/var/task/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.72.0/lib/seahorse/client/configuration.rb:162:in `block in apply_options'",
"/var/task/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.72.0/lib/seahorse/client/configuration.rb:161:in `each'",
"/var/task/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.72.0/lib/seahorse/client/configuration.rb:161:in `apply_options'",
"/var/task/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.72.0/lib/seahorse/client/configuration.rb:149:in `build!'",
"/var/task/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.72.0/lib/seahorse/client/base.rb:62:in `build_config'",
"/var/task/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.72.0/lib/seahorse/client/base.rb:19:in `initialize'",
"/var/task/vendor/bundle/ruby/2.5.0/gems/aws-sdk-kinesis-1.20.0/lib/aws-sdk-kinesis/client.rb:273:in `initialize'",
"/var/task/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.72.0/lib/seahorse/client/base.rb:99:in `new'",

My dependencies are below:

GEM
  remote: https://rubygems.org/
  specs:
    ...
    aws-eventstream (1.1.0)
    aws-partitions (1.343.0)
    aws-record (2.4.0)
      aws-sdk-dynamodb (~> 1.18)
    aws-sdk-core (3.104.1)
      aws-eventstream (~> 1, >= 1.0.2)
      aws-partitions (~> 1, >= 1.239.0)
      aws-sigv4 (~> 1.1)
      jmespath (~> 1.0)
    aws-sdk-dynamodb (1.37.0)
      aws-sdk-core (~> 3, >= 3.71.0)
      aws-sigv4 (~> 1.1)
    aws-sdk-kinesis (1.26.0)
      aws-sdk-core (~> 3, >= 3.99.0)
      aws-sigv4 (~> 1.1)
    aws-sdk-lambda (1.31.0)
      aws-sdk-core (~> 3, >= 3.71.0)
      aws-sigv4 (~> 1.1)
    aws-sdk-sns (1.20.0)
      aws-sdk-core (~> 3, >= 3.71.0)
      aws-sigv4 (~> 1.1)
    aws-sdk-xray (1.4.0)
      aws-sdk-core (~> 3)
      aws-sigv4 (~> 1.0)
    aws-sigv4 (1.2.1)
      aws-eventstream (~> 1, >= 1.0.2)
    aws-xray-sdk (0.11.5)
      aws-sdk-xray (~> 1.4.0)
      multi_json (~> 1)
...

DEPENDENCIES
  aws-record
  aws-sdk-kinesis
  aws-sdk-lambda
  aws-sdk-sns
  aws-xray-sdk
  ...

Not sure if it's related to #26 but the fix mentioned here shows the Kinesis service is whitelisted so I'm not sure what exactly is going wrong?
Would i just need to bump my dependencies on the aws-sdk-* gems?

Thank you! 🙏

@srprash srprash self-assigned this Jul 28, 2020
@srprash
Copy link
Contributor

srprash commented Aug 3, 2020

Hi @mt-kelvintaywl, apologies for the delay in response. I believe having kinesis as your event source for the lambda function wouldn't cause the error since the X-Ray instrumentation is within the function and the message from kinesis stream is passed in as an event. Are you creating a kinesis client and/or calling kinesis service from within your lambda function?

@mt-kelvintaywl
Copy link
Author

Hi @srprash, thank you for reaching out and my apologies for the late response here myself too 🙇

I believe having kinesis as your event source for the lambda function wouldn't cause the error since the X-Ray instrumentation is within the function and the message from kinesis stream is passed in as an event.

Indeed, i would believe so too. Yet, this was happening.
For context:

  • this Lambda is setup to be triggered by the Kinesis stream events, and this lambda also sends a payload to another Kinesis stream (hence the aws-sdk-kinesis gem as a dependency).
  • We are deploying the Lambda code via Serverless Framework and also updated the Lambda's IAM role to allow for X-ray actions.

in the end, i've reverted and abandoned the instrumentation.
Let me close this for now, and perhaps re-open it once I can re-produce once once 🙇

@srprash
Copy link
Contributor

srprash commented Aug 11, 2020

Hi @mt-kelvintaywl
Thanks for providing more details here. I will try to reproduce this on my end as well. Please feel to re-open if you come across it in the future.

@omenking
Copy link

omenking commented Dec 20, 2020

I'm getting this error as well, just trying to do general instrutumentation.
Following what we have here:

https://docs.amazonaws.cn/en_us/lambda/latest/dg/ruby-tracing.html

In JS we have:

var AWSXRay = require('aws-xray-sdk');
var AWS = AWSXRay.captureAWS(require('aws-sdk'));

So its my assumption once things are instrumented I'll be able to see functions passed downstream, Since right now I'm just trying to see my Lambda call SNS in my service map. but how Ruby configures vs NodeJS is confusing me since next to no examples.

[Update] I found this while clicking through on one of the files in the documentation.

# lambda_function.rb
require 'logger'
require 'json'
require 'aws-sdk-lambda'
$client = Aws::Lambda::Client.new()
$client.get_account_settings()

require 'aws-xray-sdk/lambda'

def lambda_handler(event:, context:)
  logger = Logger.new($stdout)
  logger.info('## ENVIRONMENT VARIABLES')
  vars = Hash.new
  ENV.each do |variable|
    vars[variable[0]] = variable[1]
  end
  logger.info(vars.to_json)
  logger.info('## EVENT')
  logger.info(event.to_json)
  logger.info('## CONTEXT')
  logger.info(context)
  $client.get_account_settings().account_usage.to_h
end

I suppose that would let me record subsegments, but I just want to see my SNS topic getting called:

require 'aws-xray-sdk/lambda'
require 'aws-sdk-sns'

def handler(event:,context:)
  results = { hello: 'world' }

  sns = Aws::SNS::Resource.new
  sns.client.publish({
    topic_arn: ENV['TOPIC_ARN'],
    # need a default for SNS
    message: results.merge({default: 'evaluators-ruby'}).to_json,
    message_structure: 'json'
  })

  return {
    "statusCode" => 200,
    "body" => results.to_json
  }
end

Just checking through the full backtrace to see what I can find in thes respective Ruby Gems:

{
    "errorMessage": "invalid configuration option `:xray_recorder'",
    "errorType": "Function<ArgumentError>",
    "stackTrace": [
        "/var/task/vendor/bundle/ruby/2.7.0/gems/aws-sdk-core-3.110.0/lib/seahorse/client/configuration.rb:168:in `rescue in block in apply_options'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/aws-sdk-core-3.110.0/lib/seahorse/client/configuration.rb:164:in `block in apply_options'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/aws-sdk-core-3.110.0/lib/seahorse/client/configuration.rb:163:in `each'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/aws-sdk-core-3.110.0/lib/seahorse/client/configuration.rb:163:in `apply_options'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/aws-sdk-core-3.110.0/lib/seahorse/client/configuration.rb:151:in `build!'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/aws-sdk-core-3.110.0/lib/seahorse/client/base.rb:65:in `build_config'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/aws-sdk-core-3.110.0/lib/seahorse/client/base.rb:22:in `initialize'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/aws-sdk-sns-1.36.0/lib/aws-sdk-sns/client.rb:325:in `initialize'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/aws-sdk-core-3.110.0/lib/seahorse/client/base.rb:102:in `new'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/aws-sdk-sns-1.36.0/lib/aws-sdk-sns/resource.rb:28:in `initialize'",
        "/var/task/function.rb:119:in `new'",
        "/var/task/function.rb:119:in `handler'"
    ]
}

This error is similar to this one over here suppose: #26

@omenking
Copy link

omenking commented Dec 21, 2020

I looked into the the gems and noticed something called patching, so I added the following to my configuration and this stopped the error. Still can't see my Lambda calling SNS in my Service Map.

require 'aws-xray-sdk'

config = {
  patch: %I[net_http aws_sdk]
}

XRay.recorder.configure(config)

Carefully reading the docs more. Still don't see my downstream

The X-Ray SDK for Ruby automatically instruments all AWS SDK clients when you patch the aws-sdk library. You cannot instrument individual clients.

@omenking
Copy link

What is odd is that it looks like require 'aws-xray-sdk/lambda' should already patch, which is what I'm including...

# Set `XRAY_LAMBDA_PATCH_CONFIG` before requiring `aws-xray-sdk/lambda`
# to configure which libraries (if any) to instrument.
#
# By default, both `net_http` and `aws_sdk` will be instrumented
unless defined? XRAY_LAMBDA_PATCH_CONFIG
  XRAY_LAMBDA_PATCH_CONFIG = %I[net_http aws_sdk]
end

# Configure the XRay.recorder with Lambda specific config.
#
# From here, a lambda may create subsegments manually, or via
# the instrumented libraries setup by XRAY_LAMBDA_PATCH_CONFIG
XRay.recorder.configure(
  patch: XRAY_LAMBDA_PATCH_CONFIG,
  context: XRay::LambdaContext.new,
  streamer: XRay::LambdaStreamer.new
)

@bhautikpip bhautikpip reopened this Feb 25, 2021
@jessedoyle
Copy link
Contributor

I've encountered similar errors a number of times now with this library. It's probably worth it to take some time and document the issue.

When this invalid configuration option error occurs it's generally for one of two reasons:

  1. The client/service you're trying to use isn't whitelisted in this library yet.
  2. The application code is attempting to patch the AWS SDK before the SDK clients have been loaded. This isn't documented anywhere to my knowledge, but ensure your SDK clients are required prior to loading the Xray patches:

i.e.

require 'aws-sdk-lambda'
require 'aws-sdk-kinesis'
# be sure to require any other AWS SDK clients your app will use here
require 'aws-xray-sdk/lambda'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants