Not able to see snowflakeAccount event type in New Relic after following installation steps

[ssontakke]

Description

I followed the installation steps mentioned in the README. I cannot see Snowflake events in NewRelic.

Here is my config.yaml file:

#authentication:
#  obfuscation: # optional
#    key: replaceme
credentials:
  account: ***.us-west-2
  user: NEWRELIC_AGENT
  password: ***
  role: SYSADMIN
 # warehouse: DEMO_WH

I have commented # warehouse: DEMO_WH as I want to monitor all warehouses. I hope I am right here in my understanding of that config variable.

When I checked last login in Snowflake for NEWRELIC_AGENT, I didn't see that it logged in ever.

My infra agent is sending logs to NewRelic, but nothing of Snowflake.

In NewRelic, I am checking the data in Data Explorer > Events > Event Type = SnowflakeAccount

Steps to Reproduce

I followed the steps mentioned in the README. Updated config.yaml as mentioned there.

Expected Behavior

If someone follows the README steps, things should work like a charm.

Relevant Logs / Console output

My /var/log/newrelic-infra/ directory is empty.

Contents of my /etc/systemd/system/newrelic-infra.service

[Unit]
Description=New Relic Infrastructure Agent
After=dbus.service syslog.target network.target

[Service]
RuntimeDirectory=newrelic-infra
Type=simple
ExecStart=/usr/bin/newrelic-infra-service
EnvironmentFile=/home/ssm-user/newrelic-snowflake-integration/snowflake.env
MemoryLimit=1G
# MemoryMax is only supported in systemd > 230 and replaces MemoryLimit. Some c$
# MemoryMax=1G
Restart=always
RestartSec=20
StartLimitInterval=0
StartLimitBurst=5
PIDFile=/var/run/newrelic-infra/newrelic-infra.pid

[Install]
WantedBy=multi-user.target

Here are last few messages from sudo cat /var/log/messages | grep 'newrelic-infra-service'

Dec 29 10:24:12 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:12Z" level=info msg="Creating service..."
Dec 29 10:24:12 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:12Z" level=info msg="runtime configuration" agentUser=root component="New Relic Infrastructure Agent" executablePath= maxProcs=1 pluginDir="[/etc/newrelic-infra/integrations.d /var/db/newrelic-infra/integrations.d]"
Dec 29 10:24:12 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:12Z" level=info msg="Checking network connectivity..." component=AgentService service=newrelic-infra
Dec 29 10:24:12 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:12Z" level=info msg=Initializing component=AgentService elapsedTime=207.738261ms service=newrelic-infra version=1.20.7
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="New Relic infrastructure agent is running." component=AgentService elapsedTime=1.066179655s service=newrelic-infra
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Starting up agent..." component=Agent
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=metadata/attributes
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=metadata/system
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=metadata/host_aliases
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=metadata/agent_config
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=metadata/proxy_config
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=services/upstart
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=services/systemd
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=metadata/facter_facts
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=sessions/users
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=services/daemontools
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=services/supervisord
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=system/network_interfaces
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=kernel/sysctl
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=kernel/modules
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=services/pidfile
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=config/sshd
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=packages/rpm
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=config/selinux
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Agent plugin" plugin=metadata/cloud_security_groups
Dec 29 10:24:13 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:13Z" level=info msg="Integration health check finished with success" component=integrations.runner.Runner integration_name=nri-flex
Dec 29 10:24:14 ip-10-21-24-19 newrelic-infra-service: time="2021-12-29T10:24:14Z" level=info msg="connect got id" agent-guid=Mjg3NzQ1MnxJTkZSQXxOQXwyODI4NjM2Nzc0NzY2Mjc3MTc2 agent-id=2828636774766277176 component=IdentityConnectService

Your Environment

Infra agent installed on AWS EC2 T2.micro using Amazon Linux
amzn2-ami-kernel-5.10-hvm-2.0.20211201.0-x86_64-gp2

[ssontakke]

Update:
I am getting data in NewRelic now but it's not as expected.

SELECT * FROM SnowflakeVirtualWarehouse SINCE 2 minutes ago gives me:

SELECT * FROM SnowflakeAccount SINCE 2 minutes ago gives me:

ERROR=exit status 127
ERROR EXEC=$$NEWRELIC_SNOWFLAKE_HOME/snowflakeintegration-linux $$NEWRELIC_SNOWFLAKE_HOME/config.yaml $$NEWRELIC_SNOWFLAKE_HOME/queries/database_storage_usage.sql
Error Msg: /bin/sh: 28276NEWRELIC_SNOWFLAKE_HOME/snowflakeintegration-linux: No such file or directory
Event Type: SnowflakeAccount
Snowflake Account: $SNOWSQL_ACCOUNT
Metric Type: snowflake.database_storage

Why it is sending $SNOWSQL_ACCOUNT or $$NEWRELIC_SNOWFLAKE_HOME/queries/database_storage_usage.sql? I think it's not able to convert the env variables? I am using AWS Linux Image.

[ssontakke]

If I replace all $$ env variables from /etc/newrelic-infra/integrations.d/flex-snowflake-linux.yml with actual values, it works. So things are inclined towards env variables setup issue.

[adlangx]

Were you able to get the snowflake data streaming to New Relic?

I replaced all the paths, but I still don't see the snowflake types under SnowflakeAccount or SnowflakeVirtualWarehouse