RedShift graph no data

[pratapsingh]

Hi ,

I have configured ELB, Cloudfront and those are working fine with provided template but for RedShift it is not working. I am not getting any data on the zabbix.

Cron entry.
/bin/bash /opt/zabbix/cloudwatch/cron.d/cron.RedShift.sh "prod-Redshift-2017" "prod-Redshift-2017" "zabbix.production.com" "default" "us-east-1"

AWS use have full access to cloud watch. Also the same user is used for other service monitoring too.
Template is imported correctly. Can you help me here plz?

Thanks,
Pratap

[omni-lchen]

You can try to extend the period of gathering cloudwatch data, change start time from 5 mins ago to 15 mins ago, sometimes there is a delay in receiving latest data.

STARTTIME=$(date -u "+%F %H:%M:00" -d "15 minutes ago")

[pratapsingh]

@omni-lchen I tried with that option but did not work. Other monitoring is working fine ELB, Cache etc. Only I am facing this issue with Redshift. Redshift I am seeing only [no-data] on the graph page.

[omni-lchen]

You can uncomment the line "#zabbix_sender.printData()" in the function "sendLatestCloudWatchData", then run the bash script with parameters to see what data you get in cloudwatch.

[pratapsingh]

This is what I am getting IAM user have full access to cloudwatch and read access to RedShift in this case.

/bin/bash /opt/zabbix/cloudwatch/cron.d/cron.RedShift.sh "prod-redshift-123445" "prod-redshift-123445" "zabbix.example.com" "default" "us-east-1"
{'host': 'prod-redshift-123445', 'value': 0, 'key': u'RedShift.CPUUtilization.Average', 'clock': 1501591440}
{'host': 'prod-redshift-123445', 'value': 0, 'key': u'RedShift.DatabaseConnections.Average', 'clock': 1501591440}
{'host': 'prod-redshift-123445', 'value': 0, 'key': u'RedShift.HealthStatus.Average', 'clock': 1501591440}
{'host': 'prod-redshift-123445', 'value': 0, 'key': u'RedShift.MaintenanceMode.Average', 'clock': 1501591440}
{'host': 'prod-redshift-123445', 'value': 0, 'key': u'RedShift.NetworkReceiveThroughput.Average', 'clock': 1501591440}
{'host': 'prod-redshift-123445', 'value': 0, 'key': u'RedShift.NetworkTransmitThroughput.Average', 'clock': 1501591440}
{'host': 'prod-redshift-123445', 'value': 0, 'key': u'RedShift.PercentageDiskSpaceUsed.Average', 'clock': 1501591440}
{'host': 'prod-redshift-123445', 'value': 0, 'key': u'RedShift.ReadIOPS.Average', 'clock': 1501591440}
{'host': 'prod-redshift-123445', 'value': 0, 'key': u'RedShift.ReadLatency.Average', 'clock': 1501591440}
{'host': 'prod-redshift-123445', 'value': 0, 'key': u'RedShift.ReadThroughput.Average', 'clock': 1501591440}
{'host': 'prod-redshift-123445', 'value': 0, 'key': u'RedShift.WriteIOPS.Average', 'clock': 1501591440}
{'host': 'prod-redshift-123445', 'value': 0, 'key': u'RedShift.WriteLatency.Average', 'clock': 1501591440}
{'host': 'prod-redshift-123445', 'value': 0, 'key': u'RedShift.WriteThroughput.Average', 'clock': 1501591440}
Count: 13

[pratapsingh]

@omni-lchen Can you let me know what changes I should do to make it work?

[omni-lchen]

Are you able to see redshift metrics from cloudwatch in AWS console? Please check the cluster identifier is correct.

[pratapsingh]

@omni-lchen Yes I see cloudwatch metrics from aws frontend and identifier is correct as well.

[omni-lchen]

If all are set correctly and you still get the value 0, it could be either the issue with a timestamp, dimensions or region. Could you manually set the value in the bash script?
For example:
STARTTIME="2017-08-14 21:00:00"
ENDTIME="2017-08-14 21:10:00"

You can also test different dimension with NodeID.
For example:
-d "NodeID="

[pratapsingh]

I tried again but looks something else is missing here. My Zabbix server was in us-west and redshift is in us-east however when I tried the same from us-east zabbix proxy server result is same.

All time across the servers and redshift is in UTC only which is default.

[SithSyntax]

I am having a similar issues. It almost seems like dimensions filtering.. will return a blank dataset no matter what ever I send to the command line.

./zabbixCloudWatch.py -z zabbix.venky.com -x venkyRds -a 1234 -r us-east-1 -s RedShift -d ClusterIdentifier=venkyRds -p 60 -f "2017-08-30 00:00:00" -t "2017-08-30 05:00:00"

and ./zabbixCloudWatch.py -z zabbix.venky.com -x venkyRds -a 1234 -r us-east-1 -s RedShift -d 1=2 -p 60 -f "2017-08-30 00:00:00" -t "2017-08-30 05:00:00"

return the same?

@omni-lchen I assume you have seen it working ? .. thanks again for this project its super helpful

[omni-lchen]

It was working before when I tested with Python 2.7 and Boto2, possibly AWS has changed something in Python API since then, could you try to test with AWS CLI see you can get the results.

[SithSyntax]

yup tried with aws cli and got results..

-mac:~ $ aws cloudwatch get-metric-statistics --namespace 'AWS/Redshift' --metric-name PercentageDiskSpaceUsed --start-time 2017-09-08T21:10:00Z --end-time 2017-09-08T22:10:00Z --period 300 --statistics Average --dimensions Name=NodeID,Value=Compute-0 Name=ClusterIdentifier,Value=REDSHIFT-VENKY
PercentageDiskSpaceUsed
DATAPOINTS	39.2626953125	2017-09-08T21:55:00Z	Percent
DATAPOINTS	39.2626953125	2017-09-08T21:10:00Z	Percent
DATAPOINTS	39.2626953125	2017-09-08T21:45:00Z	Percent
DATAPOINTS	39.2626953125	2017-09-08T21:30:00Z	Percent

[omni-lchen]

Try to test in python script with the parameters where you specified in AWS CLI, but change dimension as:
-d "ClusterIdentifier=REDSHIFT-VENKY,NodeID=Compute-0"

[SithSyntax]

i tried all possible dimension arguments.. its almost like dimensions part is not getting parsed?
is there an easy way to see the actual aws cli call?

[root@mon02 cloudwatch-venky]# ./zabbixCloudWatch.py -z zabbix.int.venky.com -x REDSHIFT-VENKY.us-east-1.redshift.amazonaws.com -a 1234  -r us-east-1 -s RedShift -d ClusterIdentifier=REDSHIFT-VENKY,NodeID=Compute-0 -p 60 -f "2017-09-16 00:00:00" -t "2017-09-16 05:00:00"
helo
[   {   'cloud_watch_results': [],
        'statistics': u'Average',
        'zabbix_key': u'RedShift.CPUUtilization.Average'},
    {   'cloud_watch_results': [],
        'statistics': u'Average',
        'zabbix_key': u'RedShift.DatabaseConnections.Average'},
    {   'cloud_watch_results': [],
        'statistics': u'Average',
        'zabbix_key': u'RedShift.HealthStatus.Average'},
    {   'cloud_watch_results': [],
        'statistics': u'Average',
        'zabbix_key': u'RedShift.MaintenanceMode.Average'},
    {   'cloud_watch_results': [],
        'statistics': u'Average',
        'zabbix_key': u'RedShift.NetworkReceiveThroughput.Average'},
    {   'cloud_watch_results': [],
        'statistics': u'Average',
        'zabbix_key': u'RedShift.NetworkTransmitThroughput.Average'},
    {   'cloud_watch_results': [],
        'statistics': u'Average',
        'zabbix_key': u'RedShift.PercentageDiskSpaceUsed.Average'},
    {   'cloud_watch_results': [],
        'statistics': u'Average',
        'zabbix_key': u'RedShift.ReadIOPS.Average'},
    {   'cloud_watch_results': [],
        'statistics': u'Average',
        'zabbix_key': u'RedShift.ReadLatency.Average'},
    {   'cloud_watch_results': [],
        'statistics': u'Average',
        'zabbix_key': u'RedShift.ReadThroughput.Average'},
    {   'cloud_watch_results': [],
        'statistics': u'Average',
        'zabbix_key': u'RedShift.WriteIOPS.Average'},
    {   'cloud_watch_results': [],
        'statistics': u'Average',
        'zabbix_key': u'RedShift.WriteLatency.Average'},
    {   'cloud_watch_results': [],
        'statistics': u'Average',
        'zabbix_key': u'RedShift.WriteThroughput.Average'}]

[omni-lchen]

Could be dimension issue, but the dimension format should be correct, otherwise the script will throw out an error. I have no redshift here to test, sorry can't help debug further.

[SithSyntax]

thanks ? anyways for me to see the actually aws cli call thats getting made?
if you can point me to the code thats constructs the aws cli call ? i assume thats how it works? a call to aws cli?

[omni-lchen]

This script uses aws boto api, please check this fucntion "get_metric_statistics" in http://boto.cloudhackers.com/en/latest/ref/cloudwatch.html, hope that help.

[0xWW]

@omni-lchen @venkysomes everything working for me except Redshift metrics. do you have work alternative to pull the Redshift Metrics.

[SithSyntax]

i ended up using aws alerts go directly to pagerduty.

[0xWW]

Thanks @venkysomes . @omni-lchen do you have any sample conf file for EMR.

[PUCHAJDA]

Thanks Yurij, Now it works like a charm for Red(s)hift metrics !

RESPECT 🥇