Fix returning nulls from /data/views/query if no data in BigQuery

[leszko]

Before this PR, if BigQuery returns null in some of the columns, then a call to /data/views/query returns an error:

"bigquery error: error reading query result: bigquery: NULL cannot be assigned to field `PlaytimeMins` of type float64"

After this change, it will return the following:

[
    {
        "viewCount": 0,
        "playtimeMins": null,
        "ttffMs": null,
        "rebufferRatio": null,
        "errorRate": null,
        "exitsBeforeStart": null
    }
]

Related to https://linear.app/livepeer/issue/PS-191/grafana-engagement-data-issues, though it does no solve the root cause of that issue.

[linear]

PS-191 Grafana Engagement Data Issues

[codecov]

Codecov Report

Attention: 7 lines in your changes are missing coverage. Please review.

Comparison is base (590d298) 28.00000% compared to head (bba6b97) 27.82875%.

Additional details and impacted files

@@                 Coverage Diff                 @@
##                main        #177         +/-   ##
===================================================
- Coverage   28.00000%   27.82875%   -0.17125%     
===================================================
  Files              4           4                 
  Lines            325         327          +2     
===================================================
  Hits              91          91                 
- Misses           223         225          +2     
  Partials          11          11                 

Files	Coverage Δ
views/bigquery.go	49.32432% <ø> (ø)
views/prometheus.go	0.00000% <ø> (ø)
views/client.go	0.00000% <0.00000%> (ø)

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 590d298...bba6b97. Read the comment docs.

Files	Coverage Δ
views/bigquery.go	49.32432% <ø> (ø)
views/prometheus.go	0.00000% <ø> (ø)
views/client.go	0.00000% <0.00000%> (ø)

[victorges]

LGTM in general, but I think we should never return null for the basic viewership fields (viewcount and playtime)

And thanks for picking this one up :)

[victorges]

Any idea how these can ever be null?

Not that I think we shouldn't make this fix, but these are the most basic fields that should always be there, defaulting to 0, so it kinda looks like a data pipeline problem.

[victorges]

These 2 fields are actually not detailed, they belong to the basic viewership metrics and should always be in the response. So they are always "asked".

Suggested change

	ViewCount: toInt64Ptr(row.ViewCount, spec.Detailed),
	PlaytimeMins: toFloat64Ptr(row.PlaytimeMins, spec.Detailed),
	ViewCount: toInt64Ptr(row.ViewCount, true),
	PlaytimeMins: toFloat64Ptr(row.PlaytimeMins, true),

(otherwise if you call the API without ?detailed=true you will get NO metrics on the response!)

[victorges]

Thinking further about this, we probably don't want to return null on the API responses for these fields, as we shouldn't leak the datasource bug to the end-user. So IMO these fields should always be present and instead we return 0 if they are null, WDYT? We could fix this directly on the SQL queries, like adding a COALESCE on those fields, or we could just have a logic here to check v == null ? 0 : v.

[leszko]

Before this PR, if BigQuery returns null in some of the columns, then a call to /data/views/query returns an error:

"bigquery error: error reading query result: bigquery: NULL cannot be assigned to field `PlaytimeMins` of type float64"

After this change, it will return the following:

[
    {
        "viewCount": 0,
        "playtimeMins": null,
        "ttffMs": null,
        "rebufferRatio": null,
        "errorRate": null,
        "exitsBeforeStart": null
    }
]

Related to https://linear.app/livepeer/issue/PS-191/grafana-engagement-data-issues, though it does no solve the root cause of that issue.

I think you're right, we should return 0, not null. I'm closing this PR and opened a new PR with the select query updated. PTAL: #178