Usage API: breakdownBy=creatorId

[ecmulli]

• added the ability to breakdown by creator id to the usage metrics api
• Added nullable types to remove unused grouping fields (time interval, creator id).
• Fixed UsageWithTimestamp that did not return the time interval
• increased default query quota that consistently failed

Test Queries
I used Trovo's api key

import requests
url = 'http://127.0.0.1:8080/data/usage/query'
headers={'Authorization': 'Bearer ' + key}

# return all usage rolled up to the userid
params = {
    'from': int(time.mktime((datetime.utcnow() - timedelta(days=1)).timetuple()))*1000,
    'to': int(time.mktime(datetime.utcnow().timetuple()))*1000,
}

# return all usage rolled up to the userid and creator id
params = {
    'from': int(time.mktime((datetime.utcnow() - timedelta(days=1)).timetuple()))*1000,
    'to': int(time.mktime(datetime.utcnow().timetuple()))*1000,
    'breakdownBy[]':['creatorId' ], 
}

# returns all usage for just user broken down by hour
params = {
    'from': int(time.mktime((datetime.utcnow() - timedelta(days=1)).timetuple()))*1000,
    'to': int(time.mktime(datetime.utcnow().timetuple()))*1000,
    'timeStep':'hour',
}

# returns all usage for both user and creator id broken down by hour
params = {
    'from': int(time.mktime((datetime.utcnow() - timedelta(days=1)).timetuple()))*1000,
    'to': int(time.mktime(datetime.utcnow().timetuple()))*1000,
    'breakdownBy[]':['creatorId' ], 
    'timeStep':'hour',
}

# return all usage for just a specific creator
params = {
    'from': int(time.mktime((datetime.utcnow() - timedelta(days=1)).timetuple()))*1000,
    'to': int(time.mktime(datetime.utcnow().timetuple()))*1000,
    'creatorId':'67281_11878_11878'
}

# return all usage for just a specific creator broken down by hour and also throwing in a wrinkle where we try and group by the creator too
params = {
    'from': int(time.mktime((datetime.utcnow() - timedelta(days=1)).timetuple()))*1000,
    'to': int(time.mktime(datetime.utcnow().timetuple()))*1000,
    'breakdownBy[]':['creatorId' ], 
    'timeStep':'hour',
    'creatorId':'67281_11878_11878'
}

for param in params:
    print(param)
    r = requests.get(url, headers=headers, params=param)
    # pprint(r.json())
    if r.status_code != 200:
        print(r.text)
        break
    time.sleep(2)

[linear]

ENG-1693 Add creatorId parameter to Usage API

[victorges]

LGTM overall, but requesting changes due to the breaking changes to the API

[victorges]

Should we have a separate check for when we have breakdown vs when we don't? When we don't have any breakdown there should be only 1 result, so it'd be an error to have more results

[victorges]

This change from *UsageSummaryRow to []Metric is another breaking change. If a client was calling this API before and parsing the response as an object, it will now explode cause it's getting an array.

My suggestion is to build on top of the QuerySummary vs QuerySummaryWithTimestep separation, given the latter already returns an array. So, instead of changing every response to be an array, we could change QuerySummaryWithTimestep to be QuerySummaryWithBreakdown instead and update the check here to also call the breakdown version when len(breakdownBy) > 0. And then we leave the no-breakdown version still returning an object not to break any existing integrations. WDYT?

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 7 lines in your changes are missing coverage. Please review.

Project coverage is 28.99628%. Comparing base (364fe8e) to head (9d9dcea).

❗ Current head 9d9dcea differs from pull request most recent head 1c50458. Consider uploading reports for the commit 1c50458 to get more accurate results

Additional details and impacted files

@@                 Coverage Diff                 @@
##                main        #183         +/-   ##
===================================================
- Coverage   29.25926%   28.99628%   -0.26298%     
===================================================
  Files              4           5          +1     
  Lines            270         269          -1     
===================================================
- Hits              79          78          -1     
  Misses           180         180                 
  Partials          11          11                 

Files	Coverage Δ
views/bigquery.go	51.23967% <ø> (-0.39967%)	⬇️
views/client.go	0.00000% <ø> (ø)
views/query_spec.go	0.00000% <0.00000%> (ø)

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 364fe8e...1c50458. Read the comment docs.

Files	Coverage Δ
views/bigquery.go	51.23967% <ø> (-0.39967%)	⬇️
views/client.go	0.00000% <ø> (ø)
views/query_spec.go	0.00000% <0.00000%> (ø)