How to create visualisations other than tables

[atjbramley]

Hi,

Thank you for this plugin. It looks like it will be wonderful once I've learned how to use it.

I have managed to use Infinity Datasource to pull data from an OAuth 2 API. I am now receiving a stream of .json in the following example format:

{
    "continuationToken": null,
    "data": [
        {
            "actingUserId": "83cd55a9-95bf-4eb5-a221-af4900c54bf7",
            "collectionId": null,
            "date": "2022-11-10T18:10:20.8466667Z",
            "device": 10,
            "groupId": null,
            "installationId": null,
            "ipAddress": "xxx.xxx.xxx.xxx",
            "itemId": "a8478aa6-5dc7-4973-b8f9-af49012b78fb",
            "memberId": null,
            "object": "event",
            "policyId": null,
            "type": 1100
        },
        {
            "actingUserId": "83cd55a9-95bf-4eb5-a221-af4900c54bf7",
            "collectionId": null,
            "date": "2022-11-10T15:00:29.8533333Z",
            "device": 10,
            "groupId": null,
            "installationId": null,
            "ipAddress": "xxx.xxx.xxx.xxx",
            "itemId": "4bfa6ac8-d26c-48b0-b8a3-af4900f7540b",
            "memberId": null,
            "object": "event",
            "policyId": null,
            "type": 1100
        },
        {
            "actingUserId": "83cd55a9-95bf-4eb5-a221-af4900c54bf7",
            "collectionId": null,
            "date": "2022-11-10T14:58:41.94Z",
            "device": 10,
            "groupId": null,
            "installationId": null,
            "ipAddress": "xxx.xxx.xxx.xxx",
            "itemId": null,
            "memberId": null,
            "object": "event",
            "policyId": null,
            "type": 1600
        }
    ],
    "object": "list"
}

I am able to manipulate this data in a Table Format - selecting the colums that I want and overlaying Value Mappings to make the data more useful.

I would like to start creating other visualisations based on this data, e.g.

• time series graph showing (count of entries by 'ip address') Vs time
• time series graph showing (count of entries by 'type' ) Vs time

While I am able to create such visualisations if the data was say log data streamed from Loki, I don't seem to be able to do anything with the data from Infinity, even if the tables created appear very similar.

I have a feeling that I'll need to either Transform the data, or use UQL, but after spending a couple of hours playing and looking at similar discussions on this list I'm afraid I've not made any progress.

I'd very much appreciate any guidance you could give me!

[svet-b]

My quick take on this is that you can to set the date field to time to get Grafana to recognize your data as timeseries data

which should open up all sorts of graphing opportunities.

From there on I guess you can play around to get the exact functionality you need. It's possible that Grafana's built-in charting options will allow for what you need. Though maybe UQL is indeed the answer to getting counts-by-time. I'm afraid I'm not yet super-familiar with the Infinity plugin itself, so someone else might be better positioned to give advise there.

[atjbramley]

Thanks for this suggestion. I'm not an advanced Grafana user, but Time-series graphs appear to require 2 types of data - time and number.

The issue that I'm facing isn't actually getting the time into the graph - as you say I can get a timestamp just fine. However, looking at the visualisations, I get a 'Data is missing a number field' error.

Using say Loki + Promtrail I can aggregate batches of logs together, resulting in a number. So for variable X, I get

• 3 entries in the first 5 minute span
• 4 entries in the 2nd 5 minute span
• 8 entries in the 3rd 5 minute span etc etc...

You can see how I can produce a time series visualisation. The query is expressed as:

count_over_time({$variable}[5m])

When I export using Infinity this seems to be missing as a built-in function, and I'm lacking the skill to recreate it with Transform or UQL or similar.

Any help in doing so is gratefully received :)

[svet-b]

Right, I see. I'm afraid that I'm not familiar with UQL itself to offer much guidance. Skimming the docs, I don't really see any functions that would do time-bucketing the way you're describing (and the way that other query languages generally allow for).

It's maybe not quite what you're after, but you could also try the histogram or heat-map panels, since one of those might yield the kind of visualization you want, without any additional data processing.

[atjbramley]

I've spent another few hours on this and have turned up the following:

https://sriramajeyam.com/grafana-infinity-datasource/blog/aws-status-feeds/

I basically want to create exactly that first visualisation - but when reproducing the settings shown in the screenshots I'm in the same situation as I am with my own data - I can't produce a time-series graph because I have no numerical data.

Would be most grateful if someone could tell me what I'm missing.

[buhaytza2005]

Has anyone figured this out?

[atjbramley]

No - I ended up building a custom importer and processing directly using Promtail/Loki.