New Endpoint: UTxOs with specific assets

[fallen-icarus]

Is your feature request related to a problem? Please describe.
It is not a problem per se but I believe the community would benefit from this query. The Beacon Token standard I am trying to create allows for fully p2p dApps such as these. They do not require any middle-men (batchers) due to using dedicated NFTs for indexing the UTxO set. In addition to this, since there are at least as many UTxOs as there are users, the concurrency of these p2p dApps scales with the number of users.

Blockfrost has this query that I have been using for the proof of concepts. Koios does not have an equivalent endpoint.

Describe the solution you'd like
To reach the full potential of using beacon tokens, there needs to be an endpoint for getting the information for all UTxOs with a specific asset. Blockfrost requires first knowing the address but ideally it should be a direct query. The query should require a policy id but optionally take an asset name like this Koios query.

Describe alternatives you've considered

1. Using both Blockfrost and Koios. Blockfrost does not allow querying based only on the policy id while Koios does not allow querying UTxOs with a specific asset. Neither application can be used in isolation since both queries are required for some p2p dApps. This is not ideal since Blockfrost is centralized and requires its own api key.
2. Getting all the UTxOs from the address using Koios and filtering the UTxOs locally. This would put a burden on the end user who may be trying to interact with the dApps from a smartphone or light wallet. It would be better if the endpoint did the filtering.

Additional context
The query response should return:

1. The address where the UTxO is located.
2. The tx hash.
3. The output index.
4. The asset balance for the UTxO.
5. The datum hash.
6. The inline datum (if present).
7. Whether or not it has a reference script.

The query can also return other information but these are the things I need for the p2p dApps.

[rdlrt]

There is already asset_txs endpoint - it doesn't yet give full information, which will be enhanced further with utxo_info as part of #186 . Unlike other API layers, you currently do (and will have with utxo_info) ability to specify multiple utxos in bulk to utxo_info endpoint

[fallen-icarus]

@rdlrt Will the enhanced versions have a way to query UTxOs that have a specific combination of assets? For example, "show me all UTxOs that have asset X AND asset Y". It is okay if there are other assets in the UTxO but it would need to have at least the supplied assets.

[rdlrt]

asset_utxos would provide that, yes..while address_utxos (the original request) in linked issue already has that in desired output fields

[fallen-icarus]

address_utxos (the original request) in linked issue already has that in desired output fields

While I know I can always filter the information after receiving the response (eg, get all UTxOs with asset X and then locally filter for asset Y), I am asking to have the Koios database do the filtering before sending the response to the end user. The hardware used for Koios databases would likely be stronger than hardware that end users are using (eg, tablets and phones). It is possible that the p2p protocols could have 1000-10000 UTxOs available but only a few hundred are relevant to the specific end user. Therefore I think, if possible, it makes more sense to have the Koios database do the filtering.

[rdlrt]

I am asking to have the Koios database do the filtering before sending the response to the end user.

well with asset_utxos / asset_txs, you would be able to supply multiple assets and return utxos relevant to those assets only from server side (regardless of which address is associated with that request). If instead you want to go from address -> utxo with a filter, that will not be done on free tier as it predominantly spams the servers allowing queries against tokens with millions of utxos going from address > utxo > ma_utxo joined with asset.
Ofcourse, with a dedicated infra plan - custom queries will be possible. The idea is giving a mid-way between viable options (that are sustainable in long term) on public tier, while non-sustainable exponentially increasing data to be queryable on custom solutions with desired filters using dedicated tiers

[fallen-icarus]

well with asset_utxos / asset_txs, you would be able to supply multiple assets and return utxos relevant to those assets only from server side (regardless of which address is associated with that request).

What I have in mind is to go directly from [asset] -> utxo. So as long as the returned utxos each contain all assets posted, then I think those endpoints would suffice. Thanks.

[rdlrt]

/asset_utxos added in #227 , where you can submit multiple assets as input query, and filter output to only return address if needed, also has _extended flag which could be set to true to list out asset_list or any scripts associated

[fallen-icarus]

I would like this issue to be re-opened because the new /asset_utxos endpoint does not actually satisfy what I am looking for.

For example, "show me all UTxOs that have asset X AND asset Y". It is okay if there are other assets in the UTxO but it would need to have at least the supplied assets.

I only want UTxOs returned if they have both asset X and asset Y. The current endpoint has two issues that make it unsuitable for my use case:

1. It returns UTxOs that have at least one of the assets specified. In other words, it is matching asset X OR asset Y.
2. The returned asset list only seems to contain the asset that was matched, even if the UTxO contains more than just that asset. This may be a bug?

[rdlrt]

I only want UTxOs returned if they have both asset X and asset Y. The current endpoint has two issues that make it unsuitable for my use case:

1. It returns UTxOs that have at least one of the assets specified. In other words, it is matching asset X OR asset Y.

Note that utxos endpoint by itself will list entire details, the server-side filtering can be done within URL itself, which is not specific to this endpoint, but valid across all Koios gRest API endpoints - and gives wide array of options for you to filter on. Your use case sounds like the below:

curl -sX POST -H "content-type: application/json" 'http://127.0.0.1:18050/rpc/asset_utxos?asset_list=cs.[{"fingerprint":"asset1fzp34lzx6zm59fsr94jh2ev98ll6xnsdheldzt"},{"fingerprint":"asset1qcezj4ju2c0an65t8ql4f92dfhp9md3jykfssp"}]' -d '{"_asset_list":[ ["46f987f7ed1886ba771b077c3ed5bbf3df158f54e0e3fa88d3d1e46e","744345525241"], ["ac54f96ff8a6952a0d803a435839ab64c99f4037a5a1ee9ba9349bcd","74575254"] ], "_extended": true }' -g

As regards your second point, thanks for flagging it was indeed the case - should be fixed now across networks

curl -sX POST -H "content-type: application/json" 'http://127.0.0.1:18050/rpc/asset_utxos?asset_list=cs.[{"fingerprint":"asset1fzp34lzx6zm59fsr94jh2ev98ll6xnsdheldzt"},{"fingerprint":"asset1qcezj4ju2c0an65t8ql4f92dfhp9md3jykfssp"}]' -d '{"_asset_list":[ ["46f987f7ed1886ba771b077c3ed5bbf3df158f54e0e3fa88d3d1e46e","744345525241"], ["ac54f96ff8a6952a0d803a435839ab64c99f4037a5a1ee9ba9349bcd","74575254"] ], "_extended": true }' -g | jq -r '.[].asset_list[].fingerprint' | sort | uniq -c | sort -n

Let us know if I am missing something.

[fallen-icarus]

Awesome. Thanks for showing the example. I had to move the -g flag to the beginning of the curl command but otherwise everything is working now. I appreciate the quick response.