[Feature Request] Add method to resolve keys more thoroughly

[SeverinAlexB]

Currently, pkarr returns the first successful response when a packet is uncached. This can be problematic because this first response can always be outdated, especially if resolvers are used.

It would be great to have a method that makes more thorough searches. For example:

let settings = ResolutionSettings::default()
    .min_response_count(4) // Waits for at least 4 responses before returning the result.
    .build();
let result = client.resolve_with_settings(&pubkey, settings);
...

min_response_count(count) is one idea how to do it. It could be done in a different way. But I think you get the gist.

This could also be combined with a simple caching api ,as suggested before:

let settings = ResolutionSettings::default()
    .fetch_strategy(Strategy::CacheFirst) // Call the cache first and only if empty, makes network calls.
    .build();
let result = client.resolve_extended(&pubkey, settings);
...

See the Apollo fetch policies as an example. It would improve the API a lot especially for new devs. Up to you.

[Nuhvi]

That is too much API in my opinion.

I can convert resolve_inner to resolve_rx or something and make it public, and you can do with the Receiver as you wish.

[SeverinAlexB]

It's exactly the API we need. Other APIs are way more complicated than this. Happy to provide a PR too if you wish.

Not sure I understand the resolve_tx approach so I can't judge if this is a good alternative. as far as I see, it only provides one SignedPacket? Am I wrong?

[Nuhvi]

No it would return a `flume::Receiver<SignedPacket>` which you can use as a sync or async iterator.

…

On Mon, 16 Dec 2024, 14:53 Severin Alexander Bühler, < ***@***.***> wrote: It's exactly the API we need. Other APIs are way more complicated than this. Happy to provide a PR too if you wish. Not sure I understand the resolve_tx approach so I can't judge if this is a good alternative. as far as I see, it only provides one SignedPacket? Am I wrong? — Reply to this email directly, view it on GitHub <#104 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJRH3DHNWYYUTC7JSRZL2MD2F25L7AVCNFSM6AAAAABTV2HTQGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBVGQYTSMZZGQ> . You are receiving this because you commented.Message ID: ***@***.***>

[SeverinAlexB]

Could be a start then

[Nuhvi]

@SeverinAlexB try using resolve_rx c6613ac

[SeverinAlexB]

I don't see how resolve_rx can be useful. When trying this test, it only returns 2 packets. I don't know why there are only 2 packets or which they are exactly.

    #[test]
    fn resolve_rx() {
        let client = Client::builder()
        .build().unwrap();

        let pubkey: PublicKey = "7fmjpcuuzf54hw18bsgi3zihzyh4awseeuq5tmojefaezjbd64cy".try_into().unwrap();

        let rx = client.resolve_rx(&pubkey).unwrap();
        for (i, packet) in rx.iter().enumerate() {
            println!("{i}. Packet {}", packet.timestamp())
        };
        println!("Done")
    }

Output:

0. Packet Timestamp (1734946561347670)
1. Packet Timestamp (1734946561347670)
Done

[Nuhvi]

The first one is from cache (if there are any), the second and every subsequent is from the DHT as they arrive (skipping packets that aren't more recent than the ones seen so far).

So if there was anything on the network that is more recent than the cache resolve_rx would have returned it.

How you use this is up to you, you might want to wait until all incoming packets are exhausted, or wait until you see a packet more recent than a given timestamp you already know about, or wait till you find a specific record ... or whatever, up to you, if you don't like the normal behavior of; returning a packet as soon as possible.

I think this satisfies the thoroughness requirement of this feature request, no?

[SeverinAlexB]

I see what you mean. It's still rather restrictive. Is there a way to get the full packet stream? Maybe something like this:

// Each layer has different freshness properties.
enum SourceType {
    DHT, // Theoretically most up to date but can still serve outdated data.
    Resolver,  // Uses TTL caching. May serve outdated data within TTL but is very fast otherwise. Great for slow changing entries. Not so good if entries just changed.
    Cache,  // TTL caching too but probably with less entries than the big Resolver cache.
}


for (source_type, packet) in client.resolve_rx(&pubkey)  {
    println!("{source_type} - {}", packet.timestamp());
}
println!("done")

This way, I would have full control:

• I can decided to skip the cache if needed.
• I can decided to include or skip the resolver because the resolver is fast but still works with a ttl cache.
• I can decided to wait for x out of all DHT responses to find a balance between latency and recency.

I like the freshness hierarchy of the SourceTypes. But a developer should decided what freshness his data should have.

Does this makes sense?

At the end, it comes back to my origin request because in this proposal, the dev can't decided if pkarr should hit the DHT/resolver if the received cache is already sufficient.

[Nuhvi]

I think you should use mainline directly. Pkarr Client is targeting best practices and good defaults. If we keep adding more control, you will end up bypassing the Client entirely and only using SignedPacket and mainline::Dht, but you can already do that now, the very Client is optional feature in Pkarr.

[Nuhvi]

Note that in resolve_rx you will typically see every incoming packet within ~300 ms, so even calling resolve() twice with a 300ms sleep would be sufficient. So that is the maximum upside of complicating this API further, and only when you are extremely allergic to cache, which you can disable.

[SeverinAlexB]

Let me do the following (whenever I got time): I'll try to add my preferred API on top of Mainline. I'll likely do this by forking pkarr because pkarr already provides all the caching methods and so on. If you ever decided to also add this extended API to pkarr, you can just copy/merge my fork.

A lib like pkarr should provide methods for beginners with good default values and easy api, and advanced methods. Forcing users to use another lib when they mature in their pkarr skills and switch the library is a big ask. You are the pkarr boss though so it's up to you to decide what is best for this lib.

[Nuhvi]

Sounds good.