Fetch all replies

[sneakers-the-rat]

Pitch

When expanding a post, the instance should fetch all replies from the host server.

This issue is to move more general conversation out of #8 because i think that's the wrong approach

Previous context:

• [feature] fetch all replies #8 - I'm going to close this PR because I think it should be default behavior rather than behind a button in the web interface, and it should be done by the server rather than the client calling the search endpoint.
• https://wiki.neuromatch.social/Fetch_All_Replies
• https://neuromatch.social/tags/FetchAllReplies

Motivation

Two reasons:

• It's an important discovery mechanism - people should be able to see the conversation around a post (within normal privacy settings, ie. we should not be trying to get followers-only posts, etc.)
• The "a thousand of the same replies" problem is notorious on fedi and part of what makes it somewhat exhausting, and can quickly feel like brigading if a post becomes even moderately popular.

Approach

• When a post is expanded, a call is made to the context endpoint
• If that status is on a different server AND the request is coming from a logged in user on our instance, make a call to the FetchRepliesService before yielding from the db
• Remove limitation on URIs matching the host server and the limit of 5 replies in the filtered_replies method and fetch_collection.
• Instead, to mitigate amplification/DoS, replace with a single numerical limit that first filters out URLs that the instance already has (to avoid duplicated requests). This should be tied to the pagination of the context endpoint - first fetch 40 posts, then as one scrolls the server should fetch the next 40, and so on.

Concerns

Privacy has been discussed elsewhere - we will only be getting posts that wouldn't be filtered out by normal post visibility settings. ie. the user would be able to get them on their own by just running a bunch of manual searches.

• Perf & API Consistency: Having a potentially long-running service call in the context endpoint is undesirable. We should run the service as async. This will mean that later calls will yield different results (ie. as the posts are imported by the async worker). That's really only a problem for programmatic API usage, and just requires a note on the endpoint documentation. In normal web UI usage, it should look like the posts loading into the interface as they are received. The context endpoint would behave as expected on the first call, and just have extra replies in future calls. We could add an additional option that defaults false to make the reply fetching service synchronous.