AI semantic search through the whole full-text history

[lrq3000]

My suggestion is to implement AI semantic search, possibly by taking inspiration from the codebase of the incredible SemanticFinder extension.

The use case is extremely common and I think no one ever implemented such an incredibly useful solution to it: you know you read something somewhere but you don't remember where. Currently, with falcon, it's already possible to find the URL if you can remember one exact keyword. But let's say you don't, you can only paraphrase. With semantic search, the paraphrase would be enough to find the URL easily. And any webpage with a similar "idea", not just keyword.

Now the tricky question would be when to calculate the embeddings, because this is costly, especially on such a big database of content. It can either be on-query (and the user has to wait), or periodically in the background (instant query, but regular slowdowns when the embeddings are recalculated).

Hopefully, the embeddings can be calculated iteratively too, which would drastically reduce the computational cost.