Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mentions not searching by partial strings #161

Open
KyleMaas opened this issue Feb 3, 2021 · 12 comments
Open

Mentions not searching by partial strings #161

KyleMaas opened this issue Feb 3, 2021 · 12 comments

Comments

@KyleMaas
Copy link

KyleMaas commented Feb 3, 2021

mentions operator appears to be built to check when a term is included. However, it acts more like an equal operator - I can't search based on part of a mention. It has to be the whole thing for me to get results.

@arj03
Copy link
Member

arj03 commented Feb 3, 2021

True, but for these prefix indexes it should be possible to do something akin to a LIKE 'XYS%' where XYS can then be the first 32 bits of the search term.

@staltz
Copy link
Member

staltz commented Feb 3, 2021

Could you give an example of input, output, and expected outcome?

@KyleMaas
Copy link
Author

KyleMaas commented Feb 3, 2021

On the console, this: SSB.db.query(SSB.db.operators.and(SSB.db.operators.mentions(SSB.net.id)), SSB.db.operators.toCallback((err, msgs) => { console.log("err: ", err); console.log("msgs: ", msgs); }))

Yields this (not expanding the arrays, for brevity's sake):

err:  null
msgs:  (4) [{…}, {…}, {…}, {…}]

If I instead do this: SSB.db.query(SSB.db.operators.and(SSB.db.operators.mentions(SSB.net.id.substring(0, 4))), SSB.db.operators.toCallback((err, msgs) => { console.log("err: ", err); console.log("msgs: ", msgs); }))

I get this:

err:  null
msgs:  []

If I cut off the first byte of my ID: SSB.db.query(SSB.db.operators.and(SSB.db.operators.mentions(SSB.net.id.substring(1, 5))), SSB.db.operators.toCallback((err, msgs) => { console.log("err: ", err); console.log("msgs: ", msgs); }))

I get this:

err:  null
msgs:  []

@staltz
Copy link
Member

staltz commented Feb 3, 2021

Ok, I see. But why would you need to do id.substring(1,5)? What's the purpose?

@KyleMaas
Copy link
Author

KyleMaas commented Feb 3, 2021

What I would ideally love to be able to do is search for a mention with a search term "ima" and get blobs with names "image.png" back, not just IDs. I'm not sure this is possible now, but while I was hunting through the code trying to figure out if that kind of thing was supported, I ran into this issue.

@KyleMaas
Copy link
Author

KyleMaas commented Feb 3, 2021

@staltz To limit it to a 32-bit prefix search, as @arj03 mentioned, to try to cover both test cases.

@KyleMaas
Copy link
Author

KyleMaas commented Feb 3, 2021

@staltz Oh, you mean why would you cut off the first byte? Just in case one it didn't like the initial "@" and was searching only on the first 32 bits of the ID itself.

@staltz
Copy link
Member

staltz commented Feb 3, 2021

What I would ideally love to be able to do is search for a mention with a search term "ima" and get blobs with names "image.png" back, not just IDs. I'm not sure this is possible now, but while I was hunting through the code trying to figure out if that kind of thing was supported, I ran into this issue.

Ok, I sort of see the purpose now, but I think these prefix indexes are not suited for full-text search, or search of any type. If you want to find all msgs that have msg.value.content.mentions[].name === 'image.png', that's possible with another prefix index for .name and you would pass in "image.png" as the input, not "image.png".substring(0,3). I still don't understand why would you want to pass a substring as the input.

@staltz
Copy link
Member

staltz commented Feb 3, 2021

Perhaps the term "INCLUDES" hinted towards some full-text search capabilities. That's maybe a misnomer. INCLUDES is merely an "EQUAL" over the items in an array, similar to JS Array.prototype.some.

@KyleMaas
Copy link
Author

KyleMaas commented Feb 3, 2021

Ah, gotcha. So it is still a "full match" type search and does not support partial strings, and that is intentional. Good to know.

This issue was about not being able to search for partial IDs. I should probably file another one for the "what I was hoping it would do when I ran into this". But the feature I was looking for would have been for the Markdown editors we're use in ssb-browser-demo, which allow users to type "@" and have a list of user suggestions pop up which get winnowed down as you type. I was hoping to do the same thing with "&", per @arj03's feature request arj03/ssb-browser-demo#124 and I got it working in arj03/ssb-browser-demo#162 using a port of ssb-meme to db2 here: https://github.com/ssbc/ssb-meme

But it's...kludgy. I ended up having to pull the most recent posts and run them through ssb-meme's mention search. It would have been much nicer if I could do a prefix search for mention names. I'll go ahead and file that as another issue, though, to keep from cluttering up this one, since this one was about partial ID searches.

Whether "full match search" is the expected behavior or not, it would be awfully nice if that was documented. It tripped me up, and even @arj03 mentioned above that that should be possible.

@staltz
Copy link
Member

staltz commented Feb 3, 2021

Nice, this gives me much better context on what you were trying to do, and I agree we would should find a way to make db2 support allow users to type "@" and have a list of user suggestions pop up which get winnowed down as you type and I believe prefix indexes will be limited for that.

@KyleMaas
Copy link
Author

KyleMaas commented Feb 3, 2021

Sounds good. We're using ssb-suggest right now for the "@" mentions. It's not perfect, and apparently the license of it is incompatible with a lot of other stuff (including ssb-browser-demo), but it mostly works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants