More stopword issues...

[mikesname]

This multilingual stop-word list might not be working out. For example "hans" is a stop word, but also a common German name. What to do?

[juntezhang]

Hi Mike,

I recommend not removing any stopwords.

It only makes sense if storage is an issue, but for your project that's not the case. For effective retrieval, it won't mean much (if any) improvement.

Cheers, junte

On 8 jan. 2015, at 16:32, Mike Bryant [email protected] wrote:

This multilingual stop-word list might not be working out. For example "hans" is a stop word, but also a common German name. What to do?

—
Reply to this email directly or view it on GitHub.

[KepaJRodriguez]

I'm not sure about how can we handle with this problem. When I compiled the list I removed some words that I identified, but I didn't realized that "hans" was a part of the list.
I don't think that there is an alternative way than removing some of the stopwords by hand (lists of stopwords are anyway hand crafted).