Getting Warnings when using custom KB

[farrandi]

I made my own KB as per #331 and it works perfectly. The only problem is I keep getting these warning messages.

Here is my code:

from scispacy.candidate_generation import DEFAULT_PATHS, DEFAULT_KNOWLEDGE_BASES
from scispacy.candidate_generation import (
    CandidateGenerator,
    LinkerPaths
)
from scispacy.linking_utils import KnowledgeBase
from scispacy.linking import *

CustomLinkerPaths_mycustom = LinkerPaths(
    ann_index="../Metas/knowledge base/kb/nmslib_index.bin",
    tfidf_vectorizer="../Metas/knowledge base/kb/tfidf_vectorizer.joblib",
    tfidf_vectors="../Metas/knowledge base/kb/tfidf_vectors_sparse.npz",
    concept_aliases_list="../Metas/knowledge base/kb/concept_aliases.json",
)
class myKnowledgeBase(KnowledgeBase):
    def __init__(
        self,
        file_path: str = "../Metas/knowledge base/custom_kb.jsonl",
    ):
        super().__init__(file_path)

DEFAULT_PATHS["myCustom"] = CustomLinkerPaths_mycustom
DEFAULT_KNOWLEDGE_BASES["myCustom"] = myKnowledgeBase

nlp.add_pipe("scispacy_linker", config={"resolve_abbreviations": True, "linker_name": "myCustom",
                                        "filter_for_definitions": False, "threshold": "0.5"})
linker = CandidateGenerator(name="myCustom")

and I got:

Your CPU supports instructions that this binary was not compiled to use: SSE3 SSE4.1 SSE4.2 AVX AVX2
For maximum performance, you can install NMSLIB from sources 
pip install --no-binary :all: nmslib

also when I made my own custom function and tested it, I got another warning. Here is the code:

def get_id(word):
    temp_doc = nlp(word)
    try:
        ents = temp_doc.ents[0]
        possible_ents = [ent for ent in ents._.kb_ents]
        most_likely_ent = possible_ents[0]
        return most_likely_ent[0]
    except:
        return ""
        

x = get_id("LV")
print(x)

This is the warning for the code above:

home/user/.local/lib/python3.8/site-packages/scispacy/candidate_generation.py:284: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  extended_neighbors[empty_vectors_boolean_flags] = numpy.array(neighbors)[:-1]
/home/user/.local/lib/python3.8/site-packages/scispacy/candidate_generation.py:285: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  extended_distances[empty_vectors_boolean_flags] = numpy.array(distances)[:-1]

Do you know why this is the case? is there a way to fix this or remove the warning?

[easyas314]

Just installed scispacy 0.4.0 on python 3.10 and I'm receiving the same warning while using the included linker.
nlp.add_pipe("scispacy_linker", config={"linker_name": "rxnorm"})

... so it's not just a custom KB issue.

[dakinggg]

The first warning is just about the installation of nmslib and I believe is fine to ignote, but you can try to install nmslib as it says if you want. The second warning should be fixed, but I don't know if/when it will be, and I don't think it should cause any issues other than printing the warning.