How to evaluate by Average Normalized Levenshtein Similarity (ANLS)?

[kobrafarshidi]

Hi Mr. @uakarsh
As you know I am working your source code and I am trying to evaluate by ANLS like write in article.
and I want to do that with pytorch or pytorch_lightning .
I see some guide docs, but I encounter some question. I would be grateful if you guide me.

1. I write an algorithm in below if I'm wrong could you correct my wrong?

2. If this algorithm is true, could you help me, how can I replace N, M, aij, oqi with your parameters in your source code?

for example, what should I put instead of aij or qoi ?

N is the total number of questions
M is the total number of GT answers per question
aij is the ground truth answer
oqi is the network’s answer for the i'th question qi

from similarity.normalized_levenshtein import NormalizedLevenshtein
def calculate_ANLS_score(aij,oqi ):
    x=[]
    NL = NormalizedLevenshtein()
    for i in range(N):
         x+= (1- NL(aij, oqi ))
    x/N
       
    

[uakarsh]

Hi @kobrafarshidi, much sorry for late reply. Can you refer to this thread, and let me know if you still have doubt regarding the same.