How to get the char end and char start in the generated question and answers from valhalla/t5-base-qg-hl?

[monk1337]

Hi, The output of model valhalla/t5-base-qg-hl is {question and answer} is it possible to get char end and char start?

[lightshifted]

Yes, but in a roundabout way:

import pipelines

# load text
inputs = text_dataset

# instantiate QGPipeline
qgp = pipelines.QGPipeline(model, ans_model, tokenizer, ans_tokenizer)

# extract sentence and answers
sentence, extracted_answers = qgp._prepare_inputs_for_ans_extraction(inputs)

# search an answer for location char_start
index = 0
answer = extracted_answers[index]

char_start = extracted_answer.find('<hl>') + 5

Similarly, you can use rrfind('') to search for char_end.