We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I make an LM using this tutorial, but I could not parse the resulting arpa file with other tools because of inconsistent/nonstandard whitespace.
arpa
First, in the \data\ section, there were tabs before the frequency count, indicated by <TAB> below:
\data\
<TAB>
\data\ ngram 1=<TAB>7 ngram 2=<TAB>7
Second, the automatically generated <unk> token had a space instead of a tab, although every other token was separated from its frequency by a tab.
<unk>
Third, the \2-grams: and \3-grams: lines were not preceded by an empty line.
\2-grams:
\3-grams:
The text was updated successfully, but these errors were encountered:
No branches or pull requests
I make an LM using this tutorial, but I could not parse the resulting
arpafile with other tools because of inconsistent/nonstandard whitespace.First, in the
\data\section, there were tabs before the frequency count, indicated by<TAB>below:Second, the automatically generated
<unk>token had a space instead of a tab, although every other token was separated from its frequency by a tab.Third, the
\2-grams:and\3-grams:lines were not preceded by an empty line.The text was updated successfully, but these errors were encountered: