Skip to content

Commit

Permalink
GreynirPackage -> GreynirEngine
Browse files Browse the repository at this point in the history
  • Loading branch information
sveinbjornt committed Aug 23, 2024
1 parent 33438e0 commit 975bb4f
Show file tree
Hide file tree
Showing 4 changed files with 7 additions and 8 deletions.
5 changes: 2 additions & 3 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,7 @@ Tokenizer is an independent spinoff from the `Greynir project <https://greynir.i
The `Greynir natural language parser for Icelandic <https://github.com/mideind/GreynirEngine>`_
uses Tokenizer on its input.

Note that Tokenizer is licensed under the *MIT* license
while GreynirEngine is licensed under *GPLv3*.
Tokenizer is licensed under the *MIT* license.


Deep vs. shallow tokenization
Expand Down Expand Up @@ -613,7 +612,7 @@ defined within the ``TOK`` class:
| S_END | 11002 | End of sentence | |
+---------------+---------+---------------------+---------------------------+

(*) The token types marked with an asterisk are reserved for the Greynir package
(*) The token types marked with an asterisk are reserved for the GreynirEngine package
and not currently returned by the tokenizer.

To obtain a descriptive text for a token kind, use
Expand Down
6 changes: 3 additions & 3 deletions src/tokenizer/Abbrev.conf
Original file line number Diff line number Diff line change
Expand Up @@ -621,7 +621,7 @@ m.a.o. = "meðal annarra orða" ao frasi
m.a.s. = "meira að segja" ao frasi
m.fl.* = "með fleiru" ao frasi
m.m.* = "með meiru" ao frasi
msk.* = "matskeið" kvk # Er í GreynirPackage/config/Phrases.conf sem msk
msk.* = "matskeið" kvk # Er í GreynirEngine/config/Phrases.conf sem msk
matsk.* = "matskeið" kvk
mtt. = "með tilliti til" fs # Leiðrétt í GreynirCorrect
m.t.t. = "með tilliti til" fs
Expand Down Expand Up @@ -686,7 +686,7 @@ pk.* = "pakki" kk
portúg.* = "portúgalska" kvk
portúg.* = "portúgalskur" lo
pr. = "per" hk
próf.^ = "prófessor" kk # Þessi merking er notuð í bintokenizer.py í GreynirPackage
próf.^ = "prófessor" kk # Þessi merking er notuð í bintokenizer.py í GreynirEngine
próf.^ = "prófastur" kk # en hefur ekki þýðingu í Tokenizer
pt.* = "punktastærð" kvk
pt.* = "punktar" kk
Expand Down Expand Up @@ -797,7 +797,7 @@ tilv.* = "tilvitnun" kvk
tilv.* = "tilvísun" kvk
to.* = "töluorð" hk
transl.* = "þýddur" lo erl
tsk.* = "teskeið" kvk # Er í GreynirPackage/config/Phrases.conf sem tsk
tsk.* = "teskeið" kvk # Er í GreynirEngine/config/Phrases.conf sem tsk
tyrkn.* = "tyrkneska" kvk
tyrkn.* = "tyrknesur" lo
tékkn.* = "tékkneska" kvk
Expand Down
2 changes: 1 addition & 1 deletion src/tokenizer/abbrev.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ def add(abbrev: str, meaning: str, gender: str, fl: Optional[str] = None) -> Non
# This logic is not fully present in Tokenizer as information
# about person names is needed to make it work. The full implementation,
# using the NAME_FINISHERS set, is found in bintokenizer.py in
# GreynirPackage.
# GreynirEngine.
name_finisher = True
abbrev = abbrev[0:-1]
if not abbrev.endswith("."):
Expand Down
2 changes: 1 addition & 1 deletion src/tokenizer/tokenizer.py
Original file line number Diff line number Diff line change
Expand Up @@ -3054,7 +3054,7 @@ def tokenize_without_annotation(
text_or_gen: Union[str, Iterable[str]], **options: Any
) -> Iterator[Tok]:
"""Tokenize without the last pass which can be done more thoroughly if BÍN
annotation is available, for instance in GreynirPackage."""
annotation is available, for instance in GreynirEngine."""
return tokenize(text_or_gen, with_annotation=False, **options)


Expand Down

0 comments on commit 975bb4f

Please sign in to comment.