Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wmt17-transformer example: LC_ALL=C.UTF-8 commands error #2

Closed
MaksymDel opened this issue Mar 29, 2018 · 1 comment
Closed

wmt17-transformer example: LC_ALL=C.UTF-8 commands error #2

MaksymDel opened this issue Mar 29, 2018 · 1 comment

Comments

@MaksymDel
Copy link

MaksymDel commented Mar 29, 2018

LC_ALL=C.UTF-8 commands:

LC_ALL=C.UTF-8 ../tools/sacreBLEU/sacrebleu.py -t wmt16 -l en-de --echo src > data/valid.en
LC_ALL=C.UTF-8 ../tools/sacreBLEU/sacrebleu.py -t wmt16 -l en-de --echo ref > data/valid.de
LC_ALL=C.UTF-8 ../tools/sacreBLEU/sacrebleu.py -t wmt14 -l en-de --echo src > data/test2014.en
LC_ALL=C.UTF-8 ../tools/sacreBLEU/sacrebleu.py -t wmt15 -l en-de --echo src > data/test2015.en
LC_ALL=C.UTF-8 ../tools/sacreBLEU/sacrebleu.py -t wmt16 -l en-de --echo src > data/test2016.en
LC_ALL=C.UTF-8 ../tools/sacreBLEU/sacrebleu.py -t wmt17 -l en-de --echo src > data/test2017.en

Error:

Traceback (most recent call last):
  File "../tools/sacreBLEU/sacrebleu.py", line 1434, in <module>
    main()
  File "../tools/sacreBLEU/sacrebleu.py", line 1368, in main
    print_test_set(args.test_set, args.langpair, args.echo)
  File "../tools/sacreBLEU/sacrebleu.py", line 989, in print_test_set
    print('\t'.join(map(lambda x: x.rstrip(), lines)))
"UnicodeEncodeError: 'ascii' codec can't encode character '\u2013' in position 76: ordinal not in range(128)"

Is it Linux/python dependent problem? Without these locale lines sacrebleu seems to do the job.

@MaksymDel
Copy link
Author

The issue is simply due to the missing C.UTF-8 locale file. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant