PAD-FIXED-LENGTH-STRING is wrong for non-8-bit string encodings #2
Comments
heegaiximephoomeeghahyaiseekh
pushed a commit
that referenced
this issue
Dec 1, 2016
MAKE-FIXED-LENGTH-STRING, which encodes as well as pads the string. It may still have problems in encodings with variable-length characters, in which it's possible to choose a string and padding character which can't add up to the required length. MAKE-FIXED-LENGTH-STRING also provides the option to truncate overlong input strings, but this functionality might also fail in variable-length character encodings.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
lisp-binary::pad-fixed-length-stringadds its padding before encoding. This works fine for encodings such as ASCII and Latin-1, but will fail for encodings such as UTF-8, in which the byte length may differ from the character length.Padding the string after encoding might not work either, since the padding bytes would have to respect, (and therefore the writer generator must be aware of), the rules of each encoding.
One way to fix it could be to use a binary-search algorithm to find the right number of padding characters to add to the pre-encoded string to get the desired post-encoding length. In the worst case scenario, this could require re-encoding the same string dozens of times.
The text was updated successfully, but these errors were encountered: