Tweak CABI string conversion behavior #366
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As surfaced in jco/#443, the current string conversion logic in
store_string_to_utf8doesn't quite match what an implementation naturally wants to do. In particular, while the CABI does specify starting with an optimistic allocation that assumes all code points fit into a single byte, the fallback (that reallocs to the worst-case size and does the general transcode) doesn't happen as soon as the first non-fitting code point is discovered; it happens later once the initial allocation is full. This PR tweaks the logic to instead do the realloc immediately when the non-fitting code point is discovered.Technically, this could be considered a breaking change (since it observably changes when exactly the realloc happens), but it would be surprising if any code depended upon this behavior (at this early stage, without massive usage). Thus, I think we can just make the change.
CC @alexcrichton @guybedford @dbaeumer