Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure JSON encoding supports non-ASCII characters in uploads #19120

Closed
wants to merge 3 commits into from

Conversation

arash77
Copy link
Collaborator

@arash77 arash77 commented Nov 8, 2024

Update JSON encoding in upload requests to handle non-ASCII characters correctly.
Fixes step 2 of #18584

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

@arash77
Copy link
Collaborator Author

arash77 commented Nov 11, 2024

@galaxyproject/wg-backend Should this be applied to all json.dump(s)?

@mvdbeek
Copy link
Member

mvdbeek commented Nov 18, 2024

Can you add a test please ?
In what circumstances does the load not restore the unicode string ?

In [1]: import json

In [2]: json.loads(json.dumps("💩", ensure_ascii=True))
Out[2]: '💩'

@arash77
Copy link
Collaborator Author

arash77 commented Nov 18, 2024

Can you add a test please ? In what circumstances does the load not restore the unicode string ?

In [1]: import json

In [2]: json.loads(json.dumps("💩", ensure_ascii=True))
Out[2]: '💩'

The problem here is mostly visual, so for example when we export the history we have datasets_attrs.txt or history_attrs.txt file that is not the loaded json and will not include the correct characters!
This also happens in request_json in tool parameters.

@mvdbeek
Copy link
Member

mvdbeek commented Nov 18, 2024

I don't think we should deviate from stdlib defaults for this. If python decides they're done with that practice then we'll get the new defaults, but I don't think cosmetic changes to files that are not meant for human consumption warrants this.

@arash77
Copy link
Collaborator Author

arash77 commented Nov 18, 2024

Ok.
Even the name variable in here?
image

@mvdbeek
Copy link
Member

mvdbeek commented Nov 18, 2024

I mean, should users see this at all ? Probably not ? And if so it should probably be rendered as json, not as a string ?

@arash77
Copy link
Collaborator Author

arash77 commented Nov 18, 2024

If this is the case, then I close the PR.

@arash77 arash77 closed this Nov 18, 2024
@arash77 arash77 deleted the fix-request-json branch November 19, 2024 09:27
@arash77 arash77 mentioned this pull request Nov 19, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants