Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable retrieval of binary buffers #2274

Closed
guidiaz opened this issue Sep 19, 2022 · 2 comments
Closed

Enable retrieval of binary buffers #2274

guidiaz opened this issue Sep 19, 2022 · 2 comments
Labels
breaking change ⚠️ Introduces breaking changes enhancement 📈 New feature or request

Comments

@guidiaz
Copy link
Contributor

guidiaz commented Sep 19, 2022

  • In http/s sources, binary contents are specified by setting headers defined within MIME-type spec (RFC-2045). While data is serialized to text format during transmission, at client's app level is received as binary content.
  • By enabling binary data at the Witnet protocol level, either hash() or toString() reducers would be available within data requests.
@guidiaz guidiaz changed the title Add support to retrieve binary buffers Enable retrieval of binary buffers Sep 19, 2022
@tmpolaczyk
Copy link
Contributor

The conversion from bytes to string is implemented here:

witnet-rust/rad/src/lib.rs

Lines 242 to 248 in 63920bd

let response_string = response
// TODO: replace with .body_bytes() and let RADON handle the encoding?
.body_string()
.await
.map_err(|x| RadError::HttpOther {
message: x.to_string(),
})?;

We are using this method from the surf library, which only supports UTF8: https://docs.rs/surf/2.3.2/surf/struct.Response.html#method.body_string

There is a body_bytes method which returns the raw bytes: https://docs.rs/surf/2.3.2/surf/struct.Response.html#method.body_bytes

So we could add a check like:

  • If the first operator of the script expects bytes as the input, use body_bytes
  • Otherwise, use body_string

The only problem is that we cannot access the request headers from RADON to read the content type, so converting the bytes into a string later will only be possible if we know the encoding ahead of time when creating the script. Similarly, when converting string to bytes we need to specify an encoding somehow. But encodings other than UTF8 are not supported yet, so that can be another issue.

@aesedepece aesedepece added breaking change ⚠️ Introduces breaking changes enhancement 📈 New feature or request labels Oct 3, 2022
@Tommytrg
Copy link
Member

Superseeded by #2331

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change ⚠️ Introduces breaking changes enhancement 📈 New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants