-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make [Try]IntoCtx borrow instead of copying #47
Conversation
This assumes that #[repr(packed)] types are also Copy, which is a thing we can't verify in this context. At least we're in good company: the built-in #[derive]s all require #[derive(Copy)] for packed structs. Further discussion: m4b#46 (comment)
I'm working up a big PR for |
I opened PRs in both |
@@ -163,7 +163,7 @@ fn write_io() -> Result<(), scroll::Error> { | |||
let mut bytes = [0x0u8; 10]; | |||
let mut cursor = Cursor::new(&mut bytes[..]); | |||
cursor.write_all(b"hello")?; | |||
cursor.iowrite_with(0xdeadbeef as u32, BE)?; | |||
cursor.iowrite_with(&(0xdeadbeef as u32), BE)?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This and the 3 others are a huge PR, will take me a bit to review, appreciate your patience, @willglynn.
I would like to note that having to take the reference now of variables, literals, makes me pretty sad...
I feel like it jars slightly with the original and basic purpose of scroll, which was to "serialize" numbers and PODs.
I do wish there had been a little more discussion before this enormous (and amazing PR!), specifically, whether we can have our cake and eat it too w.r.t. TryInto for the primitives.
Specifically, I'm wondering if there's a compromise, so that we can still pass either references or owned values to the pwrite/iowrite.
Maybe it's just me but i do find it a usability paper cut and find it less attractive that I have to take references to pwrite now, but perhaps I'm used to the old way.
So, is there some generic magic we can use to do this? Guessing wildy and randomly:
- Have someone take AsRef
- Have scroll_derive derive the reference versions for any user type (i'm ok with taking references of user structs)
- ???
On the other hand, in my case where the struct was too large to be put on the stack, and I had to box it, even if we did say 2., this would still probably eventually blow the stack? Or perhaps not. Have to think about it, and its late and a Thursday :)
Similarly, and against my original sadness, if I'm able to pwrite out a huge boxed struct that auto-derives pwrite, that would actually make me happy and would counteract any bitterness of no longer being able to pwrite raw deadbeefs :D
Dunno, just some thoughts, feel free to respond whenever you get the time (and that goes for anyone else who's watch).
Regardless, thanks for your amazing work here @willglynn and the robustness and tenacity of the changes, great work!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cheers! And yeah, this is why I went and did these PRs even with limited discussion: I didn't know how this concept would look until it was done 😄
I'll ponder the generics question. There might be a way we can make pwrite()
be &
-agnostic while keeping TryIntoCtx
take &self
– that'd let you pwrite()
either by value or by reference without consuming self
during the write.
If we say: /// Write `n` of type `T` at offset `I` with context `Ctx`
/// # Example
/// ```
/// use scroll::{Pwrite, Pread, LE};
/// let mut bytes: [u8; 8] = [0, 0, 0, 0, 0, 0, 0, 0];
/// bytes.pwrite_with(0xbeefbeef, 0, LE).unwrap();
/// assert_eq!(bytes.pread_with::<u32>(0, LE).unwrap(), 0xbeefbeef);
fn pwrite_with<T,V>(&mut self, n: V, offset: usize, ctx: Ctx) -> result::Result<usize, E>
where T: TryIntoCtx<Ctx, <Self as Index<RangeFrom<usize>>>::Output, Error = E> + ?Sized,
V: Borrow<T>,
{
let len = self.measure_with(&ctx);
if offset >= len {
return Err(error::Error::BadOffset(offset).into())
}
let dst = &mut self[offset..];
n.borrow().try_into_ctx(dst, ctx)
} These work:
This does not work, because now there's two type parameters:
These also work:
|
Using |
I'm pretty excited about this version @willglynn ! I checked out your branch this weekend and played around with it, if nothing else comes up, I think this might be the way forward :) So we can now do stuff like this: #[test]
fn pwrite_big_struct() {
use scroll::{LE, Pwrite};
#[derive(Pread, Pwrite, Copy, Clone)]
struct Big {
arr: [u8; 100000000],
}
let big: Box<Big> = box (unsafe { ::std::mem::uninitialized() });//Big { arr: [0; 100000000] };
let mut bytes = vec![0; 100000001];
//let big = box bytes.pread_with::<Big>(0, LE).unwrap();
let _ = bytes.pwrite_with(&* big, 0, LE).unwrap();
assert!(false);
} which was my original pain point when trying to write a large struct which was boxed :D (Preading is still an issue, and boxing doesn't fix it atm, but I think we could in principle fix that too, at another time) Just grepping through a couple of codebases, I believe @philipc is correct (as usual) and afaics only goblin atm uses
So yea, overall really like this direction, seems like with a minor sacrifice we can keep the owned usage for literals (and other Copyables?). If we do end up going this route, I think we'll need some explicit documentation about why passing a type parameter via turbofish is banned/problematic, etc. Anyway, great work ! |
Cool! I plan to:
I'll comment when that's done and I've force-pushed the results. |
Friendly ping @willglynn . Would definitely like to see this get pushed through? Do you have a branch WIP anywhere, or need to offload some work to someone else? Hoping we didn't lose steam on it as i think its definitely something that's needed :) |
@m4b 👍 I've been traveling and haven't looped back to this. My most current work is still in this branch, and it needs to be re-worked per the plan above. I expect to have more time right at year's end. |
No worries, thanks for the update! :) |
@willglynn gentle ping :) is there any chance of continuing here, or should we explore other options? If you're busy don't worry at all, or don't want to work on it anymore, don't feel bad at all/responsible whatever, just trying to get a feel for how to proceed/what to do. |
Sorry! I've been traveling some more and haven't got back to this. Probably the right thing to do is to reset to |
oops i thought this PR was in old scroll_derive repo, ok never mind :) |
I cargo fmt scroll and scroll derive; was hoping something like this would get resolved, but doesn't seem like it's going to happen, and now that fmt has happened there will be conflicts everywhere. going to close this; we can revisit again, or possibly never :) |
This PR changes
[Try]IntoCtx
to take&self
instead ofself
. It very likely breaks every downstream project that callscwrite()
orpwrite()
, but in a trivial way – callers need only specify&
.This closes #46 and also incidentally closes #42.