Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] [Help wanted] Adapt to upcoming changes in Abomonation #11

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,4 @@ quote = "1.0"
synstructure = "0.12"

[dev-dependencies]
abomonation = "0.7"
abomonation = { git = "https://github.com/HadrienG2/abomonation.git", branch = "support-more-types" }
55 changes: 34 additions & 21 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,38 +14,51 @@ fn derive_abomonation(mut s: synstructure::Structure) -> proc_macro2::TokenStrea
});

let entomb = s.each(|bi| quote! {
::abomonation::Abomonation::entomb(#bi, _write)?;
::abomonation::Entomb::entomb(#bi, _write)?;
Copy link
Author

@HadrienG2 HadrienG2 Oct 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Abomonation API change from TimelyDataflow/abomonation#28: The Abomonation trait is now API sugar for two separate traits, Entomb and Exhume, via a blanket impl which ensures that everything which implements those two traits implements Abomonation. This was done to allow deserialization of types which contain references, which requires adding a lifetime parameter to the deserialization trait (as in serde).

The semantics of Exhume<'de>'s lifetime parameter is that if T: Exhume<'de>, it means that it is okay to deserialize an &'de T from &'de mut [u8] bytes. This is not always the case for references, for example you don't want to allow people to deserialize &'static Something from stack-allocated bytes.

});

let extent = s.each(|bi| quote! {
sum += ::abomonation::Abomonation::extent(#bi);
sum += ::abomonation::Entomb::extent(#bi);
});

s.bind_with(|_| synstructure::BindStyle::RefMut);

let exhume = s.each(|bi| quote! {
let temp = bytes;
bytes = ::abomonation::Abomonation::exhume(#bi, temp)?;
bytes = ::abomonation::Exhume::exhume(From::from(#bi), bytes)?;
Copy link
Author

@HadrienG2 HadrienG2 Oct 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Abomonation API change from TimelyDataflow/abomonation#22: exhume() now expects NonNull<Self> instead of &mut self. That is because exhume() receives an invalid Self as input (it has dangling pointers/references), and creating an &mut to invalid data is UB. Raw pointers should be used whenever possible in those situations.

});

s.bound_impl(quote!(abomonation::Abomonation), quote! {
#[inline] unsafe fn entomb<W: ::std::io::Write>(&self, _write: &mut W) -> ::std::io::Result<()> {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the inline directives since I established in TimelyDataflow/abomonation#24 that they are actually very rarely needed. It's probably better to let the compiler's optimizer do its job if it does it reasonably well.

match *self { #entomb }
Ok(())
}
#[allow(unused_mut)]
#[inline] fn extent(&self) -> usize {
let mut sum = 0;
match *self { #extent }
sum
s.gen_impl(quote! {
extern crate abomonation;
extern crate std;

gen unsafe impl abomonation::Entomb for @Self {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Abomonation API change from TimelyDataflow/abomonation#25: implementing Abomonation traits is now unsafe, since an incorrect implementation (e.g. forgetting to patch a pointer) can lead to memory unsafety.

unsafe fn entomb<W: ::std::io::Write>(&self, _write: &mut W) -> ::std::io::Result<()> {
match *self { #entomb }
Ok(())
}

#[allow(unused_mut)]
fn extent(&self) -> usize {
let mut sum = 0;
match *self { #extent }
sum
}
}
#[allow(unused_mut)]
#[inline] unsafe fn exhume<'a,'b>(
&'a mut self,
mut bytes: &'b mut [u8]
) -> Option<&'b mut [u8]> {
match *self { #exhume }
Some(bytes)

gen unsafe impl<'de> abomonation::Exhume<'de> for @Self
where Self: 'de,
Copy link
Author

@HadrienG2 HadrienG2 Oct 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

synstructure help wanted here to fix a memory safety bug: assuming that Self is Generic<'a, 'b, 'c>, I would like to enforce that abomonation::Exhume<'de> is only implemented for Generic<'de, 'de, 'de>.

This where clase does not fully do what I want, because it allows Self to be e.g. &'static Something when 'de is shorter-lived than 'static, which is wrong and can lead to memory unsafety through deserialization of a fake &'static T that points into short-lived stack-allocated bytes followed by deallocation of said bytes.

A 'de: 'self bound (where 'self is the lifetime of Self) is not enough either, because abomonation::decode() will ultimately generate a &'de Self, and therefore Self must live for at least 'de.

Since Self must live for at least 'de and at most 'de, it must live for exactly 'de, i.e. Generic<'de, 'de, 'de> is what I want here. But I don't know how to do it.

{
#[allow(unused_mut)]
unsafe fn exhume(
self_: std::ptr::NonNull<Self>,
mut bytes: &'de mut [u8]
) -> Option<&'de mut [u8]> {
// FIXME: This (briefly) constructs an &mut _ to invalid data
// (via "ref mut"), which is UB. The proposed &raw mut
// operator would allow avoiding this.
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the FIXME says, this operation cannot be implemented without UB in Rust, for now, but there is a new language feature on the way that should fix that.

match *self_.as_ptr() { #exhume }
Some(bytes)
}
}
})
}
21 changes: 21 additions & 0 deletions tests/test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -202,4 +202,25 @@ mod tests {
assert!(rest.len() == 0);
}
}

#[derive(Eq, PartialEq, Abomonation)]
pub struct StructWithRef<'a> {
a: &'a str,
b: bool,
}

#[test]
fn test_struct_with_ref() {
let record = StructWithRef { a: &"test", b: true };

let mut bytes = Vec::new();
unsafe { encode(&record, &mut bytes).unwrap(); }

assert_eq!(bytes.len(), measure(&record));

if let Some((result, rest)) = unsafe { decode::<StructWithRef>(&mut bytes) } {
assert!(result == &record);
assert!(rest.len() == 0);
}
}
}