Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass ObjectReference to the is_movable API #1172

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/policy/copyspace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ impl<VM: VMBinding> SFT for CopySpace<VM> {
false
}

fn is_movable(&self) -> bool {
fn is_movable(&self, _object: ObjectReference) -> bool {
true
}

Expand Down
6 changes: 5 additions & 1 deletion src/policy/immix/immixspace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,11 @@ impl<VM: VMBinding> SFT for ImmixSpace<VM> {
fn is_object_pinned(&self, object: ObjectReference) -> bool {
VM::VMObjectModel::LOCAL_PINNING_BIT_SPEC.is_object_pinned::<VM>(object)
}
fn is_movable(&self) -> bool {
fn is_movable(&self, _object: ObjectReference) -> bool {
#[cfg(feature = "object_pinning")]
if self.is_object_pinned(_object) {
Copy link
Collaborator

@wks wks Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I agree that the trait method SFT::is_movable should have object as a one parameter (because a space can, in theory, refuse to move some objects but not others), I don't think object pinning should be involved here.

The main use of ObjectReference::is_movable is foreign function interface (FFI) where the VM passes an object reference (or interior pointer) to native code. If is_movable returns true, the VM will omit the pinning and unpinning operation around the native call (or the action of telling mmtk-core the object is an unmovable root when GC happens while the native function is being executed).

But if is_movable answers true because the pinning bit is set, the following thing may happen. Assume there are two mutator threads.

(Update: I got the return values wrong. Now corrected.)

  1. The first mutator is about to call a native function. It calls obj.is_movable and gets false true (it is movable). Then the first mutator pins the object and then calls the native function.
  2. The second mutator is about to call another function. It calls obj.is_movable and gets true false (it is not movable) because the first mutator pinned that object. It omits the pinning operation and calls the native function directly.
  3. The first mutator returns from the native function, and unpins the object.
  4. GC started. The GC thinks all mutators have stopped because the second mutator is executing native code. The GC thread sees the object is not pinned, and moves it.
  5. The second mutator is still accessing the object in native code, but the object is moved. It is a data race.

The documentation of ObjectReference::is_movable simply says "Can the object be moved?" It is not very clear about how long the answer remain valid. Intuitively, if ObjectReference::is_movable returns true, it should mean "the object will never ever move until it is dead". But if the object cannot move because the pinning bit is set, it can once again become movable after we remove the pinning bit. This may cause some subtle confusion. We need to fix the documentation so that the VM binding developers know what to expect.

Copy link
Collaborator Author

@k-sareen k-sareen Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you have the return values swapped in your example. If the object is pinned then is_movable returns false.

I agree that this is a valid use-case, but the documentation of is_movable in the SFT mentions object pinning. It is not the only use-case of is_movable, however. ART uses it when cloning objects, for example, to allocate the object with the correct semantics/allocator. I've just removed the non-moving allocator and replaced it with pin bits, and I tripped on this bug.

I'm not sure what the best way to approach this is.

Copy link
Collaborator Author

@k-sareen k-sareen Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, your example is true for the current state of affairs anyway. As in, if two threads pin the same object, then one thread may unpin it while the other thread is still using it. So this PR doesn't make the situation worse. I'm not sure if that is something we can resolve anyway as we would effectively need to implement a counter for the pinning metadata instead of a single bit.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scenario Kunshan describes is an issue with our pinning semantic (which uses a boolean rather than a count). If you replace the call to object.is_movable with memory_manager::is_pinned in the scenario, you would get the same problem.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wks What are your thoughts?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the pinning counter and the semantics of is_movable are orthogonal.

If we implement a per-object pinning counter and apply pin_object and unpin_object unconditionally when calling native, there will be no problem.

But if we use pinning counter, and also use is_movable to detect if an object is movable and elide the pin/unpin operation depending on the return value of is_movable, there will still be a problem. The scenario I described will still go wrong if the first mutator applies a pinning counter, and the second mutator elides the pinning/unpinning because it is "unmovable".

The crux is still the semantics. I insist that object.is_movable returning false should mean it can't be moved until it dies.

But if you need to clone object and also clone some special semantics (e.g. whether it can be moved), we have to find other metadata to use.

Ruby also has a dup function that clones an object. It also copies the "write-barrier-unprotected" (WB-unprotected) property and its finalizers. CRuby uses a bitmap to identify if an object is "WB-unprotected", and uses an in-header bit for whether it is finalizable. Currently I just implemented the query of whether an object is "WB-unprotected" using a table lookup, which is as efficient as CRuby.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, if you are sure there is no race, you can use memory_manager::is_pinned to query the pinning bit and then pin the copied object, too, if that's desired.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay. I don't think this will get resolved over GitHub comments. It doesn't matter in my case since I never unpin objects (until they die), so I'll just keep this in my branch.

return false;
}
!super::NEVER_MOVE_OBJECTS
}

Expand Down
2 changes: 1 addition & 1 deletion src/policy/immortalspace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ impl<VM: VMBinding> SFT for ImmortalSpace<VM> {
fn is_object_pinned(&self, _object: ObjectReference) -> bool {
true
}
fn is_movable(&self) -> bool {
fn is_movable(&self, _object: ObjectReference) -> bool {
false
}
#[cfg(feature = "sanity")]
Expand Down
2 changes: 1 addition & 1 deletion src/policy/largeobjectspace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ impl<VM: VMBinding> SFT for LargeObjectSpace<VM> {
fn is_object_pinned(&self, _object: ObjectReference) -> bool {
true
}
fn is_movable(&self) -> bool {
fn is_movable(&self, _object: ObjectReference) -> bool {
false
}
#[cfg(feature = "sanity")]
Expand Down
2 changes: 1 addition & 1 deletion src/policy/lockfreeimmortalspace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ impl<VM: VMBinding> SFT for LockFreeImmortalSpace<VM> {
fn is_object_pinned(&self, _object: ObjectReference) -> bool {
true
}
fn is_movable(&self) -> bool {
fn is_movable(&self, _object: ObjectReference) -> bool {
unimplemented!()
}
#[cfg(feature = "sanity")]
Expand Down
2 changes: 1 addition & 1 deletion src/policy/markcompactspace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ impl<VM: VMBinding> SFT for MarkCompactSpace<VM> {
false
}

fn is_movable(&self) -> bool {
fn is_movable(&self, _object: ObjectReference) -> bool {
true
}

Expand Down
2 changes: 1 addition & 1 deletion src/policy/marksweepspace/malloc_ms/global.rs
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ impl<VM: VMBinding> SFT for MallocSpace<VM> {
false
}

fn is_movable(&self) -> bool {
fn is_movable(&self, _object: ObjectReference) -> bool {
false
}

Expand Down
2 changes: 1 addition & 1 deletion src/policy/marksweepspace/native_ms/global.rs
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ impl<VM: VMBinding> SFT for MarkSweepSpace<VM> {
false
}

fn is_movable(&self) -> bool {
fn is_movable(&self, _object: ObjectReference) -> bool {
false
}

Expand Down
4 changes: 2 additions & 2 deletions src/policy/sft.rs
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ pub trait SFT {

/// Is the object movable, determined by the policy? E.g. the policy is non-moving,
/// or the object is pinned.
fn is_movable(&self) -> bool;
fn is_movable(&self, object: ObjectReference) -> bool;

/// Is the object sane? A policy should return false if there is any abnormality about
/// object - the sanity checker will fail if an object is not sane.
Expand Down Expand Up @@ -146,7 +146,7 @@ impl SFT for EmptySpaceSFT {
fn is_object_pinned(&self, _object: ObjectReference) -> bool {
false
}
fn is_movable(&self) -> bool {
fn is_movable(&self, _object: ObjectReference) -> bool {
/*
* FIXME steveb I think this should panic (ie the function should not
* be invoked on an empty space). However, JikesRVM currently does
Expand Down
2 changes: 1 addition & 1 deletion src/policy/vmspace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ impl<VM: VMBinding> SFT for VMSpace<VM> {
fn is_object_pinned(&self, _object: ObjectReference) -> bool {
true
}
fn is_movable(&self) -> bool {
fn is_movable(&self, _object: ObjectReference) -> bool {
false
}
#[cfg(feature = "sanity")]
Expand Down
2 changes: 1 addition & 1 deletion src/util/address.rs
Original file line number Diff line number Diff line change
Expand Up @@ -600,7 +600,7 @@ impl ObjectReference {

/// Can the object be moved?
pub fn is_movable<VM: VMBinding>(self) -> bool {
unsafe { SFT_MAP.get_unchecked(self.to_address::<VM>()) }.is_movable()
unsafe { SFT_MAP.get_unchecked(self.to_address::<VM>()) }.is_movable(self)
}

/// Get forwarding pointer if the object is forwarded.
Expand Down
Loading