Fix complex transi compact #547
Open
+531
−118
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
casperisfine
force-pushed
the
fix-complex-transi-compact
branch
2 times, most recently
from
9232d8c
to
a4d6719
Compare
|
It's crashing on: case VM_METHOD_TYPE_ISEQ:
if (def->body.iseq.iseqptr) gc_mark(objspace, (VALUE)def->body.iseq.iseqptr); |
due to a failure on a CI http://ci.rvm.jp/results/trunk-iseq_binary@ruby-sp2-docker/4779277 ``` expected: == disasm: #<ISeq:prism_test_forwarding_arguments_node1@<compiled>:2 (2,8)-(4,11)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: 1]) [ 1] "..."@0 0000 putself ( 3) 0001 getlocal_WC_0 ?@-2 0003 splatarray false 0005 getblockparamproxy ?@-1, 0 0008 send <calldata!mid:prism_test_forwarding_arguments_node, argc:1, ARGS_SPLAT|ARGS_BLOCKARG|FCALL>, nil 0011 leave ( 2) actual: == disasm: #<ISeq:prism_test_forwarding_arguments_node1@<compiled>:2 (2,8)-(4,11)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: 1]) [ 1] "..."@0 0000 putself ( 3) 0001 getlocal_WC_0 ?@-2 0003 splatarray false 0005 getblockparamproxy "!"@-1, 0 0008 send <calldata!mid:prism_test_forwarding_arguments_node, argc:1, ARGS_SPLAT|ARGS_BLOCKARG|FCALL>, nil 0011 leave ( 2) /tmp/ruby/src/trunk-iseq_binary/tool/lib/iseq_loader_checker.rb:36:in `exit': exit (SystemExit) ```
That function is a bit too low level to called from multiple
places. It's always used in tandem with `rb_shape_set_too_complex`
and both have to know how the object is laid out to update the
`iv_ptr`.
So instead we can provide two higher level function:
- `rb_obj_copy_ivs_to_hash_table` to prepare a `st_table` from an
arbitrary oject.
- `rb_obj_convert_to_too_complex` to assign the new `st_table`
to the old object, and safely free the old `iv_ptr`.
Unfortunately both can't be combined into one, because `rb_obj_copy_ivar`
need `rb_obj_copy_ivs_to_hash_table` to copy from one object
to another.
We ran into that case on our CI, including some sizes would help debug it much easier.
Reproduction script:
```
o = Object.new
10.times { |i| o.instance_variable_set(:"@A#{i}", i) }
i = 0
a = Object.new
while RubyVM::Shape.shapes_available > 2
a.instance_variable_set(:"@i#{i}", 1)
i += 1
end
o.remove_instance_variable(:@a0)
puts o.instance_variable_get(:@A1)
```
Before this patch, it would incorrectly output `2` and now it correctly
outputs `1`.
Now the documentation that was already in the codebase for `File::Stat#directory?` shows up.
Found through GC.stress + GC.auto_compact crashes in rubyGH-8932. Previously, the compaction run within `rb_method_entry_alloc()` could move the `def->body.iseq.cref` and `iseqptr` set up before the call and leave the `def` pointing to moved addresses. Nothing was marking `def` during that GC run. Low probability reproducer: GC.stress = true GC.auto_compact = true arr = [] alloc = 1000.times.map { [] } alloc = nil a = arr.first GC.start
When transitioning an object to TOO_COMPLEX we copy all its ivar in a table, but if GC (and compaction) trigger in the middle of the transition, the old `iv_ptr` might still be the canonical ivar list and will be updated by the GC, but the reference we copied in the table will be outdated. So we must pin these reference until they are all copied and the object `iv_ptr` is pointing to the table. To do this we allocate a temporary TOO_COMPLEX object which we use as the host for the copy. TOO_COMPLEX objects are marked with `mark_tbl` which does pin references. Co-Authored-By: Alan Wu <[email protected]>
casperisfine
force-pushed
the
fix-complex-transi-compact
branch
from
a4d6719
to
97fb07d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I think this PR fixes the compaction issue, but now I run into the bug I was looking to in the first place: