-
Notifications
You must be signed in to change notification settings - Fork 214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add APIs for tracking/debugging undesired object retention (aka "leaks") #7362
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
FUDCo
added
enhancement
New feature or request
SwingSet
package: SwingSet
performance
Performance related issues
labels
Apr 8, 2023
warner
approved these changes
Apr 9, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good, small changes recommended
FUDCo
force-pushed
the
7318-leak-diagnostics
branch
from
April 10, 2023 21:06
27f1298
to
8678f93
Compare
… "leaks") Closes #7318 Adds two and a half API features for to aid debugging and testing for leaks of objects managed by LiveSlots: * The `testHooks` object returned by liveslots now contains a function `getRetentionStats` that will return a data object containing the counts of the various non-singular objects that LiveSlots tracks internally. (LiveSlots keeps track of all these things in JavaScript Maps and Sets, so counting these objects is simply a matter of returning the sizes of these various collections. One consequence of this is that `getRetentionStats` call will execute quickly regardless of how much stuff LiveSlots is holding onto.) * The `testHooks` object now also contains references to all these various Maps and Sets directly. Note that this is powerful and dangerous, but it's confined to the `testHooks` object which is only exposed during testing. * The data record that `getRetentionStats` produces is also returned as the result of every `bringOutYourDead` operation. From there it can be examined in tests, but, more signficantly, it will be output as part of the delivery status array that is written to the slog, so that a graph of object retention stats over time can be produced from a running chain or a long running performance test executing with a live swingset. Collections passed in `testHooks` and counted by `getRetentionStats`: Collection | Type | What ------------|:-----|----- exportedRemotables | B | exported objects, to pin remotables; dropped on export drop importedDevices | B | imported devices, to pin devices; grows monotonically remotableRefCounts | B | objects ref'd from off vat (kernel or storage) kernelRecognizableRemotables | C | exports recognizable by kernel; tracks vrefs known to kernel, drop on export retire, retire when dead or kernel retire exportedVPIDs | C | promises exported; drop on resolve (vat is decider) importedVPIDs | C | promises imported; drop on resolve (kernel is decider) vrefRecognizers | C | vrefs used as keys in VirtualObjectAwareWeakMap/Set definedDurableKinds | D | durable kinds that exist kindInfoTable | D | info about kinds (durable + non-durable) nextInstanceIDs | D | next id to allocate for kind once allocation has started possiblyDeadSet | E | baseRefs to investigate for GC; leared on BOYD possiblyRetiredSet | E | vrefs to investigate for retirement; cleared on BOYD slotToVal | F | live objects with vrefs valToSlot | E | live objects with vrefs Types: A - Keyed by strings referring to virtual/durable store collections; cardinality is the total number of collections B - Keyed by direct references to explicitly in-memory objects; cardinality limited by RAM capacity C - Keyed by strings referring to explicitly in-memory objects; cardinality limited by RAM capacity D - Keyed by kindID strings; cardinality is number of defined knids (sometimes only durable kinds) E - Transient collections; note that these have no counters since weak collections are not countable F - Keyed by vref strings referring to any kind of object currently addressable in memory
FUDCo
force-pushed
the
7318-leak-diagnostics
branch
from
April 10, 2023 22:07
8678f93
to
0a7221b
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
automerge:rebase
Automatically rebase updates, then merge
enhancement
New feature or request
performance
Performance related issues
SwingSet
package: SwingSet
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #7318
Adds two and a half API features to aid debugging and testing for leaks of objects managed by LiveSlots:
The
testHooks
object returned by liveslots now contains a functiongetRetentionStats()
that will return a data object containing the counts of the various non-singular objects that LiveSlots tracks internally. (LiveSlots keeps track of all these things in JavaScript Maps and Sets, so counting these objects is simply a matter of returning the sizes of these various collections. One consequence of this is that agetRetentionStats
call will execute quickly regardless of how much stuff LiveSlots is holding onto.)The
testHooks
object now also contains references to all these various Maps and Sets directly. Note that this is powerful and dangerous, but it's confined to thetestHooks
object which is only exposed during testing.The data record that
getRetentionStats
produces is also returned as the result of everybringOutYourDead
operation. From there it can be examined in tests, but, more signficantly, it will be output as part of the delivery status array that is written to the slog, so that a graph of object retention stats over time can be produced from a running chain or a long running performance test executing with a live swingset.Collections passed in
testHooks
and counted bygetRetentionStats
:exportedRemotables
importedDevices
remotableRefCounts
kernelRecognizableRemotables
exportedVPIDs
importedVPIDs
vrefRecognizers
VirtualObjectAwareWeakMap/Set
definedDurableKinds
kindInfoTable
nextInstanceIDs
possiblyDeadSet
possiblyRetiredSet
slotToVal
valToSlot
Types:
A - Keyed by strings referring to virtual/durable store collections; cardinality is the total number of such stores
B - Keyed by direct references to explicitly in-memory objects; cardinality limited by RAM capacity
C - Keyed by strings referring to explicitly in-memory objects; cardinality limited by RAM capacity
D - Keyed by kindID strings; cardinality is number of defined knids (sometimes only durable kinds)
E - Transient collections, held weakly. Note that because they are weak collections they're not countable, so they lack corresponding counts in the retention stats
F - Keyed by vref strings referring to any kind of object currently addressable in memory