NegativeArraySizeException parsing huge heap dump

[eclipsewebmaster]

| --- | --- |
| Bugzilla Link | 552009 |
| Status | NEW |
| Importance | P3 normal |
| Reported | Oct 10, 2019 10:29 EDT |
| Modified | Jun 03, 2021 03:44 EDT |
| Version | 1.9 |
| Reporter | Andrew Johnson |

Description

From the MAT dev mailing list: https://www.eclipse.org/lists/mat-dev/msg00589.html

Hi All,

Seems even if # of objects is less than 2 billion, we still have some issues when we use MAT. I encounter following error log when I try to open heap dump with 1 billion object...

!ENTRY org.eclipse.mat.ui 1 0 2019-10-09 00:09:01.674
!MESSAGE Heap D:\tmp\19674.dump contains 1,104,267,590 objects

!ENTRY org.eclipse.core.jobs 4 2 2019-10-09 02:02:42.929
!MESSAGE An internal error occurred during: "Parsing heap dump from 'D:\tmp\19674.dump'".
!STACK 0
java.lang.NegativeArraySizeException
at java.io.ObjectOutputStream$HandleTable.growEntries(ObjectOutputStream.java:2347)
at java.io.ObjectOutputStream$HandleTable.assign(ObjectOutputStream.java:2276)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1428)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1378)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at org.eclipse.mat.collect.HashMapIntObject.writeObject(HashMapIntObject.java:438)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1140)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at org.eclipse.mat.parser.internal.SnapshotImpl.create(SnapshotImpl.java:215)
at org.eclipse.mat.parser.internal.SnapshotImplBuilder.create(SnapshotImplBuilder.java:95)
at org.eclipse.mat.parser.internal.SnapshotFactoryImpl.parse(SnapshotFactoryImpl.java:235)
at org.eclipse.mat.parser.internal.SnapshotFactoryImpl.openSnapshot(SnapshotFactoryImpl.java:126)
at org.eclipse.mat.snapshot.SnapshotFactory.openSnapshot(SnapshotFactory.java:147)
at org.eclipse.mat.ui.snapshot.ParseHeapDumpJob.run(ParseHeapDumpJob.java:83)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:60)

Thanks,
Roy

[eclipsewebmaster]

By Andrew Johnson on Oct 10, 2019 12:34

For the record, please could you confirm the version of MAT and the JVM version.

The failure looks to be in JVM class library code in ObjectOutputStream where the table holding the mapping of objects to IDs is extended. This code seems to double the size of the array each time.
The MAT code is writing out the GC roots as a HashMapIntObject<XGCRootInfo[]>
The heap size of 1,104,267,590 objects is just over 2^30

Perhaps the dump has very many GC roots (keep unreachable objects is set?).

[eclipsewebmaster]

By Andrew Johnson on Jun 03, 2021 03:44

The line in SnapshotImpl is https://git.eclipse.org/c/mat/org.eclipse.mat.git/tree/plugins/org.eclipse.mat.parser/src/org/eclipse/mat/parser/internal/SnapshotImpl.java?id=2909188cf1c0af29e7e2e4affc17ec872a490cbb#n215

Guessing version as 1.9