Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GC overhead limit exceeded when running Instrumenter from source code #2

Open
ChunghaSung opened this issue Jul 29, 2019 · 2 comments

Comments

@ChunghaSung
Copy link

Hi, I tried to compile and run "instrumenter" from the source code (instead of running instrumenter.sh).

By setting Instrumenter's main as a default function, I used this arguments.
com.twitter.android -w -allow-phantom-refs -process-multiple-dex -force-android-jar <sdk_path>/android-28/android.jar -src-prec apk -output-format dex -process-dir <my_path>/AndroidSlicer/tool/Twitter_v7.93.2-release.50_apkpure.com.apk

The app I used is from Google Play Store for version 7.93.2
After that, there is no error for a while, and it ran for like 5 minutes.

At the end, the program dies because of GC
I found that the paper used Twitter as a benchmark. I am curious how to instrument it.
I tried it with a small example, it could be done.
It would be really appreciated if you help me to figure this out.

Below is the error message.

Soot started on Sun Jul 28 16:59:22 PDT 2019
[Thread-9] ERROR heros.solver.CountingThreadPoolExecutor - Worker thread execution failed: GC overhead limit exceeded
java.lang.OutOfMemoryError: GC overhead limit exceeded
at soot.toDex.ConstantVisitor.caseStringConstant(ConstantVisitor.java:86)
at soot.jimple.StringConstant.apply(StringConstant.java:63)
at soot.toDex.StmtVisitor.caseAssignStmt(StmtVisitor.java:489)
at soot.jimple.internal.JAssignStmt.apply(JAssignStmt.java:242)
at soot.toDex.DexPrinter.toInstructions(DexPrinter.java:1511)
at soot.toDex.DexPrinter.toMethodImplementation(DexPrinter.java:1174)
at soot.toDex.DexPrinter.toMethods(DexPrinter.java:1083)
at soot.toDex.DexPrinter.addAsClassDefItem(DexPrinter.java:656)
at soot.toDex.DexPrinter.add(DexPrinter.java:1646)
at soot.PackManager.writeClass(PackManager.java:1096)
at soot.PackManager.lambda$writeOutput$1(PackManager.java:699)
at soot.PackManager$$Lambda$2/1879083009.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "Thread-9" java.lang.RuntimeException: java.lang.OutOfMemoryError: GC overhead limit exceeded
at soot.PackManager.writeOutput(PackManager.java:716)
at soot.PackManager.writeDexOutput(PackManager.java:584)
at soot.PackManager.writeOutput(PackManager.java:567)
at soot.Main.run(Main.java:271)
at soot.Main.main(Main.java:141)
at org.ucr.ds.cd.utilities.Instrumenter.main(Instrumenter.java:407)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at soot.toDex.ConstantVisitor.caseStringConstant(ConstantVisitor.java:86)
at soot.jimple.StringConstant.apply(StringConstant.java:63)
at soot.toDex.StmtVisitor.caseAssignStmt(StmtVisitor.java:489)
at soot.jimple.internal.JAssignStmt.apply(JAssignStmt.java:242)
at soot.toDex.DexPrinter.toInstructions(DexPrinter.java:1511)
at soot.toDex.DexPrinter.toMethodImplementation(DexPrinter.java:1174)
at soot.toDex.DexPrinter.toMethods(DexPrinter.java:1083)
at soot.toDex.DexPrinter.addAsClassDefItem(DexPrinter.java:656)
at soot.toDex.DexPrinter.add(DexPrinter.java:1646)
at soot.PackManager.writeClass(PackManager.java:1096)
at soot.PackManager.lambda$writeOutput$1(PackManager.java:699)
at soot.PackManager$$Lambda$2/1879083009.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
java.lang.OutOfMemoryError: GC overhead limit exceeded
at soot.toDex.ConstantVisitor.caseStringConstant(ConstantVisitor.java:86)
at soot.jimple.StringConstant.apply(StringConstant.java:63)
at soot.toDex.StmtVisitor.caseAssignStmt(StmtVisitor.java:489)
at soot.jimple.internal.JAssignStmt.apply(JAssignStmt.java:242)
at soot.toDex.DexPrinter.toInstructions(DexPrinter.java:1511)
at soot.toDex.DexPrinter.toMethodImplementation(DexPrinter.java:1174)
at soot.toDex.DexPrinter.toMethods(DexPrinter.java:1083)
at soot.toDex.DexPrinter.addAsClassDefItem(DexPrinter.java:656)
at soot.toDex.DexPrinter.add(DexPrinter.java:1646)
at soot.PackManager.writeClass(PackManager.java:1096)
at soot.PackManager.lambda$writeOutput$1(PackManager.java:699)
at soot.PackManager$$Lambda$2/1879083009.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Ouuups... something went wrong! Sorry about that.
Follow these steps to fix the problem:
1.) Are you sure you used the right command line?
Click here to double-check:
https://github.com/Sable/soot/wiki/Options-and-JavaDoc

2.) Not sure whether it's a bug? Feel free to discuss
the issue on the Soot mailing list:
https://github.com/Sable/soot/wiki/Getting-help

3.) Sure it's a bug? Click this link to report it.
https://github.com/Sable/soot/issues/new?title=java.lang.RuntimeException+when+...&body=Steps+to+reproduce%3A%0A1.%29+...%0A%0AFiles+used+to+reproduce%3A+%0A...%0A%0ASoot+version%3A+%3Cpre%3Etrunk%3C%2Fpre%3E%0A%0ACommand+line%3A%0A%3Cpre%3E-w+-allow-phantom-refs+-process-multiple-dex+-force-android-jar+%2Fhome%2Fchungha%2FAndroid%2FSdk%2Fplatforms%2Fandroid-28%2Fandroid.jar+-src-prec+apk+-output-format+dex+-process-dir+%2Fhome%2Fchungha%2Fwork%2FAndroidSlicer%2Ftool%2FTwitter_v7.93.2-release.50_apkpure.com.apk%3C%2Fpre%3E%0A%0AMax+Memory%3A%0A%3Cpre%3E3775MB%3C%2Fpre%3E%0A%0AStack+trace%3A%0A%3Cpre%3Ejava.lang.RuntimeException%3A+java.lang.OutOfMemoryError%3A+GC+overhead+limit+exceeded%0A%09at+soot.PackManager.writeOutput%28PackManager.java%3A716%29%0A%09at+soot.PackManager.writeDexOutput%28PackManager.java%3A584%29%0A%09at+soot.PackManager.writeOutput%28PackManager.java%3A567%29%0A%09at+soot.Main.run%28Main.java%3A271%29%0A%09at+soot.Main.main%28Main.java%3A141%29%0A%09at+org.ucr.ds.cd.utilities.Instrumenter.main%28Instrumenter.java%3A407%29%0ACaused+by%3A+java.lang.OutOfMemoryError%3A+GC+overhead+limit+exceeded%0A%09at+soot.toDex.ConstantVisitor.caseStringConstant%28ConstantVisitor.java%3A86%29%0A%09at+soot.jimple.StringConstant.apply%28StringConstant.java%3A63%29%0A%09at+soot.toDex.StmtVisitor.caseAssignStmt%28StmtVisitor.java%3A489%29%0A%09at+soot.jimple.internal.JAssignStmt.apply%28JAssignStmt.java%3A242%29%0A%09at+soot.toDex.DexPrinter.toInstructions%28DexPrinter.java%3A1511%29%0A%09at+soot.toDex.DexPrinter.toMethodImplementation%28DexPrinter.java%3A1174%29%0A%09at+soot.toDex.DexPrinter.toMethods%28DexPrinter.java%3A1083%29%0A%09at+soot.toDex.DexPrinter.addAsClassDefItem%28DexPrinter.java%3A656%29%0A%09at+soot.toDex.DexPrinter.add%28DexPrinter.java%3A1646%29%0A%09at+soot.PackManager.writeClass%28PackManager.java%3A1096%29%0A%09at+soot.PackManager.lambda%24writeOutput%241%28PackManager.java%3A699%29%0A%09at+soot.PackManager%24%24Lambda%242%2F1879083009.run%28Unknown+Source%29%0A%09at+java.util.concurrent.ThreadPoolExecutor.runWorker%28ThreadPoolExecutor.java%3A1149%29%0A%09at+java.util.concurrent.ThreadPoolExecutor%24Worker.run%28ThreadPoolExecutor.java%3A624%29%0A%09at+java.lang.Thread.run%28Thread.java%3A748%29%0A%3C%2Fpre%3E
Please be as precise as possible when giving us
information on how to reproduce the problem. Thanks!

Process finished with exit code 1

@ChunghaSung ChunghaSung changed the title GC out of memory GC overhead limit exceeded when running Instrumenter from source code Jul 29, 2019
@archer29m
Copy link
Member

archer29m commented Jul 29, 2019

For Twitter, this is the version that we used in the paper:

Twitter APK

This can be instrumented successfully.
Generally since AndroidSlicer’s instrumenter is based on Soot, it inherits Soot’s static analysis size limitations, so that it cannot handle very large apps.
Another hint to instrument large apps is to filter out the libraries that the analyzer knows that they are unrelated to his/her analysis, e.g., advertising, or authentication related libraries. For this purpose, you can uncomment line 173 of instrumenter.java to filter out the libraries that you don't want to instrument.

@ChunghaSung
Copy link
Author

I tried with the app you gave, I still get the same error..

These are the arguments for the main class "Instrumenter", and I am using intellij.

com.twitter.android -w -allow-phantom-refs -process-multiple-dex -force-android-jar /home/chungha/Android/Sdk/platforms/android-28/android.jar -src-prec apk -output-format dex -process-dir /home/chungha/work/AndroidSlicer/tool/com.twitter.android.apk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants