Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/develop' into feature/InvokesCG
Browse files Browse the repository at this point in the history
# Conflicts:
#	sootup.callgraph/src/main/java/sootup/callgraph/RapidTypeAnalysisAlgorithm.java
#	sootup.core/src/main/java/sootup/core/jimple/common/expr/JNewMultiArrayExpr.java
#	sootup.tests/pom.xml
  • Loading branch information
JonasKlauke committed Jul 15, 2024
2 parents 38bd53e + 3e1e20f commit 47f0e16
Show file tree
Hide file tree
Showing 472 changed files with 37,586 additions and 2,608 deletions.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/FEATURE_TEMPLATE.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ body:
attributes:
label: Missing Feature
description: Please describe the wanted functionality in detail. If possible enhance it with examples and/or scientific publications.
value: ""
value: Please describe the missing feature here..
validations:
required: true

Expand Down
12 changes: 4 additions & 8 deletions .github/workflows/gh-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,7 @@ jobs:
python-version: 3.x

# install dependencies
- run: pip install mike
- run: pip install mkdocs-material
- run: pip install mkdocs-tooltips
- run: pip install git+https://github.com/RedisLabs/mkdocs-include.git
- run: pip install git+https://github.com/swissiety/LspLexer4Pygments.git
- run: pip install mike mkdocs-material mkdocs-tooltips git+https://github.com/RedisLabs/mkdocs-include.git git+https://github.com/swissiety/LspLexer4Pygments.git
# grab latest release url of the JimpleLSP jar and download it
- run: curl -s -L -o ./jimplelsp.jar $(curl -s https://api.github.com/repos/swissiety/jimpleLsp/releases/latest | grep 'browser_download_url".*jar"' | cut -d ':' -f 2,3 | tr -d \")

Expand All @@ -68,8 +64,8 @@ jobs:
git config --local user.email "github-actions[bot]@users.noreply.github.com"
git config --local user.name "github-actions[bot]"
# sanitive head_ref name
- run: echo "DOC_VERSION_NAME=$(echo ${{ github.head_ref }} | sed "s/[^[:alnum:]-]/_/g" )" >> $GITHUB_ENV
# sanitize head_ref name
- run: echo "DOC_VERSION_NAME=$(echo ${{ github.head_ref }} | sed "s/[^([[:alnum:]_.-]/_/g" )" >> $GITHUB_ENV

# on push to develop branch - keep a doc around for develop to show the current state
- name: deploy doc in subdirectory
Expand All @@ -86,7 +82,7 @@ jobs:
uses: marocchino/sticky-pull-request-comment@v2
with:
message: |
You updated the documentation - [Doc Preview](https://${{ github.repository_owner }}.github.io/${{ github.event.repository.name }}/${{ env.DOC_VERSION_NAME }}_preview/).
[Documentation Preview](https://${{ github.repository_owner }}.github.io/${{ github.event.repository.name }}/${{ env.DOC_VERSION_NAME }}_preview/).
# on PR close - delete preview
- name: delete the deployed preview
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<p align="center">
<img width="350px" src="https://github.com/soot-oss/SootUp/blob/develop/docs/SootUpLogo.svg">
<img width="350px" src="https://github.com/soot-oss/SootUp/blob/develop/docs/img/SootUpLogo.svg">
</p>

# SootUp library ![Java CI with Maven](https://github.com/soot-oss/SootUp/workflows/Java%20CI%20with%20Maven/badge.svg?branch=develop) [![codecov](https://codecov.io/gh/soot-oss/SootUp/branch/develop/graph/badge.svg?token=ELA7U7IAWD)](https://codecov.io/gh/soot-oss/SootUp) [![javadoc](https://javadoc.io/badge2/org.soot-oss/sootup.core/javadoc.svg)](https://javadoc.io/doc/org.soot-oss/sootup.core) [![Maven Central](https://maven-badges.herokuapp.com/maven-central/org.soot-oss/sootup.core/badge.svg)](https://central.sonatype.com/artifact/org.soot-oss/sootup)
Expand Down Expand Up @@ -47,7 +47,7 @@ You want to collaborate? Please read our [coding guidelines and the contributors

## Publications
[the SootUp paper](https://doi.org/10.1007/978-3-031-57246-3_13) explains further details and the design decision behind SootUp.
[Preprint](/docs/SootUp-paper.pdf) is also available.
[Preprint](/docs/assets/SootUp-paper.pdf) is also available.

If you use SootUp in your research work, feel free to cite it as follows:

Expand Down
147 changes: 147 additions & 0 deletions docs/analysisinput.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# Analysis Input
i.e. What should be analyzed. An `AnalysisInputLocation` points to code input SootUp can analyze.
We ship multiple Implementations that can handle different input.

Additionally you can specify a SourceType. This determines what is considered e.g. in the CallGraphs generation.
Further you can specify a List of [BodyInterceptors](bodyinterceptors.md), which will optimize the raw Jimple IR that was transformed from the input.

### Java Runtime
#### Java <=8
The `DefaultRTJaAnalysisInputLocation` points to the rt.jar of the executing JVM.


```java
AnalysisInputLocation inputLocation = new DefaultRTJaAnalysisInputLocation();
JavaView view = new JavaView(inputLocation);
```

To include a different Java Runtime library point to any rt.jar via a `JavaClassPathAnalysisInputLocation` as its a usual .jar file.

#### Java >=9
The `JRTFilesystemAnalysisInputLocation` points to the jigsawed java runtime of the executing JVM.

```java
AnalysisInputLocation inputLocation = new JrtFileSystemAnalysisInputLocation();
JavaView view = new JavaView(inputLocation);
```


!!! info "If you have errors like Java.lang.String, Java.lang.Object, ... you are most likely missing this AnalysisInputLocation."

### Java Bytecode
File-Extensions: `.class, .jar, .war`

The `JavaClassPathAnalysisInputLocation` is the equivalent of the classpath you would pass to the java executable i.e. point to root(s) of package(s).

=== "Directory"
```java
AnalysisInputLocation inputLocation =
new JavaClassPathAnalysisInputLocation("target/"); // points to
JavaView view = new JavaView(inputLocation);
```

=== ".jar File"
```java
AnalysisInputLocation inputLocation = new JavaClassPathAnalysisInputLocation("myCode.jar");
JavaView view1 = new JavaView(inputLocation);

// if you want to analyze a specific language level of a multi release jar
AnalysisInputLocation inputLocation =
new MultiReleaseJarAnalysisInputLocation("myCode.jar", new JavaLanguage(10) );
JavaView view2 = new JavaView(inputLocation);
```

=== ".class File"
```java
// if you omit the package structure while pointing to a file,
// you have to pass the omitted directories as a parameter
AnalysisInputLocation inputLocation = new PathBasedAnalysisInputLocation.
ClassFileBasedAnalysisInputLocation("Banana.class", "packageName.subPackage", SourceType.Application);
JavaView view = new JavaView(inputLocation);
```

=== "Complete class path"
```java
String cp = "myCode.jar" + File.pathSeparator + "dependency.jar" + File.pathSeparator + "target/classes/";
AnalysisInputLocation inputLocation = new JavaClassPathAnalysisInputLocation(cp);
JavaView view = new JavaView(inputLocation);
```

### Java Sourcecode
File-Extensions: `.java`

With the `OTFCompileAnalysisInputLocation` you can point directly to .java files or pass a String with Java sourcecode.
The AnalysisInputLocation delegates the data to the `JavaCompiler` and transform the bytecode from the compiler to Jimple.

=== "Single File"
```java
AnalysisInputLocation inputLocation = new OTFCompileAnalysisInputLocation("Banana.java");
JavaView view = new JavaView(inputLocation);
```

=== "Multiple Files"
```java
List<Path> files = Arrays.asList(Paths.get("Apple.java"), Paths.get("Banana.java"));
AnalysisInputLocation inputLocation = new OTFCompileAnalysisInputLocation(files);
JavaView view = new JavaView(inputLocation);
```
=== "File as String"
```java
String content = "public class B{ }";
AnalysisInputLocation location = new OTFCompileAnalysisInputLocation("B.java", content );
JavaView view = new JavaView(location);
```

`JavaSourcePathInputLocation` [***experimental!***]{Has huge problems with exceptional flow!} - points to a directory that is the root source directory (containing the package directory structure).

### Jimple
File-Extensions: `.jimple`

The `JimpleAnalysisInputLocation` needs a Path to a .jimple file or a directory.

```java
Path path = Paths.get("Banana.java");
AnalysisInputLocation jimpleLocation = new JimpleAnalysisInputLocation(path);
JavaView view = new JavaView(jimpleLocation);
```


### Android Bytecode
File-Extensions: `.apk`

The `ApkAnalysisInputLocation` currently uses dex2jar internally

```java
Path path = Paths.get("Banana.apk");
AnalysisInputLocation inputLocation = new ApkAnalysisInputLocation(path);
JavaView view = new JavaView(inputLocation);
```

!!! info "A SootUp solution to directly generate Jimple is WIP!"


### Combining Multiple AnalysisInputLocations
But what if I want to point to multiple AnalysisInputLocations?

```java
AnalysisInputLocation mainJar = new JavaClassPathAnalysisInputLocation("myCode.jar");
AnalysisInputLocation jarA = new JavaClassPathAnalysisInputLocation("dependencyA.jar");
AnalysisInputLocation jarB = new JavaClassPathAnalysisInputLocation("dependencyB.jar");

List<AnalysisInputLocation> inputlocationList = Arrays.asList(mainJar, jarA, jarB);

JavaView view = new JavaView(inputlocationList);
```
!!! note "Of course you can combine different types of `AnalysisInputLocation`s as well!"


### Maven Project as Analysis Input in SootUp
This uses `#!shell mvn compile` + `JavaClassPathAnalysisInputLocation` under the hood to include a maven project.
```java
TODO: let the code sail with the upstream boat to this doc.
```

Unfortunately its harder to extract the path of the binary result of Gradle projects in a unified way for all kinds of models - If you have a solution are looking forward to merge your contribution :-).

### Java cli arguments to configure SootUp
We created a [Utility](tool_setup.md) that parses a String of java command line arguments and configures SootUp respectively.
14 changes: 7 additions & 7 deletions docs/announce.md → docs/announcement.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,17 @@ SootUp is a library that can easily be included in other projects, leaving those

Below is an overview of what’s new.

* Library by default, framework as an option
* Modular Architecture, no more singletons
* New source code frontend
* Immutable Jimple IR
* Greatly increased testability and test coverage
- Library by default, framework as an option
- Modular Architecture, no more singletons
- New source code frontend
- Immutable Jimple IR
- Greatly increased testability and test coverage ![Coverage](https://camo.githubusercontent.com/adc4ab244f7c0c2b2f3fec0a6e5d778421ddc0be7f89a608c16533c9a964766f/68747470733a2f2f636f6465636f762e696f2f67682f736f6f742d6f73732f536f6f7455702f6272616e63682f646576656c6f702f67726170682f62616467652e7376673f746f6b656e3d454c4137553749415744)

SootUp is not a drop-in replacement for Soot! Due to its completely new architecture and API it is essentially an almost complete rewrite. For a while, Soot and SootUp will coexist, as many existing tools depend on Soot, yet our maintenance efforts will henceforth be focused on SootUp, not Soot, and on extending SootUp with those capabilities that people still find missing. For now, we recommend using SootUp for greenfield projects.

For more details, check out
* The SootUp home page: https://soot-oss.github.io/SootUp/, and
* The SootUp repository: https://github.com/soot-oss/SootUp/

[This Page ;-)](https://soot-oss.github.io/SootUp/) and The SootUp repository: [https://github.com/soot-oss/SootUp/](https://soot-oss.github.io/SootUp/)

We are very much looking forward to your feedback and feature requests. To this end, best create appropriate issues in the repository.

Expand Down
File renamed without changes.
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
29 changes: 14 additions & 15 deletions docs/bodyinterceptors.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,32 +9,31 @@ The "raw" generated Jimple from the Bytecodefrontend needs a lot improvements -
- The Locals we get from the Java bytecode are typically untyped. Therefore we have to augment the Local types which is done by the TypeAssigner.
- t.b.c.

Method scoped optimisations:
Optimizations (method scope)

- ConditionalBranchFolder: removes tautologic ifs that are always true/false - if we can determine it in the scope of the method.
- EmptySwitchEliminator: removes switches that are not really switching
- ConstantPropagatorAndFolder: calculates constant values before runtime
- CastAndReturnInliner: Removes merging flows to a single return
- UnreachableCodeEliminator: speaks for itself.
- TrapTightener

Make Local names standardized:
Standardize Jimple appearance

- LocalNameStandardizer: numbers Locals with the scheme: type-initial + number of type occurence

!!! info "Soot Equivalent"

[BodyTransformer](https://github.com/soot-oss/soot/blob/develop/src/main/java/soot/BodyTransformer.java)


Below, we show how these BodyInterceptors work for the users who are interested in their internal workings.

### LocalSplitter

LocalSplitter is a<code>BodyInterceptor</code>that attempts to identify and separate uses of a local variable (as definition) that are independent of each other by renaming local variables.


Example 1:

![LocalSplitter Example_1](./figures/LocalSplitter%20Example_1.png)
![LocalSplitter Example_1](assets/figures/LocalSplitter%20Example_1.png)

As shown in the example above, the local variable<code>l1</code>is defined twice. It can be split up into two new local variables: <code>l1#1</code> and <code>l1#2</code> because the both definitions are independent of each other.

Expand All @@ -45,7 +44,7 @@ Look for foldable navigation and tabs for showing old vs new

Example 2:

![LocalSplitter Example_2](./figures/LocalSplitter%20Example_2.png)
![LocalSplitter Example_2](assets/figures/LocalSplitter%20Example_2.png)

In the second example, the local variable<code>l2</code>is defined thrice. But it cannot be split up into three new local variables as in the first example, because its definitions in the if-branches are not independent of each other. Therefore, it can only be split up into two local variables as shown in the figure.

Expand All @@ -57,7 +56,7 @@ LocalPacker is a<code>BodyInterceptor</code>that attempts to minimize the number

Example:

![LocalPacker Example](./figures/LocalPacker%20Example.png)
![LocalPacker Example](assets/figures/LocalPacker%20Example.png)

In the given example above, the local variables<code>l1</code>,<code>l3</code>are summarized to be one local variable<code>l1</code>, because they have the same type without interference with each other. Likewise, the local variables<code>l2</code>,<code>l4</code>and<code>l5</code>are summarized to be another local variable<code>l2</code>. Although the local variable<code>l0</code>doesn't interfere any other local variables, it cannot be summed up with other local variables because of its distinctive type.

Expand All @@ -70,7 +69,7 @@ TrapTightener is a<code>BodyInterceptor</code>that shrinks the protected area co

Example:

![TrapTightener Example](./figures/TrapTightener%20Example.png)
![TrapTightener Example](assets/figures/TrapTightener%20Example.png)

We assume in the example above that only the<code>Stmt</code>:<code>l2 := 2</code>might throw an exception caught by the<code>Trap</code>which is labeled with<code>label3</code>. In the jimple body before running the TrapTightener, the protected area covered by the Trap contains three<code>Stmts</code>:<code>l1 := 1; l2 := 2; l2 := 3</code>. But an exception could only arise at the<code>Stmt</code>:<code>l2 := 2</code>. After the implementation of TrapTightener, we will get a contractible protected area which contains only the<code>Stmt</code>that might throw an exception, namely the<code>Stmt</code>:<code>l2 := 2</code>.

Expand All @@ -82,7 +81,7 @@ EmptySwitchEliminator is a<code>BodyInterceptor</code>that removes empty switch

Example:

![EmptySwitchEliminator Example](./figures/EmptySwitchEliminator%20Example.png)
![EmptySwitchEliminator Example](assets/figures/EmptySwitchEliminator%20Example.png)

As shown in the example above, the switch statement in the jimple body always takes the default action. After running EmptySwitchEliminator, the switch statement is replaced with a<code>GotoStmt</code>to the default case.

Expand All @@ -94,7 +93,7 @@ UnreachableCodeEliminator is a<code>BodyInterceptor</code>that removes all unrea

Example:

![UnreachableCodeEliminator Example](./figures/UnreachableCodeEliminator%20Example.png)
![UnreachableCodeEliminator Example](assets/figures/UnreachableCodeEliminator%20Example.png)

Obviously, the code segment<code>l2 = 2; l3 = 3;</code>is unreachable. It will be removed after running the UreachableCodeEliminator.

Expand All @@ -106,7 +105,7 @@ CopyPropagator is a<code>BodyInterceptor</code>that supports the global copy pro

Example for global copy propagation:

![UnreachableCodeEliminator Example](./figures/CopyPropagator%20Example_1.png)
![UnreachableCodeEliminator Example](assets/figures/CopyPropagator%20Example_1.png)

Consider a code segment in the following form:

Expand All @@ -125,7 +124,7 @@ In the example for global copy propagation, the first used<code>l1</code>is repl

Example for constant propagation:

![CopyPropagator Example_1](figures/CopyPropagator%20Example_2.png)
![CopyPropagator Example_1](assets/figures/CopyPropagator%20Example_2.png)

Constant propagation is similar to copy propagation. Consider a code segment in the following form:

Expand Down Expand Up @@ -169,8 +168,8 @@ StaticSingleAssignmentFormer is a<code>BodyInterceptor</code>that transforms jim

Example:

![SSA Example_1](./figures/SSA%20Example_1.png)
![SSA Example_1](assets/figures/SSA%20Example_1.png)

![SSA Example_2](./figures/SSA%20Example_2.png)
![SSA Example_2](assets/figures/SSA%20Example_2.png)

In the given example, the StaticSingleAssignmentFormer assigns each<code>IdentityStmt</code>and<code>AssignStmt</code>to a new local variable . And each use uses the local variable which is most recently defined. Sometimes, it is impossible to determine the most recently defined local variable for a use in a join block. In this case, the StaticSingleAssignmentFormer will insert a<code>PhiStmt</code>in the front of the join block to merge all most recently defined local variables and assign them a new local variable.
11 changes: 6 additions & 5 deletions docs/advanced-topics.md → docs/builtin-analyses.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
# Functionalities and Utilities
# BuiltIn Analyses
More to come!

#### LocalLivenessAnalyser
### LocalLivenessAnalyser

LocalLivenessAnalyser is used for querying for the list of live local variables before and after a given <code>Stmt</code>.

Example:

![LocalLiveness Example](./figures/LocalLiveness%20Example.png)
![LocalLiveness Example](assets/figures/LocalLiveness%20Example.png)

The live local variables before and after each <code>Stmt</code> will be calculated after generating an instance of LocalLivenessAnalyser as shown the example above. They can be queried by using the methods <code>getLiveLocalsBeforeStmt</code> and <code>getLiveLocalsAfterStmt</code>.

#### DominanceFinder
### DominanceFinder

DomianceFinder is used for querying for the immediate dominator and dominance frontiers for a given basic block.

Example: ![DominanceFinder Example](figures/DominanceFinder%20Example.png)
Example: ![DominanceFinder Example](assets/figures/DominanceFinder%20Example.png)

After generating an instance of DominanceFinder for a <code>BlockGraph</code>, we will get the immediate dominator and dominance frontiers for each basic block. The both properties can be queried by using the methods<code>getImmediateDominator</code>and<code>getDominanceFrontiers</code>.
Loading

0 comments on commit 47f0e16

Please sign in to comment.