Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate some passes to Mini passes #11191

Merged
merged 90 commits into from
Oct 15, 2024
Merged
Show file tree
Hide file tree
Changes from 84 commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
c9ef2c3
Implement skeleton of mini pass framework
Akirathan Sep 26, 2024
43eafae
Rename MiniPassManager to MiniPassChainer
Akirathan Sep 26, 2024
4a0fd85
Implement default IR.withNewChildren
Akirathan Sep 26, 2024
1bbf991
Add MiniPassTester and MiniPassTraverser
Akirathan Sep 26, 2024
70f0a1a
Add tests for OperatorToFunction mini pass
Akirathan Sep 26, 2024
808dbd8
Add necessary withNewChildren implementations
Akirathan Sep 26, 2024
1abc70d
Add necessary withNewChildren implementations
Akirathan Sep 26, 2024
cdf1d87
MiniIRPass has metadata type argument
Akirathan Sep 27, 2024
d7b6016
Add mini pass version of LambdaShorthandToLambda
Akirathan Sep 27, 2024
1bd2b40
Add tests of mini pass version of LambdaShorthandToLambda
Akirathan Sep 27, 2024
61fed92
Implement Prefix.withNewChildren
Akirathan Sep 27, 2024
40252a2
Add IRTest to test children and withNewChildren API
Akirathan Sep 30, 2024
76dbc23
Update default IR.withNewChildren
Akirathan Sep 30, 2024
4a66460
Copy of children of DefinitionArgument.Specified is handled specifica…
Akirathan Sep 30, 2024
be2724d
Add necessary withNewChildren implementations
Akirathan Sep 30, 2024
fd3a727
LambdaToShorthandMini ignores blank fn
Akirathan Sep 30, 2024
8a5c435
Tests that have different internal naming are handled in different te…
Akirathan Sep 30, 2024
995dea5
DefinitionArgument.Specified implements withNewChildren
Akirathan Oct 1, 2024
985b8c2
Change prepare API
Akirathan Oct 1, 2024
28edccb
MiniIRPass is not parametrized based on metadata type
Akirathan Oct 1, 2024
6121750
Dummy implementation of minipass chaining
Akirathan Oct 1, 2024
54e5bd0
Add test for BinaryOperator.newChildren
Akirathan Oct 1, 2024
6b4c6b3
IR.withNewChildren must be called with the same length as children
Akirathan Oct 1, 2024
01561a1
MiniIRPass.prepare has parent parameter
Akirathan Oct 1, 2024
dd42554
Remove parent parameter from MiniIRPass.prepare
Akirathan Oct 2, 2024
f85f017
Remove MiniIRPass.runsAfter - it will not be necessary for this PR
Akirathan Oct 2, 2024
a4aaf8e
Add docs to MiniIRPass
Akirathan Oct 2, 2024
836c4c2
Add skeleton of TailCallMini pass
Akirathan Oct 2, 2024
079543a
runtime-compiler depends on org.graalvm.sdk:collections
Akirathan Oct 2, 2024
5d19be4
Implement withNewChildren on Expression via mapExpressions
Akirathan Oct 2, 2024
a369ed8
Fix MatchError in TailCallMini
Akirathan Oct 3, 2024
487003f
Revert "Implement withNewChildren on Expression via mapExpressions"
Akirathan Oct 4, 2024
b2f7053
Revert "runtime-compiler depends on org.graalvm.sdk:collections"
Akirathan Oct 4, 2024
edec03b
Replace org.graalvm.collections.EconomicSet with java.util.IdentityHa…
Akirathan Oct 4, 2024
67ff454
Revert tests that depend on InlineContext config
Akirathan Oct 4, 2024
547e9e6
Add more tests that test consistency with old pass version
Akirathan Oct 4, 2024
925a2de
Add logging to MiniPassTraverser
Akirathan Oct 4, 2024
dc6e7f3
Implement more withNewChildren methods
Akirathan Oct 4, 2024
c56c2be
Add another tail call mini pass test
Akirathan Oct 4, 2024
ea1462a
TailCallMini respect initial isInTailPosition flag
Akirathan Oct 4, 2024
18f4cea
Implement TailCallMini.toString
Akirathan Oct 4, 2024
e2fc538
Using IR.mapExpressions in MiniPassTraverser
JaroslavTulach Oct 8, 2024
a58ecc8
Detect and use MiniPassFactory in PassFactory
JaroslavTulach Oct 8, 2024
1aaa589
Move OperatorToFunction mega pass to test
Akirathan Oct 9, 2024
8604acb
Move LambdaShorthandToLambda mega pass to test
Akirathan Oct 9, 2024
0ddb96b
Use pattern match instead of isInstanceOf check in PassManager
Akirathan Oct 9, 2024
a85679d
Move TailCall mega pass to test
Akirathan Oct 9, 2024
80c4545
Implement MiniPass trait for testing mini and mega passes
Akirathan Oct 9, 2024
9a5dfea
TailCallMegaPass uses metadata from TailCall, not this
Akirathan Oct 9, 2024
d5c6983
Change API of MiniIRPass to accept only Expression
Akirathan Oct 10, 2024
89cde89
OperatorToFunctionMini checks post condition
Akirathan Oct 10, 2024
7c9f370
Add failing test to OperatorToFunctionTest
Akirathan Oct 10, 2024
92ae4fe
Implement IRDumper.dumpWithSvg
Akirathan Oct 10, 2024
65560c4
Ensure MiniPassTraverser returns transformed Module
Akirathan Oct 10, 2024
dfc54db
IRDumper handles SugaredType
Akirathan Oct 10, 2024
8771d58
MiniPassTraverser logs IRs with hash code.
Akirathan Oct 10, 2024
bdff605
Refactor TailCallMini to the new API
Akirathan Oct 10, 2024
ce56abb
typo
Akirathan Oct 10, 2024
d42d3e9
Must preprocess the module even for miniPass
JaroslavTulach Oct 11, 2024
ba7ab9d
Using analyseModuleBinding from TailCallMegaPass
JaroslavTulach Oct 11, 2024
90139e7
Allow for comparing mega and mini IR trees
JaroslavTulach Oct 11, 2024
8558d5b
Merging with develop. 47 failing tests. Mostly because of advanturous…
JaroslavTulach Oct 12, 2024
05fd38a
Making the TailCall pass closer to original mega pass code
JaroslavTulach Oct 12, 2024
367fc9a
Properly annotate withClue sections
JaroslavTulach Oct 12, 2024
36ebf2b
Associate TailCall pass metadata
JaroslavTulach Oct 12, 2024
0a02563
Rewrite using tailCandidates
JaroslavTulach Oct 12, 2024
83f4a57
Removal of notTailMeta
JaroslavTulach Oct 12, 2024
69b3274
Execute megaPass.runExpression at the end of the test
JaroslavTulach Oct 12, 2024
dea6dd6
One TailCallTest failure down to six
JaroslavTulach Oct 12, 2024
6c2b51d
Down to five failing tests in TailCallTest
JaroslavTulach Oct 12, 2024
f27ce6f
Down to three failures
JaroslavTulach Oct 12, 2024
f4077de
Reordering the test and down to two failures
JaroslavTulach Oct 12, 2024
1b021c2
Making all TailCallTest pass on mega as well as mini pass
JaroslavTulach Oct 12, 2024
8ce42e0
No need for TailCallMini
JaroslavTulach Oct 12, 2024
64b013f
Cleaning dead code up
JaroslavTulach Oct 13, 2024
f54ba6d
SectionsToBinOp is mini IR pass
JaroslavTulach Oct 13, 2024
40ef345
Introducing MiniIRPass.combine
JaroslavTulach Oct 13, 2024
8124e1e
Hiding traversal behind MiniIRPass.compile
JaroslavTulach Oct 13, 2024
668fdfd
Not including IRDumper changes in this PR
JaroslavTulach Oct 13, 2024
2af068f
No need for MetadataStorage.size
JaroslavTulach Oct 13, 2024
95b1756
Properly clone ChainedMiniPass when first or second delegates changes
JaroslavTulach Oct 14, 2024
c84e881
Removing duplicated test - mini pass testing is already inside of ass…
JaroslavTulach Oct 14, 2024
0964711
Rewriting MiniIRPass processing to queue - much easier for profiling …
JaroslavTulach Oct 14, 2024
ffd27df
MiniIRPass.prepare(parent, child)
JaroslavTulach Oct 14, 2024
924e404
Keep all IRs in local variables for easier debugging
JaroslavTulach Oct 14, 2024
2736a76
Combining subsequent mini passes
JaroslavTulach Oct 15, 2024
762045a
Both MiniPassFactory and IRPass are on par by extending IRProcessingPass
JaroslavTulach Oct 15, 2024
ef3450c
Only update IR metadata in transform methods
JaroslavTulach Oct 15, 2024
248c0f4
Removing dead code
JaroslavTulach Oct 15, 2024
befef0c
TailCall rewritten to Java
JaroslavTulach Oct 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
package org.enso.compiler.pass;

import java.util.Objects;
import org.enso.compiler.core.IR;
import org.enso.compiler.core.ir.Expression;

/** Utility class for chaining mini passes together. */
final class ChainedMiniPass extends MiniIRPass {
private final MiniIRPass firstPass;
private final MiniIRPass secondPass;

private ChainedMiniPass(MiniIRPass firstPass, MiniIRPass secondPass) {
this.firstPass = firstPass;
this.secondPass = secondPass;
}

static MiniIRPass chain(MiniIRPass firstPass, MiniIRPass secondPass) {
if (firstPass == null) {
return secondPass;
}
return new ChainedMiniPass(firstPass, secondPass);
}

@Override
public MiniIRPass prepare(IR parent, Expression current) {
var first = firstPass.prepare(parent, current);
var second = secondPass.prepare(parent, current);
if (first == firstPass && second == secondPass) {
return this;
} else {
return new ChainedMiniPass(first, second);
}
}

@Override
public Expression transformExpression(Expression ir) {
var transformedIr = firstPass.transformExpression(ir);
return secondPass.transformExpression(transformedIr);
}

@Override
public boolean checkPostCondition(IR ir) {
return firstPass.checkPostCondition(ir) && secondPass.checkPostCondition(ir);
}

@Override
public String toString() {
return Objects.toString(firstPass) + ":" + Objects.toString(secondPass);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
package org.enso.compiler.pass;

import java.util.function.Function;
import org.enso.compiler.core.IR;
import org.enso.compiler.core.ir.Expression;
import org.enso.compiler.core.ir.Module;
import org.enso.compiler.core.ir.ProcessingPass;

/**
* Mini IR pass operates on a single IR element at a time. The {@link org.enso.compiler.Compiler}
* traverses the whole IR tree in DFS. It works in two phases.
*
* <p><b>Note</b> that the current implementation is limited to traverse only {@link
* org.enso.compiler.core.ir.Expression} elements. This is a technical limitation because of the
* implementation of traversal via {@link IR#mapExpressions(Function)}. Hence, the additional
* methods {@link #prepareForModule(Module)} and {@link #transformModule(Module)}.
*
* <p>In the first, <b>prepare</b> phase, the compiler traverses from the root to the leaves and
* calls the {@link #prepare(Expression)} method on the mini pass. During this phase, the mini pass
* can gather information about the current IR element, but not modify it.
*
* <p>In the second, <b>transform</b> phase, the compiler returns from the leaves to the root and
* calls the {@link #transformExpression(Expression)} method on the mini pass. During this phase,
* the mini pass is free to transform the current IR element. The children of the current IR element
* are already transformed.
*
* <p>For each IR element:
*
* <ol>
* <li>The {@link #prepare(Expression)} method is called to prepare the pass for the current IR
* element. This method is called when the {@link org.enso.compiler.Compiler} traverses the IR
* tree from top to bottom. This is useful for mini passes that need to build some information
* about the current IR element before transforming it. The mini pass must not modify the IR
* element neither attach any metadata to it in this method. By returning {@code null} from
* this method, the mini pass signals to the compiler that it wishes to not process the
* subtree of the current IR element.
* <li>The {@link #transformExpression(Expression)} method is called to transform the current IR
* element. This method is called when the {@link org.enso.compiler.Compiler} traverses the
* element from bottom to top. All the children of the current IR element are already
* transformed when this method is called.
* </ol>
*
* <p>Inspired by: <a href="https://dl.acm.org/doi/10.1145/3140587.3062346">Miniphases: compilation
* using modular and efficient tree transformations</a>. PDF available at <a
* href="https://infoscience.epfl.ch/server/api/core/bitstreams/8ab72c0a-8aa6-4dee-a704-3504938dc316/content">infoscience.epfl.ch</a>
*/
public abstract class MiniIRPass implements ProcessingPass {
/**
* Prepare the pass for the provided IR element. This method is called when the {@link
* org.enso.compiler.Compiler} traverses the IR element from top to bottom.
*
* <p>The mini pass is free to gather any information about the elements it encounters (via this
* method) and use it in the {@link #transformExpression(Expression)} method. Note however, that
* it is not wise to store the references to the IR or their children for later comparison in the
* {@link #transformExpression(Expression) transform phase}, as the IR tree will most likely be
* transformed during the compilation process.
*
* <p>TL;DR; Do no store references to the IR elements or their children in this method.
*
* @param parent the the parent of the edge
* @param child the child expression element to be be processed.
* @return an instance of the pass to process the child's element subtree
*/
public MiniIRPass prepare(IR parent, Expression child) {
return this;
}

/**
* Transform the provided IR element. Children of the IR element are already transformed when this
* method is called. This method is called when the {@link org.enso.compiler.Compiler} traverses
* the IR element from bottom to top.
*
* <p>The pass should not do any traversal in this method.
*
* @param expr Expression IR element to be transformed by this pass.
* @return The transformed Expression IR, or the same IR if no transformation is needed. Must not
* return null.
*/
public abstract Expression transformExpression(Expression expr);

/**
* Transforms the module IR. This is the last method that is called.
*
* @see #transformExpression(Expression)
*/
public Module transformModule(Module moduleIr) {
return moduleIr;
}

public boolean checkPostCondition(IR ir) {
return true;
}

/**
* Name of the mini pass.
*
* @return by default it returns name of the implementing class
*/
@Override
public String toString() {
return getClass().getName();
}

/**
* Combines two mini IR passes into one that delegates to both of them.
*
* @param first first mini pass (can be {@code null})
* @param second second mini pass
* @return a combined pass that calls both non-{@code null} of the provided passes
*/
public static MiniIRPass combine(MiniIRPass first, MiniIRPass second) {
return ChainedMiniPass.chain(first, second);
}

/**
* Takes an IR element of given type {@code irType} and transforms it by provided {@link
* MiniIRPass}. When assertions are on, the resulting IR is checked with {@link
* #checkPostCondition} method of provided {@code miniPass}.
*
* @param <T> the in and out type of IR
* @param irType class of the requested IR type
* @param ir the IR element (not {@code null})
* @param miniPass the pass to apply
* @return the transformed IR
*/
public static <T extends IR> T compile(Class<T> irType, T ir, MiniIRPass miniPass) {
var newIr = MiniPassTraverser.compileDeep(ir, miniPass);
assert irType.isInstance(newIr)
: "Expected "
+ irType.getName()
+ " but got "
+ newIr.getClass().getName()
+ " by "
+ miniPass;
assert miniPass.checkPostCondition(newIr) : "Post condition failed for " + miniPass;
return irType.cast(newIr);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
package org.enso.compiler.pass;

import org.enso.compiler.context.InlineContext;
import org.enso.compiler.context.ModuleContext;

/**
* A collection of factory methods for {@link MiniIRPass IR mini passes}. If a mini pass supports
* only inline compilation, its {@link #createForModuleCompilation(ModuleContext)} method should
* return null.
*/
public interface MiniPassFactory {
/**
* Creates an instance of mini pass that is capable of transforming IR elements in the context of
* a module.
*
* @param moduleContext A mini pass can optionally save reference to this module context.
* @return May return null if module compilation is not supported.
*/
MiniIRPass createForModuleCompilation(ModuleContext moduleContext);

/**
* Creates an instance of mini pass that is capable of transforming IR elements in the context of
* an inline compilation.
*
* @param inlineContext A mini pass can optionally save reference to this inline context.
* @return Must not return null. Inline compilation should always be supported.
*/
MiniIRPass createForInlineCompilation(InlineContext inlineContext);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
package org.enso.compiler.pass;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.LinkedList;
import java.util.List;
import org.enso.compiler.core.IR;
import org.enso.compiler.core.ir.Expression;
import org.enso.compiler.core.ir.Module;

/** Implementation of {@link MiniIRPass#compile}. */
final class MiniPassTraverser {
private final MiniIRPass miniPass;
private List<IR> in;
private final List<IR> out;
private int outIndex;

private MiniPassTraverser(MiniIRPass miniPass, List<IR> out, int outIndex) {
this.miniPass = miniPass;
this.out = out;
this.outIndex = outIndex;
}

private boolean enqueue(Collection<MiniPassTraverser> queue) {
if (in == null) {
var ir = out.get(outIndex);
in = enqueueSubExpressions(queue, ir, miniPass);
return !in.isEmpty();
} else {
return false;
}
}

private void convertExpression() {
if (outIndex != -1) {
var oldIr = out.get(outIndex);
var index = new int[1];
var newIr = oldIr.mapExpressions((old) -> (Expression) in.get(index[0]++));
var transformedIr =
switch (newIr) {
case Module m -> miniPass.transformModule(m);
case Expression e -> miniPass.transformExpression(e);
default -> throw new IllegalArgumentException("" + oldIr);
};
if (oldIr != transformedIr) {
out.set(outIndex, transformedIr);
}
outIndex = -1;
}
}

static IR compileDeep(IR root, MiniIRPass miniPass) {
var result = new IR[] {root};
var rootTask = new MiniPassTraverser(miniPass, Arrays.asList(result), 0);
var stackOfPendingIrs = new LinkedList<MiniPassTraverser>();
stackOfPendingIrs.add(rootTask);
while (!stackOfPendingIrs.isEmpty()) {
if (stackOfPendingIrs.peekLast().enqueue(stackOfPendingIrs)) {
// continue descent
continue;
}
var deepestIr = stackOfPendingIrs.removeLast();
deepestIr.convertExpression();
}
assert result[0] != null;
return result[0];
}

/**
* @param queue queue to put objects in
* @param ir IR to process
* @param miniPass process with this mini pass
* @return {@code true} if the has been modified with new tries to process first
*/
private static List<IR> enqueueSubExpressions(
Collection<MiniPassTraverser> queue, IR ir, MiniIRPass miniPass) {
var childExpressions = new ArrayList<IR>();
var i = new int[1];
ir.mapExpressions(
(ch) -> {
var preparedMiniPass = miniPass.prepare(ir, ch);
childExpressions.add(ch);
queue.add(new MiniPassTraverser(preparedMiniPass, childExpressions, i[0]++));
return ch;
});
return childExpressions;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
package org.enso.compiler.pass.desugar;

import org.enso.compiler.core.IR;
import org.enso.compiler.core.ir.CallArgument;
import org.enso.compiler.core.ir.Expression;
import org.enso.compiler.core.ir.expression.Application;
import org.enso.compiler.core.ir.expression.Operator;
import org.enso.compiler.pass.MiniIRPass;
import scala.collection.mutable.ListBuffer;

public class OperatorToFunctionMini extends MiniIRPass {
OperatorToFunctionMini() {}

@Override
public Expression transformExpression(Expression ir) {
if (ir instanceof Operator.Binary binOp) {
ListBuffer<CallArgument> args = new ListBuffer<>();
args.addOne(binOp.left());
args.addOne(binOp.right());
return new Application.Prefix(
binOp.operator(),
args.toList(),
false,
binOp.location().isDefined() ? binOp.location().get() : null,
binOp.passData(),
binOp.diagnostics());
}
return ir;
}

@Override
public boolean checkPostCondition(IR ir) {
boolean[] isChildOperator = {false};
ir.children()
.foreach(
child -> {
if (child instanceof Operator.Binary) {
isChildOperator[0] = true;
}
return null;
});
return !isChildOperator[0];
}
}
Loading
Loading