Skip to content

Commit

Permalink
Continue module parsing despite Jython parsing errors (#206)
Browse files Browse the repository at this point in the history
Let's see if we can still go through (similar to
[b3c28a8](#125)).

## Differences
1. Log a warning instead of throwing an exception, which stops the
module from processing. We are already doing this further up the stack
anyway:
https://github.com/wala/ML/blob/ef1cca799d7726bfc70cec7b838ebeec2516d5e4/com.ibm.wala.cast.python/source/com/ibm/wala/cast/python/client/PythonAnalysisEngine.java#L159-L166
2. Continue to convert to CAst despite the parsing error coming out of
Jython. This is a workaround for #205.
I think this should be OK (best-effort) as I don't see it contributing
to any soundness issues. And, from what I am seeing in the
[logs](https://github.com/ponder-lab/ML/actions/runs/9897547494/job/27342295209#step:12:32780),
the parsing errors are "mild."
  • Loading branch information
khatchad authored Jul 12, 2024
1 parent ef1cca7 commit cbd558e
Showing 1 changed file with 7 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
import static com.ibm.wala.cast.python.util.Util.DYNAMIC_ANNOTATION_KEY;
import static com.ibm.wala.cast.python.util.Util.STATIC_METHOD_ANNOTATION_NAME;
import static com.ibm.wala.cast.python.util.Util.getNameStream;
import static java.util.logging.Logger.getLogger;

import com.ibm.wala.cast.ir.translator.AbstractClassEntity;
import com.ibm.wala.cast.ir.translator.AbstractCodeEntity;
Expand Down Expand Up @@ -60,6 +61,7 @@
import java.util.LinkedList;
import java.util.Map;
import java.util.function.Supplier;
import java.util.logging.Logger;
import org.python.antlr.PythonTree;
import org.python.antlr.ast.Assert;
import org.python.antlr.ast.Assign;
Expand Down Expand Up @@ -141,6 +143,8 @@

public abstract class PythonParser<T> extends AbstractParser<T> implements TranslatorToCAst {

private static final Logger LOGGER = getLogger(PythonParser.class.getName());

private static boolean COMPREHENSION_IR = true;

private CAstType codeBody =
Expand Down Expand Up @@ -2376,7 +2380,9 @@ public String getMsg() {
}
});
});
throw new TranslatorToCAst.Error(warnings);

// Log the parsing errors (best-effort).
warnings.forEach(w -> LOGGER.warning(() -> "Encountered parsing problem: " + w));
}

try {
Expand Down

0 comments on commit cbd558e

Please sign in to comment.