Skip to content

Commit

Permalink
Rename sympy/symengine switching from "backend" to "symbolic API"
Browse files Browse the repository at this point in the history
Because we are calling the codegen langauge targets "backends", it would be confusing to keep
doing so with the SymPy / SymEngine switch. Backend matches LLVM terminology, so it makes sense
there. We switch to using symbolic_api for the latter, which has some downsides in that we're
setting the implementation and not the API, but it seems like the best choice of the options
we've discussed.
  • Loading branch information
hayk-skydio committed Jun 11, 2022
1 parent a56ade9 commit 5874834
Show file tree
Hide file tree
Showing 27 changed files with 80 additions and 85 deletions.
8 changes: 4 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -86,16 +86,16 @@ TEST_CMD=-m unittest discover -s test/ -p *_test.py -v
GEN_FILES=test/*codegen*.py

test_symengine:
$(TEST_ENV) SYMFORCE_BACKEND=symengine $(PYTHON) $(TEST_CMD)
$(TEST_ENV) SYMFORCE_SYMBOLIC_API=symengine $(PYTHON) $(TEST_CMD)

test_sympy:
$(TEST_ENV) SYMFORCE_BACKEND=sympy $(PYTHON) $(TEST_CMD)
$(TEST_ENV) SYMFORCE_SYMBOLIC_API=sympy $(PYTHON) $(TEST_CMD)

test: test_symengine test_sympy

# Generic target to run a SymEngine codegen test with --update
update_%:
$(TEST_ENV) SYMFORCE_BACKEND=symengine $(PYTHON) test/$*.py --update
$(TEST_ENV) SYMFORCE_SYMBOLIC_API=symengine $(PYTHON) test/$*.py --update

# All SymForce codegen tests, formatted as update_my_codegen_test targets
GEN_FILES_UPDATE_TARGETS=$(shell \
Expand All @@ -106,7 +106,7 @@ test_update: $(GEN_FILES_UPDATE_TARGETS)

# Generic target to run a SymPy codegen test with --update --run_slow_tests
sympy_update_%:
$(TEST_ENV) SYMFORCE_BACKEND=sympy $(PYTHON) test/$*.py --update --run_slow_tests
$(TEST_ENV) SYMFORCE_SYMBOLIC_API=sympy $(PYTHON) test/$*.py --update --run_slow_tests

# All SymForce codegen tests, formatted as sympy_update_my_codegen_test targets
GEN_FILES_SYMPY_UPDATE_TARGETS=$(shell \
Expand Down
6 changes: 3 additions & 3 deletions docs/development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -95,11 +95,11 @@ Much of the core functionality of SymForce is in generating code using the `Jinj
For example template files, see ``symforce/codegen/cpp_templates``.

*************************************************
Symbolic Backends
Symbolic API
*************************************************
SymForce uses the `SymPy <https://www.sympy.org/en/index.html>`_ API, but supports two backend implementations of it. The SymPy backend is pure Python, whereas the `SymEngine <https://github.com/symengine/symengine>`_ backend is wrapped C++. It can be 100-200 times faster for many operations, but is less fully featured and requires a C++ build.
SymForce uses the `SymPy <https://www.sympy.org/en/index.html>`_ API, but supports two implementations of it. The SymPy implementation is pure Python, whereas the `SymEngine <https://github.com/symengine/symengine>`_ implementation is wrapped C++. It can be 100-200 times faster for many operations, but is less fully featured and requires a C++ build.

To set the backend, you can either use :func:`symforce.set_backend()` before any other imports, or use the ``SYMFORCE_BACKEND`` environment variable with the options ``sympy`` or ``symengine``. By default SymEngine will be used if found, otherwise SymPy.
To set the symbolic API, you can either use :func:`symforce.set_symbolic_api()` before any other imports, or use the ``SYMFORCE_SYMBOLIC_API`` environment variable with the options ``sympy`` or ``symengine``. By default SymEngine will be used if found, otherwise SymPy.

*************************************************
Building wheels
Expand Down
2 changes: 1 addition & 1 deletion notebooks/storage_D_tangent.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"source": [
"import symforce\n",
"\n",
"symforce.set_backend(\"sympy\")\n",
"symforce.set_symbolic_api(\"sympy\")\n",
"symforce.set_log_level(\"warning\")\n",
"\n",
"from symforce import geo\n",
Expand Down
2 changes: 1 addition & 1 deletion notebooks/symbolic_computation_speedups.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
"# Setup\n",
"import symforce\n",
"\n",
"symforce.set_backend(\"sympy\")\n",
"symforce.set_symbolic_api(\"sympy\")\n",
"\n",
"from symforce import geo\n",
"from symforce import sympy as sm\n",
Expand Down
2 changes: 1 addition & 1 deletion notebooks/tangent_D_storage.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"source": [
"import symforce\n",
"\n",
"symforce.set_backend(\"sympy\")\n",
"symforce.set_symbolic_api(\"sympy\")\n",
"symforce.set_log_level(\"warning\")\n",
"\n",
"from symforce import geo\n",
Expand Down
2 changes: 1 addition & 1 deletion notebooks/tutorials/cameras_tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
"# Setup\n",
"import symforce\n",
"\n",
"symforce.set_backend(\"sympy\")\n",
"symforce.set_symbolic_api(\"sympy\")\n",
"symforce.set_log_level(\"warning\")\n",
"\n",
"from symforce.notebook_util import display\n",
Expand Down
2 changes: 1 addition & 1 deletion notebooks/tutorials/codegen_tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
"import os\n",
"import symforce\n",
"\n",
"symforce.set_backend(\"symengine\")\n",
"symforce.set_symbolic_api(\"symengine\")\n",
"symforce.set_log_level(\"warning\")\n",
"\n",
"from symforce import codegen\n",
Expand Down
2 changes: 1 addition & 1 deletion notebooks/tutorials/geometry_tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
"# Setup\n",
"import symforce\n",
"\n",
"symforce.set_backend(\"sympy\")\n",
"symforce.set_symbolic_api(\"sympy\")\n",
"symforce.set_log_level(\"warning\")\n",
"\n",
"from symforce.notebook_util import display\n",
Expand Down
2 changes: 1 addition & 1 deletion notebooks/tutorials/ops_tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
"# Setup\n",
"import symforce\n",
"\n",
"symforce.set_backend(\"sympy\")\n",
"symforce.set_symbolic_api(\"sympy\")\n",
"symforce.set_log_level(\"warning\")\n",
"\n",
"from symforce.notebook_util import display\n",
Expand Down
4 changes: 2 additions & 2 deletions notebooks/tutorials/sympy_tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"# Configuration (optional)\n",
"import symforce\n",
"\n",
"symforce.set_backend(\"sympy\")\n",
"symforce.set_symbolic_api(\"sympy\")\n",
"symforce.set_log_level(\"warning\")\n",
"from symforce.notebook_util import display, print_expression_tree"
]
Expand All @@ -33,7 +33,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Always import the SymPy API through SymForce, because symforce can switch out the backend implementation of the API and adds a few minor but important augmentations. Let's define some algebraic symbols:"
"Always import the SymPy API through SymForce, because symforce can switch out the symbolic implementation of the API and adds a few minor but important augmentations. Let's define some algebraic symbols:"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion notebooks/tutorials/values_tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
"# Setup\n",
"import symforce\n",
"\n",
"symforce.set_backend(\"sympy\")\n",
"symforce.set_symbolic_api(\"sympy\")\n",
"symforce.set_log_level(\"warning\")\n",
"\n",
"from symforce import geo\n",
Expand Down
48 changes: 24 additions & 24 deletions symforce/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,18 +61,18 @@ def set_log_level(log_level: str) -> None:
set_log_level(os.environ.get("SYMFORCE_LOGLEVEL", "WARNING"))

# -------------------------------------------------------------------------------------------------
# Symbolic backend configuration
# Symbolic API configuration
# -------------------------------------------------------------------------------------------------
sympy: T.Any = None

from . import initialization


def _set_backend(sympy_module: ModuleType) -> None:
def _set_symbolic_api(sympy_module: ModuleType) -> None:
# Make symforce-specific modifications to the sympy API
initialization.modify_symbolic_api(sympy_module)

# Set this as the default backend
# Set this as the default symbolic API
global sympy # pylint: disable=global-statement
sympy = sympy_module

Expand Down Expand Up @@ -143,13 +143,13 @@ def _use_symengine() -> None:
logger.critical("Commanded to use symengine, but failed to import.")
raise

_set_backend(symengine)
_set_symbolic_api(symengine)


def _use_sympy() -> None:
import sympy as sympy_py

_set_backend(sympy_py)
_set_symbolic_api(sympy_py)
sympy_py.init_printing()


Expand All @@ -171,54 +171,54 @@ def set_symengine_eval_on_sympify(eval_on_sympy: bool = True) -> None:
logger.debug("set_symengine_fast_sympify has no effect when not using symengine")


def set_backend(backend: str) -> None:
def set_symbolic_api(name: str) -> None:
"""
Set the symbolic backend for symforce. The sympy backend is the default and pure python,
whereas the symengine backend is C++ and requires building the symengine library. It can
Set the symbolic API for symforce. The sympy API is the default and pure python,
whereas the symengine API is C++ and requires building the symengine library. It can
be 100-200 times faster for many operations, but is less fully featured.
The default is symengine if available else sympy, but can be set by one of:
1) The SYMFORCE_BACKEND environment variable
1) The SYMFORCE_SYMBOLIC_API environment variable
2) Calling this function before any other symforce imports
Args:
backend (str): {sympy, symengine}
name (str): {sympy, symengine}
"""
# TODO(hayk): Could do a better job of checking what's imported and raising an error
# if this isn't the first thing imported/called from symforce.

if sympy and backend == sympy.__package__:
logger.debug(f'already on backend "{backend}"')
if sympy and name == sympy.__package__:
logger.debug(f'already on symbolic API "{name}"')
return
else:
logger.debug(f'backend: "{backend}"')
logger.debug(f'symbolic API: "{name}"')

if backend == "sympy":
if name == "sympy":
_use_sympy()
elif backend == "symengine":
elif name == "symengine":
_use_symengine()
else:
raise NotImplementedError(f'Unknown backend: "{backend}"')
raise NotImplementedError(f'Unknown symbolic API: "{name}"')


# Set default to symengine if available, else sympy
if "SYMFORCE_BACKEND" in os.environ:
set_backend(os.environ["SYMFORCE_BACKEND"])
if "SYMFORCE_SYMBOLIC_API" in os.environ:
set_symbolic_api(os.environ["SYMFORCE_SYMBOLIC_API"])
else:
try:
symengine = _import_symengine_from_build()

logger.debug("No SYMFORCE_BACKEND set, found and using symengine.")
set_backend("symengine")
logger.debug("No SYMFORCE_SYMBOLIC_API set, found and using symengine.")
set_symbolic_api("symengine")
except ImportError:
logger.debug("No SYMFORCE_BACKEND set, no symengine found, using sympy.")
set_backend("sympy")
logger.debug("No SYMFORCE_SYMBOLIC_API set, no symengine found, using sympy.")
set_symbolic_api("sympy")


def get_backend() -> str:
def get_symbolic_api() -> str:
"""
Return the current backend as a string.
Return the current symbolic API as a string.
Returns:
str:
Expand Down
4 changes: 2 additions & 2 deletions symforce/codegen/codegen_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -240,8 +240,8 @@ def tmp_symbols() -> T.Iterable[str]:
yield sm.Symbol(f"_tmp{i}")

if cse_optimizations is not None:
if symforce.get_backend() == "symengine":
raise ValueError("cse_optimizations is not supported on the symengine backend")
if symforce.get_symbolic_api() == "symengine":
raise ValueError("cse_optimizations is not supported on symengine")

temps, flat_simplified_outputs = sm.cse(
flat_output_exprs, symbols=tmp_symbols(), optimizations=cse_optimizations
Expand Down
7 changes: 3 additions & 4 deletions symforce/databuffer.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,9 @@

class DataBuffer(sympy.MatrixSymbol):
"""
Custom class for when using the sympy backend to make MatrixSymbol consistent with
symforce's custom 1-D Databuffer symengine
We want to force Databuffers to be 1-D since otherwise CSE will (rightfully) treat each index
as a separate expression.
Custom class to make sympy's MatrixSymbol consistent with symengine, where we have a custom
1-D Databuffer. We want to force Databuffers to be 1-D since otherwise CSE will (rightfully)
treat each index as a separate expression.
"""

# HACK(harrison): needed to get around the flast that DataBuffer needs to be called from
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@
from symforce import sympy as sm
from symforce import typing as T

if symforce.get_backend() != "symengine":
logger.warning("The 3D Localization example is very slow on the sympy backend")
if symforce.get_symbolic_api() != "symengine":
logger.warning("The 3D Localization example is very slow on sympy. Use symengine.")

NUM_POSES = 5
NUM_LANDMARKS = 20
Expand Down
10 changes: 5 additions & 5 deletions symforce/initialization.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ def new_symbol(

def override_simplify(sympy_module: T.Type) -> None:
"""
Override simplify so that we can use it with the symengine backend
Override simplify so that we can use it with symengine.
Args:
sympy_module (module):
Expand All @@ -112,7 +112,7 @@ def simplify(*args: T.Any, **kwargs: T.Any) -> sympy.Basic:

def override_limit(sympy_module: T.Type) -> None:
"""
Override limit so that we can use it with the symengine backend
Override limit so that we can use it with symengine.
Args:
sympy_module (module):
Expand Down Expand Up @@ -250,7 +250,7 @@ def override_subs(sympy_module: T.Type) -> None:
self, _get_subs_dict(*args, **kwargs), **kwargs
)
else:
raise NotImplementedError(f"Unknown backend: '{sympy_module.__name__}'")
raise NotImplementedError(f"Unknown symbolic API: '{sympy_module.__name__}'")


def override_solve(sympy_module: T.Type) -> None:
Expand All @@ -274,15 +274,15 @@ def solve(*args: T.Any, **kwargs: T.Any) -> T.List[T.Scalar]:
return []
else:
raise NotImplementedError(
f"sm.solve currently only supports FiniteSet and EmptySet results on the SymEngine backend, got {type(solution)} instead"
f"sm.solve currently only supports FiniteSet and EmptySet results on SymEngine, got {type(solution)} instead"
)

sympy_module.solve = solve
elif sympy_module.__name__ == "sympy":
# This one is fine as is
return
else:
raise NotImplementedError(f"Unknown backend: '{sympy_module.__name__}'")
raise NotImplementedError(f"Unknown symbolic API: '{sympy_module.__name__}'")


def override_count_ops(sympy_module: T.Type) -> None:
Expand Down
2 changes: 1 addition & 1 deletion symforce/notebook_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ def display(*args: T.Any) -> None:
"""
Display the given expressions in latex, or print if not an expression.
"""
if symforce.get_backend() == "sympy":
if symforce.get_symbolic_api() == "sympy":
IPython.display.display(*args)
return

Expand Down
4 changes: 2 additions & 2 deletions symforce/ops/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ class ScalarExpr(abc.ABC):
"""
Metaclass for scalar expressions
DataBuffer is a subclass of sm.Expr in both backends, but we do not want it to be registered
under ScalarLieGroupOps.
DataBuffer is a subclass of sm.Expr but we do not want it to be registered under
ScalarLieGroupOps.
"""

@abc.abstractmethod
Expand Down
8 changes: 4 additions & 4 deletions symforce/test_util/epsilon_handling.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ def is_value_with_epsilon_correct(
"""

# Converting between SymPy and SymEngine breaks substitution afterwards, so we require
# running with the SymPy backend
assert symforce.get_backend() == "sympy"
# running with SymPy.
assert symforce.get_symbolic_api() == "sympy"

# Create symbols
x = sm.Symbol("x", real=True)
Expand Down Expand Up @@ -115,8 +115,8 @@ def is_derivative_with_epsilon_correct(
"""

# Converting between SymPy and SymEngine breaks substitution afterwards, so we require
# running with the SymPy backend
assert symforce.get_backend() == "sympy"
# running with SymPy.
assert symforce.get_symbolic_api() == "sympy"

# Create symbols
x = sm.Symbol("x", real=True)
Expand Down
Loading

0 comments on commit 5874834

Please sign in to comment.