Skip to content

Releases: symforce-org/symforce

v0.9.0

08 Jun 14:37
Compare
Choose a tag to compare

Full Changelog: v0.8.0...v0.9.0

New in this release, we’ve added an experimental dense optimizer, greatly improved the docs, and made some additions to the core symbolic API and codegen API. We also have lots of under-the-hood changes, to the optimizer and the build in particular. Keep reading for more details, and thanks to everyone who contributed!

🚨 Breaking Changes

We're removing a few things that have been marked as deprecated for several releases now. We're also switching our C++ API to use references for mandatory input-output arguments, and deprecating the versions that used pointers, for consistency with our styleguide and modern best practices. And finally, we've removed the early_exited field from the OptimizationStats in favor of a more comprehensive description of exit status.

🧠 Optimizer Changes

SymForce now has a dense optimizer! This is designed for problems where the problem linearization is dense enough that it's better to perform a dense Cholesky decomposition rather than a sparse one. It's currently in an experimental state, you can find example usage here.

As part of this, we've also made several changes to make it easier to swap out different parts of the optimizer like this.

Plus, we now expose more information about the result of an optimization beyond whether it just exited early - you can find that in status and failure_reason in the OptimizationStats.

📚 Documentation Improvements

We've made multiple improvements to our docs for this release, both in readability and content. We have a new theme (with dark mode 😎 ), and the API docs now have things formatted and cross-linked correctly. Check them out at https://symforce.org!

🖊️ Codegen Improvements

We've made improvements and additions to codegen usage, and to the Python and PyTorch backends. In particular, we've added lambdify and numbify functions, to easily convert symbolic functions to numerical ones for fast evaluation in Python. For example,

def foo(x: sf.Scalar, R: sf.Rot3) -> sf.Scalar:
    return (R * sf.V3(x, 0, 0)).x

foo_numeric = symforce.util.lambdify(foo)

foo_numeric(5.0, sym.Rot3())

🔣 Symbolic API Improvements

⚙️ Buildsystem / Infra Changes

👮 SkyMarshal

✨ Misc

v0.8.0

07 Feb 19:03
Compare
Choose a tag to compare

Full Changelog: v0.7.0...v0.8.0

This is probably our biggest release yet, with multiple major additions - we have built-in IMU Preintegration, new backends for CUDA and PyTorch, and a new symbolic type for unit vectors on S(2). Plus, we've made major improvements to memory usage in the C++ optimizer, made lots of additions to the symbolic API, and made plenty of improvements under the hood.

🚨 Breaking Changes

  • ae43476 Slices of 1D matrices are matrices, not lists - taking a 1D slice of a geo.Matrix now returns a geo.Matrix, previously it would return a list.
  • f60180e Add include_jacobians flag - by default, the Optimizer will now not allocate or populate the problem jacobian, since it is not needed for optimization. If you'd like it to be filled out, set include_jacobians to true.
  • There are some simplifications to the sym::Factor API, described in Code Cleanup / Refactoring below. Most users should be unaffected.
  • Generated Python functions return vectors as 1D ndarrays - see the description below

📈 IMU Factor

SymForce has IMU Preintegration! We have a C++ ImuFactor with preintegration and linearization functions generated from a symbolic implementation. See the PRs #279 and #280 for more information, and see imu_factor.h for an example.

🦾 CUDA Backend

The CUDA backend generates __host__ __device__ functions, i.e. functions that you can call from within a CUDA kernel. It's currently in an experimental state, and supports scalars, vectors/matrices, and sf.DataBuffer in the APIs of generated functions; other types like geo and cam objects can be passed via their storage vectors. See the commit for more examples.

🔥 PyTorch Backend

The PyTorch backend generates python functions that operate on PyTorch tensors, with each symbolic op becoming one PyTorch op. It supports broadcasting these operations across multiple optional batch dimensions. Like the CUDA backend, this is currently experimental - it supports scalars and vectors/matrices in generated functions APIs, and is not particularly optimized for performance.

⚪ geo.Unit3

The Unit3 symbolic type lets you represent unit vectors in 3D, including optimizing unit vector variables and performing manifold operations on them.

➡️ Generated Python functions return vectors as 1D ndarrays

By default, generated python functions will now return 1d ndarrays for outputs that are row or column vectors (including 1x1 matrices). This includes member functions of generated python types, such as sym.Pose3.position(). See the commit for more information.

💨 Optimizer Performance/API Improvements

Major improvements to memory usage in the C++ optimizer, especially for large problems like bundle adjustment that have lots of small factors. Also, converting a sym::Values to LCM is significantly faster.

🔣 Symbolic API additions

🧹 Code Cleanup / Refactoring

⚙️ Infra / Buildsystem changes

👮 SkyMarshal

v0.7.0

22 Nov 00:55
Compare
Choose a tag to compare

Full Changelog: v0.6.0...v0.7.0

🚨 Breaking Changes

sf.Pose2_SE2 and sf.Pose3_SE3 are now in geo.unsupported (also accessible through sf.unsupported). We haven't removed any functionality, just moved them to clarify that they are not intended to be supported by much of SymForce and we do not intend to add more support for them. The sf.Pose2 and sf.Pose3 classes should be used instead, since they use more efficient representations.

🐍 Python 3.11 Support

SymForce now supports Python 3.11! As of this release you should be able to build symforce from source or install it with pip on Python 3.11.

🗒️ Documentation and Examples Improvements

📉 Optimizer Improvements

🛠️ Buildsystem / Infrastructure / Testing Improvements

👮 SkyMarshal

SkyMarshal C++ bindings can now serialize and deserialize messages larger than 2GiB

✨ Misc

v0.6.0

30 Sep 02:25
Compare
Choose a tag to compare

History: v0.5.0...v0.6.0

🔣 Default Epsilon Configuration

Building on the tools from v0.5 for enabling configurable default epsilons for library functions, add checks to enable confidently generating code without worrying about forgetting to pass an epsilon somewhere, which would previously result in an unsafe generated function. Generating a function without calling symforce.set_epsilon_to_symbol() or symforce.set_epsilon_to_number() will produce a warning, since this allows unintentionally calling functions with singularities with an epsilon of 0. Similarly, setting epsilon to a symbol, but then attemptting to generate a function that doesn't take an argument for epsilon, will yield a descriptive error message. This behavior is also configurable through new additions to the CodegenConfig. The error messages are intended to have thorough explanations, so let us know if they don't make sense by creating an issue!

  • 559e521 Set the default epsilon to a symbol for codegen
  • f17d8b2 Run python tests with ctest
  • 713a00a Add sym.epsilon

➕ Call User-Defined Functions from Generated C++

The code generator supports replacing builtin functions with user-provided alternatives. For instance, a custom my_lib::fast_sin function can be replaced for std::sin. The CppConfig has a new argument override_methods which can be used to specify these functions.

Shoutout to Roy Vorster for contributing this!

  • 7f9cf25 Add function specific overwrites to sympy CPP printer

🏭 More methods on generated types

Add more methods to bring generated Python types up to parity with generated C++ types. In addition, automatically generate some previously handwritten methods.

  • 48f6968 Codegen PoseX.position and rotation
  • 90f14a0 Generate handwritten sym geo methods
  • 81fa502 Uniform position/rotation accessors
  • 1e95863 Generate to_homogenous_matrix for pose types
  • 8e2b19c Codegen ToYawPitchRoll for C++ Rot3

🐍 Generated Python functions are more rigorous about Matrix shapes

PythonConfig supports options for whether generated Python functions should expect all matrix inputs to be 2D arrays, or allow vectors to be 1D arrays. This fixes the previous unintuitive behavior of expecting storage arrays.

  • c163cb6 Reshape matrix inputs to be rank 2
  • 652eb97 Add matrix accessor format function to config and remove matrix_is_1d param

🔧 CI/CD

Add a continuous testing suite on main and on pull requests across compilers and Python versions. In addition, add automatic deployment of the symforce docs to https://symforce.org.

  • ff5a34b Rerun tests with output on failure
  • 716befd Add docs as an artifact of CI runs
  • 6dd9f9a Run CI on push to main and build badge
  • d9633f3 Preserve downloads symforce-org.github.io
  • de296a8 Auto-deploy docs on push to main
  • 20548c3 Add SymForce CI

🧹 Code Quality Improvements

Various internal improvements, fixing C++ build warnings, sorting imports, and refactoring internal modules

  • d10ac9a Sort imports with isort
  • 6d1c918 Fix warnings
  • 322faba Sort some imports
  • b268447 Removed unused imports in symforce python function codegen template
  • c3d0a25 Move things to typing_util
  • 525f229 Misc changes to tests
  • a7999d6 TemplateListEntry -> dataclass
  • d154598 Modify optionality of template_dir arg
  • 3f467fa Make TemplateList.add arg order match
  • 429e6ac Take relative path in render_template

👮 SkyMarshal

  • 58e37ea [SkyMarshal] Sort imports
  • a815051 [SkyMarshal] static functions in lcm_coretypes are __attribute__((unused))
  • d2760d7 [SkyMarshal] Explicitly static_cast hashes to int64

🎁 Misc

  • 693f270 Clarify docstrings on SE(n) classes
  • da612a9 Update installation instructions
  • 8f01c0d Bump version to 0.6
  • a948a31 Add failing test for factor cache with custom jacobians
  • 9b3c47b More factor cache fixes
  • 48fd72d Use corrected runpaths w/ setup.py develop too
  • 29310df dev_requirements.txt errors on py < 3.8
  • d2181a8 Add note about building from a release
  • 94d1bfa SparseCholesky is MPL2
  • b3d6edb Fix of pip install -e . on both macos and ubuntu
  • e97a5fd Upgrade to sympy 1.11
  • 9a20a81 Fix factor cache
  • 48699d1 Add more named Matrix types
  • 47f4adb Fix values_tutorial link
  • 324bba2 LM Optimizer prints out id with iteration stats
  • 40a6c9e Pull in the latest tl::optional
  • b339396 Add numba to dev_requirements.txt
  • e2ed962 Symforce add sparse_linearization option to factor.py
  • 7289ce0 Don't add symengine to sys.modules if import fails
  • e131eae Use SYMFORCE_PYTHON for rerun_if_needed
  • 75201b6 Fix build on cmake >=3.23
  • d8e992f Fix broken links in README
  • 71d36b0 Raise ValueError if subs args are different lengths
  • 73d5dbe Symforce Values.from_storage allow different scalar types
  • 73b22a5 Allow symforce storage ops to return numeric types
  • d50dbba Fix docs for Group and LieGroup
  • eeadcee Added getter function for symforce optimizer params
  • cea8701 Move error message for wrong number of keys
  • 367eeb8 Added helper function for converting Symforce Values to dicts
  • 68384d0 Dataclass ops support T.Tuple
  • 2d1a615 Codegen performance improvements
  • e9ef402 LinearCameraCal doesn't take Abs(z)
  • 65f6d00 Use sf.sympy.Eq in symbolic_computation_speedups.ipynb
  • 52d2b5d Regenerate with make test_update_all
  • 87c031b Better errors from wrong factor results
  • c0933d0 Fix type annotation
  • f6496d6 Remove local Catch2 (instead fetch it)
  • cdf8560 fix bug w/ lcm gen for large matrix
  • d614535 Make cc_sym types pickleable

v0.5.0

29 Jun 03:36
Compare
Choose a tag to compare

History: v0.4.0...v0.5.0

🚨 Symbolic API moved to symforce.symbolic 🚨

The symbolic math components of SymForce are now unified under one import for ease of use:

import symforce.symbolic as sf

The symforce.symbolic module contains the SymForce-modified SymPy (or SymEngine) API, with our added logic functions and helper functions (i.e. things that were previously accessible from symforce.sympy), geometric lie group types (from symforce.geo), and camera types (from symforce.cam). If you just want the geo or cam types, those modules still exist and can be imported instead.

Usage of symforce.sympy is deprecated, you should use symforce.symbolic instead. It will be removed in a future release.

  • 46dc58b matrix.fixed_type_from_shape -> matrix_type_from_shape
  • b34909b Add deprecated symforce.sympy stub
  • 5e0afd5 Add T.Scalar to sf
  • a26a9d5 Add geo and cam to symforce.symbolic
  • 192f5cf Move sympy api to symforce.symbolic

🔣 Configurable default epsilons for library functions

SymForce now provides more flexibility around setting the default value of epsilon, its mechanism for handling singularity points.

Many library functions require an argument for epsilon, and previously defaulted it to 0 for ease of prototyping e.g. in a notebook. For runtime code, however, this can be a potential source of issues, since it can be easy to unintentionally forget to pass epsilon to one of these functions, meaning that an unhandled singularity creeps in. To address this, we have added a function sf.epsilon() which should be used as the default epsilon argument for library functions. The value that this returns is configurable - the default is 0 (so by default, behavior is the same). However, it can also be set to a symbol with sf.set_epsilon_to_symbol(), which is recommended for generating production code. Similarly, it can be set to a numerical value with sf.set_epsilon_to_number(). Either of these must be called before importing symforce.symbolic.

  • f63171c Fixed default epsilon
  • 7133947 Use sm.epsilon() as default eps
  • 8965840 sm.default_epsilon -> sm.numeric_epsiln

⚙️ Unified mechanism for adding code generation backends

We've unified much of the backend-specific differences, so that it should be easier to add new code generation backends going forward. See the PR and the example of adding a JavaScript backend for more information.

  • e4c2d19 Centralize code for creating new code generation backends

👮 SkyMarshal Updates

  • 90c4891 Enable skymarshal printing by default
  • 5fb13f7 Allow unknown notations in skymarshal
  • 06be0d8 [Skymarshal] Generate operator<< for all types

🛠️ Build System Fixes

  • 2e14e4d Add minimum spdlog version
  • fba4ae8 Workflow and build support for pip wheels
  • 7f59396 Bump default pybind to 2.9.2
  • ea82c15 Remove spdlog (and eigen and catch2) from the manifest
  • 7efa7cd Misc docs, CMake, and packaging fixes
  • 322c564 Upgrade pip requirements
  • c321ca0 Install eigen_lcm headers
  • aba8ef7 Fix install paths

🎁 Misc other changes

  • e7fea68 Fix spelling of word perturbation in lie-group docstrings
  • 7c778cc Use spdlog::set_level instead of sym::internal::SetLogLevel
  • 411c845 Retry symforce_docs_test 3 times on failure
  • 3915767 Copy robot_2d_localization example
  • 234f827 Make robot_3d_localization comment true
  • 41fbca0 Remove duplicate epsilon files
  • 7bee06e Change format_py_dir to take T.Openable
  • 5f4b9fc Emit a warning if optimizing factors that don't touch any keys
  • 84a1ca2 Call modify_symbolic_api once per sympy
  • cccb324 Fix up C++ example in README
  • 43c2dfd Fix derivatives of sm.Mod
  • 165b097 Fix plotting code for robot_3d_localization example
  • 9e08a78 fix factor_utils failing to compile on clang-6.0
  • 60c9ccf Ret Path (!str) in generate_lcm_types
  • bdd50e9 Enable sparse cc_sym.Factor creation

v0.4.0

24 May 13:14
Compare
Choose a tag to compare

v0.4.0 is the first public release of SymForce outside the early-access program, and the first release after the paper was accepted to RSS 2022. SymForce can now be pip installed on Linux and Mac with pre-built wheels.

History: v0.3.0...v0.4.0

✨ Support for sparse matrices and factors in Python

Previously, SymForce could generate C++ functions of sparse matrices (in Eigen) and use them in the C++ optimizer, but it could not generate Python functions of sparse matrices or use Python optimizer. This release adds support for generating Python functions that use SciPy's sparse matrix format, and also the ability to convert them to Eigen in the pybind wrapper, so they can be used with the optimizer from Python.

  • 77b8678 Add sym::Factor::SparseJacobianFunc
  • 3ec069c Calculate hessian & rhs in one place
  • eef1c9c Add sparse outputs to generate py code
  • 3d8cd89 Expose symbolic jacobians computed by factor
  • a43a296 Factor out py residual wrapper - cc_sym
  • 2f91fb2 Make sparse_matrix_init more readable
  • 357507e Replace Matrix arg with CSCFormat arg
  • d2b1042 Remove None check in C++ sparse mat gen
  • 4e2e586 More arguments passed through OptimizationProblem and Factor
  • ff6cd01 get_sparse_mat_data to CSCFormat method
  • 1fab87c Consistent Matrix storage order
  • e2fbce0 CSCFormat type instead of dict

🧊 Data Buffers

Added initial support for data buffers, which are external arrays of data passed in to symbolic expressions, for example lookup tables, images, or volumetric maps. Code generation is currently supported, but not optimization.

  • 2cd9dfe Initial work for values to support DataBuffer entries
  • 236e27f don't allow DataBuffer to be iterable
  • e0db744 Allow symengine to simplify DatabufferElement index expressions
  • a550ed0 fix bug w/ databuffer storage ops
  • 9f1345b databuffer tests
  • 981d48a implement databuffer

⏱️ Code Generation

Many improvements to code generation quality and speed for large problems.

  • 0c67bb8 Allow specific explicit template instantiations
  • 0f03029 Give generate_function return type
  • e7c1388 Convert matrix elements to float in to_numerical()
  • 9a1069e Fix return type: str -> sm.Symbol
  • ca55474 fix bug w/ symbol formatting in codegen_util
  • 33184aa Remove unused codegen_util code
  • aea7eb0 Rename output_terms to dense_terms
  • 2862b26 Remove dead jinja code
  • 7414072 Fixed codegen bug with dataclasses
  • a626932 Allow explicit template instantiation for C++ codegen
  • d094cda add an option to symengine wrapper to skip eval on sympy
  • 04c6274 Changed codegen folder names from python2.7 to python
  • 6aa7e0d add dont_flatten_args to _get_subs_dict
  • c1d482d Disable code format when generating temp factors
  • e654de2 move lazy imports outside for loop
  • e76c6d6 Fix SymforceCodegenTest doc-string
  • 015e968 Allow symforce to generate code from functions w/ dataclasses as args

👁️ Bundle Adjustment in the Large

Added an example that optimizes the datasets from Bundle Adjustment in the Large to parallel the GTSAM and Ceres examples. This is not yet a benchmark optimized for performance.

  • bad1035 Bundle-Adjustment-in-the-Large Example

🥇 Optimization

Improve the optimization library based on testing larger multi-objective problems.

  • 4540121 Fixed typo in optimizer
  • 304654e Explicit template intantiate Optimizer
  • af280a3 Added a type check on optimized values
  • 34e6c17 fix bug w/ custom jacobians
  • 46e8a0c Added some useful timestep objectives
  • 2c926e8 Added support for loading factors from files

🛠️ Build System

Support pip install symforce with pre-built wheels, and upgrade dependencies.

  • af687c1 Pin pip requirements
  • 92c30f7 Support pip install [-e] .
  • a3b2dc0 Bump skymarshal required cmake version to 3.19
  • 24e9601 Split CMakeLists
  • 3769c2d Upgrade sympy to version 1.10.1

✍🏼 Documentation

Improved the README, tutorials, and examples across the board to prepare for public release.

🎁 Miscellaneous

  • 78d7844 [Skymarshal] lextab updates
  • fb1bd27 Delete cholesky/assert
  • f127562 Autoformat with clang-format 14
  • 8591d8f Include cassert where we use it
  • 185be31 Simplify ArrayStorageOps
  • d32d14b Have ndarray ops use AbstractVectorOps
  • 359bf11 [Skymarshal] Optional printing
  • 5d52d76 Allow geo.M.setitem with np.float64
  • 97a47dc Assert geo.Matrix shape == arg shape
  • 69ea6d6 Diffs in test failures
  • cfc0b87 Remove #hashable from sym.key_t.

v0.3.0

01 Apr 21:27
e2ac9dc
Compare
Choose a tag to compare

📷 Support for CameraCal types in C++ Values and optimizations

You can now use camera calibration objects directly as inputs to Factors for optimization, either as optimized variables or as constants.

  • 1db2d85 camera cals in cc_sym.Values
  • 73af9cc Add camera cals to python sym
  • c230173 Auto-generate C++ cam cal constructors
  • 523b5a1 Rework camera cal constructors
  • f7987ee Add camera cal types to BY_TYPE_HELPER
  • b26be84 Define template specializations of [Lie]GroupOps for camera cals in C++
  • e43406e Register CameraCal with LieGroupOps in python
  • 2509e29 Add all the camera cals to types_t and define TypeEnum

🐍 More support for dataclasses

We recently added some support for using Dataclasses in various places, such as in Values and with Ops. We've now added more thorough support for them in more places:

  • dd017bf Fixed size dataclass ops with sequences
  • 184238d Added ops support for NoneTypes so that Dataclasses can have optional fields
  • 4a079ce Added Group and Lie Group Ops for Dataclasses
  • 05debd6 Added better support for indexing dataclasses stored in Values objects
  • db4585f Added clearer error message for dynamically created dataclasses stored in Values objects

🎲 Lots more examples and benchmarks

⛸️ std::complex in generated C++ code

Generated C++ functions can now handle complex scalars! If the support_complex flag is true, generated functions will be allowed to contain complex constants and otherwise work correctly if instantiated with Scalar = std::complex<T>.

  • 94d37e9 Add support for std::complex in C++ functions

⏱️ Code generation performance improvements

Some changes to speed up code generation by eliminating a lot of unnecessary work

  • c26926d [SymEngine] Add symbols argument to CSE
  • 45c4eb3 Cache generated residuals in Factor
  • 2294616 Cache jinja environment
  • d8563ed Speed up assert in Codegen construction
  • 6a4fb41 Don't autoformat python, for real
  • 3e13c3a Speed up symforce_py_optimizer_test.py
  • d907c37 Lazily print code in Codegen
  • 630e0d3 codegen exposes an option to disable autoformatting

🧮 Barrier functions for optimization

A set of functions with standard interfaces to make it easy to create soft constraints in optimization problems

  • d8300cb Added min/max centering barrier function
  • cd04ffd Added one-sided barrier functions
  • 1149e8a Min/max barrier function
  • 93a8727 Added symmetric barrier function

🎶 More noise models

  • e48e590 Added pseudo huber noise model
  • 48a42c7 Isotropic and diagonal noise models

✍🏼 Documentation

  • 90bdf74 Add missing docstrings for cc_sym
  • f93ae35 Add docstrings for cc_sym.Key
  • d5c66f3 Add module-level docstring to cc_sym
  • 958b098 Get cc_sym docs to generate correctly
  • 8dba435 Fix custom jacobian func argument name, docstring
  • f0d5922 Update comment on index_entry_t.offset

🎁 Misc other changes

  • 1fabece ComputeKeysToOptimize defaults comparator to LexicalLessThan
  • f89f60d add in databuffer + databuffer element to symengine
  • efe830e fix bug w/ symengine floor + ceil
  • 367beef Add omega property to ATANCameraCal
  • e4834f0 Added "length" metadata reader to timestep sub problem
  • d93c486 Tie tic tocs to log level
  • 8d94d38 Enable setting c++ log level from python
  • 9317314 Add property setters for DoubleSphereCC
  • 38f89cc Consolidate version numbers
  • c04b86a Pose2 tangent space matches Pose3
  • 88192d9 count ops after sympy-ification
  • 7d4f5c5 Replace sympy.count_ops w/ better impl
  • 202affc Remove extra quote in CLASS.py.jinja
  • ff98cd4 Wrap Optimizer.factors
  • 6e879fa Wrap Factor.IsSparse in cc_sym
  • 77e34c9 Add Rot2.from_angle
  • 12f6025 Better variable naming in opt factor construction
  • ccef9e6 Improve symengine.count_ops
  • 8f645a9 Misc documentation and EAP fixes
  • a2fa9ae Changed "assertNear" to "assertStorageNear"
  • c5ee7aa compute_jacobians is a method of ResidualBlock
  • 8484b2d Implemented group and lie group ops for ndarrays.
  • e2049f9 Add custom jacobian support to OptimizationProblem
  • 83a9be4 Add function to fetch variable position in problem ordering
  • 3c9dc21 Update license to Apache 2.0
  • faf77d2 Add option to prune the values object used for optimization.
  • 93cb131 Added support for pickling and unpickling Values objects
  • 3cf8854 change Matrix.dot to only support vector dot product
  • 36ebdde Default C++ log level to warning
  • 67f99e4 Downgrade METIS

👮🏼‍♀️ SkyMarshal

  • 4b8b9f3 Move _computeHash into class declaration
  • f5d2a13 Introduce #cpp_display notation for structs with primitive types