Releases: symforce-org/symforce
v0.9.0
Full Changelog: v0.8.0...v0.9.0
New in this release, we’ve added an experimental dense optimizer, greatly improved the docs, and made some additions to the core symbolic API and codegen API. We also have lots of under-the-hood changes, to the optimizer and the build in particular. Keep reading for more details, and thanks to everyone who contributed!
🚨 Breaking Changes
We're removing a few things that have been marked as deprecated for several releases now. We're also switching our C++ API to use references for mandatory input-output arguments, and deprecating the versions that used pointers, for consistency with our styleguide and modern best practices. And finally, we've removed the early_exited
field from the OptimizationStats
in favor of a more comprehensive description of exit status.
- 2683835 Remove deprecated stuff @aaron-skydio
- b660c97 opt output-args by reference, not ptr @bradley-solliday-skydio
- c3d2f3c Expose optimization status @aaron-skydio
🧠 Optimizer Changes
SymForce now has a dense optimizer! This is designed for problems where the problem linearization is dense enough that it's better to perform a dense Cholesky decomposition rather than a sparse one. It's currently in an experimental state, you can find example usage here.
As part of this, we've also made several changes to make it easier to swap out different parts of the optimizer like this.
Plus, we now expose more information about the result of an optimization beyond whether it just exited early - you can find that in status
and failure_reason
in the OptimizationStats
.
- 1c6ad68 Add basic test of Dense optimizer @bradley-solliday-skydio
- d30d1b2 CheckDerivatives gets any linearizer @bradley-solliday-skydio
- bdc665b Be Sparse/Dense Linearizer agnostic @bradley-solliday-skydio
- 9d0e3e1 Allow dense linearization in optimizer @bradley-solliday-skydio
- fd74776 OptimizationStats Linearizat'n agnostic @bradley-solliday-skydio
- 6501a45 Add DenseCholeskySolver @bradley-solliday-skydio
- c3d2f3c Expose optimization status @aaron-skydio
- ff17a89 Overload ComputeCovariance for MatrixX @bradley-solliday-skydio
- d792e99 Add dense linearizer @bradley-solliday-skydio
- 47b4ccc Factor out core of ComputeFactorHelper @bradley-solliday-skydio
- f5356d8 Rename opt/cholesky opt/sparse_cholesky @bradley-solliday-skydio
- caea9da add in compute_all_covariances to the py optimizer @harrison-skydio
- f37a783 Make LevenbergMarquardt Sparse agnostic @bradley-solliday-skydio
- f8cad34 Add helpful err msg if unopt'zed factor @bradley-solliday-skydio
- 6c3902d Template Linearization on sparsity @bradley-solliday-skydio
- 7a34ec0 Rename linearization_sparse_key_helper @bradley-solliday-skydio
- 0abeb62 Reduce # of public Linearizer methods @bradley-solliday-skydio
- 978ea34 Make optimization stats sparse agnostic @bradley-solliday-skydio
- 4f70119 Move ComputeKeysToOptimize to factor.h @bradley-solliday-skydio
- f03e353 Correct init residual order @bradley-solliday-skydio
- 8fd8c1c Update doc-string to reflect signature @bradley-solliday-skydio
- 59a0309 output_arg& UpdateXXXFactorIntoTriplets @bradley-solliday-skydio
- 7eda493 Add PreallocatedLinearizedDenseFactors @bradley-solliday-skydio
- a066808 Reduce # of index lookups in linearizer @bradley-solliday-skydio
- bdb848b Avoid second unordered_map lookup @bradley-solliday-skydio
- f8743e7 rm index from LinearizedFactor types @bradley-solliday-skydio
- d1b538c Add shape to sparsity pattern in stats @aaron-skydio
- b660c97 opt output-args by reference, not ptr @bradley-solliday-skydio
- 9266eb5 Add full stats to Python result object @aaron-skydio
- 596e3bc Explicit instantiations for opt @bradley-solliday-skydio
- e37758d Add missing pragma once @bradley-solliday-skydio
- 52fc289 Declare explicit instantiation @bradley-solliday-skydio
📚 Documentation Improvements
We've made multiple improvements to our docs for this release, both in readability and content. We have a new theme (with dark mode 😎 ), and the API docs now have things formatted and cross-linked correctly. Check them out at https://symforce.org!
- 3aef619 Add note about numerical epsilon @aaron-skydio
- c1d1ecb More docs @aaron-skydio
- ee77f27 Fix returns in docstrings @aaron-skydio
- 2dc5dda Clean up python sym docs @aaron-skydio
- 43cdfff Switch docs theme to Furo @aaron-skydio
- 12529d3 Clean up
opt
docs @aaron-skydio - 0e0fafc Partially clean up sym opt C++ docs @aaron-skydio
- cce64be Fix links from generated to symbolic classes @aaron-skydio
- caf55f4 Docs cleanup, take 2 @aaron-skydio
- e75ffb0 Docs cleanup, take 1 @aaron-skydio
- e833903 Update cameras tutorial @xipengwang
- 8ad6bab Added example code for initialization of sf.Rot3 using a quaternion @VineetTambe
- fb0e6cb Fix docs for
Pose*.(retract|local_coordinates)
@aaron-skydio - dfa4d18 Add comment on sf.atan2 @aaron-skydio
- 7269521 Reorder optimizer.py docstring @aaron-skydio
- 40f129c Clean up tangent_D_storage @aaron-skydio
🖊️ Codegen Improvements
We've made improvements and additions to codegen usage, and to the Python and PyTorch backends. In particular, we've added lambdify
and numbify
functions, to easily convert symbolic functions to numerical ones for fast evaluation in Python. For example,
def foo(x: sf.Scalar, R: sf.Rot3) -> sf.Scalar:
return (R * sf.V3(x, 0, 0)).x
foo_numeric = symforce.util.lambdify(foo)
foo_numeric(5.0, sym.Rot3())
- 1387297 Broadcast tensors before stacking in pytorch backend @zachary-teed-skydio
- 5c0d48d TemplateList.render supports specifying external search paths @gareth-cross
- b7bc699 Add update_template_data to CodegenConfig @gareth-cross
- 03fc8be Add lambdify, numbify @aaron-skydio
- 5a36e4c Use Codegen.name as python function name @bradley-solliday-skydio
- d805697 NumericFactor sets return_2d_vectors @aaron-skydio
- 6adfbee Add
evict
option forload_generated_function
@aaron-skydio - 68f3924 Generate Rot3.(from_angle_axis, from_two_unit_vectors) @aaron-skydio
🔣 Symbolic API Improvements
- 5bb0bdd Consolidate lists of types @aaron-skydio
- c1b37fd no more logmap_acos_clamp_max @aaron-skydio
- c1fdca1 Check types of geo constructor arguments @aaron-skydio
- d337809 Add
Matrix.det()
@aaron-skydio
⚙️ Buildsystem / Infra Changes
- df4f86c Print out fmt and spdlog version when finding packages @chao-qu-skydio
- e64a1d8 Small fix to tl-optional handling in cmakelists @chao-qu-skydio
- fe2a559 Fix symengine_wrapper search @aaron-skydio
- ae2173a Clarify error on failed symengine import @aaron-skydio
- f9c21ad Specify fmt version in find_package @chao-qu-skydio
- d7a40fa Bump cmake_minimum_required version to 3.19 @chao-qu-skydio
- 5bce25c Upgrade requirements @aaron-skydio
- c2283db Fix test include paths @aaron-skydio
- 33f46d8 Packaging cleanup @aaron-skydio
- 6c54bf5 Force pybind11 version @asaba96
- c91bd03 Add max cmake policy version @aaron-skydio
- db687f0 Remove unnecessary tl::optional alias @aaron-skydio
- d56c10c Install symforce slam @johhat
- 951884c Use URL for FetchContent @aaron-skydio
- c7de467 Fix symforce dependency of TartanLlama/optional @arturo-skydio
- 8545df6 Upgrade dependencies @aaron-skydio
- 30bd18e Drop bionic from CI @aaron-skydio
- ee6ac28 build_wheels works on forks @aaron-skydio
- 29b1ad1 Upgrade pylint @aaron-skydio
👮 SkyMarshal
- 7714ea7 [SkyMarshal] Print eigen_lcm types @aaron-skydio
- fd6543b [SkyMarshal] Enums have getTypeNameArrayPtr @aaron-skydio
- 61d70f0 skymarshal: only parse argfiles for the first argument @william-smith-skydio
- ccb5620 [SkyMarshal] Make enums mypy-checkable @aaron-skydio
- 5815fbd Add/fix some type annotations in skymarshal @aaron-skydio
✨ Misc
- 0940fa5 Fix new_symbol on py3.10+ @aaron-skydio
- 2afcce3 Objects returned from generated functions are const @aaron-skydio
- ace29ab More stuff includes-what-it-uses @aaron-skydio
- 024818f SYM_ASSERT takes messages @aaron-skydio
- df0a8d2 Remove redundant virtual in GncOptimizer @chao-qu-skydio
- b8821c9 A couple of random typos @chao-qu-skydio
- 3cc6061 Replace empty ctor with =default in factor and values @chao-qu-skydio
- 82d3a1a Use range-based for instead of index-based @chao-qu-skydio
- 07a53fd Finish removing os.path @aaron-skydio
- 5be27bd Fix a shadowing declaration in PopulateIterationStats @chao-qu-skydio
- 4539394 Add a SetNew method that throws if key already exists @chao-qu-skydio
- 72c45e4 Use the _t variant instead of ::type @chao-qu-skydio
- cfa72f1 Replace a couple of typedefs with using @chao-qu-skydio
- 0f0c802 Remove redundant static in an unnamed namespace @chao-qu-skydio
- 7b22108 Add Unit3 to cc_sym @aaron-skydio
- 35fd2d9 return size_t instead of int32_t from values::NumEntries() @chao-qu-skydio
v0.8.0
Full Changelog: v0.7.0...v0.8.0
This is probably our biggest release yet, with multiple major additions - we have built-in IMU Preintegration, new backends for CUDA and PyTorch, and a new symbolic type for unit vectors on S(2). Plus, we've made major improvements to memory usage in the C++ optimizer, made lots of additions to the symbolic API, and made plenty of improvements under the hood.
🚨 Breaking Changes
- ae43476 Slices of 1D matrices are matrices, not lists - taking a 1D slice of a
geo.Matrix
now returns ageo.Matrix
, previously it would return a list. - f60180e Add include_jacobians flag - by default, the
Optimizer
will now not allocate or populate the problem jacobian, since it is not needed for optimization. If you'd like it to be filled out, setinclude_jacobians
totrue
. - There are some simplifications to the
sym::Factor
API, described inCode Cleanup / Refactoring
below. Most users should be unaffected. - Generated Python functions return vectors as 1D ndarrays - see the description below
📈 IMU Factor
SymForce has IMU Preintegration! We have a C++ ImuFactor
with preintegration and linearization functions generated from a symbolic implementation. See the PRs #279 and #280 for more information, and see imu_factor.h
for an example.
- ab1c621 Add an on manifold Imu Factor @bradley-solliday-skydio
🦾 CUDA Backend
The CUDA backend generates __host__ __device__
functions, i.e. functions that you can call from within a CUDA kernel. It's currently in an experimental state, and supports scalars, vectors/matrices, and sf.DataBuffer
in the APIs of generated functions; other types like geo
and cam
objects can be passed via their storage vectors. See the commit for more examples.
- 45b2ce2 CUDA Backend @aaron-skydio
🔥 PyTorch Backend
The PyTorch backend generates python functions that operate on PyTorch tensors, with each symbolic op becoming one PyTorch op. It supports broadcasting these operations across multiple optional batch dimensions. Like the CUDA backend, this is currently experimental - it supports scalars and vectors/matrices in generated functions APIs, and is not particularly optimized for performance.
- a5341d8 PyTorch backend @aaron-skydio
- 5835edf Better PyTorch test coverage @aaron-skydio
⚪ geo.Unit3
The Unit3
symbolic type lets you represent unit vectors in 3D, including optimizing unit vector variables and performing manifold operations on them.
- 6dadff9 Add geo.Unit3 to model directions in R^3 @chao-qu-skydio
➡️ Generated Python functions return vectors as 1D ndarrays
By default, generated python functions will now return 1d ndarrays for outputs that are row or column vectors (including 1x1 matrices). This includes member functions of generated python types, such as sym.Pose3.position()
. See the commit for more information.
- 718d061 return 1d vectors if !return_2d_vectors @bradley-solliday-skydio
💨 Optimizer Performance/API Improvements
Major improvements to memory usage in the C++ optimizer, especially for large problems like bundle adjustment that have lots of small factors. Also, converting a sym::Values
to LCM is significantly faster.
- f60180e Add include_jacobians flag @ryan-brott-skydio
- 5b756a6 Linearizer: stop storing dense linearized factors @ryan-brott-skydio
- 1c1268a Turn off sort_by_offset for keys when get LCM message @xipeng-wang-skydio
- c7b2715 Factor Jacobian factory functions are constructors @aaron-skydio
- 8111eb2 Factor supports more functor types @samuel-wang-skydio
🔣 Symbolic API additions
- 892885b Default to automatically picking output_names @aaron-skydio
- ae43476 Slices of 1D matrices are matrices, not lists @nathan-skydio
- 652bc74 Add
CSCFormat.to_matrix
@nathan-skydio - bd42ee9 Add metadata field to ResidualBlock @nathan-skydio
- a959826 Simplify geo_factors_codegen w/ skip_dir_nesting @bradley-solliday-skydio
- fc4b4d5 More logic methods @aaron-skydio
- 6ed28d9 Add V2.x,y and V3.x,y,z @aaron-skydio
- 42b9694 Add sf.clamp @aaron-skydio
- 9769ee3 Add unit_* shorthands to Matrix @aaron-skydio
- 1c44a94 Pipe sparse_matrices through Codegen.function @aaron-skydio
- 24eaf20 Add CodegenConfig.custom_preamble @aaron-skydio
- a4d4bbc Check that namespaces are valid identifiers @aaron-skydio
- 04acf8a Allow setting default epsilon to same value @aaron-skydio
- f8f7eb4 Assert Matrix shape is correct when copying matrix @bradley-solliday-skydio
- 23b1f84 Change default epsilon value to symbol @bradley-solliday-skydio
🧹 Code Cleanup / Refactoring
- 6c3cc2d Cache factor indexes in the Linearizer @aaron-skydio
- 9774673 Raise exception during codegen if name is not valid @nathan-skydio
- 3e11ebd Check sparse matrix condition for Jacobian and Hessian constructors @changh95
- 783ede7 Reduce the overload set of Factor constructors @chao-qu-skydio
- af42fd8 DataBuffers not marked as unused @aaron-skydio
- 305387d Reduce surface area of keys_to_func and keys_to_optimize @chao-qu-skydio
- 1244d2c Move Factor::IsSparse to the Helpers section @chao-qu-skydio
- af2dff0 Optimizer asserts keys and factors are nonempty @aaron-skydio
- 4a3fd72 Clear before reserve in OptimizationStats.Reset() @chao-qu-skydio
- b2cceee Use the correct default epsilon in optimizer @chao-qu-skydio
- 49018a4 Use operator bool of std::function to determine is_sparse @chao-qu-skydio
- 9171906 Pass sink parameters by value and move @chao-qu-skydio
- d3bb744 Change some usages of ScalarType to Scalar @chao-qu-skydio
- 474f1bd Fix a linter error when a geo type does not define @chao-qu-skydio
- 26dfa72 Matrix types cleanup @aaron-skydio
- c835dbd Reduce number of functor copies in factor creation @chao-qu-skydio
- ecdeeef Make Key constexpr @chao-qu-skydio
- da7b88d Include cstdint instead of stdint.h @chao-qu-skydio
⚙️ Infra / Buildsystem changes
- 9741edf Set min version of lazy-object-proxy to 1.9.0 @bradley-solliday-skydio
- 0f4ca33 Fix symengine python_requires for setuptools 67 @aaron-skydio
- 4c36881 Run tests once @aaron-skydio
- 4a0e69b Update requirements @aaron-skydio
- 2207030 requirements_test supports running from different locations @aaron-skydio
- 1f53590 Add clang to CI, simplify matrix @aaron-skydio
- a04ae35 symforce/test/*.py covered by pylint @aaron-skydio
- 02e86d9 Use gcc 5.5 instead of 5.3 in CI @aaron-skydio
- 791347a GitHub Actions Workflow Updates @aaron-skydio
👮 SkyMarshal
- 732ce35 Typescript updates @vincent-lecrubier-skydio
- faeb8d1 All name functions are constexpr @aaron-skydio
- 6184693 Remove unused variable @aaron-skydio
- aaa593e Typescript changes @rachel-skydio
- 6dc889a Add support for optional
UNKNOWN=0
alias for enums in LCM protobuf generation. @karl-rosaen-skydio - 78ad229 Require python LCM arguments to construct using
from_all_fields
@peter-skydio
v0.7.0
Full Changelog: v0.6.0...v0.7.0
🚨 Breaking Changes
sf.Pose2_SE2
and sf.Pose3_SE3
are now in geo.unsupported
(also accessible through sf.unsupported
). We haven't removed any functionality, just moved them to clarify that they are not intended to be supported by much of SymForce and we do not intend to add more support for them. The sf.Pose2
and sf.Pose3
classes should be used instead, since they use more efficient representations.
- 1c1cfd1 Move SE(n) classes to geo.unsupported @aaron-skydio
🐍 Python 3.11 Support
SymForce now supports Python 3.11! As of this release you should be able to build symforce from source or install it with pip
on Python 3.11.
- 84f0fc7 Add py3.11 to list of supported versions @aaron-skydio
- 0bc8ea6 Python 3.11 support @aaron-skydio
- 443997a Update requirements @aaron-skydio
🗒️ Documentation and Examples Improvements
- 6828030 Fix links and warnings in tutorials @aaron-skydio
- 982ac12 Add internship callout, video link @aaron-skydio
- c55badc Add note to README about epsilon @aaron-skydio
- 6f100c5 Fix robot_2d_localization plotting @aaron-skydio
- 5d81680 Add info on
make install
to README @aaron-skydio - fd644ff Create Values before Factors in examples @aaron-skydio
📉 Optimizer Improvements
- e11c49f unique ID for Values and reset factor index_entries when input values changes @samuel-wang-skydio
- da76e73 Count linearized factors then allocate @aaron-skydio
- 2273737 Initialize default key contructor @simutisernestas
- d9132d2 Expose jacobian and cholesky factor sparsity patterns @aaron-skydio
🛠️ Buildsystem / Infrastructure / Testing Improvements
- b9795e1 Use checkout@v3 everywhere @aaron-skydio
- 417f566 Fix make test_update on macos @bradley-solliday-skydio
- ec17afd stubs codegen test doesn't require the manifest @aaron-skydio
- d2bb8a4 Add symforce_root and symforce_data_root @aaron-skydio
- aaa7846 Move unused-import pylint ignore to one line @aaron-skydio
- 352116f Add workflow_dispatch trigger for CI @aaron-skydio
- 3de6517 Remove unused interface library @bradley-solliday-skydio
- 3571634 Update Catch2 to v3.1.1; Test on ubuntu jammy @bradley-solliday-skydio
- af29620 Forward CMAKE_PREFIX_PATH to symengine @brian-kubisiak-skydio
- 7ca1e43 Fix lcmtypes install during editable installation @bradley-solliday-skydio
- cfa8e04 Editable install test fixes @aaron-skydio
👮 SkyMarshal
SkyMarshal C++ bindings can now serialize and deserialize messages larger than 2GiB
- 1552cbd Use ptrdiff_t for encode/decode sizes @aaron-skydio
- 95dd10b Misc typescript change @vincent-lecrubier-skydio
✨ Misc
- 694e2ca Add
_symengine_
attr to matrix class @roy-vorster-skydio - 9619229 Added assert to prevent codgen with nested databuffers @nathan-skydio
- 5cf1840 Fix bug that caused symengine symbols types to be pass-by-ref @roy-vorster-skydio
- d7220a0 More dataclasses over dicts @aaron-skydio
- 6447174 Purge os.path @aaron-skydio
- b16b2cb Add codegen_util.load_generated_function @bradley-solliday-skydio
- 238765e Add interpolate lie group operation @roy-vorster-skydio
- 3e543e8 Always raise IndexError if wrong shape genned code @bradley-solliday-skydio
- 7d58556 Default log level is info @aaron-skydio
- f0ef328 sym class constructors accept column vectors @bradley-solliday-skydio
v0.6.0
History: v0.5.0...v0.6.0
🔣 Default Epsilon Configuration
Building on the tools from v0.5 for enabling configurable default epsilons for library functions, add checks to enable confidently generating code without worrying about forgetting to pass an epsilon somewhere, which would previously result in an unsafe generated function. Generating a function without calling symforce.set_epsilon_to_symbol()
or symforce.set_epsilon_to_number()
will produce a warning, since this allows unintentionally calling functions with singularities with an epsilon of 0. Similarly, setting epsilon to a symbol, but then attemptting to generate a function that doesn't take an argument for epsilon, will yield a descriptive error message. This behavior is also configurable through new additions to the CodegenConfig
. The error messages are intended to have thorough explanations, so let us know if they don't make sense by creating an issue!
- 559e521 Set the default epsilon to a symbol for codegen
- f17d8b2 Run python tests with ctest
- 713a00a Add sym.epsilon
➕ Call User-Defined Functions from Generated C++
The code generator supports replacing builtin functions with user-provided alternatives. For instance, a custom my_lib::fast_sin
function can be replaced for std::sin
. The CppConfig
has a new argument override_methods
which can be used to specify these functions.
Shoutout to Roy Vorster for contributing this!
- 7f9cf25 Add function specific overwrites to sympy CPP printer
🏭 More methods on generated types
Add more methods to bring generated Python types up to parity with generated C++ types. In addition, automatically generate some previously handwritten methods.
- 48f6968 Codegen PoseX.position and rotation
- 90f14a0 Generate handwritten sym geo methods
- 81fa502 Uniform position/rotation accessors
- 1e95863 Generate to_homogenous_matrix for pose types
- 8e2b19c Codegen ToYawPitchRoll for C++ Rot3
🐍 Generated Python functions are more rigorous about Matrix shapes
PythonConfig
supports options for whether generated Python functions should expect all matrix inputs to be 2D arrays, or allow vectors to be 1D arrays. This fixes the previous unintuitive behavior of expecting storage arrays.
- c163cb6 Reshape matrix inputs to be rank 2
- 652eb97 Add matrix accessor format function to config and remove matrix_is_1d param
🔧 CI/CD
Add a continuous testing suite on main and on pull requests across compilers and Python versions. In addition, add automatic deployment of the symforce docs to https://symforce.org.
- ff5a34b Rerun tests with output on failure
- 716befd Add docs as an artifact of CI runs
- 6dd9f9a Run CI on push to main and build badge
- d9633f3 Preserve downloads symforce-org.github.io
- de296a8 Auto-deploy docs on push to main
- 20548c3 Add SymForce CI
🧹 Code Quality Improvements
Various internal improvements, fixing C++ build warnings, sorting imports, and refactoring internal modules
- d10ac9a Sort imports with isort
- 6d1c918 Fix warnings
- 322faba Sort some imports
- b268447 Removed unused imports in symforce python function codegen template
- c3d0a25 Move things to typing_util
- 525f229 Misc changes to tests
- a7999d6 TemplateListEntry -> dataclass
- d154598 Modify optionality of template_dir arg
- 3f467fa Make TemplateList.add arg order match
- 429e6ac Take relative path in render_template
👮 SkyMarshal
- 58e37ea [SkyMarshal] Sort imports
- a815051 [SkyMarshal] static functions in lcm_coretypes are
__attribute__((unused))
- d2760d7 [SkyMarshal] Explicitly static_cast hashes to int64
🎁 Misc
- 693f270 Clarify docstrings on SE(n) classes
- da612a9 Update installation instructions
- 8f01c0d Bump version to 0.6
- a948a31 Add failing test for factor cache with custom jacobians
- 9b3c47b More factor cache fixes
- 48fd72d Use corrected runpaths w/ setup.py develop too
- 29310df dev_requirements.txt errors on py < 3.8
- d2181a8 Add note about building from a release
- 94d1bfa SparseCholesky is MPL2
- b3d6edb Fix of pip install -e . on both macos and ubuntu
- e97a5fd Upgrade to sympy 1.11
- 9a20a81 Fix factor cache
- 48699d1 Add more named Matrix types
- 47f4adb Fix values_tutorial link
- 324bba2 LM Optimizer prints out id with iteration stats
- 40a6c9e Pull in the latest tl::optional
- b339396 Add numba to dev_requirements.txt
- e2ed962 Symforce add sparse_linearization option to factor.py
- 7289ce0 Don't add symengine to sys.modules if import fails
- e131eae Use SYMFORCE_PYTHON for rerun_if_needed
- 75201b6 Fix build on cmake >=3.23
- d8e992f Fix broken links in README
- 71d36b0 Raise ValueError if subs args are different lengths
- 73d5dbe Symforce Values.from_storage allow different scalar types
- 73b22a5 Allow symforce storage ops to return numeric types
- d50dbba Fix docs for Group and LieGroup
- eeadcee Added getter function for symforce optimizer params
- cea8701 Move error message for wrong number of keys
- 367eeb8 Added helper function for converting Symforce Values to dicts
- 68384d0 Dataclass ops support T.Tuple
- 2d1a615 Codegen performance improvements
- e9ef402 LinearCameraCal doesn't take Abs(z)
- 65f6d00 Use sf.sympy.Eq in symbolic_computation_speedups.ipynb
- 52d2b5d Regenerate with make test_update_all
- 87c031b Better errors from wrong factor results
- c0933d0 Fix type annotation
- f6496d6 Remove local Catch2 (instead fetch it)
- cdf8560 fix bug w/ lcm gen for large matrix
- d614535 Make cc_sym types pickleable
v0.5.0
History: v0.4.0...v0.5.0
🚨 Symbolic API moved to symforce.symbolic
🚨
The symbolic math components of SymForce are now unified under one import for ease of use:
import symforce.symbolic as sf
The symforce.symbolic
module contains the SymForce-modified SymPy (or SymEngine) API, with our added logic functions and helper functions (i.e. things that were previously accessible from symforce.sympy
), geometric lie group types (from symforce.geo
), and camera types (from symforce.cam
). If you just want the geo
or cam
types, those modules still exist and can be imported instead.
Usage of symforce.sympy
is deprecated, you should use symforce.symbolic
instead. It will be removed in a future release.
- 46dc58b matrix.fixed_type_from_shape -> matrix_type_from_shape
- b34909b Add deprecated symforce.sympy stub
- 5e0afd5 Add T.Scalar to sf
- a26a9d5 Add geo and cam to symforce.symbolic
- 192f5cf Move sympy api to symforce.symbolic
🔣 Configurable default epsilons for library functions
SymForce now provides more flexibility around setting the default value of epsilon, its mechanism for handling singularity points.
Many library functions require an argument for epsilon
, and previously defaulted it to 0 for ease of prototyping e.g. in a notebook. For runtime code, however, this can be a potential source of issues, since it can be easy to unintentionally forget to pass epsilon to one of these functions, meaning that an unhandled singularity creeps in. To address this, we have added a function sf.epsilon()
which should be used as the default epsilon argument for library functions. The value that this returns is configurable - the default is 0 (so by default, behavior is the same). However, it can also be set to a symbol with sf.set_epsilon_to_symbol()
, which is recommended for generating production code. Similarly, it can be set to a numerical value with sf.set_epsilon_to_number()
. Either of these must be called before importing symforce.symbolic
.
- f63171c Fixed default epsilon
- 7133947 Use sm.epsilon() as default eps
- 8965840 sm.default_epsilon -> sm.numeric_epsiln
⚙️ Unified mechanism for adding code generation backends
We've unified much of the backend-specific differences, so that it should be easier to add new code generation backends going forward. See the PR and the example of adding a JavaScript backend for more information.
- e4c2d19 Centralize code for creating new code generation backends
👮 SkyMarshal Updates
- 90c4891 Enable skymarshal printing by default
- 5fb13f7 Allow unknown notations in skymarshal
- 06be0d8 [Skymarshal] Generate operator<< for all types
🛠️ Build System Fixes
- 2e14e4d Add minimum spdlog version
- fba4ae8 Workflow and build support for pip wheels
- 7f59396 Bump default pybind to 2.9.2
- ea82c15 Remove spdlog (and eigen and catch2) from the manifest
- 7efa7cd Misc docs, CMake, and packaging fixes
- 322c564 Upgrade pip requirements
- c321ca0 Install eigen_lcm headers
- aba8ef7 Fix install paths
🎁 Misc other changes
- e7fea68 Fix spelling of word perturbation in lie-group docstrings
- 7c778cc Use spdlog::set_level instead of sym::internal::SetLogLevel
- 411c845 Retry symforce_docs_test 3 times on failure
- 3915767 Copy robot_2d_localization example
- 234f827 Make robot_3d_localization comment true
- 41fbca0 Remove duplicate epsilon files
- 7bee06e Change format_py_dir to take T.Openable
- 5f4b9fc Emit a warning if optimizing factors that don't touch any keys
- 84a1ca2 Call modify_symbolic_api once per sympy
- cccb324 Fix up C++ example in README
- 43c2dfd Fix derivatives of sm.Mod
- 165b097 Fix plotting code for robot_3d_localization example
- 9e08a78 fix factor_utils failing to compile on clang-6.0
- 60c9ccf Ret Path (!str) in generate_lcm_types
- bdd50e9 Enable sparse cc_sym.Factor creation
v0.4.0
v0.4.0 is the first public release of SymForce outside the early-access program, and the first release after the paper was accepted to RSS 2022. SymForce can now be pip installed on Linux and Mac with pre-built wheels.
History: v0.3.0...v0.4.0
✨ Support for sparse matrices and factors in Python
Previously, SymForce could generate C++ functions of sparse matrices (in Eigen) and use them in the C++ optimizer, but it could not generate Python functions of sparse matrices or use Python optimizer. This release adds support for generating Python functions that use SciPy's sparse matrix format, and also the ability to convert them to Eigen in the pybind wrapper, so they can be used with the optimizer from Python.
- 77b8678 Add sym::Factor::SparseJacobianFunc
- 3ec069c Calculate hessian & rhs in one place
- eef1c9c Add sparse outputs to generate py code
- 3d8cd89 Expose symbolic jacobians computed by factor
- a43a296 Factor out py residual wrapper - cc_sym
- 2f91fb2 Make sparse_matrix_init more readable
- 357507e Replace Matrix arg with CSCFormat arg
- d2b1042 Remove None check in C++ sparse mat gen
- 4e2e586 More arguments passed through OptimizationProblem and Factor
- ff6cd01 get_sparse_mat_data to CSCFormat method
- 1fab87c Consistent Matrix storage order
- e2fbce0 CSCFormat type instead of dict
🧊 Data Buffers
Added initial support for data buffers, which are external arrays of data passed in to symbolic expressions, for example lookup tables, images, or volumetric maps. Code generation is currently supported, but not optimization.
- 2cd9dfe Initial work for values to support DataBuffer entries
- 236e27f don't allow DataBuffer to be iterable
- e0db744 Allow symengine to simplify DatabufferElement index expressions
- a550ed0 fix bug w/ databuffer storage ops
- 9f1345b databuffer tests
- 981d48a implement databuffer
⏱️ Code Generation
Many improvements to code generation quality and speed for large problems.
- 0c67bb8 Allow specific explicit template instantiations
- 0f03029 Give generate_function return type
- e7c1388 Convert matrix elements to float in to_numerical()
- 9a1069e Fix return type: str -> sm.Symbol
- ca55474 fix bug w/ symbol formatting in codegen_util
- 33184aa Remove unused codegen_util code
- aea7eb0 Rename output_terms to dense_terms
- 2862b26 Remove dead jinja code
- 7414072 Fixed codegen bug with dataclasses
- a626932 Allow explicit template instantiation for C++ codegen
- d094cda add an option to symengine wrapper to skip eval on sympy
- 04c6274 Changed codegen folder names from python2.7 to python
- 6aa7e0d add dont_flatten_args to _get_subs_dict
- c1d482d Disable code format when generating temp factors
- e654de2 move lazy imports outside for loop
- e76c6d6 Fix SymforceCodegenTest doc-string
- 015e968 Allow symforce to generate code from functions w/ dataclasses as args
👁️ Bundle Adjustment in the Large
Added an example that optimizes the datasets from Bundle Adjustment in the Large to parallel the GTSAM and Ceres examples. This is not yet a benchmark optimized for performance.
- bad1035 Bundle-Adjustment-in-the-Large Example
🥇 Optimization
Improve the optimization library based on testing larger multi-objective problems.
- 4540121 Fixed typo in optimizer
- 304654e Explicit template intantiate Optimizer
- af280a3 Added a type check on optimized values
- 34e6c17 fix bug w/ custom jacobians
- 46e8a0c Added some useful timestep objectives
- 2c926e8 Added support for loading factors from files
🛠️ Build System
Support pip install symforce
with pre-built wheels, and upgrade dependencies.
- af687c1 Pin pip requirements
- 92c30f7 Support pip install [-e] .
- a3b2dc0 Bump skymarshal required cmake version to 3.19
- 24e9601 Split CMakeLists
- 3769c2d Upgrade sympy to version 1.10.1
✍🏼 Documentation
Improved the README, tutorials, and examples across the board to prepare for public release.
- ad2a796 Epsilon tutorial, rename ops tutorial
- 0bb830d Tweak README to prepare for public beta
- 0c2f9f2 Fix images in README
- d6eb5cc Add issue templates
- 91e0b18 Add arXiv link to README
- 94b1abd Add citation to repo, README
- bd26812 New Images
- d451552 Fix docs version and date
- 340c74d Regenerate after README and version updates
- 60042bb Update README.md
- 76e785e Update README.md
- a1da5b0 Minor README update
- 982f6d4 Update matrix multiplication benchmark
🎁 Miscellaneous
- 78d7844 [Skymarshal] lextab updates
- fb1bd27 Delete cholesky/assert
- f127562 Autoformat with clang-format 14
- 8591d8f Include cassert where we use it
- 185be31 Simplify ArrayStorageOps
- d32d14b Have ndarray ops use AbstractVectorOps
- 359bf11 [Skymarshal] Optional printing
- 5d52d76 Allow geo.M.setitem with np.float64
- 97a47dc Assert geo.Matrix shape == arg shape
- 69ea6d6 Diffs in test failures
- cfc0b87 Remove #hashable from sym.key_t.
v0.3.0
📷 Support for CameraCal types in C++ Values and optimizations
You can now use camera calibration objects directly as inputs to Factors for optimization, either as optimized variables or as constants.
- 1db2d85 camera cals in cc_sym.Values
- 73af9cc Add camera cals to python sym
- c230173 Auto-generate C++ cam cal constructors
- 523b5a1 Rework camera cal constructors
- f7987ee Add camera cal types to BY_TYPE_HELPER
- b26be84 Define template specializations of [Lie]GroupOps for camera cals in C++
- e43406e Register CameraCal with LieGroupOps in python
- 2509e29 Add all the camera cals to types_t and define TypeEnum
🐍 More support for dataclasses
We recently added some support for using Dataclasses in various places, such as in Values
and with Ops
. We've now added more thorough support for them in more places:
- dd017bf Fixed size dataclass ops with sequences
- 184238d Added ops support for NoneTypes so that Dataclasses can have optional fields
- 4a079ce Added Group and Lie Group Ops for Dataclasses
- 05debd6 Added better support for indexing dataclasses stored in Values objects
- db4585f Added clearer error message for dynamically created dataclasses stored in Values objects
🎲 Lots more examples and benchmarks
- ea6c6a4 Examples and Benchmarks
⛸️ std::complex
in generated C++ code
Generated C++ functions can now handle complex scalars! If the support_complex
flag is true, generated functions will be allowed to contain complex constants and otherwise work correctly if instantiated with Scalar = std::complex<T>
.
- 94d37e9 Add support for std::complex in C++ functions
⏱️ Code generation performance improvements
Some changes to speed up code generation by eliminating a lot of unnecessary work
- c26926d [SymEngine] Add
symbols
argument to CSE - 45c4eb3 Cache generated residuals in Factor
- 2294616 Cache jinja environment
- d8563ed Speed up assert in Codegen construction
- 6a4fb41 Don't autoformat python, for real
- 3e13c3a Speed up symforce_py_optimizer_test.py
- d907c37 Lazily print code in Codegen
- 630e0d3 codegen exposes an option to disable autoformatting
🧮 Barrier functions for optimization
A set of functions with standard interfaces to make it easy to create soft constraints in optimization problems
- d8300cb Added min/max centering barrier function
- cd04ffd Added one-sided barrier functions
- 1149e8a Min/max barrier function
- 93a8727 Added symmetric barrier function
🎶 More noise models
✍🏼 Documentation
- 90bdf74 Add missing docstrings for cc_sym
- f93ae35 Add docstrings for cc_sym.Key
- d5c66f3 Add module-level docstring to cc_sym
- 958b098 Get cc_sym docs to generate correctly
- 8dba435 Fix custom jacobian func argument name, docstring
- f0d5922 Update comment on index_entry_t.offset
🎁 Misc other changes
- 1fabece ComputeKeysToOptimize defaults comparator to LexicalLessThan
- f89f60d add in databuffer + databuffer element to symengine
- efe830e fix bug w/ symengine floor + ceil
- 367beef Add omega property to ATANCameraCal
- e4834f0 Added "length" metadata reader to timestep sub problem
- d93c486 Tie tic tocs to log level
- 8d94d38 Enable setting c++ log level from python
- 9317314 Add property setters for DoubleSphereCC
- 38f89cc Consolidate version numbers
- c04b86a Pose2 tangent space matches Pose3
- 88192d9 count ops after sympy-ification
- 7d4f5c5 Replace sympy.count_ops w/ better impl
- 202affc Remove extra quote in CLASS.py.jinja
- ff98cd4 Wrap Optimizer.factors
- 6e879fa Wrap Factor.IsSparse in cc_sym
- 77e34c9 Add Rot2.from_angle
- 12f6025 Better variable naming in opt factor construction
- ccef9e6 Improve symengine.count_ops
- 8f645a9 Misc documentation and EAP fixes
- a2fa9ae Changed "assertNear" to "assertStorageNear"
- c5ee7aa compute_jacobians is a method of ResidualBlock
- 8484b2d Implemented group and lie group ops for ndarrays.
- e2049f9 Add custom jacobian support to OptimizationProblem
- 83a9be4 Add function to fetch variable position in problem ordering
- 3c9dc21 Update license to Apache 2.0
- faf77d2 Add option to prune the values object used for optimization.
- 93cb131 Added support for pickling and unpickling Values objects
- 3cf8854 change Matrix.dot to only support vector dot product
- 36ebdde Default C++ log level to warning
- 67f99e4 Downgrade METIS