SUNDIALS patch release v6.4.1

Changes to SUNDIALS in release 6.4.1

Fixed a bug with the Kokkos interfaces that would arise when using clang.

Fixed a compilation error with the Intel oneAPI 2022.2 Fortran compiler in the
Fortran 2003 interface test for the serial N_Vector.

Fixed a bug in the SUNLINSOL_LAPACKBAND and SUNLINSOL_LAPACKDENSE modules
which would cause the tests to fail on some platforms.

SUNDIALS minor release v6.4.0

Changes to SUNDIALS in release 6.4.0

CMake 3.18.0 or newer is now required for CUDA support.

A C++14 compliant compiler is now required for C++ based features and examples
e.g., CUDA, HIP, RAJA, Trilinos, SuperLU_DIST, MAGMA, GINKGO, and KOKKOS.

Added support for GPU enabled SuperLU_DIST and SuperLU_DIST v8.x.x. Removed
support for SuperLU_DIST v6.x.x or older. Fix mismatched definition and
declaration bug in SuperLU_DIST matrix constructor.

Added support for the Ginkgo linear algebra
library. This support includes new SUNMatrix and SUNLinearSolver
implementations, see the SUNMATRIX_GINKGO and SUNLINEARSOLVER_GINKGO
sections in the documentation for more information.

Added new NVector, dense SUNMatrix, and dense SUNLinearSolver
implementations utilizing Kokkos Ecosystem for
performance portability, see the NVECTOR_KOKKOS, SUNMATRIX_KOKKOSDENSE and
SUNLINEARSOLVER_KOKKOSDENSE sections in the documentation for more
information.

Added the functions ARKStepSetTableName, ERKStepSetTableName,
MRIStepCoupling_LoadTableByName, ARKodeButcherTable_LoadDIRKByName, and
ARKodeButcherTable_LoadERKByName to load a table from a string.

Fixed a bug in the CUDA and HIP vectors where N_VMaxNorm would return the
minimum positive floating-point value for the zero vector.

Fixed memory leaks/out of bounds memory accesses in the ARKODE MRIStep module
that could occur when attaching a coupling table after reinitialization with a
different number of stages than originally selected.

Fixed a memory leak in CVODE and CVODES where the projection memory would not be
deallocated when calling CVodeFree.

SUNDIALS minor release v6.3.0

Changes to SUNDIALS in release 6.3.0

Added GetUserData functions in each package to retrieve the user data pointer
provided to SetUserData functions. See ARKStepGetUserData,
ERKStepGetUserData, MRIStepGetUserData, CVodeGetUserData,
IDAGetUserData, or KINGetUserData for more information.

Fixed a bug in ERKStepReset, ERKStepReInit, ARKStepReset, ARKStepReInit,
MRIStepReset, and MRIStepReInit where a previously-set value of tstop (from
a call to ERKStepSetStopTime, ARKStepSetStopTime, or MRIStepSetStopTime,
respectively) would not be cleared.

Updated MRIStepReset to call the corresponding MRIStepInnerResetFn with the same
(tR,yR) arguments for the MRIStepInnerStepper object that is used to evolve the
MRI "fast" time scale subproblems.

Added a new example which demonstrates using CVODE with a discontinuous right-hand-side function and rootfinding.

Added a variety of embedded DIRK methods from [Kennedy & Carpenter,
NASA TM-2016-219173, 2016] and [Kennedy & Carpenter, Appl. Numer. Math., 146, 2019] to
ARKODE.

Fixed the unituitive behavior of the USE_GENERIC_MATH CMake option which
caused the double precision math functions to be used regardless of the value of
SUNDIALS_PRECISION. Now, SUNDIALS will use precision appropriate math
functions when they are available and the user may provide the math library to
link to via the advanced CMake option SUNDIALS_MATH_LIBRARY.

Changed SUNDIALS_LOGGING_ENABLE_MPI CMake option default to be 'OFF'.

SUNDIALS minor release v6.2.0

https://github.com/LLNL/sundials is now the official SUNDIALS development repository.

Added the SUNLogger API which provides a SUNDIALS-wide
mechanism for logging of errors, warnings, informational output,
and debugging output.

Deprecated the following functions, it is recommended to use the SUNLogger API
instead.

• ARKStepSetDiagnostics
• ERKStepSetDiagnostics
• MRIStepSetDiagnostics
• KINSetInfoFile
• SUNNonlinSolSetPrintLevel_Newton
• SUNNonlinSolSetInfoFile_Newton
• SUNNonlinSolSetPrintLevel_FixedPoint
• SUNNonlinSolSetInfoFile_FixedPoint
• SUNLinSolSetInfoFile_PCG
• SUNLinSolSetPrintLevel_PCG
• SUNLinSolSetInfoFile_SPGMR
• SUNLinSolSetPrintLevel_SPGMR
• SUNLinSolSetInfoFile_SPFGMR
• SUNLinSolSetPrintLevel_SPFGMR
• SUNLinSolSetInfoFile_SPTFQM
• SUNLinSolSetPrintLevel_SPTFQMR
• SUNLinSolSetInfoFile_SPBCGS
• SUNLinSolSetPrintLevel_SPBCGS

The SUNLinSolSetInfoFile_** and SUNNonlinSolSetInfoFile_* family of
functions are now enabled by setting the CMake option SUNDIALS_LOGGING_LEVEL
to a value >= 3.

Added the function SUNProfiler_Reset to reset the region timings and counters
to zero.

Added the functions ARKStepPrintAllStats, ERKStepPrintAllStats,
MRIStepPrintAll, CVodePrintAllStats, IDAPrintAllStats, and
KINPrintAllStats to output all of the integrator, nonlinear solver, linear
solver, and other statistics in one call. The file scripts/sundials_csv.py
contains functions for parsing the comma-separated value output files.

Added functions to CVODE, CVODES, IDA, and IDAS to change the default step size
adaptivity parameters. For more information see the documentation for:

• CVodeSetEtaFixedStepBounds
• CVodeSetEtaMaxFirstStep
• CVodeSetEtaMaxEarlyStep
• CVodeSetNumStepsEtaMaxEarlyStep
• CVodeSetEtaMax
• CVodeSetEtaMin
• CVodeSetEtaMinErrFailEta
• CVodeSetEtaMaxErrFailEta
• CVodeSetNumFailsEtaMaxErrFail
• CVodeSetEtaConvFail
• IDASetEtaFixedStepBounds
• IDAsetEtaMax
• IDASetEtaMin
• IDASetEtaLow
• IDASetEtaMinErrFail
• IDASetEtaConvFail

Added the functions CVodeSetDeltaGammaMaxLSetup and
CVodeSetDeltaGammaMaxBadJac in CVODE and CVODES to adjust the gamma change
thresholds to require a linear solver setup or Jacobian/precondition update,
respectively.

Added the function IDASetDetlaCjLSetup in IDA and IDAS to adjust the parameter
that determines when a change in c_j requires calling the linear solver setup
function.

Added the function MRIStepSetOrder to select the default MRI method of a given
order.

Added support to CVODES for integrating IVPs with constraints using BDF methods
and projecting the solution onto the constraint manifold with a user defined
projection function. This implementation is accompanied by additions to the
CVODES user documentation and examples.

The behavior of N_VSetKernelExecPolicy_Sycl has been updated to be consistent
with the CUDA and HIP vectors. The input execution policies are now cloned and
may be freed after calling N_VSetKernelExecPolicy_Sycl. Additionally, NULL
inputs are now allowed and, if provided, will reset the vector execution
policies to the defaults.

Fixed the SUNContext convenience class for C++ users to disallow copy
construction and allow move construction.

A memory leak in the SYCL vector was fixed where the execution policies were
not freed when the vector was destroyed.

The include guard in nvector_mpimanyvector.h has been corrected to enable
using both the ManyVector and MPIManyVector NVector implementations in the same
simulation.

Changed exported SUNDIALS PETSc CMake targets to be INTERFACE IMPORTED instead
of UNKNOWN IMPORTED.

A bug was fixed in the integrator functions to retrieve the number of nonlinear
solver failures. The failure count returned was the number of failed steps due
to a nonlinear solver failure i.e., if a nonlinear solve failed with a stale
Jacobian or preconditioner but succeeded after updating the Jacobian or
preconditioner, the initial failure was not included in the nonlinear solver
failure count. The following functions have been updated to return the total
number of nonlinear solver failures:

• ARKStepGetNumNonlinSolvConvFails
• ARKStepGetNonlinSolvStats
• MRIStepGetNumNonlinSolvConvFails
• MRIStepGetNonlinSolvStats
• CVodeGetNumNonlinSolvConvFails
• CVodeGetNonlinSolvStats
• CVodeGetSensNumNonlinSolvConvFails
• CVodeGetSensNonlinSolvStats
• CVodeGetStgrSensNumNonlinSolvConvFails
• CVodeGetStgrSensNonlinSolvStats
• IDAGetNumNonlinSolvConvFails
• IDAGetNonlinSolvStats
• IDAGetSensNumNonlinSolvConvFails
• IDAGetSensNonlinSolvStats

As such users may see an increase in the number of failures reported from the
above functions. The following functions have been added to retrieve the number
of failed steps due to a nonlinear solver failure i.e., the counts previously
returned by the above functions:

• ARKStepGetNumStepSolveFails
• MRIStepGetNumStepSolveFails
• CVodeGetNumStepSolveFails
• CVodeGetNumStepSensSolveFails
• CVodeGetNumStepStgrSensSolveFails
• IDAGetNumStepSolveFails
• IDAGetNumStepSensSolveFails

SUNDIALS patch release v6.1.1

Changes to SUNDIALS in release 6.1.1

Fixed exported SUNDIALSConfig.cmake.

Fixed Fortran interface to MRIStepInnerStepper and ``MRIStepCoupling`
structures and functions.

Added new Fortran example program,
examples/arkode/F2003_serial/ark_kpr_mri_f2003.f90 demonstrating MRI
capabilities.

SUNDIALS minor release v6.1.0

Changes to SUNDIALS in release 6.1.0

Added new reduction implementations for the CUDA and HIP NVECTORs that use
shared memory (local data storage) instead of atomics. These new implementations
are recommended when the target hardware does not provide atomic support for the
floating point precision that SUNDIALS is being built with. The HIP vector uses
these by default, but the N_VSetKernelExecPolicy_Cuda and
N_VSetKernelExecPolicy_Hip functions can be used to choose between
different reduction implementations.

SUNDIALS::<lib> targets with no static/shared suffix have been added for use
within the build directory (this mirrors the targets exported on installation).

CMAKE_C_STANDARD is now set to 99 by default.

Fixed exported SUNDIALSConfig.cmake when profiling is enabled without Caliper.

Fixed sundials_export.h include in sundials_config.h.

Fixed memory leaks in the SUNLINSOL_SUPERLUMT linear solver.

Several other minor bug-fixes and improvements.

SUNDIALS major release v6.0.0

Changes to SUNDIALS in release 6.0.0

SUNContext

SUNDIALS v6.0.0 introduces a new SUNContext object on which all other SUNDIALS
objects depend. As such, the constructors for all SUNDIALS packages, vectors,
matrices, linear solvers, nonlinear solvers, and memory helpers have been
updated to accept a context as the last input. Users upgrading to SUNDIALS
v6.0.0 will need to call SUNContext_Create to create a context object with
before calling any other SUNDIALS library function, and then provide this object
to other SUNDIALS constructors. The context object has been introduced to allow
SUNDIALS to provide new features, such as the profiling/instrumentation also
introduced in this release, while maintaining thread-safety. See the
documentation section on the SUNContext for more details.

A script upgrade-to-sundials-6-from-5.sh has been provided with the release
(obtainable from the GitHub release page) to help ease the transition to
SUNDIALS v6.0.0. The script will add a SUNCTX_PLACEHOLDER argument to all of
the calls to SUNDIALS constructors that now require a SUNContext object. It
can also update deprecated SUNDIALS constants/types to the new names. It can be
run like this:

> ./upgrade-to-sundials-6-from-5.sh <files to update>

SUNProfiler

A capability to profile/instrument SUNDIALS library code has been added. This
can be enabled with the CMake option SUNDIALS_BUILD_WITH_PROFILING. A built-in
profiler will be used by default, but the
Caliper library can also be used instead with
the CMake option ENABLE_CALIPER. See the documentation section on profiling
for more details. WARNING: Profiling will impact performance, and should be
enabled judiciously.

SUNMemoryHelper

The SUNMemoryHelper functions Alloc, Dealloc, and Copy have been updated
to accept an opaque handle as the last input. At a minimum, existing
SUNMemoryHelper implementations will need to update these functions to accept
the additional argument. Typically, this handle is the execution stream (e.g., a
CUDA/HIP stream or SYCL queue) for the operation. The CUDA, HIP, and SYCL
SUNMemoryHelper implementations have been updated accordingly. Additionally,
the constructor for the SYCL implementation has been updated to remove the SYCL
queue as an input.

NVector

Two new optional vector operations, N_VDotProdMultiLocal and
N_VDotProdMultiAllReduce, have been added to support low-synchronization
methods for Anderson acceleration.

The CUDA, HIP, and SYCL execution policies have been moved from the sundials
namespace to the sundials::cuda, sundials::hip, and sundials::sycl
namespaces respectively. Accordingly, the prefixes "Cuda", "Hip", and "Sycl"
have been removed from the execution policy classes and methods.

The Sundials namespace used by the Trilinos Tpetra NVector has been replaced
with the sundials::trilinos::nvector_tpetra namespace.

The serial, PThreads, PETSc, hypre, Parallel, OpenMP_DEV, and OpenMP vector
functions N_VCloneVectorArray_* and N_VDestroyVectorArray_* have been
deprecated. The generic N_VCloneVectorArray and N_VDestroyVectorArray
functions should be used instead.

The previously deprecated constructor N_VMakeWithManagedAllocator_Cuda and
the function N_VSetCudaStream_Cuda have been removed and replaced with
N_VNewWithMemHelp_Cuda and N_VSetKerrnelExecPolicy_Cuda respectively.

The previously deprecated macros PVEC_REAL_MPI_TYPE and
PVEC_INTEGER_MPI_TYPE have been removed and replaced with
MPI_SUNREALTYPE and MPI_SUNINDEXTYPE respectively.

SUNLinearSolver

The following previously deprecated functions have been removed

Removed	Replaced with
SUNBandLinearSolver	SUNLinSol_Band
SUNDenseLinearSolver	SUNLinSol_Dense
SUNKLU	SUNLinSol_KLU
SUNKLUReInit	SUNLinSol_KLUReInit
SUNKLUSetOrdering	SUNLinSol_KLUSetOrdering
SUNLapackBand	SUNLinSol_LapackBand
SUNLapackDense	SUNLinSol_LapackDense
SUNPCG	SUNLinSol_PCG
SUNPCGSetPrecType	SUNLinSol_PCGSetPrecType
SUNPCGSetMaxl	SUNLinSol_PCGSetMaxl
SUNSPBCGS	SUNLinSol_SPBCGS
SUNSPBCGSSetPrecType	SUNLinSol_SPBCGSSetPrecType
SUNSPBCGSSetMaxl	SUNLinSol_SPBCGSSetMaxl
SUNSPFGMR	SUNLinSol_SPFGMR
SUNSPFGMRSetPrecType	SUNLinSol_SPFGMRSetPrecType
SUNSPFGMRSetGSType	SUNLinSol_SPFGMRSetGSType
SUNSPFGMRSetMaxRestarts	SUNLinSol_SPFGMRSetMaxRestarts
SUNSPGMR	SUNLinSol_SPGMR
SUNSPGMRSetPrecType	SUNLinSol_SPGMRSetPrecType
SUNSPGMRSetGSType	SUNLinSol_SPGMRSetGSType
SUNSPGMRSetMaxRestarts	SUNLinSol_SPGMRSetMaxRestarts
SUNSPTFQMR	SUNLinSol_SPTFQMR
SUNSPTFQMRSetPrecType	SUNLinSol_SPTFQMRSetPrecType
SUNSPTFQMRSetMaxl	SUNLinSol_SPTFQMRSetMaxl
SUNSuperLUMT	SUNLinSol_SuperLUMT
SUNSuperLUMTSetOrdering	SUNLinSol_SuperLUMTSetOrdering

Fortran Interfaces

The ARKODE, CVODE, IDA, and KINSOL Fortran 77 interfaces have been removed. See
the "SUNDIALS Fortran Interface" section in the user guides and the F2003
example programs for more details using the SUNDIALS Fortran 2003 module
interfaces.

ARKODE

The ARKODE MRIStep module has been extended to support implicit-explicit (IMEX)
multirate infinitesimal generalized additive Runge-Kutta (MRI-GARK) methods. As
such, MRIStepCreate has been updated to include arguments for the slow
explicit and slow implicit ODE right-hand side functions. MRIStepCreate has
also been updated to require attaching an MRIStepInnerStepper for evolving the
fast time scale. MRIStepReInit has been similarly updated to take explicit
and implicit right-hand side functions as input. Codes using explicit or
implicit MRI methods will need to update MRIStepCreate and MRIStepReInit
calls to pass NULL for either the explicit or implicit right-hand side
function as appropriate. If ARKStep is used as the fast time scale integrator,
codes will need to call ARKStepCreateMRIStepInnerStepper to wrap the ARKStep
memory as an MRIStepInnerStepper object. Additionally, MRIStepGetNumRhsEvals
has been updated to return the number of slow implicit and explicit function
evaluations. The coupling table structure MRIStepCouplingMem and the
functions MRIStepCoupling_Alloc and MRIStepCoupling_Create have also
been updated to support IMEX-MRI-GARK methods.

The deprecated functions MRIStepGetCurrentButcherTables and
MRIStepWriteButcher and the utility functions MRIStepSetTable and
MRIStepSetTableNum have been removed. Users wishing to create an MRI-GARK
method from a Butcher table should use MRIStepCoupling_MIStoMRI to create
the corresponding MRI coupling table and attach it with MRIStepSetCoupling.

The implementation of solve-decoupled implicit MRI-GARK methods has been updated
to remove extraneous slow implicit function calls and reduce the memory
requirements.

Deprecated ARKODE nonlinear solver predictors: specification of the ARKStep
"bootstrap" or "minimum correction" predictors (options 4 and 5 from
ARKStepSetPredictorMethod), or MRIStep "bootstrap" predictor (option 4 from
MRIStepSetPredictorMethod), will output a deprecation warning message.
These options will be removed in a future release.

The previously deprecated functions ARKStepSetMaxStepsBetweenLSet and
ARKStepSetMaxStepsBetweenJac have been removed and replaced with
ARKStepSetLSetupFrequency and ARKStepSetMaxStepsBetweenJac respectively.

CVODE

The previously deprecated function CVodeSetMaxStepsBetweenJac has been removed
and replaced with CVodeSetJacEvalFrequency.

CVODES

Added a new function CVodeGetLinSolveStats to get the CVODES linear solver
statistics as a group.

Added a new function, CVodeSetMonitorFn, that takes a user-function
to be called by CVODES after every nst successfully completed time-steps.
This is intended to provide a way of monitoring the CVODES statistics
throughout the simulation.

The previously deprecated function CVodeSetMaxStepsBetweenJac has been removed
and replaced with CVodeSetJacEvalFrequency.

KINSOL

New orthogonalization methods were added for use within Anderson acceleration
in KINSOL. See the "Anderson Acceleration QR Factorization" subsection within
the mathematical considerations chapter of the user guide and the
KINSetOrthAA function documentation for more details.

Deprecations

In addition to the deprecations noted elsewhere, many constants, types, and
functions have been renamed so that they are properly namespaced. The old names
have been deprecated and will be removed in SUNDIALS v7.0.0.

The following constants, macros, and typedefs are now deprecated:

Deprecated Name	New Name
realtype	sunrealtype
booleantype	sunbooleantype
RCONST	SUN_RCONST
BIG_REAL	SUN_BIG_REAL ...

SUNDIALS v5.8.0

Changes to SUNDIALS in release 5.8.0

The RAJA NVECTOR implementation has been updated to support the SYCL backend in addition to the CUDA and HIP backend. Users can choose the backend when configuring SUNDIALS by using the SUNDIALS_RAJA_BACKENDS CMake variable. This module remains experimental and is subject to change from version to version.

A new SUNMatrix and SUNLinearSolver implementation were added to interface with the Intel oneAPI Math Kernel Library (oneMKL). Both the matrix and the linear solver support general dense linear systems as well as block diagonal linear systems. This module is experimental and is subject to change from version to version.

Added a new optional function to the SUNLinearSolver API, SUNLinSolSetZeroGuess, to indicate that the next call to SUNlinSolSolve will be made with a zero initial guess. SUNLinearSolver implementations that do not use the SUNLinSolNewEmpty constructor will, at a minimum, need set the setzeroguess function pointer in the linear solver ops structure to NULL. The SUNDIALS iterative linear solver implementations have been updated to leverage this new set function to remove one dot product per solve.

The time integrator packages (ARKODE, CVODE(S), and IDA(S)) all now support a new "matrix-embedded" SUNLinearSolver type. This type supports user-supplied SUNLinearSolver implementations that set up and solve the specified linear system at each linear solve call. Any matrix-related data structures are held internally to the linear solver itself, and are not provided by the SUNDIALS package.

Added functions to ARKODE and CVODE(S) for supplying an alternative right-hand side function and to IDA(S) for supplying an alternative residual for use within nonlinear system function evaluations.

Support for user-defined inner (fast) integrators has been to the MRIStep module in ARKODE. See the "MRIStep Custom Inner Steppers" section in the user guide for more information on providing a user-defined integration method.

Added specialized fused HIP kernels to CVODE which may offer better performance on smaller problems when using CVODE with the NVECTOR_HIP module. See the optional input function CVodeSetUseIntegratorFusedKernels for more information. As with other SUNDIALS HIP features, this is feature is experimental and may change from version to version.

New KINSOL options have been added to apply a constant damping factor in the fixed point and Picard iterations (see KINSetDamping), to delay the start of Anderson acceleration with the fixed point and Picard iterations (see KINSetDelayAA), and to return the newest solution with the fixed point iteration (see KINSetReturnNewest).

The installed SUNDIALSConfig.cmake file now supports the COMPONENTS option to find_package. The exported targets no longer have IMPORTED_GLOBAL set.

A bug was fixed in SUNMatCopyOps where the matrix-vector product setup function pointer was not copied.

A bug was fixed in the SPBCGS and SPTFQMR solvers for the case where a non-zero initial guess and a solution scaling vector are provided. This fix only impacts codes using SPBCGS or SPTFQMR as standalone solvers as all SUNDIALS packages utilize a zero initial guess.

A bug was fixed in the ARKODE stepper modules where the stop time may be passed after resetting the integrator.

A bug was fixed in IDASetJacTimesResFn in IDAS where the supplied function was used in the dense finite difference Jacobian computation rather than the finite difference Jacobian-vector product approximation.

A bug was fixed in the KINSOL Picard iteration where the value of KINSetMaxSetupCalls would be ignored.

SUNDIALS v5.7.0

Changes to SUNDIALS in release 5.7.0

A new NVECTOR implementation based on the SYCL abstraction layer has been added targeting Intel GPUs. At present the only SYCL compiler supported is the DPC++ (Intel oneAPI) compiler. See the SYCL NVECTOR section in the user guide for more details.

A new SUNMatrix and SUNLinearSolver implementation were added to interface with the MAGMA linear algebra library. Both the matrix and the linear solver support general dense linear systems as well as block diagonal linear systems, and both are targeted at GPUs (AMD or NVIDIA).

These features are considered experimental and are subject to major changes even in minor releases.

SUNDIALS v5.6.1

Changes to SUNDIALS in release 5.6.1

Fixed a bug in the SUNDIALS CMake which caused an error
if the CMAKE_CXX_STANDARD and SUNDIALS_RAJA_BACKENDS options
were not provided.

Fixed some compiler warnings when using the IBM XL compilers.