diff --git a/joss.06605/10.21105.joss.06605.crossref.xml b/joss.06605/10.21105.joss.06605.crossref.xml new file mode 100644 index 0000000000..723c1fd4d5 --- /dev/null +++ b/joss.06605/10.21105.joss.06605.crossref.xml @@ -0,0 +1,230 @@ + + + + 20240619001026-8c71e669db341366d002709f14cf4e303f1377e2 + 20240619001026 + + JOSS Admin + admin@theoj.org + + The Open Journal + + + + + Journal of Open Source Software + JOSS + 2475-9066 + + 10.21105/joss + https://joss.theoj.org + + + + + 06 + 2024 + + + 9 + + 98 + + + + DynamicOED.jl: A Julia package for solving optimum +experimental design problems + + + + Carl Julius + Martensen + https://orcid.org/0000-0003-4143-3040 + + + Christoph + Plate + https://orcid.org/0000-0003-0354-8904 + + + Sebastian + Sager + https://orcid.org/0000-0002-0283-9075 + + + + 06 + 19 + 2024 + + + 6605 + + + 10.21105/joss.06605 + + + http://creativecommons.org/licenses/by/4.0/ + http://creativecommons.org/licenses/by/4.0/ + http://creativecommons.org/licenses/by/4.0/ + + + + Software archive + 10.5281/zenodo.12084055 + + + GitHub review issue + https://github.com/openjournals/joss-reviews/issues/6605 + + + + 10.21105/joss.06605 + https://joss.theoj.org/papers/10.21105/joss.06605 + + + https://joss.theoj.org/papers/10.21105/joss.06605.pdf + + + + + + Julia: A fresh approach to numerical +computing + Bezanson + SIAM Rev. + 1 + 59 + 10.1137/141000671 + 0036-1445 + 2017 + Bezanson, J., Edelman, A., Karpinski, +S., & Shah, V. B. (2017). Julia: A fresh approach to numerical +computing. SIAM Rev., 59(1), 65–98. +https://doi.org/10.1137/141000671 + + + DifferentialEquations.jl – a performant and +feature-rich ecosystem for solving differential equations in +Julia + Rackauckas + The Journal of Open Research +Software + 1 + 5 + 10.5334/jors.151 + 2017 + Rackauckas, C., & Nie, Q. (2017). +DifferentialEquations.jl – a performant and feature-rich ecosystem for +solving differential equations in Julia. The Journal of Open Research +Software, 5(1). https://doi.org/10.5334/jors.151 + + + Optimization.jl: A unified optimization +package + Dixit + 10.5281/zenodo.7738525 + 2023 + Dixit, V. K., & Rackauckas, C. +(2023). Optimization.jl: A unified optimization package. Zenodo. +https://doi.org/10.5281/zenodo.7738525 + + + Numerische methoden für optimale +versuchsplanungsprobleme bei nichtlinearen DAE-Modellen + Körkel + 10.11588/heidok.00002980 + 2002 + Körkel, S. (2002). Numerische +methoden für optimale versuchsplanungsprobleme bei nichtlinearen +DAE-Modellen [PhD thesis, Universität Heidelberg]. +https://doi.org/10.11588/heidok.00002980 + + + Juniper: An open-source nonlinear +branch-and-bound solver in Julia + Kröger + Integration of Constraint Programming, +Artificial Intelligence, and Operations Research + 10.1007/978-3-319-93031-2_27 + 978-3-319-93031-2 + 2018 + Kröger, O., Coffrin, C., Hijazi, H., +& Nagarajan, H. (2018). Juniper: An open-source nonlinear +branch-and-bound solver in Julia. In W.-J. van Hoeve (Ed.), Integration +of Constraint Programming, Artificial Intelligence, and Operations +Research (pp. 377–386). Springer International Publishing. +https://doi.org/10.1007/978-3-319-93031-2_27 + + + Sensitivity analysis of +differential–algebraic equations: A comparison of methods on a special +problem + Li + Applied Numerical Mathematics + 2 + 32 + 10.1016/S0168-9274(99)00020-3 + 0168-9274 + 2000 + Li, S., Petzold, L., & Zhu, W. +(2000). Sensitivity analysis of differential–algebraic equations: A +comparison of methods on a special problem. Applied Numerical +Mathematics, 32(2), 161–174. +https://doi.org/10.1016/S0168-9274(99)00020-3 + + + ModelingToolkit: A composable graph +transformation system for equation-based modeling + Ma + 10.48550/arXiv.2103.05244 + 2022 + Ma, Y., Gowda, S., Anantharaman, R., +Laughman, C., Shah, V., & Rackauckas, C. (2022). ModelingToolkit: A +composable graph transformation system for equation-based modeling (No. +arXiv:2103.05244). arXiv. +https://doi.org/10.48550/arXiv.2103.05244 + + + Sampling decisions in optimum experimental +design in the light of Pontryagin’s maximum principle + Sager + SIAM J. Control Optim. + 4 + 51 + 10.1137/110835098 + 0363-0129 + 2013 + Sager, S. (2013). Sampling decisions +in optimum experimental design in the light of Pontryagin’s maximum +principle. SIAM J. Control Optim., 51(4), 3181–3207. +https://doi.org/10.1137/110835098 + + + On the implementation of an interior-point +filter line-search algorithm for large-scale nonlinear +programming + Wächter + Mathematical Programming + 1 + 106 + 10.1007/s10107-004-0559-y + 2006 + Wächter, A., & Biegler, L. T. +(2006). On the implementation of an interior-point filter line-search +algorithm for large-scale nonlinear programming. Mathematical +Programming, 106(1), 25–57. +https://doi.org/10.1007/s10107-004-0559-y + + + + + + diff --git a/joss.06605/10.21105.joss.06605.pdf b/joss.06605/10.21105.joss.06605.pdf new file mode 100644 index 0000000000..32df6df033 Binary files /dev/null and b/joss.06605/10.21105.joss.06605.pdf differ diff --git a/joss.06605/paper.jats/10.21105.joss.06605.jats b/joss.06605/paper.jats/10.21105.joss.06605.jats new file mode 100644 index 0000000000..8894220e9b --- /dev/null +++ b/joss.06605/paper.jats/10.21105.joss.06605.jats @@ -0,0 +1,507 @@ + + +
+ + + + +Journal of Open Source Software +JOSS + +2475-9066 + +Open Journals + + + +6605 +10.21105/joss.06605 + +DynamicOED.jl: A Julia package for solving optimum +experimental design problems + + + +https://orcid.org/0000-0003-4143-3040 + +Martensen +Carl Julius + + +* + + +https://orcid.org/0000-0003-0354-8904 + +Plate +Christoph + + + + +https://orcid.org/0000-0002-0283-9075 + +Sager +Sebastian + + + + + +Otto von Guericke University Magdeburg, +Germany + + + + +* E-mail: + + +14 +2 +2023 + +9 +98 +6605 + +Authors of papers retain copyright and release the +work under a Creative Commons Attribution 4.0 International License (CC +BY 4.0) +2022 +The article authors + +Authors of papers retain copyright and release the work under +a Creative Commons Attribution 4.0 International License (CC BY +4.0) + + + +Julia +optimization +experimental design +parameter estimation + + + + + + Summary +

Optimum experimental design (OED) problems are typically + encountered when unknown or uncertain parameters of mathematical + models are to be estimated from an observable, maybe even + controllable, process. In this scenario, OED can be used to decide on + an experimental setup before collecting the data, i.e., deciding on + when to measure and / or how to stimulate a dynamic process in order + to maximize the amount of information gathered such that the + parameters can be accurately estimated.

+

Our software package, DynamicOED.jl, facilitates the solution of + optimum experimental design problems for dynamical systems. Following + ideas presented in Sager + (2013), we + cast the OED problem into an optimal control problem. This is done by + augmenting the user-provided system of ordinary differential equations + (ODE) or differential algebraic equations (DAE) with their variational + differential (algebraic) equations and the differential equation + governing the evolution of the Fisher information matrix (FIM). A + suitable criterion based on the FIM is then optimized in the resulting + optimal control problem using a direct first discretize, then + optimize approach.

+ + + Statement of need +

DynamicOED.jl is a Julia + (Bezanson + et al., 2017) package for solving optimum experimental design + problems. Solving OED problems is of interest for several reasons. + First, all model-based optimization strategies rely on the knowledge + of the accurate values of the model’s parameters. Second, computing + optimal experimental designs before performing the actual experiments + to collect data allows to reduce the number of needed experiments or + measurements. This is important in practical applications when + measuring quantities of interest is only possible to a limited extent, + e.g., due to high costs of performing the measurements.

+

Our package is designed for high flexibility and ease of use. For + formulating the underlying dynamical system, our package is based on + the ODESystem from + ModelingToolkit.jl + (Ma + et al., 2022). This enables researchers and modelers to easily + investigate and analyze their models and allows them to collect + insightful data for their parameter estimation problems.

+

To our knowledge, this is the first dedicated package for solving + general optimal experimental design problems with dynamical systems + written in the Julia programming language. It may therefore be a + valuable resource to different communities dealing with experimental + data and parameter estimation problems.

+ + + Problem statement and usage example +

The problem we are interested in solving reads + where + + 𝒯=[t0,tf] + is the fixed time horizon and + + x:𝒯↦ℝnx + are the differential states. The first and second constraint denote + the dynamical system and the sensitivities of the solution of the + dynamical system with respect to the uncertain parameters, + respectively, and are given in an implicit form. Here, + + + fẋ, + ( + + fx) + denote the partial derivative of + + f + with respect to + + ẋ + and ( + + x). + The objective + + ϕ(F(tf)) + of Bolza type is a suited objective function, e.g., the D-criterion + + + ϕ(F(tf))=det(F−1(tf)). + The evolution of the symmetric FIM + + F:𝒯↦ℝnp×np + is governed by the measurement function + + h:ℝnx↦ℝnh, + the sensitivities + + G:𝒯↦ℝnx×np + and the sampling decisions + + w(t)∈{0,1}nh. + The latter are the main optimization variables and represent the + decision whether to measure at a given time point or not. In our + direct approach, these variables are discretized, hence we write + + + w(t)∈{0,1}Nw×nh, + where + + Nw + is the (user-supplied) number of discretization intervals on + + + 𝒯. + The sampling decisions are then accumulated in the variables + + + z + and constrained by + + M∈ℝ+nh. + The controls + + u∈𝒰 + can either be fixed or also be viewed as optimization variables after + discretization.

+

For more information on optimal experimental design for DAEs and + their sensitivity analysis, we refer to Körkel + (2002) + and Li et al. + (2000).

+

The functionality in this package integrates into Julia’s + SciML + ecosystem. The model is provided in symbolic form as an + ODESystem using + ModelingToolkit.jl + (Ma + et al., 2022) with additional frequency information for the + observed and control variables. Both ODE or DAE systems can be + provided. DynamicOED.jl augments the given + system symbolically with its sensitivity equations and the dynamics of + the FIM. The resulting system together with a sufficient information + criterion defines an OEDProblem, solveable + using DifferentialEquations.jl + (Rackauckas + & Nie, 2017). Here, all sampling and control decisions are + discretized in time and can be used to model additional constraints. + At last, the OEDProblem can be transformed into + an OptimizationProblem as a sufficient input to + Optimization.jl + (Dixit + & Rackauckas, 2023). Here, a variety of optimization + solvers for nonlinear programming and mixed-integer nonlinear + programming available as additional backends, e.g., + Juniper + (Kröger + et al., 2018) or Ipopt + (Wächter + & Biegler, 2006). A simple example demonstrates the usage + of DynamicOED.jl for the Lotka-Volterra system + (Sager, + 2013).

+

[fig:lotka] + shows the solution of the example above including the differential + states, sensitivities + + G + and the sampling decisions + + w. + More examples can be found in the + documentation.

+ using DynamicOED +using ModelingToolkit +using Optimization, OptimizationMOI, Ipopt + +@variables t +@variables x(t)=0.5 [description="Biomass Prey"] +@variables y(t)=0.7 [description="Biomass Predator"] +@variables u(t) [description="Control"] +@parameters p[1:2]=[1.0;1.0] [description="Fixed Parameters", tunable=false] +@parameters p_est[1:2]=[1.0;1.0] [description="Tunable Parameters", tunable=true] +D = Differential(t) +@variables obs(t)[1:2] [description = "Observed", measurement_rate=96] +obs = collect(obs) + +@named lotka_volterra = ODESystem( + [ + D(x) ~ p[1]*x - p_est[1]*x*y; + D(y) ~ -p[2]*y + p_est[2]*x*y + ], tspan = (0.0, 12.0), + observed = obs .~ [x; y] +) +@named oed_system = OEDSystem(lotka_volterra) +oed_problem = OEDProblem(structural_simplify(oed_system), DCriterion()) + +optimization_variables = states(oed_problem) + +w1, w2 = keys(optimization_variables.measurements) + +constraint_equations = [ + sum(optimization_variables.measurements[w1]) ≲ 32, + sum(optimization_variables.measurements[w2]) ≲ 32, +] + +@named constraint_system = ConstraintsSystem( + constraint_equations, optimization_variables, Num[] +) + +optimization_problem = OptimizationProblem( + oed_problem, AutoForwardDiff(), constraints = constraint_system, + integer_constraints = false +) + +optimal_design = solve(optimization_problem, Ipopt.Optimizer(); + hessian_approximation="limited-memory") + + + + Extensions +

Several extensions are planned for the future. First, a multiple + shooting approach is planned. Also, other steps to increase the + efficiency of our implementation may be considered. For example, in + the case of fixed initial values and controls, the integration of + + + x + and + + G + need to be done only once and can be decoupled from the numerical + integration of + + F + and the subsequent optimization over + + w.

+ +

Differential states, sensitivities of the states with + respect to the parameters and the optimal sampling design for + Lotka-Volterra system. +

+ + + + + Acknowledgements +

The work was funded by the German Research Foundation DFG within + the priority program 2331 ‘Machine Learning in Chemical Engineering’ + under grants KI 417/9-1, SA 2016/3-1, SE 586/25-1

+ + + + + + + + + BezansonJeff + EdelmanAlan + KarpinskiStefan + ShahViral B. + + Julia: A fresh approach to numerical computing + SIAM Rev. + Society for Industrial and Applied Mathematics + 201701 + 20240315 + 59 + 1 + 0036-1445 + 10.1137/141000671 + 65 + 98 + + + + + + RackauckasChristopher + NieQing + + DifferentialEquations.jl – a performant and feature-rich ecosystem for solving differential equations in Julia + The Journal of Open Research Software + 2017 + 5 + 1 + 10.5334/jors.151 + + + + + + DixitVaibhav Kumar + RackauckasChristopher + + Optimization.jl: A unified optimization package + Zenodo + 202303 + 10.5281/zenodo.7738525 + + + + + + KörkelS. + + Numerische methoden für optimale versuchsplanungsprobleme bei nichtlinearen DAE-Modellen + Universität Heidelberg + Heidelberg + 2002 + 10.11588/heidok.00002980 + + + + + + KrögerOle + CoffrinCarleton + HijaziHassan + NagarajanHarsha + + Juniper: An open-source nonlinear branch-and-bound solver in Julia + Integration of Constraint Programming, Artificial Intelligence, and Operations Research + + van HoeveWillem-Jan + + Springer International Publishing + Cham + 2018 + 978-3-319-93031-2 + 10.1007/978-3-319-93031-2_27 + 377 + 386 + + + + + + LiShengtai + PetzoldLinda + ZhuWenjie + + Sensitivity analysis of differential–algebraic equations: A comparison of methods on a special problem + Applied Numerical Mathematics + 200002 + 20230614 + 32 + 2 + 0168-9274 + 10.1016/S0168-9274(99)00020-3 + 161 + 174 + + + + + + MaYingbo + GowdaShashi + AnantharamanRanjan + LaughmanChris + ShahViral + RackauckasChris + + ModelingToolkit: A composable graph transformation system for equation-based modeling + arXiv + 202202 + 20240315 + https://arxiv.org/abs/2103.05244 + 10.48550/arXiv.2103.05244 + + + + + + SagerSebastian + + Sampling decisions in optimum experimental design in the light of Pontryagin’s maximum principle + SIAM J. Control Optim. + 201301 + 20231106 + 51 + 4 + 0363-0129 + 10.1137/110835098 + 3181 + 3207 + + + + + + WächterA. + BieglerL. T. + + On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming + Mathematical Programming + 2006 + 106 + 1 + 10.1007/s10107-004-0559-y + 25 + 57 + + + + +
diff --git a/joss.06605/paper.jats/lotka.png b/joss.06605/paper.jats/lotka.png new file mode 100644 index 0000000000..a33857c1cd Binary files /dev/null and b/joss.06605/paper.jats/lotka.png differ