diff --git a/joss.06326/10.21105.joss.06326.crossref.xml b/joss.06326/10.21105.joss.06326.crossref.xml new file mode 100644 index 0000000000..c3551f8c97 --- /dev/null +++ b/joss.06326/10.21105.joss.06326.crossref.xml @@ -0,0 +1,314 @@ + + + + 20240313T193609-27fc01c0bd885340c576af853ba600367661fecc + 20240313193609 + + JOSS Admin + admin@theoj.org + + The Open Journal + + + + + Journal of Open Source Software + JOSS + 2475-9066 + + 10.21105/joss + https://joss.theoj.org + + + + + 03 + 2024 + + + 9 + + 95 + + + + PyProximal - scalable convex optimization in +Python + + + + Matteo + Ravasi + https://orcid.org/0000-0003-0020-2721 + + + Marcus Valtonen + Örnhag + https://orcid.org/0000-0001-8687-227X + + + Nick + Luiken + https://orcid.org/0000-0003-3307-1748 + + + Olivier + Leblanc + https://orcid.org/0000-0003-3641-1875 + + + Eneko + Uruñuela + https://orcid.org/0000-0001-6849-9088 + + + + 03 + 13 + 2024 + + + 6326 + + + 10.21105/joss.06326 + + + http://creativecommons.org/licenses/by/4.0/ + http://creativecommons.org/licenses/by/4.0/ + http://creativecommons.org/licenses/by/4.0/ + + + + Software archive + 10.5281/zenodo.10805997 + + + GitHub review issue + https://github.com/openjournals/joss-reviews/issues/6326 + + + + 10.21105/joss.06326 + https://joss.theoj.org/papers/10.21105/joss.06326 + + + https://joss.theoj.org/papers/10.21105/joss.06326.pdf + + + + + + PyLops - A linear-operator Python library for +scalable algebra and optimization + Ravasi + SoftwareX + 11 + 10.1016/j.softx.2019.100361 + 2020 + Ravasi, M., & Vasconcelos, I. +(2020). PyLops - A linear-operator Python library for scalable algebra +and optimization. SoftwareX, 11. +https://doi.org/10.1016/j.softx.2019.100361 + + + Parikh + 10.1561/2400000003 + 2013 + Parikh, N. (2013). Foundations; +Trends in Optimization. +https://doi.org/10.1561/2400000003 + + + Proximal splitting methods in signal +processing + Combettes + 10.1007/978-1-4419-9569-8_10 + 2011 + Combettes, P., & Pesquet, J.-C. +(2011). Proximal splitting methods in signal processing. Springer +Optimization; Its Applications. +https://doi.org/10.1007/978-1-4419-9569-8_10 + + + Distributed optimization and statistical +learning via the alternating direction method of +multipliers + Boyd + Foundations and Trends in Machine +Learning + 3 + 10.1561/2200000016 + 2011 + Boyd, S., Parikh, N., Chu, E., +Peleato, B., & Eckstein, J. (2011). Distributed optimization and +statistical learning via the alternating direction method of +multipliers. Foundations and Trends in Machine Learning, 3. +https://doi.org/10.1561/2200000016 + + + A first-order primal-dual algorithm for +convex problems with applications to imaging + Chambolle + Journal of Mathematical Imaging and +Vision + 40 + 10.1007/s10851-010-0251-1 + 2011 + Chambolle, A., & Pock, T. (2011). +A first-order primal-dual algorithm for convex problems with +applications to imaging. Journal of Mathematical Imaging and Vision, 40. +https://doi.org/10.1007/s10851-010-0251-1 + + + A joint inversion-segmentation approach to +assisted seismic interpretation + Ravasi + Geophysical Journal +International + 228 + 10.1093/gji/ggab388 + 2022 + Ravasi, M., & Birnie, C. (2022). +A joint inversion-segmentation approach to assisted seismic +interpretation. Geophysical Journal International, 228. +https://doi.org/10.1093/gji/ggab388 + + + Plug-and-Play Priors for Model Based +Reconstruction + Venkatakrishnan + 2013 IEEE Global Conference on Signal and +Information Processing + 10.1109/GlobalSIP.2013.6737048 + 2013 + Venkatakrishnan, S. V., Bouman, C. +A., & Wohlberg, B. (2013). Plug-and-Play Priors for Model Based +Reconstruction. 2013 IEEE Global Conference on Signal and Information +Processing. +https://doi.org/10.1109/GlobalSIP.2013.6737048 + + + Seeing through the CO2 plume: Joint +inversion-segmentation of the Sleipner 4D seismic data +set + Romero + The Leading Edge + 42 + 10.1190/tle42070457.1 + 2023 + Romero, J., Luiken, N., & Ravasi, +M. (2023). Seeing through the CO2 plume: Joint inversion-segmentation of +the Sleipner 4D seismic data set. The Leading Edge, 42. +https://doi.org/10.1190/tle42070457.1 + + + Plug and Play Post-Stack Seismic Inversion +with CNN-Based Denoisers + Romero + Second EAGE Subsurface Intelligence +Workshop + 1 + 10.3997/2214-4609.2022616015 + 2022 + Romero, J., Luiken, M. C. N., & +Ravasi, M. (2022). Plug and Play Post-Stack Seismic Inversion with +CNN-Based Denoisers. Second EAGE Subsurface Intelligence Workshop, 1. +https://doi.org/10.3997/2214-4609.2022616015 + + + Interferometric Lensless Imaging: Rank-one +Projections of Image Frequencies with Speckle +Illuminations + Leblanc + ArXiv e-prints + 10.1109/tci.2024.3359178 + 2023 + Leblanc, O., Hofer, M., Sivankutty, +S., Rigneault, H., & Jacques, L. (2023). Interferometric Lensless +Imaging: Rank-one Projections of Image Frequencies with Speckle +Illuminations. ArXiv e-Prints. +https://doi.org/10.1109/tci.2024.3359178 + + + Leveraging GPUs for matrix-free optimization +with PyLops + Ravasi + Fifth EAGE Workshop on High Performance +Computing for Upstream + 1 + 10.3997/2214-4609.2021612003 + 2021 + Ravasi, M. (2021). Leveraging GPUs +for matrix-free optimization with PyLops. Fifth EAGE Workshop on High +Performance Computing for Upstream, 1. +https://doi.org/10.3997/2214-4609.2021612003 + + + Proxalgs + Maheswaranathan + 2019 + Maheswaranathan, N., Zapp, S., & +Poole, B. (2019). Proxalgs. +https://github.com/ganguli-lab/proxalgs/ + + + Block-simultaneous direction method of +multipliers: a proximal primal-dual splitting algorithm for nonconvex +problems with multiple constraints + Moolekamp + Optimization and Engineering + 19 + 10.1007/s11081-018-9380-y + 2018 + Moolekamp, F., & Melchior, P. +(2018). Block-simultaneous direction method of multipliers: a proximal +primal-dual splitting algorithm for nonconvex problems with multiple +constraints. Optimization and Engineering, 19. +https://doi.org/10.1007/s11081-018-9380-y + + + The proximity operator +repository + Chierchia + 2024 + Chierchia, G., Chouzenoux, E., +Combettes, P. L., & Pesquet, J.-C. (2024). The proximity operator +repository. https://proximity-operator.net/ + + + Pyxu-org/pyxu: pyxu + Simeoni + 10.5281/zenodo.4486431 + 2024 + Simeoni, M., Kashani, S., +Rué-Queralt, J., & Developers, P. (2024). Pyxu-org/pyxu: pyxu. +Zenodo. https://doi.org/10.5281/zenodo.4486431 + + + ProxImaL: Efficient image optimization using +proximal algorithms + Heide + 4 + 35 + 10.1145/2897824.2925875 + 2016 + Heide, F., Diamond, S., Nießner, M., +Ragan-Kelley, J., Heidrich, W., & Wetzstein, G. (2016). ProxImaL: +Efficient image optimization using proximal algorithms. 35(4). +https://doi.org/10.1145/2897824.2925875 + + + + + + diff --git a/joss.06326/10.21105.joss.06326.jats b/joss.06326/10.21105.joss.06326.jats new file mode 100644 index 0000000000..a533b27974 --- /dev/null +++ b/joss.06326/10.21105.joss.06326.jats @@ -0,0 +1,618 @@ + + +
+ + + + +Journal of Open Source Software +JOSS + +2475-9066 + +Open Journals + + + +6326 +10.21105/joss.06326 + +PyProximal - scalable convex optimization in +Python + + + +https://orcid.org/0000-0003-0020-2721 + +Ravasi +Matteo + + +* + + +https://orcid.org/0000-0001-8687-227X + +Örnhag +Marcus Valtonen + + + + +https://orcid.org/0000-0003-3307-1748 + +Luiken +Nick + + + + +https://orcid.org/0000-0003-3641-1875 + +Leblanc +Olivier + + + + +https://orcid.org/0000-0001-6849-9088 + +Uruñuela +Eneko + + + + + +Earth Science and Engineering, Physical Sciences and +Engineering (PSE), King Abdullah University of Science and Technology +(KAUST), Thuwal, Kingdom of Saudi Arabia + + + + +Ericsson Research, Lund, Sweden. + + + + +ISPGroup, INMA/ICTEAM, UCLouvain, Louvain-la-Neuve, +Belgium. + + + + +Basque Center on Cognition, Brain and Language (BCBL), +Donostia-San Sebastián, Spain. + + + + +* E-mail: + + +19 +12 +2023 + +9 +95 +6326 + +Authors of papers retain copyright and release the +work under a Creative Commons Attribution 4.0 International License (CC +BY 4.0) +2022 +The article authors + +Authors of papers retain copyright and release the work under +a Creative Commons Attribution 4.0 International License (CC BY +4.0) + + + +Python +convex optimization +proximal + + + + + + Summary +

A broad class of problems in scientific disciplines ranging from + image processing and astrophysics, to geophysics and medical imaging + call for the optimization of convex, non-smooth objective functions. + Whereas practitioners are usually familiar with gradient-based + algorithms, commonly used to solve unconstrained, smooth optimization + problems, proximal algorithms can be viewed as analogous tools for + non-smooth and possibly constrained versions of such problems. These + algorithms sit at a higher level of abstraction than gradient-based + algorithms and require a basic operation to be performed at each + iteration: the evaluation of the so-called proximal operator of the + functional to be optimized. PyProximal is a + Python-based library aimed at democratizing the application of convex + optimization to scientific problems; it provides the required building + blocks (i.e., proximal operators and algorithms) to define and solve + complex, convex objective functions in a high-level, abstract fashion, + shielding users away from any unneeded mathematical and implementation + details.

+ + + Statement of need +

PyProximal is a Python library for convex + optimization, developed as an integral part of the + PyLops framework. It provides practitioners + with an easy-to-use framework to define and solve composite convex + objective functions arising in many modern inverse problems. Its API + is designed to offer a class-based and user-friendly interface to + proximal operators, coupled with function-based optimizers; because of + its modular design, researchers in the field of convex optimization + can also benefit from this library in a number of ways when developing + new algorithms: first, they can easily include their newly developed + proximal operators and solvers; second, they can compare these methods + with state-of-the-art algorithms already provided in the library.

+

Several projects in the Python ecosystem provide implementations of + proximal operators and/or algorithms, which present some overlap with + those available in PyProximal. A (possibly not + exhaustive) list of other projects is composed of + proxalgs + (Maheswaranathan + et al., 2019), proxmin + (Moolekamp + & Melchior, 2018), The Proximity Operator + Repository + (Chierchia + et al., 2024), ProxImaL + (Heide + et al., 2016), and pyxu + (Simeoni + et al., 2024). A key common feature of all of the above + mentioned packages is to be self-contained; as such, not only proximal + operators and solvers are provided, but also linear operators that are + useful for the applications that the package targets. Moreover, to the + best of our knowledge, all of these packages provide purely CPU-based + implementations (apart from pyxu). On the other hand, + PyProximal heavily relies on and seamlessly + integrates with PyLops + (Ravasi + & Vasconcelos, 2020), a Python library for matrix-free + linear algebra and optimization. As such, it can easily handle + problems with millions of unknowns and inherits the interchangle + CPU/GPU backend of PyLops + (Ravasi, + 2021). More specifically, PyLops is + leveraged in the implementation of proximal operators that require + access to linear operators (e.g., numerical derivatives) and/or + least-squares solvers (e.g., conjugate gradient). Whilst libraries + with similar capabilities exist in the Python ecosystem, their design + usually leads to a tight coupling between linear and proximal + operators, and their respective solvers. On the other hand, by + following the Separation of Concerns (SoC) design principle, the + overlap between PyLops and + PyProximal is reduced to a minimum, easing both + their development and maintenance, as well as allowing newcomers to + learn how to solve inverse problems in a step-by-step fashion. As + such, PyProximal can be ultimately described as + a light-weight extension of PyLops that users + of the latter can easily learn and adopt with minimal additional + effort.

+ + + Mathematical framework +

Convex optimization is routinely used to solve problems of the form + (Parikh, + 2013):

+

+ + min𝐱f(𝐱)+g(𝐋𝐱)

+

where + + f + and + + g + are possibly non-smooth convex functionals and + + + 𝐋 + is a linear operator. A special case appearing in many scientific + applications is represented by + + f=1/2‖𝐲−𝒜(𝐱)‖22. + Here, + + 𝒜 + is a (possibly non-linear) modeling operator, describing the + underlying physical process that links the unknown model vector + + + 𝐱 + to the vector of observations + + 𝐲. + In this case, we usually refer to + + g + as the regularizer, where one or multiple functions are added to the + data misfit term to promote certain features in the sought after + solution and/or constraint the solution to fall within a given set of + allowed vectors.

+

A common feature of all proximal algorithms is represented by the + fact that one must be able to repeatedly evaluate the proximal + operator of + + f + and/or + + g. + The proximal operator of a function + + f + is defined as

+

+ + proxτf(𝐱)=min𝐲f(𝐲)+12τ||𝐲−𝐱||22

+

Whilst evaluating a proximal operator does itself require solving + an optimization problem, these problems often admit closed form + solutions or can be solved very efficiently with ad-hoc specialized + methods. Several of such proximal operators are efficiently + implemented in the PyProximal library.

+

Finally, there exists three main families of proximal algorithms + that can be used to solve various flavors of + Equation 1, + namely:

+ + +

Proximal Gradient + (Combettes + & Pesquet, 2011): this method, also commonly referred + to as the Forward-Backward Splitting (FBS) algorithm, is usually + the preferred choice when + + 𝐋=𝐈 + (i.e. identity operator). Accelerated versions such as the FISTA + and TwIST algorithms exist and are usually preferred to the + vanilla FBS method;

+ + +

Alternating Direction Method of Multipliers + (Boyd + et al., 2011): this method is based on a splitting strategy + and can be used for a broader class of problem than FBS and its + accelerated versions.

+ + +

Primal-Dual + (Chambolle + & Pock, 2011): another popular algorithm able to tackle + problems in the form of + Equation 1 with + any choice of + + 𝐋. + It reformulates the original problem into its primal-dual version + and solves a saddle optimization problem.

+ + +

PyProximal provides implementations for + these three families of algorithms; moreover, all solvers include + additional features such as back-tracking for automatic selection of + step-sizes, logging of the objective function evolution through + iterations, and possibility to inject custom callbacks.

+ + + Code structure +

PyProximal’s modular and easy-to-use + Application Programming Interface (API) allows scientists to define + and solve convex objective functions by means of proximal algorithms. + The API is composed of two main part as shown in Fig. 1.

+

The first part contains the entire suite of proximal operators, + which are class-based objects subclassing the + pylops.ProxOperator parent class. For each of + these operators, the solution to the proximal optimization problem in + Equation 2 (and/or the + dual proximal problem) is implemented in the + prox (and/or dualprox) + method. As in most cases a closed-form solution exists for such a + problem, our implementation provides users with the most efficient way + to evaluate a proximal operator. The second part comprises of + so-called proximal solvers, optimization algorithms that are suited to + solve problems of the form in + Equation 1. + Finally, some specialized solvers that rely on one or more of the + previously described optimizers are also provided.

+ +

Schematic representation of the + PyProximal API.

+ + + + + Representative PyProximal Use Cases +

Examples of PyProximal applications in different scientific fields + include:

+ + +

Joint inversion and segmentation of subsurface + models: when inverting geophysical data for subsurface + priorities, prior information can be provided to inversion process + in the form of discrete number of rock units; this can be + parametrized in terms of their expected mean (or most likely + value). Ravasi & Birnie + (2022) + and Romero et al. + (2023) + frame such a problem as a joint inversion and segmentation task, + where the underlying objective function is optimized in + alternating fashion using the Primal-Dual algorithm.

+ + +

Plug-and-Play (PnP) priors: introduced in 2013 + by Venkatakrishnan et al. + (2013), + the PnP framework lays its foundation on the interpretation of the + proximal operator as a denoising problem; as such, powerful + statistical or deep learning based denoisers are used to evaluate + the proximal operator of implicit regularizers. Romero et al. + (2022) + applies this concept in the context of seismic inversion, + achieving results of superior quality in comparison to traditional + model-based regularization techniques.

+ + +

Multi-Core Fiber Lensless Imaging (MCFLI) is a + computational imaging technique to reconstruct biological samples + at cellular scale. Leveraging the rank-one projected + interferometric sensing of the MCFLI has been shown to improve the + efficiency of the acquisition process + (Leblanc + et al., 2023); this entails solving a regularized inverse + problem with the proximal gradient method. Depending on the image + to be reconstructed, the regularization term may for instance be + + + L1 + or TV.

+ + + + + + + + + + RavasiM. + VasconcelosI. + + PyLops - A linear-operator Python library for scalable algebra and optimization + SoftwareX + 2020 + 11 + https://www.sciencedirect.com/science/article/pii/S2352711019301086 + 10.1016/j.softx.2019.100361 + + + + + + ParikhN. + + Foundations; Trends in Optimization + 2013 + https://web.stanford.edu/~boyd/papers/pdf/prox_algs.pdf + 10.1561/2400000003 + + + + + + CombettesP. + PesquetJ.-C. + + Proximal splitting methods in signal processing + Springer Optimization; Its Applications + 2011 + https://link.springer.com/chapter/10.1007/978-1-4419-9569-8_10 + 10.1007/978-1-4419-9569-8_10 + + + + + + BoydS. + ParikhN. + ChuE. + PeleatoB. + EcksteinJ. + + Distributed optimization and statistical learning via the alternating direction method of multipliers + Foundations and Trends in Machine Learning + 2011 + 3 + https://web.stanford.edu/~boyd/papers/pdf/admm_distr_stats.pdf + 10.1561/2200000016 + + + + + + ChambolleA. + PockT. + + A first-order primal-dual algorithm for convex problems with applications to imaging + Journal of Mathematical Imaging and Vision + 2011 + 40 + https://link.springer.com/article/10.1007/s10851-010-0251-1 + 10.1007/s10851-010-0251-1 + + + + + + RavasiM. + BirnieC. + + A joint inversion-segmentation approach to assisted seismic interpretation + Geophysical Journal International + 2022 + 228 + https://academic.oup.com/gji/article/228/2/893/6374557 + 10.1093/gji/ggab388 + + + + + + VenkatakrishnanS. V + BoumanC. A. + WohlbergB. + + Plug-and-Play Priors for Model Based Reconstruction + 2013 IEEE Global Conference on Signal and Information Processing + 2013 + https://ieeexplore.ieee.org/document/6737048 + 10.1109/GlobalSIP.2013.6737048 + + + + + + RomeroJ. + LuikenN. + RavasiM. + + Seeing through the CO2 plume: Joint inversion-segmentation of the Sleipner 4D seismic data set + The Leading Edge + 2023 + 42 + https://library.seg.org/doi/full/10.1190/tle42070457.1 + 10.1190/tle42070457.1 + + + + + + RomeroJ. + LuikenM. Corrales N. + RavasiM. + + Plug and Play Post-Stack Seismic Inversion with CNN-Based Denoisers + Second EAGE Subsurface Intelligence Workshop + 2022 + 1 + https://www.earthdoc.org/content/papers/10.3997/2214-4609.2022616015 + 10.3997/2214-4609.2022616015 + + + + + + LeblancO. + HoferM. + SivankuttyS. + RigneaultH. + JacquesL. + + Interferometric Lensless Imaging: Rank-one Projections of Image Frequencies with Speckle Illuminations + ArXiv e-prints + 2023 + https://arxiv.org/abs/2306.12698 + 10.1109/tci.2024.3359178 + + + + + + RavasiM. + + Leveraging GPUs for matrix-free optimization with PyLops + Fifth EAGE Workshop on High Performance Computing for Upstream + 2021 + 1 + https://www.earthdoc.org/content/papers/10.3997/2214-4609.2021612003 + 10.3997/2214-4609.2021612003 + + + + + + MaheswaranathanN. + ZappS. + PooleB. + + Proxalgs + 2019 + https://github.com/ganguli-lab/proxalgs/ + + + + + + MoolekampF. + MelchiorP. + + Block-simultaneous direction method of multipliers: a proximal primal-dual splitting algorithm for nonconvex problems with multiple constraints + Optimization and Engineering + 2018 + 19 + https://link.springer.com/article/10.1007/s11081-018-9380-y + 10.1007/s11081-018-9380-y + + + + + + ChierchiaG. + ChouzenouxE. + CombettesP. L. + PesquetJ.-C. + + The proximity operator repository + 2024 + https://proximity-operator.net/ + + + + + + SimeoniMatthieu + KashaniSepand + Rué-QueraltJoan + DevelopersPyxu + + Pyxu-org/pyxu: pyxu + Zenodo + 2024 + https://doi.org/10.5281/zenodo.4486431 + 10.5281/zenodo.4486431 + + + + + + HeideFelix + DiamondSteven + NießnerMatthias + Ragan-KelleyJonathan + HeidrichWolfgang + WetzsteinGordon + + ProxImaL: Efficient image optimization using proximal algorithms + Association for Computing Machinery + New York, NY, USA + 2016 + 35 + 4 + 10.1145/2897824.2925875 + + + + +
diff --git a/joss.06326/10.21105.joss.06326.pdf b/joss.06326/10.21105.joss.06326.pdf new file mode 100644 index 0000000000..2930676fcb Binary files /dev/null and b/joss.06326/10.21105.joss.06326.pdf differ diff --git a/joss.06326/media/figs/software.png b/joss.06326/media/figs/software.png new file mode 100644 index 0000000000..c00c59ebf5 Binary files /dev/null and b/joss.06326/media/figs/software.png differ