Merge Develop for a new release (#322)

* Add support for tensorflow backend which allows for differentiability (#112) * Added support for tensorflow * Updates to get tests passing * Or --> And * Moving modopt to allow working with tensorflow * Fix issues with wos * Fix all flakes finally! * Update modopt/base/backend.py Co-authored-by: Samuel Farrens <[email protected]> * Update modopt/base/backend.py Co-authored-by: Samuel Farrens <[email protected]> * Minute updates to codes * Add dynamic module * Fix docu * Fix PEP Co-authored-by: chaithyagr <[email protected]> Co-authored-by: Samuel Farrens <[email protected]> * Fix 115 (#116) * Fix issues * Add right tests * Fix PEP Co-authored-by: chaithyagr <[email protected]> * Minor bug fix, remove elif (#124) Co-authored-by: chaithyagr <[email protected]> * Add tests for modopt.base.backend and fix minute bug uncovered (#126) * Minor bug fix, remove elif * Add tests for backend * Fix tests * Add tests * Remove cupy * PEP fixes * Fix PEP * Fix PEP and update * Final PEP * Update setup.cfg Co-authored-by: Samuel Farrens <[email protected]> * Update test_base.py Co-authored-by: chaithyagr <[email protected]> Co-authored-by: Samuel Farrens <[email protected]> * Release cleanup (#128) * updated GPU dependencies * added logo to manifest * updated package version and release date * Unpin package dependencies (#189) * unpinned dependencies * updated pinned documentation dependency versions * Add Gradient descent algorithms (#196) * Version 1.5.1 patch release (#114) * Add support for tensorflow backend which allows for differentiability (#112) * Added support for tensorflow * Updates to get tests passing * Or --> And * Moving modopt to allow working with tensorflow * Fix issues with wos * Fix all flakes finally! * Update modopt/base/backend.py Co-authored-by: Samuel Farrens <[email protected]> * Update modopt/base/backend.py Co-authored-by: Samuel Farrens <[email protected]> * Minute updates to codes * Add dynamic module * Fix docu * Fix PEP Co-authored-by: chaithyagr <[email protected]> Co-authored-by: Samuel Farrens <[email protected]> * Fix 115 (#116) * Fix issues * Add right tests * Fix PEP Co-authored-by: chaithyagr <[email protected]> * Minor bug fix, remove elif (#124) Co-authored-by: chaithyagr <[email protected]> * Add tests for modopt.base.backend and fix minute bug uncovered (#126) * Minor bug fix, remove elif * Add tests for backend * Fix tests * Add tests * Remove cupy * PEP fixes * Fix PEP * Fix PEP and update * Final PEP * Update setup.cfg Co-authored-by: Samuel Farrens <[email protected]> * Update test_base.py Co-authored-by: chaithyagr <[email protected]> Co-authored-by: Samuel Farrens <[email protected]> * Release cleanup (#128) * updated GPU dependencies * added logo to manifest * updated package version and release date Co-authored-by: Chaithya G R <[email protected]> Co-authored-by: chaithyagr <[email protected]> * make algorithms a module. * add Gradient Descent Algorithms * enforce WPS compliance. * add test for gradient descent * Docstrings improvements * Add See Also and minor corrections * add idx initialisation for all algorithms. * fix merge error * fix typo Co-authored-by: Samuel Farrens <[email protected]> Co-authored-by: Chaithya G R <[email protected]> Co-authored-by: chaithyagr <[email protected]> * Release cleanup (#198) * started clean up for next release * update progress * further clean up * additional clean up * cleaned up link to logo * fixed index.rst * fixed conflict * Fast Singular Value Thresholding (#209) * add SingularValueThreshold This Method provides 10x faster SVT estimation than the LowRankMatrix Operator. * linting * add test for fast computation. * flake8 compliance * Ignore DAR000 Error. * Update modopt/signal/svd.py tuples in docstring Co-authored-by: Samuel Farrens <[email protected]> * Update modopt/signal/svd.py typo Co-authored-by: Samuel Farrens <[email protected]> * Update modopt/opt/proximity.py typo Co-authored-by: Samuel Farrens <[email protected]> * update docstring * fix isort * Update modopt/signal/svd.py Co-authored-by: Samuel Farrens <[email protected]> * Update modopt/signal/svd.py Co-authored-by: Samuel Farrens <[email protected]> * run isort Co-authored-by: Samuel Farrens <[email protected]> * added writeable input data array feature for benchopt (#213) * removed flake8 limit * updated patch version * [lint] pydocstyle compliance. (#228) * [lint] pydocstyle compliance. * use pytest-pydocstyle * Power method: fix #211 (#212) * Correct the norm update for Power Method x_new should be divided by its norm, not by x_old_norm. * fix test value We are testing for eigen value of Identity. It should be one. * fix WPS350 * fix test value for unconverged case Co-authored-by: Samuel Farrens <[email protected]> * Switch from progressbar to tqdm (#231) * switch from progressbar to tqdm. The progress bar can be provided externally for nested usage. * exposes the progress bar argument. * Child classes better have to implement these. (my linter was complaining) * update docs for progress bar using tqdm. * fix WPS errors * drop progressbar requirement, add tqdm. * [lint] disable warning for non implemented function. * simplify progbar check and argument passthrough * Update README for tqdm dependency (#240) Remote progressbar, use tqdm. * add small help for the metric argument. (#241) * add small help for the metric argument. * RST validation * use single quote * use double backticks. Co-authored-by: Samuel Farrens <[email protected]> * add implementation for admm and fast admm. Based on Goldstein2014 * add Goldstein ref. * WPS compliance. * Abstract class for cost function. * add custom cost operator for admm. * fix WPS compliance. * Ci update (#268) * update python version support. * use string for CI. * remove flake8 and wemake-python-styleguide This anticipates the change to black formatting. * remove wps checks * apparently conda does not support 3.11 for now * remove all linting testing. * fix np.int warning/error * fix dtype error * fix precision for doctest * added black and isort support * Update python version in README * add 3.7 for test back * don't test 3.10 twice * Test rewrite (#266) * add MatrixOperator. * move base test to pytest. * [fixme] remove flake8 and emoji config. * rewrite test_math module using pytest. * use fail/skipparam helper function. * generalize usage of failparam * refactor test_signal. * refactor test_signal, the end. * lint * fix missing parameter. * add dummy object test helper. * rewrite test for cost and gradients. * show missing lines in coverage reports * rewrite of proximity operators testing. * add fail low rank method. * add cases for algorithms test * add algorithm test. * add pytest-cases and pytest-xdists support. * add support for testing metrics. * improve base module coverage. * test for wrong mask in metric module. * add docstring. * update email adress and authors field. * 100% coverage for transform module. * move linear operator to class * update docstring. * paramet(e)rization. * update docstring. * improve test_helper module. * raises should be specified for each failparam call. * encapsulate module's test in classes. * skip test if sklearn is not installed. * pin pydocstyle * removed unnormalised Gaussian kernel option and corresponding test * Restrict scikit-image version for testing * added fix for basic test suite * set behaviour for different astropy versions * updated docstring for gaussian_kernel * Use example scripts as tests. (#277) * Initialize the example module. * do not export the assert statements. * add matplotlib as requirement. * add support for sphinx-gallery * Update modopt/examples/README.rst Co-authored-by: Samuel Farrens <[email protected]> * Update modopt/examples/__init__.py Co-authored-by: Samuel Farrens <[email protected]> * Update modopt/examples/conftest.py Co-authored-by: Samuel Farrens <[email protected]> * Update modopt/examples/example_lasso_forward_backward.py Co-authored-by: Samuel Farrens <[email protected]> * Update modopt/examples/example_lasso_forward_backward.py Co-authored-by: Samuel Farrens <[email protected]> * ignore auto_example folder * doc formatting. * add pogm and basic comparison. * fix: add matplotlib for the plotting in examples scripts. * fix: add matplotlib for basic ci too. * ci: run pytest with xdist for faster testing --------- Co-authored-by: Samuel Farrens <[email protected]> * fix: specify data_range for ssim. Refs: #290 * typos. * feat(test): add test for admm. * feat(admm): improve doc. * refactor: rename abstract cost to CostParent. * feat: add test for fast admm. * feat(admm): improve docstrings. * style: remove extra line.c * feat: make POGM more memory efficient. * feat: add a dummy cost for the identity operator. * feat: create a linear operator module, add wavelet transform. * feat: add test case for wavelet transform. * Update setup.py --------- Co-authored-by: chaithyagr <[email protected]> Co-authored-by: Samuel Farrens <[email protected]> Co-authored-by: Pierre-Antoine Comby <[email protected]> Co-authored-by: Pierre-antoine Comby <[email protected]> Co-authored-by: Pierre-Antoine Comby <[email protected]>
CEA-COSMIC · Jan 19, 2024 · 2b1764c · 2b1764c
1 parent 3bae0ac
commit 2b1764c
Show file tree

Hide file tree

Showing 39 changed files with 2,439 additions and 2,601 deletions.
diff --git a/.github/workflows/cd-build.yml b/.github/workflows/cd-build.yml
@@ -62,9 +62,7 @@ jobs:
       - name: Set up Conda with Python 3.8
         uses: conda-incubator/setup-miniconda@v2
         with:
-          auto-update-conda: true
-          python-version: 3.8
-          auto-activate-base: false
+          python-version: "3.8"
 
       - name: Install dependencies
         shell: bash -l {0}

diff --git a/.github/workflows/ci-build.yml b/.github/workflows/ci-build.yml
@@ -16,21 +16,12 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest, macos-latest]
-        python-version: [3.8]
+        python-version: ["3.10"]
 
     steps:
       - name: Checkout
         uses: actions/checkout@v2
 
-      - name: Report WPS Errors
-        uses: wemake-services/[email protected]
-        continue-on-error: true
-        with:
-          reporter: 'github-pr-review'
-          path: './modopt'
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-
       - name: Set up Conda with Python ${{ matrix.python-version }}
         uses: conda-incubator/setup-miniconda@v2
         with:
@@ -52,7 +43,7 @@ jobs:
           python -m pip install --upgrade pip
           python -m pip install -r develop.txt
           python -m pip install -r docs/requirements.txt
-          python -m pip install astropy scikit-image scikit-learn
+          python -m pip install astropy "scikit-image<0.20" scikit-learn matplotlib
           python -m pip install tensorflow>=2.4.1
           python -m pip install twine
           python -m pip install .
@@ -61,7 +52,7 @@ jobs:
         shell: bash -l {0}
         run: |
           export PATH=/usr/share/miniconda/bin:$PATH
-          python setup.py test
+          pytest -n 2
 
       - name: Save Test Results
         if: always()
@@ -98,7 +89,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest, macos-latest]
-        python-version: [3.6, 3.7, 3.9]
+        python-version: ["3.7", "3.8", "3.9"]
 
     steps:
       - name: Checkout
@@ -117,11 +108,11 @@ jobs:
           python --version
           python -m pip install --upgrade pip
           python -m pip install -r develop.txt
-          python -m pip install astropy scikit-image scikit-learn
+          python -m pip install astropy "scikit-image<0.20" scikit-learn matplotlib
           python -m pip install .
 
       - name: Run Tests
         shell: bash -l {0}
         run: |
           export PATH=/usr/share/miniconda/bin:$PATH
-          python setup.py test
+          pytest -n 2
diff --git a/.gitignore b/.gitignore
@@ -73,6 +73,7 @@ instance/
 docs/_build/
 docs/source/fortuna.*
 docs/source/scripts.*
+docs/source/auto_examples/
 docs/source/*.nblink
 
 # PyBuilder

diff --git a/README.md b/README.md
@@ -37,11 +37,11 @@ All packages required by ModOpt should be installed automatically. Optional pack
 In order to run the code in this repository the following packages must be
 installed:
 
-* [Python](https://www.python.org/) [> 3.6]
+* [Python](https://www.python.org/) [> 3.7]
 * [importlib_metadata](https://importlib-metadata.readthedocs.io/en/latest/) [==3.7.0]
 * [Numpy](http://www.numpy.org/) [==1.19.5]
 * [Scipy](http://www.scipy.org/) [==1.5.4]
-* [Progressbar 2](https://progressbar-2.readthedocs.io/) [==3.53.1]
+* [tqdm](https://tqdm.github.io/) [>=4.64.0]
 
 ### Optional Packages
 

diff --git a/develop.txt b/develop.txt
@@ -1,9 +1,12 @@
 coverage>=5.5
-flake8>=4
-nose>=1.3.7
 pytest>=6.2.2
+pytest-raises>=0.10
+pytest-cases>= 3.6
+pytest-xdist>= 3.0.1
 pytest-cov>=2.11.1
-pytest-pep8>=1.0.6
 pytest-emoji>=0.2.0
-pytest-flake8>=1.0.7
-wemake-python-styleguide>=0.15.2
+pydocstyle==6.1.1
+pytest-pydocstyle>=2.2.0
+black
+isort
+pytest-black
diff --git a/docs/requirements.txt b/docs/requirements.txt
@@ -6,3 +6,4 @@ numpydoc==1.1.0
 sphinx==4.3.1
 sphinxcontrib-bibtex==2.4.1
 sphinxawesome-theme==3.2.1
+sphinx-gallery==0.11.1
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -45,6 +45,7 @@
     'nbsphinx',
     'nbsphinx_link',
     'numpydoc',
+    "sphinx_gallery.gen_gallery"
 ]
 
 # Include module names for objects
@@ -145,6 +146,18 @@
 # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
 html_show_copyright = True
 
+
+
+# -- Options for Sphinx Gallery ----------------------------------------------
+
+sphinx_gallery_conf = {
+    "examples_dirs": ["../../modopt/examples/"],
+    "filename_pattern": "/example_",
+    "ignore_pattern": r"/(__init__|conftest)\.py",
+}
+
+
+
 # -- Options for nbshpinx output ------------------------------------------
 
 

diff --git a/docs/source/refs.bib b/docs/source/refs.bib
@@ -207,3 +207,15 @@ @article{zou2005
   journal = {Journal of the Royal Statistical Society Series B},
   doi = {10.1111/j.1467-9868.2005.00527.x}
 }
+
+@article{Goldstein2014,
+ author={Goldstein, Tom and O’Donoghue, Brendan and Setzer, Simon and Baraniuk, Richard},
+ year={2014},
+ month={Jan},
+ pages={1588–1623},
+ title={Fast Alternating Direction Optimization Methods},
+ journal={SIAM Journal on Imaging Sciences},
+ volume={7},
+ ISSN={1936-4954},
+ doi={10/gdwr49},
+}
diff --git a/docs/source/toc.rst b/docs/source/toc.rst
@@ -25,6 +25,7 @@
 
   plugin_example
   notebooks
+  auto_examples/index
 
 .. toctree::
   :hidden:

diff --git a/modopt/examples/README.rst b/modopt/examples/README.rst
@@ -0,0 +1,5 @@
+========
+Examples
+========
+
+This is a collection of Python scripts demonstrating the use of ModOpt.
diff --git a/modopt/examples/__init__.py b/modopt/examples/__init__.py
@@ -0,0 +1,10 @@
+"""EXAMPLES.
+
+This module contains documented examples that demonstrate the usage of various
+ModOpt tools.
+
+These examples also serve as integration tests for various methods.
+
+:Author: Pierre-Antoine Comby
+
+"""
diff --git a/modopt/examples/conftest.py b/modopt/examples/conftest.py
@@ -0,0 +1,46 @@
+"""TEST CONFIGURATION.
+
+This module contains methods for configuring the testing of the example
+scripts.
+
+:Author: Pierre-Antoine Comby
+
+Notes
+-----
+Based on:
+https://stackoverflow.com/questions/56807698/how-to-run-script-as-pytest-test
+
+"""
+from pathlib import Path
+import runpy
+import pytest
+
+def pytest_collect_file(path, parent):
+    """Pytest hook.
+
+    Create a collector for the given path, or None if not relevant.
+    The new node needs to have the specified parent as parent.
+    """
+    p = Path(path)
+    if p.suffix == '.py' and 'example' in p.name:
+        return Script.from_parent(parent, path=p, name=p.name)
+
+
+class Script(pytest.File):
+    """Script files collected by pytest."""
+
+    def collect(self):
+        """Collect the script as its own item."""
+        yield ScriptItem.from_parent(self, name=self.name)
+
+class ScriptItem(pytest.Item):
+    """Item script collected by pytest."""
+
+    def runtest(self):
+        """Run the script as a test."""
+        runpy.run_path(str(self.path))
+
+    def repr_failure(self, excinfo):
+        """Return only the error traceback of the script."""
+        excinfo.traceback = excinfo.traceback.cut(path=self.path)
+        return super().repr_failure(excinfo)
diff --git a/modopt/examples/example_lasso_forward_backward.py b/modopt/examples/example_lasso_forward_backward.py
@@ -0,0 +1,153 @@
+# noqa: D205
+"""
+Solving the LASSO Problem with the Forward Backward Algorithm.
+==============================================================
+
+This an example to show how to solve an example LASSO Problem
+using the Forward-Backward Algorithm.
+
+In this example we are going to use:
+ - Modopt Operators (Linear, Gradient, Proximal)
+ - Modopt implementation of solvers
+ - Modopt Metric API.
+TODO: add reference to LASSO paper.
+"""
+
+import numpy as np
+import matplotlib.pyplot as plt
+
+from modopt.opt.algorithms import ForwardBackward, POGM
+from modopt.opt.cost import costObj
+from modopt.opt.linear import LinearParent, Identity
+from modopt.opt.gradient import GradBasic
+from modopt.opt.proximity import SparseThreshold
+from modopt.math.matrix import PowerMethod
+from modopt.math.stats import mse
+
+# %%
+# Here we create a instance of the LASSO Problem
+
+BETA_TRUE = np.array(
+    [3.0, 1.5, 0, 0, 2, 0, 0, 0]
+)  # 8 original values from lLASSO Paper
+DIM = len(BETA_TRUE)
+
+
+rng = np.random.default_rng()
+sigma_noise = 1
+obs = 20
+# create a measurement matrix with decaying covariance matrix.
+cov = 0.4 ** abs((np.arange(DIM) * np.ones((DIM, DIM))).T - np.arange(DIM))
+x = rng.multivariate_normal(np.zeros(DIM), cov, obs)
+
+y = x @ BETA_TRUE
+y_noise = y + (sigma_noise * np.random.standard_normal(obs))
+
+
+# %%
+# Next we create Operators for solving the problem.
+
+# MatrixOperator could also work here.
+lin_op = LinearParent(lambda b: x @ b, lambda bb: x.T @ bb)
+grad_op = GradBasic(y_noise, op=lin_op.op, trans_op=lin_op.adj_op)
+
+prox_op = SparseThreshold(Identity(), 1, thresh_type="soft")
+
+# %%
+# In order to get the best convergence rate, we first determine the Lipschitz constant of the gradient Operator
+#
+
+calc_lips = PowerMethod(grad_op.trans_op_op, 8, data_type="float32", auto_run=True)
+lip = calc_lips.spec_rad
+print("lipschitz constant:", lip)
+
+# %%
+# Solving using FISTA algorithm
+# -----------------------------
+#
+# TODO: Add description/Reference of FISTA.
+
+cost_op_fista = costObj([grad_op, prox_op], verbose=False)
+
+fb_fista = ForwardBackward(
+    np.zeros(8),
+    beta_param=1 / lip,
+    grad=grad_op,
+    prox=prox_op,
+    cost=cost_op_fista,
+    metric_call_period=1,
+    auto_iterate=False, # Just to give us the pleasure of doing things by ourself.
+)
+
+fb_fista.iterate()
+
+# %%
+# After the run we can have a look at the results
+
+print(fb_fista.x_final)
+mse_fista = mse(fb_fista.x_final, BETA_TRUE)
+plt.stem(fb_fista.x_final, label="estimation", linefmt="C0-")
+plt.stem(BETA_TRUE, label="reference", linefmt="C1-")
+plt.legend()
+plt.title(f"FISTA Estimation MSE={mse_fista:.4f}")
+
+# sphinx_gallery_start_ignore
+assert mse(fb_fista.x_final, BETA_TRUE) < 1
+# sphinx_gallery_end_ignore
+
+
+# %%
+# Solving Using the POGM Algorithm
+# --------------------------------
+#
+# TODO: Add description/Reference to POGM.
+
+
+cost_op_pogm = costObj([grad_op, prox_op], verbose=False)
+
+fb_pogm = POGM(
+    np.zeros(8),
+    np.zeros(8),
+    np.zeros(8),
+    np.zeros(8),
+    beta_param=1 / lip,
+    grad=grad_op,
+    prox=prox_op,
+    cost=cost_op_pogm,
+    metric_call_period=1,
+    auto_iterate=False, # Just to give us the pleasure of doing things by ourself.
+)
+
+fb_pogm.iterate()
+
+# %%
+# After the run we can have a look at the results
+
+print(fb_pogm.x_final)
+mse_pogm = mse(fb_pogm.x_final, BETA_TRUE)
+
+plt.stem(fb_pogm.x_final, label="estimation", linefmt="C0-")
+plt.stem(BETA_TRUE, label="reference", linefmt="C1-")
+plt.legend()
+plt.title(f"FISTA Estimation MSE={mse_pogm:.4f}")
+#
+# sphinx_gallery_start_ignore
+assert mse(fb_pogm.x_final, BETA_TRUE) < 1
+
+# %%
+# Comparing the Two algorithms
+# ----------------------------
+
+plt.figure()
+plt.semilogy(cost_op_fista._cost_list, label="FISTA convergence")
+plt.semilogy(cost_op_pogm._cost_list, label="POGM convergence")
+plt.xlabel("iterations")
+plt.ylabel("Cost Function")
+plt.legend()
+plt.show()
+
+
+# %%
+# We can see that the two algorithm converges quickly, and POGM requires less iterations.
+# However the POGM iterations are more costly, so a proper benchmark with time measurement is needed.
+# Check the benchopt benchmark for more details.