Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ENH: Add inertially accelerated proximal cd #63

Closed
wants to merge 12 commits into from
Closed

ENH: Add inertially accelerated proximal cd #63

wants to merge 12 commits into from

Conversation

mathurinm
Copy link
Collaborator

@mathurinm mathurinm commented Mar 2, 2022 •
edited
Loading

fixes #62
Implementation by @ofercoq

I currently install from a branch of my fork, otherwise the repo is not pip installable. When/if the PR is merged over there, we can update.

Graphs:
image

Going towards smaller regularizer value, on some datasets it looks nice:
image

@mathurinm mathurinm requested a review from agramfort March 2, 2022 12:58
@mathurinm
Copy link
Collaborator Author

I don't know why the install fails. On my machine I can create a fresh env, install the package through pip from bitbucket (as is done by benchopt), and it works

(base) ➜  ~ conda activate tmp
(tmp) ➜  ~ pip install git+https://bitbucket.org/mathurinm/cd_solver.git@module_structure
Collecting git+https://bitbucket.org/mathurinm/cd_solver.git@module_structure
  Cloning https://bitbucket.org/mathurinm/cd_solver.git (to revision module_structure) to /tmp/pip-req-build-lj5bkp05
  Running command git clone -q https://bitbucket.org/mathurinm/cd_solver.git /tmp/pip-req-build-lj5bkp05
  Running command git checkout -b module_structure --track origin/module_structure
  Switched to a new branch 'module_structure'
  Branch 'module_structure' set up to track remote branch 'module_structure' from 'origin'.
  Resolved https://bitbucket.org/mathurinm/cd_solver.git to commit 11a72c49d429eb5940e838f0944e4d238ab16114
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting numpy
  Using cached numpy-1.22.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB)
Collecting scipy
  Using cached scipy-1.8.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (42.1 MB)
Building wheels for collected packages: cd-solver
  Building wheel for cd-solver (PEP 517) ... done
  Created wheel for cd-solver: filename=cd_solver-0.1-cp39-cp39-linux_x86_64.whl size=831021 sha256=8c8091a8b5cb18cd1e2847a40504c7f5d8bfdd9777b41decdc40b6a5bd35702d
  Stored in directory: /tmp/pip-ephem-wheel-cache-kvfql8af/wheels/7d/d8/4a/d6193fc1fe3b8298ba828f661fb3d8e80641471cf9af49e89a
Successfully built cd-solver
Installing collected packages: numpy, scipy, cd-solver
Successfully installed cd-solver-0.1 numpy-1.22.2 scipy-1.8.0
(tmp) ➜  ~ python
Python 3.9.7 (default, Sep 16 2021, 13:09:58) 
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cd_solver
>>>

@agramfort any clue ?

@agramfort
Copy link
Contributor

you still need help here @mathurinm ?

@mathurinm
Copy link
Collaborator Author

mathurinm commented Mar 3, 2022 •
edited
Loading

yes, I don't know why the import fails (resulting in NameError: name cd_solver is not defined)
the branch of my fork of cd_solver that I'm trying to install is this one :

https://bitbucket.org/mathurinm/cd_solver/src/module_structure/

In a fresh conda env with pip, pip install git+https://bitbucket.org/mathurinm/cd_solver.git@module_structure works,
and after installing benchopt in the env,

@mathurinm
Copy link
Collaborator Author

I managed to reproduce with benchopt install . --env -f -s apcd:

# Install
Collecting packages:
... done
Installing required packages for:
- apcd
...Collecting git+https://bitbucket.org/mathurinm/cd_solver.git@module_structure
  Cloning https://bitbucket.org/mathurinm/cd_solver.git (to revision module_structure) to /tmp/pip-req-build-173seqly
  Running command git clone --filter=blob:none --quiet https://bitbucket.org/mathurinm/cd_solver.git /tmp/pip-req-build-173seqly
  warning: filtering not recognized by server, ignoring
  Running command git checkout -b module_structure --track origin/module_structure
  Switched to a new branch 'module_structure'
  Branch 'module_structure' set up to track remote branch 'module_structure' from 'origin'.
  Resolved https://bitbucket.org/mathurinm/cd_solver.git to commit b2cc2e684d48574166f41d9b1bf269d138bf5588
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting numpy
  Using cached numpy-1.22.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB)
Collecting scipy
  Using cached scipy-1.8.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (41.6 MB)
Building wheels for collected packages: cd-solver
  Building wheel for cd-solver (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for cd-solver (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [26 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.8
      creating build/lib.linux-x86_64-3.8/cd_solver
      copying cd_solver/dualizer.py -> build/lib.linux-x86_64-3.8/cd_solver
      copying cd_solver/__init__.py -> build/lib.linux-x86_64-3.8/cd_solver
      running build_ext
      building 'cd_solver.atoms' extension
      creating build/temp.linux-x86_64-3.8
      creating build/temp.linux-x86_64-3.8/cd_solver
      /home/mathurin/anaconda3/envs/benchopt_lasso/bin/x86_64-conda-linux-gnu-cc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/mathurin/anaconda3/envs/benchopt_lasso/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/mathurin/anaconda3/envs/benchopt_lasso/include -fPIC -I/tmp/pip-build-env-24wm2oms/overlay/lib/python3.8/site-packages/numpy/core/include -I/home/mathurin/anaconda3/envs/benchopt_lasso/include/python3.8 -c cd_solver/atoms.cpp -o build/temp.linux-x86_64-3.8/cd_solver/atoms.o -O3
      cc1plus: warning: command-line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
      In file included from /tmp/pip-build-env-24wm2oms/overlay/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1960,
                       from /tmp/pip-build-env-24wm2oms/overlay/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                       from /tmp/pip-build-env-24wm2oms/overlay/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:5,
                       from cd_solver/atoms.cpp:728:
      /tmp/pip-build-env-24wm2oms/overlay/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
         17 | #warning "Using deprecated NumPy API, disable it with " \
            |  ^~~~~~~
      /home/mathurin/anaconda3/envs/benchopt_lasso/bin/x86_64-conda-linux-gnu-c++ -pthread -shared -B /home/mathurin/anaconda3/envs/benchopt_lasso/compiler_compat -L/home/mathurin/anaconda3/envs/benchopt_lasso/lib -Wl,-rpath=/home/mathurin/anaconda3/envs/benchopt_lasso/lib -Wl,--no-as-needed -Wl,--sysroot=/ -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/home/mathurin/anaconda3/envs/benchopt_lasso/lib -Wl,-rpath-link,/home/mathurin/anaconda3/envs/benchopt_lasso/lib -L/home/mathurin/anaconda3/envs/benchopt_lasso/lib -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/mathurin/anaconda3/envs/benchopt_lasso/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/mathurin/anaconda3/envs/benchopt_lasso/include build/temp.linux-x86_64-3.8/cd_solver/atoms.o -o build/lib.linux-x86_64-3.8/cd_solver/atoms.cpython-38-x86_64-linux-gnu.so
      /home/mathurin/anaconda3/envs/benchopt_lasso/compiler_compat/ld: cannot find /lib64/libpthread.so.0
      /home/mathurin/anaconda3/envs/benchopt_lasso/compiler_compat/ld: cannot find /usr/lib64/libpthread_nonshared.a
      collect2: error: ld returned 1 exit status
      error: command '/home/mathurin/anaconda3/envs/benchopt_lasso/bin/x86_64-conda-linux-gnu-c++' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for cd-solver
Failed to build cd-solver
ERROR: Could not build wheels for cd-solver, which is required to install pyproject.toml-based projects

googling "cannot find /lib64/libpthread.so.0" is not of much help so far

@mathurinm
Copy link
Collaborator Author

Seems to pop up quite frequently:
mamba-org/mamba#842

pandas-dev/pandas#40706

@Badr-MOUFAD Badr-MOUFAD mentioned this pull request May 11, 2023
Open
@Badr-MOUFAD
Copy link
Member

closing in favor of #131

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@agramfort agramfort Awaiting requested review from agramfort

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

ENH add cd_solver with accelerated coordinate descent
3 participants
@mathurinm @agramfort @Badr-MOUFAD