ENH: Add inertially accelerated proximal cd

[mathurinm]

fixes #62
Implementation by @ofercoq

I currently install from a branch of my fork, otherwise the repo is not pip installable. When/if the PR is merged over there, we can update.

Graphs:

Going towards smaller regularizer value, on some datasets it looks nice:

[mathurinm]

I don't know why the install fails. On my machine I can create a fresh env, install the package through pip from bitbucket (as is done by benchopt), and it works

(base) ➜  ~ conda activate tmp
(tmp) ➜  ~ pip install git+https://bitbucket.org/mathurinm/cd_solver.git@module_structure
Collecting git+https://bitbucket.org/mathurinm/cd_solver.git@module_structure
  Cloning https://bitbucket.org/mathurinm/cd_solver.git (to revision module_structure) to /tmp/pip-req-build-lj5bkp05
  Running command git clone -q https://bitbucket.org/mathurinm/cd_solver.git /tmp/pip-req-build-lj5bkp05
  Running command git checkout -b module_structure --track origin/module_structure
  Switched to a new branch 'module_structure'
  Branch 'module_structure' set up to track remote branch 'module_structure' from 'origin'.
  Resolved https://bitbucket.org/mathurinm/cd_solver.git to commit 11a72c49d429eb5940e838f0944e4d238ab16114
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting numpy
  Using cached numpy-1.22.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB)
Collecting scipy
  Using cached scipy-1.8.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (42.1 MB)
Building wheels for collected packages: cd-solver
  Building wheel for cd-solver (PEP 517) ... done
  Created wheel for cd-solver: filename=cd_solver-0.1-cp39-cp39-linux_x86_64.whl size=831021 sha256=8c8091a8b5cb18cd1e2847a40504c7f5d8bfdd9777b41decdc40b6a5bd35702d
  Stored in directory: /tmp/pip-ephem-wheel-cache-kvfql8af/wheels/7d/d8/4a/d6193fc1fe3b8298ba828f661fb3d8e80641471cf9af49e89a
Successfully built cd-solver
Installing collected packages: numpy, scipy, cd-solver
Successfully installed cd-solver-0.1 numpy-1.22.2 scipy-1.8.0
(tmp) ➜  ~ python
Python 3.9.7 (default, Sep 16 2021, 13:09:58) 
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cd_solver
>>> 

@agramfort any clue ?

[agramfort]

you still need help here @mathurinm ?

[mathurinm]

yes, I don't know why the import fails (resulting in NameError: name cd_solver is not defined)
the branch of my fork of cd_solver that I'm trying to install is this one :

https://bitbucket.org/mathurinm/cd_solver/src/module_structure/

In a fresh conda env with pip, pip install git+https://bitbucket.org/mathurinm/cd_solver.git@module_structure works,
and after installing benchopt in the env,

[mathurinm]

I managed to reproduce with benchopt install . --env -f -s apcd:

# Install
Collecting packages:
... done
Installing required packages for:
- apcd
...Collecting git+https://bitbucket.org/mathurinm/cd_solver.git@module_structure
  Cloning https://bitbucket.org/mathurinm/cd_solver.git (to revision module_structure) to /tmp/pip-req-build-173seqly
  Running command git clone --filter=blob:none --quiet https://bitbucket.org/mathurinm/cd_solver.git /tmp/pip-req-build-173seqly
  warning: filtering not recognized by server, ignoring
  Running command git checkout -b module_structure --track origin/module_structure
  Switched to a new branch 'module_structure'
  Branch 'module_structure' set up to track remote branch 'module_structure' from 'origin'.
  Resolved https://bitbucket.org/mathurinm/cd_solver.git to commit b2cc2e684d48574166f41d9b1bf269d138bf5588
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting numpy
  Using cached numpy-1.22.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB)
Collecting scipy
  Using cached scipy-1.8.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (41.6 MB)
Building wheels for collected packages: cd-solver
  Building wheel for cd-solver (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for cd-solver (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [26 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.8
      creating build/lib.linux-x86_64-3.8/cd_solver
      copying cd_solver/dualizer.py -> build/lib.linux-x86_64-3.8/cd_solver
      copying cd_solver/__init__.py -> build/lib.linux-x86_64-3.8/cd_solver
      running build_ext
      building 'cd_solver.atoms' extension
      creating build/temp.linux-x86_64-3.8
      creating build/temp.linux-x86_64-3.8/cd_solver
      /home/mathurin/anaconda3/envs/benchopt_lasso/bin/x86_64-conda-linux-gnu-cc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/mathurin/anaconda3/envs/benchopt_lasso/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/mathurin/anaconda3/envs/benchopt_lasso/include -fPIC -I/tmp/pip-build-env-24wm2oms/overlay/lib/python3.8/site-packages/numpy/core/include -I/home/mathurin/anaconda3/envs/benchopt_lasso/include/python3.8 -c cd_solver/atoms.cpp -o build/temp.linux-x86_64-3.8/cd_solver/atoms.o -O3
      cc1plus: warning: command-line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
      In file included from /tmp/pip-build-env-24wm2oms/overlay/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1960,
                       from /tmp/pip-build-env-24wm2oms/overlay/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                       from /tmp/pip-build-env-24wm2oms/overlay/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:5,
                       from cd_solver/atoms.cpp:728:
      /tmp/pip-build-env-24wm2oms/overlay/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
         17 | #warning "Using deprecated NumPy API, disable it with " \
            |  ^~~~~~~
      /home/mathurin/anaconda3/envs/benchopt_lasso/bin/x86_64-conda-linux-gnu-c++ -pthread -shared -B /home/mathurin/anaconda3/envs/benchopt_lasso/compiler_compat -L/home/mathurin/anaconda3/envs/benchopt_lasso/lib -Wl,-rpath=/home/mathurin/anaconda3/envs/benchopt_lasso/lib -Wl,--no-as-needed -Wl,--sysroot=/ -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/home/mathurin/anaconda3/envs/benchopt_lasso/lib -Wl,-rpath-link,/home/mathurin/anaconda3/envs/benchopt_lasso/lib -L/home/mathurin/anaconda3/envs/benchopt_lasso/lib -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/mathurin/anaconda3/envs/benchopt_lasso/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/mathurin/anaconda3/envs/benchopt_lasso/include build/temp.linux-x86_64-3.8/cd_solver/atoms.o -o build/lib.linux-x86_64-3.8/cd_solver/atoms.cpython-38-x86_64-linux-gnu.so
      /home/mathurin/anaconda3/envs/benchopt_lasso/compiler_compat/ld: cannot find /lib64/libpthread.so.0
      /home/mathurin/anaconda3/envs/benchopt_lasso/compiler_compat/ld: cannot find /usr/lib64/libpthread_nonshared.a
      collect2: error: ld returned 1 exit status
      error: command '/home/mathurin/anaconda3/envs/benchopt_lasso/bin/x86_64-conda-linux-gnu-c++' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for cd-solver
Failed to build cd-solver
ERROR: Could not build wheels for cd-solver, which is required to install pyproject.toml-based projects

googling "cannot find /lib64/libpthread.so.0" is not of much help so far

[mathurinm]

Seems to pop up quite frequently:
mamba-org/mamba#842

pandas-dev/pandas#40706

[Badr-MOUFAD]

closing in favor of #131