Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
140 changes: 96 additions & 44 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,69 +1,121 @@
language: python
python:
- '3.6'
- '3.7-dev'
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3.7 version now is available


git:
depth: 200
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't go back to any old version of git so no need for this line

sudo: required
dist: trusty
matrix:
include:
- name: "Bionic python 3.7"
os: linux
dist: bionic
python: 3.7
env:
- REPO=https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
- OS=linux-64
- name: "trusty python 3.6"
os: linux
dist: trusty
python: 3.6
env:
- REPO=https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
- OS=linux-64
- name: "xenial python 3.7"
os: linux
dist: xenial
python: 3.7
env:
- REPO=https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
- OS=linux-64
- name: "xenial python 3.6"
os: linux
dist: xenial
python: 3.6
env:
- REPO=https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
- OS=linux-64
- name: "osx python 3.7"
os: osx
language: generic
env:
- REPO=https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
- TRAVIS_PYTHON_VERSION=3.7
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variable TRAVIS_PYTHON_VERSION is not defined in all cases, is it normal ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it is defined when it's the default platform language. However on Mac it is not possible to define a job with default language "python" (thus the language: generic).
See for example:

https://docs.travis-ci.com/user/multi-os/#python-example-unsupported-languages

- OS=osx-64
- name: "osx python 3.6"
os: osx
language: generic
env:
- REPO=https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
- TRAVIS_PYTHON_VERSION=3.6
- OS=osx-64

before_install:
- sudo apt-get update
- ldd --version
- gcc --version
- export START=$(pwd)

install:
- if [[ "$TRAVIS_PYTHON_VERSION" == "3.7-dev" ]]; then export VADD="py37"; else export VADD="py36"; fi
- wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
-O miniconda.sh
- bash miniconda.sh -b -p $HOME/miniconda
- export PATH="$HOME/miniconda/bin:$PATH"
- hash -r
- conda config --set always_yes yes --set changeps1 no
- conda install conda-build
- conda update -q conda
- conda update -n root conda-build
- conda config --set anaconda_upload no
- conda config --append channels conda-forge
- conda config --append channels tofuproject
- conda info -a
- if [[ "$TRAVIS_PYTHON_VERSION" == "3.7-dev" ]]; then export THIS_PY_VERSION="3.7";
else THIS_PY_VERSION=$TRAVIS_PYTHON_VERSION;
fi
- conda install -q python=$THIS_PY_VERSION conda-build anaconda-client nose
nose-timer coverage codecov
- export REV=$(python -c "import _updateversion as up; out=up.updateversion(); print(out)")
- export VERSION=$(echo $REV | tr - .)
- echo $REV
- conda build conda_recipe
- export PKG_DIR=$HOME/miniconda/conda-bld/linux-64/
- conda install tofu --use-local
- wget "$REPO" -O miniconda.sh
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change here since we created an environmental variable for the source of the conda repo

- bash miniconda.sh -b -p $HOME/miniconda
- export PATH="$HOME/miniconda/bin:$PATH"
- hash -r
- conda config --set always_yes yes --set changeps1 no
- conda update -q conda
- conda config --append channels conda-forge
- conda config --append channels tofuproject
- conda info -a
- conda install -q python="$TRAVIS_PYTHON_VERSION" conda-verify
nose nose-timer coverage codecov
- export REV=$(python -c "import _updateversion as up; out=up.updateversion(); print(out)")
- export VERSION=$(echo $REV | tr - .)
- echo $REV
- pip install -e ".[dev]"
Comment thread
Didou09 marked this conversation as resolved.

script:
- cd $HOME
- nosetests tofu.tests --nocapture -v --with-id --with-timer --with-coverage
--cover-package=tofu
- cd $HOME
- nosetests tofu.tests --nocapture -v --with-id --with-timer --with-coverage
--cover-package=tofu

after_success:
- codecov
- chmod +x $START/anaconda_upload.sh
- echo $TRAVIS_TAG
- codecov
- chmod +x $START/anaconda_upload.sh
- echo $TRAVIS_TAG

before_deploy:
- ls $START
- cd $START
- ls $START
- cd $START

deploy:
- provider: script
user: "ToFuProject"
script: $START/anaconda_upload.sh
on:
tags: true
all_branches: true
branch: master
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to deploy for all branches only master

skip_cleanup: true
- provider: pypi
user: "Didou09"
distributions: "sdist"
skip_existing: true
skip_cleanup: true
on:
tags: true
branch: master
password:
secure: JNEDTDJVx/2fXNfHntNQ99iDRNuQ4uB3y+DBWVIBycCT95+UCb36YPtKzmruEk/UUS29Xgq4IYCGdfCSWE9smKqG8tV1PcHiw705m+AzcpKy77YtzbVECFBxqY4W36O2pHrkwEUzP/7acjFwNsnUFzArqEzsBJ+KdLaa4OPHJXCh30GA0GyqlrXYbBKG+DA9hX5vtsGo4C6w9noALYF3fS7pKPiI6ipKFnAlzGgHQ7Ke0uQME8N3IAFhmh+Z5xMtIIDWxlnqv+KszdG4DIaGV/W6NIJNAbRhzkqUd+Chu6LoPAd/XkHDTeirR/MBkNUc5UcRJxRnP9rUTRo1gCO/buTYuNRgFkMvqhV5a033+x9edWgtUiKNJIMPLXOxe0RJvc5GWji+Co77HtHxRmGRM2rnYqWMtZeYZlFbUdvHu/8jf0d6I8jyUgAoJYdlMA2u/ipENP3S6by4epE9qycUPXiIVh6r3DZbf3vPTMFvTZYAjBrA0NOzihv1xgcXwemmNUFOQSpe0io4UcFxtS9lLMo+30UMQjCHSnbEVM3zSlZmbMOKpkVOlKlt8Lz5NxwVgWtu9FuW2pGukLtE8AWbqvY9urXAPZCQqZlOIklIjJQIqOITnuw9LEV09cgvPHXfdvNni3ldbMlIQ89zryM6dYvhYryTiEZGK4JDR3wAKJA=
- provider: pypi
user: "Didou09"
distributions: "bdist_wheel"
skip_existing: true
skip_cleanup: true
on:
tags: true
all_branches: true
condition: $OS = osx-64
branch: master
password:
secure: JNEDTDJVx/2fXNfHntNQ99iDRNuQ4uB3y+DBWVIBycCT95+UCb36YPtKzmruEk/UUS29Xgq4IYCGdfCSWE9smKqG8tV1PcHiw705m+AzcpKy77YtzbVECFBxqY4W36O2pHrkwEUzP/7acjFwNsnUFzArqEzsBJ+KdLaa4OPHJXCh30GA0GyqlrXYbBKG+DA9hX5vtsGo4C6w9noALYF3fS7pKPiI6ipKFnAlzGgHQ7Ke0uQME8N3IAFhmh+Z5xMtIIDWxlnqv+KszdG4DIaGV/W6NIJNAbRhzkqUd+Chu6LoPAd/XkHDTeirR/MBkNUc5UcRJxRnP9rUTRo1gCO/buTYuNRgFkMvqhV5a033+x9edWgtUiKNJIMPLXOxe0RJvc5GWji+Co77HtHxRmGRM2rnYqWMtZeYZlFbUdvHu/8jf0d6I8jyUgAoJYdlMA2u/ipENP3S6by4epE9qycUPXiIVh6r3DZbf3vPTMFvTZYAjBrA0NOzihv1xgcXwemmNUFOQSpe0io4UcFxtS9lLMo+30UMQjCHSnbEVM3zSlZmbMOKpkVOlKlt8Lz5NxwVgWtu9FuW2pGukLtE8AWbqvY9urXAPZCQqZlOIklIjJQIqOITnuw9LEV09cgvPHXfdvNni3ldbMlIQ89zryM6dYvhYryTiEZGK4JDR3wAKJA=
- provider: pypi
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I have created an account on test.pipy.org

user: "Didou09"
distributions: "sdist"
skip_existing: true
skip_cleanup: true
on:
branch: deploy-test
server: https://test.pypi.org/legacy/
password:
secure: JNEDTDJVx/2fXNfHntNQ99iDRNuQ4uB3y+DBWVIBycCT95+UCb36YPtKzmruEk/UUS29Xgq4IYCGdfCSWE9smKqG8tV1PcHiw705m+AzcpKy77YtzbVECFBxqY4W36O2pHrkwEUzP/7acjFwNsnUFzArqEzsBJ+KdLaa4OPHJXCh30GA0GyqlrXYbBKG+DA9hX5vtsGo4C6w9noALYF3fS7pKPiI6ipKFnAlzGgHQ7Ke0uQME8N3IAFhmh+Z5xMtIIDWxlnqv+KszdG4DIaGV/W6NIJNAbRhzkqUd+Chu6LoPAd/XkHDTeirR/MBkNUc5UcRJxRnP9rUTRo1gCO/buTYuNRgFkMvqhV5a033+x9edWgtUiKNJIMPLXOxe0RJvc5GWji+Co77HtHxRmGRM2rnYqWMtZeYZlFbUdvHu/8jf0d6I8jyUgAoJYdlMA2u/ipENP3S6by4epE9qycUPXiIVh6r3DZbf3vPTMFvTZYAjBrA0NOzihv1xgcXwemmNUFOQSpe0io4UcFxtS9lLMo+30UMQjCHSnbEVM3zSlZmbMOKpkVOlKlt8Lz5NxwVgWtu9FuW2pGukLtE8AWbqvY9urXAPZCQqZlOIklIjJQIqOITnuw9LEV09cgvPHXfdvNni3ldbMlIQ89zryM6dYvhYryTiEZGK4JDR3wAKJA=
67 changes: 43 additions & 24 deletions Notebooks/Cython_speedup_notes.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -94,20 +94,20 @@
"output_type": "stream",
"text": [
"For L = 1000\n",
"0.360000 μs, using the untyped_func\n",
"0.123000 μs, using the somewhat_typed_func\n",
"0.113000 μs, using the typed_func\n",
"0.029000 μs, using the inline_typed_func\n",
"0.798000 μs, using the untyped_func\n",
"0.027000 μs, using the somewhat_typed_func\n",
"0.017000 μs, using the typed_func\n",
"0.008000 μs, using the inline_typed_func\n",
"For L = 10000\n",
"5.174000 μs, using the untyped_func\n",
"0.024000 μs, using the somewhat_typed_func\n",
"0.012000 μs, using the typed_func\n",
"0.015000 μs, using the inline_typed_func\n",
"8.871000 μs, using the untyped_func\n",
"0.027000 μs, using the somewhat_typed_func\n",
"0.014000 μs, using the typed_func\n",
"0.014000 μs, using the inline_typed_func\n",
"For L = 100000\n",
"43.197000 μs, using the untyped_func\n",
"0.151000 μs, using the somewhat_typed_func\n",
"0.025000 μs, using the typed_func\n",
"0.024000 μs, using the inline_typed_func\n"
"81.221000 μs, using the untyped_func\n",
"0.148000 μs, using the somewhat_typed_func\n",
"0.010000 μs, using the typed_func\n",
"0.015000 μs, using the inline_typed_func\n"
]
}
],
Expand Down Expand Up @@ -200,7 +200,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 4,
"metadata": {},
"outputs": [
{
Expand All @@ -210,19 +210,19 @@
"\n",
"-------- TESTS -------\n",
"Running numpy buffers\n",
"0.020667 0.009333 0.098333 0.882333 6.571667 58.261333 670.627333 μs\n",
"0.010667 0.027000 0.108667 0.926667 11.027667 95.586333 1118.653333 μs\n",
"Running cpython.array buffer\n",
"0.327333 0.090000 0.323333 0.742333 7.339333 70.054000 1853.809000 μs\n",
"0.074667 0.091667 0.351667 3.606667 25.803333 230.891000 2605.097000 μs\n",
"Running cpython.array memoryview\n",
"0.890333 0.785000 1.215667 1.305667 6.270667 44.810333 538.815667 μs\n",
"0.325333 0.398667 1.079000 1.887667 7.430000 79.003667 1005.866000 μs\n",
"Running cpython.array raw C type with trick\n",
"0.044000 0.048667 0.380000 1.419333 10.037000 100.486667 2541.475333 μs\n",
"0.035000 0.041667 0.119667 0.787667 8.991000 93.519667 1401.235667 μs\n",
"Running C pointers\n",
"0.006000 0.006333 0.023667 0.159000 2.129667 25.866667 363.967667 μs\n",
"0.005000 0.006667 0.057333 0.563333 5.081000 35.753000 887.650667 μs\n",
"Running malloc memoryview\n",
"0.588667 0.630333 0.663667 1.224333 2.392333 23.987667 379.138667 μs\n",
"0.482333 1.043333 0.778000 1.127333 6.685000 53.277667 798.593333 μs\n",
"Running argument memoryview\n",
"0.011667 0.016333 0.112000 0.722667 5.305000 45.683333 525.975333 μs\n"
"0.008000 0.019000 0.156000 1.217000 10.740000 96.713667 1305.644333 μs\n"
]
}
],
Expand Down Expand Up @@ -391,7 +391,7 @@
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -402,9 +402,28 @@
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": 6,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"For L = 1000\n",
"10.211000 μs, using the sequential loop\n",
"58.846000 μs, using the parallel 1 loop\n",
"25.343000 μs, using the parallel 2 loop\n",
"For L = 10000\n",
"53.134000 μs, using the sequential loop\n",
"180.918000 μs, using the parallel 1 loop\n",
"49.014000 μs, using the parallel 2 loop\n",
"For L = 100000\n",
"827.005000 μs, using the sequential loop\n",
"601.007000 μs, using the parallel 1 loop\n",
"132.242000 μs, using the parallel 2 loop\n"
]
}
],
"source": [
"%%cython --compile=-fopenmp --link-args=-fopenmp\n",
"\n",
Expand Down Expand Up @@ -650,7 +669,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.2"
"version": "3.7.5"
}
},
"nbformat": 4,
Expand Down
17 changes: 4 additions & 13 deletions conda_recipe/conda_upload.sh
Original file line number Diff line number Diff line change
@@ -1,20 +1,11 @@
# Only need to change these two variables
PKG_NAME=tofu
USER=ToFuProject
OS=linux-64
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now this is was created in .travis.yml


#mkdir ~/conda-bld
#conda config --set anaconda_upload no
#conda update -n root conda-build
#conda config --append channels conda-forge
#conda config --append channels tofuproject
#export CONDA_BLD_PATH=~/conda-bld
#export VERSION=`date +%Y.%m.%d`
#export VERSION=$(head -n 1 version.txt)

#conda build conda_recipe
conda config --set anaconda_upload no
conda install anaconda-client conda-build
conda build conda_recipe
export PKG_REAL=$(conda build . --output | tail -1)
echo "Available conda packages:"
echo $(find $CONDA_BLD_PATH/$OS/ -type f -name $PKG_NAME*.tar.bz2)
PKG_REAL=$(find $CONDA_BLD_PATH/$OS/ -type f -name $PKG_NAME-$VERSION-$VADD*.tar.bz2)
echo $PKG_REAL
anaconda -t $CONDA_UPLOAD_TOKEN upload -u $USER -l main $PKG_REAL --force
7 changes: 1 addition & 6 deletions conda_recipe/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,24 +4,19 @@ package:

source:
git_url: https://github.com/ToFuProject/tofu.git
#git_branch: {{ environ['TRAVIS_BRANCH'] }}
git_rev: {{ environ['REV'] }}
# "patches:" might be the answer for clang compilers ?

build:
script_env:
# - REV
Comment thread
Didou09 marked this conversation as resolved.
- VERSION
- TRAVIS_BRANCH
- CONDA_BLD_PATH

requirements:

# build: necessary for build.sh
# here same as run, as we are using cython
build:
- python
- setuptools
- setuptools >=40.8.0
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated dependency

- setuptools_scm
- numpy
- scipy
Expand Down