Skip to content

ARROW-6513: [CI] Rename conda requirements files to have txt extension instead of yml #10572

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kszucs wants to merge 3 commits into from
Closed

ARROW-6513: [CI] Rename conda requirements files to have txt extension instead of yml #10572

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0601c4c
Rename conda requirements files to have txt extension instead of yml
kszucs Jun 22, 2021
8f1913f
Fix conda isntegration dockerfile
kszucs Jun 29, 2021
0bbd748
Pin aws sdk
kszucs Jun 29, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions .github/workflows/python.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,12 +140,6 @@ jobs:
- name: Fetch Submodules and Tags
shell: bash
run: ci/scripts/util_checkout.sh
- name: Show available RAM size
Copy link
Member Author

@kszucs kszucs Jun 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Debugging snippet from 8aeec28

shell: bash
run: |
hwmemsize=$(sysctl -n hw.memsize)
ramsize=$(expr $hwmemsize / $((1024**3)))
echo "System Memory: ${ramsize} GB"
- name: Install Dependencies
shell: bash
run: |
Expand Down
6 changes: 3 additions & 3 deletions ci/appveyor-cpp-setup.bat
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,16 +50,16 @@ set CONDA_PACKAGES=

if "%ARROW_BUILD_GANDIVA%" == "ON" (
@rem Install llvmdev in the toolchain if building gandiva.dll
set CONDA_PACKAGES=%CONDA_PACKAGES% --file=ci\conda_env_gandiva_win.yml
set CONDA_PACKAGES=%CONDA_PACKAGES% --file=ci\conda_env_gandiva_win.txt
)
if "%JOB%" == "Toolchain" (
@rem Install pre-built "toolchain" packages for faster builds
set CONDA_PACKAGES=%CONDA_PACKAGES% --file=ci\conda_env_cpp.yml
set CONDA_PACKAGES=%CONDA_PACKAGES% --file=ci\conda_env_cpp.txt
)
if "%JOB%" NEQ "Build_Debug" (
@rem Arrow conda environment is only required for the Build and Toolchain jobs
conda create -n arrow -q -y -c conda-forge ^
--file=ci\conda_env_python.yml ^
--file=ci\conda_env_python.txt ^
%CONDA_PACKAGES% ^
"cmake=3.17" ^
"ninja" ^
Expand Down
0 ci/conda_env_archery.yml → ci/conda_env_archery.txt
View file
Open in desktop
File renamed without changes.
1 change: 1 addition & 0 deletions ci/conda_env_cpp.yml → ci/conda_env_cpp.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
# specific language governing permissions and limitations
# under the License.

# workaround for https://issues.apache.org/jira/browse/ARROW-13134
aws-sdk-cpp<1.9
benchmark=1.5.2
boost-cpp>=1.68.0
Expand Down
0 ci/conda_env_gandiva.yml → ci/conda_env_gandiva.txt
View file
Open in desktop
File renamed without changes.
0 ci/conda_env_gandiva_win.yml → ci/conda_env_gandiva_win.txt
View file
Open in desktop
File renamed without changes.
0 ci/conda_env_python.yml → ci/conda_env_python.txt
View file
Open in desktop
File renamed without changes.
0 ci/conda_env_r.yml → ci/conda_env_r.txt
View file
Open in desktop
File renamed without changes.
0 ci/conda_env_sphinx.yml → ci/conda_env_sphinx.txt
View file
Open in desktop
File renamed without changes.
0 ci/conda_env_unix.yml → ci/conda_env_unix.txt
View file
Open in desktop
File renamed without changes.
8 changes: 4 additions & 4 deletions ci/docker/conda-cpp.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@ ARG arch
FROM ${repo}:${arch}-conda

# install the required conda packages into the test environment
COPY ci/conda_env_cpp.yml \
ci/conda_env_gandiva.yml \
COPY ci/conda_env_cpp.txt \
ci/conda_env_gandiva.txt \
/arrow/ci/
RUN conda install \
--file arrow/ci/conda_env_cpp.yml \
--file arrow/ci/conda_env_gandiva.yml \
--file arrow/ci/conda_env_cpp.txt \
--file arrow/ci/conda_env_gandiva.txt \
compilers \
doxygen \
valgrind && \
Expand Down
5 changes: 2 additions & 3 deletions ci/docker/conda-integration.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,9 @@ ARG jdk=8
ARG go=1.15

# Install Archery and integration dependencies
COPY ci/conda_env_archery.yml /arrow/ci/
COPY ci/conda_env_archery.txt /arrow/ci/
RUN conda install -q \
--file arrow/ci/conda_env_cpp.yml \
--file arrow/ci/conda_env_archery.yml \
--file arrow/ci/conda_env_archery.txt \
numpy \
compilers \
maven=${maven} \
Expand Down
4 changes: 2 additions & 2 deletions ci/docker/conda-python.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ FROM ${repo}:${arch}-conda-cpp

# install python specific packages
ARG python=3.6
COPY ci/conda_env_python.yml /arrow/ci/
COPY ci/conda_env_python.txt /arrow/ci/
RUN conda install -q \
--file arrow/ci/conda_env_python.yml \
--file arrow/ci/conda_env_python.txt \
$([ "$python" == "3.6" -o "$python" == "3.7" ] && echo "pickle5") \
python=${python} \
nomkl && \
Expand Down
4 changes: 2 additions & 2 deletions ci/docker/conda.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ RUN /arrow/ci/scripts/install_conda.sh ${arch} linux latest ${prefix}
RUN /arrow/ci/scripts/install_minio.sh ${arch} linux latest ${prefix}

# create a conda environment
ADD ci/conda_env_unix.yml /arrow/ci/
RUN conda create -n arrow --file arrow/ci/conda_env_unix.yml git && \
ADD ci/conda_env_unix.txt /arrow/ci/
RUN conda create -n arrow --file arrow/ci/conda_env_unix.txt git && \
conda clean --all

# activate the created environment by default
Expand Down