Installing arrow on debian (stretch)

[tschm]

Hi, I am trying to install arrow in a rocker container. I struggle to install plasma but I have managed parquet. The error message I get is:

Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 libplasma-dev : Depends: libarrow-cuda-dev (= 0.14.1-1) but it is not going to be installed
                 Depends: libplasma14 (= 0.14.1-1) but it is not going to be installed
 libplasma-glib-dev : Depends: libarrow-cuda-glib-dev (= 0.14.1-1) but it is not going to be installed
                      Depends: libplasma-glib14 (= 0.14.1-1) but it is not going to be installed
                      Depends: gir1.2-plasma-1.0 (= 0.14.1-1) but it is not going to be installed

The dockerfile I am using is

# rocker/rstudio is based on debian
RUN apt update && \
    apt install -y -V apt-transport-https curl gnupg lsb-release apt-utils locate

RUN echo "deb http://deb.debian.org/debian $(lsb_release --codename --short)-backports main" >> /etc/apt/sources.list.d/backports.list && \
    curl --output /usr/share/keyrings/apache-arrow-keyring.gpg https://dl.bintray.com/apache/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-keyring.gpg && \
    echo "deb [arch=amd64 signed-by=/usr/share/keyrings/apache-arrow-keyring.gpg] https://dl.bintray.com/apache/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/ $(lsb_release --codename --short) main" >> /etc/apt/sources.list.d/apache-arrow.list && \
    echo "deb-src [signed-by=/usr/share/keyrings/apache-arrow-keyring.gpg] https://dl.bintray.com/apache/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/ $(lsb_release --codename --short) main" >> /etc/apt/sources.list.d/apache-arrow.list && \
    curl https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add - && \
    echo "deb http://apt.llvm.org/$(lsb_release --codename --short)/ llvm-toolchain-$(lsb_release --codename --short)-7 main" >> /etc/apt/sources.list.d/llvm.list && \
    echo "deb-src http://apt.llvm.org/$(lsb_release --codename --short)/ llvm-toolchain-$(lsb_release --codename --short)-7 main" >> /etc/apt/sources.list.d/llvm.list

RUN apt update && \
    apt install -y -V -f \
    libarrow-dev \
    libarrow-glib-dev \
    libarrow-flight-dev \
    libparquet-dev \
    libparquet-glib-dev \
    libxml2-dev

RUN install2.r --error \
    --deps TRUE \
    arrow

# this doesn't quite work yet
RUN apt install -y -V -f \
    libgandiva-dev \
    libgandiva-glib-dev \
    libplasma-dev \
    libplasma-glib-dev

[https://github.com/rocker-org/rocker-versioned/issues/163#issuecomment-535110987]

[kou]

You need to enable "non-free" section to install *plasma* packages because it depends on nvidia-cuda-toolkit package: https://packages.debian.org/search?keywords=nvidia-cuda-toolkit

[tschm]

@kou Thank you for your help. Do you mind going into details? I am not an expert for Debian. Would it be possible to update https://arrow.apache.org/install/? Attending contrib non-free to all APT-LINES didn't work for me.

[kou]

I will do: https://issues.apache.org/jira/browse/ARROW-6715
We also need "contrib" section:

echo deb http://deb.debian.org/debian $(lsb_release --codename --short) contrib | tee /etc/apt/sources.list.d/contrib.list
echo deb http://deb.debian.org/debian $(lsb_release --codename --short) non-free | tee /etc/apt/sources.list.d/non-free.list

Sorry.

[kou]

You need to run apt update after you update apt-lines.

[tschm]

Thank you so much. I have managed to install all libraries. BTW: I assume that's very unrelated but when I then install the arrow-package (install2.r --error --deps TRUE arrow) I get a lot of those deprecated warnings, e.g.

g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include -I. -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c configurationreader.cpp -o configurationreader.o
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include -I. -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c ctagsreader.cpp -o ctagsreader.o
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include -I. -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c datadir.cpp -o datadir.o
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include -I. -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c documentstyle.cpp -o documentstyle.o
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include -I. -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c elementstyle.cpp -o elementstyle.o
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include -I. -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c highlight_wrapper.cpp -o highlight_wrapper.o
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include -I. -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c htmlgenerator.cpp -o htmlgenerator.o
gcc -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include  -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c init.c -o init.o
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include -I. -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c languagedefinition.cpp -o languagedefinition.o
languagedefinition.cpp: In member function ‘highlight::RegexDef highlight::LanguageDefinition::extractRegex(const string&)’:
languagedefinition.cpp:100:3: warning: ‘template<class> class std::auto_ptr’ is deprecated [-Wdeprecated-declarations]
   auto_ptr<Matcher> m ( reDefPattern->createMatcher ( paramValue ) );
   ^~~~~~~~
In file included from /usr/include/c++/6/memory:81:0,
                 from languagedefinition.cpp:28:
/usr/include/c++/6/bits/unique_ptr.h:49:28: note: declared here
   template<typename> class auto_ptr;
                            ^~~~~~~~

Is that something I could ignore for now? Not sure this is the right place to ask for help on that problem? New issue?

[kou]

Apache Arrow doesn't have languagedefinition.cpp. So it may be a file in other package. But I don't know about R deeply.
Could you send an e-mail to dev@arrow.apache.org? You can join the mailing list by sending an e-mail to dev-subscribe@arrow.apache.org.