Unable to download dependencies

[maheshps1]

I could not resolve below issue. This is I am compiling polyglot from Virtual Machine where proxy is set.

No issue when I compiled in my laptop with public network.

root@BLR1000015302:/polyglot# bazel build src/main/java/me/dinowernli/grpc/polyglot
Unhandled exception thrown during build; message: Unrecoverable error while evaluating node 'REPOSITORY_DIRECTORY:@protoc_osx_x86_64' (requested by nodes 'REPOSITORY:@protoc_osx_x86_64')
INFO: Elapsed time: 12.518s
java.lang.RuntimeException: Unrecoverable error while evaluating node 'REPOSITORY_DIRECTORY:@protoc_osx_x86_64' (requested by nodes 'REPOSITORY:@protoc_osx_x86_64')
at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:447)
at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:496)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at com.sun.security.ntlm.Client.type3(Client.java:161)
at sun.net.www.protocol.http.ntlm.NTLMAuthentication.buildType3Msg(NTLMAuthentication.java:241)
at sun.net.www.protocol.http.ntlm.NTLMAuthentication.setHeaders(NTLMAuthentication.java:216)
at sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:2096)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpConnection.createAndConnectViaHttp(HttpConnection.java:144)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpConnection.createAndConnect(HttpConnection.java:92)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpDownloader.download(HttpDownloader.java:149)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpDownloader.download(HttpDownloader.java:92)
at com.google.devtools.build.lib.bazel.repository.HttpArchiveFunction.fetch(HttpArchiveFunction.java:66)
at com.google.devtools.build.lib.rules.repository.RepositoryDelegatorFunction.compute(RepositoryDelegatorFunction.java:155)
at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:388)
... 4 more
java.lang.RuntimeException: Unrecoverable error while evaluating node 'REPOSITORY_DIRECTORY:@protoc_osx_x86_64' (requested by nodes 'REPOSITORY:@protoc_osx_x86_64')
at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:447)
at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:496)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at com.sun.security.ntlm.Client.type3(Client.java:161)
at sun.net.www.protocol.http.ntlm.NTLMAuthentication.buildType3Msg(NTLMAuthentication.java:241)
at sun.net.www.protocol.http.ntlm.NTLMAuthentication.setHeaders(NTLMAuthentication.java:216)
at sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:2096)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpConnection.createAndConnectViaHttp(HttpConnection.java:144)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpConnection.createAndConnect(HttpConnection.java:92)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpDownloader.download(HttpDownloader.java:149)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpDownloader.download(HttpDownloader.java:92)
at com.google.devtools.build.lib.bazel.repository.HttpArchiveFunction.fetch(HttpArchiveFunction.java:66)
at com.google.devtools.build.lib.rules.repository.RepositoryDelegatorFunction.compute(RepositoryDelegatorFunction.java:155)
at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:388)
... 4 more
root@BLR1000015302:/polyglot#

Regards
-Mahesh

[katre]

It looks like your proxy is expecting the service to respond to NTLM authentication, which we can't do. Can you change the proxy configuration to not require authentication, or use a local proxy which handles the authentication directly?

[maheshps1]

sslverify I made false during proxy configuration. And I configured squid tool to allow all IP. But still same error.
While debugging I have seen, bazel tells unable to download, but if I use wget also unable to download for the same site, but even if I pass --no-certificate to wget then wget able to download. I think, somewhere this certificate exchange causes issue. Could not understand how to come out from this issue.

[maheshps1]

I cannot disable authentication in local proxy as I discussed with admin team. Is there any other solution?

[katre]

It seems Bazel doesn't have support for the proxy authentication you are using. Can you let us know
a) What is the name of the proxy software
b) What is the configured authentication method
c) Can other Java programs successfully authenticate to this proxy and download data over HTTP/S?

Thanks

[maheshps1]

(a) used only local proxy now: configured in /etc/environment and /etc/apt/apt.conf
(b)
root@BLR1000015302:~/polyglot# cat /etc/environment
HTTP_PROXY="http://username:password@proxy.co.com:8080"
HTTPS_PROXY="https://username:password@proxy.co.com:8080"
FTP_PROXY="ftp://username:password@proxy.co.com:8080"
HTTP_SSLVERIFY=false
NO_PROXY="localhost,127.0.0.1,localaddress,.localdomain.com"

root@BLR1000015302:~/polyglot# cat /etc/apt/apt.conf
Acquire::http::proxy "http://username:password@proxy.co.com:8080";
Acquire::ftp::proxy "ftp://username:password@proxy.co.com:8080";
Acquire::https::proxy "https://username:password@proxy.co.com:8080";
(c) yes, grpc-java tool is working fine.
Note: for security reason not mentioned here the username, password and proxy name.

[maheshps1]

Is it possible to download dependencies in some other machine and copy those dependencies to local machine to build polyglot?? I tried this, but still it is trying to download dependencies in local machine. How to over come from this?

[katre]

Where are you installing the dependencies?

Have you looked at the "--experimental_repository_cache" flag? You can use that on your local version to cache downloaded artifacts in a single per-user location, then copy that cache to the remote machine and use the same flag so that it will be used there. The cache defaults to a location in your home directory, but by passing a value with the flag (ie, "--experimental_repository_cache=/path/to/cache") you can choose the directory to use.

[maheshps1]

root@BLR1000015302:~/polyglot_temp# bazel build src/main/java/me/dinowernli/grpc/polyglot --experimental_repository_cache=/root/backup
Unrecognized option: --experimental_repository_cache=/root/backup

Is it correct command to build by using --experimental_repository_cache ??
Normally I used
root@BLR1000015302:/polyglot_temp# bazel build src/main/java/me/dinowernli/grpc/polyglot
ERROR: /root/polyglot_temp/third_party/protobuf/BUILD:13:1: no such package '@protoc_osx_x86_64//file': Error downloading from https://repo1.maven.org/maven2/com/google/protobuf/protoc/3.0.0-beta-3/protoc-3.0.0-beta-3-osx-x86_64.exe to /root/.cache/bazel/_bazel_root/5592dc4abbf4e732d4b7252f2ae4e980/external/protoc_osx_x86_64: Error downloading https://repo1.maven.org/maven2/com/google/protobuf/protoc/3.0.0-beta-3/protoc-3.0.0-beta-3-osx-x86_64.exe to /root/.cache/bazel/_bazel_root/5592dc4abbf4e732d4b7252f2ae4e980/external/protoc_osx_x86_64/protoc-3.0.0-beta-3-osx-x86_64.exe: Proxy address https://s00370346:OSPFospf is not a valid URL and referenced by '//third_party/protobuf:protoc_rule'.
ERROR: Analysis of target '//src/main/java/me/dinowernli/grpc/polyglot:polyglot' failed; build aborted.
INFO: Elapsed time: 2.423s
root@BLR1000015302:/polyglot_temp#

Above are the dependency downloading files what I had mentioned.

[maheshps1]

Hi katre,
Any input on this (above)?

[katre]

That is the correct flag. What version of Bazel are you using? The flag was added recently, make sure you are using the latest release.

[maheshps1]

I tried to install bazel-0.4.1-installer-linux-x86_64.sh now, but getting error
root@BLR1000015302:/opt# chmod +x bazel-0.4.1-installer-linux-x86_64.sh
root@BLR1000015302:/opt# ./bazel-0.4.1-installer-linux-x86_64.sh
...
Uncompressing..error [./bazel-0.4.0-installer-linux-x86_64.sh]: missing 1509 bytes in zipfile
(attempting to process anyway)
file #1: bad zipfile offset (local header sig): 5466
(attempting to re-compensate)
error: invalid compressed data to inflate /usr/local/lib/bazel/bin/bazel-real
I am using ubuntu OS
root@BLR1000015302:/opt# uname -a
Linux BLR1000015302 3.13.0-24-generic #47-Ubuntu SMP Fri May 2 23:30:00 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
root@BLR1000015302:/opt#

[damienmg]

What is the SHA-256 of the shell installer?

…

On Mon, Dec 5, 2016, 7:44 AM Mahesh Poojary S ***@***.***> wrote: I tried to install bazel-0.4.1-installer-linux-x86_64.sh now, but getting error Uncompressing..error [./bazel-0.4.0-installer-linux-x86_64.sh]: missing 1509 bytes in zipfile (attempting to process anyway) file #1 <#1>: bad zipfile offset (local header sig): 5466 (attempting to re-compensate) error: invalid compressed data to inflate /usr/local/lib/bazel/bin/bazel-real I am using ubuntu OS ***@***.***:/opt# uname -a Linux BLR1000015302 3.13.0-24-generic #47 <#47>-Ubuntu SMP Fri May 2 23:30:00 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux ***@***.***:/opt# — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1979 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADjHf3SRi9uecSoBjGNjbGjryQSBU3goks5rE7K6gaJpZM4Keolw> .

[maheshps1]

root@BLR1000015302:/opt# sha256sum bazel-0.4.1-installer-linux-x86_64.sh
11452bafd9735d79d2a1fdae4b865944d10d544ec4d7b90b7d5a8d63aecf6758 bazel-0.4.1-installer-linux-x86_64.sh
root@BLR1000015302:/opt#

[damienmg]

186f22ea776fba7dd5931241700f60974a681e90eb5831d40257baac9a58e436 is the correct shasum (see on https://github.com/bazelbuild/bazel/releases/0.4.1) Can you try to redownload the artifact.

[maheshps1]

I tried many times, but same issue. I wanted to know whether bazel-0.4.1-installer-linux-x86_64.sh is works
for Ubuntu OS ?