<p dir="auto"><em>Please make sure that this is a build/installation issue. As per our <a href="https://github.com/tensorflow/tensorflow/blob/master/ISSUES.md">GitHub Policy</a>, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template</em></p>
<p dir="auto"><strong>System information</strong></p>
<ul dir="auto">
<li>Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No</li>
<li>OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04</li>
<li>Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A</li>
<li>TensorFlow installed from (source or binary): source</li>
<li>TensorFlow version (use command below): r1.12</li>
<li>Python version: 3.6.5</li>
<li>Bazel version (if compiling from source): 0.19.0<br>
GCC/Compiler version (if compiling from source): gcc (Ubuntu 7.3.0-16ubuntu3) 7.3.0<br>
CUDA/cuDNN version: 10.0/7.4.1<br>
GPU model and memory: NVIDIA GeForce 940MX<br>
Exact command to reproduce: bazel build -c opt --config=cuda --config=gdr --config=mkl --config=ngraph --config=verbs  //tensorflow/tools/pip_package:build_pip_package --verbose_failures</li>
</ul>
<p dir="auto"><strong>Describe the problem</strong><br>
While building the tensorflow from the branch r1.12 with GPU support, the build fails with the following error:<br>
ERROR: ~/Documents/dev/git/tensorflow/tensorflow/contrib/verbs/BUILD:105:1: C++ compilation of rule '//tensorflow/contrib/verbs:rdma_mgr' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command<br>
The issue was reproducable in tensorflow 1.11, It was said that it will be fixed in 1.12 after the merge from master, but apparently this did not happen.</p>
<p dir="auto"><strong>Provide the exact sequence of commands / steps that you executed before running into the problem</strong><br>
./configure<br>
bazel build -c opt --config=cuda --config=gdr --config=mkl --config=ngraph --config=verbs  //tensorflow/tools/pip_package:build_pip_package --verbose_failures</p>
<p dir="auto"><strong>Any other info / logs</strong><br>
The following command line is available:<br>
ERROR: <del>/Documents/dev/git/tensorflow/tensorflow/contrib/verbs/BUILD:105:1: C++ compilation of rule '//tensorflow/contrib/verbs:rdma_mgr' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command<br>
(cd <del>/.cache/bazel/_bazel_username/cf67b2b2e967476eb2b1ee98e33ab5bd/execroot/org_tensorflow &amp;&amp; <br>
exec env - <br>
CUDA_TOOLKIT_PATH=/usr/local/cuda <br>
CUDNN_INSTALL_PATH=/usr/lib/x86_64-linux-gnu <br>
GCC_HOST_COMPILER_PATH=/usr/bin/gcc <br>
NCCL_HDR_PATH=/usr/include <br>
NCCL_INSTALL_PATH=/usr/lib/x86_64-linux-gnu <br>
PATH=</del>/bin:/usr/local/sbin:/usr/local/lib:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/home/linuxbrew/.linuxbrew/opt/coreutils/libexec/gnubin:/usr/local/cuda/bin:/usr/local/share/apache/hadoop/sbin:/usr/local/share/apache/hadoop/bin:/usr/local/share/apache/spark/sbin:/usr/local/share/apache/spark/bin:/usr/games:/usr/local/games:</del>/bin:/snap/bin <br>
PWD=/proc/self/cwd <br>
PYTHON_BIN_PATH=/usr/bin/python3 <br>
PYTHON_LIB_PATH=/usr/lib/python3/dist-packages <br>
TF_CUDA_CLANG=0 <br>
TF_CUDA_COMPUTE_CAPABILITIES=5.0 <br>
TF_CUDA_VERSION=10.0 <br>
TF_CUDNN_VERSION=7 <br>
TF_NCCL_VERSION=2 <br>
TF_NEED_CUDA=1 <br>
TF_NEED_OPENCL_SYCL=0 <br>
TF_NEED_ROCM=0 <br>
external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -MD -MF bazel-out/k8-opt/bin/tensorflow/contrib/verbs/<em>objs/rdma_mgr/rdma_mgr.pic.d '-frandom-seed=bazel-out/k8-opt/bin/tensorflow/contrib/verbs/<em>objs/rdma_mgr/rdma_mgr.pic.o' '-DGRPC_ARES=0' '-DPB_FIELD_16BIT=1' -D__CLANG_SUPPORT_DYN_ANNOTATION</em></em> -DEIGEN_MPL2_ONLY '-DEIGEN_MAX_ALIGN_BYTES=64' -DTF_USE_SNAPPY -DTENSORFLOW_USE_VERBS -DTENSORFLOW_USE_GDR -DCURL_STATICLIB -DPLATFORM_LINUX -DENABLE_CURL_CLIENT -DENABLE_NO_ENCRYPTION -iquote . -iquote bazel-out/k8-opt/genfiles -iquote bazel-out/k8-opt/bin -iquote external/protobuf_archive -iquote bazel-out/k8-opt/genfiles/external/protobuf_archive -iquote bazel-out/k8-opt/bin/external/protobuf_archive -iquote external/bazel_tools -iquote bazel-out/k8-opt/genfiles/external/bazel_tools -iquote bazel-out/k8-opt/bin/external/bazel_tools -iquote external/grpc -iquote bazel-out/k8-opt/genfiles/external/grpc -iquote bazel-out/k8-opt/bin/external/grpc -iquote external/zlib_archive -iquote bazel-out/k8-opt/genfiles/external/zlib_archive -iquote bazel-out/k8-opt/bin/external/zlib_archive -iquote external/boringssl -iquote bazel-out/k8-opt/genfiles/external/boringssl -iquote bazel-out/k8-opt/bin/external/boringssl -iquote external/com_google_absl -iquote bazel-out/k8-opt/genfiles/external/com_google_absl -iquote bazel-out/k8-opt/bin/external/com_google_absl -iquote external/nsync -iquote bazel-out/k8-opt/genfiles/external/nsync -iquote bazel-out/k8-opt/bin/external/nsync -iquote external/eigen_archive -iquote bazel-out/k8-opt/genfiles/external/eigen_archive -iquote bazel-out/k8-opt/bin/external/eigen_archive -iquote external/local_config_sycl -iquote bazel-out/k8-opt/genfiles/external/local_config_sycl -iquote bazel-out/k8-opt/bin/external/local_config_sycl -iquote external/gif_archive -iquote bazel-out/k8-opt/genfiles/external/gif_archive -iquote bazel-out/k8-opt/bin/external/gif_archive -iquote external/jpeg -iquote bazel-out/k8-opt/genfiles/external/jpeg -iquote bazel-out/k8-opt/bin/external/jpeg -iquote external/com_googlesource_code_re2 -iquote bazel-out/k8-opt/genfiles/external/com_googlesource_code_re2 -iquote bazel-out/k8-opt/bin/external/com_googlesource_code_re2 -iquote external/farmhash_archive -iquote bazel-out/k8-opt/genfiles/external/farmhash_archive -iquote bazel-out/k8-opt/bin/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/k8-opt/genfiles/external/fft2d -iquote bazel-out/k8-opt/bin/external/fft2d -iquote external/highwayhash -iquote bazel-out/k8-opt/genfiles/external/highwayhash -iquote bazel-out/k8-opt/bin/external/highwayhash -iquote external/local_config_cuda -iquote bazel-out/k8-opt/genfiles/external/local_config_cuda -iquote bazel-out/k8-opt/bin/external/local_config_cuda -iquote external/double_conversion -iquote bazel-out/k8-opt/genfiles/external/double_conversion -iquote bazel-out/k8-opt/bin/external/double_conversion -iquote external/mkl_linux -iquote bazel-out/k8-opt/genfiles/external/mkl_linux -iquote bazel-out/k8-opt/bin/external/mkl_linux -iquote external/mkl_dnn -iquote bazel-out/k8-opt/genfiles/external/mkl_dnn -iquote bazel-out/k8-opt/bin/external/mkl_dnn -iquote external/curl -iquote bazel-out/k8-opt/genfiles/external/curl -iquote bazel-out/k8-opt/bin/external/curl -iquote external/jsoncpp_git -iquote bazel-out/k8-opt/genfiles/external/jsoncpp_git -iquote bazel-out/k8-opt/bin/external/jsoncpp_git -iquote external/aws -iquote bazel-out/k8-opt/genfiles/external/aws -iquote bazel-out/k8-opt/bin/external/aws -isystem external/protobuf_archive/src -isystem bazel-out/k8-opt/genfiles/external/protobuf_archive/src -isystem bazel-out/k8-opt/bin/external/protobuf_archive/src -isystem external/grpc/include -isystem bazel-out/k8-opt/genfiles/external/grpc/include -isystem bazel-out/k8-opt/bin/external/grpc/include -isystem external/zlib_archive -isystem bazel-out/k8-opt/genfiles/external/zlib_archive -isystem bazel-out/k8-opt/bin/external/zlib_archive -isystem external/grpc/third_party/address_sorting/include -isystem bazel-out/k8-opt/genfiles/external/grpc/third_party/address_sorting/include -isystem bazel-out/k8-opt/bin/external/grpc/third_party/address_sorting/include -isystem external/boringssl/src/include -isystem bazel-out/k8-opt/genfiles/external/boringssl/src/include -isystem bazel-out/k8-opt/bin/external/boringssl/src/include -isystem external/nsync/public -isystem bazel-out/k8-opt/genfiles/external/nsync/public -isystem bazel-out/k8-opt/bin/external/nsync/public -isystem third_party/eigen3/mkl_include -isystem bazel-out/k8-opt/genfiles/third_party/eigen3/mkl_include -isystem bazel-out/k8-opt/bin/third_party/eigen3/mkl_include -isystem external/eigen_archive -isystem bazel-out/k8-opt/genfiles/external/eigen_archive -isystem bazel-out/k8-opt/bin/external/eigen_archive -isystem external/gif_archive/lib -isystem bazel-out/k8-opt/genfiles/external/gif_archive/lib -isystem bazel-out/k8-opt/bin/external/gif_archive/lib -isystem external/farmhash_archive/src -isystem bazel-out/k8-opt/genfiles/external/farmhash_archive/src -isystem bazel-out/k8-opt/bin/external/farmhash_archive/src -isystem external/local_config_cuda/cuda -isystem bazel-out/k8-opt/genfiles/external/local_config_cuda/cuda -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda -isystem external/local_config_cuda/cuda/cuda/include -isystem bazel-out/k8-opt/genfiles/external/local_config_cuda/cuda/cuda/include -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda/cuda/include -isystem external/local_config_cuda/cuda/cuda/include/crt -isystem bazel-out/k8-opt/genfiles/external/local_config_cuda/cuda/cuda/include/crt -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda/cuda/include/crt -isystem external/double_conversion -isystem bazel-out/k8-opt/genfiles/external/double_conversion -isystem bazel-out/k8-opt/bin/external/double_conversion -isystem external/mkl_linux/include -isystem bazel-out/k8-opt/genfiles/external/mkl_linux/include -isystem bazel-out/k8-opt/bin/external/mkl_linux/include -isystem external/mkl_dnn/include -isystem bazel-out/k8-opt/genfiles/external/mkl_dnn/include -isystem bazel-out/k8-opt/bin/external/mkl_dnn/include -isystem external/mkl_dnn/src -isystem bazel-out/k8-opt/genfiles/external/mkl_dnn/src -isystem bazel-out/k8-opt/bin/external/mkl_dnn/src -isystem external/mkl_dnn/src/common -isystem bazel-out/k8-opt/genfiles/external/mkl_dnn/src/common -isystem bazel-out/k8-opt/bin/external/mkl_dnn/src/common -isystem external/mkl_dnn/src/cpu -isystem bazel-out/k8-opt/genfiles/external/mkl_dnn/src/cpu -isystem bazel-out/k8-opt/bin/external/mkl_dnn/src/cpu -isystem external/mkl_dnn/src/cpu/xbyak -isystem bazel-out/k8-opt/genfiles/external/mkl_dnn/src/cpu/xbyak -isystem bazel-out/k8-opt/bin/external/mkl_dnn/src/cpu/xbyak -isystem external/mkl_dnn/src/cpu/gemm -isystem bazel-out/k8-opt/genfiles/external/mkl_dnn/src/cpu/gemm -isystem bazel-out/k8-opt/bin/external/mkl_dnn/src/cpu/gemm -isystem external/curl/include -isystem bazel-out/k8-opt/genfiles/external/curl/include -isystem bazel-out/k8-opt/bin/external/curl/include -isystem external/jsoncpp_git/include -isystem bazel-out/k8-opt/genfiles/external/jsoncpp_git/include -isystem bazel-out/k8-opt/bin/external/jsoncpp_git/include -isystem external/aws/aws-cpp-sdk-core/include -isystem bazel-out/k8-opt/genfiles/external/aws/aws-cpp-sdk-core/include -isystem bazel-out/k8-opt/bin/external/aws/aws-cpp-sdk-core/include -isystem external/aws/aws-cpp-sdk-kinesis/include -isystem bazel-out/k8-opt/genfiles/external/aws/aws-cpp-sdk-kinesis/include -isystem bazel-out/k8-opt/bin/external/aws/aws-cpp-sdk-kinesis/include -isystem external/aws/aws-cpp-sdk-s3/include -isystem bazel-out/k8-opt/genfiles/external/aws/aws-cpp-sdk-s3/include -isystem bazel-out/k8-opt/bin/external/aws/aws-cpp-sdk-s3/include '-std=c++11' -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -fPIC -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -fno-omit-frame-pointer -no-canonical-prefixes -DNDEBUG -g0 -O2 -ffunction-sections -fdata-sections -mavx -mavx2 -mfma '-mfpmath=both' -msse4.2 -DEIGEN_AVOID_STL_ARRAY -Iexternal/gemmlowp -Wno-sign-compare -fno-exceptions '-ftemplate-depth=900' '-DGOOGLE_CUDA=1' '-DINTEL_MKL=1' -DEIGEN_USE_VML -DENABLE_MKL '-DINTEL_NGRAPH=1' -fopenmp -msse3 -pthread '-DGOOGLE_CUDA=1' '-DINTEL_MKL=1' -DENABLE_MKL -c tensorflow/contrib/verbs/rdma_mgr.cc -o bazel-out/k8-opt/bin/tensorflow/contrib/verbs/<em>objs/rdma_mgr/rdma_mgr.pic.o)<br>
tensorflow/contrib/verbs/rdma_mgr.cc: In static member function 'static void tensorflow::RdmaMgr::RegMemVisitors()':<br>
tensorflow/contrib/verbs/rdma_mgr.cc:282:40: error: invalid use of member 'tensorflow::RdmaMgr::rdma_adapter</em>' in static member function<br>
int32_t bus_id = TryToReadNumaNode(rdma_adapter_-&gt;context_-&gt;device) + 1;<br>
^~~~~~~~~~~~~<br>
In file included from tensorflow/contrib/verbs/rdma_mgr.cc:18:0:<br>
./tensorflow/contrib/verbs/rdma_mgr.h:50:16: note: declared here<br>
RdmaAdapter* rdma_adapter_;<br>
^~~~~~~~~~~~~<br>
Target //tensorflow/tools/pip_package:build_pip_package failed to build<br>
INFO: Elapsed time: 4521.698s, Critical Path: 158.49s, Remote (0.00% of the time): [queue: 0.00%, setup: 0.00%, process: 0.00%]<br>
INFO: 4797 processes: 4797 local.<br>
FAILED: Build did NOT complete successfully</p>
<p dir="auto">Please go to Stack Overflow for help and support:</p>
<p dir="auto"><a href="https://stackoverflow.com/questions/tagged/tensorflow" rel="nofollow">https://stackoverflow.com/questions/tagged/tensorflow</a></p>
<p dir="auto">If you open a GitHub issue, here is our policy:</p>
<ol dir="auto">
<li>It must be a bug, a feature request, or a significant problem with documentation (for small docs fixes please send a PR instead).</li>
<li>The form below must be filled out.</li>
<li>It shouldn't be a TensorBoard issue. Those go <a href="https://github.com/tensorflow/tensorboard/issues">here</a>.</li>
</ol>
<p dir="auto"><strong>Here's why we have that policy</strong>: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.</p>
<hr>
<h3 dir="auto">System information</h3>
<ul dir="auto">
<li><strong>Have I written custom code (as opposed to using a stock example script provided in TensorFlow)</strong>:</li>
<li><strong>OS Platform and Distribution (e.g., Linux Ubuntu 16.04)</strong>:</li>
<li><strong>Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device</strong>:</li>
<li><strong>TensorFlow installed from (source or binary)</strong>:</li>
<li><strong>TensorFlow version (use command below)</strong>:</li>
<li><strong>Python version</strong>:</li>
<li><strong>Bazel version (if compiling from source)</strong>:</li>
<li><strong>GCC/Compiler version (if compiling from source)</strong>:</li>
<li><strong>CUDA/cuDNN version</strong>:</li>
<li><strong>GPU model and memory</strong>:</li>
<li><strong>Exact command to reproduce</strong>:</li>
</ul>
<p dir="auto">You can collect some of this information using our environment capture script:</p>
<p dir="auto"><a href="https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh">https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh</a></p>
<p dir="auto">You can obtain the TensorFlow version with</p>
<p dir="auto">python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"</p>
<h3 dir="auto">Describe the problem</h3>
<p dir="auto">Describe the problem clearly here. Be sure to convey here why it's a bug in TensorFlow or a feature request.</p>
<h3 dir="auto">Source code / logs</h3>
<p dir="auto">Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.</p>