Failure to create build_pip_package Ubuntu 22.04 LTS / Python 3.10 / Cuda 11.7
Hi @cnovel ,

You can find the required pacakges for specific version packages in [setup.py](https://github.com/tensorflow/tensorflow/blob/r2.9/tensorflow/tools/pip_package/setup.py#L75) file.

For TF 2.9v the required numpy version should be >=1.20

https://github.com/tensorflow/tensorflow/blob/a5ed5f39b675a1c6f315e0caf3ad4b38478fa571/tensorflow/tools/pip_package/setup.py#L87

Apart from that you need to install CUDA and cuDNN also as per tested [configurations](https://www.tensorflow.org/install/source#gpu) mentioned here wrt TF version. For Tf2.9v the tested configurations are mentioned below.



Version             | Python version | Compiler | Build tools | cuDNN | CUDA
-- | -- | -- | -- | -- | --





tensorflow-2.9.0 | 3.7-3.10       | GCC 9.3.1 | Bazel 5.0.0 | 8.1 | 11.2
-- | -- | -- | -- | -- | --


Please try the build again and let us know if still have problem.

Thanks!



 This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61926">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61926">No</a>

### Issue type

Build/Install

### Have you reproduced the bug with TensorFlow Nightly?

No

### Source

source

### TensorFlow version

2.9.1

### Custom code

No

### OS platform and distribution

Ubuntu 22.04 LTS

### Mobile device

/

### Python version

3.10

### Bazel version

5.0.0

### GCC/compiler version

11.4.0

### CUDA/cuDNN version

11.7/8.5.0

### GPU model and memory

RTX A6000, GTX 1070

### Current behavior?

I've installed CUDA 11.7 toolkit only and libcudnn 8.5.0 on Ubuntu 22.04 LTS.
I'm using python 3.10 using the `python3.10` cmd.

Here's the script I'm running

```bash
export PATH=$PATH:/usr/local/cuda-11.7/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.7/lib64
export TF_CUDA_VERSION=11.7
export TF_CUDNN_VERSION=8.5.0
export TF_CUBLAS_VERSION=11.10.1

python3.10 -m virtualenv venv
venv/bin/pip install numpy==1.23.5 wheel packaging requests opt_einsum
venv/bin/pip install keras_preprocessing --no-deps

mkdir tmp
cd tmp
git clone https://github.com/tensorflow/tensorflow.git
cd tensorflow
git checkout v2.9.1

cd ..
. venv/bin/activate
cd tmp/tensorflow
./configure  # No ROCm, Yes CUDA, No TensorRT, 3.5,5.2,6.0,6.1,7.0,7.5,8.0 capabilities, No clang, default for the rest

bazel build --config=opt --verbose_failures //tensorflow:libtensorflow_cc.so

bazel build --config=opt --verbose_failures //tensorflow:install_headers

bazel build --config=opt --verbose_failures //tensorflow/tools/pip_package:build_pip_package  # Failure here

./bazel-bin/tensorflow/tools/pip_package/build_pip_package ./bazel-bin/tensorflow/tools/pip_package
```

The output of `bazel build --config=opt --verbose_failures //tensorflow/tools/pip_package:build_pip_package` is available in the relevant log output section.

The interesting line seems to be:
```
cp: cannot stat '/home/cluster/CN_TF/ContextCapture/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/_dtype_api.h': No such file or directory
```
Indeed this file does not exist. I'm unsure why it is needed, but I tried different `numpy` version and failed to find one that works (either I'm missing this file, or `.doxyfile` or something else).

Can you advise which version of numpy should I be using for the compilation to be successful?

### Standalone code to reproduce the issue

```shell
I'm using vanilla Tensorflow checkout, no modifications are applied.
```


### Relevant log output

```shell
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=116
INFO: Reading rc options for 'build' from /home/cluster/work/ThirdParty/Tensorflow/distrib/tmp/tensorflow/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /home/cluster/work/ThirdParty/Tensorflow/distrib/tmp/tensorflow/.bazelrc:
  'build' options: --define framework_shared_object=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library
INFO: Reading rc options for 'build' from /home/cluster/work/ThirdParty/Tensorflow/distrib/tmp/tensorflow/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/bin/python3 --action_env PYTHON_LIB_PATH=/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages --python_path=/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/bin/python3 --action_env TF_CUDA_VERSION=11.7 --action_env TF_CUBLAS_VERSION=11.10.1 --action_env TF_CUDNN_VERSION=8.5.0 --action_env CUDA_TOOLKIT_PATH=/usr/local/cuda-11.7 --action_env TF_CUDA_COMPUTE_CAPABILITIES=3.5,5.2,6.0,6.1,7.0,7.5,8.0 --action_env LD_LIBRARY_PATH=:/usr/local/cuda-11.7/lib64 --action_env GCC_HOST_COMPILER_PATH=/usr/bin/x86_64-linux-gnu-gcc-11 --config=cuda
INFO: Reading rc options for 'build' from /home/cluster/work/ThirdParty/Tensorflow/distrib/tmp/tensorflow/.bazelrc:
  'build' options: --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/common,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils
INFO: Found applicable config definition build:short_logs in file /home/cluster/work/ThirdParty/Tensorflow/distrib/tmp/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /home/cluster/work/ThirdParty/Tensorflow/distrib/tmp/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:cuda in file /home/cluster/work/ThirdParty/Tensorflow/distrib/tmp/tensorflow/.bazelrc: --repo_env TF_NEED_CUDA=1 --crosstool_top=@local_config_cuda//crosstool:toolchain --@local_config_cuda//:enable_cuda
INFO: Found applicable config definition build:opt in file /home/cluster/work/ThirdParty/Tensorflow/distrib/tmp/tensorflow/.tf_configure.bazelrc: --copt=-Wno-sign-compare --host_copt=-Wno-sign-compare
INFO: Found applicable config definition build:linux in file /home/cluster/work/ThirdParty/Tensorflow/distrib/tmp/tensorflow/.bazelrc: --copt=-w --host_copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --config=dynamic_kernels --distinct_host_configuration=false --experimental_guard_against_concurrent_changes
INFO: Found applicable config definition build:dynamic_kernels in file /home/cluster/work/ThirdParty/Tensorflow/distrib/tmp/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS
INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (236 packages loaded, 7837 targets configured).
INFO: Found 1 target...
ERROR: /home/cluster/.cache/bazel/_bazel_cluster/350d8f2feeb21f42226977fe6efcfb0a/external/local_config_python/BUILD:254:8: Executing genrule @local_config_python//:numpy_include failed: (Exit 1): bash failed: error executing command 
  (cd /home/cluster/.cache/bazel/_bazel_cluster/350d8f2feeb21f42226977fe6efcfb0a/execroot/org_tensorflow && \
  exec env - \
    CUDA_TOOLKIT_PATH=/usr/local/cuda-11.7 \
    GCC_HOST_COMPILER_PATH=/usr/bin/x86_64-linux-gnu-gcc-11 \
    LD_LIBRARY_PATH=:/usr/local/cuda-11.7/lib64 \
    PATH=/home/cluster/.cache/bazelisk/downloads/sha256/399eedb225cff7a13f9f027f7ea2aad02ddb668a8eb89b1d975d222e4dc12ed9/bin:/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/bin:/home/cluster/.vscode-server/bin/8b617bd08fd9e3fc94d14adb8d358b56e3f72314/bin/remote-cli:/home/cluster/.local/bin:/home/cluster/miniconda3/bin:/home/cluster/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/cuda-11.7/bin \
    PYTHON_BIN_PATH=/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/bin/python3 \
    PYTHON_LIB_PATH=/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages \
    TF2_BEHAVIOR=1 \
    TF_CUBLAS_VERSION=11.10.1 \
    TF_CUDA_COMPUTE_CAPABILITIES=3.5,5.2,6.0,6.1,7.0,7.5,8.0 \
    TF_CUDA_VERSION=11.7 \
    TF_CUDNN_VERSION=8.5.0 \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; 
cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/.doxyfile" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/.doxyfile" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/__multiarray_api.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/__multiarray_api.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/__ufunc_api.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/__ufunc_api.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/_dtype_api.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/_dtype_api.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/_neighborhood_iterator_imp.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/_neighborhood_iterator_imp.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/_numpyconfig.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/_numpyconfig.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/_numpyconfig.h.in" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/_numpyconfig.h.in" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/arrayobject.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/arrayobject.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/arrayscalars.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/arrayscalars.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/experimental_dtype_api.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/experimental_dtype_api.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/halffloat.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/halffloat.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/libdivide/LICENSE.txt" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/libdivide/LICENSE.txt" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/libdivide/libdivide.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/libdivide/libdivide.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/ndarrayobject.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/ndarrayobject.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/ndarraytypes.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/ndarraytypes.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/noprefix.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/noprefix.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_1_7_deprecated_api.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_3kcompat.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_3kcompat.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_common.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_common.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_cpu.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_cpu.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_endian.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_endian.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_interrupt.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_interrupt.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_math.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_math.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_no_deprecated_api.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_no_deprecated_api.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_os.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_os.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/numpyconfig.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/numpyconfig.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/old_defines.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/old_defines.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/oldnumeric.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/oldnumeric.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/random/bitgen.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/random/bitgen.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/random/distributions.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/random/distributions.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/ufuncobject.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/ufuncobject.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/utils.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/utils.h"
   ')
# Configuration: d04d20a1a4a46df9c7580a4efee273c7e78fdcec51eaaa1a5c3a4ea89b71ce88
# Execution platform: @local_execution_config_platform//:platform
cp: cannot stat '/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/_dtype_api.h': No such file or directory
Target //tensorflow/tools/pip_package:build_pip_package failed to build
ERROR: /home/cluster/.cache/bazel/_bazel_cluster/350d8f2feeb21f42226977fe6efcfb0a/external/local_config_python/BUILD:66:11 Middleman _middlemen/@local_Uconfig_Upython_S_S_Cnumpy_Uheaders-BazelCppSemantics_build_arch_k8-opt failed: (Exit 1): bash failed: error executing command 
  (cd /home/cluster/.cache/bazel/_bazel_cluster/350d8f2feeb21f42226977fe6efcfb0a/execroot/org_tensorflow && \
  exec env - \
    CUDA_TOOLKIT_PATH=/usr/local/cuda-11.7 \
    GCC_HOST_COMPILER_PATH=/usr/bin/x86_64-linux-gnu-gcc-11 \
    LD_LIBRARY_PATH=:/usr/local/cuda-11.7/lib64 \
    PATH=/home/cluster/.cache/bazelisk/downloads/sha256/399eedb225cff7a13f9f027f7ea2aad02ddb668a8eb89b1d975d222e4dc12ed9/bin:/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/bin:/home/cluster/.vscode-server/bin/8b617bd08fd9e3fc94d14adb8d358b56e3f72314/bin/remote-cli:/home/cluster/.local/bin:/home/cluster/miniconda3/bin:/home/cluster/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/cuda-11.7/bin \
    PYTHON_BIN_PATH=/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/bin/python3 \
    PYTHON_LIB_PATH=/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages \
    TF2_BEHAVIOR=1 \
    TF_CUBLAS_VERSION=11.10.1 \
    TF_CUDA_COMPUTE_CAPABILITIES=3.5,5.2,6.0,6.1,7.0,7.5,8.0 \
    TF_CUDA_VERSION=11.7 \
    TF_CUDNN_VERSION=8.5.0 \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; 
cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/.doxyfile" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/.doxyfile" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/__multiarray_api.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/__multiarray_api.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/__ufunc_api.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/__ufunc_api.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/_dtype_api.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/_dtype_api.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/_neighborhood_iterator_imp.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/_neighborhood_iterator_imp.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/_numpyconfig.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/_numpyconfig.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/_numpyconfig.h.in" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/_numpyconfig.h.in" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/arrayobject.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/arrayobject.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/arrayscalars.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/arrayscalars.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/experimental_dtype_api.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/experimental_dtype_api.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/halffloat.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/halffloat.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/libdivide/LICENSE.txt" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/libdivide/LICENSE.txt" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/libdivide/libdivide.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/libdivide/libdivide.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/ndarrayobject.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/ndarrayobject.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/ndarraytypes.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/ndarraytypes.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/noprefix.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/noprefix.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_1_7_deprecated_api.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_3kcompat.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_3kcompat.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_common.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_common.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_cpu.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_cpu.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_endian.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_endian.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_interrupt.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_interrupt.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_math.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_math.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_no_deprecated_api.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_no_deprecated_api.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_os.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/npy_os.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/numpyconfig.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/numpyconfig.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/old_defines.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/old_defines.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/oldnumeric.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/oldnumeric.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/random/bitgen.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/random/bitgen.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/random/distributions.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/random/distributions.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/ufuncobject.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/ufuncobject.h" && cp -f "/home/cluster/work/ThirdParty/Tensorflow/distrib/venv/lib/python3.10/site-packages/numpy/core/include/numpy/utils.h" "bazel-out/k8-opt/bin/external/local_config_python/numpy_include/numpy/utils.h"
   ')
# Configuration: d04d20a1a4a46df9c7580a4efee273c7e78fdcec51eaaa1a5c3a4ea89b71ce88
# Execution platform: @local_execution_config_platform//:platform
INFO: Elapsed time: 136.025s, Critical Path: 44.21s
INFO: 977 processes: 370 internal, 607 local.
FAILED: Build did NOT complete successfully
```
