TF 2.13.0-rc0 fails to compile on Ubuntu 22.04
Log when compiling using CUDA:
[mochi_O3.txt](https://github.com/tensorflow/tensorflow/files/11461476/mochi_O3.txt)
 A similar issue is present for MacOS 13.3 using python 3.10. Log attached. 
[MacOS.txt](https://github.com/tensorflow/tensorflow/files/11461636/MacOS.txt)
 I'm trying to reproduce in Ubuntu 22.04 with all of the above conditions.  I'll let you know if I'm able to. @SuryanarayanaY,
Notes: I tried to replicate the issue using tf v2.13.0-rc0 and CUDA 12.0 and was facing the below error when trying to configure the setup using ./configure.
![Screenshot 2023-05-16 2 40 47 PM](https://github.com/tensorflow/tensorflow/assets/81610181/39715001-4b2c-49e2-9127-1394e2ad0668)
 > @SuryanarayanaY, Notes: I tried to replicate the issue using tf v2.13.0-rc0 and CUDA 12.0 and was facing the below error when trying to configure the setup using ./configure. ![Screenshot 2023-05-16 2 40 47 PM](https://user-images.githubusercontent.com/81610181/238594523-39715001-4b2c-49e2-9127-1394e2ad0668.png)
This is not related to the issue and in fact it is not a bug. Make sure you have Cuda 12.0 properly installed. In both cases it's x86_64.

The issue here seem to point to a configuration issue (see below). Yet, I am using a stock configuration, unless there is a specific python package with a non-compatible package...

ERROR: 
/home/nicola/.cache/bazel/_bazel_nicola/c53ed0be17816f9e0970b1ba234e403c/external/local_config_python/BUILD:78:8: 
in outs attribute of genrule rule @local_config_python//:python_include: 
Genrules without outputs don't make sense
ERROR: 
/home/nicola/.cache/bazel/_bazel_nicola/c53ed0be17816f9e0970b1ba234e403c/external/local_config_python/BUILD:78:8: 
Analysis of target ***@***.***_config_python//:python_include' failed
INFO: Repository cython instantiated at:
   /home/nicola/Software/tensorflow/gpu/tensorflow/WORKSPACE:15:14: in 
<toplevel>
/home/nicola/Software/tensorflow/gpu/tensorflow/tensorflow/workspace2.bzl:972:21: 
in workspace
/home/nicola/Software/tensorflow/gpu/tensorflow/tensorflow/workspace2.bzl:706:20: 
in _tf_repositories
/home/nicola/Software/tensorflow/gpu/tensorflow/third_party/repo.bzl:136:21: 
in tf_http_archive
Repository rule _tf_http_archive defined at:
/home/nicola/Software/tensorflow/gpu/tensorflow/third_party/repo.bzl:89:35: 
in <toplevel>
ERROR: Analysis of target 
'//tensorflow/tools/pip_package:build_pip_package' failed; build aborted:
INFO: Elapsed time: 88.076s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (388 packages loaded, 6429 
targets configured)

On 5/16/23 9:11 AM, Yeison Rodriguez wrote:
>
> I was not able to reproduce the error, my build succeeded. What is 
> your system arch?
>
> —
> Reply to this email directly, view it on GitHub 
> <https://github.com/tensorflow/tensorflow/issues/60581#issuecomment-1549112759>, 
> or unsubscribe 
> <https://github.com/notifications/unsubscribe-auth/AAAIY7Y24PVXJBBXVXRPIVDXGMSATANCNFSM6AAAAAAX7IC5YE>.
> You are receiving this because you authored the thread.Message ID: 
> ***@***.***>
>
 Hi, Thanks for reporting the bug, Since we are trying to figure out if this is regression issue. 
Could you please help us with the below analysis and let us know if you see change in behavior.

Please test against the 2.12.0 version with below configurations.

<h4 id="gpu" data-text="GPU" role="presentation" style="box-sizing: inherit; margin: 32px 0px 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-numeric: ; font-variant-east-asian: ; font-variant-alternates: ; font-weight: ; font-stretch: ; font-size: 16px; line-height: ; font-family: Roboto, &quot;Noto Sans&quot;, &quot;Noto Sans JP&quot;, &quot;Noto Sans KR&quot;, &quot;Noto Naskh Arabic&quot;, &quot;Noto Sans Thai&quot;, &quot;Noto Sans Hebrew&quot;, &quot;Noto Sans Bengali&quot;, sans-serif; font-optical-sizing: ; font-kerning: ; font-feature-settings: ; font-variation-settings: ; letter-spacing: normal; overflow: hidden; text-overflow: ellipsis; margin-inline-end: -40px; padding-inline-end: 40px; color: rgb(32, 33, 36); orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><span class="devsite-heading" role="heading" aria-level="4" style="box-sizing: inherit;">GPU</span><button type="button" class="devsite-heading-link button-flat material-icons" aria-label="Copy link to this section: GPU" data-title="Copy link to this section: GPU" data-id="gpu" style="box-sizing: border-box; appearance: none; background: 0px center; border: 0px; border-radius: var(--devsite-button-border-radius,2px); box-shadow: none; color: var(--devsite-icon-color,var(--devsite-secondary-text-color)); cursor: pointer; display: inline-block; font-style: normal; font-variant-ligatures: ; font-variant-caps: ; font-variant-numeric: ; font-variant-east-asian: ; font-variant-alternates: ; font-weight: normal; font-stretch: ; font-size: 24px; font-family: &quot;Material Icons&quot;; font-optical-sizing: ; font-kerning: ; font-feature-settings: &quot;liga&quot;; font-variation-settings: ; height: 24px; letter-spacing: normal; line-height: 1; margin: var(--devsite-button-margin,0); margin-inline-end: var(--devsite-button-margin-x-end); max-width: var(--devsite-button-max-width,none); min-width: 36px; outline: 0px; overflow: hidden; padding: 0px 8px; text-align: center; text-decoration: none; text-overflow: ellipsis; text-transform: none; transition: background-color 0.2s ease 0s, border 0.2s ease 0s, box-shadow 0.2s ease 0s; vertical-align: bottom; white-space: nowrap; width: var(--devsite-button-width,auto); overflow-wrap: normal; direction: ltr; -webkit-font-smoothing: antialiased; opacity: 0;"></button></h4><div class="devsite-table-wrapper" style="box-sizing: inherit; margin: var(--devsite-table-margin,16px 0); padding: 0px; overflow: auto; color: rgb(32, 33, 36); font-family: Roboto, &quot;Noto Sans&quot;, &quot;Noto Sans JP&quot;, &quot;Noto Sans KR&quot;, &quot;Noto Naskh Arabic&quot;, &quot;Noto Sans Thai&quot;, &quot;Noto Sans Hebrew&quot;, &quot;Noto Sans Bengali&quot;, sans-serif; font-size: 16px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">

Version | Python version | Compiler | Build tools | cuDNN | CUDA
-- | -- | -- | -- | -- | --
tensorflow-2.12.0 | 3.8-3.11 | GCC 9.3.1 | Bazel 5.3.0 | 8.6 | 11.8

</div> On MacOS, the error is related but slightly different:

ERROR: /Users/feranick/Desktop/tensorflow/tensorflow/BUILD:1134:21: declared output 'tensorflow/libtensorflow_framework.2.dylib' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely)
ERROR: /Users/feranick/Desktop/tensorflow/tensorflow/BUILD:1134:21: Executing genrule //tensorflow:libtensorflow_framework.2.dylib_sym failed: not all outputs were created or valid
realpath: illegal option -- -
usage: realpath [-q] [path ...]
Target //tensorflow/tools/pip_package:build_pip_package failed to build > Hi, Thanks for reporting the bug, Since we are trying to figure out if this is regression issue. Could you please help us with the below analysis and let us know if you see change in behavior.
> 
> Please test against the 2.12.0 version with below configurations.
> 
> #### GPU
> Version	Python version	Compiler	Build tools	cuDNN	CUDA
> tensorflow-2.12.0	3.8-3.11	GCC 9.3.1	Bazel 5.3.0	8.6	11.8

On Ubuntu 22.04. TF 2.12.0 fails just as well (using the parameters below). 
tensorflow-2.12.0
python: 3.10.6	
GCC gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Bazel 5.3.0
cuDNN: 8.6 (or nothing)
CUDA: 11.8 (or nothing)
(i.e. the issue is present regardless of using CUDA or not).

Weirdly enough, it compiled just fine when released. I am not sure what changed, suggestions welcome.

I am testing MacOS, will report shortly. > I'm trying to reproduce in Ubuntu 22.04 with all of the above conditions. I'll let you know if I'm able to.

If you are successful, can you please list your system configuration (including your list of python packages and version)? I was not able to reproduce it in a fresh docker environment. On MacOS 13.3.1, TF 2.12.0 compiles just fine. 

tensorflow-2.12.0
python: 3.10.11
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Bazel 5.3.0 [UPDATE] I finally figured out the issue with Ubuntu. Compilation fails when Cython is newer than 0.29.28 is installed (currently the latest is 0.29.32). Cython needs to be v0.29.28, which is also that provided by Ubuntu via apt. Maybe this should be listed  in the recommended list of required software. 

I am still trying to identifying the issue with MacOS, which seems to be  related (and yet compilation fails also with Cytin 0.29.28). [UPDATE 2 - MacOS] It turns out that for MacOS the issue is with realpath, as listed in issue #60179. While that issue is closed, the complete lack of documentation on how to fix it (which involves third-party libraries installed) in the documentation is still an issue. So for MacOS the issue is a duplicate of #60179, so I removed the MacOS from the issue title. It would be great to update the documentation to reflect the needs of realpath from Coreutils (via brew or macports) to successfully complete the build. For Ubuntu the issue is not a duplicate, but again, documentation needs to be updated with information on the correct version number for Cython. @feranick ,

Sorry for the late response. I have tried simple CPU build with r2.13 and its success. Is this issue is specific to any optimizations ? 

<img width="1500" alt="r2 13 Ubuntu22 CPU build" src="https://github.com/tensorflow/tensorflow/assets/116063290/e23a9b58-c85c-4ead-bc33-391eeef0d50e">
 This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. Action is on Google here. I suggested a solution above.  @feranick ,

From Tf2.13 onwards TF uses Clang 16.0.0 as compiler instead of GCC as per updated [documentation](https://www.tensorflow.org/install/source#linux). I will also test the build with Clang and update you results. Hi @feranick , Could you please try to compile it in the 2.13.0 or 2.13.0-rc2 and let us know if that works for you. Thanks! Yes, it compiles fine with 2.13.0. But that is not because of this specific release (compared to earlier rcs), but because, as mentioned above the use of the correct libraries. The issue is in the lack of any documentation of what version of cython is needed for compilation. Such info should be listed in the required libraries. Hi, 

Would it be possible for you to create a PR with the specific version of cython which worked for you.Thanks! As described above there is no change needed in TF, just making sure that Cython is in the correct version:

Compilation fails when Cython is newer than 0.29.28 is installed (currently the latest is 0.29.32). Cython needs to be v0.29.28, which is also that provided by Ubuntu via apt. 

I am not sure what I should push as PR. Rather, this should be listed within the set of requirements for compilation... I have also not been able to build on lubuntu 22.04.3 using docker.io.

I follow the instructions on https://www.tensorflow.org/install/source#linux

The following appears to work OK

docker pull tensorflow/tensorflow:devel
docker run -it -w /tensorflow_src -v $PWD:/mnt -e HOST_PERMS="$(id -u):$(id -g)" \
    tensorflow/tensorflow:devel bash

git pull 

However, when running ./configure it fails to see the clang directories

root@54f07d1ee3ba:/tensorflow_src# ./configure 
You have bazel 6.1.0 installed.
Please specify the location of python. [Default is /usr/bin/python3]: 

Found possible Python library paths:
  /usr/lib/python3/dist-packages
  /usr/local/lib/python3.8/dist-packages
Please input the desired Python library path to use.  Default is [/usr/lib/python3/dist-packages]

Do you wish to build TensorFlow with ROCm support? [y/N]: 
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: 
No CUDA support will be enabled for TensorFlow.

Do you want to use Clang to build TensorFlow? [Y/n]: Y  
Clang will be used to compile TensorFlow.

Please specify the path to clang executable. [Default is ]: 


If I try to compile not using clang it does download and start to compile but fails at many points.

Bazel command used is either:

bazel build --config=opt --config=v2 //tensorflow/tools/pip_package:build_pip_package

or 

bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package

When it starts is tells me python 3.9 chosen.

I am compiling this on an old pre avx system.


 Don't use clang when asked. Should work.

On 9/12/23 9:43 AM, cuchio wrote:
>
> I have also not been able to build on lubuntu 22.04.3 using docker.io.
>
> I follow the instructions on 
> https://www.tensorflow.org/install/source#linux
>
> The following appears to work OK
>
> docker pull tensorflow/tensorflow:devel
> docker run -it -w /tensorflow_src -v $PWD:/mnt -e HOST_PERMS="$(id 
> -u):$(id -g)"
> tensorflow/tensorflow:devel bash
>
> git pull
>
> However, when running ./configure it fails to see the clang directories
>
> ***@***.***:/tensorflow_src# ./configure
> You have bazel 6.1.0 installed.
> Please specify the location of python. [Default is /usr/bin/python3]:
>
> Found possible Python library paths:
> /usr/lib/python3/dist-packages
> /usr/local/lib/python3.8/dist-packages
> Please input the desired Python library path to use. Default is 
> [/usr/lib/python3/dist-packages]
>
> Do you wish to build TensorFlow with ROCm support? [y/N]:
> No ROCm support will be enabled for TensorFlow.
>
> Do you wish to build TensorFlow with CUDA support? [y/N]:
> No CUDA support will be enabled for TensorFlow.
>
> Do you want to use Clang to build TensorFlow? [Y/n]: Y
> Clang will be used to compile TensorFlow.
>
> Please specify the path to clang executable. [Default is ]:
>
> If I try to compile not using clang it does download and start to 
> compile but fails at many points.
>
> Bazel command used is either:
>
> bazel build --config=opt --config=v2 
> //tensorflow/tools/pip_package:build_pip_package
>
> or
>
> bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package
>
> When it starts is tells me python 3.9 chosen.
>
> I am compiling this on an old pre avx system.
>
> —
> Reply to this email directly, view it on GitHub 
> <https://github.com/tensorflow/tensorflow/issues/60581#issuecomment-1715753213>, 
> or unsubscribe 
> <https://github.com/notifications/unsubscribe-auth/AAAIY76YGMCL6OPMGTQL6HLX2BRHLANCNFSM6AAAAAAX7IC5YE>.
> You are receiving this because you were mentioned.Message ID: 
> ***@***.***>
>

--------------uhVVUHL7PrMAlHCnksw33Bcd
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body>
    Don't use clang when asked. Should work.<br>
    <br>
    <div class="moz-cite-prefix">On 9/12/23 9:43 AM, cuchio wrote:<br>
    </div>
    <blockquote type="cite" ***@***.***">
      
      <p dir="auto">I have also not been able to build on lubuntu
        22.04.3 using docker.io.</p>
      <p dir="auto">I follow the instructions on <a href="https://www.tensorflow.org/install/source#linux" rel="nofollow" moz-do-not-send="true" class="moz-txt-link-freetext">https://www.tensorflow.org/install/source#linux</a></p>
      <p dir="auto">The following appears to work OK</p>
      <p dir="auto">docker pull tensorflow/tensorflow:devel<br>
        docker run -it -w /tensorflow_src -v <math-renderer class="js-inline-math" style="display: inline" data-static-url="https://github.githubassets.com/static" data-run-id="b0d76be046d4ad734683225467a7b462">$PWD:/mnt -e
          HOST_PERMS=&quot;$</math-renderer>(id -u):$(id -g)&quot; <br>
        tensorflow/tensorflow:devel bash</p>
      <p dir="auto">git pull</p>
      <p dir="auto">However, when running ./configure it fails to see
        the clang directories</p>
      <p ***@***.***:/tensorflow_src# ./configure<br>
        You have bazel 6.1.0 installed.<br>
        Please specify the location of python. [Default is
        /usr/bin/python3]:</p>
      <p dir="auto">Found possible Python library paths:<br>
        /usr/lib/python3/dist-packages<br>
        /usr/local/lib/python3.8/dist-packages<br>
        Please input the desired Python library path to use. Default is
        [/usr/lib/python3/dist-packages]</p>
      <p dir="auto">Do you wish to build TensorFlow with ROCm support?
        [y/N]:<br>
        No ROCm support will be enabled for TensorFlow.</p>
      <p dir="auto">Do you wish to build TensorFlow with CUDA support?
        [y/N]:<br>
        No CUDA support will be enabled for TensorFlow.</p>
      <p dir="auto">Do you want to use Clang to build TensorFlow? [Y/n]:
        Y<br>
        Clang will be used to compile TensorFlow.</p>
      <p dir="auto">Please specify the path to clang executable.
        [Default is ]:</p>
      <p dir="auto">If I try to compile not using clang it does download
        and start to compile but fails at many points.</p>
      <p dir="auto">Bazel command used is either:</p>
      <p dir="auto">bazel build --config=opt --config=v2
        //tensorflow/tools/pip_package:build_pip_package</p>
      <p dir="auto">or</p>
      <p dir="auto">bazel build --config=opt
        //tensorflow/tools/pip_package:build_pip_package</p>
      <p dir="auto">When it starts is tells me python 3.9 chosen.</p>
      <p dir="auto">I am compiling this on an old pre avx system.</p>
      <p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br>
        Reply to this email directly, <a href="https://github.com/tensorflow/tensorflow/issues/60581#issuecomment-1715753213" moz-do-not-send="true">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/AAAIY76YGMCL6OPMGTQL6HLX2BRHLANCNFSM6AAAAAAX7IC5YE" moz-do-not-send="true">unsubscribe</a>.<br>
        You are receiving this because you were mentioned.<img src="https://github.com/notifications/beacon/AAAIY7YJCZC4WSJB5N72WK3X2BRHLA5CNFSM6AAAAAAX7IC5YGWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTGIRIP2.gif" alt="" moz-do-not-send="true" width="1" height="1"><span style="color: transparent; font-size: 0; display: none; visibility: hidden; overflow: hidden; opacity: 0; width: 0; height: 0; max-width: 0; max-height: 0; mso-hide: all">Message
          ID: <span>&lt;tensorflow/tensorflow/issues/60581/1715753213</span><span>@</span><span>github</span><span>.</span><span>com&gt;</span></span></p>
      <script type="application/ld+json">[
{
***@***.***": "http://schema.org",
***@***.***": "EmailMessage",
"potentialAction": {
***@***.***": "ViewAction",
"target": "https://github.com/tensorflow/tensorflow/issues/60581#issuecomment-1715753213",
"url": "https://github.com/tensorflow/tensorflow/issues/60581#issuecomment-1715753213",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
***@***.***": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]</script>
    </blockquote>
    <br>
  </body>
</html>

--------------uhVVUHL7PrMAlHCnksw33Bcd--
 To be clear: You can install clang (from Ubuntu's repos via apt), it will be recognized, and compilation will start. However, it will fail later on for the lack of the atomic library (if I remember correctly). 

Easiest path is not to use clang for compilation. I have tried, but it does still have problems. Many stops in build occurs. > I have tried, but it does still have problems. Many stops in build occurs.

File separate and detailed issue reports. Will do.

FYI, I do have the clang libraries on my machine.
 Have you tried TF2.14.0-rc?
<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Build/Install

### Have you reproduced the bug with TF nightly?

No

### Source

source

### Tensorflow Version

2.13.0-rc0

### Custom Code

No

### OS Platform and Distribution

Linux Ubuntu 22.04 / MacOS 13.3

### Mobile device

_No response_

### Python version

3.10.6

### Bazel version

5.3.0

### GCC/Compiler version

gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0

### CUDA/cuDNN version

11.8

### GPU model and memory

Quadro RTX 6000

### Current Behaviour?

When compiling TF 2.13.0-rc0 from source with default bazel parameters (see attached log) compilation fails. 

### Standalone code to reproduce the issue

```shell
Use version of TF 2.13.0-rc0. Follow the default bazel ./configure parameters. The issues happens regardless of compilation with or without CUDA. 

Please input the desired Python library path to use.  Default is [/usr/lib/python3/dist-packages]

Do you wish to build TensorFlow with ROCm support? [y/N]: 
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: 
No CUDA support will be enabled for TensorFlow.

Do you wish to download a fresh release of clang? (Experimental) [y/N]: 
Clang will not be downloaded.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -Wno-sign-compare]: -Wno-sign-compare 


Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: 
Not configuring the WORKSPACE for Android builds.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
	--config=mkl         	# Build with MKL support.
	--config=mkl_aarch64 	# Build with oneDNN and Compute Library for the Arm Architecture (ACL).
	--config=monolithic  	# Config for mostly static monolithic build.
	--config=numa        	# Build with NUMA support.
	--config=dynamic_kernels	# (Experimental) Build kernels into separate shared objects.
	--config=v1          	# Build with TensorFlow 1 API instead of TF 2 API.
Preconfigured Bazel build configs to DISABLE default on features:
	--config=nogcp       	# Disable GCP support.
	--config=nonccl      	# Disable NVIDIA NCCL support.
Configuration finished
```


### Relevant log output

```shell
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=100
INFO: Reading rc options for 'build' from /home/nicola/Software/tensorflow/gpu/tensorflow/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /home/nicola/Software/tensorflow/gpu/tensorflow/.bazelrc:
  'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility
INFO: Reading rc options for 'build' from /home/nicola/Software/tensorflow/gpu/tensorflow/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/usr/bin/python3 --action_env PYTHON_LIB_PATH=/usr/lib/python3/dist-packages --python_path=/usr/bin/python3
INFO: Reading rc options for 'build' from /home/nicola/Software/tensorflow/gpu/tensorflow/.bazelrc:
  'build' options: --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_jitrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils,tensorflow/core/tfrt/utils/debug
INFO: Found applicable config definition build:short_logs in file /home/nicola/Software/tensorflow/gpu/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /home/nicola/Software/tensorflow/gpu/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:opt in file /home/nicola/Software/tensorflow/gpu/tensorflow/.tf_configure.bazelrc: --copt=-Wno-sign-compare --host_copt=-Wno-sign-compare
INFO: Found applicable config definition build:linux in file /home/nicola/Software/tensorflow/gpu/tensorflow/.bazelrc: --define=build_with_onednn_v2=true --host_copt=-w --copt=-Wno-all --copt=-Wno-extra --copt=-Wno-deprecated --copt=-Wno-deprecated-declarations --copt=-Wno-ignored-attributes --copt=-Wno-array-bounds --copt=-Wunused-result --copt=-Werror=unused-result --copt=-Wswitch --copt=-Werror=switch --copt=-Wno-error=unused-but-set-variable --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=dynamic_kernels --experimental_guard_against_concurrent_changes
INFO: Found applicable config definition build:dynamic_kernels in file /home/nicola/Software/tensorflow/gpu/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/7d879c8b161085a4374ea481b93a52adb19c0529.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/dc275fd03254d67d29cc70a5a0569acf24d2280d.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/google/benchmark/archive/f7547e29ccaed7b64ef4f7495ecfff1c9f6f3d03.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://mirror.bazel.build/github.com/bazelbuild/rules_cc/archive/081771d4a0e9d7d3aa0eed2ef389fa4700dfb23e.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/google/flatbuffers/archive/v23.1.21.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/abseil/abseil-cpp/archive/b971ac5250ea8de900eae9f95e06548d14cd95fe.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/glennrp/libpng/archive/v1.6.39.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/google/XNNPACK/archive/b9d4073a6913891ce9cbd8965c8d506075d2a45a.zip failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/openxla/stablehlo/archive/43d81c6883ade82052920bd367c61f9e52f09954.zip failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/google/boringssl/archive/c00d7ca810e93780bd0c8ee4eea28f4f2ea4bcdc.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/pybind/pybind11_abseil/archive/2c4932ed6f6204f1656e245838f4f5eae69d2e29.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
ERROR: /home/nicola/.cache/bazel/_bazel_nicola/c53ed0be17816f9e0970b1ba234e403c/external/local_config_python/BUILD:78:8: in outs attribute of genrule rule @local_config_python//:python_include: Genrules without outputs don't make sense
ERROR: /home/nicola/.cache/bazel/_bazel_nicola/c53ed0be17816f9e0970b1ba234e403c/external/local_config_python/BUILD:78:8: Analysis of target '@local_config_python//:python_include' failed
INFO: Repository cython instantiated at:
  /home/nicola/Software/tensorflow/gpu/tensorflow/WORKSPACE:15:14: in <toplevel>
  /home/nicola/Software/tensorflow/gpu/tensorflow/tensorflow/workspace2.bzl:972:21: in workspace
  /home/nicola/Software/tensorflow/gpu/tensorflow/tensorflow/workspace2.bzl:706:20: in _tf_repositories
  /home/nicola/Software/tensorflow/gpu/tensorflow/third_party/repo.bzl:136:21: in tf_http_archive
Repository rule _tf_http_archive defined at:
  /home/nicola/Software/tensorflow/gpu/tensorflow/third_party/repo.bzl:89:35: in <toplevel>
ERROR: Analysis of target '//tensorflow/tools/pip_package:build_pip_package' failed; build aborted: 
INFO: Elapsed time: 87.858s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (389 packages loaded, 6975 targets configured)
    Fetching https://storage.googleapis.com/.../github.com/cython/cython/archive/3.0.0a11.tar.gz
```
</details>