title: Tensorflow install from Source (Win7): bazel, python -> No module named 'encodings'
body: <h1>Goal</h1>
  
  <p>Use Python-trained (keras) model in c/c++.
  See section work-around for c-work-around.</p>
  
  <p>Compile Tensorflow from source (rev 1.0 or 1.2.1)
  following the instructions from</p>
  
  <p><a href="https://www.tensorflow.org/install/install_windows" rel="nofollow noreferrer">https://www.tensorflow.org/install/install_windows</a></p>
  
  <p>(tensorflow already is installed via pip in python3, but I need to use it from c++, so installing from source seems mandatory)</p>
  
  <h1>System</h1>
  
  <ul>
  <li>win7(64)</li>
  <li>bazel 0.5.2,</li>
  <li>python(env) 3.5.2 (64),</li>
  <li>pip(env) 9.0.1,</li>
  <li>msys64/msys2_shell.cmd (GNU bash, 4.4.12  (64-pc-msys))</li>
  <li>gcc 6.3.0</li>
  <li>java SE "1.8.0_131",</li>
  <li>CUDA 8.0.60 (not used).</li>
  </ul>
  
  <h1>Error</h1>
  
  <p>(full logs are below in extra sections)</p>
  
  <p>in msys2-shell, setting environment vars, activate python virt-env, doing bazel clean, ./configure, </p>
  
  <pre><code>bazel build --verbose_failures --config=opt //tensorflow/tools/pip_package:build_pip_package
  </code></pre>
  
  <p>I get this error:</p>
  
  <pre><code>  ERROR: D:/user/sourcen/external/tensorflow-master-201707051514/
              tensorflow-master-rev-1-2-1/tensorflow/core/BUILD:1415:1: 
  
    Executing genrule //tensorflow/core:version_info_gen failed: 
  
    bash.exe failed: error executing command
  
    cd C:/users/user/appdata/local/temp/_bazel_user/chogfwdz/execroot/org_tensorflow
  
    C:/tools/msys64/usr/bin/bash.exe 
                                -c source external/bazel_tools/tools/genrule/genrule-setup.sh; 
                                          tensorflow/tools/git/gen_git_source.py 
                                --generate tensorflow/tools/git/gen/spec.json 
                                           tensorflow/tools/git/gen/head 
                                           tensorflow/tools/git/gen/branch_ref 
                                              "bazel-out/host/genfiles/tensorflow/core/util/version_info.cc": 
                                                com.google.devtools.build.lib.shell.
    BadExitStatusException: Process exited with status 3.
  
    Fatal Python error: Py_Initialize: unable to load the file system codec
    ImportError: No module named 'encodings'
  </code></pre>
  
  <p>Sorry, I am absolutely lost here, this message tells me nothing.
  Here some snippets from the files mentioned above, maybe somebody has a glue and gives me a hint for enlightment.... </p>
  
  <hr>
  
  <p>//tensorflow/core/BUILD:1415:1
       tf_version_info_genrule()</p>
  
  <pre><code> cc_library(
       name = "version_lib",
       srcs = ["util/version_info.cc"],
       hdrs = ["public/version.h"],
       copts = tf_copts(),
   )
  </code></pre>
  
  <hr>
  
  <p>bazel/.../genrule-setup.sh</p>
  
  <pre><code> set -e
   set -u
   set -o pipefail
  </code></pre>
  
  <hr>
  
  <p>//tensorflow/tools/git/gen_git_source.py</p>
  
  <pre><code>  """... creates symlinks from the internal git repository directory so
    that the build system can see changes in the version state. ..."""
  
    def parse_branch_ref(filename)...
    def configure(src_base_path, debug=False)...
    def get_git_version(git_base_path)...
    def write_version_info(filename, git_version)...
    def generate(arglist)...
    def raw_generate(output_file)...
  </code></pre>
  
  <hr>
  
  <p>I suppose, that the wrong python version is used (not the virtualenv I activated) - there are several other python versions (2.7-32, 2.7-64, 3.5-32, 3.5-64) installed (which are needed otherwise).</p>
  
  <h1>Questions</h1>
  
  <p>What can I do to check, if bazel is using the right python (virtualenv)?</p>
  
  <p>If bazel is using the wrong one, how can I change this?</p>
  
  <p>If bazel is using the right version, how can I get more details about what is going wrong (and fix it)?</p>
  
  <h2>Meta-Questions</h2>
  
  <p>What further information are needed (environment variables)?</p>
  
  <p>What can I do to improve the question to enable people to give advices?
  (please give comments, what is missing, wrong, unclear, too much)</p>
  
  <h2>Next steps</h2>
  
  <p>I searched for "Fatal Python error: Py_Initialize: unable to load the file system codec
  ImportError: No module named 'encodings'"</p>
  
  <ul>
  <li><p>tried to do the procedure below with a local admin account -> blocked by the company firewall</p></li>
  <li><p>tried to set PYTHONPATH and PYTHONHOME, no effect.</p>
  
  <p>export PYTHONHOME=/D/bin/Python64-3-Anaconda/envs/python64-35</p>
  
  <p>export PYTHONPATH=/D/bin/Python64-3-Anaconda/envs/python64-35:
                    /D/bin/Python64-3-Anaconda/envs/python64-35/lib/site-packages:
                    /d/bin/Python64-3-Anaconda/envs/python64-35/libs</p></li>
  <li><p>try to set up another virtualenv (as admin)</p></li>
  <li><p>try to install tensorflow via cmake </p></li>
  </ul>
  
  <h2>Procedure</h2>
  
  <p>This is what I did after starting msys2:</p>
  
  <p>In TF 1.0, I did a work-around for the first problem I faced (download failure of numeric_min_js).
  After some IRC talk &amp; lots of tries - finally I commented out one line in
   /tensorflow-rev1-0/tensorflow/tensorboard/bower/BUILD (see below).</p>
  
  <p>So now ./configure script (TF 1.0)  runs without errors (warnings about http_proxy can be ignored, as I 've been told on IRC and on some URLs I googled). Below I added full logs of the full build-attempt.</p>
  
  <p>My Python virtual-env is in</p>
  
  <pre><code>d:/bin/Python64-3-Anaconda/envs/python64-35/python.exe,
  </code></pre>
  
  <p>As library-path I used:</p>
  
  <pre><code>D:/bin/Python64-3-Anaconda/envs/python64-35/lib/site-packages.
  </code></pre>
  
  <h1>TF 1.0 attempt</h1>
  
  <h2>patching BUILD (TF1.0)</h2>
  
  <pre><code>.../tensorflow-rev1-0/tensorflow/tensorboard/bower/BUILD
  
   filegroup(
       name = "bower",
       srcs = [
      ...
           # "@numericjs_numeric_min_js//file", # by rg20170710
      ...
  
  
  
  
   user@CRPC0815 MSYS ~     
  $ cd /d/user/Sourcen/external/tensorflow-rev1-0/
  </code></pre>
  
  <h2>step 0: set env, activate python-env, cd </h2>
  
  <p>start msys2-shell:</p>
  
  <pre><code> $ c:/tools/msys64/msys2_shell.cmd
  </code></pre>
  
  <p>set the environment variables</p>
  
  <pre><code>  user@CRPC0815 MSYS ~
   $ source d:/user/00-Projekte/VrVis/env-cuda-python64-3--1.sh
  </code></pre>
  
  <p>activate virtual-env</p>
  
  <pre><code> $ cd /d/bin/Python64-3-Anaconda/
   $ source activate python64-35
  </code></pre>
  
  <p>goto sources</p>
  
  <pre><code> $ cd /d/user/Sourcen/external/tensorflow-rev1-0
  
    (python64-35)
    user@CRPC0815 MSYS /d/user/Sourcen/external/tensorflow-rev1-0
   $ 
  </code></pre>
  
  <h2>step 1: bazel clean</h2>
  
  <pre><code> (python64-35)
   user@CRPC0815 MSYS /d/user/Sourcen/external/tensorflow-rev1-0
  $ bazel clean
   WARNING: ignoring http_proxy in environment.
   ....................
   INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.
  </code></pre>
  
  <h2>step 2: ./configure</h2>
  
  <pre><code> (python64-35)
   user@CRPC0815 MSYS /d/user/Sourcen/external/tensorflow-rev1-0
  $ ./configure
  
   Please specify the location of python.
       [Default is /D/bin/Python64-3-Anaconda/envs/python64-35/python]:
  
   Please specify optimization flags to use during compilation [Default is -march=native]:
  
   Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
    No XLA support will be enabled for TensorFlow
  
   Found possible Python library paths:
     D:\bin\Python64-3-Anaconda\envs\python64-35\lib\site-packages
     D:\bin\Python64-3-Anaconda\envs\python64-35
   Please input the desired Python library path to use.
     Default is [D:\bin\Python64-3-Anaconda\envs\python64-35\lib\site-packages]
  
   Using python library path: D:\bin\Python64-3-Anaconda\envs\python64-35\lib\site-packages
  
   Junction created for util\python\python_include &lt;&lt;===&gt;&gt; D:\bin\Python64-3-Anaconda\envs\python64-35\include
  
   Junction created for util\python\python_lib &lt;&lt;===&gt;&gt; D:\bin\Python64-3-Anaconda\envs\python64-35\lib\site-packages
  
   Junction created for third_party\py\numpy\numpy_include &lt;&lt;===&gt;&gt; D:\bin\Python64-3-Anaconda\envs\
                                                                      python64-35\lib\site-packages\numpy\core\include
  
   Do you wish to build TensorFlow with CUDA support? [y/N]
   No CUDA support will be enabled for TensorFlow
  
   Configuration finished
  
   WARNING: ignoring http_proxy in environment.
   INFO: All external dependencies fetched successfully.
  </code></pre>
  
  <h2>step 3: bazel build &amp; error</h2>
  
  <pre><code> $ bazel build --verbose_failures --config=opt //tensorflow/tools/pip_package:build_pip_package
  
    WARNING: ignoring http_proxy in environment.
  
    INFO: Found 1 target...
  
    ERROR: D:/user/sourcen/external/tensorflow-rev1-0/tensorflow/core/BUILD:1227:1:
     Executing genrule //tensorflow/core:version_info_gen failed:
      bash.exe failed:
       error executing command
  
        cd C:/users/user/appdata/local/temp/_bazel_user/hnxszhzu/execroot/org_tensorflow
  
        SET PATH=C:\tools\msys64\usr\bin;C:\tools\msys64\bin;
                 D:\bin\Python64-3-Anaconda\envs\python64-35;
                 D:\bin\Python64-3-Anaconda\envs\python64-35\Library\mingw-w64\bin;
                 D:\bin\Python64-3-Anaconda\envs\python64-35\Library\usr\bin;
                 D:\bin\Python64-3-Anaconda\envs\python64-35\Library\bin;
                 D:\bin\Python64-3-Anaconda\envs\python64-35\Scripts;
                 C:\tools\msys64\usr\bin;
                 D:\user\bin\BLAS;
                 D:\bin\TDM-GCC-64\bin;
                 C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\bp2qlotc.p22\NativeBinaries\x86;
                 C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;
                 C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\;
                 C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64_x86\;
                 C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\;
                 C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\;
                 C:\Program Files (x86)\MSBuild\14.0\bin;
                 C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin;
                 C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\libnvvp;
                 C:\Program Files\NVIDIA GPU Computing Toolkit\cuDNN\bin;
                 C:\Program Files\NVIDIA GPU Computing Toolkit\cuDNN\lib;
                 C:\Program Files\TortoiseGit\bin;
                 C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;
                 C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;
                 C:\ProgramData\Oracle\Java\javapath;
                 C:\Qt\Qt5.7.0\5.7\msvc2015;
                 C:\WINDOWS;
                 C:\WINDOWS\System32\Wbem;
                 C:\WINDOWS\System32\WindowsPowerShell\v1.0\;
                 C:\WINDOWS\system32;
                 D:\user\bin\MinGW\msys\1.0\bin;
                 D:\user\bin\MinGW\msys\1.0\sbin;
                 D:\user\bin\MinGW\msys\1.0\sbin\awk;
                 D:\bin\CMake\bin;
                 D:\bin\Git\bin;
                 [...]
                 D:\bin\Python64-3-Anaconda\envs\python64-35\DLLs;
                 D:\bin\Python64-3-Anaconda\envs\python64-35\Lib;
                 D:\bin\Python64-3-Anaconda\envs\python64-35\libs;
                 D:\bin\Slik-Subversion-1-9-2\bin;
                 D:\bin\SubversionColabnet\bin;
                 D:\bin\TDM-GCC-64\bin;
                 C:\tools\msys64\usr\local\bin;
                 C:\tools\msys64\usr\bin;
                 C:\tools\msys64\usr\bin;
                 C:\tools\msys64\opt\bin;
                 C:\Windows\System32;
                 C:\Windows;
                 C:\Windows\System32\Wbem;
                 C:\Windows\System32\WindowsPowerShell\v1.0\;
                 C:\tools\msys64\usr\bin\site_perl;
                 C:\tools\msys64\usr\bin\vendor_perl;
                 C:\tools\msys64\usr\bin\core_perl
  
      C:/tools/msys64/usr/bin/bash.exe -c source external/bazel_tools/tools/genrule/genrule-setup.sh;
                                          tensorflow/tools/git/gen_git_source.py
                                       --generate tensorflow/tools/git/gen/spec.json
                                                  tensorflow/tools/git/gen/head
                                                  tensorflow/tools/git/gen/branch_ref
                                                   "bazel-out/host/genfiles/tensorflow/core/util/version_info.cc":
                                                    com.google.devtools.build.lib.
  
      shell.BadExitStatusException: Process exited with status 3.
  
    Fatal Python error: Py_Initialize: unable to load the file system codec
    ImportError: No module named 'encodings'
  
    Current thread 0x0000628c (most recent call first):
    Target //tensorflow/tools/pip_package:build_pip_package failed to build
    INFO: Elapsed time: 5,500s, Critical Path: 0,41s
  </code></pre>
  
  <h1>TF 1.2.1 build attempt</h1>
  
  <h2>step 0: set env, activate python-env, cd </h2>
  
  <p>(the same as in TF 1.0)</p>
  
  <pre><code> $ source d:/user/00-Projekte/VrVis/env-cuda-python64-3--1.sh
   $ cd /d/bin/Python64-3-Anaconda/
   $ source activate python64-35
   $ cd /d/user/Sourcen/external/tensorflow-master-201707051514/tensorflow-master-rev-1-2-1/
  </code></pre>
  
  <h2>step 1: bazel clean</h2>
  
  <pre><code> $ bazel clean
  
    WARNING: ignoring http_proxy in environment.
    INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes
  </code></pre>
  
  <h2>step 2: ./configure</h2>
  
  <pre><code> $ ./configure
  
    WARNING: ignoring http_proxy in environment.
  
    You have bazel 0.5.2- installed.
  
    Please specify the location of python.
      [Default is /D/bin/Python64-3-Anaconda/envs/python64-35/python]:
  
    Found possible Python library paths:
      D:\bin\Python64-3-Anaconda\envs\python64-35
      D:\bin\Python64-3-Anaconda\envs\python64-35\lib\site-packages
    Please input the desired Python library path to use.
      Default is [D:\bin\Python64-3-Anaconda\envs\python64-35]
    D:/bin/Python64-3-Anaconda/envs/python64-35/lib/site-packages
  
    Do you wish to build TensorFlow with MKL support? [y/N]
    No MKL support will be enabled for TensorFlow
  
    Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified
      [Default is -march=native]:
  
    Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
    No XLA support will be enabled for TensorFlow
  
    Do you wish to build TensorFlow with VERBS support? [y/N]
    No VERBS support will be enabled for TensorFlow
  
    Do you wish to build TensorFlow with CUDA support? [y/N]
    No CUDA support will be enabled for TensorFlow
  
    Do you wish to build TensorFlow with MPI support? [y/N]
    MPI support will not be enabled for TensorFlow
  
    Configuration finished
  </code></pre>
  
  <h2>step 3: bazel build &amp; error</h2>
  
  <pre><code>  (python64-35)
    user@CRPC0815 MSYS /d/user/Sourcen/external/tensorflow-master-201707051514/tensorflow-master-rev-1-2-1
   $ bazel build --verbose_failures --config=opt //tensorflow/tools/pip_package:build_pip_package
  
    WARNING: ignoring http_proxy in environment.
  
    INFO: Found 1 target...
  
    ERROR: D:/user/sourcen/external/tensorflow-master-201707051514/tensorflow-master-rev-1-2-1/tensorflow/core/BUILD:1415:1:
           Executing genrule //tensorflow/core:version_info_gen failed:
            bash.exe failed:
             error executing command
  
             cd C:/users/user/appdata/local/temp/_bazel_user/chogfwdz/execroot/org_tensorflow
  
             SET PATH=C:\tools\msys64\usr\bin;
                      C:\tools\msys64\bin;
                      C:\tools\msys64\usr\bin;
                      D:\user\bin\BLAS;
                      D:\bin\TDM-GCC-64\bin;
                      C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\bp2qlotc.p22\NativeBinaries\x86;
                      C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;
                      C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\;
                      C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64_x86\;
                      C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\;
                      C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\;
                      C:\Program Files (x86)\MSBuild\14.0\bin;
                      C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin;
                      C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\libnvvp;
                      C:\Program Files\NVIDIA GPU Computing Toolkit\cuDNN\bin;
                      C:\Program Files\NVIDIA GPU Computing Toolkit\cuDNN\lib;
                      C:\Program Files\TortoiseGit\bin;
                      C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;
                      C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;
                      C:\ProgramData\Oracle\Java\javapath;
                      C:\Qt\Qt5.7.0\5.7\msvc2015;
                      C:\WINDOWS;
                      C:\WINDOWS\System32\Wbem;
                      C:\WINDOWS\System32\WindowsPowerShell\v1.0\;
                      C:\WINDOWS\system32;
                      D:\BaslerPylon5\Runtime\Win32\;
                      D:\BaslerPylon5\Runtime\x64\;
                      D:\user\bin\MinGW\msys\1.0\bin;
                      D:\user\bin\MinGW\msys\1.0\sbin;
                      D:\user\bin\MinGW\msys\1.0\sbin\awk;
                      D:\bin\CMake\bin;
                      D:\bin\Git\bin;
                      D:\bin\Python64-3-Anaconda\envs\python64-35;
                      D:\bin\Python64-3-Anaconda\envs\python64-35\Library\bin;
                      D:\bin\Python64-3-Anaconda\envs\python64-35\Scripts;
                      D:\bin\Python64-3-Anaconda\envs\python64-35\DLLs;
                      D:\bin\Python64-3-Anaconda\envs\python64-35\Lib;
                      D:\bin\Python64-3-Anaconda\envs\python64-35\libs;
                      D:\bin\Slik-Subversion-1-9-2\bin;
                      D:\bin\SubversionColabnet\bin;
                      D:\bin\TDM-GCC-64\bin;
                      D:\bin\gnuplot\bin;
                      D:\user\bin;
      .
      C:/tools/msys64/usr/bin/bash.exe -c source external/bazel_tools/tools/genrule/genrule-setup.sh;
                                          tensorflow/tools/git/gen_git_source.py
                                       --generate tensorflow/tools/git/gen/spec.json
                                                  tensorflow/tools/git/gen/head tensorflow/tools/git/gen/branch_ref
                                                   "bazel-out/host/genfiles/tensorflow/core/util/version_info.cc":
                                                    com.google.devtools.build.lib.shell.
  
      BadExitStatusException: Process exited with status 3.
  
    Fatal Python error: Py_Initialize: unable to load the file system codec
    ImportError: No module named 'encodings'
  
    Current thread 0x000036e8 (most recent call first):
    Target //tensorflow/tools/pip_package:build_pip_package failed to build
  
    INFO: Elapsed time: 5,413s, Critical Path: 0,52s
  </code></pre>
  
  <h2>Work-Around</h2>
  
  <p>Meanwhile I found a work-around for deploying Tensorflow (Keras) Models (win7/win10) without need of bazel/cmake/sources....:</p>
  
  <ul>
  <li>use the C-API c_api.h from <a href="https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-windows-x86_64-1.5.0.zip" rel="nofollow noreferrer">win/cpu</a>, <a href="https://stackoverflow.com/questions/44305647/segmentation-fault-when-using-tf-sessionrun-to-run-tensorflow-graph-in-c-not-c/44588221#44588221">c-api example</a></li>
  <li><p>create lib from dll via vs2015 command prompt: </p>
  
  <p>dumpbin /exports xxx.dll > xxx.def</p>
  
  <p>pywrap_tensorflow_internal.lib</p></li>
  <li><p>extract gpu lib from Python pywrap_tensorflow_internal.lib</p></li>
  <li>copy pyd lib _pywrap_tensorflow_internal.pyd to dll-path</li>
  <li><p>maybe use gpu-options (python -> c):</p></li>
  <li><p>python code</p>
  
  <p>gpu_options = tensorflow.GPUOptions(allow_growth=True)</p>
  
  <p>config = tensorflow.ConfigProto(gpu_options=gpu_options)</p>
  
  <p>serialized = config.SerializeToString()</p>
  
  <p>[int(i) for i in serialized]</p></li>
  <li><p>c-code for gpu-options:</p>
  
  <p>TF_SessionOptions* sess_opts = TF_NewSessionOptions();</p>
  
  <p>// tensorflow.GPUOptions(allow_growth=True) ...</p>
  
  <p>uint8_t configProtoGPU[] = { 50, 2, 32, 1}; </p>
  
  <p>size_t configProtoGPUlen = sizeof(configProtoGPU) / sizeof(uint8_t);</p>
  
  <p>TF_SetConfig(sess_opts, configProtoGPU, configProtoGPUlen, status);</p></li>
  </ul>