How does one install torchmeta for a ppc64le architecture in pytorch?
I was trying to use torchmeta in a ppc64le architecture. Unfortunately it's not been easy to install since ppc64le requires special binaries to work.
I eventually managed to get the right binaries for pytorch and torchvision by following these instructions (that prepend the right ibm channel with the conda binaries, plus installs all the required files too):
conda config --prepend channels https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/
conda create -n my_new_env python=3.7 powerai=1.7.0
conda activate my_new_env

after that I proceeded to install the right version of torchmeta, which was 1.3.1 since ppc64le only has pytorch 1.3.1 and torchvision 0.4.2. So I did:
pip install torchmeta==1.3.1

but now I have a new error that it cannot find the right version of h5py compatible with what I want to do. The error message is to large to paste but I will paste what I hope are useful part of it:
(my_new_env) [miranda9@hal-login ~]$ pip install torchmeta==1.3.1
Collecting torchmeta==1.3.1
  Using cached torchmeta-1.3.1-py3-none-any.whl (144 kB)
Requirement already satisfied: requests in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (2.22.0)
Requirement already satisfied: torchvision&lt;0.6.0,&gt;=0.4.0 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (0.4.2)
Requirement already satisfied: torch&lt;1.5.0,&gt;=1.3.0 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (1.3.1)
Processing ./.cache/pip/wheels/87/f5/ad/9f04a48453875e8054c19f9fe3f50cbbe0c09b956835555019/Pillow-6.2.2-cp37-cp37m-linux_ppc64le.whl
Requirement already satisfied: numpy&gt;=1.14.0 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (1.17.4)
Requirement already satisfied: tqdm&gt;=4.0.0 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (4.36.1)
Collecting h5py~=2.9.0
  Using cached h5py-2.9.0.tar.gz (287 kB)
Requirement already satisfied: chardet&lt;3.1.0,&gt;=3.0.2 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from requests-&gt;torchmeta==1.3.1) (3.0.4)
Requirement already satisfied: certifi&gt;=2017.4.17 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from requests-&gt;torchmeta==1.3.1) (2020.6.20)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,&lt;1.26,&gt;=1.21.1 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from requests-&gt;torchmeta==1.3.1) (1.25.10)
Requirement already satisfied: idna&lt;2.9,&gt;=2.5 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from requests-&gt;torchmeta==1.3.1) (2.8)
Requirement already satisfied: six in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchvision&lt;0.6.0,&gt;=0.4.0-&gt;torchmeta==1.3.1) (1.13.0)
Building wheels for collected packages: h5py
  Building wheel for h5py (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/miranda9/.conda/envs/my_new_env/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '&quot;'&quot;'/tmp/pip-install-bpmeop26/h5py/setup.py'&quot;'&quot;'; __file__='&quot;'&quot;'/tmp/pip-install-bpmeop26/h5py/setup.py'&quot;'&quot;';f=getattr(tokenize, '&quot;'&quot;'open'&quot;'&quot;', open)(__file__);code=f.read().replace('&quot;'&quot;'\r\n'&quot;'&quot;', '&quot;'&quot;'\n'&quot;'&quot;');f.close();exec(compile(code, __file__, '&quot;'&quot;'exec'&quot;'&quot;'))' bdist_wheel -d /tmp/pip-wheel-ccg1oj0n
       cwd: /tmp/pip-install-bpmeop26/h5py/
  Complete output (1321 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-ppc64le-3.7
  creating build/lib.linux-ppc64le-3.7/h5py
  copying h5py/__init__.py -&gt; build/lib.linux-ppc64le-3.7/h5py
  copying h5py/h5py_warnings.py -&gt; build/lib.linux-ppc64le-3.7/h5py
  copying h5py/highlevel.py -&gt; build/lib.linux-ppc64le-3.7/h5py
  copying h5py/ipy_completer.py -&gt; build/lib.linux-ppc64le-3.7/h5py
  copying h5py/version.py -&gt; build/lib.linux-ppc64le-3.7/h5py
  creating build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/__init__.py -&gt; build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/attrs.py -&gt; build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/base.py -&gt; build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/compat.py -&gt; build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/dataset.py -&gt; build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/datatype.py -&gt; build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/dims.py -&gt; build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/files.py -&gt; build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/filters.py -&gt; build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/group.py -&gt; build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/selections.py -&gt; build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/selections2.py -&gt; build/lib.linux-ppc64le-3.7/h5py/_hl
  copying h5py/_hl/vds.py -&gt; build/lib.linux-ppc64le-3.7/h5py/_hl
  creating build/lib.linux-ppc64le-3.7/h5py/tests
  copying h5py/tests/__init__.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests
  copying h5py/tests/common.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests
  creating build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/__init__.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_attrs.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_attrs_data.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_base.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_dataset.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_datatype.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_dimension_scales.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_file.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_file_image.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_group.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_h5.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_h5d_direct_chunk_write.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_h5f.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_h5p.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_h5t.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_objects.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_selections.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  copying h5py/tests/old/test_slicing.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/old
  creating build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/__init__.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_attribute_create.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_dataset_getitem.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_dataset_swmr.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_datatype.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_deprecation.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_dims_dimensionproxy.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_file.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_filters.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl
  copying h5py/tests/hl/test_threads.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl
  creating build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
  copying h5py/tests/hl/test_vds/__init__.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
  copying h5py/tests/hl/test_vds/test_highlevel_vds.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
  copying h5py/tests/hl/test_vds/test_lowlevel_vds.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
  copying h5py/tests/hl/test_vds/test_virtual_source.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
  running build_ext
  Autodetected HDF5 1.10.2
  ********************************************************************************
                         Summary of the h5py configuration
  
      Path to HDF5: None
      HDF5 Version: '1.10.2'
       MPI Enabled: False
  Rebuild Required: True
  
  ********************************************************************************
  Executing api_gen rebuild of defs
  Executing cythonize()
  [ 1/22] Cythonizing /tmp/pip-install-bpmeop26/h5py/h5py/_conv.pyx
  /tmp/pip-install-bpmeop26/h5py/.eggs/Cython-0.29.21-py3.7.egg/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-bpmeop26/h5py/h5py/_conv.pxd
    tree = Parsing.p_module(s, pxd, full_module_name)

...

  /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning &quot;Using deprecated NumPy API, disable it with &quot; &quot;#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION&quot; [-Wcpp]
   #warning &quot;Using deprecated NumPy API, disable it with &quot; \
    ^
  In file included from /tmp/pip-install-bpmeop26/h5py/h5py/defs.c:654:0:
  /tmp/pip-install-bpmeop26/h5py/h5py/api_compat.h:27:18: fatal error: hdf5.h: No such file or directory
   #include &quot;hdf5.h&quot;
                    ^
  compilation terminated.
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for h5py
  Running setup.py clean for h5py
Failed to build h5py
DEPRECATION: Could not build wheels for h5py which do not use PEP 517. pip will fall back to legacy 'setup.py install' for these. pip 21.0 will remove support for this functionality. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.
Installing collected packages: Pillow, h5py, torchmeta
  Attempting uninstall: Pillow
    Found existing installation: Pillow 7.1.2
    Uninstalling Pillow-7.1.2:
      Successfully uninstalled Pillow-7.1.2
  Attempting uninstall: h5py
    Found existing installation: h5py 2.8.0
    Uninstalling h5py-2.8.0:
      Successfully uninstalled h5py-2.8.0
    Running setup.py install for h5py ... error
    ERROR: Command errored out with exit status 1:
     command: /home/miranda9/.conda/envs/my_new_env/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '&quot;'&quot;'/tmp/pip-install-bpmeop26/h5py/setup.py'&quot;'&quot;'; __file__='&quot;'&quot;'/tmp/pip-install-bpmeop26/h5py/setup.py'&quot;'&quot;';f=getattr(tokenize, '&quot;'&quot;'open'&quot;'&quot;', open)(__file__);code=f.read().replace('&quot;'&quot;'\r\n'&quot;'&quot;', '&quot;'&quot;'\n'&quot;'&quot;');f.close();exec(compile(code, __file__, '&quot;'&quot;'exec'&quot;'&quot;'))' install --record /tmp/pip-record-hlwpfooj/install-record.txt --single-version-externally-managed --compile --install-headers /home/miranda9/.conda/envs/my_new_env/include/python3.7m/h5py


... 

    copying h5py/tests/hl/test_vds/test_lowlevel_vds.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
    copying h5py/tests/hl/test_vds/test_virtual_source.py -&gt; build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
    running build_ext
    Autodetected HDF5 1.10.2
    ********************************************************************************
                           Summary of the h5py configuration
    
        Path to HDF5: None
        HDF5 Version: '1.10.2'
         MPI Enabled: False
    Rebuild Required: True
    
    ********************************************************************************
    Executing cythonize()
    [ 1/22] Cythonizing /tmp/pip-install-bpmeop26/h5py/h5py/_conv.pyx
    /tmp/pip-install-bpmeop26/h5py/.eggs/Cython-0.29.21-py3.7.egg/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-bpmeop26/h5py/h5py/_conv.pxd


...

    warning: h5py/api_types_hdf5.pxd:730:6: 'H5Z_ERROR_EDC' redeclared
    warning: h5py/api_types_hdf5.pxd:731:6: 'H5Z_DISABLE_EDC' redeclared
    warning: h5py/api_types_hdf5.pxd:732:6: 'H5Z_ENABLE_EDC' redeclared
    warning: h5py/api_types_hdf5.pxd:733:6: 'H5Z_NO_EDC' redeclared
    building 'h5py.defs' extension
    creating build/temp.linux-ppc64le-3.7
    creating build/temp.linux-ppc64le-3.7/tmp
    creating build/temp.linux-ppc64le-3.7/tmp/pip-install-bpmeop26
    creating build/temp.linux-ppc64le-3.7/tmp/pip-install-bpmeop26/h5py
    creating build/temp.linux-ppc64le-3.7/tmp/pip-install-bpmeop26/h5py/h5py
    gcc -pthread -B /home/miranda9/.conda/envs/my_new_env/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DH5_USE_16_API -I./h5py -I/tmp/pip-install-bpmeop26/h5py/lzf -I/opt/local/include -I/usr/local/include -I/home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include -I/home/miranda9/.conda/envs/my_new_env/include/python3.7m -c /tmp/pip-install-bpmeop26/h5py/h5py/defs.c -o build/temp.linux-ppc64le-3.7/tmp/pip-install-bpmeop26/h5py/h5py/defs.o
    In file included from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1830:0,
                     from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                     from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                     from /tmp/pip-install-bpmeop26/h5py/h5py/api_compat.h:26,
                     from /tmp/pip-install-bpmeop26/h5py/h5py/defs.c:654:
    /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning &quot;Using deprecated NumPy API, disable it with &quot; &quot;#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION&quot; [-Wcpp]
     #warning &quot;Using deprecated NumPy API, disable it with &quot; \
      ^
    In file included from /tmp/pip-install-bpmeop26/h5py/h5py/defs.c:654:0:
    /tmp/pip-install-bpmeop26/h5py/h5py/api_compat.h:27:18: fatal error: hdf5.h: No such file or directory
     #include &quot;hdf5.h&quot;
                      ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
  Rolling back uninstall of h5py
  Moving to /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/h5py
   from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/~5py
  Moving to /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/h5py-2.8.0-py3.7.egg-info
   from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/~5py-2.8.0-py3.7.egg-info
ERROR: Command errored out with exit status 1: /home/miranda9/.conda/envs/my_new_env/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '&quot;'&quot;'/tmp/pip-install-bpmeop26/h5py/setup.py'&quot;'&quot;'; __file__='&quot;'&quot;'/tmp/pip-install-bpmeop26/h5py/setup.py'&quot;'&quot;';f=getattr(tokenize, '&quot;'&quot;'open'&quot;'&quot;', open)(__file__);code=f.read().replace('&quot;'&quot;'\r\n'&quot;'&quot;', '&quot;'&quot;'\n'&quot;'&quot;');f.close();exec(compile(code, __file__, '&quot;'&quot;'exec'&quot;'&quot;'))' install --record /tmp/pip-record-hlwpfooj/install-record.txt --single-version-externally-managed --compile --install-headers /home/miranda9/.conda/envs/my_new_env/include/python3.7m/h5py Check the logs for full command output.


anyone know how I can successfully install a working torchmeta version in a ppc64le (using wmcle 1.7.0)?

related:

gitissue for torchmeta: https://github.com/tristandeleu/pytorch-meta/issues/95

IBM gitissue for torchmeta support: https://github.com/IBM/powerai/issues/269

h5py gitissue for torchmeta: https://github.com/h5py/h5py/issues/1678

IBM h5py support for torchmeta: https://github.com/IBM/powerai/issues/270



Because there are not wheels for powerpc for h5py you are installing h5py from source (from the tarball).  This requires both the Python and h5py development headers to be available, see https://docs.h5py.org/en/stable/build.html#source-installation.
Either install h5py from conda or install the required build dependencies.
