Reading files with hdfs3 fails
Question: I am trying to read a file on HDFS with Python using the hdfs3 module.

    
    
    import hdfs3
    hdfs = hdfs3.HDFileSystem(host='xxx.xxx.com', port=12345)
    hdfs.ls('/projects/samplecsv/part-r-00000')
    

This produces

    
    
    [{'block_size': 134345348,
      'group': 'supergroup',
      'kind': 'file',
      'last_access': 1473453452,
      'last_mod': 1473454723,
      'name': '/projects/samplecsv/part-r-00000/',
      'owner': 'dr',
      'permissions': 420,
      'replication': 3,
      'size': 98765631}]
    

So it seems to be able to access the HDFS and read the directory structure.
However, reading the file fails.

    
    
    with hdfs.open('/projects/samplecsv/part-r-00000', 'rb') as f:
        print(f.read(100))
    

gives

    
    
    ---------------------------------------------------------------------------
    OSError                                   Traceback (most recent call last)
    .
    .<snipped>
    .
    
    OSError: [Errno Read file /projects/samplecsv/part-r-00000 Failed:] 1
    

What could be the issue? I am using Python3.5.


Answer: if You want any operation on files then you have to pass full File path .

    
    
    import hdfs3
    hdfs = hdfs3.HDFileSystem(host='xxx.xxx.com', port=12345)
    hdfs.ls('/projects/samplecsv/part-r-00000')
    
    #you have to add file to location
    hdfs.put('local-file.txt', '/projects/samplecsv/part-r-00000')
    
    with hdfs.open('projects/samplecsv/part-r-00000/local-file.txt', 'rb') as f:
        print(f.read(100))
    

