title: Running tensorflow with file on HDFS (cannot find libhdfs.so)
body: <p>I am running into an error "libhdfs.so: cannot open shared object file: No such file or directory" (stack trace below) while trying to run a python script invoking a Tensorflow reader on a file stored in HDFS. I am running the script  on a node on the cluster which has Tensorflow in a virtualenv, activated at the time of execution. I set the following environment variables before execution:</p>
  
  <ul>
  <li>export HADOOP_HDFS_HOME=$HADOOP_HDFS_HOME:/opt/cloudera/parcels/CDH</li>
  <li>export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64</li>
  <li>export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/cloudera/parcels/CDH/lib/libhdfs.so</li>
  <li>export
  LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${JAVA_HOME}/jre/lib/amd64/server</li>
  </ul>
  
  <p>I execute the script as such:</p>
  
  <ul>
  <li>CLASSPATH=$($LD_LIBRARY_PATH} classpath --glob) python TEST.py</li>
  </ul>
  
  <p>This is the code in the script:</p>
  
  <pre><code>filename_queue = tf.train.string_input_producer([   
     "hdfs://hostname:port/user/hdfs/test.avro" ]) 
  reader =
     tf.WholeFileReader() key, value = reader.read(filename_queue)
  
     with tf.Session() as sess:
         coord = tf.train.Coordinator()
         threads = tf.train.start_queue_runners(coord=coord)
         sess.run([key,value])
         coord.request_stop()
         coord.join(threads)
  </code></pre>
  
  <p>Below is the stack trace of the error. Any ideas on what is causing this error is appreciated (I have already checked that the LD_LIBRARY_PATH variable has an explicit pointer to the libhdfs.so file prior to execution, unable to figure out why it still cannot find the file).</p>
  
  <pre><code>Traceback (most recent call last):
    File "TEST.py", line 25, in &lt;module&gt;
      sess.run([key,value])
    File "/home/username/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 767, in run
      run_metadata_ptr)
    File "/home/username/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 965, in _run
      feed_dict_string, options, run_metadata)
    File "/home/username/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1015, in _do_run
      target_list, options, run_metadata)
    File "/home/username/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1035, in _do_call
      raise type(e)(node_def, op, message)
  tensorflow.python.framework.errors_impl.NotFoundError: libhdfs.so: cannot open shared object file: No such file or directory
       [[Node: ReaderReadV2 = ReaderReadV2[_device="/job:localhost/replica:0/task:0/cpu:0"](WholeFileReaderV2, input_producer)]]
  
  Caused by op u'ReaderReadV2', defined at:
    File "TEST.py", line 19, in &lt;module&gt;
      key, value = reader.read(filename_queue)
    File "/home/username/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/io_ops.py", line 272, in read
      return gen_io_ops._reader_read_v2(self._reader_ref, queue_ref, name=name)
    File "/home/username/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_io_ops.py", line 410, in _reader_read_v2
      queue_handle=queue_handle, name=name)
    File "/home/username/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
      op_def=op_def)
    File "/home/username/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2395, in create_op
      original_op=self._default_original_op, op_def=op_def)
    File "/home/username/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1264, in __init__
      self._traceback = _extract_stack()
  
  NotFoundError (see above for traceback): libhdfs.so: cannot open shared object file: No such file or directory
  </code></pre>