title: An error occurred while calling o137.partitions. : org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://ip
body: <p>I am trying to execute this github project in an aws emr spark cluster
  <a href="https://github.com/pran4ajith/spark-twitter-streaming.git" rel="nofollow noreferrer">https://github.com/pran4ajith/spark-twitter-streaming.git</a></p>
  <p>I've succeeded to run 2 fisrt codes</p>
  <ul>
  <li>tweet_stream_producer.py</li>
  <li>sparkml_train_model.py</li>
  </ul>
  <p>But when I run consumer part with command</p>
  <pre><code>spark-submit --packages org.apache.spark:spark-sql-kafka-0-10_2.12:3.0.0,io.delta:delta-core_2.12:0.7.0 tweet_stream_consumer.py
  </code></pre>
  <p>I got file path error</p>
  <pre><code>Py4JJavaError: An error occurred while calling o137.partitions.
  : org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://ip-10-0-0-61.ec2.internal:8020/home/hadoop/spark-twitter-streaming/TwitterStreaming/src/app/models/metadata
  </code></pre>
  <p>It seems that the problem is located mapping between local file system path and hadoop file system path</p>
  <pre><code>model_path = str(SRC_DIR / 'models')
  
  pipeline_model = PipelineModel.load(model_path)
  </code></pre>