title: Spark Count number of lines with a particular word in it
body: <p>I have a log file which has lines containing the word "error" in it. How do i count the total number of lines containing this term in apache spark?</p>
  
  <p>So far i am using this approach.</p>
  
  <pre><code>from pyspark import SparkConf, SparkContext
  
  conf = SparkConf().setMaster("local").setAppName("WordCount")
  sc = SparkContext(conf = conf)
  
  input = sc.textFile("errors.txt")
  words = input.flatMap(lambda x: x for x if "errors" in input)
  wordCounts = input.countByValue()
  
  for word, count in wordCounts.items():
      print str(count)
  </code></pre>
  
  <p>But this method doesnt work. Can anyone tell me how do i get the count?</p>
  
  <p>Edit: Equivalent in scala is </p>
  
  <pre><code>lines = spark.textFile("hdfs://...")
  errors = lines.filter(_.startsWith("ERROR"))
  errors.persist()
  </code></pre>
  
  <p>what is the python equivalent of this line.</p>