title: Scala RDD with pattern matching from a text file
body: <p><strong><em>kinda new to Spark</em></strong>. So here's the problem, I have a txt file with some general format, lets say: <code>Time</code> <code>:</code> <code>Message</code>  So i have to implement two things: RDD and pattern groups and matching.</p>
  
  <p>To take the file as an rdd:</p>
  
  <pre><code>val rdd1 = sc.textFile(location)
  </code></pre>
  
  <p>To Build the pattern:</p>
  
  <pre><code>private val f1 = "([0-9]*)"
  private val f2 = "([:])"
  private val f3 = "(.*?)"
  private val regex = s"$f1 $f2 $f3"
  private val p = Pattern.compile(regex)
  </code></pre>
  
  <p>now i want to integrate both of these, </p>
  
  <pre><code>rdd1.map(//What to do here) 
  </code></pre>
  
  <p>I want to check each line if it matches the general format.If it doesn't i want to display an error message for each line that doesn't match.</p>
  
  <p>if it does match, i wanna make groups for the pattern stated above. f1 is group1, f2 is group2, f3 is group three.Finally i wanna search f3(message field) for keywords like error,failure.</p>
  
  <p>I know this is a lot to ask for.Thx in advance.</p>
  
  <p>What i have already tried:</p>
  
  <pre><code>def parseLine(s1: String): Option[Groups] = {
  val matcher = p.matcher(s1)
  if (matcher.find) {
    println("General Format correct")
    //group
    Some(group(matcher))
    //after format is verified search f3 for error,failure keyword.
  
  }
  else {
    println("Format Wrong")
    None
  }
  }
  
  def group(matcher: Matcher) = {
  Line(
    matcher.group(1),
    matcher.group(2),
    matcher.group(3))}
  
  case class Line(Time: String, colon: String, Message: String)
  </code></pre>
  
  <p>Now i am stuck on how to iterate the rdd to pass each line of the text file to the function.if i pass the entire rdd to the function ie, RDD[String] type.Other elements like matcher doesn't work cause it expects String type.
  While reading about the rdd functions:(correct me if i am wrong), the foreach method should iterate the rdd but i get type mismatches. currently trying the map function but havn't got it yet.</p>
  
  <p>As i said i am new to spark rdd's.I don't know if using partition function would help me out instead of grouping.</p>
  
  <p>I really need some guidance from the experianced. Any help is appriciated.</p>