title: How to validate file data with Hive table data?
body: <p>i am trying to fetch a csv file which contains list of tables in a databases and rows and columns in each table  and compare with Hive tables. the intention is to match each table rows and columns in hive.</p>
  
  <blockquote>
    <p>You have log file like below. Like below many rows </p>
  </blockquote>
  
  <pre><code>DBName, Table name, rows, date 
  =========================== 
  bigdata, emp, 10000,01-01-2010 
  bigdata,finance,3000,03-09-2010 
  bugdata,mktg,2000,10-10-2010 
  </code></pre>
  
  <hr>
  
  <p>Now, write a program/script, to check these log file table rows are equal to Hive tables or not. If the table rows are equal print “Table rows are equal” else “Table rows are not equal” Assume that you have Hive and table names are same</p>
  
  <p>i have tried multiple things to traverse through dataframe also tried also with RDD but i am unable to pick each record of file and compare with each table in hive using SCALA.</p>
  
  <p>what i have tried: </p>
  
  <pre><code> val sc = new SparkContext()
      val readFileDB= sc.textFile("/databases.csv")
      val mapData= readFileDB.map(l=&gt;l.split(",")).map(c=&gt;(c(0),c(1),c(2)))
      val filterData = mapData.foreach{
        case (dbname,table,rows)=&gt; spark.catalog.setCurrentDatabase(dbname)
       } 
  </code></pre>