title: dataframe look up and optimization
body: <p>I am using spark-sql-2.4.3v with java.
  I have scenario below</p>
  <pre><code>val data = List(
    (&quot;20&quot;, &quot;score&quot;, &quot;school&quot;,  14 ,12),
    (&quot;21&quot;, &quot;score&quot;, &quot;school&quot;,  13 , 13),
    (&quot;22&quot;, &quot;rate&quot;, &quot;school&quot;,  11 ,14),
    (&quot;23&quot;, &quot;score&quot;, &quot;school&quot;,  11 ,14),
    (&quot;24&quot;, &quot;rate&quot;, &quot;school&quot;,  12 ,12),
    (&quot;25&quot;, &quot;score&quot;, &quot;school&quot;, 11 ,14)
   )
  val df = data.toDF(&quot;id&quot;, &quot;code&quot;, &quot;entity&quot;, &quot;value1&quot;,&quot;value2&quot;)
  df.show
  
  //this look up data populated from DB.
  
  val ll = List(
     (&quot;aaaa&quot;, 11),
    (&quot;aaa&quot;, 12),
    (&quot;aa&quot;, 13),
    (&quot;a&quot;, 14)
   )
  val codeValudeDf = ll.toDF( &quot;code&quot;, &quot;value&quot;)
  codeValudeDf.show
  </code></pre>
  <p>I need to map &quot;code&quot; with &quot;value&quot; in the final output, only for those rows/records which has &quot;code&quot; as &quot;score&quot; in the &quot;data&quot; dataframe.</p>
  <p>How can i make a look up hashmap from codeValudeDf , so that I can get output as below</p>
  <pre><code>+---+-----+-------+------+-----+
  | id| code|entity|value1|value2|
  +---+-----+-------+------+-----+
  | 20|score|school|     a|   aaa|
  | 21|score|school|    aa|    aa|
  | 22| rate|school|    11|    14|
  | 23|score|school|  aaaa|     a|
  | 24| rate|school|    12|    12|
  | 25|score|school|  aaaa|     a|
  +---+-----+------+------+------+
  </code></pre>
  <p>Is there any possibility to make this look up optimum i.e. every time i should not pull the dataframe data from DB ??</p>