title: Spark scala - Nested StructType conversion to Map
body: <p>I am using Spark 1.6 in scala.</p>
  
  <p>I created an index in ElasticSearch with an object. The object "params" was created as a Map[String, Map[String, String]]. Example: </p>
  
  <pre><code>val params : Map[String, Map[String, String]] = ("p1" -&gt; ("p1_detail" -&gt; "table1"), "p2" -&gt; (("p2_detail" -&gt; "table2"), ("p2_filter" -&gt; "filter2")), "p3" -&gt; ("p3_detail" -&gt; "table3"))
  </code></pre>
  
  <p>That gives me records that look like the following:</p>
  
  <pre><code>{
          "_index": "x",
          "_type": "1",
          "_id": "xxxxxxxxxxxx",
          "_score": 1,
          "_timestamp": 1506537199650,
          "_source": {
             "a": "toto",
             "b": "tata",
             "c": "description",
             "params": {
                "p1": {
                   "p1_detail": "table1"
                },
                "p2": {
                   "p2_detail": "table2",
                   "p2_filter": "filter2"
                },
                "p3": {
                   "p3_detail": "table3"
                }
             }
          }
       },
  </code></pre>
  
  <p>Then I am trying to read the Elasticsearch index in order to update the values. </p>
  
  <p>Spark reads the index with the following schema:</p>
  
  <pre><code>|-- a: string (nullable = true)
  |-- b: string (nullable = true)
  |-- c: string (nullable = true)
  |-- params: struct (nullable = true)
  |    |-- p1: struct (nullable = true)
  |    |    |-- p1_detail: string (nullable = true)
  |    |-- p2: struct (nullable = true)
  |    |    |-- p2_detail: string (nullable = true)
  |    |    |-- p2_filter: string (nullable = true)
  |    |-- p3: struct (nullable = true)
  |    |    |-- p3_detail: string (nullable = true)
  </code></pre>
  
  <p>My problem is that the object is read as a struct. In order to manage and easily update the fields I want to have a Map as I am not very familiar with StructType.</p>
  
  <p>I tried to get the object in a UDF as a Map but I have the following error:</p>
  
  <pre><code> User class threw exception: org.apache.spark.sql.AnalysisException: cannot resolve 'UDF(params)' due to data type mismatch: argument 1 requires map&lt;string,map&lt;string,string&gt;&gt; type, however, 'params' is of struct&lt;p1:struct&lt;p1_detail:string&gt;,p2:struct&lt;p2_detail:string,p2_filter:string&gt;,p3:struct&lt;p3_detail:string&gt;&gt; type.;
  </code></pre>
  
  <p>UDF code snippet:</p>
  
  <pre><code>val getSubField : Map[String, Map[String, String]] =&gt; String = (params : Map[String, Map[String, String]]) =&gt; { val return_string = (params ("p1") getOrElse("p1_detail", null.asInstanceOf[String]) return_string }
  </code></pre>
  
  <p>My question: How can we convert this Struct to a Map? I already read saw the toMap method available in the documentation but can not find how to use it (not very familiar with implicit parameters) as I am a scala beginner.</p>
  
  <p>Thanks in advance,</p>