title: Spark IllegalArgumentException: Column features must be of type struct<type:tinyint,size:int,indices:array<int>,values:array<double>>
body: <p>I'm trying to use the <code>org.apache.spark.ml.regression.LinearRegression</code> fit my data. So I've got the original RDD transformed to <em>dataframe</em>, and have tried to feed it to the <em>linearRegression</em> model.</p>
  <pre><code>val spark: SparkSession = SparkSession.builder.master(&quot;local&quot;).getOrCreate
  val parsedData = dataRDD.map{
    item =&gt;
      val doubleArray = Array(item._1.toDouble, item._2.toDouble, item._3.toDouble)
      val features = Vectors.dense(doubleArray)
      Row(item._4.toDouble, features)
  }
  
  val schema = List(
    StructField(&quot;label&quot;, DoubleType, true),
    StructField(&quot;features&quot;, new org.apache.spark.mllib.linalg.VectorUDT, true)
  )
  
  val df = spark.createDataFrame(
    parsedData,
    StructType(schema)
  )
  val lr = new LinearRegression()
    .setMaxIter(10)
    .setRegParam(0.3)
    .setElasticNetParam(0.8)
  
  val lr_model = lr.fit(df)
  </code></pre>
  <p>And here is what the dataframe looks like:</p>
  <pre><code>+---------+-------------+
  |    label|     features|
  +---------+-------------+
  |      5.0|[0.0,1.0,0.0]|
  |     20.0|[0.0,1.0,0.0]|
  |    689.0|[0.0,1.0,0.0]|
  |    627.0|[0.0,1.0,0.0]|
  |    127.0|[0.0,1.0,0.0]|
  |      0.0|[0.0,1.0,0.0]|
  |      0.0|[0.0,1.0,0.0]|
  |      0.0|[0.0,1.0,0.0]|
  |     76.0|[0.0,1.0,0.0]|
  |      5.0|[0.0,1.0,0.0]|
  |      0.0|[0.0,1.0,0.0]|
  |      0.0|[0.0,1.0,0.0]|
  |      0.0|[0.0,1.0,0.0]|
  |      0.0|[0.0,1.0,0.0]|
  |      0.0|[0.0,1.0,0.0]|
  |      2.0|[0.0,1.0,0.0]|
  |    329.0|[0.0,1.0,0.0]|
  |2354115.0|[0.0,1.0,0.0]|
  |      5.0|[0.0,1.0,0.0]|
  |   4303.0|[0.0,1.0,0.0]|
  +---------+-------------+
  </code></pre>
  <p>But it presented the error below.</p>
  <pre><code>java.lang.IllegalArgumentException: requirement failed: Column features must be of type struct&lt;type:tinyint,size:int,indices:array&lt;int&gt;,values:array&lt;double&gt;&gt; but was actually struct&lt;type:tinyint,size:int,indices:array&lt;int&gt;,values:array&lt;double&gt;&gt;.
  </code></pre>
  <p>The later data type doesn't seem to be different from the one required. Can anyone help?</p>