title: AWS Glue Job Method pyWriteDynamicFrame does not exist
body: <p>My goal is to read dataframe from existing catalog table, make some transformations and create a new table out of it. So according to <a href="https://docs.aws.amazon.com/glue/latest/dg/update-from-job.html" rel="nofollow noreferrer">https://docs.aws.amazon.com/glue/latest/dg/update-from-job.html</a>, I use the <code>sink.writeFrame</code> method:</p>
  <pre><code>datasource0 = glueContext.create_dynamic_frame.from_catalog(database = &quot;my_db&quot;, table_name = &quot;table1&quot;, transformation_ctx = &quot;datasource0&quot;)
  datasource1 = datasource0.toDF().withColumn(&quot;date&quot;, current_date().cast(&quot;string&quot;))
  
  datasource2 = DynamicFrame.fromDF(datasource1, glueContext, &quot;datasource2&quot;)
                                                      
  sink = glueContext.getSink(connection_type=&quot;s3&quot;, path=&quot;s3://my_bucket/output&quot;, enableUpdateCatalog=True)
  sink.setFormat(&quot;json&quot;)
  sink.setCatalogInfo(catalogDatabase='my_db', catalogTableName='table2')
  sink.writeFrame(datasource2)
  
  job.commit()
  
  </code></pre>
  <p>But as a result I get a misleading error, that method pyWriteDynamicFrame doesn't exist:</p>
  <pre><code>Traceback (most recent call last):
    File &quot;/tmp/test&quot;, line 39, in &lt;module&gt;
      sink.writeFrame(datasource1)
    File &quot;/opt/amazon/lib/python3.6/site-packages/awsglue/data_sink.py&quot;, line 31, in writeFrame
      return DynamicFrame(self._jsink.pyWriteDynamicFrame(dynamic_frame._jdf, callsite(), info), dynamic_frame.glue_ctx, dynamic_frame.name + &quot;_errors&quot;)
    File &quot;/opt/amazon/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py&quot;, line 1257, in __call__
      answer, self.gateway_client, self.target_id, self.name)
    File &quot;/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/utils.py&quot;, line 63, in deco
      return f(*a, **kw)
    File &quot;/opt/amazon/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py&quot;, line 332, in get_return_value
      format(target_id, &quot;.&quot;, name, value))
  py4j.protocol.Py4JError: An error occurred while calling o75.pyWriteDynamicFrame. Trace:
  py4j.Py4JException: Method pyWriteDynamicFrame([class org.apache.spark.sql.Dataset, class java.lang.String, class java.lang.String]) does not exist
      at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
      at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326)
      at py4j.Gateway.invoke(Gateway.java:274)
      at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
      at py4j.commands.CallCommand.execute(CallCommand.java:79)
      at py4j.GatewayConnection.run(GatewayConnection.java:238)
      at java.lang.Thread.run(Thread.java:748)
  </code></pre>
  <p>Versions:
  Spark: 2.4, Python: 3, Glue: 2</p>