title: Print df in python class
body: <p>I would like to print <code>spark_df</code>dataframe without rerunning the entire query. In the first step I am running the <code>run_query</code> function and print the df and in the second step I would like to print <code>spark_df</code> by accessing <code>new_df</code> that has been saved in memory to avoid re-running the entire query.</p>
  <p>I added the following code snippet after the <code>pandas_to_spark</code> function but I none is returned</p>
  <pre><code>report = DataPipeline()
  DataPipeline.pandas_to_spark(report)
  print(report.spark_df)
  </code></pre>
  <pre><code>class DataPipeline:
      def __init__(self):
          self.new_df = None 
          self.spark_df = None
          
      def run_script(self):
          self.run_query() #return new_df in memory 
          self.pandas_to_spark()
  
     def run_query(self):
          &quot;&quot;&quot; Run query  &quot;&quot;&quot;
          query = '''select * from table'''
          query = pd.read_sql_query(query, conn_lake)
          new_df = query.fillna('NaN')
          self.new_df = new_df 
          return self
  
     def pandas_to_spark(self): 
          &quot;&quot;&quot; Function to convert pandas df to spark df &quot;&quot;&quot;
          mySchema = StructType([StructField(&quot;date&quot;, StringType(), True)
                                 , StructField(&quot;customer_id&quot;, StringType(), True)])
          new_df = self.new_df
          spark_df = spark.createDataFrame(new_df, schema=mySchema)
          self.spark_df = spark_df 
          return self 
  
  
  report = DataPipeline()
  DataPipeline.run_query(report)
  print(report.new_df)
  </code></pre>