title: How can I log sql execution results in airflow?
body: <p>I use airflow python operators to execute sql queries against a redshift/postgres database. In order to debug, I'd like the DAG to return the results of the sql execution, similar to what you would see if executing locally in a console:</p>
  
  <p>I'm using <code>psycop2</code> to create a connection/cursor and execute the sql. Having this logged would be extremely helpful to confirm the parsed parameterized sql, and confirm that data was actually inserted (I have painfully experiences issues where differences in environments caused unexpected behavior)</p>
  
  <p>I do not have deep knowledge of airflow or the low level workings of the python DBAPI, but the <code>pscyopg2</code> documentation does seem to refer to some methods and connection configurations that may allow this.</p>
  
  <p>I find it very perplexing that this is difficult to do, as I'd imagine it would be a primary use case of running ETLs on this platform. I've heard suggestions to simply create additional tasks that query the table before and after, but this seems clunky and ineffective.</p>
  
  <p>Could anyone please explain how this may be possible, and if not, explain why? Alternate methods of achieving similar results welcome. Thanks!</p>
  
  <p>So far I have tried the <code>connection.status_message()</code> method, but it only seems to return the first line of the sql and not the results. I have also attempted to create a logging cursor, which produces the sql, but not the console results</p>
  
  <pre class="lang-py prettyprint-override"><code>import logging
  import psycopg2 as pg
  from psycopg2.extras import LoggingConnection
  
  conn = pg.connect(
      connection_factory=LoggingConnection,
      ...
  )
  conn.autocommit = True
  
  logging.basicConfig(level=logging.DEBUG)
  logger = logging.getLogger(__name__)
  logger.addHandler(logging.StreamHandler(sys.stdout))
  conn.initialize(logger)
  
  cur = conn.cursor()
  
  sql = """    
      INSERT INTO mytable (
      SELECT *
      FROM other_table
      );
  """
  
  cur.execute(sql)
  </code></pre>
  
  <p>I'd like the logger to return something like:</p>
  
  <pre><code>sql&gt; INSERT INTO mytable (
       SELECT ...
  [2019-07-25 23:00:54] 912 rows affected in 4 s 442 ms
  </code></pre>