title: Why doesn't Spark Mongo connector push down filters?
body: <p>I have a large Mongo collection that I want to use in my Spark application, using Spark Mongo connector. The collection is quite large (>10 GB) and has daily data, with an index on <code>original_item.CreatedDate</code> field. Queries to select a couple of days in Mongo are extremely fast (under a second). However when I write the same query using dataframes, that filter is not pushed down to Mongo, resulting in extremely slow performance as Spark apparently fetches entire collection and does filtering itself.</p>
  
  <p>Query looks the following way:</p>
  
  <pre><code>collection
        .filter("original_item.CreatedDate  &gt; %s" % str(start_date_timestamp_ms)) \
        .filter("original_item.CreatedDate  &lt; %s" % str(end_date_timestamp_ms)) \
        .select(...)
  </code></pre>
  
  <p>In physical plan I see:
  <code>PushedFilters: [IsNotNull(original_item)]</code></p>
  
  <p>When I make a similar query with filtering on another field of that collection, mongo successfully pushes it down - <code>PushedFilters: [IsNotNull(original_item), IsNotNull(doc_type), EqualTo(doc_type,case)]</code>!</p>
  
  <p>Could it be the case that <code>GreaterThan</code> filter pushing is not supported by Mongo Spark connector, or that there is a bug with it?</p>
  
  <p>Thanks!</p>