title: Query running very slow on big MongoDB db
body: <p>I have a MongoDB db with a single rather large collection of documents (13GB for about 2M documents) sitting on a single server with 8GB RAM. Each document has a text field that can be relatively large (it can be a whole blog post) and the other fields are data about the text content and the text author. Here's what the schema looks like:</p>
  
  <pre><code>{
      text: "Last night there was a storm in San Francisco...",
      author: {
          name: "Firstname Lastname",
          website_url: "http://..."
      },
      date: "201403075612",
      language: "en",
      concepts: [
          {name: "WeatherConcept", hit: "storm", start: 23, stop: 28},
          {name: "LocationConcept", hit: "San Francisco", start: 32, stop: 45}
      ],
      location: "us",
      coordinates: []
  }
  </code></pre>
  
  <p>I'm planning to query the data in different ways:</p>
  
  <ol>
  <li><p>Full-text search on the "text" field. So let's say my text search query is q:</p>
  
  <pre><code>db.coll.aggregate([
      {
          $match:{
              $text: {
                  $search:q
              }
          }
      }
  ])
  </code></pre></li>
  <li><p>Aggregate documents by author:</p>
  
  <pre><code>db.coll.aggregate([
      {
          $project: {
              name: "$author.name",
              url: "$author.website_url"
          }
      },
      {
          $group: {
              _id: "$name",
              size: {
                  $sum:1
              },
              url: {
                  $first: "$url"
              }
          }
      },
      {
          $sort:{
              size:-1
          }
      }
  ])
  </code></pre></li>
  <li><p>Aggregate documents by concepts: </p>
  
  <pre><code>db.coll.aggregate([
      {
          $unwind: "$concepts"
      },
      {
          $group: {
              _id: "$concepts.name",
              size: {
                  $sum:1
              }
          }
      },
      {
          $sort:{
              size:-1
          }
      }
  ])
  </code></pre></li>
  </ol>
  
  <p>These three queries may also include filtering on the following fields: date, location, coordinates, language, author.</p>
  
  <p>I don't have indexes yet in place, so the queries run very slow. But since the indexes would be very different for the different ways I hit the data, does that rule out indexing as a solution? Or is there a way to index for all these cases and not have to shard the collection? Basically my questions are:</p>
  
  <ul>
  <li>What would be a good indexing strategy in this case? </li>
  <li>Do I need to create separate collections for authors and concepts?</li>
  <li>Should I somehow restructure my data?</li>
  <li>Do I need to shard my collection or is my 8GB single-server powerful enough to handle that data?</li>
  </ul>