title: Mongoose Populate after Aggregate
body: <p>I am trying to get a specific data model after I run an aggregate pipeline followed by populate but I am falling just short of it.</p>
  
  <p>The desired result in the end is the following:</p>
  
  <pre><code>[
    {
      _accountId: "5beee0966d17bc42501f1234",
      name: "Company Name 1",
      contactEmail: "email1@email.com",
      contactName: "contact Name 1"
      reason: "Warranties",
      total: 1152,
      lineItems: [
        {
          _id: "5beee0966d17bc42501f5086",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        },
        {
          _id: "5bf43929e7179a56e21382bc",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        },
        {
          _id: "5bf4392fe7179a56e21382bd",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        }
      ]
    },
    {
      _accountId: "5beee0966d17bc42501f1235",
      name: "Company Name 2",
      contactEmail: "email2@email.com",
      contactName: "contact Name 2"
      reason: "Warranties",
      total: 1152,
      lineItems: [
        {
          _id: "5beee0966d17bc42501f5086",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        },
        {
          _id: "5bf43929e7179a56e21382bc",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        },
        {
          _id: "5bf4392fe7179a56e21382bd",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        }
      ]
    }
  ]
  </code></pre>
  
  <p>I am collecting this data from the following two models:</p>
  
  <p><strong>Warranty</strong></p>
  
  <pre><code>{
    _id: "5beee0966d17bc42501f5086",
    jobsiteAddressStreet: String,
    jobsiteAddressCity: String,
    jobsiteAddressState" String,
    jobsiteAddressZip: Number,
    warrantyFee: Number,
    _accountId: {
      type: Schema.Types.ObjectId,
      ref: "accounts"
    },
    payStatus: String
  }
  </code></pre>
  
  <p><strong>Account</strong></p>
  
  <pre><code>{
    _id: "5beee0966d17bc42501f1235",
    name: String,
    contactName: String,
    contactEmail: String
  }
  </code></pre>
  
  <p>My current query is the following:</p>
  
  <pre><code>Warranty.aggregate([
      {
        $match: {
          payStatus: "Invoiced Next Billing Cycle"
        }
      },
      {
        $group: {
          _id: "$_accountId",
          total: {
            $sum: "$warrantyFee"
          },
          lineItems: {
            $push: {
              _id: "$_id",
              jobsiteAddress: {
                $concat: [
                  "$jobsiteAddressStreet",
                  " ",
                  "$jobsiteAddressCity",
                  ", ",
                  "$jobsiteAddressState",
                  " ",
                  "$jobsiteAddressZip"
                ]
              },
              warrantyFee: "$warrantyFee"
            }
          }
        }
      },
      {
        $project: {
          reason: "Warranties",
          total: "$total",
          lineItems: "$lineItems"
        }
      }
    ])
      .then(warranties =&gt; {
        console.log(warranties);
        Account.populate(warranties, {
          path: "_id",
          select: "contactName contactEmail name"
        })
          .then(warranties =&gt; {
            res.send(warranties);
          })
          .catch(err =&gt; {
            res.status(422).send(err);
            throw err;
          });
      })
      .catch(err =&gt; {
        res.status(422).send(err);
        throw err;
      });
  </code></pre>
  
  <p>Which results in the following:</p>
  
  <pre><code>[
    {
      _id: {
        _id: "5bc39dfa331c0e2cb897b61e",
        name: "Company Name 1",
        contactEmail: "email1@email.com",
        contactName: "Contact Name 1"
      },
      reason: "Warranties",
      total: 1152,
      lineItems: [
        {
          _id: "5beee0966d17bc42501f5086",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        },
        {
          _id: "5bf43929e7179a56e21382bc",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        },
        {
          _id: "5bf4392fe7179a56e21382bd",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        }
      ]
    },
    {
      _id: {
        _id: "5bc39dfa331c0e2cb897b61e",
        name: "Company Name 2",
        contactEmail: "email2@email.com",
        contactName: "Contact Name 2"
      },
      reason: "Warranties",
      total: 1152,
      lineItems: [
        {
          _id: "5beee0966d17bc42501f5086",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        },
        {
          _id: "5bf43929e7179a56e21382bc",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        },
        {
          _id: "5bf4392fe7179a56e21382bd",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        }
      ]
    }
  ]
  </code></pre>
  
  <p>As you can see this is really close with some minor issues.</p>
  
  <ol>
  <li>It shows _id instead of _accountId.  I defaulted to this because whenever I try to return the _accountId in $group it labels it as a non-accumulator field and when I do it in $project it just doesn't show up.  The data sets must be grouped by the _accountId in the Warranty model.</li>
  <li>I would prefer to add the additional (contactName, contactEmail, name) fields to the top level object instead of creating a subdocument if possible.  This may be simple or impossible as I am not super familiar with populate but can't find anything to answer my question directly.</li>
  </ol>
  
  <p>The goal at the end of this is to take the returned object and use the array of objects to do a bulk create of documents to another collection.</p>
  
  <p>-- Answer to my specific use case --</p>
  
  <pre><code>Warranty.aggregate([
      {
        $match: {
          payStatus: "Invoiced Next Billing Cycle"
        }
      },
      {
        $group: {
          _id: "$_accountId",
          total: {
            $sum: "$warrantyFee"
          },
          lineItems: {
            $push: {
              _id: "$_id",
              jobsiteAddress: {
                $concat: [
                  "$jobsiteAddressStreet",
                  " ",
                  "$jobsiteAddressCity",
                  ", ",
                  "$jobsiteAddressState",
                  " ",
                  "$jobsiteAddressZip"
                ]
              },
              warrantyFee: "$warrantyFee"
            }
          }
        }
      },
      {
        $lookup: {
          from: Account.collection.name,
          localField: "_id",
          foreignField: "_id",
          as: "accounts"
        }
      },
      {
        $unwind: "$accounts"
      },
      {
        $project: {
          lineItems: "$lineItems",
          reason: "Warranties",
          total: "$total",
          type: "Invoice",
          date: new Date(),
          company: "$accounts.name",
          contactName: "$accounts.contactName",
          contactEmail: "$accounts.contactEmail"
        }
      },
      {
        $addFields: {
          _accountId: "$_id"
        }
      },
      {
        $project: {
          _id: 0
        }
      }
    ])
  </code></pre>
  
  <p>This gives me the result:</p>
  
  <pre><code>[
    {
      lineItems: [
        {
          _id: "5be203eb3afd8098d4988152",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        }
      ],
      reason: "Warranties",
      total: 384,
      type: "Invoice",
      date: "2018-11-21T14:08:15.052Z",
      company: "Company Name 1",
      contactName: "Contact Name 1",
      contactEmail: "email1@email.com",
      _accountId: "5be203eb3afd8098d4988152",
      referenceNumber: 1542809296615
    },
    {
      lineItems: [
        {
          _id: "5beee0966d17bc42501f5086",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        },
        {
          _id: "5bf43929e7179a56e21382bc",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        },
        {
          _id: "5bf4392fe7179a56e21382bd",
          jobsiteAddress: "1234 Street Southwest Sunnyville, Wyoming 12345",
          warrantyFee: 384
        }
      ],
      reason: "Warranties",
      total: 1152,
      type: "Invoice",
      date: "2018-11-21T14:08:15.052Z",
      company: "Company Name 2",
      contactName: "Contact Name 2",
      contactEmail: "email2@email.com",
      _accountId: "5bc39dfa331c0e2cb897b61e",
      referenceNumber: 1542809295680
    }
  ]
  </code></pre>