How does Bitcoin Core sort memory pool transactions by fee? I'm trying to understand the algorithm Bitcoin Core uses to sort memory pool transactions by fee.

Gavin's fee rework proposal states:


  Next, sort all remaining memory pool transactions by fee-paid-per-kilobyte, and include as many as will fit until the block is maximum block size bytes big, not including "free" transactions (transactions with fee-per-kb of less than the default spam threshold of 0.0001 BTC/kilobyte).


This makes sense, because a miner should want to load a block with transactions carrying the most "fee density", or fee per byte.

However, reviewing the CreateNewBlock function in miner.cpp, it's not clear how Bitcoin Core accounts for transaction length when prioritizing memory pool items:

// Prioritise by fee once past the priority size or we run out of high-priority
// transactions:
if (!fSortedByFee &amp;&amp;
    ((nBlockSize + nTxSize &gt;= nBlockPrioritySize) || !AllowFree(dPriority)))
{
    fSortedByFee = true;
    comparer = TxPriorityCompare(fSortedByFee);
    std::make_heap(vecPriority.begin(), vecPriority.end(), comparer);
}


TXPriorityCompare seems to just look at absolute fee:

public:
    TxPriorityCompare(bool _byFee) : byFee(_byFee) { }

    bool operator()(const TxPriority&amp; a, const TxPriority&amp; b)
    {
        if (byFee)
        {
            if (a.get&lt;1&gt;() == b.get&lt;1&gt;())
                return a.get&lt;0&gt;() &lt; b.get&lt;0&gt;();
            return a.get&lt;1&gt;() &lt; b.get&lt;1&gt;();
        }
        else
        {
            if (a.get&lt;0&gt;() == b.get&lt;0&gt;())
                return a.get&lt;1&gt;() &lt; b.get&lt;1&gt;();
            return a.get&lt;0&gt;() &lt; b.get&lt;0&gt;();
        }
    }
};


In other words, this looks like a sort on absolute transaction fee, not fee density. What am I missing?
The excerpt you're showing is for the priority area in blocks. The concept of "priority" in the source code corresponds modified bitcoin-days-destroyed per kilobyte. It is only used for a small portion of the block size (max 50kB by default in Bitcoin Core 0.11.x, and max 0kB by default in 0.12.x, it can be changed with a configuration setting).

Look at the rest of CreateNewBlock, which uses sorting by feerate, by iterating an index on the mempool that is pre-sorted by feerate (=satoshi per kB).