MemoryCache as a message broker?
What you have here is, IMHO, excellent.

Couple nitpicks: 


I don't get why RemoveAll is returning the original content instead of void - also I'm expecting an exception to be thrown in GetQueue if the specified key doesn't exist.
This part (in the Pop(string) method):

var queue = GetQueue(key);
string result = "";


Would look more consistent if written like this:

var queue = GetQueue(key);
var result = string.Empty;


Or, since it's used as an out parameter value on the next line (and therefore is compiler-enforced to be initialized after the call to TryDequeue(out string)), simply like this:

var queue = GetQueue(key);
string result;



As for the Clear extension method, there's a potential tiny little issue here - see this MSDN thread:


  ConcurrentQueue&lt;T&gt;'s implementation is based on a particular lock-free algorithm that doesn't permit atomically clearing of the collection, hence why no Clear method is exposed.  As such, you have two primary options.
  
  The first is simply to create a new ConcurrentQueue&lt;T&gt; instead of clearing the original, and then swap in the new for the original.
  
  The second is to continually remove from the collection until it's empty, e.g. T ignored; while(cq.TryDequeue(out ignored));. Which you choose is largely dependent on the needs of your application. 
  
  The former will be faster, but you also run the risk of losing some items you didn't intend to lose (then again, if the whole point is to empty the collection, that might not matter), and it only works if you have access to all of the variables holding references to the queue. 
  
  The latter is effective, but it runs the risk of running indefinitely if other threads are concurrently appending; it's also more expensive that simply allocating a new queue.