ObservablePriorityQueue Implementation
At first, I was confused what exactly did the implementation of IObservable&lt;T&gt; mean. You should properly document that.
You don't need to have a separate lock objects. If you're locking on a specific object, I think you should use that object in the lock.
If you're just calling a single method in your lambda, you can use more succinct syntax: new Disposable(() =&gt; this.Unsubscribe(observer)).
If you want to make your code thread-safe, you need to lock all your reads too. In Unsubscribe() you don't do that, which means someone could write to the list while you're reading it. The same problem is in Peek().
Dequeue() shouldn't return default(T) if the queue is empty. This can be problematic especially with value types. Instead, you should have a method like bool TryDequeue(out T result).
You're accessing Count outside of a lock. I wouldn't rely on the fact that doing this is safe, I think you should lock before accessing it too.
Since you're using Rx, you might as well use their Disposable.Create() instead of creating your own.