What sort of algorithm/communication model/data structure do collaborative real time editors use?
The usual approach is operational transformation. The wikipedia page seems to be pretty good. Here are some other references:


Understanding and Applying Operational Transformation
Operational Transformation: An Introduction
OT FAQ
ShareJS - javascript library implementing OT


Here's a paper on a different approach: Real time group editors without operational transformation. (Note: I haven't actually read it, so can't speak to how good it is)