Duplicates when creating view with entity reference
You can give a try to Views Distinct module.

Also, if there the results the logically different you can think of showing them in way that they look more of information rather than repeated results
For example if you group by conference, you can show aggregated speakers for a given conferenece