Find common value amongst rows
Group by reference, count distinct numbers in each group and compare that to the total count of distinct numbers in the table:

SELECT
  Reference
FROM
  dbo.atable
GROUP BY
  Reference
HAVING
  COUNT(DISTINCT Number) = (SELECT COUNT(DISTINCT Number) FROM dbo.atable)
;