What is required in Affinity Propagation
According to sklearn.cluster.AP (in case in AP chosen affinity = "precomputed") the method "fit" requires X  where X is matrix of similarities / affinities, that is, if distance matrix is estimated, use as:

X $ = matrix\ of\ similarities = -distance\ matrix$.