How Warm Was the Paleocene?
Redrawn after Zachos et al. 2001.

This is a plot of the Cenozoic variations of $\delta\ce{^{18}O}_{Cibicidoides}$ according to Zachos et al. 2001 (this is a bit outdated now but the Paleocene is roughly the same in more modern curves such as Zachos et al. 2008 or Cramer et al. 2011). It is supposed to follow variations in deep ocean temperatures and thus reflect the climate state at a given time. The Paleocene is the timeframe delimited by the two vertical bars (you can see the PETM quite clearly on one of the two bars).
From that plot it seems the Paleocene deep-sea temperature prior to the PETM is roughly at the same level at it is during the Middle Eocene (ca. 45 Ma), i. e. warmer than most of the Cenozoic but colder than the Early Eocene or most of the Cretaceous.
There isn't actually that many studies focussing on the Paleocene itself, this is clearly a time interval that is studied so far mostly because of the K/T boundary or the PETM, i. e either because of its start or its end.