What are the units of utility?
No, you can not really say that. Here is the reason:

Utility is generally used to describe preferences. Let's say my happiness is described by this utility of chocolates: $Y(X) = X$. So, two chocolates will give me two units of happiness. However, by definition of preferences, I am allowed to apply monotone transformations to them. Hence, you can just transform that function to $Y(X) = X^2$. Now two chocolates will give me 4 units of happiness. 

Again, by definition, this is a completely valid transformation. However, now I am getting 4 dollars instead of $2 because I simply transformed the function. For obvious reasons, you can not do that.

Hence, utility is unitless.