Why an ARM processor with 32 bits address bus can address 4 billion different bytes? Why an ARM processor with 32 bits address bus can address 4 billion different bytes? 
I know that $2^{32}$ is equal to about 4 billions, but shouldn't it be 4 billion bits and not bytes?

Hence if I want to find how many different words it can address it should be $\frac{2^{32}}{8}$ so I will have the number of bytes, and if I want to find the number of words I divide it again by 4 (because a word in ARM is 32 bits = 4 bytes).
While there have been computers built that use bit addressing, notably the Burroughs 1700, 1800, and 1900 mainframes and the Intel iAPX-432, the vast majority of machines use byte addressing.

This means that each possible address corresponds to a potential byte in memory.
Thus a 32 bit address would have a span of $2^{32}$ or $4,294,967,296$ bytes.

For completeness, the other category of machines use word addressing, where each address value corresponds to a word of memory. Examples here include the PDP-10 and CDC-6600 as well as many specialized DSP and embedded processors still in use.