What all can be said when you say that the CPU is 32 bit?
An $n$-bit processor is a processor for which the preferred integer size is $n$ bits.  That's usually the size of the integer or general purpose registers (a processor may not have exposed registers in its ISA, a processor may have some other kind of registers of different width, a processor may provide instructions to do some integer operations on different -- smaller or wider -- width, the width of buses used in the ISA implementation do not define its architectural width: there may be several implementations using buses of several sizes).

Data of that size is usually called a word, but when an ISA exist as a family and is extended to provide wider registers, the term word tend to continue to refer to data of the width adequate for the first member of the family (thus the continued use of word to refer to 16-bit quantities in the x86 world which has grow now to a 64-bit ISA).

The address space size is determined by the ISA width only if the ISA is using the same registers for address computation as for integer one.  That's a very common property of later architectures, but it has not always be the case (the 8086 used 24-bit addresses but its word size was 16 bits, Cray had 64-bit data register but its address registers were 24 bits IIRC).  Even when the registers used are the same, the amount of addressable memory may be different, either because some bits are not used for addresses (the 68000 for instance, and programmers making use of that caused issues for their followers when all of the bits were token into consideration), or because virtual memory allows for a process $n$-bit addressable space to be mapped into a wider physical address space (you could consider the 24-bit address of the 8086 a special case of that).