Do I need to implement multi precision arithmetic operations? I want to implement Elliptic Curves arithmetics (for edu purpose and better understanding) for special NIST primes: point addition, exponentiation, etc.. all operations needed for EC encryption/decryption, digital signatures, and key exchanges. I'm interested in only special NIST primes, not in implementation for a general curve. Do I need to implement a multi-precision arithmetic for some fixed word size (32, 64) such as add/sub, reduction, inversion, Montgomery reduction, and so on? As far as I understand these primes are selected so that implementation is fast comparing to a general implementation.
Whatever the curve, it's needed to use multi-precision integer arithmetic in ECC cryptography, at least for a few modular operations modulo the Ellitic Curve group order, since that's large (192-bit is the bare modern minimum). For the field, where the most compute-intensive operations are, there are alternatives with fields other than prime order groups. See for example Thomas Pornin's Efficient Elliptic Curve Operations On Microcontrollers With Finite Field Extensions (note: in this work, extensions applies to field, not microcontrollers).
For curves on prime-order field (as in the question), it's indispensable to have multi-precision integer arithmetic for field operations, and the performance of modular operations is central to the overall performance. Special attention to side channels, including timing, must be taken when manipulating confidential information (when signing and decrypting, and to some degree when encrypting; that's usually not a concern for signature verification).
For prime field using special NIST primes (as in the question), speed optimizations are possible. That's the reason to use such primes. All correct generic multi-precision integer arithmetic techniques and libraries (including GMP) work with these primes, but in general do not take advantage of the special form for speedup. Such techniques and libraries are thus adequate to get correct results, but not to get the best possible performance. More often that not, they also do not give adequate protection against side-channel leakage. Only some functions of GMP are written with this in mind.