Can I remove User Dictionary Application itself ? not just words in it? My process monitor shows the User Dictionary Application is taking up all the CPU the device has got so even phonecaller app crashes. 

I'd like to remove the whole User Dictionary Application if possible.
I don't want to use this feature anyway.

Would I be able to remove it ? 

If I could, would removing User Dictionary cause any serious issues ?
On my Galaxy S3, running 4.1.2, the User dictionary is a background service that can be disabled. If you are running a similar device look into the following:

Settings\App Manager\All\User Dictionary
Click the disable button.

If you are using a different device or system this could get you started in the right direction.