Swap Fusion Drive for SSD
The short answer to this is YES.

I have done this exact thing in my iMac and it works perfectly.  You don't have to "uninstall" the Fusion drive.  Once you remove the HDD, you would have effectively destroyed the Fusion drive requiring the original (128GB) SSD to be reformatted.

I personally opted to create a new Fusion drive of the 128GB and 1TB Fusion drive for a 1.128TB drive, but you for format them and use them independently as well.