Macbook Pro 15" Late 2011 - Swap HDD with SSD - Fan always 2000 rpm
You most likely have a malfunctioning sensor, and it's not necessarily restricted to the CPU.

There are sensors for ambient air, the heat sink/pipe, the palm rest, the CPU, the GPU, the hard drive, etc.  Any/all of these could be malfunctioning (or detecting) a malfunction to cause the SMC to spin up your fan.

To diagnose this problem:


Return your MacBook Pro to it's original configuration.  If the problem goes away, then you know it's due to an issue that was introduced due to the modification.
Run Apple Diagnostics.  Regardless if the problem goes away in the previous step, run Apple Hardware Test (AHT).  Hold the D key while booting from a powered off state with the AC adapter connected.    If there is an problem, this tool will likely tell you what the issue is (if there is one).


No issue - you have a good baseline to start working with
Issue - you will have a diagnostic code to start working with to solve the problem

Install the SSD in the HDD location.  Since this is your objective, this is the next logical step.  If you encounter issues here, it's related to the SSD installation.  You will want to run AHT here again to verify all is well.
Install the HDD in the optical bay caddy.  This being your final step, install it and check performance.  Run AHT again. Verify that you don't get any issues.


The key here is to do each step individually to see where something could have gone wrong.  It's also entirely plausible with the disconnection/reconnection of components something wasn't connected properly.  Double check your work. Double check that you didn't accidentally knock something loose.

A note on the inner workings of your MBP


The fans never stop, they are always spinning.  1000 RPM is typically the lowest speed.  
Fans spin up because a sensor indicated there was a temp spike and the SMC responded by spinning up the fan
Controlling the fan is a bad idea.  Fans remove heat that's generated by the different components.  If the fan can't remove the heat (quickly enough) it will throttle your CPU to generate less heat.  


When you control the fan, you treat the symptom.  If you really must have it quiet, treat the cause and lower your CPU speed, thereby reducing the heat, the fans will follow.  You will sacrifice performance but not endanger your CPU.