Understanding a PID controller
The output of your tension sensor aren't Newtons, but volts or miliamps or wathever.Then yes if you multiply all conversion units in that loop, you will get for Kp something like rpm/N.
If the error becomes zero, then the output of regulator will not be zero if it has an integrator. Integrator "stores" the output state, sytem without integration cannot acheive zero static error, you will find this in any book. In case you would use only P-regulator (Kp), then if the error is zero the output is also zero, but this is not the case if you use PI regulator.  

Your application can be also further expanded into multiple loops. If you know the speed of the next roller, then first you have to match speed only then you can add/subtract deviation speed with use of tension sensor. This approach is better.