How can I fix my rear derailleur's B-tension adjustment?
If I understand you correctly, you don't have a B-Tension issue. B-Tension sets the clearance between the upper derailleur pulley and the chain, and should not be used to add spring return force to a derailleur. 

In this case, I believe that the derailleur pivots are in need of cleaning and lubrication, and I would start there. 

If that doesn't fix the issue, you can look at replacing the derailleur return spring (#2 in photo)behind the fixing bolt of the derailleur and/or the cage spring (#1 in photo)between the derailleur body and the derailleur cage. 



Ultegra 6700 RD Manual 

Exploded View PDF