How do you make an SSS texture (Using Principled BSDF) only affect part of a mesh? (Image/gif inside) (Albedo texture on left and SSS texture on right just for reference) https://i.stack.imgur.com/veTGy.png

I've been trying to figure out how to have an SSS shader apply only to certain parts of the mesh it's attached to (The skin, usually) with little luck.

It's not making the entire mesh look like a tomato, but it gives all parts of the texture that aren't skin a strange artifacting(?) effect that makes all parts of the model that aren't skin look terrible. Shown here: https://imgur.com/a/ADmiXiG (Note that the Subsurface intensity starts at 0.1 in this gif, then goes to 0 (disabled), and ends on 1.) 

Strangely enough, even if the entire texture is transparent other than the skin itself this weird effect still happens ("Use Alpha" was enabled on the texture too, I checked.) 

Any tips on how to solve this issue? I might be going at this totally wrong, so any responses would be greatly appreciated!
The best way I can think of is to have Image Texture for Subsurface value which will behave as a SSS intensity control. Where black color = 0 and white = 1. 

In my example I've used such setting with two textures, one for SSS color and another one (black and white) for SSS intensity. This way you can control where SSS will be visible (and how intense).