Weird Protruding Vertices in Sculpt Mode
It's very difficult to tell what might be going on, without more information. If you can attach the .blend file, someone might be able to take a look at it.

Aside from that, there are a whole variety of things that might be going on. Sculpt mode might be "confused" about trying to smooth an inverted crease, or it might be "confused" by a single vertex that's not attached to anything. It doesn't look like you have any other modifiers on the object, but things like like "mirror x" on a mesh that has a mirror modifier can have weird results.

I'd recommend taking a deliberate, methodical approach, to sort out any oddities manually. Firstly, use edit mode to manually move your vertices back into the general vicinity of where they're supposed to be. If you pay attention during this process, it might reveal some clues about what's going on in sculpt mode. You might be able to use Proportional Edit mode to make things a little faster.

I suspect that you will discover some faces that are doing odd things you didn't intend, or edges that are connected in the wrong order, or something like that. Unfortunately, the only way to discover them is to pick individual vertices, and the only alternative is to blow that section away and rebuild it from scratch.

I hope that helps!