How do I remove crumbling mortar from fieldstone foundation?
I've no experience of lime mortar, but I would work under the same assumptions as I use for plaster and normal mortar.

That is, when removing the loose material go a little way into the sound areas, but don't be too aggressive.

So in this case I'd start with a steel brush to get the really loose material out and then use a chisel (or an old long screwdriver) to scrape away a little bit further and test out the remaining mortar to see how loose and crumbly it is.

If you find that you are removing a lot of mortar then I'd start to get concerned, but if you only do a small area at a time you shouldn't affect the stability of the wall.

It might be useful to get a couple of people in to give you quotes for the work. They should explain what they would do and give you an indication of just how bad the mortar is. Ask them if there could be structural problems when repointing - it's a perfectly valid question.

Just don't tell them you intend to the work yourself :)