How to remove excess nail hole filler
It depends on the filler material used, but the most likely candidates would  be caulk or wood filler/putty.

Caulk remains somewhat flexible after drying. It will probably be white. You can trim it with a razor blade to remove the excess. Assuming the caulk is a paintable type, you can then apply primer and paint your trim to conceal the patch. If you need to apply more, simply smooth it with your finger or a putty knife while it's wet.

Wood filler or putty will dry hard. It will probably be some shade of brown. Most putties are designed to be sand-able, like real wood. Simply sand the patched area smooth with the surrounding wood, apply primer, and paint. Putty can come in a can or squeeze tube, and is usually applied with a putty knife (but a finger can work too).

If neither of these works for you, please post a picture so we can better identify the material.