Replacing 14" x 1.95" With 14" x 2.125"?
14" tires are actually quite common on department store bikes (Wal Mart, etc), but they are not found on "bike shop" brands. I think this is where some of the confusion on this issue has come in.

You will almost certainly be fine with the slightly wider tire. For one thing, there is no true standardization on tire widths. I.e., every company uses its own gauge for what it means to be 1.95", etc (I once had a 2.55" tire from brand X that was narrower than a 2.2" tire from brand Y).

To be on the safe side, make sure you have a little clearance wherever the tires pass the frame or fork; but I would bet money that the 2.125" tires will fit, and on top of that you probably won't be able to tell any difference between the old and new tires.

EDIT:
Apparently I don't have enough points or something to comment on the answer that links to the Schwalbe website. I was going to say, "Those tires with fractional widths on the Schwalbe website will be entirely different from the tire with a decimal width mentioned by the O.P. If the O.P. gets a 14" tire with a decimal width he will be fine. In the USA I doubt there is anything else to be had."

Finally, if Wal Mart does not sell replacement tires for its own bikes (regardless of the marked width), any bike shop can order the correct tires for you and offer you a guarantee that they will fit.