Prevent apps being uninstalled
There are quite a few apps in the Market which will lock certain apps behind a password, a search in the Market for "applock" should find most of them.

I haven't actually used any, so can't give you a recommendation, but these should allow you to lock the Market and Settings apps behind a PIN number, to keep your kids out of them.