Drain plug or nut in basement hidden in drywall
Those both look like clean out plugs. I wouldn't drywall over either. They could be needed at some point and then you'd regret covering them as you start punching holes in the wall looking for them.

If you want to seal out the cold, create a proper frame for the 1st and then get an access panel for it that has weather stripping around it or use the foam weather stripping on it to create an air tight seal.

The second, I would just use spray foam in the gap.