Temporarily filling a hole with a shim
My first thought is square flush tubing plugs. they come in all sizes, for tubing up to like 6".

Check out this web site: Mocap square tubing plugs



Should be pretty easy to pry out with a small tool when you need to. They are not advertised as perfect seals to keep out water, but unless an area depression causes standing water right by the hole, I bet they would do a pretty good job.