Should I worry about rain getting inside my frame?
It depends on the frame material what effect the water inside the frame will have. A lot of frames have a small hole drilled at the bottom of the bottom bracket casing. If there is no hole there you can drill one yourself. I have done this on aluminium frames before and it is very easy. Use a 4mm drill bit and drill through till you cut into the bottom bracket void. There will be a gap internally between the frame casing and the actual bottom bracket. A hole this small in the centre of the bottom bracket casing will not affect the strength of the frame but will allow water to drain. I have found frames fill up with water even with the seat post in place if ridden in the wet, so putting a cork in the seat post hole probably won't make much difference.