Relationship of antenna size to gain and directivity
The shape very much affects how the size will play.

For the simple case of a dipole antenna, the best performance (in terms of power transfer) will be obtained with half the wavelength of the signal to be transmitted or received. You also get local minima in the impedance for values of \$k \lambda + \frac{\lambda}{2}\$, and increasing lambda will mean more directionality on the main lobe(s) but also more side lobes. Shortly put, it can be worse and typically \$\frac{\lambda}{2}\$ is the best.

If you have a more directional antenna like the yagi, the size will be primarily influenced by the main folded dipole , and the reflectors and directors will be sized accordingly. But a larger antenna may mean more directors (usually the reflector is one) and therefore, again, more directivity.

For other antennas like the parabolic reflector, size can improve gain/directionality but the benefits will get progressively lower.