How do equilibrium shifts affect solids?
When a particular chemical process is at equilibrium, the respective rates of the forward and backward reactions are equal, meaning that no net change in the concentrations of products and reactants occurs and the composition of the reaction mixture remains stable. If $\ce{OH-}$ is added to a solution already at equilibrium, then there will be an excess of product relative to reactants and the rate of the reverse reaction will increase relative to the forward reaction until equilibrium is reestablished. This means that the ions will recombine into a crystal lattice and form a precipitate. So, to answer your first question, no, the amount of $\ce{NaOH_{(s)}}$ does not remain constant; more of it will be formed if additional ions are added to a solution already at equilibrium.

The reason why pure solids are not factored into equilibrium expressions is that they are not in fact part of the solution. Any excess precipitate, irrespective of the exact quantity, has no impact on the composition of the solution at equilibrium because it exists outside of the solution by definition. If equilibrium is disturbed due to changing conditions, then more precipitate can form, or some of the existing precipitate may dissolve back into solution. Under any given set of controlled environmental conditions, however, the product of the concentrations of the dissolved species remains constant at equilibrium, which is precisely why the equilibrium will shift if conditions or concentrations change.

For a more rigorous and mathematical explanation, you could read up on dynamic equilibria, equilibrium constants, and thermodynamic activity.