Is Pandemic Legacy as good with 3 players as it is with 4?
We've played the game with both 3 and 4 in the same campaign.  It didn't affect the feel of the game either way.  For a variety of reasons you likely won't play the same characters in every mission anyway, so there is no reason that you can't add or drop someone part way through.