What should my priorities be in designing a home generator?
The factors that you are describing are just components of what matters here which is power (Instantaneous ability to do work or deliver power) and energy (work done or power x time acting).

You need a better understanding of power and energy and there is so much on the internet on this that repeating it here is not a good idea.

T address your questions, plus a few notes:


  ... to sell back to the grid. 


To do this the equipment must be certified as suitable. Among other things it MUST be certified as "anti islanding" - ie it cannot send energy into the grid when the grid is unpowered. There is zero chance of getting approval for connection of DIY equipment to the gid without a certified anti-islanding arrangement from an approved supplier. The most usual and probably easiest method is to drive a certified ant-islanding "grid tie" inverter. These ost about $US1 per Watt on ebay for top quality brands (eg SMA) and rather less for brands of lesser repute. As a rule you don;t get what you don't pay for in this area. You can but non-certified grid tie inverters at low prices but connection is prohibited and probably also illegal.  FWIW - the reasons fro requireing anti-isalnding are not overly good technically but rules are rules.


  Which generates greater power: running at a faster RPM, or building a larger generator? 


Essentially "No", although too small a generator for available power will limit output. .
Speed is not directly tied to power. It is often possible to build a more compact generator if it is DESIGNED to operate at higher speed but this is not "a given" and is complex. The power output is set by the available power to drive the generator as long as the generator is able to handle it. If eg a wind turbine receives 100 Watt of power at the shaft, you could design a slow or fast generator (or alternator) that worked about as well BUT you could only approach and not exceed 100 Watts no matter what speed you used.


  Is there a limit on RPM usefulness in building a generator? 


Yes,sort of - but aspects such as how is the power produced, are as great or greater influences in design speed. Most wind turbines rortate such that the blade tips exceed wind speed by a factor K. A k of 1 to 2 is slow, 2 to 6 say is normal, 6 to 9 is rather fast and 9+ is getting frenetic. Wind Turbines with TSR (Tip speed ratios wrt wind) of 12:1 exist but are noisy and prone to erosion by rain and dust. High speed devices fly apart more easily. A TSR = 12 machine is a work of art &amp; insanity and I'd be wary of standing too near to one.


  If I can make a generator run at 20,000 RPM (yes, I believe I could achieve that rotational speed), will that produce a useful current gain over the same generator running at, say, 3,600 RPM?


As above, it's essentially irrelevant, and 20,000 RPM is hard to design for well in that sort of environment - a rotor that rotates 300+ times every second is trying to kill itself and you every moment it is running.


  I am also not sure if I should construct an AC generator, or a DC generator hooked to a DC to AC inverter. 


AC is almost always used for reasons which become clear once you look at real world designs.