What's in an Arduino?
You did not specify what Arduino board you are talking about. It would
have been useful to tag the question with the appropriate board name. In
the following, I assume you want something similar to an Arduino Uno,
i.e. an ATmega328P at 5&nbsp;V and 16&nbsp;MHz.


  I realize I can go look up the actual design itself [...]


You should. There really isn't that much to it.

Let us take a look at the Uno rev.&nbsp;3 schematic, and see what may be
useful to you:


ATmega16U2 and associated circuitry: This is for USB communication.
You have no use of it if you are not doing USB.
3.3&nbsp;V regulator: This is for providing the Arduino 3.3V output.
Only a convenience feature, not used by the Arduino itself. You should
leave this out unless you need a 3.3&nbsp;V supply.
5&nbsp;V regulator. Only you know how you want to power your board.
This regulator is needed unless you have some reasonably stable
5&nbsp;V supply like, e.g., a USB bus.
ISCP header: This is rarely used on an Arduino, but you are very
likely to use it in your custom board, as ICSP is the easiest way to
program an ATmega with no bootloader.
Reset button: Not always needed, as you can just power-cycle your
board.
10&nbsp;kΩ pullup on RESET: Your board is very likely to work just
fine without this, but there is a little risk that you suffer from
random resets. You want this resistor on anything that is
“production”-like, for reliability concerns.
Decoupling capacitors: Just like the RESET pullup, the board may work
without them, but they provide reliability.
The MOSFET and OP-amp right before the 3.3&nbsp;V regulator: This is
for automatically switching between the Vin and USB power supplies.
Unlikely to be useful on a custom board.
Ceramic resonator: For running the ATmega at 16&nbsp;MHz, you need
either this or, for better timing characteristics, a quartz with
suitable capacitors.
LEDs: You know if you want them.
AREF capacitor: Only useful if you do analog readings, as it reduces
noise.