Tips on ESD protection for hobbyist designs
This is an awesome question because it touches on two things that 'beginners' (if I may) often don't have a good feeling for:


How important electrical protection and conditioning is
Where to put electrical protection


Here is what you should do:

You should protect any outgoing and incoming electrical signals that a user will touch


Any I/O lines that exit your board may be connected to a source of ESD - be it another device or a person. The easiest way to protect pure outputs is to buffer them with a gate (lots of 74 and 4000 series parts to do this), or in the case of an analog output an opamp. For pure inputs, you will want to use a zener/TVS+series resistor as the easiest way to protect such lines.

However, you should also keep in mind that most, if not all, microcontrollers and other devices that you may want to use have built-in ESD protection, sometimes really really good stuff. They have both (micro)zener and schottky-to ground/schottky-to-Vcc protection, which basically takes care of all your worries. In order to beef up your design you may still want to add series resistors of about 1kohm on outgoing and incoming lines - if this doesn't affect operation. 

A word of warning: the fact that all MCU CMOS in/outputs have protection on them does not mean that all outputs have this protection. For instance, open-drain outputs (often used for I2C peripherals) are notoriously ESD sensitive and require additional input protection.

Also very sensitive to ESD are USB lines. However, a bog standard zener and especially a TVS will not suffice because of the comparatively high capacitance of these devices. High-speed buses require specialist protection diodes, for instance NXP PRTR5V0U4D. These devices are just schottky diodes going to your power rails, so you need to additionally overvoltage protect the power supply as well if you have not done that already!

You should protect and condition all incoming power lines


Power lines are by far your worst enemy when it comes to destructive events. Of course, a malfunction on your board may cause excess current to flow and cause fires - this is what we use fuses for. Always fuse off your boards if you expect such a scenario to be possible. Not likely - just the possibility is enough. Don't worry about the fuse rating, it doesn't need to be tightly matched to the expected current draw of your board. The only function of fuses is to prevent fires, so make sure it does that!

However, continuous high current is not the only thing that can happen with power lines. Incoming power lines are often long wires with associated high inductance - often in the order of µH or tens of µH. If you have an application that consumes 1A, this means that in steady state this (parasitic) inductance will contain \$E=\frac{1}{2}LI^2\$, which will be in the order of µJ with this power line. If a user now suddenly disconnects the power line, the current path is broken but there is still this energy in the power line that it needs to lose. The way this energy is discharged is via a spark that happens just after you disconnect the device - you have probably seen this 'inductive kick'-spark before when connecting or disconnecting devices. Even though it's often just between 1-100µJ of energy, when discharged into a low-capacitance bus this can cause dangerous high voltage spikes that damage microelectronics.

This is why on power lines, a TVS or MOV is good practice to include. Of course, some bulk capacitance is also very welcome. 

On-board protection and conditioning is very seldomly necessary


Beginners often go totally nuts with fuses and protection devices everywhere. This is not necessary, especially if there is no way a user or other source of ESD will ever touch these lines. The same goes for EMI protection - often just not necessary, and if you have an EMI problem there are usually better ways to solve these (like decreasing source/load impedance with termination or buffering).