This LM317 diagram doesn't make any sense to me
The datasheet has a pretty thorough description of the use of the ADJ pin with \$R_1\$ and \$R_2\$:



Since both \$R_1\$ and \$R_2\$ appear in the equation for the output voltage

$$V_{\text{out}} = 1.25\text{ V} \times \left(1 + \frac{R_2}{R_1}\right) + I_{\text{ADJ}}R_2$$

you need both in order to realize an arbitrary output voltage. Depending on your expected load and desired output voltage you may be able to remove \$R_1\$. However, you must maintain a minimum load current (which the datasheet specifies as 10mA) so if your load may fall below that you must rely on the \$R_1\$ and \$R_2\$ divider to draw enough current to meet that minimum load current requirement.

With a voltage divider you normally have an input voltage that you want to divide down using a pair of resistors. You set the ratio of resistors in order to set the divided down voltage:

$$V_{\text{div}} = \left(\frac{R_1}{R_1 + R_2}\right)V_{\text{input}}$$

In this case, the divided down voltage \$V_{\text{div}}\$ is set by the device (1.25V) so you are setting the ratio of resistors in order to set the voltage divider's "input" voltage \$V_{\text{input}}\$, which is the LM317's \$V_{\text{out}}\$.


  Lastly, forgive my ignorance of caps but if a capacitor isn't a load, isn't \$C_1\$ creating a short circuit?


A capacitor has very high (ideally, infinite) impedance at DC so there is no short circuit. This capacitor will short circuit high frequency signals (i.e. noise) on \$V_{\text{in}}\$, which is desirable since \$V_{\text{in}}\$ is supposed to be a DC voltage source.