How poles are related to frequency response
One thing that really helped me understand poles and zeros is to visualize them as amplitude surfaces.  Several of these plots can be found in A Filter Primer.  Some notes:


It's probably easier to learn the analog S plane first, and after you understand it, then learn how the digital Z plane works.
A zero is a point at which the gain of the transfer function is zero.
A pole is a point at which the gain of the transfer function is infinite.
Often there are zeros or poles at infinity, which aren't always included in descriptions of the transfer function, but are necessary to understand it.
The frequency response in the S plane happens along the jω axis only.  


The origin is 0 Hz, or DC, and the cutoff frequency of filters increases radially away from the origin.  Putting a pole at any point along a circle at a certain distance from the origin will produce the same cutoff frequency.
To increase the cutoff frequency of a filter, move the poles radially outward.
To increase the Q of a biquad filter, move the poles along the circle towards the jω axis, which keeps the cutoff frequency constant, but increases the effect that the pole has on the frequency response, making it more "peaky".


If a zero appears on the jω axis, then the frequency response will drop to zero at that frequency; if you input a sine wave at that frequency, the output will be 0.
If a pole appears on the jω axis, then the impulse response is an oscillator; any impulse will cause it to ring forever at that frequency.  Impulses have finite energy, but the response of the filter has infinite energy, so it has infinite gain.


A simple example is an integrator H(s) = 1/s:


This function equals 0 when s is infinite, so it has a zero at infinity.
This function equals infinity when s is zero, so it has a pole at zero.


In other words, it has infinite gain at DC (the step response of an integrator is forever-increasing), and the gain decreases as frequency increases:



Moving the pole away from the origin, along the imaginary axis into the left hand of the S plane, makes the gain at 0 Hz on the jw axis finite again, and now you have a low-pass filter: