What are lateral, longitudinal and directional stability?
The answer here is found in the Pilot's Handbook of Aeronautical Knowledge (and probably elsewhere) and is as follows:



The longitudinal axis of an aircraft is more or less a straight line through the aircraft's nose cone or prop hub and the endpoint of the fuselage (the aircraft's center of gravity will usually lie along or just slightly above/below this line as well). It is the axis around which the aircraft rolls, controlled by the ailerons. The lateral axis is parallel to the wings and passes through the aircraft's center of gravity. It is the axis around which the aircraft pitches, as controlled by the elevators. Finally, the vertical axis is "normal" (perpendicular in all directions to) the geometric plane formed by the longitudinal and lateral axes, parallel to the aircraft's primary lift vector and (in level flight) its weight vector. It is the axis around which the aircraft yaws, controlled by the rudder.

Rotation about any one axis is the job of one linked set of control surfaces as mentioned above. Stability in that same axis, roughly defined as keeping that line passing through your aircraft pointing in the same direction, is the job of the other two surfaces in concert, but primarily the one that would move the endpoints of that axis up or down relative to the aircraft. So, elevators provide lateral rotation to achieve longitudinal stability, and vice versa for ailerons.

It differs slightly for the vertical axis, as if your plane is both longitudinally and laterally stable, it's also "vertically stable", however the plane is longitudinally and laterally stable, but completely out of control, in a "flat spin". Stability in the vertical axis therefore is secondary to "directional stability", which keeps the longitudinal axis pointing in a particular direction along the geometric plane formed by the lateral and longitudinal axes. In this case the rudder both controls yaw and provides directional stability.

In addition to control surfaces, weight and especially the aircraft's center of gravity is important in stability. Ideally, most small planes are most stable in level flight when the CG of the aircraft is exactly on the centerline of the aircraft (between the tip of the nose and the tip of the tail) and slightly forward of the aircraft's center of lift (which depends on the wing's angle of attack but is usually close to the thickest point in the wing's cross-section). In this configuration, while the aircraft is travelling forward normally, downwash from the wings flow over the top of the horizontal stabilizer, keeping the nose level. In a stall, the slight nose-heavy configuration along with the stabilizers in the rear will cause the nose to point downward gently, restoring normal airflow and allowing the pilot to recover.

If the CG is too far forward, the pilot will need to apply upward pitch or trim to keep the nose level. This will reduce the amount of travel he will have available to pitch up, and in a stall, the plane will nose down sharply and the elevator may provide insufficient force for the pilot to pull out of the dive.

If the CG is too far aft, the plane will want to nose up constantly, and the pilot will have to apply downward elevator or trim. In a stall, a plane with an aft CG won't nose down, preventing the pilot being able to restore normal airflow over the wings. This is especially dangerous in an uncoordinated stall aka a spin; the aft center of gravity in combination with the forward thrust of the engine will "stabilize" the plane in the spin and make it impossible to recover.

If the CG is off the aircraft's centerline, the plane will tend to roll towards its heavier side. This is compensated for with ailerons or aileron trim, and for most everyday flight it's the easiest to compensate for, but it can cause unfamiliar roll behavior and a tendency to spiral down, which the pilot must be aware of and correct for.