Why is a mild earthquake often mistaken as dizziness or vertigo? Is it due to the longitudinal nature of the seismic waves?

When a heavy bus or truck goes by at high-speed, the shaking in nearby ground is not mistaken as dizziness, rather the feet and body feel a vertical jerk (seemingly a transverse wave vertical to the ground).

But when a natural, mild earthquake takes place, it is not felt as a jerk or switch in a direction at all. Rather we feel it for a while as dizziness. I have experienced this, and heard of many others who have as well.  In-fact it takes a few seconds to distinguish whether it is dizziness or an earthquake. 

Is it due to the longitudinal waves? Is it because they are horizontal with the ground, matching the direction of dizziness?

Also I guess maybe there may be similarity between the frequency of seismic waves and this video from Wikipedia's article on vertigo:



Does this phenomenon's frequency act similarly to the motion and simulates  dizziness?
Having experienced many minor earthquakes (Mag 3 to 5 in the East African Rift Valley, Samoa, Solomon Islands, Philippines and Indonesia), and one major life-threatening earthquake (Magnitude 8, Nepal), I can make a few personal observations about physiological reactions.
First, You are quite correct about the body picking up any swaying feeling, and interpreting this as dizziness. I was in an 11th floor hotel room in Bangkok when an aftershock of the Aceh earthquake occurred. My first thought was "Did I drink too much beer this evening"? I wasn't alone. When I opened the door into the corridor, the occupants of nearly every room were looking out and wondering much the same thing.

Second, after experiencing a big one the body becomes hypersensitive - conditioned to pick up vibrations that could be the onset of a major quake. Now, at the onset of any vibration, my brain goes straight to "how big is it - should I run to the nearest exit"? This response is now much faster than the sensation of dizziness. Also I now pick up many vibrations that nobody else around me seems to notice. 

Thirdly, I am not sure I agree with you about vertical motion being the trigger. It seems to me that lateral motion - moving one's feet sideways relative to the head, creates a slight imbalance, which the ears interpret as 'dizziness'.