What causes aircraft to swerve during takeoff?
I know you said that you don't think it's a crosswind, but honestly it most likely is.  As the plane speeds up and the wings start to generate more and more lift the plane is more easily affected by cross wind.  So, as you speed up, it gets harder and harder to keep the plane straight.  Throw in some gusts, and you're going to get some jerking motions as well.  Especially if that gust decreases the lift of the aircraft, the wheels suddenly catch the ground more and the craft is suddenly tugged harder than the pilot anticipated...

Here's a video of a Boeing 777 taking off in a heavy crosswind that should illustrate the point more.