Axle to wheel bore ratio?
Using an axle much smaller than the hole it fits into is sure to end badly.  However, you can make up the difference with a sleeve bushing.  Maybe you can find a standard size bushing that fits well, or maybe you'll need to invent one e.g. from copper plumbing pipe.  If you want to try the pipe approach, use 3/8" copper surrounded by 1/2" copper and you might just find a good enough fit.

You don't want the wheels to wobble...if they do, they will wear out rather quickly, and the vehicle's--ahem--performance may be adversely affected.