My particles are halfway inside the floor, can I move them up the z axis?
The easiest way to fix your problem is (from the file you uploaded) rotate your bullet object on the x axis by -90 degrees in object mode, then apply the rotation on your bullet with ctrl-a.

The reason your bullets are like that is because blender thinks that (for some reason) the y axis is what should be facing up. Rotating the bullet as describes above sets the origin point of the bullet on the bottom relative to the y axis.