Make force field only affect objects while they are within a specific area
You can set a max distance on the force field, after doing that the force field will only affect the objects if they are within the set distance rather than affecting the whole scene.