Due to the significant gap between gradient-based training and inference, there may come a point where it's more efficient to directly search for the target network using a gradient-free method rather than training a large network and reducing it to a much smaller one.