How important is it to have a consistent workout time?

If I workout one day at 6am, the next day at 8pm and then the next day at 3pm for example, will this have a negative impact or a negligible impact?

I'm asking in general, although the types of exercises I am doing are pushups, situps and various dumbbell exercises for strength building.