Away rotations are very important and are only possible in 4th year. It's very important in specialties like Ortho, EM, Neurosurgery, etc to do away rotations to audition. This typically happens in the first 3 to 4 months of fourth year. Applications for residency don't even open for submission until September, so it's completely false to suggest that the beginning of 4th year isn't crucial for the residency process.

Of course you'll be doing other things like research, etc. to build your application, but program directors do weigh audition rotations heavily in who they interview/rank.