Bad First Undergraduate Semester
Your least important grades are those earliest in your college career; if you have shown improvement, that improvement will be considered by most admissions committee members, especially if you show improvement in related courses at higher difficulty level (for example, based on your majors, I expect you have had many mathematics courses and physics courses since your problem courses in those areas).

However, you should know that if you are applying to top universities in your field, you will need a strong application overall - not only grades count, and admissions might be tight even for great applications.

Although I would not stress these issues in your initial application, you can use this experience to show growth when you interview for jobs or graduate positions. The classic "what is your worst quality as an employee/student" sort of question is a great opportunity for you to discuss your struggle when beginning college and especially to discuss how you were able to turn things around. Congratulations for not being discouraged - nice work.