Should I turn down PhD offer if I think I may not pass qualifying exam (risking immigration issues if I fail)?
Professors and admissions committees have a lot of experience in accepting students- they wouldn't offer you a position unless they were confident that you were capable of handling the program. It's just as bad a situation for your professor if you fail at the oral exam stage: they will be stuck in year 2 of a 5 year grant without an experienced student and no immediate replacement to carry on that work. 

Two specific thoughts:


Find out who the Graduate Program Chair or the Graduate Admissions Committee Chair is for your department and ask them the same question you asked your professor. These are people with a lot of experience handling and admitting graduate students, and they can give you an impartial opinion.
See if this program has an alternative to the oral examination. My program allowed thesis-option MS students to waive the oral exam under some conditions.  In some labs it was very common for students (typically international students with weaker presentation skills) to fulfill the MS requirements and write a thesis rather than doing an oral presentation.