It came as figures obtained by BBC Radio Wales showed 15% of medical positions were vacant or filled by temporary staff.
In the Aneurin Bevan University Health Board, one in 10 of the medical staff positions were vacant.
The Welsh government said there were difficulties recruiting A&E doctors.
BBC Radio Wales submitted Freedom of Information requests to the seven boards in Wales asking about staff levels for consultants, doctors and nurses.
All responded for the 11 months to August 2013 and showed that all the A&E wards were regularly operating below agreed staff levels.
The worst was Aneurin Bevan University Health Board in south east Wales, where one in 10 of the medical staff positions were vacant.
In the Hywel Dda Health Board in south west Wales, a quarter of the A&E staff were temporary.
The chair of College of Emergency Medicine Mark Poulden called the issue in Wales a "vacancy staffing crisis".
"Over the last year or 18 months we have seen a change. We've essentially reached a tipping point in our staffing levels. Some departments you cannot deliver what you want to for patients," he said.
"Sometimes you can't deliver the quality and certain times we can't deliver the safety we require either which can be difficult."
Across Wales, more than 15% or almost one in seven positions at A&Es were either vacant or being filled by temporary staff.
However the average NHS vacancy rate was more than 5% which is less than the English NHS rate of 9%.
In September the Auditor General for Wales identified a deterioration of care at A&Es.
Nicola Davis-Job, who represents A&E nurses for the Royal College of Nursing, said: "If you have a nurse down on your shift and nobody to fill that the patient care level it doesn't go down, it just means that the nurses have less time to spend with each patient.
"You have to choose which patients that you're going to care for and that's not something that nurses like to do.
"They feel they should give dignified care to all those patients on their shift."
Responding to the figures, the Welsh government said: "While just over 96% of all medical and dental posts are filled, demonstrating we do not have a general recruitment problem, there are geographical hotspots and recruitment difficulties in some specialities...
"If we want senior A&E doctors present for more hours of the week, we will only achieve this if we stop spreading them so thinly and focus their working in fewer centres."
Health Minister Mark Drakeford added: "Right across the United Kingdom there is a shortage of senior doctors to work in A&E departments and Wales is no exception."
He told BBC Radio Wales that the NHS in Wales was being overhauled, including A&E care in an attempt to prevent such issues arising.
"The problems of recruitment don't go away by reorganisation but they become more manageable because you are able to concentrate your forces," he said.
The shadow health minister Darren Millar called the figures unacceptable and said First Minister Carwyn Jones was breaking promises of improvement.
A shortage of staff at Welsh hospital emergency departments is at "tipping point" and putting patient safety at risk, a top consultant has said.