Overall staffing in NHS Scotland is at a record high, but auditors said the government must plan ahead better to "urgently address workforce issues".
It said there were signs NHS services were "under increasing stress".
The Scottish government said it was committed to having the right mix of skills and staff in the right places.
But opposition parties said the "damning" report was confirmation of a "deepening workforce crisis" in the health service.
The new report, the first part of a two-stage audit of workforce planning in Scotland's NHS, examined clinical staffing in hospitals and other secondary care settings.
It found that health boards "consistently underestimate" the number of staff they need, while the Scottish government had "not yet adequately estimated what impact increasing and changing demand for NHS services could have on the workforce or skills required to meet this need".
The report comes after medical leaders urged decision-makers to be "courageous" over NHS reform in future budgets, amid warnings from unions that "the NHS in Scotland isn't getting the resources it needs".
The report, by Auditor General Caroline Gardner, said it was vital that the government and NHS bosses plan more effectively for the future.
It highlighted that spending on NHS staff had increased by 11% between 2011-12 and 2016-17, with the overall staff hitting a record high of 139,341 whole-time equivalent workers.
However, in reaching these staffing levels almost all NHS boards overspent against their pay budgets, and agency staffing costs have more than doubled over six years to more than Â£171m.
And the report warned that the government and NHS boards "have not planned their NHS workforce effectively for the long term".
It said the health service faces "urgent" staffing challenges, including:
The Scottish government is setting up a National Workforce Planning group to improve joint working. However, Ms Gardner's report said responsibility for workforce planning was "confused" - it is currently split between the government, health boards and regional workforce groups - and was at risk of fragmenting further with the integration of health and social care services.
Ms Gardner said: "Thousands of people work hard in Scotland's NHS to deliver vital public services every day, but there are signs that the health service is under stress and that staff face increasing workload pressures.
"The Scottish government and NHS boards recognise the challenges, but urgently need to improve their understanding of future demand, staff projections and associated costs, and set out in detail how they plan to create a workforce that can meet the long-term health needs of the population."
The report was welcomed by staff groups and unions.
Theresa Fyffe, director of the Royal College of Nursing Scotland, said the report should highlight "the urgency with which workforce planning now needs to be addressed".
She said: "Audit Scotland has hit the nail on the head. For too long plans have been restricted by what is affordable and achievable with the staff available, rather than focusing on strategic, long-term planning to meet demand. The result is that Scotland has too few nursing staff in post and too few nurses being trained.
"Staff across NHS Scotland are under enormous, unrelenting pressure to meet ever growing demand. The significant workforce challenges set out in the report must be addressed robustly, realistically and rapidly if patients are to get the care that they need."
Dr Peter Bennie, chairman of the British Medical Association Scotland, said there were "serious workforce challenges facing the health service in Scotland which must be tackled by the Scottish government as a matter of urgency".
And Matt McLaughlin, Unison Scotland's head of health, said there was "no doubt that the service needs to significantly up its game when it comes to workforce planning".
He added: "Management teams, staff and unions are being asked to save millions, maintain services at the current level and plan for a different future. That reality means that workforce planning is being pulled in different directions as short term needs outweighs long term planning."
Health Secretary Shona Robison told BBC Scotland that she accepted all of the recommendations in the report.
Ms Robison noted that the government published the first part of its National Health and Social Care Workforce Plan in June, which sets out how to improve planning to recruit, develop and retain staff.
She said: "We've listened to Audit Scotland's comments about workforce planning well before they've produced this report. That's why we worked on a very detailed workforce plan that sets out a very clear framework on how we make the improvements, how we build on the work already done, but take a real regional and national perspective on that.
"Not just for current service models, but for future service models. And that's why the detail has been done and now will be taken forward. The pace behind that will absolutely be there.
"I understand the necessity to not just have more staff, but having the right staff in the right places, to fulfil this new model of keeping people out of hospital, keeping people in the community, and that's what we're giving close attention."
Opposition parties said the report was evidence of a "workforce crisis" within the NHS.
Scottish Conservative health spokesman Miles Briggs said: "Time and again we have seen warnings about long term workforce planning, and these figures show the situation is only getting worse.
"Agency costs are soaring, and the percentage of vacancies in consultancy and nursing posts have more than doubled. More worryingly, we are not seeing enough new nurses being trained to keep up with an increasingly elderly workforce."
Scottish Labour's Anas Sarwar said the report was "absolutely damning" saying NHS staff were "over-worked, under-valued and under-resourced".
He said: "Staff morale is at rock bottom in the health service, with staff reporting there simply aren't enough of them to do the job properly."
And Lib Dem MSP Alex Cole-Hamilton said: "This damning audit rightly takes a dim view of Nicola Sturgeon and her health secretary's attempts to avert this crisis. The Scottish Government ignored warning after warning that staffing isn't keeping up with demand."
Scotland's government and health boards have failed to plan effectively for long-term staffing challenges, Audit Scotland has said.