In 2016, there were 5,826 reported cases, with holidaymakers losing Â£7.2 million in total, an average of Â£1,200 per person.
The most common scams involved plane tickets, online accommodation bookings and timeshare sales.
Action Fraud said reports of scams had risen consistently over the past five years.
More than a quarter of those affected said the scams had a significant impact on their health or finances.
About 260 people said they needed medical treatment as a result, or were at risk of bankruptcy, according to Action Fraud, the UK's national fraud and cyber crime reporting centre.
Sports events and religious festivals are a common target for scammers due to the limited supply of tickets resulting in higher prices.
Holidaymakers are being persuaded to part with their cash with fake accommodation websites and adverts, with fraudsters are also hacking into official accounts to target people.
Tony Neate of fraud prevention group Get Safe Online said holidays are seen as a "big-ticket item" so present "the perfect opportunity for cyber criminals to swindle unsuspecting victims out of their hard-earned money".
He said people should do as much research as they could about the organisation they are booking through "and ensure that they are a reputable travel operator that is a member of a recognised trade body like Abta".
"By booking in haste, you could not only risk losing a huge amount of money, but also disappoint family and friends when it comes to that long-awaited escape," he added.
Scam victim Stephanie Gager used cash and a bank transfer to pay for a non-existent holiday.
She says when she received paperwork with all the right logos and flight information on, she assumed it was legitimate.
She was asked to pay a deposit and then told the company she would pay the remaining balance within a fortnight. However, she was then told she had to pay into a bank account "immediately" to get the holiday at a low price.
"When I got to the airport they said to me, 'you're not booked on the flight'. Of course I was devastated," she told BBC Breakfast.
"[Airport staff] said this happens a lot, it's worth paying that extra more to actually make the flight."
She admitted in hindsight she should have heard alarm bells at the point she was told to pay money into a bank account, and has since been given advice about how to avoid fraud.
Authorities say they believe criminals are taking advantage of the lack of awareness of the UK's strict regulations on travel companies.
Fake websites have encouraged tourists to pay via bank transfer or cash, claiming these are the only methods which are protected by their own bogus insurance policies.
But in reality, these methods make it harder for people to get their money back.
Mark Tanzer, chief executive of travel trade organisation Abta, said: "Abta is regularly contacted by members of the public who have been caught out by increasingly sophisticated travel-related frauds.
"We know at first-hand that the loss and shock of finding that your flight or holiday accommodation has not been booked can be very significant."
Get Safe Online, Abta and City of London police have launched a campaign to warn travellers about the dangers of holiday booking fraud.
Get news from the BBC in your inbox, each weekday morning
The number of British tourists hit by booking scams rose by 19% last year, according to Action Fraud.