The 50-year-old from Felstead in Essex rowed the Atlantic two years ago in the Woodvale Challenge.
The 3,000-mile (4,800km) voyage took him 52 days - 10 of which were spent going backwards. Despite this, he set a new UK solo and race record.
In January, Mr Pitcher plans to do it all again. This time, he has his sights set on the 40-day trans-Atlantic solo rowing world record.
He will live aboard his rowing boat, the Soma of Essex. Made of carbon fibre, it is 6.5m (21ft) long and weighs just 460kg (1,014lb).
Each day he will have to eat about 7,000 calories and make his own water to drink from sea water.
Asked why he would row the Atlantic a second time, Mr Pitcher said: "The first time, it was an adventure. I stumbled across that adventure and ended up rowing across the Atlantic in a race from the Canaries to Antigua.
"But we had very bad weather and it wasn't really what what I had bought into.
"When I got to Antigua I decided I wanted to do it again, hopefully with the right weather.
"I want to experience those famous trade winds which blow of the coast of Africa towards the Americas.
"Of course, I also want to try and break the world record to be the fastest solo rower across the Atlantic.
"Last time I spent 10 days going backwards and that is very mentally challenging," he said.
"But if we get good weather and  I don't go backwards for 10 days, I think there's a possibility of breaking the record."
His days will be divided into 16 hours of rowing, four hours sleeping and the remaining four hours will be spent on boat management.
Mr Pitcher will sleep on the floor of his compact cabin, which has a carbon fibre seat.
For entertainment, Mr Pitcher has a number of audio books to listen to and more than 4,000 songs on an iPod. His music includes a mix of music from classical to hard rock and Abba.
The Soma of Essex has been designed to be self-righting in the event of a capsize.
"It's designed to come up very quickly if it turns turtle," he said, "and as long as I hold on tight I will be alright - even if I'm like laundry in a tumble dryer."
"Those people who think I'm crazy don't really know me.
"There's a risk and there's an assessment of that risk. And we reduce those risks as much as possible."
The plan is to get the rowing vessel to Tenerife and then to the island of La Gomera.
Once there, Mr Pitcher will wait for the the right conditions.
Asked what he was expecting on the open ocean, he said: "You only need a tropical storm and it can be very bad. We are not departing in tropical storm season, though there is still a risk. It is less than 10%.
"But there are other things which can go wrong. I have a water machine that turns salt water into drinkable water. If that breaks, it'll be game over.
"It doesn't have to be storm weather that stops me, there are other things that could be. There's a liferaft on the boat as well," he said.
As well as attempting to break the current record - which is 40 days, 9 hours and 44 minutes - Mr Pitcher is raising money for two charities, Great Ormond Street Hospital and The C Group, which seeks to generate support for former Royal Marines.
Charlie Pitcher says people who call him "crazy" do not know him very well.