Mr Tillerson, former head of Exxon Mobil with no previous political experience, said in an interview he was "stunned" with the invitation.
"I didn't want this job. I didn't seek this job," he told the conservative website Independent Journal Review (IJR) during his recent visit to Asia.
"My wife told me I'm supposed to do this."
In the wide-ranging interview, Mr Tillerson said he had been due to retire this month after spending more than 40 years at oil giant Exxon Mobil. He turns 65 on Thursday.
"I was going to go to the ranch to be with my grandkids," he added in the interview, as he returned to the US from Beijing.
Mr Tillerson said he had never met Donald Trump before his election in November.
When he was invited for a conversation with the then president-elect, he thought they would talk "about the world" given his experiences at the oil company, he added.
"When he asked me at the end of that conversation to be secretary of state, I was stunned."
Mr Tillerson then gave the news to his wife, Renda St Clair, who said: "I told you God's not through with you."
He added: "My wife convinced me. She was right. I'm supposed to do this."
It was Mr Tillerson's first interview since he took office and correspondents say he has so far kept a low profile at the state department.
He came under criticism after the state department press corps was not taken along with him on his trip to Asia.
The IJR's Erin McPike was the only reporter allowed to travel with him. The state department said this was because of the size of the plane.
But in the interview Mr Tillerson said: "I'm not a big media press access person. I personally don't need it.
"I understand it's important to get the message of what we're doing out, but I also think there's only a purpose in getting the message out when there's something to be done."
Mr Tillerson has also been criticised for his close ties with Russian President Vladimir Putin. But when the subject came up in the interview, the IJR said: "He was so cagey when Russia came up, for example, that his answer wasn't even worthy of inclusion."
The US Secretary of State Rex Tillerson has said he only accepted the job after being convinced by his wife.