The Planet Earth presenter said his work took him away for months at a time when his two children, Susan and Robert, were growing up.
Sir David told the Radio Times: "If you have a child of six or eight and you miss three months of his or her life, it's irreplaceable. You miss something.
"Perhaps you can't have your cake and eat it."
Attenborough said his wife, Jane, who died in 1997, had been very "understanding", however.
A month after celebrating his 91st birthday, Sir David told interviewer Louis Theroux that he thinks about his own mortality "all the time", because "it's more and more likely that I'm going to die tomorrow."
Asked if he expects anything to happen afterwards, he said simply: "No."
On the subject of global warming, Sir David warned: "We should be very, very worried about it.... the land is being scorched, deserts are spreading, and the seas are warming - all those factors cause great changes in our fortunes."
When asked which animal he felt a kinship with, Sir David replied: "That has to be an ape. Because our kinship is a reality. I don't feel it with a mosquito or, indeed, a whale."
Follow us on Facebook, on Twitter @BBCNewsEnts, or on Instagram at bbcnewsents. If you have a story suggestion email entertainment.news@bbc.co.uk.
Sir David Attenborough says his only career regret is missing out on time with his children.