The warning by Age UK and Carers UK came as the charities released figures showing there were 1.2m carers over 65 - a 25% rise in the past decade.
The biggest increase though has been seen in the over 85s, with the numbers more than doubling to 87,000.
The charities said these older carers needed better support.
This issue includes NHS funding, GP access and social care, particularly of older people.
Policy guide: Where the parties stand
The majority of carers over 65 are looking after a partner, although some are looking after elderly parents, or grandchildren, or relations with disabilities, according to the charities.
The research by the charities - based on Census data from 2001 and 2011 and their own analysis - estimated the care provided by older carers was worth Â£15bn a year.
It warned that despite the growth in numbers, few of the older carers were being offered support by the state. Last year just over 175,000 were given assessments by their local councils.
One in three carers aged 65 to 74 provide over 50 hours of care a week, but for the over 85s group that rises to more than half.
More than half of the younger age group said they were not in good health, compared to 40% of those who do not have caring responsibilities.
Feelings of depression and anxiety were cited by one in three carers aged 65 to 74 and nearly half of the older age group.
The BBC has launched an online guide to the care system for the over-65s. The "care calculator" covers residential care and the support provided in people's own homes, for tasks such as washing and dressing.
Users can submit their postcode and find out how much each service costs where they live in the UK.
There is also a dedicated BBC Cost of Care website, with news stories, analysis and video.
Dave, 90, is typical of the generation of older people who find themselves caring for loved ones.
The former tram driver has been looking after his wife Ethel, who has poor eyesight and limited mobility, for the last 20 years.
He does not get any help. "We like to be independent and between us we manage to get by. We just accept this as the way it is. You just get on with it don't you?"
Carers UK chief executive Helena Herklots said: "Caring is something that touches all of us at some point in our lives but this research shows that the number of older people are caring for others at a time when they are more likely to need care themselves.
"Action is needed to ensure that older carers have the support they need so they don't have to care alone."
Councillor Izzi Seccombe, of the Local Government Association, said: "Councils recognise that informal carers do a challenging job in very difficult circumstances."
She said local authorities were doing their best to support them and she urged carers to come forward for help, saying that they did not always realise they were entitled to help.
The number of older carers in England is rising, with signs the pressures of looking after loved ones is damaging their health, research suggests.