Scientists at Imperial College London examined the bone structure of hip-fracture patients who had been treated with bisphosphonates.
They found evidence the drugs were linked to microscopic cracks, making bones more fragile and prone to break.
Osteoporosis affects three million people in the UK.
What is osteoporosis?
Losing bone is a normal part of the ageing process, but some people lose bone density much faster than normal. This can lead to osteoporosis and an increased risk of fractures.
Bisphosphonates - the main treatment for osteoporosis - are an extremely successful and commonly prescribed class of drugs that slow down the natural processes by which the body removes ageing or damaged bone.
But doctors have raised concerns about the number of fractures occurring among elderly patients who have been taking the drugs for a long time.
To find out why, the team led by Dr Richie Abel took samples of bone from 16 hip-fracture patients and studied them at the Diamond Light Source - the massive doughnut-shaped Syncatron or particle accelerator at the Harwell campus in south Oxfordshire.
"What we wanted to see was whether the bone from bisphosphonate patients was weaker or stronger than bone from untreated controls," Dr Abel explained.
"Rather startlingly, we found the bone from the bisphosphonate patients was weaker. That's a conundrum because the bone should be stronger."
By bombarding the samples with X-rays 10 billion times brighter than the Sun, the team were able to generate images of the internal structure of the bones in unprecedented detail.
These showed microscopic cracks building up in the bones of patients treated with bisphosphonates.
Dr Abel said: "The drug is clearly working, but it also leads to the build-up of micro-cracks in the bone and that could increase the likelihood of a fracture."
It's a surprising result, but the study was small and the work is at an early stage.
Even so, Prof Justin Cobb, a co-author on the paper, says the discovery raises important questions about how we prescribe bisphosphonates for long-term conditions such as osteoporosis.
"There's no hurry, but we should think about how long people are taking them for, and how we might monitor the development of these micro-cracks," he said.
In the meantime the researchers say people should continue to take medications prescribed by their doctor.
If you answered: "Yes," to more than one of these questions, then you may be more at risk of developing osteoporosis:
•Has anyone in your family ever been diagnosed with osteoporosis?
•Have you ever broken a bone after a minor bump or fall?
•Are you female and aged over 50?
•Do you drink more than three units of alcohol a day?
•Do you miss out on summer sunlight (through being housebound, avoiding the sun, always covering your skin or wearing sunscreen)?
•Do you miss out on doing at least 30 minutes of activity five times a week?
Source: National Osteoporosis Society
Drugs used to treat weak bones in elderly patients suffering from osteoporosis may actually make them weaker, research suggests.