The shrew-like mammals, whose unusual name - meaning island murderer - points to their insectivorous nature, became extinct around 500 years ago.
Despite this, the scientists were able to use the DNA to trace the mammals' evolutionary history back millions of years to their split with their nearest living relatives, the solenodons of Cuba and Haiti.
Solenodons, at around 30 centimetres long (excluding their tails), resemble large, fat shrews with long, flexible snouts, and are one of the few venomous mammals.
The research ends the status of Nesophontes as one of the few recent mammal groups without DNA data to help place it on the evolutionary family tree.
It also offers scientists a picture of how these small, insect-eating mammals evolved.
'It's exciting to see how far back their lineage goes,' says Professor Ian Barnes, a research leader at the Museum and one of the authors of the study.
For the study, the scientists collected DNA from a Nesophontes skull found in a 750-year-old owl pellet from the Dominican Republic, and compared it with genetic information from 25 living species from the same order - known as Eulipotyphla - which includes mammals such as moles, shrews and hedgehogs.
They found that Nesophontes were most closely related to solenodons, with the two families having diverged from one another around 57 million years ago - a period of significant volcanic activity in the Caribbean that could have fragmented the populations, leading to their development as separate families.
The scientists also found that both Nesophontes and solenodons sit far from their next nearest relatives, having diverged from them around 72 million years ago - a time when dinosaurs still roamed the Earth.
This divergence may have occurred when their ancestors, having colonised the islands that now form the Caribbean, were separated from North America as the islands moved from the Pacific towards their current location.
Obtaining DNA from tropical fossils is notoriously difficult, and the team made use of the latest technology and specialised labs in London, UK, and Mainz, Germany, to conduct the study.
'These advances allow us to study the evolutionary history of these extinct species in a way that we wouldn't have thought possible 10 years ago,' says Dr Selina Brace, researcher at the Museum and a co-author on the paper.
The eight species of Nesophontes became extinct around 500 years ago, coinciding with the arrival of European ships in the Caribbean.
One proposed explanation for this is that they were driven extinct by the arrival of rats that stowed aboard these ships.
With the research suggesting that the genus had survived for over 50 million years prior to the arrival of Europeans, the demise of Nesophontes shows the devastating effect that humans and invasive species can have.
'Nesophontes was just one of the dozens of mammals that went extinct in the Caribbean during recent times,' says Prof Barnes. 'In fact, there are very few native mammal species left.
'Yet islands are vital ecosystems, often acting as "museums of diversity", preserving ancient species and promoting adaptations in modern ones.
This skull was dug up from the Tower of London moat. The lion guarded the entrance 700 years ago as a symbol of strength.
A major study provides the first global view of biodiversity damage since the 1500s and suggests how losses can be reversed.