Scottish Natural Heritage (SNH) said rock was dug away from cliffs near Bearreraig Bay in an apparent organised search for valuable specimens.
The agency said Skye was known as Scotland's "Dinosaur Isle" because of its important fossil records.
SNH has appealed for witnesses to contact the police.
Skye is a key dinosaur fossil site in Scotland.
Bearreraig Bay, north of Portree, is within a Site of Special Scientific Interest (SSSI). A crowbar is suspected to have been used to prise away some of the rock.
Dinosaur footprints may also have been removed from Valtos on Skye, SNH said.
Evidence gathered at Valtos has been used by palaeontologists to explain what has been called the Dino Stampede in Australia, an incident where a group of dinosaurs were pursued by a predator.
SNH said the Bearreraig Bay dig had been done without the landowner's permission or the consent of SNH, which manages the SSSI.
Dr Colin MacFadyen, SNH's geologist, also said the actions went against guidelines in the Scottish Fossil Code.
The codes does allow for the use of mechanical diggers, rock saws and even explosives for extracting fossils, but only when it was to the benefit of palaeontological research.
Dr MacFadyen said: "Fossil collecting is important for scientific and educational purposes, and is a popular hobby.
"It is better for fossils that fall from cliffs to be found, collected and enjoyed rather than be eroded and washed away by the tide.
"However, speeding up the process by large scale rock removal as in this alarming case is irresponsible and illegal, and also potentially dangerous to people as the cliff faces are undermined and destabilised."
Dr MacFadyen said permission to remove material from the SSSI would have been needed from SNH, but it had not been sought.
He said fossil-rich rock had been damaged at the Bearreraig Bay site.
Dr Neil Clark, of University of Glasgow's Hunterian Museum and Art Gallery, has described Skye as one of the world's most important palaeontology sites.
Its standing is underlined by the number of finds from the Middle Jurassic, about 170 million years ago.
Dr Clark told the BBC News Scotland website: "This is quite a shock. It is not something I have heard of happening before.
"Without knowing exactly where the damage is I can't say what they were after. It may have been plesiosaur."
He added: "Whatever has been taken is now lost to science."
Finds on Skye have included the earliest turtles known to have lived in water.
The 164-million-year-old reptile fossils were found embedded in a block of rock at the bay of Cladach a'Ghlinne, on the Strathaird peninsula.
The new species formed a missing link between ancient terrestrial turtles and their modern, aquatic descendants.
The discovery of Eileanchelys waldmani, which translates as "the turtle from the island", was reported in the Royal Society journals in 2008.
Tonnes of rock has been disturbed at a Jurassic site on Skye in what has been described as one of Scotland's most reckless acts of fossil collecting.