Rural Affairs Secretary Richard Lochhead announced the move earlier this month.
Almost 30 organisations have signed an open letter seeking an urgent meeting with him to discuss their concerns.
Mr Lochhead said the changes would not affect research.
Under EU rules, GM crops must be formally authorised before they can be cultivated.
An amendment came into force earlier this year, allowing member states and devolved administrations to restrict or ban the cultivation of genetically modified organisms within their territory.
Mr Lochhead announced he would use the amendment to request Scotland be excluded from European consents for the cultivation of GM crops.
The letter - whose signatories include the National Farmers' Union, the Royal Society of Edinburgh, the Roslin Institute and the European Academies Science Advisory Council - said they were "extremely concerned" about the "negative impact" a ban could have.
It claimed the decision was "political and not based on any informed scientific assessment of risk".
The scientists and academics said outlawing the cultivation of GM crops "risks constraining Scotland's contribution to research and leaving Scotland without access to agricultural innovations which are making farming more sustainable elsewhere in the world."
The letter cited examples where "a GM method has a contribution to make".
It highlighted ongoing research on things which "might benefit Scotland's farmers, consumers and environment" including potatoes that can reduce fungicide use and omega-3 enriched oilseeds that may offer a more sustainable source of feed for salmon farming.
Responding to the letter, Mr Lochhead said he respected the views of those in the scientific community and would be happy to meet them.
He said he would "reassure them that these changes will not affect research as it is currently carried out in Scotland, where the contained use of GM plants is permitted for scientific purposes, such as in laboratories or sealed glasshouse facilities".
He added: "However, just because GM crops can be cultivated in Scotland it doesn't mean they should be.
"Scotland's Â£14bn food sector has a reputation for a clean and green image across the world and allowing the cultivation of GM crops could damage that unique selling point."
Last week, a former chief science adviser to the Scottish government warned on ban on GM crops could have "apocalyptic" consequences and threaten the country's food and drinks industry.
Prof Muffy Calder, who stepped down from the role in December and has yet to be replaced, said she was "disappointed and angry" and called on ministers to publish the scientific basis for their decision.
Professor Neva Haites, vice president for life sciences at The Royal Society of Edinburgh, told BBC Scotland: "Most governments try to get the best advice possible before they make these decisions.
"In the past we have had chief scientific advisers such as Dame Anne Glover and Prof Calder, who were there to give advice on such subjects.
"We no longer have such an adviser in government at the moment and the scientific body that usually gives advice has not met for some time.
"So we are suggesting it is time they actually looked again and sought some very senior advice on this subject."
A ban on growing genetically-modified crops in Scotland could threaten the country's contribution to scientific research, according to scientists, universities and farming leaders.