The research, funded by World Rugby over 29 months, took in 485 men and compared amateur and professional players with matched control groups.
World Rugby said it was "difficult" to draw robust conclusions, adding "further research was required".
But Professor Patria Hume said it was "irresponsible" to ignore the link.
She said rugby players who had suffered four or more concussions performed worse in tests measuring mental and physical coordination, motor speed and multi-tasking.
The Auckland University of Technology study, which is in the process of being peer reviewed for publication, was commissioned by World Rugby in 2012 amid growing fears about the potential for long-term brain injuries in high-impact sports.
Hume, who said that "94% of elite level rugby players experienced one or more concussions", added that the study showed a statistically significant link between repeated concussion and brain damage.
She said: "We've got to go through that scientific process, but what I'm saying is that, as a scientist, it's irresponsible for people to say there are no long-term brain health issues.
"Because all indications so far from the analysis we have done indicates that there possibly are for the rugby players and for people who have been concussed more than four times."
The study involved 131 ex-players, 281 retired amateur players and 73 retired non-contact sportspeople.
World Rugby said in a statement: "The study showed that rugby players performed above average in some tests compared to non-contact sports players and less well in others.
"It is therefore difficult to draw robust conclusions about the links between rugby and long-term cognitive health issues and highlights the need for further in-depth research."
Concerns about concussions in rugby follow a £490m settlement in 2013 between American football's NFL and former players, who said the league hid the dangers of long-term brain damage.
And the issue was highlighted when Wales and Northampton winger George North and England's Harlequins full-back Mike Brown were both stood down from the game for an extended period after suffering concussion earlier this year.
The latest annual injury audit for English rugby shows that the number of reported concussions rose by 59% in 2013-14 compared to the previous season.
But rugby's governing bodies believe the increase is because of improved understanding of head injuries.
In the new findings, athletes who reported four or more concussions during their careers performed worse in some, but not all, neuropsychological testing and those who had one to three concussions had a worse result in one of five balance tests.
But researchers said further targeted research was needed to assess the impact of concussion on long-term balance and cognitive brain function. Arthritis and cardiovascular health in retired athletes also requires further investigation.
A major study of rugby union players points to a potential link between frequent concussion and brain function, the project's lead researcher said.