Overnight visitors from the Republic of Ireland declined by 7% in 2013.
A report for the Department of Enterprise, Trade and Investment (DETI) said "social unrest over parades and flags" had potentially had an impact.
It also said the local tourism industry was worried about "continuing sectarianism and racial prejudice".
The report said there needed to be "concerted action to promote a safe and secure image of Northern Ireland, including targeted marketing".
The review was ordered by DETI minister Arlene Foster last year.
It calls for a need "to deepen the relationship" between the Northern Ireland Tourist Board (NITB) and Invest NI.
It rules out a merger, but suggests a shared office and renaming the Tourist Board either Visit NI or Discover NI.
The recommendations contained in the report have been put out to public consultation.
It calls for an updated strategy and says the Tourist Board needs to improve "partnership working".
"NITB is at a crossroads," the report's author writes.
"Given the board's achievements in recent years... I am confident that NITB will meet the challenge."
An "image problem" may explain a fall in visitor numbers to Northern Ireland from across the border, according to a review of the tourism industry.