Human Rights Watch believes Nato air strikes killed at least 72 civilians and says the organisation needs to bear responsibility where appropriate.
"We're calling for prompt, credible and thorough investigations," HRW's Fred Abrahams told BBC News.
Nato insists it took unprecedented care to minimise civilian casualties.
It argues that it cannot take responsibility because it has had no presence on the ground to confirm the deaths.
Aircraft from the US, the UK and France conducted most of the 9,658 strike sorties last year, targeting forces loyal to Libyan leader Col Muammar Gaddafi.
In March, another human rights organisation,  
          Amnesty International, said it had documented 55 cases
 of named civilians, including 16 children and 14 women, killed in air strikes.
It described Nato's failure to investigate these cases thoroughly as "deeply disappointing".
The point of the Nato air campaign in Libya last year was to protect civilians, so how many innocent people died is still a sensitive issue, BBC diplomatic correspondent Bridget Kendall reports.
In its report published on Monday, HRW said it had examined in detail evidence of claims of civilian deaths from eight separate Nato strikes.
In one instance, it said a first Nato bomb killed 14 people and a second bomb, moments later, killed 18 more who had rushed to help victims.
What concerns Mr Abrahams, the main author of the report, is that the deaths remain unacknowledged and the families have been offered no compensation.
"Until now, Nato has taken a position of denial," he said.
Source: Nato
"They refuse to give information about how they died and they refuse to investigate, and it is this lack of transparency that is deeply troubling.
"I think it will lead to unnecessary civilian deaths in the future if Nato refuses to look at what went wrong and make corrections."
Nato says it is ready to co-operate with the new Libyan authorities in assessing what further action is appropriate.
But so far the task force being set up in Tripoli seems to have made little headway, our diplomatic correspondent adds.
A leading human rights organisation has urged Nato to investigate fully the deaths of civilians in air strikes in Libya last year.