Reform Scotland said its research suggested more than half of delays to trains in Scotland were due to Network Rail faults rather than ScotRail.
It said devolving the organisation would allow a "clear line of accountability" when things go wrong.
Network Rail is funded by Transport Scotland but is ultimately accountable to Westminster, not Holyrood.
A spokesman for Network Rail said: "We work closely with the Scottish and UK governments to continually enhance and improve our railways. Any discussions over further devolution of Network Rail in Scotland would be a matter for those governments".
The full devolution of Network Rail - which is responsible for maintaining rail infrastructure - has been a long-standing ambition of the Scottish government.
Transport Minister Humza Yousaf said last month that the move would help to reverse "the current unacceptable industry performance" in Scotland.
It comes amid ongoing concern over delays and cancellations to ScotRail trains since Dutch firm Abellio took over the contract last year, which has led to Mr Yousaf suggesting that the rail franchise could be taken over by the public sector in the future.
Mr Yousaf is to attend a conference on Tuesday alongside Abellio and Network Rail officials aimed at exploring the future of Scotland's railways.
In its Track to the Future report, Reform Scotland said 54% of trains which were delayed by more than three minutes had been attributed to Network Rail faults.
The report was written by Reform Scotland's research director Alison Payne and Tom Harris, the former Labour MP and UK transport minister.
Mr Harris said: "The current debate over whether or not to nationalise ScotRail misses the point. 54% of delays are the fault of Network Rail. Nationalising ScotRail won't make the trains run on time and it is self-defeating for any politician to imply otherwise.
"Instead, we need fundamental change to the governance of Network Rail. The Scottish government is responsible for the strategic direction and funding of the Scottish rail network, but this responsibility cannot be properly exercised while Network Rail remains answerable to the UK government.
"Reform Scotland believes that Network Rail in Scotland should be fully accountable to the Scottish government, and that means it must be devolved."
The report, which examines the modernisation of Scottish railways, also found that journey times in Scotland compared unfavourably with those of a similar distance in England.
For example, Edinburgh to Aberdeen and London to Birmingham are roughly the same distance, but the former takes well over two hours and the latter less than 90 minutes.
Birmingham and Manchester are 25% further apart than Glasgow and Dundee, but the train takes 15 minutes less to arrive.
While it welcomed recent investment in the railways, the report said upgrading lines while trying to use them could cause additional problems and added that investments such as the Borders Railway had been "short-sighted".
The new line from Edinburgh to Tweedbank is not electrified and is single-track.
The report said the Scottish government should future-proof all new rail investment by ensuring that it is double-track and electrified where possible.
And it called for the creation of a "bold and ambitious" Scottish Rail Infrastructure Commission to look into faster rail links for the future, including from the central belt to both the north and the south, and looking at strategic inter- and intra-city rail links within Scotland.
The report found that, under current proposals, over the next 30 years people will be able to travel by train from the central belt to London faster than from the central belt to Inverness, despite the former being around 400 miles while the latter is only 160 miles.
Mr Harris added: "Ongoing improvements are vital and the Scottish government should be congratulated for investing in this area, but we need to think about the bigger picture and create a bold strategy for the future.
"Our rail infrastructure is deficient. We should not tolerate a situation where a resident of Glasgow can get to London faster than she can get to Inverness, because Scotland's cities are too poorly connected."
Calls for the Scottish Parliament to be given full responsibility over Network Rail have been backed by a think tank.