Making responses to public nodes only viewable by the creator and the originator
A quick search on D.o. came up with this forum post asking to do the exact same thing: http://drupal.org/node/919914

They suggest using Node Comment and Content Access together to achieve it, but I'm not certain Content Access will give you the specific option you're looking for.

It would be a start though, and you might be able to hook into Content Access and create a new "Parent Node author" or similar option if you are up for a coding session.