What are the implications of duplicate content issues with Drupal comments?
After investigating, I don't think there's an SEO problem here. It's an annoyance for sure but it's not a problem and here's why:

As per Google's recommendations here http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html on canonical URLs and duplicate content, you will not have any problems if you have duplicate content only if you properly specify your canonical URLs. Also, Google (and most probably all other search engines) will only display the canonical URL in search results and not the other URLs.

In Drupal's case, the canonical URLs are properly set. For example, I have created a test article (node/2) and added comments and replies to it and got the following pages:


/node/2
/comment/2
/comment/3


After inspecting their generated source code, they all have the same &lt;link rel="canonical" href="/node/2" /&gt; in their source code properly created. So basically only the main article page will display in the search results.

Now, whether to modify core to only generate a single page or to have the robots.txt file patched as you specified is beyond me. I'm sure there will be lots of considerations and special cases to consider before implementing any of the above and since there really isn't a problem maybe it's best to keep things as is.

Cheers!

Note: I'm using Drupal 7.17 (latest version as of today).