Google keeps indexing /comment/reply URL With the new update of Google algorithm called Penguin, I think my site was being penalized due to webspam. But of course I don't create post which seems to be spam to Google. It is just I think how Google index my site.

I found out that Google index the URL of my site like:

http://www.example.com/comment/reply/3866/26556

So there are so many comment/reply URL index by Google. I have already added:

Disallow: /comment/reply/
Disallow: /?q=comment/reply/

but still Google still index this URL.

Any idea how to prevent Google from indexing comments?

Here's my .htaccess:

&lt;FilesMatch "\.(engine|inc|info|install|make|module|profile|test|po|sh|.*sql|theme|tpl(\.php)?|xtmpl)$|^(\..*|Entries.*|Repository|Root|Tag|Template)$"&gt;
  Order allow,deny
&lt;/FilesMatch&gt;

Options -Indexes
Options +FollowSymLinks
ErrorDocument 404 /index.php
DirectoryIndex index.php index.html index.htm

&lt;IfModule mod_php5.c&gt;
  php_flag magic_quotes_gpc                 off
  php_flag magic_quotes_sybase              off
  php_flag register_globals                 off
  php_flag session.auto_start               off
  php_value mbstring.http_input             pass
  php_value mbstring.http_output            pass
  php_flag mbstring.encoding_translation    off
&lt;/IfModule&gt;

&lt;IfModule mod_expires.c&gt;
  ExpiresActive On

  ExpiresDefault A1209600

  &lt;FilesMatch \.php$&gt;
    ExpiresActive Off
  &lt;/FilesMatch&gt;
&lt;/IfModule&gt;

&lt;IfModule mod_rewrite.c&gt;
  RewriteEngine on
  RewriteRule "(^|/)\." - [F]
  RewriteCond %{HTTP_HOST} !^www\. [NC]
  RewriteRule ^ http://www.%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteCond %{REQUEST_URI} !=/favicon.ico
  RewriteRule ^ index.php [L]

  &lt;IfModule mod_headers.c&gt;
    RewriteCond %{HTTP:Accept-encoding} gzip
    RewriteCond %{REQUEST_FILENAME}\.gz -s
    RewriteRule ^(.*)\.css $1\.css\.gz [QSA]

    RewriteCond %{HTTP:Accept-encoding} gzip
    RewriteCond %{REQUEST_FILENAME}\.gz -s
    RewriteRule ^(.*)\.js $1\.js\.gz [QSA]

    RewriteRule \.css\.gz$ - [T=text/css,E=no-gzip:1]
    RewriteRule \.js\.gz$ - [T=text/javascript,E=no-gzip:1]

    &lt;FilesMatch "(\.js\.gz|\.css\.gz)$"&gt;
      Header set Content-Encoding gzip
      Header append Vary Accept-Encoding
    &lt;/FilesMatch&gt;
  &lt;/IfModule&gt;
&lt;/IfModule&gt;
I would try adding

RewriteCond %{HTTP_USER_AGENT} ^Googlebot
RewriteRule ^comment/reply/$ - [G,L]
RewriteRule ^comment/reply/(.*) - [G,L]


to .htaccess or Apache config to inform spiders that the links should be removed.  Just make sure to put it before the catch-all rule at the bottom of the standard Drupal .htaccess.  The problem is coming up with a list of spiders to add as [OR] conditions to the RewriteCond so that they get a 410 but Drupal handles normal requests.

Google Webmaster Tools has a URL removal tool, too, but I have never tried using it with patterns before.