Thanks Michael. I think using the noindex in the robots.txt and meta tags can be a great tool for totally removing pages from a search index.
In our case we had to keep some pages on the site in a duplicated fashion (along side the snippets), so we decided to nofollow links to duplicate pages to help chop them off the link graph. We should probably apply the noindex rule to those pages as well since that would really prevent any of them from making it into the index ...and I just thought of using the canonical URL tag on those pages in case someone does link to any of them.
Thanks Michael. I think
Thanks Michael. I think using the noindex in the robots.txt and meta tags can be a great tool for totally removing pages from a search index.
In our case we had to keep some pages on the site in a duplicated fashion (along side the snippets), so we decided to nofollow links to duplicate pages to help chop them off the link graph. We should probably apply the noindex rule to those pages as well since that would really prevent any of them from making it into the index ...and I just thought of using the canonical URL tag on those pages in case someone does link to any of them.