Robots Meta Tag
The following information comes straight from Google.
Multiple content values
We recommend that you place all content values in one meta tag. This keeps the meta tags easy to read and reduces the chance for conflicts. For instance:
<META NAME=”ROBOTS” CONTENT=”NOINDEX, NOFOLLOW”>
Unnecessary content values
By default, Googlebot will index a page and follow links to it. So there’s no need to tag pages with content values of INDEX or FOLLOW.
If you use both a robots.txt file and robots meta tags
If the robots.txt and meta tag instructions for a page conflict, Googlebot follows the most restrictive. More specifically:
- If you block a page with robots.txt, Googlebot will never crawl the page and will never read any meta tags on the page.
- If you allow a page with robots.txt but block it from being indexed using a meta tag, Googlebot will access the page, read the meta tag, and subsequently not index it.
Valid meta robots content values
Googlebot interprets the following robots meta tag values:
- NOINDEX - prevents the page from being included in the index.
- NOFOLLOW - prevents Googlebot from following any links on the page. (Note that this is different from the link-level NOFOLLOW attribute, which prevents Googlebot from following an individual link.)
- NOARCHIVE - prevents a cached copy of this page from being available in the search results.
- NOSNIPPET - prevents a description from appearing below the page in the search results, as well as prevents caching of the page.
- NOODP - blocks the Open Directory Project description of the page from being used in the description that appears below the page in the search results.
- NONE - equivalent to “NOINDEX, NOFOLLOW”.
A word about content value “NONE”
As defined by robotstxt.org, the following direction means NOINDEX, NOFOLLOW.
<META NAME=”ROBOTS” CONTENT=”NONE”>
Warning: some webmasters use this tag to indicate no robots restrictions and inadvertently block all search engines from their content.
–> End of Google Content <–
New tag coming soon:
Google is coming out with a new tag called “unavailable_after” which will allow people to tell Google when a particular page will no longer be available for crawling. This will be great for e-commerce sites that have special offers that expire after a certain period of time.
Order The SEO Book for more great information on search engine optimization. I highly recommend it. You will also find helpful SEO tools on the site.
Tags: googlebot tags, meta tags, robots meta tags, robots tags
