be2f37df_er_Checklist___SearchTools_com__Action
[Content] META ROBOTS tags [Action] 
In addition to the robots.txt file, the Robots Exclusion Protocol allows page creators to set up robot controls within the header of each page. Be sure you recognize these options: do not index the contents of the page, but do follow the links. , you can index the page but do not follow the links , you should neither index the contents nor follow the links. I'd recommend re-fetching these pages and checking them in future indexing crawls, however, because the settings may change. For more information, see the HTML Author's Guide to the Robots META tag.
Indexing Robot Crawler Checklist - SearchTools.com
Action