Solved: Pages were being blocked by meta robots deliberatelyA lot of pages are being blocked in the robots.txt file and when I checked the file there were no indications of a rule that blocks these.The robots.txt file is structured:Sitemap: https://domain.com/sitemap.xmlUser-agent: *Disallow: /directory-1/Disallow: /directory-2/Disallow: /directory-3/Disallow: /directory-4/Disallow: /directory-5/User-agent: Googlebot-ImageDisallow:None of the directories in the robots file match the URLs that are being blocked.I was wondering if the incomplete Disallow could be the issue, despite it being only applicable to Googlebot-Image?
Effect of incomplete Disallow rule in robots.txt file
robots.txt
I have just found that an incomplete Disallow allows all robots complete access.Found the answer here: http://www.robotstxt.org/robotstxt.html