User-agent: *
Disallow: /ingest/health
Disallow: /ingest/providers/
Disallow: /ingest/translate/
Disallow: /ingest/jobs/

Sitemap: https://cmr.earthdata.nasa.gov/search/sitemap.xml

Disallow: /search/health
Disallow: /search/tiles/
Disallow: /search/reset
Disallow: /search/community-usage-metrics
Disallow: /search/humanizers/
Disallow: /search/concepts/
Disallow: /search/granules/
Disallow: /search/collections/

Disallow: /opensearch/granules/

Disallow: /access-control/groups/
Disallow: /access-control/acls/
Disallow: /access-control/permissions/
Disallow: /access-control/health

User-agent: *
Disallow: /ingest/health
Disallow: /ingest/providers/
Disallow: /ingest/translate/
Disallow: /ingest/jobs/

Sitemap: https://cmr.earthdata.nasa.gov/search/sitemap.xml

Disallow: /search/health
Disallow: /search/tiles/
Disallow: /search/reset
Disallow: /search/community-usage-metrics
Disallow: /search/humanizers/
Disallow: /search/granules/
Disallow: /search/collections/

Disallow: /opensearch/granules/

Disallow: /access-control/groups/
Disallow: /access-control/acls/
Disallow: /access-control/permissions/
Disallow: /access-control/health

Allow crawlers to index our collection landing pages under /search/concepts route.
CMR-4379: Allow crawlers to index our collection landing pages under /search/concepts route.
