No Need to configure a personalized search before the crawl. You are able to do all sorts of searches in authentic-time and Examine code or textual content modifications in-concerning crawls. Check if robots.txt timed out or blocked URLs. Also, validate if a URL was from the sitemap or if AhrefsSiteAuditBot https://www.botsaas.io/transform-your-website-into-a-conversion-machine/