How long does it take to crawl a website? It appears that my audit is stuck.
The amount of time it takes for our bot to crawl a website varies depending on your crawl-delay setting and the size of your audit (how many pages we need to crawl).
In some cases, you can experience a delay in the Site Audit after the crawling pages counter stops. Usually, this happens in the last stage of the audit. At this time Site Audit bot checks on the external links and images and saves the information. If your website has a lot of external links and images, it is not unusual to see a delay, and it can take some time for your audit to finish.
Most of the time, there is nothing to worry about, and you can stop crawling yourself if needed:
If you stop and save results in the middle of a crawl, the audit will present the new results of its unfinished crawl as your up-to-date audit.
If you discard results, Site Audit will not crawl or audit your pages and your crawl budget will be saved.
However, if your audit is stuck for more than 24 hours, and you cannot stop it yourself, please contact our Support team.
- How is Site Health Score calculated in the Site Audit tool?
- Why does Semrush say I have duplicate content?
- How many pages can I crawl in a Site Audit?
- How do I audit a subdomain?
- What Issues Can Site Audit Identify?
- Why does Semrush say I have an incorrect certificate?
- What are unoptimized anchors and how does Site Audit identify them?
- What do the Structured Data Markup Items in Site Audit Mean?
- Why do working pages on my website appear as broken?
- Can I change automatic Audit re-run schedule or stop it completely?
- Why only a few pages of my website are being crawled?
- Why can’t I find URLs from the Audit report on my website?
- Configuring Site Audit
- Site Audit Overview Report
- Site Audit Thematic Reports
- Reviewing Your Site Audit Issues
- Site Audit Crawled Pages Report
- Site Audit Statistics
- Compare Crawls and Progress
- Troubleshooting Site Audit
- Exporting Site Audit Results
- How to Optimize your Site Audit Crawl Speed
- How To Integrate Site Audit with Zapier
- Using Semrush to Find Areas for Improvement on Your Website
- Optimizing Page Title and Meta Description via Semrush