In order to help site owners better understand how Googlebot crawls their websites, Google has launched a brand new version of its Crawl Stats report.
For those unfamiliar with the search giant’s Crawl Stats report in Search Console, it can be a useful tool which allows you to better understand how well your site and its content is performing online.
For instance, if the crawl rate on a site suddenly drops this could indicate that a server is misconfigured or that there is a problem with the quality of the site’s content. However, the crawl rate can also drop if a site is experiencing a DDoS attack or if other issues are hurting its performance.
An increase in crawling though can either show that useful content has been added or that a misconfiguration is causing a site to auto-generate duplicate or thin web pages.
New Crawl Stats report
Google’s improved Crawl Stats report includes a number of exciting new features such as the ability to view detailed information on host status, total number of requests, URL examples to show where in your site requests occurred and more.
The report even provides site owners with access to over-time charts that show total requests, total download size and average response time. If you’re average response time is higher than it should be, then it may be time to consider switching your site to another web hosting service.
The new version of the Crawl Stats report also includes data on crawl requests broken down by response, file type of the fetched URL, purpose of the crawl request and Googlebot agent. Meanwhile, host status details in the report provide site owners with information on their site’s general availability to Google over the course of the last 90 days.
Keeping a website up and running can certainly be difficult but by using the data available in Google’s Crawl Stats report, site owners can make appropriate changes to ensure that their sites are available in the company’s search engine and reachable by their users or customers.
- We’ve also highlighted the best VPN services
