About Scoreling

Scoreling checks one web page at a time for SEO, markup, links, and speed, gives it a score out of 100, and explains every problem it finds. It is free and needs no account.

ScorelingBot

If you found this page in your server logs, the requests came from our checker. Its User-Agent is shaped like a phone browser, the way Googlebot's smartphone agent is, so sites that choose their markup by User-Agent send the same mobile page Google indexes. It always names itself at the end:

Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Mobile Safari/537.36 (compatible; ScorelingBot/1.0; +https://scoreling.com/about)

What it requests

It never crawls on its own schedule. It visits a site only when somebody pastes one of its pages into the form on scoreling.com, and then makes these requests, one at a time:

one GET of the page itself, following up to five redirects;
one GET of /robots.txt, and one of the sitemap it names (or /sitemap.xml);
a HEAD to each distinct link on the page, at most 50 of them, and a single GET only for links that refuse HEAD.

The speed figures come from Google's PageSpeed Insights, so that part of a check is a visit from Google, not from us. The same page checked again within 15 minutes reuses the earlier report instead of fetching anything.

robots.txt and blocking

ScorelingBot does not read robots.txt for permission. Every visit is started by a person who asked about that page, the same way a browser or an HTML validator would be, so crawl rules written for search engines do not apply. It reads robots.txt only to report whether Google may crawl the page.

To keep it out, refuse any User-Agent containing ScorelingBot, or requests from 109.206.178.44, at your server or firewall. A check of a blocked page then says the page could not be fetched, and nothing else is requested.

Limits we hold ourselves to

Public addresses only: private, loopback, and cloud metadata addresses are refused, including after a redirect.
Only http and https on the standard ports. A response is read up to 2 MB and dropped after that.
Each visitor can start 10 checks an hour.
Reports are shared by link but kept out of search engines, and the stored copy of a page is deleted after 30 days.