How To Use The Google Indexation Checker

Note: This post has been updated in 2019 as we needed to make significant changes to how this function works, in order to combat changes from Google. This is a step-by-step guide on how to use the Google Indexation Checker feature in URL Profiler. How To Set Up Indexation Checks The most common use-case for […]

Read More >>

Update 1.92 – Revamped Google Indexation Checker

Since our last big update when we added Lighthouse data to Page Speed, we’ve run a number of low-key updates that didn’t warrant a blog post about them. But this time we’ve got a fix to a core feature, so we suggest everyone jumps on this new version of the software. Improved: Google Indexation Checker […]

Read More >>

Update 1.85 – Lighthouse & CrUX Data at Scale

The latest URL Profiler update contains a new and improved PageSpeed API integration, utilizing the very recently released Lighthouse data, which you can now see in Google PageSpeed Insights: This is a departure from the old PageSpeed metrics, which were based on performance optimization rules – whereas these metrics are much more user-centric. Lighthouse Data […]

Read More >>

Update 1.83

The latest URL Profiler update contains a big fix for Mac, plus a bunch of other smaller fixes and changes which affect both Windows and Mac. Fixed: Mac TLS Issue As some of you have noticed, URL Profiler had become unable to correctly process data on some HTTPS URLs, and users were getting HTTP 400 […]

Read More >>

Update 1.78 – Critical Updates

The latest URL Profiler update contains a big fix for Mac, plus a bunch of other smaller fixes and changes which affect both Windows and Mac. Fixed: Mac TLS Issue As some of you have noticed, URL Profiler had become unable to correctly process data on some HTTPS URLs, and users were getting HTTP 400 […]

Read More >>

Update 1.74 – Removing Facebook Shares

With this update, we had to take the regrettable step of removing Facebook share count from our social share feature. The eagle-eyed among you will have noticed that it has been returning zeros for the last couple of weeks, and this is due to Facebook depreciating the API we used to gather this data. Whilst […]

Read More >>

Update 1.68 (and 1.67, 1.66, 1.65, 1.64, 1.63…)

We’ve not done a big update in a while, and this isn’t a big update either. Instead we’ve been doing lots of small updates, so this is a quick announcement post to get everyone up to speed with the things that have changed over the last couple of months. Let’s face it, you don’t want […]

Read More >>

How to Find Google Indexation Gaps in Your Sitemap

URL Profiler has a Google Index Checker feature, which allows you to check if any URL is indexed by Google. This means that you can literally take your sitemap, dump it into URL Profiler, and confirm if all your pages are properly indexed. Pretty sweet, right? Not always. If you have a site with a […]

Read More >>

Using Proxies with URL Profiler

If you’re thinking about using proxies with URL Profiler, it is most likely you are trying to check Google Indexation or Duplicate Content. And with good reason, too, since these tasks require the software to scrape Google’s results pages. Google’s Attitude Towards Scraping In short, Google hate people scraping their search results. This has been the […]

Read More >>

How to Scrape Data From Twitter Profile Pages

Update: 1/8/2019 Twitter has updated their layout to render data primarily through Javascript. You will need to scrape https://mobile.twitter.com rather than https://twitter.com. As mobile twitter still renders html in the source. We will be updating the scraper soon to be able to crawl js rendered content. I have been meaning to write this blog post […]

Read More >>