Update 1.78 – Critical Updates

Posted on: November 20th, 2017 by Patrick Hathaway in Updates

The latest URL Profiler update contains a big fix for Mac, plus a bunch of other smaller fixes and changes which affect both Windows and Mac.

Fixed: Mac TLS Issue

As some of you have noticed, URL Profiler had become unable to correctly process data on some HTTPS URLs, and users were getting HTTP 400 (Bad Request) response codes.

This issue was limited to the Mac version, and only on certain URLs. Without going into the boring details, we’ve managed to fix the problem now, so users should no longer come up against this issue.

Since this issue affects data quality, we’d recommend all Mac users update to the latest version.

Added: Render Timeout to Screen Capture

The screen capture feature runs through a headless browser called PhantomJS. Using Phantom, URL Profiler renders the page content and then generates the screenshot – almost as if it were opening the page up in Chrome or Firefox.

Previously, the screen capture would fire as soon as the DOM was loaded. However, some websites use JavaScript to change the page AFTER the DOM has loaded, so you would often find that the screen captures did not perfectly match what you could see in a browser.

To help mitigate such issues, we have added a ‘render timeout’ feature, which allows you to add a delay to the process, so that URL Profiler will wait for a few seconds before triggering the screen capture.

To control the render timeout, click ‘Screen Capture’ and then when the box pops open, adjust the render timeout number (which is in seconds):

Render Timeout

To learn more about render timeouts, check out my Sitebulb article on crawling JavaScript websites (Sitebulb is our new website crawler).

Added: More SEMrush Regional Support

SEMrush have added support for a whole host of new countries over the last few months, and URL Profiler now supports all 135 regional databases from SEMrush.

Fire up your Icelandic keyword search volumes!

Improved: Ahrefs metrics granularity

In Ahrefs, you can choose to look at metrics on a range of levels, depending on how granular you want to get. On a domain level, the main choices are ‘HTTP’, ‘HTTPS’ or ‘both’; and whether you include subdomains or not. In Ahrefs it looks like this:

Ahrefs metrics granularity

Previously, URL Profiler would simply use the recommended option, but in response to a few requests for more granularity, we’ve added this customisation as an option. Head to Settings -> Link Analysis where you can specify your Ahrefs settings.

Ahrefs settings


  • Alexa changed the data they offer, so our Alexa API was affected. For a while there is was giving back empty results – now it doesn’t 🙂
  • URL Profiler was not correctly parsing TLDs for some South African TLDs.
  • Google+ share count has been removed as an option, since Google stopped supporting the feature.
  • URL Profiler will now better handle situations where there are no ‘Keyword Opportunities’ suggestions for your Search Analytics results.


Existing customers or existing trial users can grab the new update from here:

If you’ve not tried URL Profiler yet, you can start a free 14 day trial here. The trial is fully featured, and you don’t need to give us any payment details to get started.