Niels Bosma

SeoTools 4.0.7 is released – with MajesticSEO!

nielsbosma — Sat, 12 Jan 2013 17:50:36 +0000

I’m really happy to introduce support for MajesticSEO!

Fetch directly in Excel:

Metrics for any URL such as CitationFlow & TrustFlow, External backlinks, Indexed urls and much more.
Backlinks!
Anchor texts
All domains hosted on an IP
Keyword metrics
New & lost backlinks by date range

Read more about this on the MajesticSEO documentation page. (I’ll post a video tutorial soon!)

I’ve also reimplemented the Google Analytics tool from scratch with a new UI and authentication is now through OAUTH.

Give me SeoTools 4.0.7 now!

Happy SEOing!

/Niels

SEOlytics integration in SeoTools 3.3

nielsbosma — Mon, 13 Feb 2012 19:27:14 +0000

As a first external SEO tool, the german company – SEOlytics – has chosen to sponsor an integration of their new API.

New functions!

Other…

Changed how SeoTools handles errors. You can now opt-out to get popups for subsequent errors in that session.
Improved error handling when SeoTools.config.xml isn’t found. Also now displaying path to SeoTools.config.xml in About dialog.
Blecko functions removed due to Blecko now requires authentication.
Updated CheckBacklink so expectedLinkHref and expectedLinkText now supports wildcards.
Example:
=CheckBacklink(“http://nielsbosma.se”, “http://*.uservoice.com”,”Suggestion*”,TRUE)
Thanks to Serge Esteves for suggesting this.
Fixed problems with result strings from XPathOnUrl containing “/t” (for example “http://t.co/vrPCw2Nu” was returned as “http:/ .co/vrPCw2Nu”).
Thanks Javi Garcia fore reporting this.
Updated GooglePageRank function so PR(n/a) returns -1 (instead of 0) to separate from PR(0).
Thanks to Pieter-Jan Vanneste for suggesting this.
Fixed workaround for bug in HtmlAgility Pack (http://htmlagilitypack.codeplex.com/workitem/28317) when parsing documents with tags.
Support for Open graph protocol in HtmlMeta:
=HtmlMeta(“http://ogp.me/”,”og:title”)
Fixed bug in Google Analytics Wizard where last profileId wasn’t remembered correctly.
Added “Insert values” option in Google Analytics wizard.
Updated to the latest ExcelDna version (r.70906).
Added links to Forums, Documentation, Like on Facebook and What’s new in Help section.
Added attributes for Tlds in SeoTools.config.xml to support additional date formats, such as “dd/MM/yyyy” (.FR). New Tld attributes are WhoIsCreatedDateFormat, WhoIsUpdatedDateFormat and WhoIsExpiresDateFormat. Date formats like “dd/MM” are interpreted as the “next future date”. If today is dec. 14 2011, “01/02″ is interpreted as febr. 1 2012.
Added ability to skip update reminders until next version.

Download 3.3 today!

How to setup Tlds in SeoTools.config.xml

nielsbosma — Wed, 07 Dec 2011 05:47:50 +0000

I got a question in the SeoTool Community Forum on how to setup a .fr TLD in SeoTools.config.xml and decided to make a blog post about it.

This post assumes some knowledge in regular expressions. If you don’t know this already, regular expression are extremely powerful and really useful when working with SEO. Start by reading An SEO’s Guide to RegEx.

In SeoTools.config.xml you can add additional TLDs to support functions that use whois such as WhoIs, WhoIsDomainExpires, WhoIsDomainUpdated, WhoIsDomainCreated, DomainAge and IsDomainRegistered.

In SeoTools.config.xml, in the section add:

First we figure out a good WhoIsServer for .fr. Googling “Whois server france” gives us “whois.nic.fr”. (Also .whois-servers.net usually works).

Now we do a whois search for a domain we know exist. Either find a whois service for your particular TLD or use a command line tools such as Whois in Sysinternals.

Then we figure out the regular expression for parsing out the part that we need. To do this, I copy the whois text into RegExr.

For WhoIsCreatedRegex we create the regular expression “(?:created:\s*(\d{2}/\d{2}/\d{4}))”. Do the same for WhoIsUpdatedRegex and WhoIsExpiresRegex.

We now finally need to figure out what happens when getting whois for a domain that doesn’t exist.

We see that if the domain isn’t registered, the text “no entries found” is present in the whois result.

Our final is now:

While working on this post I noticed that SeoTools wont propertly parse DD/MM/YYYY formatted dates. I will fix this in the next release of SeoTools (3.3). Until then only IsDomainRegistered will work properly with this configuration.

UPDATE:
This was fixed in 3.3. Final for .fr is:

If you manage to make a TLD setting, please share with the community by posting in the comment section below.

New 3.2 brings you GoogleAnalytics support, SpinText and a whole bunch of fixes!

nielsbosma — Mon, 28 Nov 2011 05:28:33 +0000

It was a long time I released a new version of SeoTools, mostly because of the birth of my beautiful daughter Julia. But between diaper changes I’ve had some time to develop next release of SeoTools. And this time I think you will not be disappointed.

The primary new feature is an implementation of Google Data Export API to get any Google Analytics data into your spreadsheet.

The magic happens in the new function GoogleAnalytics but as these formulas can become quite complicated, I have built a wizard that helps pick metric, dimension etc. It’s smart enough to figure out what Goal, Custom variable and Dynamic segments you’re using.

New functions

The rest…

- Found a major problem with my builds making some of the cool stuff from previous release not working such as JsonPathOnUrl.

- Fixed problems with HtmlH1, HtmlH2, HtmlH3.

- Fixed so HtmlMeta functions can read content like “In Frida’s house there’s a lot of awesome people”. (Before “In Frida” would be returned).

- I have disabled preview for most functions (especially those with long running times) when in Excel function wizard.

- Fixed broken GoogleIndexCount, GoogleLinkCount.

- If no separator is specified in StringJoin, “” is used as the default separator
=StringJoin(,{“Niels”,”Frida”,”Julia”})

- RegexpFind now works like RegexpFindOnUrl and returns an array with all matches.

- Added support for defining HTTP authentication using urls like http://username:password@domain.com (instead of using HttpDownloaderConfigs).
See example of this in cookbook/SeoMozUrlMetrics.xls

- Added an alias for ‘extention’ called ‘extension’ in UrlProperty. (Thanks Darroch).

Be the first to download!

Fixed PR, Cookbook, and a whole bunch of scraping functionality – Helloworld SeoTools 2.3

nielsbosma — Sat, 08 Oct 2011 16:46:06 +0000

Time for a new release of SeoTools 2.3!

New functions:

For this release I’ve spent a lot of time on the scraping functionality, introducing HttpDownloader configs that allows you to control most aspects of the HTTP request in functions like XPathOnUrl and RegexpFindOnUrl.

From now on I also ship a folder called cookbook/ with a bunch of examples how to use the different functions in SeoTools.

Thanks to Amit for pointing out that GooglePageRank was broken due to Google changing their urls for PageRank toolbars yesterday.

Get your copy!

Minor fixes

- Added optional attribute queryValue to function UrlProperty for setting queryParam values on an url.

Example:
=UrlProperty(“http://domain.com/Default.aspx?a=b&foo=x”;”query”;”foo”;”bar”)
=> http://domain.com/Default.aspx?a=b&foo=bar

- Changed XPathOnUrl to return an array of nodes (similar to ImportXML) instead of a “;”-concatenated string (no more messing with (xpath)[1]).
In a similar way RegexpFindOnUrl is now returning an array if there are multiple matches.

- RegexpFind now has the ability to take an vector as input. The first match is returned.
=RegexpFind(A3:C3;”\b[A-Z0-9._%-]+@[A-Z0-9.-]+\.[A-Z]{2,4}\b”;0) => (* Returns the first email adress found in in columns A3:C3 *)

- Updated About dialog to be able to show more third party software.

- All function that take an url (such as XPathOnUrl, RegexpFindOnUrl, etc.) now supports syntax for accessing local files:
file:///C:/path/to/file.xml
file:///path/relative/workbook.xml
file:///../../path/relative/workbook.xml

(You can tell SeoTools what encoding to use, using xmlHttpDownloaderSettings.ResponseEncoding.)

Office 64bit version, W3CValidate and DomainAge in new SeoTools 2.2

nielsbosma — Mon, 19 Sep 2011 19:36:10 +0000

It’s time for a new SeoTools release!

I happy to announce that I’ve finally managed to produce a version that works with Office 2010 64bit! Make sure you download the correct version marked with “64″.

New functions

- InternetArchiveFirstSeen

- DomainAge

Other updates

- On startup SeoTools now checks http://seotools.nielsbosma.se/latestversion.txt for the latest version and prompts the user to download a new version if available. This can be inactivated in SeoTools.config.xml.

- Improved the ResponseTime function with two additional (optional) parameters: iterations (default 1) and pauseBetweenIterations (default 1000).The ResponseTime function returns the median of the values collected in each iteration.

=ResponseTime(“http://offerta.se”;10;1000)

Will measure the responsetime of offerta.se 10 times with 1000ms between each iteration and return the median response time.

- Improved ResolvedIp using “host” property in UrlProperty so i.e.: =ResolveIp(“http://mydomain.com/file.aspx”) works. Same for GoogleIndexCount, WhoIs and IsDomainRegistered.

- Improved the Format function to fully support .Net composite formatting (string.Format).

=Format(“{0:0.00}”,123.4567) => “123.46″
=Format(“{0:0000}”,3) => “0003″

For full syntax reference: http://msdn.microsoft.com/en-us/library/txafckwd.aspx

SeoTools v2.1 with backlink checker and text-to-code ratio metrics

nielsbosma — Tue, 30 Aug 2011 18:59:56 +0000

I’ve uploaded a new release of SeoTools with some interesting features.

Some of these features have been added thanks to suggestions from SeoTools users. If you’re missing anything please let me know. Remember donators get their feature suggestions prioritized.

Following new features and improvements has been added in 2.1:

CheckBacklink(string pageUrl, string expectedLinkHref, [string expectedLinkText], [bool checkNofollow])

Verifies that a backlink is present on a page with the correct link text and that there’s no rel=nofollow.

This is a very useful function when validating that links you have “acquired” are still present.

Example:
=CheckBacklink(“http://linkdirectory.com/subpage”;”http://mypage.com/”;”An awesome page”;FALSE)

In the example we verify that there a link to “http://mypage.com” on “http://linkdirectory.com/subpage” with the anchorText “An awesome page” and that this link is not rel=nofollow.

IsFoundOnPage(string pageUrl, string countOccurrencesOfString, [bool ignoreCase])

Returns the number of occurrences a string can be found on a page.

Example: Check that Google analytics tracking is present on you page:
=IsFoundOnPage(“http://offerta.se”;”google-analytics.com/ga.js”)

HtmlMetaRobots(string url)

Returns HTML meta robots on a page (cached).

HtmlMeta(sting url, string metaName)

Returns arbitrary HTML meta content on a page (cached).

Common meta tags:

author
googlebot
robots
keywords
description
refresh
copyright
contact
last-modified
cache-control
content-language
content-type
expires
pragma

Examples:
=HtmlMeta(“http://offerta.se”;”author”);
=HtmlMeta(“http://offerta.se”;”googlebot”)

HtmlCanonical(sting url)

Returns the HTML canonical on a page (cached).

PageSize(string pageUrl)

Returns page size in bytes (cached).

PageTextSize(string pageUrl)

Returns page text size in bytes (cached).

Calculates how much of the page source is text. Compared to other tools available online my version is quite restrictive, counting as little as possible. If you’re technically interested, I’m discussing this on stackoverflow. (I might change the algorithm in the future to be less restrictive and more aligned with other tools.)

PageCodeSize(string pageUrl)

Returns page code size in bytes (cached).

PageCodeToTextRatio(string pageUrl)

Returns page code to text ratio (cached). Same as PageTextSize(pageUrl)/PageCodeSize(pageUrl).

PageEncoding(string pageUrl)

Returns the page encoding detected by SeoTools.

HttpHeader(string pageUrl, string headerName)

Returns response HTTP header (cached).

Example:
=HttpHeader(“http://google.com”;”Content-Type”) => text/html; charset=ISO-8859-1

Mics

Added 1000ms sleep in GooglePageRank and GooglePlusCount to avoid your IP getting blocked by Google when performing a lot of lookups.

Download SeoTools.

New release of SeoTools!

nielsbosma — Sat, 20 Aug 2011 10:30:34 +0000

Added a new Ribbons interface. Checkout the “Donate” button

Added optional parameter “attribute” To XPathOnUrl method for selecting attributes on nodes.
Example:
=XPathOnUrl(“http://www.google.com/search?q=example”;”(//a)[1]“;”href”)

Added new function SqlFirst(string SqlQuery)Returns the first column of the first row in the result of a SQL query. Requires that you setup a ConnectionString in SeoTools.config.xml. Works great together with Format.
Example:
=SqlFirst(Format(“SELECT Id FROM Clients WHERE Email=’{0}’ AND Password=’{1}’”;A1:A2))

Added new function StringJoin(string separator; vector data) Returns concatenated string of a vector with specified separator.
Example:
A
1 Niels
2 Frida
3 Julia
=StringJoin(“, “;A1:A3)
Returns “Niels, Frida, Julia”

Added optional parameter to FacebookLikes(string url; [string specific]) Where you can specify either “total” (default), “shares”, “likes” or “comments”.
Example:
=FacebookLikes(“http://offerta.se”;”shares”);

Added new function GooglePlusCount(string url) Retrieves the Google+ count for an url.
Example:
=GooglePlusCount(“http://yahoo.com”)

Added new function TwitterCount(string url) Retrieves the number of times an url has been tweeted.
Example:
=TwitterCount(“http://nielsbosma.se”)

Added “Generate link report…” found in the “Onpage” menu.

Added new function GoogleLinkCount(string url) Returns an approximation of the number of backlinks indexed by Google. (Similar to searching for links: in Google.)

Download from SeoTools project page.

Retreive Google+ count using C#

nielsbosma — Sat, 13 Aug 2011 07:58:14 +0000

In my work with SeoTools I’m adding a Excel function for retreiving a page’s Google+ count (on request by @richardbaxterseo) and I found an excellent implementation in PHP.

In the spirit of sharing, here’s a port in C#:

private static Regex REGEX_GETURLCOUNT =
    new Regex(@"]+id=""aggregateCount""[^>]+>(\d*)");

public static int GetUrlCount(string url)
{
    string fetchUrl =
        "https://plusone.google.com/u/0/_/+1/fastbutton?url="+
        HttpUtility.UrlEncode(url)+"&count=true";
    HttpWebRequest request =
        (HttpWebRequest)WebRequest.Create(fetchUrl);
    string response = new StreamReader(request.GetResponse()
        .GetResponseStream()).ReadToEnd();
    Match match = REGEX_GETURLCOUNT.Match(response);
    if (match.Success)
    {
        return int.Parse(match.Groups[1].Value);
    }
    return 0;
}

And while I’m at it, here’s how to get the number of times an url has been tweeted:

private static Regex REGEX_GETURLCOUNT =
    new Regex(@"(\d+)");

public static int GetUrlCount(string url)
{
    string fetchUrl = "http://api.tweetmeme.com/url_info?url=" +
        HttpUtility.UrlEncode(url);

    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(fetchUrl);
    string response = new StreamReader(request.GetResponse()
        .GetResponseStream()).ReadToEnd();

    Match match = REGEX_GETURLCOUNT.Match(response);
    if (match.Success)
    {
        return int.Parse(match.Groups[1].Value);
    }
    return 0;
}

Working with regular expressions in Excel

nielsbosma — Sun, 06 Mar 2011 09:33:28 +0000

Regular expressions can be very useful for validating and manipulating data. Unfortunately there’s no built in support for regular expressions in Excel. You can use VBA but I think that’s a bit too messy.

In my Excel plugin SeoTools, I’ve added four methods:

=RegexpIsMatch(string input; string expr)	Matches an input string with a regular expression and returns true if there’s a match.
=RegexpFind(string input; string expr; int group)	Takes a regular expression and retrieves the matched group from an input string.
=RegexpFindOnUrl(string url; string regexp; int group)	Same as RegexpFind but the input string is instead the contents of a webpage.
=RegexpReplace(string input; string expr; string replacement)	Within a specified input string, replaces all strings that match a specified regular expression with a specified replacement string.

All examples below assume that you have downloaded and added SeoTools to the document you’re working on.

Example1 Validating email addresses with RegexpIsMatch

Let’s say you have a column with a bunch of email addresses and you want to find all invalid addresses. The following formula will return FALSE for all invalid email addresses:

=RegexpIsMatch(“niels@bosmainteractive.se”;”^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,6}$”

Example2 Find a particular part in a text

Now let’s say you have a column with text where you’d like to extract a particular part from, i.e. “Sports (id:4)”, where we like to extract the “4″ into a new column.

=RegexpFind(A1;”$id:(\d+)$”;1)

Example3 Get the number of pages indexed in Google for a set of keywords

For the last example we try some simple web scraping using the regular expression support in SeoTools. In our example we have a set of keywords and want to have the formula return the number of pages a search returns.

=RegexpFindOnUrl(“http://www.google.com/search?q=”&UrlEncode(A1);”About ([0-9,]*) results”;1)

With functions in SeoTools that are marked with “(cached)” (such RegexpFindOnUrl) the URL is only fetched once per “session”. So if you have several pieces you want to extract from a webpage the source of the page is only downloaded once even if you reference the URL several times in different formulas.

Resources for working with regular expressions

.Net regular expressions.
At Regexlib.com your can find a bunch of regular expressions that you can use for validating different types of data.