Understand the competitor

Understanding the Neighborhood

Before I do any work on a website I try to get an idea of where it fits into the grand scheme of things on the World Wide Web. The easiest way to do this is to run searches for some of the competitive terms in the website’s niche. If you imagine the Internet as one giant city, you can picture domains as buildings. The first step I take before working on a client’s website is figuring out in which neighborhood its building (domain) resides.

This search result page is similar to seeing a map of the given Internet neighborhood. You usually can quickly identify the neighborhood anchors (due to their link popularity) and specialists in the top 10 (due to their relevancy). You can also start to get an idea of the maturity of the result based on the presence of spam or low-quality websites.

Also, take note that regardless of whether or not you are logged into a Google account, the search engine will automatically customize your search results based on links you click most. This can be misleading because it will make your favorite websites rank higher for you than they do for the rest of the population.

Along with looking at the results themselves, I look at the other data present on the page. The amount of advertisements on the search result gives a rough idea of how competitive it is.

In addition to the ads, I also look for signs of temporal algorithms. Temporal algorithms are ranking equations that take into account the element of time with regards to relevancy. These tend to manifest themselves as news results and blog posts.

Taking Advantage of Temporal Algorithms

You can use the temporal algorithms to your advantage. I accidentally did this once with great success. I wrote a blog post about Michael Jackson’s death and its effect on the search engines a day after he died. As a result of temporal algorithms my post ranked in the top 10 for the query “Michael Jackson” for a short period following his death. Because of this high ranking, tens of thousands of people read my article. I thought it was because I was so awesome, but after digging into my analytics I realized it was because of unplanned use of the temporal algorithms. If you are a blogger, this tactic of quickly writing about news events can be a great traffic booster.

After scanning search result pages for the given website’s niche, I generally get a sense for that neighborhood of the Internet. The important takeaway is to get an idea of the level of competition, not to figure out the ins and outs of how specific websites are ranking. That comes later.

Link Relevancy

Popularity and relevancy are the two concepts that make up the bulk of Search Engine Optimization theory.

In the previous tutorial we have discussed about The Secrets of Relevancy

Now we will understand Link Relevancy.

As search engines matured, they started identifying more metrics for determining rankings. One that stood out among the rest was link relevancy.

The difference between link relevancy and link popularity is that link relevancy does not take into account the power of the link. Instead, it is a natural phenomenon that works when people link out to other content.

Let me give you an example of how it works. Say I own a blog where I write about whiteboard markers. (Yes, I did just look around my office for an example to use, and yes, there are actually people who blog about whiteboard markers. I checked.) Ever inclined to learn more about my passion for these magical writing utensils, I spend part of my day reading online what other people have to say about whiteboard markers.

On my hypothetical online reading journey, I find an article about the psychological effects of marker color choice. Excited, I go back to my website to blog about the article so (both of) my friends can read about it.

Now here is the critical takeaway. When I write the blog post and link to the article, I get to choose the anchor text. I could choose something like “click here,” but more likely I choose something that it is relevant to the article. In this case I choose “psychological effects of marker color choice.”

Someone else who links to the article might use the link anchor text “marker color choice and the effect on the brain.”

People have a tendency to link to content using the anchor text of either the domain name or the title of the page. Use this to your advantage by including keywords you want to rank for in these two elements.

This human-powered information is essential to modern-day search

engines. These descriptions are relatively unbiased and produced by real people. This metric, in combination with complicated natural language processing, makes up the lion’s share of relevancy indicators online.

The Secrets of Relevancy

In the previous tutorial How page is ranked by search engine, I discussed how popular pages (as judged by links) rank higher. By this logic, you might expect that the Internet’s most popular pages would rank for everything. 

To a certain extent they do (think Wikipedia!), but the reason they don’t dominate the rankings for every search result page is that search engines put a lot of emphasis on determining relevancy.

Relevancy is the measurement of the theoretical distance between two corresponding items with regards to relationship. Luckily for Google and Microsoft, modern-day computers are quite good at calculating this measurement for text.

Size of Google Indices

So what does this emphasis on textual content mean for SEOs? To me, it indicates that my time is better spent optimizing text than images or videos. This strategy will likely have to change in the future as computers get more powerful and energy efficient, but for right now text should be every SEO’s primary focus.

But Why Content?

The search engines must use their analysis of content as their primary indication of relevancy for determining rankings for a given search query.

For SEOs, this means the content on a given page is essential for

manipulating—that is, earning—rankings.

In the old days of AltaVista and other search engines, SEOs would just need to write “Jessica Simpson” hundreds times on the site to make it rank #1 for that query. 

What could be more relevant for the query “Jessica Simpson” than a page that says Jessica Simpson 100 times? (Clever SEOs will realize the answer is a page that says “Jessica Simpson” 101 times.) 

This metric, called keyword density, was quickly manipulated, and the search engines of the time diluted the power of this metric on rankings until it became almost useless. Similar dilution has happened to the keywords meta tag, some kinds of internal links, and H1 tags.

The funny thing is that modern-day search engines still work essentially the same way they did back in the time of keyword density. The big difference is that they are now much more sophisticated. Instead of simply counting the number of times a word or phrase is on a webpage, they use natural language processing algorithms and other signals on a page to determine relevancy. 

For example, it is now fairly trivial for search engines to determine that a piece of content is about Jessica Simpson if it mentions related phrases like “Nick Lachey” (her exhusband),“Ashlee Simpson” (her sister), and “Chicken of the Sea” (she is infamous for thinking the tuna brand “Chicken of the Sea” was made from chicken). The engines can do this for a multitude of languages and with astonishing accuracy.

In addition to the words on a page, search engines use signals like image meta information (alt attribute), link profile and site architecture, and information hierarchy to determine how relevant a given page that mentions “Jessica” is to a search query for “The Simpsons.”

How page is ranked by search engine

Domain and Page Popularity

There are hundreds of factors that help engines decide how to rank a page. And in general, those hundreds of factors can be broken into two categories—relevance and popularity (or “authority”). For the purposes of this demonstration you will need to completely ignore relevancy for a second. 

Further, within the category of popularity, there are two primary types—domain popularity and page popularity. 

Modern search engines rank pages by a combination of these two kinds of popularity metrics. These metrics are measurements of link profiles. 

To rank number one for a given query you need to have the highest amount of total popularity on the Internet. (Again, bear with me as we ignore relevancy for this section.)

This is very clear if you start looking for patterns in search result pages. Have you ever noticed that popular domains like Wikipedia.org tend to rank for everything? This is because they have an enormous amount of domain popularity. 

But what about those competitors who outrank me for a specific term with a practically unknown domain? This happens when they have an excess of page popularity. See Figure 1-1.

Figure 1-1: Graph showing different combinations of relevancy and popularity metrics that can be used to achieve high rankings 

Although en.wikipedia.org has a lot of domain popularity and get.adobe.com/reader/ has a lot of page popularity, www.awesome.com ranks higher because it has a higher total amount of popularity.