AJAX & SEO: A strategic approach to rankings

Dealing with the limitations of AJAX and Flash can be an SEOs worst nightmare. There are so many issues that come into play & usually you can deal with many AJAX SEO issues by developing Progressively Enhanced code. While, I’ve written about how to address crawling and indexing with AJAX and SEO. I wanted to take a little bit of a deep dive into this again, along with the bigger issue, ranking.

The URL Problem:

As the web becomes more and more dynamic, especially with the introduction of the stream, real-time updates, etc., the ability to crawl and index that content becomes a problem. AJAX introduced a problem to search engines by dynamically changing page or website content with URL Fragments, or hash marks, “#”. this becomes a problem because traditionally (esp. in web 1.0 days) hash marks were used as an anchors to content on a single page. Since search engines would already crawl the page, they would ignore the URL fragments (“#”) because they were already crawling the page.

For example, these two URLs would be seen as the same and would have no reason to crawl the url fragment because it was already being done.:

• http://www.tonyadam.com/faq
• http://www.tonyadam.com/faq#question-10

Fixing Crawling and Indexing through Progressive Enhancement

While AJAX is not new to the web, there is a problem that it introduces to search engine crawlability and indexing that wasn’t around way back when. We now have websites that dynamically generate content within it’s current page architecture. So, sites started implementing AJAX as a way to dynamically change the page content without page refreshes and therefore create much nicer (and sexier) user experiences, as they wouldn’t have to wait for a new page to load, etc.

That said, you are left with this URL fragment that a search engine was not built to crawl and index. So, it becomes increasingly important to use web standard techniques like Progressive Enhancement and leveraging the Hijax method. Most technology teams adopt these methods and should be standard practice across technology and engineering functions.

Relevancy and Ranking Problems that arise

As more and more of the web becomes dynamic, being strategic in the uses of the dynamic content is important. The way we rewrite URLs and change page content is extremely important to having relevant content show up in the Search Engine Results Pages (SERPs). Just like building an entire site in Flash can be detrimental to your organic search traffic, building an entire site in AJAX can do the same thing. The reason being is that you end up with URL structures that contain #’s (URL Fragments) all over the place.

The drawbacks with #’s and rankings
This becomes a problem when people like bloggers, writers, site owners, etc. start linking to your content with #’s in the URLs. Search engines would ignore everything after the # and assume all links go to the homepage. For example,If I decide to build an entire site on basketball, so, assuming I want to rank for terms like basketball news, basketball scores, etc., I would probably have them in my top nav.

If I was to have the content refresh using AJAX, here is what my URL structure would look like:

• http://www.basketball.com/
• http://www.basketball.com/#/news/
• http://www.basketball.com/#/scores/
• http://www.basketball.com/#/players/
• http://www.basketball.com/#/stats/

Or deeper sections of the site like

• http://www.basketball.com/#/players/ray-allen/stats/
• http://www.basketball.com/#/players/phil-jackson/awards/

Do you see the problem with the URL Structure? If this site was purely built with AJAX, as mentioned above, anybody that links to my site would link to http://www.basketball.com/ and I would get no deep content links. If I get links to my homepage with obscure keywords like “Ray Allen player stats” or “Phil Jackson coaching awards” and since everything after the # is ignored, those keywords would then be pointed at the homepage, which is not search engine relevant.

Using #’s and AJAX to your advantage
Alternatively, there are situations where it makes more sense not to have unique URLs on content to conserve link equity and target at one main page. For example, I would not want a URL for various filters on a users stream in their profile because I would dilute the link equity to those stream items.

If I was to have a unique URL and link to the following, I would dilute link equity:

• http://www.myspace.com/tonyadam/stream/all
• http://www.myspace.com/tonyadam/stream/photos
• http://www.myspace.com/tonyadam/stream/videos

In this situation, I would rather have URLs that conserve the link equity to the main profile url:

• http://www.myspace.com/tonyadam/#/stream/all
• http://www.myspace.com/tonyadam/#/stream/photos
• http://www.myspace.com/tonyadam/#/stream/videos

By doing so, links to the # URLs will then all be targeted at the main URL http://www.myspace.com/tonyadam/ which will create more link equity to a stronger, more relevant page.

Now, we could sit here all day and talk about all the ways to manipulate results by sending links to versions of the URL that are rewritten, but, that is not a scalable tactic, is temporary, and could lead to search penalties. At the end of the day, the goal should be to create a dynamic website that is relevant search engines and users, that creates the most holistic experience.

Think of it this way, after all your hardwork and innovation, if you can’t get users to your website, what does the shiny new object matter anyway?