October 9, 2006 In creating a website we try to desperately to make something aesthetically pleasing, and there are so many methods to that madness. But the prettier the page, often means the more code we have to implement. More code equals less SEO friendly. Some things, in fact, will create a page that is not «crawler friendly,» and thus create problems when trying to get a site indexed or ranked. You want the crawlers (robots, spiders, se’s, whatever) to have as easy a time as possible. So let’s touch on a few things that will make a world of difference.

Robots tag vs. Robots.txt file

The robots.txt file is used in regard to directing a search engine. Not so much on where to go, as where not to go. You can use it to tell a search engine not to index a certain page or pages, or not to follow certain links. This is usually the case where there is confidential information you wouldn’t want a search engine to index, but can sometimes inadvertently cause a page to not be crawled at all. You will want to be careful in your robots.txt creation to avoid preventing a crawler on your site. This has, many times, been the cause for a site not being indexed. The same holds true for the robots tag. The robots tag is a little less customizable, and is rarely needed because most people put it on «index, follow» which is the default anyway.

JavaScript

A lot of cool functions and graphics can be achieved with the use of JavaScript. The problem is that sometimes the use requires a lot of code to be placed in the HEAD of the site. Though the code is ignored by the search engines, they still have to drudge through all of it to get to the readable code. Some search engines have a timeout or a maximum character read before they will just leave and move to the next site. It’s best to call the JavaScript from an outside file if at all possible. Also, be mindful that an error in JavaScript can cause a crawler to stop completely, again, moving to the next site. So, just be mindful of the extra stuff you put on your site. Try to keep the code clean and prevent a possible problem. Just remember that the crawler for a search engines has a great deal of work to do. It has to be fast, efficient, and simple. This simplicity means that it’s not going to work too hard to get what it needs before it gives up.

Thank you, Mr. Roboto. (I really couldn’t resist)

The URI to TrackBack this entry is: http://www.seo-factor.com/Blog/bblog/trackback.php/16/

Line and paragraph breaks automatic, HTML allowed: