Or are they? Today’s treasure isn’t buried. It’s spread across a vast online network called the Internet and it involves the nascent science of SEO.

As the science of SEO has developed, so have the SEO tools. One tool that you should be aware of is called a web crawler simulator. The web crawler simulator essentially shows you how search engines are viewing your website so you can respond accordingly, tweaking your SEO strategies to reflect the algorithms of the search engines.

What is a web crawler simulator?

First, let’s make sure we understand what a web crawler simulator is. That requires backing up a bit and making sure we understand what a web crawler is. A web crawler is a complex computer program that systematically “crawls” or searches the Internet and forms search engine entries from what it reads. A good way to think of it is like a giant automated indexing tool for the Internet. The “index” in this case is search engines like Google, Yahoo, or Bing. Some people refer to web crawlers as web spiders.

That helps us understand what a web crawler simulator is. A web crawler simulator helps you to analyze your website according to the search engine web crawlers.

What can a web crawler simulator do?

If you have a tool that tells you how Google is looking at your site, you have a treasure map. Taking your site to the first page of a Google search engine is essentially like striking gold. The web crawler simulator can help to take you there.

Web crawler simulators do not look like very attractive maps. Gone are the frayed parchments and the red “X.” A web crawler simulator simply shows you what the web crawlers “sees” when it crawls your page. The result is a bland readout of HTML coding, sans pictures.

The new perspective—viewing your website through the eyes of a crawler—can help you make changes that will improve the SEO quality of your site. It demonstrates your keyword density and your metadata. It is a “dry run” of how the crawler will look at your website, and therefore where you’ll end up in line when Google, Yahoo, or Bing return results from a search engine query.

Free Web Crawler Simulators

Here are a few common web crawler simulators that you can try:

• http://www.smart-it-consulting.com/internet/google/googlebot-spoofer/index.htm
• http://www.webconfs.com/search-engine-spider-simulator.php
• http://www.seobench.com/search-engine-crawler-simulator/
• http://www.mydigitallife.info/tools/search-engine-spider-and-crawler-simulator/

Nobody ever said that treasure hunting was easy work. It still takes time, requires that you dig a little bit, and you may end up with some sore muscles. But I think you’ll be glad you did.

Related posts:

1. Bing Crawler Optimization Tips
2. Is AJAX SEO Possible?
3. See Your Website With the Eyes of a Spider

The Problem with Ajax SEO

SEO crawlers were originally designed for HTML. Somehow, they’ve never been able to totally crawl out of the HTML age, though they are getting better. However, because of this problem many Ajax pages simply are not crawled by the SEO robots. Other problems are prevalent among Ajax pages. Since one URL governs an entire Ajax page, SEO-crucial links and navigation are missing. SEO crawlers may even suspect unethical SEO techniques when crawling Ajax pages. Further problems abound—lack of indexing, crummy site navigation, and saving/sending/bookmarking problems abound.

A Helpful Tool for Ajax SEO

Ajax doesn’t support deep linking. It could create some frustration for bookmarking and site navigation. Even though Ajax alone doesn’t give you deep linking capabilities, there are ways to skirt this problem. Perhaps the most popular and widely used tool for creating unique virtual URLs is SWFAddress. This powerful tool essentially turns Flash and Ajax into simple HTML, allowing users to navigate, bookmark, save, e-mail and refresh Ajax pages. The SEO power of SWFAddress is that it creates actual HTML links that give the Ajax site more visibility to SEO robots. As long as the Ajax engineer submits an XML sitemap using SWFAddress to the search engines, there is a much better possibility of improving Ajax SEO.

Insightful Tips for Ajax SEO

Rather than create an entire site in Ajax, make a distinct URL for individual pages in order to get more visibility with the SEO robot.

Include keywords, and place them at the beginning of the site where they are more likely to be found by the SEO crawler. Use Ajax only when it’s necessary, not merely for a flashy site. Ajax is helpful for achieving a page’s dynamic interaction with the server. The fact still holds true that although you can obtain some degree of Ajax SEO, it will not be as high as a non-Ajax site.

In case you are craving a yes or no answer to the article title, here it is. Yes, AJAX SEO is possible. Of course, there is a disclaimer. Ajax SEO is limited. By following the insightful tips above, you’ll obtain at least some level of AJAX SEO.

Related posts:

1. The 60 Minute SEO Website Audit
2. Bing Crawler Optimization Tips

Enter a new head-scratching dilemma:  what is involved in Bing crawler optimization? Even though Google still tops the charts when it comes to search engine usage, there are some important facts you need to know when optimizing your site for Bing’s SEO crawler. Crawler mavens and SEO masters have come up with some answers. Here’s what you need to know about Bing crawler optimization. First, some general advice, and then a list of tips.

General Advice:  Bing is Not Google

First, you must be aware that Bing is not Google. Yes, that’s a painfully obvious statement, but here’s why it’s worth stating:  Bing uses a totally different system of crawling than Google does. For this reason, the optimization techniques and expectations that you may be familiar with under Google simply will not work with Bing. Here are some specifics.

Bing crawls slower than Google. Bing is what you could call a very cautious search engine. Rather than instantaneously crawl and rank post on the web that has high keyword or backlink levels, Bing is patient enough to wait for verification, warranting, and approval from a network of sites. Obviously, this means a longer wait for better Bing search engine ranking, but the slow-and-steady approach also protects Bing’s reputation. Bing demands a higher degree of honesty and integrity from the get-go than does Google. Bing wants high quality content. Bing’s SEO robot wants unique pages. Focus on quality and be patient. You’ll hit Bing’s pages soon enough.

Bing wants you to submit your site. For faster site ranking, you should submit your site to Bing. Bing wants it that way. If you open a webmaster account with Bing, which requires signing up with a Windows Live ID, you will have a higher degree of ‘trust’ from Bing. Adding your site to Bing and including a site map is an important first step in getting recognized by Bing.

Bing is young  … but growing. In the final analysis, don’t get overly panicky if you just can’t figure out Bing. SEO Crawlers and SEO robots are confusing. Bing is a young search engine, and as formidable as it may seem, it simply hasn’t hit the tipping point of being the major search engine. Google should still be your bread and butter when it comes to optimization techniques. Bing may catch up over time, and by then we’ll have some more things figured out about Bing’s SEO robot.

Specific Advice: A list of tips

So for the practical, brass tacks kind of advice, here’s a list of tips on Bing crawler optimization:

• Go for low keyword density. If you have more than three keywords per page, you’ll get the hairy eyebrow from the SEO crawler.
• Go for uniqueness. Specifically, you should have unique <title> tags and <meta> description tags. The title tags should match the keywords on the rest of the content in your site.
• Where possible, use text navigation links, ranger than graphical links within the site.
• Go for stellar content. Think, “A very intelligent human being is going to read this,” not, “Hmm. Wonder how the Bing crawler is going to do here?” Aim for powerful content, not just Bing crawler optimization.
• Focus. A narrow collection of keywords is important. Bing crawler optimization means that you should not use a broad range of keywords in your site content. Stick to a topic and focus on the keywords within that topic.
• Stay ethical. Any no-nos of SEO technique are still no-nos on Bing. They nab keyword stuffing and hidden text remarkably well.
• Aim for high quality backlinks, not just a big quantity of backlinks. To borrow a cliché: quality not quantity. Older sites are considered “high quality.” Bing doesn’t like Blogspot, nor really any blog sites for that matter. On the other hand, Bing likes Hubpages a lot.
• Put keywords into your URL.

That list of advice is pretty simple and short, but will go a long way as you engage in the kind of Bing optimization that will put you at the top of the page.

Related posts:

1. Anchors Aways: Anchor Text Optimization for SEM
2. Local Search Engine Optimization Tips
3. 10 SEO Facebook Tips

According to Google.com, the Changefreq XML tag may be set to one of seven frequencies: “never”, “yearly”, “monthly”, “weekly”, “daily”, “hourly”, or “always”. This tells the search engines approximately how often each page is updated. An update refers to actual changes to the HTML code or text of the page, not updated Flash content or modified images. Changefreq examples…

NEVER: Old news stories, press releases, etc.
YEARLY: Contact, “About Us”, login, registration pages.
MONTHLY: FAQs, instructions, occasionally updated articles.
WEEKLY: Product info pages, website directories.
DAILY: Blog entry index, classifieds, small message board.
HOURLY: Major news site, weather information, forum.
ALWAYS: Stock market data, social bookmarking categories.

The Priority XML sitemap tag is useful, although not quite as important. It is set to a number ranging from zero to one; if no number is assigned, a page’s priority is 0.5. A high priority page may be indexed more often and/or appear above other pages from the same site in search results. Here are some examples of different types of pages and how their Priority sitemap XML tag value might be set, depending upon how important they are…

0.8-1.0: Homepage, subdomains, product info, major features.
0.4-0.7: Articles and blog entries, category pages, FAQs.
0.0-0.3: Outdated news, info that has become irrelevant.

How strictly they want to follow the Priority and Changefreq sitemap specifications is up to the search engines; these XML tags are considered preferences, not orders. This doesn’t mean search engines don’t consider Priority and Changefreq important, just that they won’t put sitemap instructions before their own interests (like making sure a site hasn’t changed its subject or become pornographic).

But why is it important when or how frequently search engine “spiders” index your pages? When a “spider” visits a web page, it records information about the page’s content, title, META tags, links, and other characteristics. This ensures that search results reflect its latest content and take into account any recent improvements (such as new META tags or repaired links).

However, it is unnecessary for “spiders” to regularly scan pages that are seldom or never updated. Spider indexing consumes bandwidth (which can increase the cost of operating your website), and may briefly slow access to your site if it is run on a low-capacity server. Thus it is important to set the Changefreq sitemap tag to accurately reflect how often individual pages are updated.

No related posts.

1. Optimize The PDF File Name. When you create the pdfs, be sure to include your SEO targeted keywords in the file name.  Make sure that Google can identify what the PDF is about.  Don’t overdo it, but try to work in a few keywords into the file name like target-keyword.pdf

2. Complete Document Properties. Most PDFs are indexed without specified document properties, the most important of which is the Title. This document property is the equivalent of the html title tag.  If you don’t complete the Title property, the search engine is going to creat a title from the PDF’s content, and it may not be optimized for your targeted SEO keywords.  There are other PDF meta data properties that can be completed, the only other one of importance is the Subject property which is the equivalent of a meta description for .html pages.

3. Optimize your text in the PDF. For your PDF’s optmize the copy just like you would web page copy.  Don’t overdo it, but use target keywords in the first hundred words.  Try to use some variations to support long-tail while focusing on a main keyword for optimization.

4. Build links into PDFs. Include links in your PDFs, and pay attention to the anchor text used as Search engines recognize these links. In addition to including links in PDFs for search-related purposes, there’s also a good business reason as PDFs are often passed along to others via email.

For B2B websites with PDFs and White Papers, this optimization of PDFs can be incredibly important for your overall SEO strategy.  Qualified B2B visitors can be hard to find, make sure you give yourself every opportunity by leveraging your PDFs for top rankings.

Related posts:

1. How To Optimize WordPress Permalink Structure
2. Search Engine Optimization and Flash

1. Since the introduction of support for SWFObject in July, Google hasn’t associated text content in Flash with the correct parent URL or as a single entity.
2. Flash files can accrue PageRank independent of their own parent URLs.
3. Google doesn’t index URLs containing #anchors (fragment identifiers) in Flash per W3C Guidelines.
4. Currently, Google doesn’t seem to support translations of text content in Flash.

These case studies continue to show that SEO for Flash can be achieved but only if done properly and then it will have limited value when compared to a site without Flash.  SEO can be done for Flash, but it is not optimal.  The Adobe announcement has not changed the recommendations for the immediate future because:

• Google can see text and links inside the Flash file which is good but it will not split them up a Flash file into multiple pages and index them separately. That means that your Flash file will be the equivalent of one, massive HTML page, unless you break it up into multiple HTML landing pages.  Remember that this can’t use the #anchor because Google will not treat that as a separate URL.
• Google will not run all types of Javascript and this makes it uncertain in the fact that Googlebot will index a Flash file that is embedded using the SWFObject script.
• If you have content dynamically loading into your Flash movies from an 3rd party XML file, this content may not be indexed by Google.

Until this is settled, one of the best alternatives is to make  an HTML alternative that is not only helpful to the search engines, it’s also great for people without the Flash plugin and for disabled site visitors.

Sources for Blog Post:

http://googlewebmastercentral.blogspot.com/2008/06/improved-flash-indexing.html

http://www.beussery.com/blog/index.php/2008/10/google-flash-seo/

http://www.yourseoplan.com/seo-for-flash.html

Related posts:

1. Search Engine Optimization and Flash

The reason so much importance is placed on the page title is that it helps search engines understand what a web page is all about, thereby increasing the relevance of search results. However, recently there has been some question as to whether longer page titles rank better or not. Here are some thoughts on that topic:

Many experts wonder if there is a downside to having title tags that match a popular search query exactly. The reasoning behind this train of thought is that these types of title tags will be seen as an advertising attempt to trick the user into clicking rather than true content to help searchers.

So, this may be one reason a longer page title could rank better than one that includes only the searched term or phrase.  We have found that a 5-6 phrases followed by the company seems to work best.  Here is one example of a page title we have done:

Another thought is that the end user experience is better with a longer page title. This would be because it’s generally easier to understand a page’s content with a longer title.

If you do decide to go with a longer title, though you don’t want to make it overly long. Keep in mind that Google only displays the first 63 characters in search results so the searcher won’t see more than that.

While Yahoo and MSN show more characters it’s best to go with a title that can be displayed fully on all 3 major search engines. You also only want to use your best, or most important, keywords. Typically pages rank better when there is more than one keyword in the tag.

So, it seems as though the answer may be that medium page titles work best. You don’t want a title that seems spammy to the search engines or the end user on one hand. But, on the other hand you don’t want a title that will be cut off so that your topic isn’t clearly shown to the search engines and end users.

We see many sites that go over-board and place 100 – 125 characters in the title tag.   While not confirmed, Google is always looking for ‘signs’ of a spammy site and a long, keyword rich TITLE.   Our recommendation is to use the full 63 characters for the page TITLE without going over and place your most important keywords first.   In addition, if you are going to put your company or website name into the title, we recommend doing that at the end of the page title.

No related posts.

Watch Video Here

You can get more information at Google Webmaster Blog Post.   We are glad to see the constant improvements by Google on the webmaster tools and we avidly monitor and manage the information in the Google Webmaster Tool interface.

If you want to take advantage of this new feature to not only reduce 404′s but increase links, here is a sample email you could send to those sites with 404 links to your site to increase both your link count and your anchor text relevance:

(Person),

This is (clients name) from (client) and we noticed through our Google Webmaster Tools that you have a link to our site. First off, I would like to say thank you for linking to (client website) as we believe it is a valuable resources for your website visitors. However, we did find that there is an issue with the link that is pointing to our site. The issue is that the page you linking to no longer exists on our site and we would really appreciate it if you could update this link to an active page. To help you find the page that is linking to our site we have included a screen shot below that contains this information. Right below this information is the link to the new page we would like you to point this link to with the anchor text. We would like you to replace the broken link with this new information.

Broken Link:
(Put bad link here)

New Link:
(Put new link here with optional description if page allows it)

Please let me know if you have any questions and I can help you work through those.

Thank you ,
(Clients Name)

No related posts.

URL styles affect SEO (Search Engine Optimization) in two significant ways. First, the words contained in the URL – its subdomain, domain, subdirectory, and filename – help the corresponding webpage to gain a higher ranking in search results for these keywords. If there is more than one word in the domain name, filename, or other part of the structure, it is important for the words to be separated (when possible). They can be separated using hyphens or underscores, although hyphens are typically preferred. This allows the search engine’s computer software to distinguish between each of the separate keywords. It is best if these keywords are also used in the text of the webpage in question.

Second, URL structure is also important to SEO because people see URLs when looking at search result listings (on most search engines). They are more likely to visit a site if its address is short and/or contains words related to the subject of interest. A webpage with a relevant domain or subdomain name seems more likely to provide useful information than a page without this advantage. Most major search engines make any search keywords in the URL appear bold, helping to attract the attention of searchers. Yahoo shows the domain name in bold as well. This factor probably isn’t as important to SEO as the impact URL structure has upon search result positioning, but it is worth keeping in mind.

Both of these factors encourage website operators to use a long URL structure with numerous separated keywords. However, very long URLs do have some non-SEO drawbacks. People are reluctant to write them down or use them in print, they take longer to copy & paste, and they use more storage space and bandwidth (because of the long URLs appearing in HTML link codes). A domain name with many words may be forgotten if it is not immediately added to a user’s bookmark/favorites list. It is best to balance these concerns by avoiding the use of multiple subdirectories, leaving out unnecessary connecting words, and curtailing any excessive insertion of keywords in URLs.

Related posts:

1. How To Optimize WordPress Permalink Structure

This change comes from Google due to the fact that they can better understand the purpose of a page based on its URL structure now. John Mu of the Google team had this to say: “One of the reasons for that is that we can use the information provided through the parameters to better understand what your site is doing with those parameters. For instance,the URL http://www.mysite.com/search.php?q=keyword can give us information about what is happening, it could even allow us to recognize that this is a search form and perhaps let us attempt other keywords that might lead us to content that we haven’t seen for your site. On the other hand, a URL like http://www.mysite.com/search/keyword does not give us any information at all about what the “file name” is used for.”

The question now is do you need to undo all of the dynamic URL rewrites you already have. While there is not a punishment for switching your URL formats it may take a bit of time. So, essentially changing your URL rewrites really depends on how many rewrites you have and how well those pages currently rank. One suggestion is to start with your lower performing URLs and see the effects of switching them.

Otherwise you may find that you lose ranking popularity and lots of time making a switch. And what’s most important is making sure your site doesn’t suffer in search engine optimization.

No related posts.