There is little point spending time and money creating a website that cannot be found. Search engine optimisation can make a site appeal to the spiders crawling the web Web administrators have previously been able to secure high rankings for sites on search pages by outwitting the spiders that crawl and index web pages. Those...
Optimising a site for search engines
Allow search bots to crawl your site without session IDs or arguments that track their path. These techniques are useful for tracking individual user behaviour, but the access pattern of bots is entirely different.
Using these techniques may also result in incomplete indexing of your site, as bots may not be able to eliminate URLs that look different but actually point to the same page
Make sure your web server supports the If-Modified-Since HTTP header. This feature allows your web server to tell Google whether your content has changed since it last crawled the site. Supporting this feature saves you bandwidth and overhead
Make use of the robots.txt file on your web server. This file tells crawlers which directories can or cannot be crawled. Make sure it is current for your site so that you do not accidentally block any crawlers
If your company buys a content management system, make sure the system can export the content so that search engine spiders can crawl your site.