Internet Search Tools

• Search Engines
• The Spider
• The Indexer
• The Search Engine Software
• Directories
• Meta Search Tools
• Getting a Site Listed
• Improving Ranking

---

Approximately 85 percent of Internet users find website via search engines. Understanding how search engines work can assist a business in learning how to use them to not only find products and services, but also to drive potential customers to their own sites.

The term "search engine" is often used generically to describe both true search engines and subject directories. However, directories and search engines are not the same. While directories depend on human judgment for their listings, search engines create their listings electronically and automatically.

Search engines run automatically and index many millions of web pages. As a result, they often locate information that is not listed in directories.

Using a search engine can result in more precise information being returned than searching using directories. However, it can also produce results that, while matching the search terms, have nothing or very little to do with the subject being searched.

Search Engines

All search engines are made up of three basic parts:

• The spider
• The indexer
• The search engine software

Each search engine implements these elements differently. That is why conducting the same search on different search engines often produces different results.

The Spider

The spider, also referred to as the crawler, literally crawls the web and fetches documents. Most search engines run several spiders that explore the web as a team. Which pages the spiders visit and how often they do so varies from search engine to search engine.

The Indexer

The indexer reads the documents collected by the spider and creates an index. The indexis where the spider stores what it finds. In most cases, the index contains a text-only copy of every web page visited. Because this process is automated, the resources are vast, with the biggest search engines indexing several million sites.

The Search Engine Software

The search engine software is the program that sifts through the several million web pages recorded in the index to find matches to a search and then rank them in order of what it believes is most relevant.

Each search engine uses its own algorithm to determine relevancy and ranking. How each search engine's algorithm works is a closely kept secret. However, the following conditions are often considered:

• The location of keywords on a web page
  Title tags containing the search terms are often assumed to be more relevant than titles that don't. Similarly, pages that contain the search term near the top of the page are also considered to be of higher relevance than those that don't.
• Keyword density in relation to the search term
  Search engine algorithms will analyze how often search keywords occur in relation to other words on a web page. The greater the keyword density, the more relevancy the page might be given.
• Link analysis
  By analyzing how pages link to each other, a search engine can determine the subject of a web page and its importance to the subject.
• Clickthrough metrics
  A search engine will often analyze which search result links are being chosen by web users and attach greater relevancy to pages chosen more frequently.

Attempts to artificially or deceptively manipulate the way search engines determine relevancy is called search engine "spamming". Search engines are constantly monitoring sites for spamming behaviour. If caught, a site risks being removed from the index altogether.

Directories

Directories are hierarchically organized indexes of subject categories that allow the user to browse through lists of website by subject, in search of relevant information.

Directories are a practical place to start when looking for information on a fairly broad subject. In addition to being manually compiled, directories use staff searchers to evaluate, review and categorize website. A directory provides a direct link to only a tiny fraction of the total pages on the web. More detailed information can be located by moving through the general subject categories to more specific categories.

As real people compile directories, results are pre-selected and more likely to give a general, but relevant, result.

Meta Search Tools

Meta search tools don't search the entire Internet, instead they access multiple search engines, collect the results and present them to the user. The best meta search tools perform additional processes such as eliminating duplicate hits and reordering hits according to their own evaluation of relevancy. Their weakness is that search results are often less precise and sometimes slower.

Getting A Site Listed

All site owners want their site listed in the major search engines. The good news is that, with no effort, all sites will eventually be listed. The bad news is that without some effort it could take many months before the search engine spiders locate and index the site.

To speed up the process, a site can be submitted to a search engine. Prior to submitting a site the site should be prepared for submission. At minimum a description needs to be created using as many key terms as possible. Generally the description should be less than 25 words, but some services have other restrictions. It's best to check.

Key terms are words or phrases that people may use to search for the site that is being submitted. When selecting key terms, it is a good practice to include location terms such as city names, product and service offerings and specific information that differentiates the website from others.

There are two options available for getting the site listed:

• Free Listings
  The least expensive way to get a site listed is to visit the major directories and search engines, look for the "add URL" button and follow the submission instructions. While this type of submission is absolutely free, there is no guarantee that the submission will be accepted or when the site will be listed, although it generally takes four to six weeks. Once listed with a major directory, other search engines will eventually list the site, as the directories are spidered by the major search engine crawlers.
• Paid Listings
  To make sure a site is listed and the process happens faster, a listing can be purchased directly from the search engines. A range of costs, features and options is available. Search engines have recently undergone tremendous consolidation. When selecting which search engines to list with, it is valuable to understand that most of the search engines are supplied by one of only two indexes, those maintained by Google or Yahoo.

Search Engine Services

Due to the complexity of the search engine submission process, it may be most practical to engage a company that specializes in search engine submission and website optimization. When selecting a firm, assess their deliverables, follow up services, customer service and track record. Price can be deceiving in this industry.

Improving Ranking

Getting a site indexed and listed with the search engines is only the first step. Most site owners want their site to rank in the top ten. Due to the sheer volume of website, the odds of achieving this are low unless a site offers information on a very unique topic, product or service.

However, it is still desirable to make sure that a site ranks as high as possible for specific search terms. There are several tactics that can be implemented to achieve this:

Paid link advertising

Most search engines offer a paid listing service. Some of these basic paid listings services will guarantee placement in search results for related key terms. Other search engines have paid link advertising options that involve purchasing or bidding on placement of links within search results for specific terms or categories. While this is an effective means of improving ranking, the value of the money spent will need to be evaluated periodically.

Site optimization

Site optimization is the process of preparing a website for optimal search engine indexing. Site optimization is an ongoing process, which begins with the initial preparations when submitting a site to the search engines for the first time. This first phase requires the creation of meta tags, link building, search engine submission and initial monitoring of the results.

After the initial phase, site optimization consists of making adjustments to the site that will assist in maintaining or improving ranking, including:

• Adjustments to key term positioning and densities
• Changing which key terms to focus on
• Keeping the site's content current and relevant to selected key terms
• Continuing to build and reassess links created as part of a linking strategy
• Evaluating paid listings and paid link advertising strategies
• Resubmitting to search engines when major new content is added or a particular page is dropped from an index

Site optimization is a continuous process that includes monitoring results and evaluating adjustments made. It is time consuming but necessary if high search engine rankings are a requirement for the success of the business.