Web Harvesting
Web harvesting has the ability to automate the process of capturing data from the web pages. It can also be thought of as a focused or directed format of Web Crawling. Search engines do help in gathering information from web pages, but a lot of labor is involved in copying the content and converting them to required formats. Web harvesting indexes all the content that is related to the audience search term from the web pages available on the Internet. It provides a very fast form of searching since it focuses on indexing the URLs to which they are directed thus reducing the size of the index. It projects a more refined search because the indexed URLs are pre-filtered for a particular topic or areas or interest.

The process is started by providing a list of URLs that map to a specific collection or source of information. The hyperlinks associated with these URLs can be ignored or used depending on the type of intended usage.

Web Harvesting can refer to various processes, for example web structure harvesting, web content harvesting and web usage harvesting.

When performing web content harvesting, a particular aspect of the web documents is focused on, such as hypertext files, electronic messages, pictures or product pricing. Web usage harvesting collects its data from web servers keeping in mind the users’ needs so it can better anticipate user behavior.

A robust, feature-rich web harvesting application is Visual Web Ripper. It can extract information from dynamic web pages even when the information is in a format other than plain HTML, such as AJAX, ASP.NET or any other technology. The tools used for web harvesting follow a step-by-step procedure to fetch the desired information. Firstly, the user formats the harvest based on one page from site from which the data is required to be extracted. Visual Web Ripper then mimics this extraction from the remainder of the site’s specified pages.

The harvested data from the web pages can be converted to various formats like word, excel, CSV, XML, text, or other database formats. Government agencies also use web harvesting to enforce policies. It helps the business professionals to analyze their competitors and also the marketing techniques used by them. They can also use it to gather information about the selling price, competitors information, customer data and financial information of various types.

A Road Map to the World Wide Web: Web Site Planning and Preparation

Article by Gabrielle Melisende

The World Wide Web, also referred to as the Internet, offers businesses yet another media through which to educate, promote or sell their product or service, giving businesses and consumers instant access to one another 24/7. Can you believe it was once projected that the Internet would peak in the early 2000s? Today, statistics show that nearly one million new web sites are launched in the U.S. every year.

Having a web address and online presence has become as common place, and expected, as having a logo and business card. Business-to-business and business-to-consumer marketing places a great demand as well as dependence on having a domain name (web address), company email address and web site. Why? They instill consumer confidence just like a physical office or store front, mailing address, phone number and business card.

It is a mistake for any business to overlook the value and importance the Internet plays in business success, not just globally but regionally too. Free web email addresses are acceptable for personal use but using them for business undermines credibility. Many businesses can exist and be successful without a web site however few can operate without email access. It is highly advisable that all businesses purchase a domain name and set up and use a company email address. Free web mail providers such as Yahoo.com make it easy and affordable to set up an online business identity without the cost of hosting, developing a web site or buying an email program.

At some point your business will be ready to enter the World Wide Web. When it is, be aware and prepared that your web site, if used to generate traffic and sales, is like running another business. Planning and preparation are both critical in setting up your web site, maintaining it and marketing your business online. Do your homework or you will be sorry. Research web hosting providers, packages and web design firms, know the fair market rate in your area and for the type of web site you desire. If possible consult with a marketing professional to determine your goals for the web site before your purchase hosting or web design services. Acquire a basic understanding of internet marketing before your web site is designed. Both design and content are critical to your online marketing, especially in Search Engine Optimization.

If your web site is going to be heavily content managed consider purchasing a web site building software or select a hosting provider and plan that includes a free web site builder. Note, this advice is not for everyone. There is a huge learning curve and time commitment to do it yourself, many free web site builders will not allow full customization of the template or graphics, and you will need to learn how to use web administration tools. This is not a feasible option for everyone.

If you select a web design firm or freelance web designer for your web site get everything in writing, all components and details of the design services, hosting and domain name registration as well as time frame to complete the project. It is the consumer’s responsibility to know and understand what services are provided by the web designer and what items you are expected to provide, such as content and graphics. Be aware that not all web designers are graphic designers, and vice versa. Do not expect the web designer to educate you on marketing your web site, including keywords and search engine optimization. These services may, however, be available through the web designer at additional cost.

Once published, your web site becomes a living, breathing entity that you will need to take care of: market, update or refresh, and maintain. If used correctly, your web site will continue to evolve as an integral part of your business, if not your sole business.