high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

collection of scrapers, plus online toolkit for building them

interesting on many levels

html scraping; free and commercial versions

Beautiful Soup is a Python HTML/XML parser designed for quick turnaround projects like screen-scraping. Two features make it powerful:

1. Beautiful Soup won't choke if you give it bad markup. It yields a parse tree that makes approximately as much s

The Rise (and Fall?) of the Scraper-conomy