a webkit web client written in python

is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

ghost.py is a webkit web client written in python.

As you might remember, I gave my Scrape the Web tutorial again at PyCon this year, 2010.

I finally got to publishing the 2010 tidbits on the web. So without further ado:

A video of me on blip.tv
A tar.gz file of code samples
A brief four-page cheat sheet
My full slides, photos, jokes and all

PhantomJS is a headless WebKit with JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.

New tumblr post!