platform for acquiring, integrating, cleansing, analyzing and publishing data on the web.  Using Needlebase through a web browser, without programmers or DBAs, you can easily:
* acquire data from multiple sources
* merge, deduplicate and cleanse
* build and publish custom data views
MERGING INTO GOOGLE and shutting down June 1, 2012

#FrankfurtGestalten #OpenSource #Scraping #Frankfurt #PARLIS //cc @insideX @ELFPiraten

A few days ago I was doing a TON of scraping, and as you know, without the right tools, scraping can be a REAL pain. Out of my pain comes your pleasure — here’s a list of scraping tools and resources which will make your life MUCH easier the next time you need some information from a crufty old website. If you’re short on time, skip to the end and read the tl;dr.

PhantomJS is a headless WebKit with JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.

PhantomJS is an optimal solution for fast headless testing, site scraping, pages capture, SVG renderer, network monitoring and many other use cases.

List of scrapers

Hacker News
This tiny script is a scraper for http://news.ycombinator.com. It shows just how little Ruby is required to run a powerful web filter.

What it be doin':
Fetches the YCombinator news page
HPricot's it into pieces
Builds an RSS feed
Uses links to parse the destination article page and include the main text inline
Runs as a stand-alone server thanks to Rack and Mongrel

"NASCAR.com, to my knowledge, does not provide an API for acquiring driver stats. Here is a screen scraper for NASCAR.com to acquire all the relevant stats and races."