"In God We Trust," tweeted New York City Mayor Mike Bloomberg this month. "Everyone else, bring data."

Bloomberg, the billionaire founder of Bloomberg L.P., is now in his third term as mayor of the Big Apple. During his tenure, New York City has embraced a more data-driven approach to governing, even when the results of that data-driven transparency show a slump in city services.

This should be no surprise to anyone familiar with the mission statement of his financial data company:

Bloomberg started out with one core belief: that bringing transparency to capital markets through access to information could increase capital flows, produce economic growth and jobs, and significantly reduce the cost of doing business.

To reshape that mission statement for New York City, one might reasonably suggest that Bloomberg's data-driven approach to government is founded upon that belief that bringing transparency to government through access to information could increase capital flows, produce economic growth and jobs, and significantly reduce the cost of the business of government.

As Gov 2.0 goes local, New York City has become the epicenter for many experiments in governance, from citizensourcing smarter government to participatory budgeting to embracing a broader future as a data platform.

One of the most prominent New Yorkers supporting architecting a city as a platform is the city's first chief digital officer, Rachel Sterne.

Sterne gave a keynote speech at this year's Strata NY conference that explained how data-driven innovation informs New York's aim to be the nation's premier digital city.

"I'm especially excited to be speaking with you because as a city, we need your help," said Sterne to the assembled Strata attendees. "As the data practitioners and data scientists who are at the forefront of this revolution, all of our efforts are for naught if you are not part of them and not helping us to expand them and helping to really take advantage of all of the resources that the city of New York is trying put at your disposal."

Video of Sterne's talk is embedded below.

New York City's digital strategy is focused on access to technology, open government, engagement and industry. "Industry is important because we need to make sure the private sector has all the supports it needs to grow and thrive and help to create these solutions that will help the government to ultimately better serve the public," said Sterne. "Open government is important because if our data and our internal structure and priorities aren't completely open, we're not going to be able to enable increased [open] services, that kind of [open] exchange of information, etc. Engagement is crucial because we need to be constantly gathering feedback from the public, informing and serving. And access is the foundation because everyone needs access to these technologies."

Big data in the Big Apple

What does data-driven innovation look like in New York City? Sterne focused on how data "evolves government," asserting that it leads to a more efficient allocation of resources, a more effective execution, and a better response to the real-time needs of citizens. Although she allowed that, "as everyone knows, data can be manipulated."

Sterne highlighted several data-driven initiatives across the city, including the Metropolitan Transit Authority's Bus Time Initiative. "Initially, it was scoped out to hundreds of millions of dollars. The MTA ended up working with a local open-source development shop, [which] did it for a fraction of that, below a million dollars, and now you can get real-time updates on your phone based on where the buses are located using very low-cost technologies."

New York City is also using data internally, explained Sterne — like applying predictive analytics to building code violations and housing data to try to understand where potential fire risks might exist. If that sounds familiar to Radar readers, it should: Chicago is also looking to use data, developers and citizens to become a smarter city. "This is as much about citizens talking to the infrastructure of the city as infrastructure talking to itself," said Chicago CTO John Tolva in an interview last March. "It's where urban informatics and smarter cities cross over to Gov 2.0."

Web 2.0 Summit, being held October 17-19 in San Francisco, will examine "The Data Frame" — focusing on the impact of data in today's networked economy.

Save $300 on registration with the code RADAR

New York City, however, has a vastly greater "digital reach" than Chicago. It's bigger than many corporations and states, in fact, connecting to more than four million people through NYC.gov and social media channels that have expanded to include Twitter, Facebook, Tumblr, YouTube and Foursquare. Sterne envisions the city's 200-plus social media platforms as a kind of "digital switchboard," where citizens ask questions and government workers direct them to the appropriate resources, much in the same way that California connects citizens to e-services with social media.

The web as the 21st century public square

"What we're really seeing that's interesting about all these things is that they're happening in public, so people are informing one another," said Sterne. "They're engaging one another, and it's not so much the city telling you what to do but creating a forum for that conversation to take place." If you visit NYC's custom bitly URL shortener, on.nyc.gov, you can see what content is popular within that community.

Back in May, when NYC's digital roadmap was released, Anil Dash highlighted something important: the roadmap captured New York City government thinking about the web as a public space. This has profound implications about how it should be regulated, treated or described. "The single biggest lesson I got from the
65-page, 11.8mb PDF is a simple one," Dash, a native New Yorker, blogger and entrepreneur, wrote. "The greatest city in the world can take shared public spaces online as seriously as it takes its public spaces in the physical world."

City as a platform

Sterne's description of a "city as a platform" is one of the purest articulations of Tim O'Reilly's "government as a platform" vision that I've heard any public servant articulate this year.

"The thing that's really exciting to me, better than internal data, of course, is open data," Sterne said during her Strata Conference talk. "This, I think, is where we really start to reach the potential of New York City becoming a platform like some of the bigger commercial platforms and open data platforms. How can New York City, with the enormous amount of data and resources we have, think of itself the same way Facebook has an API ecosystem or Twitter does? This can enable us to produce a more user-centric experience of government. It democratizes the exchange of information and services. If someone wants to do a better job than we are in communicating something, it's all out there. It empowers citizens to collaboratively create solutions. It's not just the consumption but the co-production of government services and democracy."

Sterne highlighted the most important open data initiative that the city has pursued to date, the NYC DataMine. Soon, she said, they will be introducing "NYC Platform," which she described as "the city's API." All of their work opening the data, however, "doesn't matter if we're not evangelizing it and making sure people are using it."

NYC has used an app competition to draw more attention to its open data. As I've written elsewhere, by tying specific citizen needs to development, NYC Bigs Apps 3.0 is part of the next-generation of government apps competitions that incorporate sustainability, community, and civic value.

"We've had about 150 apps developed," said Sterne. "There are apps that would be a significant cost to the city. Instead, they're at basically no cost because the prize money is all donated. We provide 350 datasets. Until now, they were not API-enabled. They were not dynamic, but we're going to be doing that because that's the overwhelming response that we're receiving from everyone."

That feedback is widespread in the open government data community, where studies show that developers prefer to explore and interact with data online, as opposed to downloading datasets. When it comes to developers working with public data, dynamic access can open up entire new horizons for potential applications, as the release of real-time transit data has demonstrated.

Sterne shared some useful examples of apps that have been created using NYC open government data, including Roadify, which allows you to find parking spots or transit information, and Don't Eat At, a Foursquare app that sends users a text message when they check into a NYC restaurant that is at risk of being closed for health code violations.

Sterne's message to data scientists was generally quite well received at Strata. "Pleased to see @RachelSterne's keynote today," tweeted Alistair Coote, a NYC Web developer at RecordSetter. "If done right, open govt will be far more important than anything announced at #f8 today," he observed, referring to Facebook's new look.

Why open government data matters to New Yorkers

The experience in NYC during Hurricane Irene "once again proved the utility and importance of open data and the NYC DataMine, as several organizations used OEM's Hurricane Evacuation Zone geographic data to build maps that served and informed the public," Sterne told me via email. "This data has been public for over a year. Parties developing tools built on city platforms included WNYC, NYTimes, Google, Mobile Commons and Crisis Commons. NYC Digital was also in regular contact with these parties to alert them of information changes."

The key insight coming out of that August weekend, with respect to the city acting as a platform during unprecedented demands for information, was that th[…]

Map of solar potential (and currently implemented solar installations) in New York.

In contrast to green technologies like solar power and biofuels, which take decades of research and massive funds to scale, big data tools offer a relatively capital-efficient way to use proven technology to better manage resources, fight climate change and get ready for the 9 billion people on the planet by 2050. If there’s no Moore’s Law delivering rapid and exponential progress for things like batteries, and solar panels, why not try to leverage Moore’s Law in a more indirect way for greentech via big data? Here are 5 companies that are leveraging big data to help the planet:

UC Berkeley’s Geospatial Innovation Facility and the Public Interest Energy Research (PIER) program offers new tools for researchers, policymakers and the general public to explore climate change

An Introduction to Data Mining

Op deze website geven we een zo compleet mogelijk overzicht van interessante (nieuwe) databronnen van de gemeente Amsterdam. Uiteraard staat het voor deelnemers open zelf nieuwe/andere databronnen te vinden en te gebruiken tijdens de wedstrijd bij het ontwikkelen van de applicatie. Wel dient er minimaal één databron van de gemeente Amsterdam gebruikt te worden.

Want to see all of the data we have reported? Here's all the data we've covered over the last two years, that's almost 600 spreadsheets linked from one spreadsheet

I have prepared dozens of charts & graphs looking at every aspect of global warming that I could. My early Excel based analysis is here. My more recent R based analysis is here. I have used publicly available data sources to reach my own conclusions. I provide the data links and my R script for each chart/ graph so that interested readers can confirm my work for themselves.

Here's a quick summary of my evidence for global warming.

Wal-Mart, a retail giant, handles more than 1m customer transactions every hour, feeding databases estimated at more than 2.5 petabytes—the equivalent of 167 times the books in America’s Library of Congress (see article for an explanation of how data are quantified). Facebook, a social-networking website, is home to 40 billion photos. And decoding the human genome involves analysing 3 billion base pairs—which took ten years the first time it was done, in 2003, but can now be achieved in one week.

All these examples tell the same story: that the world contains an unimaginably vast amount of digital information which is getting ever vaster ever more rapidly. This makes it possible to do many things that previously could not be done: spot business trends, prevent diseases, combat crime and so on. Managed well, the data can be used to unlock new sources of economic value, provide fresh insights into science and hold governments to account.

But they are also creating a host of new problems.

At the Internet Governance Forum in Sharm El Sheikh this week, there was a conversation on social networking data. Someone made the point that there are several different types of data, and it would be useful to separate them. This is my taxonomy of social networking data.

Exclusive: Watchdog's estimates of reserves inflated says top official

How DataSF.org, which publishes dozens of official data sets, is starting to transform life and politics in San Francisco

Welcome to our new environment data store, a bank of information, facts and figures on today's most pressing environmental issues. We've partnered with organisations including the World Resources Institute to bring you the latest data on climate change, natural resources, conservation, consumption, green living and more.

Track how millions of fans interact with online music everyday.

"Functional visualizations are more than innovative statistical analyses and computational algorithms. They must make sense to the user and require a visual language system that uses colour, shape, line, hierarchy and composition to communicate clearly and appropriately, much like the alphabetic and character-based languages used worldwide between humans." Matt Woolman Digital Information Graphics Goal VisualComplexity.com intends to be a unified resource space for anyone interested in the visualization of complex networks. The project's main goal is to leverage a critical understanding of different visualization methods, across a series of disciplines, as diverse as Biology, Social Networks or the World Wide Web. I truly hope this space can inspire, motivate and enlighten any person doing research on this field.

I recently attended an excellent Made by Many event hosted at BBH which featured a re-presentation by Manuel Lima of his 2009 TED talk on data visualisation. Manuel is the curator of visualcomplexity.com and is an eloquent, modest, charming pioneer in this fascinating field. As a novice myself, I could not help wondering why we are all so immediately and instinctively attracted to the best of data visualisation.To start with, I’m sure there is some fundamental truth that for most of us data become meaningful only when we can see scale, change, patterns and relationships. Seeing is understanding.

Make or explore Freedom of Information requests

A place where you can collaborate with other people to investigate things 1. Start an investigation 2. Invite other people to help you investigate it 3. Collaborate and share answers

The European Commission's Digital Competitiveness report published today shows that Europe's digital sector has made strong progress since 2005: 56% of Europeans now regularly use the internet, 80% of them via a high-speed connection (compared to only one third in 2004), making Europe the world leader in broadband internet. Europe is the world's first truly mobile continent with more mobile subscribers than citizens (a take up rate of 119%). Europe can advance even further as a generation of "digitally savvy" young Europeans becomes a strong market driver for growth and innovation. Building on the potential of the digital economy is essential for Europe's sustainable recovery from the economic crisis. Today the Commission has asked the public what future strategy the EU should adopt to make the digital economy run at full speed.

OpenStreetMap is a free editable map of the whole world. It is made by people like you. OpenStreetMap allows you to view, edit and use geographical data in a collaborative way from

Open Plaques is a project to find and document all the UK’s blue heritage plaques, which commemorate sites where famous events occurred, or with a connection to notable historical figures.

Britain has “sleepwalked into a surveillance society”, it was claimed last night after figures disclosed that public bodies had obtained access to private telephone and e-mail records about 1,400 times a day. Council, police and other organisations made more than half a million requests for confidential communications data last year. The statistics constitute a 44 per cent rise in requests over the past two years.

The American Time Use Survey asks thousands of American residents to recall every minute of a day. Here is how people over age 15 spent their time in 2008.

Computing and communicating through the Web makes it virtually impossible to leave the past behind. College Facebook posts or pictures can resurface during a job interview; a lost or stolen laptop can expose personal photos or messages; or a legal investigation can subpoena the entire contents of a home or work computer, uncovering incriminating or just embarrassing details from the past.

A data protection specialist claims that users can gain control of their browsing history and have it protected by the UK’s Data Protection Act just by contacting companies such as Google, Yahoo! and Microsoft and telling them their identities.

So presently the City of Vancouver only shares its garbage schedule (which it divides into north and south) as a PDF file. This is a pity as it means that no one can build any apps around it. Imagine a website or Iphone app that mashed up google maps with a constantly up to date city garbage pick up schedule. With such a application one could: (..)

Today, at the Semantic Technology Conference, Rob Larson and Evan Sandhaus of the New York Times announced together that the Times will soon be publishing its copious index as Linked Data.

This is a self-initiated project based on the scenario – If the world were a village of 100 people. There are a few different versions of this text in circulation about the world’s statistics. I found the data very striking and neatly summarises the world that we live in. So I used information graphics to re-tell the story in another creative way. I designed a set of 20 posters, which contain most of the information. I used simple vector graphics that related to a statistic in order to present the information in the simplest and most accessible way.

The reason I am obsessed with self-tracking is because I think there is a way to track yourself in such a way that it leads to epiphanies about yourself, about the cause and effect of things, in such a way that these numbers would eventually be able to tell you things about yourself that you didn’t already know. This is the only reason to self-track, in my opinion.

What is more important, the rights of the mass, or the rights of the individual. In the western world we tend to assume an inverse relationship between individual rights and social control. More social control leads to less individual rights and vice versa. Marshall suggests that individual rights may be less important than the ‘greater cause’ of being able to provide more value to users if data is freely accessible.

The Cloud is looming large, offering us ways to store and share our data in ways that were never before possible. We can effortlessly share our documents and photos with our families and friends, while maintaining control over their spread using powerful granular privacy controls. But it's quickly becoming clear that the cloud isn't ready for us. Because the services we rely on are letting us down with a frequency that is simply unacceptable.

Fact: World Bank has been collecting massive amounts of data, for the past 50+ years, and now possesses one of the richest repository of information about economic development in the world. World Bank Open API is an initiative of the World Bank that opens the wealth of the World Bank's global economic data to the outside world, in a standard, easily accessible way. Open API allows third parties to develop mash-ups and applications with the World Bank data and easily create different kinds of interesting visualizations and insightful reports.

The classic book on statistical graphics, charts, tables. Theory and practice in the design of data graphics, 250 illustrations of the best (and a few of the worst) statistical graphics, with detailed analysis of how to display data for precise, effective, quick analysis. Design of the high-resolution displays, small multiples. Editing and improving graphics. The data-ink ratio. Time-series, relational graphics, data maps, multivariate designs. Detection of graphical deception: design variation vs. data variation. Sources of deception. Aesthetics and data graphical displays.

This website is part of the Master's thesis The Form of Facts and Figures, developed by Christian Behrens in the Interface Design program at Potsdam University of Applied Sciences. Its goal is the development of a design pattern taxonomy for the field of data visualization and information design.

An Introduction to Information Design

Today's consumer seems to have an insatiable appetite for information, but until recently making sense of all of that raw data was too daunting for most. Enter the new "visual scientists" who are turning bits and bytes of data -- once purely the domain of mathematicians and coders -- into stories for our digital age.

Waar gaat het om? De Tweede Kamer heeft ingestemd met een aanpassing op de Energiewet waardoor straks iedereen verplicht is een zogenaamde slimme energiemeter te laten installeren. Deze energiemeter registreert op detailniveau (per kwartier!) uw verbruik, al deze gegevens worden dan verzonden naar uw energiemaatschappij en lekker lang opgeslagen zonder dat u er veel aan kunt doen. Tevens is het aftappen of het manipuleren van de slimme meters door derden niet al te moeilijk. Kortom, inbreuk op de privacy nummertje zoveel is aan de lijst toegevoegd.

The Open Platform is the suite of services that make it possible for our partners to build applications with the Guardian. We've opened up our platform so that everyone can benefit from our journalism, our brand, and the technologies that power guardian.co.uk.

This un-conference is about convening a trans-partisan tribe of open government advocates from all walks — government representatives, technologists, developers, NGOs, wonks and activists — to share knowledge on how to use new technologies to make our government transparent and meaningfully accessible to the public.

A googol of Googlers published a paper at VLDB 2008, "Google's Deep-Web Crawl" (PDF), that describes how Google pokes and prods at web forms to see if it can find things to submit in the form that yield interesting data from the underlying database.

That least-understood principle from my original Web 2.0 manifesto, "Data is the Intel Inside," is finally coming out of the closet. A post on the Google Operating System Blog entitled Google is Really About Large Amounts of Data notes that in an interview at the Web 2.0 Summit in October, Marissa Mayer, Google's VP of Search Products and User Experience, "confessed that having access to large amounts of data is in many instances more important than creating great algorithms."

Simon Rogers gathers some of the key figures driving the political agenda in the United States. Download the data or access the API to build your own charts and visualizations.

In a nod toward privacy, Yahoo today said it would only keep personal data on searchers and portal users for 90 days (double that in cases of fraud or suspicious activity). This ups the ante for search firms Google, which halved its data retention time to nine months in September, and Microsoft, which has said it would drop its data retention times to six months if its competitors did.

As Gary mentioned in his earlier post, I track myself - 40 things about my body, mind, and activity - every day. The fact that I do this tracking seems to interest people. Whether they are driven by curiosity about the phenomenon of personal data collection, or by the desire for a yardstick by which to measure and compare themselves, the fascination exists. To address this interest, and by way of introducing myself as a hopefully regular guest blogger at The Quantified Self, I have put together a FAQ about my personal tracking. Read on and you'll probably know more about me than you ever wanted to know.

Het College Bescherming Persoonsgegevens is een onderzoek gestart om te toetsen hoe de RET omgaat met de afhandeling van privégegevens.

EagleFiler in a nutshell: 1. Main WindowWith a single keypress, import Web pages, mail messages, or any file on your Mac. 2. EagleFiler stores them in an open format: regular files and folders that are fully accessible to your other applications. 3. Optionally: encrypt the files, add tags, notes, color-coded labels, and other metadata. 4. Browse by folder or by tag, or use the live search to find the information you need (faster than Spotlight). 5. View, edit, or create documents directly in EagleFiler’s streamlined interface, or double-click to edit using another application.

Door de blogs op www.ambtenaar20.nl, de discussies die daaruit volgden en de vele ontmoetingen die ik daarna heb gehad, ben ik er bewust van geworden dat rondom Open Overheid er flink wat initiatieven zijn. Ook blijkt dat het onderwerp soms verwarring oproept. Want wat is de relatie met begrippen als open source of open standaarden? En gaat het niet gewoon om openbaarheid? En waarom gaat het alleen over de overheid? Tijd voor een toelichting op het begrip Open Overheid.

I’ve learned that the World Bank now offers an API for several of its data sets on development, governance, and business conditions, plus a collection of photos.

Open Data Definition allows you to copy your data from one social network to another, keep track of your friends network and synchronise your data across services. Simple to use and trivial to implement, this is your fastest route to true data portability.

GovTrack.us is a tool for you to keep tabs on the U.S. Congress. Use our feeds or research pending and past legislation. GovTrack is a community & open source project.

SurveyMonkey has a single purpose: to enable anyone to create professional online surveys quickly and easily.

With Glom you can design database systems - the database and the user interface. The design is loosely based on FileMaker Pro, but with a separate database server. Its simple framework should be enough to implement most database applications.

Straightforward backup advice

Ajatus is a revolutionary CRM that runs as a local Ajax web application on your own computer.

Het CBS heeft veel gegevens op regionaal niveau. Die zijn al lange tijd beschikbaar via de statistische database Statline. Cartografische systemen geven de mogelijkheid de regionale gegevens ook op een meer aantrekkelijke manier te presenteren. Daarvoor h

Standardized Data Portability is the next great frontier for the web. As users, our identity, photos, videos and other forms of personal data should be discoverable by, and shared between our chosen tools or vendors. We need a DHCP for Identity. A distrib