May 19th 2012 | | The Economist | Anonymous

the growth of internet usage on smartphones, the rise of “big data” computer processing and the increasing willingness of customers to do complicated things online. These developments have long promised to transform the way banks do business and organise themselves....If this was just a more convenient way of paying, the banks would probably shrug. But it also promises to overturn your existing financial relationships. Instead of reaching for the first card that happens to be in your wallet to pay for a $2 cup of coffee (and risk being charged a $35 penalty by your bank for exceeding your overdraft limit), your phone will choose the best method of payment.

This issue of LIMN focuses on new social media, data mining and surveillance, crowdsourcing, cloud computing, big data, and Internet revolutions. Our contributors address issues from the power and politics of statistics and algorithms to crowdsourcing’s discontents to the capriciousness of collectives in an election; from the focus group and the casino to the worlds of micro-finance and data-intensive policing.

Personalized recommendation is the new marketing. Nick Seaver explains how ‘collaborative filtering’ de- fines people through their purchases.

From business intelligence to financial performance and strategy management to analytics applications, Cognos software can provide what your organization needs to become top-performing and analytics-driven. With products for the individual, workgroup, department, midsize business and large enterprise, Cognos software is designed to help everyone in your organization make the decisions that achieve better business outcomes—for now and in the future.

"Predicting X from Twitter is a popular fad within the Twitter research subculture. It seems both appealing and relatively easy. Among such kind of studies, electoral prediction is maybe the most attractive, and at this moment there is a growing body of literature on such a topic. This is not only an interesting research problem but, above all, it is extremely difficult. However, most of the authors seem to be more interested in claiming positive results than in providing sound and reproducible methods. It is also especially worrisome that many recent papers seem to only acknowledge those studies supporting the idea of Twitter predicting elections, instead of conducting a balanced literature review showing both sides of the matter. After reading many of such papers I have decided to write such a survey myself. Hence, in this paper, every study relevant to the matter of electoral prediction using social media is commented. From this review it can be concluded that the predictive power of Twitter regarding elections has been greatly exaggerated, and that hard research problems still lie ahead."

However, according to a report published last year by McKinsey, there is a problem. "A significant constraint on realizing value from Big Data will be a shortage of talent, particularly of people with deep expertise in statistics and machine learning, and the managers and analysts who know how to operate companies by using insights from Big Data," the report said. "We project a need for 1.5 million additional managers and analysts in the United States who can ask the right questions and consume the results of the analysis of Big Data effectively." What the industry needs is a new type of person: the data scientist.

"Inferential summaries of tree estimates are useful in the setting of evolutionary biology, where phylogenetic trees have been built from DNA data since the 1960's. In bioinformatics, psychometrics and data mining, hierarchical clustering techniques output the same mathematical objects, and practitioners have similar questions about the stability and `generalizability' of these summaries. This paper provides an implementation of the geometric distance between trees developed by Billera, Holmes and Vogtmann (2001) [BHV] equally applicable to phylogenetic trees and hieirarchical clustering trees, and shows some of the applications in statistical inference for which this distance can be useful. In particular, since BHV have shown that the space of trees is negatively curved (a CAT(0) space), a natural representation of a collection of trees is a tree. We compare this representation to the Euclidean approximations of treespace made available through Multidimensional Scaling of the matrix of distances between trees. We also provide applications of the distances between trees to hierarchical clustering trees constructed from microarrays. Our method gives a new way of evaluating the influence both of certain columns (positions, variables or genes) and of certain rows (whether species, observations or arrays)."

"Searching for relevant content in a massive amount of multimedia information is facilitated by accurately annotating each image, video, or song with a large number of relevant semantic keywords, or tags. We introduce game-powered machine learning, an integrated approach to annotating multimedia content that combines the effectiveness of human computation, through online games, with the scalability of machine learning. We investigate this framework for labeling music. First, a socially-oriented music annotation game called Herd It collects reliable music annotations based on the “wisdom of the crowds.” Second, these annotated examples are used to train a supervised machine learning system. Third, the machine learning system actively directs the annotation games to collect new data that will most benefit future model iterations. Once trained, the system can automatically annotate a corpus of music much larger than what could be labeled using human computation alone. Automatically annotated songs can be retrieved based on their semantic relevance to text-based queries (e.g., “funky jazz with saxophone,” “spooky electronica,” etc.). Based on the results presented in this paper, we find that actively coupling annotation games with machine learning provides a reliable and scalable approach to making searchable massive amounts of multimedia data."

--- This is more than a bit of a stunt, but it points in an interesting direction.

In his first television interview since he resigned from the National Security Agency over its domestic surveillance program, William Binney discusses the NSA’s massive power to spy on Americans and why the FBI raided his home after he became a whistleblower. Binney was a key source for investigative journalist James Bamford’s recent exposé in Wired Magazine about how the NSA is quietly building the largest spy center in the country in Bluffdale, Utah. The Utah spy center will contain near-bottomless databases to store all forms of communication collected by the agency, including private emails, cell phone calls, Google searches and other personal data. Binney served in the NSA for over 30 years, including a time as technical director of the NSA’s World Geopolitical and Military Analysis Reporting Group. Since retiring from the NSA in 2001, he has warned that the NSA’s data-mining program has become so vast that it could "create an Orwellian state."
democracy now, 20.04.2012

monthly account activity report

By Larry Selden and Geoffrey Colvin
September 30, 2002

Get ready for a big idea that's about to sweep through most companies: managing the enterprise not as a collection of products and services, not as a group of territories, but as a portfolio of customers. Of course, managers have always known that some customers are more profitable than others. But it's amazing how many executives, like those of that big retailer, haven't the least idea just how profitable (or unprofitable) individual customers or customer segments are.

Brewery is a Python framework for data analysis and data quality measurement. The framework uses streams of structured data that flow between processing nodes.

The framework consists of several modules:

metadata – field types and field type operations, describe structure of data (available directly from the brewery package namespace)
ds – structured data streams data sources and data targets
streams – data processing streams
nodes – analytical and processing stream nodes (see Node Reference)
probes – analytical and quality data probes