In Fluid Concepts and Creative Analogies, Douglas Hofstadter (and a crew of talented students) argue that analogy-making is a core characteristic of human intelligence. The book is full of delightful puzzles. One class of puzzle goes like this:

What is the Ob of Nebraska? (The Platte. Hence the name for these: Ob-Platte puzzles)

What is the Newark of Delaware?

What is the Gettysburg of Hawaii?

The authors write:

If one were to ask, “What is the Atlantic City of France?”, a large number of people would undoubtedly answer “Monaco”, even knowing full well that Monaco is not a city inside France, but rather an independent, albeit small, country that is not in France, but borders it (the gambling city within Monaco is Monte Carlo). However, Monaco is world-famous for its casinos, is small enough to be thought of as a city, and like Atlantic City, is located on the sea.

I’m working on an Ob-Platte puzzle that you may be able to me with. It goes like this:

Who is the Andy Morikawa of Ann Arbor?

I’m working on the same puzzle for Berkeley, Boston, Providence, Portland, Seattle, Toronto, and a few other places where I’d like to convene a seminar on community information management. The first of these was held in Blacksburg, Virgina, during my recent visit there, and was successful because Andy Morikawa knew and invited all the right people. The Andy Morikawa of Blacksburg is a fellow at Virginia Tech’s Institute for Policy and Governance, and was until recently the executive director of the Community Foundation of the New River Valley.

Just as Monaco is not actually a city in France, the Andy Morikawa of Ann Arbor, Berkeley, and Providence might or might not be affiliated with the universities in those towns. Another Andy might be an independent community organizer, or a city councilor, or an economic development specialist, or the director of a nonprofit chartered to consolidate and promote cultural arts. He or she will certainly, though, be a well-known connector who has the ability to convene the kinds of folks I want to reach. They’ll come from the public schools, the colleges, the city government, the newspapers and online media, the music and dance scenes, the public library, the hospital, the chamber of commerce, the downtown merchants association, arts and culture nonprofits, social services organizations, sports leagues, environmental groups, and other centers of public life.

Crucially I am not looking for webmasters and IT staffers, but rather for leaders responsible for public engagement. The principles I’ll explain and demonstrate are ones that everyone can understand and apply. Once leaders understand what’s possible they can delegate to staff, but what they’ll delegate is nothing they couldn’t pretty easily do for themselves.

What makes this a tricky problem in social networking is that my best contacts are in the technical world, but I’m trying to reach outside it. That’s why I need an Andy to help me bridge the gap. If you are an Andy, or if you know an Andy, I’d really like to hear from you.

A few thoughts from a weekend visit to Manhattan…

Metropolitan Museum: Gertrude Stein exhibit (see it); new Islamic art wing (nice, but the non-expert may have trouble understanding why these particular objects are different than those decorative items one might see in a wealthy Turkish home, for example); Byzantium and Islam show (skip; any connections between Byzantine/Christian art and Islamic art are tough to see); Naked before the Camera (skip; photographs from the Met’s collection in which people happen to appear naked); Fu Baoshi (see it; the career of a Chinese artist develops during the Communist revolution).

MOMA: Diego River murals (see it, especially for the sketches from a 1927 visit to Moscow (Rivera was a devout Marxist)); Cindy Sherman exhibit (sad because the late 1970s small scale black and white untitled film stills were the best work; the more recent pictures taken with super high-res cameras and blessed with unlimited printing and framing budgets were hard to understand).

Seminar: Funny script, brilliant acting (especially from Zoe Lister-Jones).  Alan Rickman has been replaced by Jeff Goldblum, but in some ways he seems better for the role due to the need for him to be attractive to a couple of young women. The ending is a little weak/confusing.

I noticed, apparently rather late, that the Triborough Bridge has been renamed the “RFK Bridge”. It made me kind of sad that the U.S. is building so little new stuff that we have to find something built in the 1930s and rename it. It also saddened me that the current crop of politicians is so uninspiring that we have to keep reaching back to the 1960s and naming more stuff after the Kennedys. (My companion in the taxi, not a U.S. citizen, pointed out that it would have been more sensible to rename the Dike Bridge on Chappaquiddick after Ted Kennedy.)

I had a nerdy conversation on what might be the next mainstream framework for building web products, and in particular whether the node.js community would ultimately create this framework, or if node.js will just be a fad. This blog post is a bit of a deviation from my usual focus around marketing, so just ignore if you have no interest in the area.

Here’s the summary:

Programming languages/frameworks are like marketplaces – they have network effects
Rails, PHP, and Visual Basic were all successful because they made it easy to build form-based applications
Form-based apps are a popular/dominant design pattern
The web is moving to products with real-time updates, but building real-time apps hard
Node.js could become a popular framework by making it dead simple to create modern, real-time form-based apps
Node.js will be niche if it continues to emphasize Javascript purity or high-scalability

The longer argument below:

Large communities of novice/intermediate programmers are important
One of the biggest technology decisions for building a new product is the choice of development language and framework. Right now for web products, the most popular choice is Ruby on Rails – it’s used to build some of the most popular websites in the world, including Github, Scribd, Groupon, and Basecamp.

Programming languages are like marketplaces – you need a large functional community of people both demanding and contributing code, documentation, libraries, consulting dollars, and more. It’s critical that these marketplaces have scale – it needs to appeal to the large ecosystem of novices, freelancers and consultants that constitute the vast majority of programmers in the world. It turns out, just because a small # of Stanford-trained Silicon Valley expert engineers use something doesn’t guarantee success.

Before Rails, the most popular language for the web was PHP, which had a similar value proposition – it was easy to build websites really fast, and it was used by a large group of novice/intermediate programmers as well. This includes a 19-yo Mark Zuckerberg to build the initial version of Facebook. Although PHP gained the reputation of churning out spaghetti code, the ability for people to start by writing HTML and then start adding application logic all in one file made it extremely convenient for development.

And even before Rails and PHP, it was Visual Basic that engaged this same development community. It appealed to novice programmers who could quickly set up an application by dragging-and-dropping controls, write application logic with BASIC, etc.

I think there’s a unifying pattern that explains much of the success of these three frameworks.

The power of form-based applications
The biggest “killer app” for all of these languages is how easy it is to build the most common application that mainstream novice-to-intermediate programmers are paid to build: Basic form-based applications.

These kinds of apps let you do a some basic variation of:

Give the user a form for data-entry
Store this content in a database
Edit, view, and delete entries from this database

It turns out that this describes a very high % of useful applications, particularly in business contexts including addressbooks, medical records, event-management, but also consumer applications like blogs, photo-sharing, Q&A, etc. Because of the importance of products in this format, it’s no surprise one of Visual Basic’s strongest value props was a visual form building tool.

Similarly, what drove a lot of the buzz behind Rails’s initial was a screencast below:

How to build a blog engine in 15 min with Rails (presented in 2005)

Even if you haven’t done any programming, it’s worthwhile to watch the above video to get a sense for how magical it is to get a basic form-based application up and running in Rails. You can get the basics up super quickly. The biggest advantages in using Rails are the built-in data validation and how easy it is to create usable forms that create/update/delete entries in a database.

Different languages/frameworks have different advantages – but easy form-based apps are key
The point is, every new language/framework that gets buzz has some kind of advantage over others- but sometimes these advantages are esoteric and sometimes they tap into a huge market of developers who are all trying to solve the same problem. In my opinion, if a new language primarily helps solve scalability problems, but is inferior in most other respects, then it will fail to attract a mainstream audience. This is because most products don’t have to deal with scalability issues, though there’s no end to programmers who pick technologies dedicated to scale just in case! But much more often than not, it’s all just aspirational.

Contrast this to a language lets you develop on iOS and reach its huge audience – no matter how horrible it is, people will flock to it.

Thus, my big prediction is:

The next dominant web framework will be the one that allows you to build form-based apps that are better and easier than Rails

Let’s compare this idea with one of the most recent frameworks/languages that has gotten a ton of buzz is node.js. I’ve been reading a bit about it but haven’t used it much – so let me caveat everything in the second half with my post with that. Anyway, based on what I’ve seen there’s a bunch of different value props ascribed to its use:

Build server-side applications with Javascript, so you don’t need two languages in the backend and frontend
High-performance/scalability
Allows for easier event-driven applications

A lot of the demo applications that are built seem to revolve around chat, which is easy to build in node but harder to build in Rails. Ultimately though, in its current form, there’s a lot missing from what would be required for node.js to hit the same level of popularity as Rails, PHP, or Visual Basic for that. I’d argue that the first thing that the node.js community has to do is to drive towards a framework that makes modern form-based applications dead simple to build.

What would make a framework based on node.js more mainstream?
Right now, modern webapps like Quora, Asana, Google Docs, Facebook, Twitter, and others are setting the bar high for sites that can reflect changes in data across multiple users in real-time. However, building a site like this in Rails is extremely cumbersome in many ways that the node.js community may be able to solve more fundamentally.

That’s why I’d love to see a “Build a blog engine in 15 minutes with node.js” that proves that node could become the best way to build modern form-based applications in the future. In order to do this, I think you’d have to show:

Baseline functionality around scaffolding that makes it as easy as Rails
Real-time updates for comment counts, title changes, etc that automatically show across any viewers of the blog
Collaborative editing of a single blog post
Dead simple implementation of a real-time feed driving the site’s homepage

All of the above features are super annoying to implement in Rails, yet could be easy to do in node. It would be a huge improvement.

Until then, I think people will still continue to mostly build in Rails with a large contingent going to iOS – the latter not due to the superiority of the development platform, but rather because that’s what is needed to access iOS users.

UPDATE: I just saw Meteor on Hacker News which looks promising. Very cool.

Dean Baker dismisses them

In fact, measured productivity numbers are unlikely to pick up the full gains that may be associated with lower populations. Large populations and crowding put enormous stress on the environment. Imagine having commuting times cut in half, if smaller populations eliminated rush-hour congestion. This would not be picked up in productivity measures.

Similarly, increased access to desirable locations, such as lower prices for waterfront property, would not be picked up by conventional measures of productivity. And, of course, the reduced pollution, including lower levels of greenhouse gas emissions, would also not be picked up in standard measures of productivity.

. . .

In short, there is no demographic problem facing wealthy countries. The only problem is that people with poor math skills and imperialistic designs hold positions of influence and power.

The problem of flat population growth is more serious than Dean imagines.

Folks concentrate on the problems associated with the Welfare States but ironically that is simply because the Welfare State has better accounting practices.

The core problem is that folks want to transfer good and services into the future. This is a fundamentally difficult thing to do.

However, one trick is to construct buildings and then to rent those buildings out to future generations. This is what the vast majority of the capital stock in America is devoted to.

However, in a world where population is declining this trick will no longer work.

What that means is that the real interest rate that establishes full employment can easily go negative in times of stress. This confuses policy makers who are so ingrained that savings ought to pay, because that’s how we get more wealth.

In truth savings can just as easily be a socially costly endeavor as a socially profitable one. There is nothing written into the laws of the universe that say it should be one way or the other.

Nor will productivity growth alone do it. Even in the best of times the productivity growth ‘spread” is small and can be overwhelmed by risk spreads, as no one actually knows ahead of time what investments will be complimentary to productivity growth.

The surer thing is per capita depreciation, since its much more likely that in the future people will continue to at least have some affinity for the things they wanted in the past. However, when the per capita part goes away all you have left is the depreciation part and for a huge portion of the capital stock, that rate is quite low.

Filed under: Uncategorized

A speed limit makes sense because driving too fast or too slow puts other drivers on the road at risk, thus the decision how fast to drive can create an externality. But how should policy makers set the right speed limit? Engineers can weigh the costs of higher speeds (more accidents) against the benefits (getting places faster), and determine the optimal level. But in reality they are set at discrete levels that don’t vary nearly as much as the optimal speed on various lengths of road would appear to vary. Furthermore the optimal speed clearly depends on the preferences of the drivers, the current weather, and other factors that shift by hour of the day.

Variable speed limits, in contrast, present a more flexible, even Hayekian, way of setting the speed limit. One example is Interestate 80 in Wyoming, where sensors detect driver speeds, which are then used in an algorithm, along with weather conditions and other factors, to set speed limits that vary. An interesting article, via Radley Balko, provides more information on this road:

Drivers’ speeds are tracked by sensors embedded in the pavement and installed on markers alongside the highway.

However, that’s just one element the Wyoming Department of Transportation uses to calculate and set variable speed limits.

Other factors include weather, road condition and recommendations from Wyoming Highway Patrol troopers and department maintenance operators

According to Wikipedia, variable speed limits date back to at least 1965, with a road between Munich and Salzberg able to have a speed limit of 60, 80, or 100 km/h. These speeds were set by individuals who monitored traffic speeds, but today computers can do this automatically. This seems like the kind of law/technology that should would be more widespread, but speeding tickets mean government revenues. And at the local level, research shows that towns that are undergoing fiscal problems are more likely to issue traffic tickets (yes, traffic tickets are countercyclical). In addition, I think people suffer from a general fear of allowing safety to be at the mercy of algorithms.

But if it is true that the variance of traffic speed matters more than the average in determining the probability of an accident, then it would seem sensible to let speed limits vary with drivers perceptions of what is optimal, making adjustments for the externality of driving too fast. After all, a speed limit to far below the “natural level” probably creates more variance.

Filed under: Uncategorized

I certainly try not to be.

I think at some level every single one of us is prejudiced.  There are things we all innately fear or are uncomfortable with.  I don’t know if its nature or nurture, but I don’t believe any of us on this earth is so pure that we absolutely accept everyone as they are.

I do however feel that all of us can learn to accept everyone.  I have come to realize, intellectually,  that I personally don’t give a shit about your sex life, your  spiritual life, your personal life, whatever. I have no problem with you being you.  I accept who you are.

I also don’t care who you tell about it or if you flaunt it.  I’m happy to take the responsibility to be your friend, your acquaintance or  if I don’t like you, what you do or how you do it for whatever reason, to completely  ignore you.

That doesn’t mean that I’m always completely sensitive to everyone  I engage with.   I have my own sense of humor. Things  make me laugh that may or may not make you laugh. I’m the first to admit that sometimes my humor runs to the sophomoric. Like my high school buddy Todd, who I still refer to by his high school nickname, Boafy (don’t even remember how he got that name) says, every guy laughs at a fart joke. Not every guy does. But I do. And I’m good with that. I hope I’m 95 and still laughing at dumbass jokes.

But sometimes what I laugh at isn’t appreciated by others. It may even offend them if it comes out at the wrong time.

So why bring this up ?

This past week I did an interview at a sports conference. It was a 1 on 1 sit down.  During the conversation the topic of the Kisscam at sporting events came up . The interviewer mentioned it and commented  ”I like the Kiss Cam,”  In response to the interviewer comment,  I said “That’s because you and your boyfriend are always on it,”

Totally sophomoric.  I quickly realized that the comment wasn’t appropriate, so I added “”Or his girlfriend, this is gender-independent commentary,”

I made a mistake in making the comment. I wasn’t trying to be hurtful. It wasn’t a comment on anyone’s sexuality. It was just me trying to be funny. It wasn’t. I quickly realized it and tried to fix it. I hoped at the time I didn’t offend anyone.

This blog post is not about trying  to defend what I said. I’m not trying to defend my sense of humor. I’m not trying to convince you I’m not a homophobe. I’m not trying to justify anything at all.

I guess what I am doing is admitting that at some level I am prejudiced and that I recognize that I am .  There are a lot of things in my life that I need to improve at. This is one of them. Sometimes I make stupid throw away comments that I quickly realize are wrong. It doesn’t happen often, but it happens. It was a mistake and I realized it. I learned from it.

I’m the last to be politically correct and the last thing I am trying to be here is politically correct. I honestly don’t give a shit what you think about me. But I think being the person I want to be includes not blurting out throw away jokes about sexuality, race, ethnicity, size,  disability or other things people  have no say in about themselves.  I’m the guy who still feels bad about punching Michael Cooper in the stomach in  6th grade purely because he was overweight, even though I made the point to apologize to him when I ran into him at a reunion years later.

 Even if I don’t care about you, it doesn’t mean I’m ok with making you uncomfortable or upset with a comment that references anything  that is out of your control. That is not the person I want to be.

I’m happy to pick on you if you root for the wrong team. I’m happy to pick on you if you like doing The Wave. I’m happy to pick on you for a lot of reasons. Your sexuality should never be one of those reasons.

I like who I am. I love my life. But that doesn’t mean I won’t always try to be a better version of me

And yes, I feel better having written this blog post

It’s been a while since I hung up my spurs as a columnist, and lately I’ve been missing the opportunity to write regularly for a venue other than this blog. So when Mike Barton asked me to contribute to Wired’s Cloudline I said yes. I’m calling the column The Personal Cloud and it’ll run most Fridays. Last week was the introduction, this week I explore what Phil Windley and his crew are doing to help realize Doc Searls’ vision of a Live Web.

The URL for the series is http://www.wired.com/cloudline/tag/the-personal-cloud, and the corresponding RSS feed is http://www.wired.com/cloudline/tag/the-personal-cloud/feed.

RT @albertoconti: stephen wolfram the personal analytics of my life

A week ago today – Randosteve showing Kershaw Finlay (X-Reed’s son) how to properly grip a ski pole. Steve was so happy! Steve and Chris we love you! – Randokitty and X-Reed

(This article was first published on RDataMining, and kindly contributed to R-bloggers)

By Yanchang zhao, RDataMining.com

Building prediction/classification models is one of the most widely-seen data mining tasks in business applications. To share experience on building prediction models with R, I have started a discussion at RDataMining group on LinkedIn with the following questions. And my experience can be found at the end of question list. Pls join our discussion if you are interested.

1. What is your application about?
2. What techniques did you use? E.g., linear/logistic/non-linear regression, decision tree, random forest, neural networks, SVM, k-NN classification.
3. Which tool/function/package did you use? E.g., package rpart or party in R.
4. Why did you choose the above technique/tool/package?
5. Was your data a mixture of numerical and categorical attributes? If yes, what did you do to preprocess data before feeding them into the above techniques/functions?
6. Was your data of imbalanced classes? If yes, what did you do to achieve good prediction results?
7. Did your data have many missing values or extreme values? If yes, what did you do with them?
8. Was your table very wide, i.e., having many attributes/variables? If yes, how did you do variable/feature selection or dimensionality reduction before building predictive models?

Below are my experiences in a business application.

It was an application to model risk of customers. There were two classes, good and bad, labelled as 0 and 1. A model was to build to predict classes of new cases.

I used decision tree, because the tree is easy to understand by business people and managers, and the rules are simple and easily to be accepted by business, as compared to SVM or neural networks. We finally built a tree with less than 30 leaf nodes, that is, less than 30 rules. Although random forest can make a similar model, but it would end up with too many rules. To sum up, decision trees are easy to understand, have good performance, accommodate both categorical and numerical data and also missing values, and produce simple models.

I used ctree() in package party. The reason for this choice is not technical at all. I first tried rpart, but when I plotted rpart tree, I got a tree without any labels, and the tree does not look like a tree at all. Then I tried ctree(), and fell in love with its plot, a nice looking tree with all labels I needed. I believe both packages produce similar results, and my choice of ctreeI() is simply personal preference.

The reason I didn’t use package randomForest is that it cannot handle missing values, and in our application, we didn’t believe filling those missing values would produce good result, because many missing values were introduced when joining tables (e.g., there were no records for a customer in some tables). However, it may work in other applications. I did try cforest() in package party, it produced similar result as ctree(), so I finally chose ctree() because it produces much simpler trees than a random forest. When performance is similar, the simpler, the better.

The data was a mixture of numerical and categorical attributes, and ctree() handles that very well.

There were many missing values. We didn’t fill them. However, for attributes with too many missing values, e.g., if above 60% are missing, we simply excluded the attribute from modelling. SAS EM uses a similar strategy when importing data into EM, and I think the default threshold for missing values are 20% or 30% in EM.

The data were large and also composed of over 300 variables from dozens of tables. We didn’t do any feature selection first, but found that it took too long to train a model, especially with some categorical variables having many value levels. An option is to use a small sample to train models. I took a different way to use as many training data as possible. To make it work, I draw 20 random samples of training data, and built 20 decision trees, one tree for each sample. There are around 20-30 variables in each tree, and many trees share similar set of variables. Then I collected all variables appearing in those trees, and got around 60 variables. After that, I used all original training data for training without any sampling, but with the above 60 variables only. That’s my way to select features in that application, where all training cases were used to build a final model, but with only those attributes having appeared in the 20 trees on sampled data.

More information on R and data mining is available at:
RDataMining: http://www.rdatamining.com
Twitter: http://www.twitter.com/RDataMining
Group on Linkedin: http://group.rdatamining.com
Group on Google: http://group2.rdatamining.com



To leave a comment for the author, please follow the link and comment on his blog: RDataMining.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series,ecdf, trading) and more...

Once you’ve been working in Silicon Valley for a bit, you’re often offered advice such as:

Are you launching at X conference?  … where X is whatever hot conference is coming up, like SXSW or Launch
Do you have an X app? … where X is whatever new platform just emerged, be it Open Social, iPhone, or whatever
Have you pitched X venture capitalist? … where X is a prolific headline-grabbing investor with a recent hot deal
You should do feature X that company Y does! … where X is some sexy (but possibly superficial feature) that a hot startup has done
Do you know what your X metric is? … where X is some metric a recent blog post was written about
Have you met X? … where X is some highly connected expert in the field
Maybe you should pivot into X space! … where X is a space with a hot company that just raised a ton of funding
Did you think about applying framework X to this? … where X is a new framework, be it gamification or viral loops or Lean

Sound familiar? I confess that I’ve both received and given much advice along the lines of the above. I call it “advice autopilot.”

The perils of “advice autopilot”
Advice autopilot is when you’re too lazy to think originally about a problem, instead regurgitate whatever smart thing you read on Quora or Hacker News. If you’re a bit more connected, instead you might parrot back what’s being spoken at during Silicon Valley events and boardrooms, yet the activity is still the same – everyone gets the same advice, regardless of situation. The problem is, the best advice rarely comes in this kind of format – instead, the advice will start out with “it depends…” and takes into account an infinite array of contextual and situational things that aren’t obvious. However, we are all lazy and so instead we go on autopilot, and do, read, say, and build, all the same things.

That’s not to say that sometimes generic advice isn’t good advice – sometimes it is, especially for noob teams who are working off an incomplete set of knowledge. Often you may not have the answers, but the questions can lead to interesting conversations. You may not be able to say “you should do an iPhone app” but it’s definitely useful to ask, “how does mobile fit into this?” This can help a lot.

The other manifestation of this advice autopilot is the dreaded use of “pattern matching” to recommend solutions and actions.

Pattern-matching in a world of low probability, exceptional outcomes
One of Silicon Valley’s biggest contradictions is the love of two diametrically opposed things:

The use of pattern-recognition to predict the future…
… and the obsession with a small number of exceptional successes.

Exceptional outcomes for startups are limited – let’s say it’s really only 5-10 companies per year. In this group, you’d include companies like Facebook and Google that have “made it” and hit $100B valuations. On the emerging side, this would include startups who might ultimately have a shot at this, like Dropbox, Square, Airbnb, Twitter, etc. This is an extraordinarily small set of companies, and it isn’t much data.

The problem is, we’re hairless apes that like to recognize patterns, even in random noise. So as a result, we make little rules for ourselves – Entrepreneurs who are Harvard dropouts are good, but dropping out of Stanford grad school is even better. It’s good if they start a company in their 20s unless they’re Jeff Bezos. Being an alum of Google is good, but being an alum of Paypal is even better. Hardcore engineers as founders is good, but the list of exceptions is long: Airbnb, Pinterest, Zynga, Fab, and many others. And whatever you do, don’t fund husband-wife teams, unless they start VMWare or Cisco, in which case forget that piece of advice.

As anyone who’s taken a little statistics knows, when you have a small dataset and lots of variables, you can’t predict shit. And yet we try!

The intense focus on a small set of companies also introduces a well-known logical fallacy called Survivorship Bias. Here’s the Wikipedia page, it’s interesting reading. Basically, the idea is that we draw our pattern-recognition from well-publicized successful companies while ignoring the negative data from companies that might have done many of the same things, but end up with unpublicized failures. We’re all so intimately familiar with stories like “two PhDs from Stanford start Google” that we ignore all the cases where two PhDs from Stanford try to start a company and fail. Or similarly, YCombinator has built a great rep on companies like Airbnb and Dropbox, and yet you’d think that if you invest in 600+ startups that you’d get a few hits. Because of factors like this, it might seem as though A predicts B when in fact, it does nothing of the sort – we’re just not taking the entire dataset into account.

Conformity leads to average outcomes when we seek exceptional outcomes
The problem with giving and taking so much of the same advice is that ultimately it breeds conformity, which is another way if saying it reduces the variance in the outcome. And if you conform enough, you end up creating the average outcome:

The average outcome for entrepreneurs is, your startup fails.

Lets not forget that. And so one part of Naval Ravikant’s talk on fundable startups that resonated with me is the idea of playing to your extremes. He says in the talk:

“Investors are trying to find the exceptional outcomes, so they are looking for something exceptional about the company. Instead of trying to do everything well (traction, team, product, social proof, pitch, etc), do one thing exceptional. As a startup you have to be exceptional in at least one regard.” -Naval Ravikant @naval

Be extremely good at something, and invest in it disproportionately relative to your competition – this gives you the opportunity to actually create an extreme outcome. Otherwise, the average outcome doesn’t seem so good.

The flipside of innovation
The funny thing with all of this, of course, is that this is what innovation looks like. The remarkable ability for practical knowledge to disseminate amongst the Bay Area tech community is what makes it so strong. Before something becomes autopilot advice for a wide variety of people, often a small number of hard-working teams who know what they’re doing leverage it to great success. Follow those people, and you might find yourself successful – just like them.

So the billion dollar question is – how do you separate out trendy/junk advice from what really matters?

… well, it depends!

(This article was first published on Research tips » R, and kindly contributed to R-bloggers)

For those who have not read the seminal works of Tufte and Cleveland, please hang your heads in shame. To salvage some sense of self-worth, you can then head over to Solomon Messing’s blog where he is starting a series on data visualization based on the principles developed by Tufte and Cleveland (with R examples).

The classics are also worth reading, and remain relevant despite the 20 or 30 years that have elapsed since they appeared.

To leave a comment for the author, please follow the link and comment on his blog: Research tips » R.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series,ecdf, trading) and more...

Today I’m excited to be able to announce the launch of Wolfram|Alpha Pro—the biggest single step in the development of Wolfram|Alpha since its original introduction.

Over the two and a half years since we first launched, Wolfram|Alpha has been growing rapidly in content and capabilities. But today’s introduction of Wolfram|Alpha Pro in effect adds a whole new model for interacting with Wolfram|Alpha—and brings all sorts of fundamentally new and remarkable capabilities.

Starting today, everyone has access to Wolfram|Alpha Pro at wolframalpha.com. Unlike the “tourist” version of Wolfram|Alpha, though, you have to log in, and, yes, to get full capabilities there’s a subscription ($4.99/month, or $2.99/month for students). (Right now, you can try it for free with a trial subscription.)

So, what does Wolfram|Alpha Pro do?

There are some big things here. But at the level of the Wolfram|Alpha interface, they’re just summarized by little icons.

Let’s talk first about output. Once you’ve logged in, you have access to your history, and you can define favorites. You can also set preferences, like what location Wolfram|Alpha should assume, or what unit system you want to use. And you can do things like change the overall size of Wolfram|Alpha output.

As soon as you mouse over a Wolfram|Alpha output pod, you’ll immediately see:

(or, actually, clicking almost anywhere in the pod) does something very simple, but useful: it gives you an enlarged version of the pod, so you can for example see all the details of elaborate plots.

does something a lot of people have asked for: lets you customize output from Wolfram|Alpha, and get it in various formats—so you can put it directly into your presentation, or whatever.

Another much-requested capability, accessed with , is being able to download the raw data behind a Wolfram|Alpha output—say as a spreadsheet or the like.

(Needless to say, spreadsheets can’t faithfully represent the full breadth of data, units, etc. that Wolfram|Alpha generates, so Wolfram|Alpha Pro uses tricks like having separate sheets for “Raw Data” and “Formatted Data”.)

When one says “downloading data”, one might think just of data behind tables and plots. But Wolfram|Alpha Pro can download all sorts of other data too: 3D geometry data (say to use for a modeling program or a 3D printer), sound data, graph connectivity data, molecular specification data, etc.—in altogether more than 60 formats.

In addition to handling material in individual pods, Wolfram|Alpha also lets you download a complete output page as PDF—or CDF.

CDF (Computable Document Format) is the format that we introduced last year to let people create documents containing computations. It’s already gaining a lot of momentum in areas like textbooks and interactive reports. But now CDF is also part of Wolfram|Alpha Pro.

In all sorts of output pods, there’ll be a button labeled “Enable interactivity”. Click it and the pod will turn into CDF, that you can immediately interact with.

At a basic level, you’ll be able to resize any graphic, and rotate 3D graphics. But many kinds of graphics and other outputs will also sprout controls that let you directly modify and interact with them. (Often there’s a “More controls” section that opens out to give lots of additional controls.) And because CDF computation is done locally on your computer, the interaction is typically very zippy.

An interesting feature of CDF in Wolfram|Alpha Pro is that it effectively lets you create interactive programs directly from free-form linguistic input. You can tell Wolfram|Alpha to animate with respect to some variable, or somesuch, and it’ll generate a CDF that does that.

So there are all sorts of new things associated with output in Wolfram|Alpha Pro. But what about input?

Right below the main input box there’s a row of icons. Each of them brings out a “tray” for some special kind of input.

gives a special character keyboard, modeled after the soft keyboards that exist in Wolfram|Alpha mobile apps.

The other icons all relate to a big idea of Wolfram|Alpha Pro. With ordinary Wolfram|Alpha and its free-form linguistics, we’ve really opened up the kind of textual input that you expect a computer to be able to handle. But a big idea of Wolfram|Alpha is to go still further, and to allow input that isn’t text at all.

lets you give an image as input.

Once you’ve got the image in, it’ll be indicated by a little yellow box in the Wolfram|Alpha input field. And if you just hit Enter, Wolfram|Alpha Pro will do an automatic analysis of your image.

There’s some general analysis that always gets done, but a lot of the analysis depends on your image. If there’s text in the image, it’ll get OCR’d. If there are separate components, they’ll be identified. And so on.

But in addition to purely automatic analysis, you can tell Wolfram|Alpha Pro what to do with your image, just using standard free-form linguistics. In a sense, Wolfram|Alpha Pro is a direct beneficiary of the very powerful image handling capabilities that were added in recent versions of Mathematica. But the end result is that it’s able to do a very large range of image processing and image analysis—both “Photoshop-style”, and of a type usually seen only in specialized, expensive, image processing systems.

Particularly powerful is combining image upload with CDF—and getting interactive interfaces for image processing.

So what about other kinds of files? Well, Wolfram|Alpha Pro can handle about 60 types.

In each case, it can do general automatic analysis of what’s in the file. And you can specifically tell it what you want to do. For different types of files, the results are very different. Like here’s the result of uploading a sound file:

And here’s a general analysis of a pure binary file:

What about files that contain data? Here’s where it gets even more exciting. And actually the data doesn’t need to be laid out in a spreadsheet or CSV or whatever. lets you just copy a block of data from anywhere, and feed it to Wolfram|Alpha Pro.

To many people who’ve seen preliminary versions of Wolfram|Alpha Pro, this is then the part that’s most surprising and remarkable: Wolfram|Alpha Pro will automatically analyze the data, and generate a report about it.

The report is completely tailored to the particular data you give—and it can look very different for different kinds of data. Usually, though, it’ll contain some mixture of visualizations and analyses. It’ll have all kinds of charts and graphs and tables—often together with explicit conclusions generated by statistical and other methods.

And of course, it’s not just a static report. There are always all sorts of buttons and pull-downs that allow you to drill down, select different options, and so on. But the notion is that when you upload your data to Wolfram|Alpha Pro, it’ll immediately be able to tell you interesting things about it.

I’ll write more about this elsewhere, but in a sense the concept is to imagine what a good data scientist would do if confronted with your data, then just immediately and automatically do that—and show you the results.

We’re certainly not finished with everything that’s possible, but already in the version of Wolfram|Alpha Pro that we’re releasing today, I think what we can do with data is pretty impressive. Of course, it helps that we can build on all the sophisticated data and statistics-related capabilities that are now built in to Mathematica. And it also helps that we can make use of all the other parts of Wolfram|Alpha.

For example, if you read in data with dates, or units, or place names, or whatever, Wolfram|Alpha Pro is able to call on Wolfram|Alpha’s linguistic capabilities to understand whatever forms were entered. And when it comes to output, Wolfram|Alpha Pro can freely use the built-in knowledge in Wolfram|Alpha. So, for example, it can immediately place on a map cities or countries or whatever given in the data. But what is more, it can use its built-in knowledge to let you do things like automatically normalizing by population.

As everywhere in Wolfram|Alpha, we’re aiming for very broad and deep coverage. We want to implement every method and algorithm that’s relevant to analyzing data, and then we want to apply these automatically whenever and wherever they make sense. Already we’ve got lots of data handling and visualization, lots of standard and not-so-standard statistical methods, and lots of new methods, many original to Wolfram|Alpha Pro.

Taken with the other capabilities of Wolfram|Alpha Pro, it’s all a pretty major extension of ordinary Wolfram|Alpha—supporting a whole new model of using Wolfram|Alpha. In a sense the new capabilities emphasize more than ever the computational nature of Wolfram|Alpha: the ability to do complete, fresh, computations for every query.

We’ve been able to go a remarkably long way with the basic paradigm of ordinary Wolfram|Alpha. But now Wolfram|Alpha Pro dramatically extends this paradigm—and it’s going to be exciting to see all the new things that become conceivable. But for now, I hope that as many people as possible will use Wolfram|Alpha Pro, and will take advantage of the largest single step in the development of Wolfram|Alpha since it was first launched.

Andrew Gelman just posted an interesting article on the philosophy of Bayesian statistics. Here’s my favorite passage.

This reminds me of a standard question that Don Rubin … asks in virtually any situation: “What would you do if you had all the data?” For me, that “what would you do” question is one of the universal solvents of statistics.

Emphasis added.

I had not heard Don Rubin’s question before, but I think I’ll be asking it often. It reminds me of Alice’s famous dialog with the Cheshire Cat:

“Would you tell me, please, which way I ought to go from here?”

“That depends a good deal on where you want to get to,” said the Cat.

“I don’t much care where–” said Alice.

“Then it doesn’t matter which way you go,” said the Cat.

Related post: Irrelevant uncertainty

The other day Tim Bray tweeted a Google+ item entitled Geeks and repetitive tasks along with the comment: “Geeks win, eventually.”

Here’s the chart posted on Google+ by Bruno Oliveira:

A couple of things bothered me about this. First, there’s the adversarial tone. The subtext is a favorite geek quotation:

First they ignore you, then they laugh at you, then they fight you, then you win.

That gem is often attributed to Gandhi. Wikiquote disputes that and finds a close variant in Nicholas Klein’s 1918 speech to the Amalgamated Clothing Workers of America. Either way it’s the story of a persistent underdog overcoming an oppressor.

In geek ideology the oppressors are pointy-haired bosses and clueless users. Geeks believe (correctly) that clueless users can’t imagine, never mind implement, automated improvements to repetitive manual chores. The chart divides the world into geeks and non-geeks, and it portrays software-assisted process improvement as a contest that geeks eventually win. This Manichean worldview is unhelpful.

But the chart also fails to capture the reality of repetition and automation in the realm of information systems. Here’s an alternative world view that I choose to imagine and strive to create:

In this view of the world, tasks that involve data manipulation (as so many modern chores do) are undertaken by teams. There is an infinite supply of manual chores. Everybody tackles them. Ideally there is one member of the team I call the toolsmith. Working shoulder to shoulder with the team, the toolsmith spots an opportunity to automate some piece of the work, writes some code, deploys it, observes how it gets used (or doesn’t get used), assesses its impact (or lack of impact), and then iterates on the code. Meanwhile the toolsmith keeps working alongside the team, chipping away at the never-ending and always-evolving list of manual chores, looking for more opportunities to automate, and exploiting them in an incremental and collaborative way.

Software-assisted automation of repetitive work isn’t an event, it’s a process. And if you see it as a contest with winners and losers you are, in my view, doing it wrong.

News reports of how badly Japan was doing never quite squared with my experience as a visitor to that country. A recent New York Times editorial attempts to explain the apparent discrepancy.

[The article is interesting to read, but I'm not sure that I buy the entire argument. The Japanese government has indulged in some massively wasteful projects. Still, they probably can't compare in profligacy to the U.S. federal and state governments (for example, retirement age for public employees, including police officers, in Japan is 60, compared to as young as 41 or 42 here in the U.S.).]

Will Wilkinson has a must-read post on libertarianism, why he’s more of a liberal than a libertarian, and how he’d rather argue more with liberals than libertarians. He also condemns Ron Paul about as well as anyone could in a paragraph of his usual rhetorical genius:

Somebody’s going to ask “Isn’t Ron Paul making a difference?” So I’m going to say, “Yes.” None of this is to say that right-fusionism of the Ron Paul variety isn’t now having an influence, or that none of it is good. I’m glad to see Paul spreading a few profoundly important ideas about foreign policy. But that doesn’t mean Paul’s decades of bilking paranoid bigots with bullshit prophesies of hyperinflationary race war was really a stroke of strategic genius after all. Or maybe it means it was. But that doesn’t make it right. I don’t think Paul would be where he is today without all those years of vile fear-mongering. And I don’t think anyone ought to get away with climbing up that evil ladder, kicking it away, then pretending he was born a thousand feet off the ground in the pure mountain air right there next to heaven. He knew what he was doing, chose to do it, and none of it can be justified by a little TV-time for salutary anti-imperialist and free-market ideas. I’d rather not be affiliated with a “movement” that includes him in even a conflicted way.

I am not quite persuaded by Will’s rejection of the term libertarian to describe himself. Or rather I am not persuaded by his rejection of the term to describe ourselves, since I share a lot of the beliefs he says differentiate him from libertarians. Will appeals to the prevailing public understanding of “libertarianism”, but I think that this is more about the relative importance of a high level of economic freedom to both freedom overall and general welfare than about which rights and liberties are “off the table”.

Maybe I am biased because I don’t want to surrender libertarianism to those I see as the radicals among us. But then again maybe Will is biased because it’s easier to persuade liberals when you call yourself a liberal.

Filed under: Uncategorized

As I build out the elmcity network, launching calendar hubs in towns and cities around the country, I’ve been gathering examples of excellent web thinking. In Ann Arbor’s public schools are thinking like the web I noted that the schools in that town — and most particularly the Slauson Middle School — are Doing It Right with respect to online calendars. How, I wondered, does that happen? How does a middle school figure out a solution that eludes most universities, theaters, city governments, nightclubs, museums, and other organizations with calendars of interest to the public?

It’s not technology. Slauson Middle School is using the same web services (in this case, Google Calendar) available to everyone.

It’s not budget. The web services required for this solution are free.

It’s a way of thinking. I wrote to Slauson’s principal, Chris Curtis, to congratulate him on the excellent example his school is setting, and to identify the thinker responsible. That thinker turns out to be Chris Curtis himself. And it’s no accident that the implementation pattern on display at Slauson is also evident at Pioneer High. Chris did the same thing there before coming to Slauson.

Now I am not an educator, I only watch from the sidelines. But to me the K-12 “computer skills” curriculum seems uniformed by the kinds of core principles that will make students effective in a web-augmented world. So I asked Chris:

What you’ve done at Pioneer and now Slauson builds on an important conceptual foundation. Do you think that K-12 education could build that foundation?

Here’s what he said:

I agree with the notion that the basic principles of computer science should be generalized more broadly across the curriculum. In many ways, teaching computer and technology skills courses absent a meaningful application of them is ineffective and silly. We wouldn’t teach driver’s education and not let students drive. We don’t teach a “pencil skills class” in which we learn the skills for using this technology tool without an immediate opportunity to apply the skills and then begin to consider and explore the many ways that the pencil and writing change how we organize, perceive, and interact with our world.

This issue gets at the heart of the challenge of technology and education. Often the world seems to divide into separate interest areas: those interested in technology and those interested in education. The result is often to send the technology nerds to a room and make them teach technology and send the other teachers to their rooms and let them teach. In order to be effective at integrating technology into the instructional environment there has to be a merger between a technology interest and and educational interest, within the same person. The awareness of what is possible via technology resources and the desire to perform educational functions can lead to the educator realizing that a task could be done differently, more efficiently, more effectively, with more precision, or in some other manner improved.

Of course the schism that separates technologists from educators also affects practitioners of all kinds. In his most recent essay, Bret Victor meditates on this point:

My piano teacher played the piano. Like, all the time. He had to; it’s not easy to make a living as a musician. Between tours, his band played restaurants, bars, weddings, anywhere they could get a gig. He chose this life because he loved music, and when he taught music, he was teaching what he did. In that way, his teaching was honest.

…

Back in high school, I was taught differential equations by a working engineer. He spent his days at Lawrence Livermore Laboratory, and for whatever reason, chose to spend his evenings at the local community college. Differential equations wasn’t some abstract arcana to him. It was his bread-and-butter, and he apparently found it important enough to share.

…

My information theory professor would teach me information theory in the morning, and then spend the afternoon furthering the field. Sure, what she taught was somewhat elementary by her standards, but she was well aware that this elementary theory was the foundation on which her life’s research was built. It showed, and it stayed with me.

…

Real teaching is not about transferring “the material”, as if knowledge were some sort of mass-produced commodity that ships from Amazon. Real teaching is about conveying a way of thinking. How can a teacher convey a way of thinking when he doesn’t genuinely think that way?

I’m preoccupied with a related question. The way of thinking that I most want to convey is web thinking. Which is, by definition, openly available to anyone who wants to learn. Schools everywhere can observe and emulate what Chris Curtis is doing at Slauson. In so doing they can become practitioners in the way that Chris is. Their students might then see them as practitioners and learn from their examples.

I would be delighted if the elmcity project could help bootstrap that virtuous cycle.

The most common sin of economists is to take a point which is true in part and mistake it for one which is true in full. We so enjoy being contrarian and refuting sacrosanct beliefs with the clean and frictionless logic of economics that it doesn’t quite satisfy to point out that these beliefs are simply less true than people think, but must insist that they are fully false. Case in point is this Steven Landsburg post, which Robin Hanson praises, in which he argues that misers are just as admirable as philanthropists:

In this whole world, there is nobody more generous than the miser—the man who could deplete the world’s resources but chooses not to. The only difference between miserliness and philanthropy is that the philanthropist serves a favored few while the miser spreads his largess far and wide.

If you build a house and refuse to buy a house, the rest of the world is one house richer. If you earn a dollar and refuse to spend a dollar, the rest of the world is one dollar richer—because you produced a dollar’s worth of goods and didn’t consume them.

Who exactly gets those goods? That depends on how you save. Put a dollar in the bank and you’ll bid down the interest rate by just enough so someone somewhere can afford an extra dollar’s worth of vacation or home improvement. Put a dollar in your mattress and (by effectively reducing the money supply) you’ll drive down prices by just enough so someone somewhere can have an extra dollar’s worth of coffee with his dinner….

Landsburg’s point would be completely valid and convincing had he simply argued that the miser is more like a philanthropist than is commonly believed, but to fully satisfy the economist itch he must argue that the miser not morally distinct from the philanthropist. What’s missing from Landsburg’s analysis is the common sense fact that the world is full of areas where the social gains from a dollar spent are much larger than a dollar. Whether it is giving the global poor mosquito nets, education, vaccines, other medical care, or just cash, it is clear that it is easy to spend the money in ways that generate huge social gains. Likewise it is not hard to identify areas of basic research with large spillovers that are underinvested in. Of course Landsburg knows there are many ways to spend money and generate much higher social returns than burning money or putting it in the bank, which is why his article in Slate advising people how best to donate to charity didn’t simply advise them to burn it.

In reply to Landburg, Karl argues:

This is because the miser is withholding his assessment of the most utility maximizing uses of his money and that assessment is a valuable thing. Even if the miser knows very little he knows something and as always ignorance is not abdication.

I agree, but I would go a step further than this: the miser is not simply withholding his assessment, but he is signaling that he does not care that the money could generate massive welfare gains, and so he does not care about massive welfare gains. This is because it is simply not credible for a miser to argue that he cannot identify areas where there are large social benefits, or use the money to hire people to identify areas with large social benefits, therefore the only option that remains is sociopathic indifference.

In reply to Karl, Robin argues that he’d “still guess that the miser does more good than the average rich-nation philanthropist”. Surely there are many philanthropists who are terrible at philanthropy. Someone who spends millions creating a public museum filled with Damien Hirst sculptures is generating lower social returns than if he were to miserly put the money in a bank or burn it. But this simply reinforces my original point: Landsburg could’ve made a truer and more persuasive argument had he simply gone with a more modest version of it. That is, he should have said misers are better than some philanthropists. The problem is that most people are already pretty capable of identifying low value philanthropy and begrudging it as a waste of money. If you explained to them that putting the money in the bank or burning it is similar to giving it away to society at large, they would agree that some this is better than some philanthropy. But the contrarian victory there would not be enough to satisfy an economist, and so the argument is oversold.

Filed under: Uncategorized

After a few years of using Android, I received an iPhone 4S (Verizon network) to use for a consulting project. I’m experimenting with using it as my primary phone.

One of the things that I have liked best about Android is how easy it is to set up a new phone by typing in one’s Gmail account name and password (just once). This automatically syncs calendar, contacts, email, etc.

For a Gmail user such as myself, the process of setting up the iPhone is not simple and involves Googling for a variety of tutorials. The first step is to set up the phone with one’s Apple ID. In addition to entering the username and password on the phone, I had to respond to an email message from Apple asking me to verify my Apple ID and password. Thus did I have to type in Apple ID and password twice.

Next step was trying to use Gmail and sync contacts and calendar. I downloaded the Gmail app from the App Store and typed in my Gmail account name and password. This populated the Gmail app with mail but did not sync calendar or contacts. So I went to the Settings -> Mail control panel and selected “Gmail” as an account to add. At the cost of typing in my name, Gmail username, and password, this synced mail, calendars, and notes, but not contacts. The Google help page on the subject says that to sync Gmail contacts one tells the iPhone that one is using Microsoft Exchange and then follows a series of instructions with server names to type in.

Back in 2008, I started picking out “Favorites” on a series of Android phones. Google puts these into a “Starred in Android” group. You would hope that these would be automatically mapped into the “Favorites” contact group on a new iPhone, but they aren’t. The Favorites group remains empty even after all of one’s contacts have been sync’d.

By the end of the set up process I think that I’d typed in my Apple ID and password at least twice and my Gmail username and password three or more times. Typing passwords on the virtual keyboard was much more difficult than on the Droid 2 physical keyboard.

One good thing about Android phones is that they don’t further clutter my house with cables. I keep micro-USB and mini-USB cables permanently attached to my desktop computer and use them to charge a variety of devices from different manufacturers. Neither cable works with the iPhone, which requires a custom cable.

Siri works better than expected as far as speech-to-text is concerned, but has a way to go in common sense reasoning. For example, I asked “Will the weather be nice on Thursday?” Siri correctly recognized the words but said “no”. The forecast was for sunny skies and 50 degree temperatures. That’s pretty nice for December 1 as far as most folks in Massachusets are concerned, but Siri apparently differs.

Question: What do folks like to use for turn-by-turn navigation (voice prompts) on the iPhone? I haven’t yet figured out how to get that out of the built-in Maps application.

[Oh yes, how does it work as a phone? So far, not so great in an area of weak coverage that I call "my house". With the Droid 2 I could make and receive calls inside the house, though the sound quality was better stepping outside. With the iPhone, the phone will ring but the call drops after a minute or two unless I walk outside. This could become an issue in January...]

Paul Krugman has some words for job creators and other high skilled people: we don’t need you.

…textbook economics says that in a competitive economy, the contribution any individual (or for that matter any factor of production) makes to the economy at the margin is what that individual earns — period. What a worker contributes to GDP with an additional hour of work is that worker’s hourly wage, whether that hourly wage is $6 or $60,000 an hour. This in turn means that the effect on everyone else’s income if a worker chooses to work one hour less is precisely zero. If a hedge fund manager gets $60,000 an hour, and he works one hour less, he reduces GDP by $60,000 — but he also reduces his pay by $60,000, so the net effect on other peoples’ incomes is zip.

First let me just say that the extent to which what people earn is equal to their marginal product is greatly unappreciated, so before I disagree with Krugman I just want to pause and point out that this is truer than most people think. But it’s not true everywhere and always. Note that Krugman does not go so far to say that marginal product *does* in fact equal income, but that “textbook economics says that in a competitive economy…”, and that job creator praise is not “something that comes out of the free-market economic principles these people claim to believe in”, and that “Even if you believe that the top 1% or better yet the top 0.1% are actually earning the money they make…”.

He doesn’t actually say “people earn what they make”, nor does he say how good of an approximation to reality marginal product theory is. But in his economics textbook with Robin Wells, they are a bit more explicit, and do call the marginal product theory a pretty good approximation:

The main conclusion you should raw from this discussion is that marginal productivity theory is not a perfect description of how factor incomes are determined, but that it works pretty well. The deviations are important. But, by and large, in a modern economy with well-functioning labor markets, factors of production are paid the equilibrium value of the marginal product -the value of the marginal product of the last unit employed in the market as a whole.

This is a really important point, and I don’t disagree. But I think that many high skilled, high paid workers, and job creators in the U.S. can be an important deviation from this general rule. Many of these people work at moving the productivity frontier forward, and thus increasing the marginal productivity of other workers. After all, one of the important things that entrepreneurs do is find ways to increase the productivity of other workers so they can underprice their competitors. The process of creative destruction is not manna from heaven. I won’t pretend to know have all the answers about what drives this process, but entrepreneurs, job creators, and high skilled people are an important part of it.

Consider, for instance, that if we suddenly kicked out the top 10% of high IQ people (or 10% most productive people, or 10% most creative people, or whatever) in the U.S.. It strikes me as fairly likely that the total output of the remaining 90% would go down. Krugman seems to argue that this would not be the case. But even if you disagree with me in the short run, in the long-run the productivity increasing innovations these people would have made won’t show up, and the rest of us would have lower productivity as a result.

Now, instead of kicking out the top 10% of workers, just make them work less as a result of high income taxes.  See my concern?

Lowered incentives of job creators and other innovators should be considered as one of the likely downsides to higher taxes.

Note that if you don’t think this is true, then what business do we have subsidizing higher education? If workers capture the entirety of their higher productivity, then I don’t see who gains by giving young people money to go to college rather than just cash.

Filed under: Uncategorized

These are not meant to be researched items. These are “streams of consciousness” from the conversation yesterday’s post created.

First some housekeeping. I DO NOT like paying more in taxes. HOWEVER, I think that this country has created unique opportunities for entrepreneurs and paying taxes is a small price to pay.  Taxes are not a bad thing.  What bothers me are not the taxes I pay to help others and to support the services our country needs. What bothers me is the  mis-allocation and inefficient distribution of our tax money. Particularly when it leads to taking more money from those who can not afford it, and in this economy, even those making 250k per year can not afford it.

Our Congress, BOTH parties, has progressively lost the moral hazard of doing what is right for the country rather than doing what is right for the party and politician. Why wouldn’t a politician  go for the gold in their political career ? Why wouldn’t a party profess that their way is the only way  ? What is the financial or career downside of doing so ? None.  It may not serve the country very well, but careers and fortunes will be made . Our politicians are no better than the financial whores who helped get us into this mess. They put personal gain over the people they are elected to serve.

So what can be done ? Here you go:

1. Transparency.

It had been often promised  and never delivered.  If there was transparency in our budgets and the actual spending of our dollars, down to the nickel we as citizens would have much more insight and leverage in the budget process. As the saying goes, “Sunlight is a great disinfectant”.   US citizens (with the exception of classified defense spending) should be able to see it how our money is being spent in real-time.    The value of transparency is that we would benefit from the collective brain power of the American people.  I’m usually not a fan of crowdsourcing, but when it comes to managing how our nation’s money is spent, I think it could be a very powerful enhancement to the process.The power of the people at its best.

With complete transparency we could have hundreds of volunteer deficit reduction Super Committees to look for the best places to cut costs and improve efficiency. Without it, we are at the mercy of a “Super Committee” formed purely to make politicians through compromise and political expediency. That is not how problems are solved. That is how they are passed on to future generations.

2. 10 Year Budgets

There is no better example of how politicians lie to themselves and the American people than the fact that our budgets are framed within a 10 year plan.  There is no business person on the planet who would think that a 10 year plan would have even a remote possibility of playing out as planned.  Yet that is what we use to try to convince our country that we are “taking action ” to cure our problems. Hell, even communist countries have plans that are 5 years long.  Any budget plan that is longer than the end of the current POTUS term is basically a crock of shit. You can’t plan what you can’t control.  Any effort to do so is an out and out lie . It’s probably the only truly bi-partisan program that is unanimously agreed upon., and it is a lie.

3. Taxes Vs  Job Creation

There is an ongoing refrain from some that any increase in Taxes will have a negative impact on investment and job creation. Not true in 99.99pct of cases. Never has been. Never will be.  First the .01% times where it may be true. Potentially, a person could have some amount of money less than what they need to start a company because they paid say 1k dollars more in taxes this year than they did last year.  This could happen and I’m sure it has happened. but its the exception that proves the rule.

Now the rule…

People driven to succeed are driven to succeed. People driven by money are driven by money. People driven to compete, compete.  We live in a country that puts an emphasis on achievement. Not just financial achievement. The ability to set goals and achieve them. We celebrate and reward those that accomplish their goals. It is part of the very fabric of what makes this country so amazingly unique.

Those of us who are driven by money have a number that we strive for. People like me. (If you want to learn more about people like me, read this).  We want to be a millionaire. Once we become a millionaire, some of us want more. Some of us don’t. But once you hit the first number you begin to make decisions in your life about how you might get to the next number or just use what you have to make your life (and possibly the life of others) better.

Others set goals and define success and achievement in any number of ways. In no cases do any of them examine the tax rate. In fact, I would be willing to bet that 99pct of us completely ignore the tax rate. Why ? Because we know that the rewards we all value the most came as the result of our efforts. Something that no tax rate is going to take away from us.

 The risk of starting a business . The risk of making an investment in the sweat equity of someone else’s efforts. The risk of starting a charity. The risk of taking a new job. The risk of adding a new employee, etc, etc, etc.  I have NEVER met a motivated person  who has said they would not chase their goals  because of tax rates.

Personal achievement is not the only motivating factor that over-rides taxation.  Business to business competition ALWAYS over-rides taxation. If you own or run a business you have to best your competitors.  As long as they pay under the same tax structure as your business, it’s all about who can do a better job. Not what the tax rate is.

Of course none of this is going to stop big companies from arguing that higher taxes impacts job creation. Of course they are going to argue it. The less they pay in taxes, the higher their earnings per share and the greater the value of their stock and options.  If a big company needs employees to stay competitive in their industry(s) you better believe they are going to hire that person no matter what it takes. They will find the money some how. Even if it means lowering their political contributions and lobbying costs or bringing in cash held overseas.

This is a country that competes to win. That is not going to change.

In fact, follow this logic. Its counter intuitive, but its absolutely true.  The higher the tax rate on income the more risks us money chasers have to take in order to hit our number.  If you want that number, you are going to go for it. Period, end of story. More risk, more companies started, more people hired.

3a. Let Me Be Clear

I am not advocating that we raise taxes for everyone. I think that is a huge mistake.  I do think billionaires should pay more. We have benefited the most financially from this great country, and it is the right thing to do to give more back in a time of need.  I believe those of  us who have achieved windfalls in the stockmarket should pay more as well.  My tax rate back in 2000 was far greater than today, and I had no problem with it. My tax rate when I sold my first company in 1990 was even higher. I had no problem with it.  Nor should any entrepreneur or investor who makes the big score. As I said in my last blog post, it’s a great problem to have.

Do I realize that much  of the incremental tax money I send to the Treasury is going to be wasted ? Yes. Do I realize that after all the bureaucracy and overhead associated with running our government and the programs it creates that probably less than 50pct of tax money reaches the programs that the money is intended to support ? Yes. There is no question we are throwing good money after bad. There is no question that something needs to be done and I believe transparency will help solve this problem.

HOWEVER,  if money is going to be wasted by our government, it is better that Mark Cuban, Warren Buffet’s and other mega rich people’s money be wasted than YOUR money be wasted.  Agree ?

My point here is to say that the argument that higher taxes are a disincentive is very wrong.  The argument we should be making against taxes is that the government does a very, very poor job of effectively distributing our tax dollars where they are needed. Lets stick to reality rather than trying to make dogmatic proclamations that are incorrect.

4.  Tear Down Foreclosed Houses.

There is no question that the housing market has a huge impact on the economy.  There is also no question that the housing market is one of supply and demand. There is also no question that the government owns hundreds of thousands of foreclosed homes and growing. Every day those homes cost money to maintain and service. It’s expensive and they hold down housing prices. The solution ? Tear them down.

Tearing down foreclosed homes by the hundreds of thousands will be the ultimate infrastructure project. Thousands of jobs bulldozing and clearing homes. Reduction in inventories. Reduction in overhead to service the vacant homes. The quicker you take the homes off the market, the sooner the market for new and used homes will recover and prices will go up. I had been planning on blogging about this several months ago, but Time.com beat me too it . http://www.time.com/time/magazine/article/0,9171,2090368,00.html .

Bottom line is that best way to build up the housing market is to tear down every foreclosed home owned by government agencies. It will not only energize the housing market, but the process of tearing down the homes will create jobs for less educated/skilled labor.

5. Spending Money on Infrastructure/Infrastructure Bank

Speaking of infrastructure. I was watching a talking head show where an “expert” commented that the Chinese proved out the value of spending on infrastructure. It got me thinking about whether or not spending on Infrastructure or funding an Infrastructure B[…]

I’ve tried various strategies and pieces of software for personal organization and haven’t been happy with most of them. I’ll briefly describe my criteria and what I’ve found.

My needs are fairly simple. I don’t need or want something that could scale to running a multinational corporation.

I’d like something with a portable, transparent data format. I don’t want the data stored in a hidden file or in a proprietary format. I’d like to be able to read the data without the software that was used to write it.

I’d like to be as structured or unstructured as I choose and not have to conform to a rigid database schema. I’d like to be able to do ad hoc queries as well as strongly typed queries.

I’d like something that exports to paper easily.

Here’s what I found: org-mode. It’s an Emacs mode for editing text files. It provides sophisticated functionality, but all the sophistication is in the software, not the data format. It’s more convenient to work with org-mode files in Emacs, but the raw file format is just a light-weight mark-down, easy for a person or a computer to parse.

When I went back to using Emacs a year ago after a 15-year hiatus, I heard good things about org-mode but didn’t understand what people liked about it. I heard it described as a to-do list manager and was not impressed. I’m not interested in the features I was first introduced to: tracking the status of to-do items and making agendas. I still don’t use those features. It took me a while to realize that org-mode was what I had been looking for. It was similar in spirit to something I’d thought about writing.

Emacs is an acquired taste. But someone who doesn’t use Emacs could get some good ideas from looking at org-mode. I imagine some people have borrowed its ideas and implemented them for other editors. If not, someone should.

The org-mode site has links to numerous introductions and tutorials. I like the FLOSS Weekly interview with org-mode’s creator Carsten Dominik. In it he explains his motivation for writing org-mode and gives a high-level overview of its features.

Related posts:

Giving Emacs another try
Forced to be simple
Not for everyone
Software that gets used

The latest New Yorker magazine contains “The Grand Tour”, an article by Evan Osnos, who joins a Chinese bus tour to see Europe through the eyes of Guide Li:

“if [the bus driver gets caught working more than 12 hours/day, audited by an automated system], the fine starts at eighty-eight hundred euros, and they take away your license! That’s the way Europe is. On the surface, it appears to rely on everyone’s self-discipline, but behind it all there are strict laws.”

“We have to get used to the fact that Europeans sometimes move slowly,” he said. When shopping in China, he went on, “we’re accustomed to three of us putting our items on the counter at the same time, and then the old lady gives change to three people without making a mistake. Europeans don’t do that.” He continued, “I’m not saying that they’re stupid. If they were, they wouldn’t have developed all this technology, which requires very subtle calculations. They just deal with math in a different way. Let them do things their way, because if we’re rushing then they’ll feel rushed, and that will put them in a bad mood, and then we’ll think that they’re discriminating against us, which is not necessarily the case.”

Li made a great show of acting out a Mediterranean life style: “Wake up slowly, brush teeth, make a cup of espresso, take in the aroma.” The crowd laughed. “With a pace like that, how can their economies keep growing? It’s impossible.” He added, “In this world, only when you have diligent, hardworking people will the nation’s economy grow.”

“The European economy is in decline,” he said bluntly. “Times have changed.”

“Can a place where workers go on strike every day grow economically? Certainly not,” he said.

[Handy, a fellow tourist,] was a sanitation specialist by training, and he couldn’t help but notice Milan’s abundant graffiti and overstuffed trash bins. As Li had explained it, “The government wants to clean, but it doesn’t have enough money.” Handy tried to be polite, but he said, “If it was like this in Shanghai, old folks would be calling us all afternoon to complain.”

He pointed out the window to the highway and said that it had taken decades for Italy to build it, because of local opposition. “If this were China, it would be done in six months! And that’s the only way to keep the economy growing.”

He mentioned a Western friend who had quit his job to go backpacking and find his calling in life. “Would our parents accept that? Of course not! They’d point a finger and say, ‘You’re a waste!’ ” he said. But, in Europe, “young people are allowed to pursue what they want to pursue.”

More: Read the article.

I was on the phone last night with a friend I’ll call “Susie”. She talked about how angry she was with her husband for not doing enough of the work associated with their two children. “I probably do 85 percent of the work,” Susie noted, “and in any case there is nothing that Bill could do now to make up for a year of lost sleep. I’m still angry about things that happened 10 years ago.” Did she imagine that other American women were getting more help? “No, all of my friends are in the same situation.” If her husband was no worse than others, why then was she angry? “If you were born into slavery and everyone you knew was a slave, it would still make sense to be angry about the situation. Maybe every woman being angry is the first step towards justice.”

It then occurred to me that perhaps Americans with kids would be happier if all couples got a divorce when the kids were about 10 years old and did not need too much hands-on child care. If a woman desired “the society of a more or less coarse-minded person of another sex” [Isabel Archer in Henry James's Portrait of a Lady] she could remarry a man who’d wronged some other woman by not doing his fair share. Every married woman would still be stuck with a slacker, but it wouldn’t be the same slacker who prevented her from enjoying a full night’s sleep when her children were young.

At a service stop on the Merritt Parkway over the weekend, I was approached by a young couple in a jam. They were halfway to their destination, had pulled in for gas, then realized neither had brought a wallet. They were both on their phones, working the problem, and the guy looked up to ask if I’d heard of a roadside assistance program that could help in that situation. I wound up giving them ten bucks. Maybe it was a scam, in which case I only lost $10. But maybe it wasn’t, in which case I helped some folks in need.

Ten bucks wasn’t enough to get them as far as they said they needed to go, though. And later I got to thinking about how we might have created enough trust, in an ad-hoc way, for me to make a short-term loan of, say, $50. It’s an interesting thought experiment. I wonder what solutions you can imagine? Here are a few that occurred to me.

Web identity. Given a web connection, I could have searched for the couple’s names, found their web footprints, and verified that their photographs, locations, and other attributes matched what they claimed.

Six degrees of separation. If we could trace our connection through social network space, that might be enough. It might even be possible to do that with voice calls, but with a web connection it could be almost trivial.

PayPal. Given a web connection, we could have brought up a browser and done a PayPal transaction. In that case I wouldn’t even be making a loan, I’d know that the funds had been transferred before handing over cash.

Losing my wallet while traveling is a nightmare scenario for me. It’s never happened but I dread the thought. I hate being so dependent on documents that I carry around in a wallet that could easily be lost or stolen.

Those documents embody claims made on my behalf by identity providers that we have all agreed to trust. That arrangement became necessary when society grew beyond what interpersonal trust could scale out to support. And it will remain necessary. But as voice and data connectivity become ubiquitous, and as interpersonal trust scales out in ways it never could before, I wonder if we’ll see a re-emergence of pre-bureaucratic modes of identity.

In some disciplines, there is the notion of a Comment on a published article, which is what it sounds like: a short comment about the contents of the article (for example, that it’s wrong). Cat Dynamics links to an interesting account of physicist Rick Trebino’s (lengthy but ultimately successful) attempts to publish a Comment explaining why a published article is wrong.

I don’t think I’ve ever seen a Comment in a pure math journal. They’re common in statistics journals.

One of the readers with whom I shared coffee this evening in Orlando had what looked like a stack of papers. One of the papers turned out to be a notebook computer with an aluminum case. It was marvelously light and very sleek. What was it? A Dell V13, which is buried among hundreds of other products on the Dell Web site. It sells for $450 with Linux and 2 GB of RAM, a 250 GB hard drive, and a built-in Webcam. Deficits: no optical drive (impossible at this weight and thickness); no Thinkpad-style pointer nub in the middle of the keyboard so you’re stuck with a trackpad (the owner said he would have preferred the keyboard nub  but liked the machine anyway due to its “disposable price” so that he doesn’t have to worry about losing it or dropping it).

I’m wondering who designed this thing because it doesn’t look like anything that Dell has ever sold before.

[Update: I ordered a Lenovo ThinkPad Edge after reading all of the comments heaping scorn on the Dell and praising the Edge's durability. Also, I have always loved the trackpoint in-keyboard nubby pointer thingy.]

Twenty “why” posts from this blog:

Why programmers are not paid in proportion to their productivity
Why 90% solutions beat 100% solutions
Why Unicode is subtle
Why programmers write unneeded code
Why care about spherical trig?
Why Shakespeare is hard to read
Why are bad guys so interesting in novels?
Why proof by pattern of examples doesn’t work
Why Ruby is a good language for DSLs
Why SQL failed
Why is TeX so popular in Germany?
Why microarray study conclusions are so often wrong
Why technical arguments get heated
Why there will always be programmers
Why so few electronic medical records
Why heights are normally distributed
Why heights are not normally distributed
Why computer scientists count from zero
Why functional programming hasn’t taken off
Why Mr. Scott is Scottish

Good news for cricket fans: YouTube is making the semifinal and final matches of this year’s Indian Premier League cricket tournament available live for all viewers, including those in the U.S. The IPL’s final four matches of the season, which begin on Wednesday and culminate with the championship match on April 25, will be streamed live on YouTube’s IPL channel at youtube.com/ipl.

As part of a multiyear agreement with the IPL that YouTube struck in January, the online video site has been streaming the entire season of IPL cricket matches live — for all markets except the U.S. If cricket fans here wanted to view the action live, they had to sign up for subscription-based online sports site Willow.tv and pay $60 to stream the games in real time.

Despite this, YouTube saw massive demand for on-demand viewing of the IPL matches in the U.S., as matches became available here 15 minutes after the live streams concluded. Even without the availability of live streams, the U.S. ranked second in the world (behind India) in terms of number of subscribers to the IPL channel, as well as rating, commenting on and favoriting videos. As a result of the enthusiasm by U.S. viewers, YouTube opened up the last four matches for viewing live.

The inaugural season of the IPL on YouTube — the video-sharing site’s first foray into streaming a sporting event live — appears to be a success. Over the first 55 matches of the IPL season, YouTube says it’s received more than 40 million views across the globe, and that’s before the beginning of the playoff matchups. With the availability of live streams in the U.S., we can expect plenty more viewers tuning in for the final four matchups. A schedule is available on the IPL web site here.

Related content on GigaOM Pro: Not Your Grandfather’s Streaming Video Business (subscription required)

It’s Saturday morning at the eBay/PayPal headquarters, and developers are busy preparing for the second day of iPad development at the first official iPadDevCamp (brought to us by the same people who previously organized three iPhoneDevCamp events). The coffee is brewed and the bagels are served. Teams were formed last night but there’s still a few people walking around comparing skill sets and looking for a team to join. Of course there are iPads everywhere being held up by business card holders and other random homemade contraptions.

Developers have flown in from Argentina, China, Germany, Sweden, English, and Canada. The goal is to meet new people on Friday night and by Sunday have an awesome app. A hackathon. The best apps receive various prizes from iPads to keyboards to cases.

In addition to being heads down for several days focused on this exciting new platform, there are talks about marketing your apps, integrating PayPal, placing advertisements, Objective-C lessons, and other presentations from well-known players in Silicon Valley.

Developers and Their Apps

I sat down with Dan Grover, the creator of ShoveBox, to discuss his latest project: Etude. It’s a sheet music app that looks beautiful on the iPad (although it’s only available for the iPhone right now). Imagine propping up your iPad on your piano and playing along with famous compositions. Below is an example piece of music.

I also discussed Audiotorium with Michael Emmons, a former Symbian developer who recently left that platform for iPhone OS. Audiotorium is both a recording and note-taking app that is perfect for college students and working professionals. Instead of carrying around a laptop that is arguably overkill for lectures and meetings, you use an iPad to make sure you capture everything.

Music Creation

Now this is cool. Rana Sobhany uses two iPads as a DJ setup. Her blog about the experience and its progress is called Destroy the Silence.

The Results

Here’s a list of many of the apps presented Sunday afternoon after a rough two nights. Somehow these magicians were able to produce functioning apps that appear to be ready for App Store submission. However, many are still in the development phase on Github, or are now open source for anyone to download and try.

Relay — This app will be truly amazing when completed. The demo received a huge applause. Users can drag websites, text, and music to and from the iPad and computer. Music seamlessly stops playing on one device and continues on the other. Websites you are currently reading instantly load on the other device. This app won the “Most Useful” award.

PAD – Personal Armour Defense — A mobile security system. Users set up wireless sensors (smoke, motion, etc.) in a hotel room, campsite, or home to ensure protection. The system can be armed or disarmed using RFID. PAD received the “Most Alarming” award.

iPad Slot Machine — Another huge applause. One person throws an iPhone as the slot machine’s lever, and three iPads show the spinning objects. iPad Slot Machine received the “Coolest App” award and is pictured below.

iuiPad – Extending the iUI web development framework to support the iPad. This won the “Best Web App” award.

Shopkeep – Mine your email to find online purchases and track packages.

Melena21 — Finally an app looking towards helping people with special needs. Children can touch large images to indicate what they need or want. This app won the “Accessibility App” award.

Airhawk — Air Hockey on the iPad. This app won the “Most Monetizable” award because of its in-app purchases and use of ads.

iPad Boombox — A full screen mp3 player that looks and behaves like an old school boombox. This won the “Retro” award.

Tank or Die — Use iPhones to control tanks on an iPad. This won the “Best Game” award.

iConessionStand — Users can order food and drinks at a sporting event right from their seats. This won the “Best Use of PayPal API” award.

The most common use of the word “argument” is to describe a disagreement.  So the first time you hear “argument” to mean something you pass into a function (either a mathematical function or a programming language function), it sounds odd. How did “argument” come to mean two very different things? Here is an explanation.

It is curious to track the path by which the word “argument” came to have two different meanings, one in mathematics and the other in everyday English. According to the Oxford English Dictionary, the word derives from the Latin for “to make clear, prove”; thus it came to mean, by one thread of derivation, “the evidence offered as proof”, which is to say, “the information offered”, which led to its meaning in Lisp. But in the other thread of derivation, it came to mean “to assert in a manner against which others may make counter assertions”, which led to the meaning of the word as a disputation.

Taken from An Introduction to Programming in Emacs Lisp.

Update: As Dave Richeson points out in the comments below, there are really three meanings of “argument” being discussed.

Union Square Ventures’ Fred Wilson will be giving a talk at the 99% conference later this week entitled “10 Ways To Be Your Own Boss.” Prior to the talk, Wilson has provided a preview of his presentation, which gives his perspective on the various categories of entrepreneur:

The Sole Proprietor: Matt Drudge
The Partnership: Openshop Studios
The Husband & Wife Team: Daily Lit
The Boutique: Union Square Ventures
The Federation: Allen & Co
The Project: Avatar
The Tour Bus: The Hype Machine
The Startup:Red Stamp
The Breakout: Foursquare
The Company: Twitter

Wilson is probably the most prominent technology investor in the world today. Though he’s led investments in some of the web’s greatest successes stories — including Boxee, Delicious, Feedburner, Foursquare, Meetup, Tumblr and Twitter — he’s perhaps more widely known for his blog, A VC, and the insights he shares on running technology startups.

He only hints at what he’s planning to talk about at the conference, but his slides reveal a useful perspective from an authoritative voice: the notion that web businesses can be successful and influential without using the venture-backed startup model.

The examples cited sit on a continuum, ranging from small, loosely bound collaborators to larger, more traditional business structures. For web workers seeking to fulfill their entrepreneurial ambition, this illustrates that there are many paths to success. Perhaps choosing the structure most appropriate is more important than the product or service being produced.

I’d be interested to hear from you whether these categories are suitable descriptions of your own circumstance. What kind of entrepreneur are you?

The speculation and anticipation over the iPad is now over, at least in the U.S., and we fickle mobile enthusiasts now turn our thoughts to what will come next. The past two years have been exciting in the mobile space, with smartphones gaining full computing power and netbooks taking the world by storm. Apple has shaken things up with the iPad, and slates are on the minds of many enthusiasts. Will the Next Big Thing (NBT) in the mobile space come from the phone sector, or the notebook segment or will it be a tablet of some sort? That’s not clear at this point, but I believe that the driver will not be technology, it will be usage driven.

What I mean by usage driven is the NBT is going to come from creating a new way to do things that people want to do. That may come from online activities, or handling daily tasks or a combination of those. There are already mobile devices that can do everything that people want, whether in a laptop form or with a phone. The trick is going to lie in showing a new way to do these things that blows everything else out of the water. That’s not going to be easy for companies to produce.

Smartphones of today are full computers that can do many of the things formerly relegated to bigger devices. The processors are fast and platforms have evolved to take advantage of that in the phone space. While phones are often able to do the things people want, the restrictions of the screen size and input mechanisms are not going away. While I agree with Stacey at GigaOM that multiple core processors in phones will make them incredibly powerful, the input/output limitations will prevent them from being able to use that power to its fullest potential. We will see the smartphone continue to evolve, but the NBT will not be a phone due to these limitations.

Notebooks are evolving at a slower pace than phones, and we’re not seeing new usage scenarios for them. They will continue to dominate the computing space, but improvements will be incremental at best. Netbooks are still selling in good numbers, but we’ve seen just about everything that can be packed into the small budget-friendly form already done. The notebook/netbook will continue to be the primary computers for most folks, but hardly the NBT.

Slates have gotten new life in the minds of many, but they are not new technology. The iPad has pushed slates to the forefront of the mobile space, and while the form makes it possible to do many things in a more appropriate way, it doesn’t bring anything new to the user. It doesn’t create new uses for consumers like the NBT should do. The iPad is fun and useful, but at its core it is only providing uses that are already provided by other devices. Apple will sell a boatload of iPads, but it’s not the NBT.

The same is going to hold true for the many slates we are bound to see coming down the pike. They will do many things, and some of them very well, but nothing really new. They will attempt to duplicate the popularity of the iPad, and some may come close, but there won’t be anything truly innovative in this process. The slate will simply be another way to do the things that folks already do.

It may sound like I am cynical, and perhaps that is true. But I also find it a great time to be a mobile enthusiast, as the advancements in the past few years have been extraordinary. I am itching to see something new come along that will blow me away in sheer usefulness. That’s the primary requirement for the NBT for me — give me a new use that will rock my world.

The Microsoft Courier concept that has leaked out in videos is the primary candidate for the NBT. This device doesn’t fit into current categories, it is creating something totally new. It is an attempt to bring the day planner of old into the digital age, and that can be huge. It was only a few years ago when seemingly everyone carried one of those paper day planners. I attended countless meetings where the conference table was covered with planners. I used one myself, it was a way to organize my life, keep track of the things important to me and to make sure I didn’t overlook anything significant.

Smartphones began to take over some of the roles of the day planner, and they started to disappear. But the smartphone, due to the screen limitations mentioned, can’t really replace all of the benefits the day planner provided. This is where the Courier has amazing potential. The videos of the Courier show a day planner that totally integrates the web, the professional life and the social life of the user. It is the glue that binds all of that together, and has the potential to leverage that in a way that can vary as needed for the individual. It is a way to make mobile tech highly personal, and this can be big.

Where the digital planner (Courier) can make the biggest contribution for the user is through search. The day planner of old suffered from the inability to easily find things as needed, especially those things not in the immediate past. The Courier will not suffer from that disadvantage; it will enable the user to find anything, or more appropriately everything as needed. It will turn the web into a vast fountain of information that when coupled with each user’s personal information, will be the most important tool to come along in a great while.

So my requirement that the NBT present a new use for the individual is met by the Courier as demonstrated. Keeping all of the stuff that is important to the individual at hand, in an easily accessible way. It is personal by nature, and that can change the way we work. This could be the Next Big Thing.

Related content on GigaOM Pro (subscription required):

Hot Topic: Apple’s iPad

Last March, I wrote a piece about the 10 apps a web worker can’t do without. A year later, and the app landscape has shifted significantly, so here’s an updated list of what I consider to be the best solutions for our critical tasks in 2010.

Backoffice

1. Invoice management

Freshbooks is still an excellent invoicing service, but there are some new and some improved invoicing apps in this space, such as:

WinkBill
SimplyBill
WorkingPoint

You can also track expenses with:

Expensify (see our review here)

And add functionality to your online invoicing:

OfficeDrop (includes Freshbooks integration. See my review here)

Also check out Celine’s piece on managing your finances online. If you are looking for solutions for creating and managing contracts, check out Celine’s roundup of contract resources. You may also want to check out Outright, a bookkeeping and tax prep tool that Scott reviewed.

2. Time tracking

Need to get a handle on how much time you’ve spent on a particular project or task? There are apps for that — some standalone, some that integrate with your invoicing systems, and some like Where is My Time that help you to analyze how the time flew by and how productive you were during that time. Here are some others:

Klok
BubbleTimer
toggl
Paymo (tracking with invoicing)
Intervals (plus task and project management)
RescueTime (with time management and project tracking. See our review here.)

And in the time-, cash- and distance-tracking category, take a peek at 1DayLater (see our review here).

3. Social CRM

Here are some startups that provide useful Social CRM products:

BatchBlue
SugarCRM

For quick contact information exchange, I love the Poken social business card and am so disappointed that they aren’t taking off like I think they should. I wear my cute skull Poken at every conference I go to but have yet to get “poked” unless Poken is a conference sponsor.

I’m also keen on Bump for the iPhone and Android, and also covered some other contact sharing apps in this roundup: DropCard, Rmbrme, BeamMe, ShareCard, SnapDat.

At SXSW, I was given a very impressive demo of relationship management tool Gist (see our review here) that promised a lot, though I have yet to incorporate it into my daily work.

4. RSS Reader

I don’t know about you, but I’m over RSS readers. But to be fair to those who have yet to discover the social firehose, you could go with the ever-popular Google Reader and the novelty of Snackr, which puts a little ticker at the bottom of your computer screen for passive, almost subliminal consumption of your feeds.

And here are a few other popular feed readers:

Filtrbox (see my initial review here)
Netvibes (this is really more of a “build your own landing page” service that incorporates feed reading features)
Feedly (a similar landing page service)

For saving articles to read later, I currently use a combination of Delicious, Instapaper, and “favoriting” tweets containing links on Twitter.

Communications

5. Email management Social communications management

I have changed the heading of this section because I find that my communications are no longer mainly taking place through email, and are increasingly moving into my social networks.

While I am trying to move away from Gmail and start using email management tool PostBox again (its attachment management tools make it a compelling option for me), I am also looking out other social communications management systems and apps.

I was panicked to find Threadsy — the intriguing integrated communications client that you could use to see your email, social networks, and Twitter in a single place — under “re-construction” but have signed up to see what is happening with the app.

You might also like to check out a few email productivity add-ons we’ve reviewed, such as Xobni, Rapportive and MailBrowser.

6. Calls, Conferencing and Instant Messaging

Right now, my company has been moving away from Skype. Although we all love the app, it seems to drops our calls almost constantly now. We are moving back to the old-fashioned telephone for calls, while for conferencing we have been using FreeConference.com.

Here are a few phone conferencing and webinar-style conferencing systems that I also use:

Calliflower
DimDim
GoToMeeting

Note that I didn’t include WebEx in the list. I am convinced that the company, which once dominated this space, has had a hard time keeping pace with the more nimble startups.

One other phone-related service that my company is trying is eVoice, because we need a virtual PBX system that can accommodate our UK office as well as multiple U.S. locations. Unfortunately,  I don’t have enough experience with it yet to tell you how it is working for us. Stay tuned.

Work Process

7. Project management

My company first used Basecamp for project management before switching to 5pm. Today, I’m seriously checking out glasscubes as it provides project management together with collaborative space. It is much lighter on the project management side — it’s really just a task management  app — but I’m getting a feel for the company’s interesting take on how virtual groups can work better together. More on that soon, too.

Just for giggles and grins, check out my old post about this topic: “Project Management, Collaboration and How Our Brains Work.”

8. Calendars and Schedules

I’m excited about the web-based services that allow me to give out a link to my calendar — or just a portion of my calendar — so  people can get on my schedule. But as my post about a scheduling bungle at SXSW due to system time zone issues, I know that there is still no single tool that “does it all.”

Still, here is a quick rundown of a few tools I’m still using or trying out:

TimeBridge and their Meetwith.me feature
TimeDriver
Tungle
ScheduleOnce

9. Cloud-based collaboration/document sharing

While I still use Google Docs, some fundamental integration issues are making me look elsewhere. As I mentioned earlier, my company is currently experimenting with glasscubes.

Here are a few others:

iWork.com (from Apple)
Zoho
Colaab (see our review here)
Backboard (useful for feedback on docs and files)

And I know you’re going to think this is kooky, but the 2.0 version of Spinscape combines mind mapping principles and collaborative communications in a way that is quite compelling to me.

10. File storage/backup/sync

Take a look at SugarSync. It provides backup, file sync and file sharing “on-the-go” on any Mac, PC or mobile device (check out my review here). Here are some other useful options:

DropBox
Box.net
Mozy

A newbie in the “active backup” and file storage space is Soonr, which also has Mac, PC and mobile capabilities.

What are some of the must-have applications you’ve found invaluable in your own web work?

stock.xchng images by johnnyberg, gun, toutouke

Related GigaOM Pro content (sub. req.): Enabling the Web Work Revolution

The New York Times has an online math blog by Steven Strogatz. Given the venue, it is written for an elementary audience. Here is a recent post on the method of exhaustion and how it allows you to approximate π.

I’ve drafted a review of a new book, The Design of Design. Comments/corrections would be appreciated.

Jonathan Rauch contrasts David Frum’s vision of the world with Fox News’ vision of the world:

In Fox World, he says, Obama is a radical ideologue determined to impose European-style socialism on the United States; in reality, he is a pragmatic consensus-seeker who gets his ideas from the Left but wants to win re-election with 60 percent of the vote. In Fox World, Americans in the millions are rising up to protest Obama’s expansion of government; in reality, many Americans are distressed by the economy and will simmer down when prosperity returns.

In Fox World, liberals have wrecked the country; in reality, 21st-century America is better in almost every way than the America of 1975 or 1955. In Fox World, people such as Palin and the conservative talking heads Rush Limbaugh and Glenn Beck speak for the overlooked American middle; in reality, they speak for a fringe, one big enough to make them rich but not to elect anyone to national office.

Where you land on the spectrum between these two world views will determine how radical you see the Tea Party movement. I think part of the fundamental disagreement between Arnold Kling and Will Wilkinson about who libertarians should align with and how to view the Tea Party Movement is that Arnold sees the Fox World as closer to reality and Will sees the Frum world as closer to reality. The more radical the movement, the farther from reality their beliefs, the less sustainable and influential they they will be as a political movement.

Another part of the disagreement between these competing visions is how to brand libertarianism. Will thinks the association with the Tea Party will push libertarianism, or at least the perception of libertarianism, even more towards an undesirable brand that is already holding it back.  Arnold, in contrast, believes that the Tea Party movement is a relatively reasonable movement with libertarian aspects, and is a practical and desirable vehicle for libertarianism. (Of course, I may be completely projecting with this assessment of Arnold and Will’s beliefs, in which case replace “Arnold” and “Will” with “some libertarians” and “other libertarians”.)

I guess the question is whether the Tea Party Movement can actually deliver libertarian policies, or will it achieve nothing and tarnish the name of libertarianism along the way?

H/T M.R.

Filed under: Uncategorized

I’m editing an interview with John Hancock, who leads the PowerPivot charge and championed its support of OData. During our conversation, I told him this story about how pleased I was to discover that OData “just works” with PubSubHubbub. His response made me smile, and I had to stop and transcribe it:

Any two teams can invent a really efficient way to exchange data. But every time you do that, every time you create a custom protocol, you block yourself off from the effect you just described. If you can get every team — and this is something we went for a long time telling people around the company — look, REST and Atom aren’t the most efficient things you can possibly imagine. We could take some of your existing APIs and our engine and wire them together. But we’d be going around and doing that forever, with every single pair of things we wanted to wire up. So if we take a step back and look at what is the right way to do this, what’s the right way to exchange data between applications, and bet on a standard thing that’s out there already, namely Atom, other things will come along that we haven’t imagined. Dallas is a good example of that. It developed independently of PowerPivot. It was quite late in the game before we finally connected up and started working with it, but we had a prototype in an afternoon. It was so simple, just because we had taken the right bets.

There are, of course, many kinds of efficiency. Standards like Atom aren’t most efficient in all ways. But they are definitely the most efficient in the “it just works” way.

With all the talk about Apple’s iPad, I’m noticing two trends. First, the device is polarizing opinions — people either lust for it or say they don’t see the point of it. Very few seem to be “on the fence” about the iPad. But there’s a common thread among all — folks are wondering how (or if) an iPad fits into their daily workflow, and that gets me to my second trend. People are looking to the iPad for more than content consumption — which it is excellent for — they’re trying to be productive with it. I’m doing just that, now that I found an app that not only links my iPad with my Google Docs account, but lets me edit documents and spreadsheets in the cloud.

Office2 Pro is the app and it cost me $7.99, which I think is more than reasonable for the functionality it provides. The software allows me to use the iPad as if I was running Google Docs locally on my computer. With it, I can view any of my documents, spreadsheets, presentations or even Adobe PDF files I have stored in the cloud. But Google Docs viewers are a dime-a-dozen — the difference here is that I can edit files on the iPad and the changes are saved online to my Google Docs account.

It’s not perfect (yet), but it works

Now there are some limitations and quite a few kinks to be worked out. For starters, you can view all kinds of file types, but for now, you can only edit documents or spreadsheets. Presentations are view-only. I also encountered a fair number of bugs when using the software. Adding an image to a document worked, for example, but when I zoomed the page to get a better view of the format, the entire doc zoomed off the screen. I was able to save the changes, even in this state, but it was a scary few moments. That’s not the only bug, but rather than run though them all, I’ll say this: the developer is aware of several issues and has already submitted a version with fixes to Apple for approval. I’ve captured the list of fixes in the image gallery below.

iPad: meet the cloud

It was tricky to connect Office2 Pro with Google Docs, and the support information could use an update. Once I figured it out however, the steps were actually quite simple. And I didn’t just connect the iPad with my personal Gmail account — the software supports Google Apps accounts too, so I’ve used it for both work and personal documents. Yes, you can link Office2 Pro to multiple web accounts simultaneously. Although I don’t have a MobileMe account, the developer says you can also connect Office2 Pro to an iDisk on MobileMe. And it also supports Box.net or any WebDAV server, so you’re not limited to Google Docs. In the file explorer, you can create folders and move documents just as if you were accessing your Docs account in a browser. Plus you can email docs right from the app. From a connectivity standpoint, most users should be covered.

Usability

Data entry is relatively straightforward. The iPad’s native on-screen keyboard is available by tapping a button — tap your screen where you want to insert or change data and start typing. All of the special function buttons are at the top of your document, just where you’d expect them. And the app works in portrait or landscape, although the file explorer stays put in landscape — I’d like to see the document in full screen but haven’t found a way to do that yet in landscape. In a spreadsheet, data entry can be a little cumbersome as you first tap a cell and then enter data in a field above the spreadsheet — not ideal, but it works.

Most functionality that people use for documents and spreadsheets is available in Office2 Pro. From a word processing standpoint, you have text and paragraph formatting, table creation, images and even real-time spelling correction to name a few features. Spreadsheet support includes multiple worksheets (although there’s a bug with this), sorting, formatting, cell merging, pane freezing, and 112 functions.

I’d estimate that most of what people need in a Google Docs editor is in the app, or is coming soon. Office2 Pro is surprisingly full-featured if you approach it with a Google Docs mentality — it’s good enough for most users, but won’t replace every function found in the Microsoft Office suite. Aside from charts and other advanced features offered by Google Docs, the basics are all here.

Note: to view any images in full size, simply right-click on one and open in a new browser tab.

Buy now or wait?

So if you can connect your iPad with a Google Docs account and edit information, why isn’t anyone talking about this application? I think there’s a few reasons — lack of awareness for starters, not to mention the buggy nature I’ve experienced. Unless you’re a patient person, I’d strongly consider waiting for the updated version with fixes before plunking down the $7.99. Yes, the app works, but it’s not quite where it needs to be. I’ve had to create workarounds for some issues and not everyone is willing to do that. A good example: there’s no Close button when you’re done with edits. I end up opening a new document in this case, which forces the app to prompt for a document save. And that’s just one of several user interface examples that need some tweaking.

I’m looking forward to the updated version as I see great promise in Office2 Pro. I made the transition to Google Docs well over two years ago and this software is breathing an air of productivity into my iPad that I didn’t expect. I anticipate that Apple will update its iWorks apps to address compatibility and file management issues but I’m not so sure Apple will fully embrace Google Docs in iWorks. If it does, great, but if not, Office2 Pro is worth the look.

(Special shout out to Joshua, a reader that tipped me on this app!)

Related GigaOM Pro Content (sub req’d)

I put together the following “Twitter Cheat Sheet” for a client, and thought it might be also helpful for others who are still struggling with how to get started with Twitter. It provides some gentle guidelines that should help to keep emerging best practices in mind.

Why Use Twitter

There are many reasons to use Twitter, but here are my top three.

Branding. The more others refer to you or “retweet” what you tweet, the greater your reach. Extend your brand by tweeting “retweetable” tweets.
Communications. Twitter can be a powerful communications tool, particularly because of its potential to reach not just your own followers, but your followers’ followers and so on. Google  and other search engines are also including tweets in search results now.
Community. By engaging your followers, you can begin to build a following, and join in conversations in meaningful ways.

How to Use Twitter

When you first open up Twitter each day, here are a few key things you’ll want to do to make the most of your “tweeting time.”

Check @ messages. First check to see who has publicly referenced you in their tweets and acknowledge, answer or respond.
Check DMs. Next check your direct messages (DMs) from people who have messaged you privately. Note that you don’t need to respond to auto-responders that say things like “Hey, thanks for following me! Can’t wait to learn more about you.” Responding to those is not expected.
Check your Twitterstream. Read through the first few pages or screenfuls of tweets to see what the people you are following are saying, and to see where there might be appropriate opportunities to retweet, respond or reference them.
Check Searches. You save searches for terms such as your brand name, product names, and keywords pertaining to your business or industry. These can be good sources of additional tweet fodder.

How to Make Your Tweets “Retweetable”

Twitter limits the length of tweets to 140 characters. In order to make your tweets “retweetable” (to make it possible for others to retweet you without having to edit the tweet), you need to reduce the the maximum length even further. Add five to the number of letters in your Twittername, then subtract that from 140. So, for example, there are 12 letters in “alizasherman” — adding five to that is 17. So I subtract 17 from 140, which equals 123. That is the maximum length of my “retweetable” tweet.

How to Retweet

There are two standard ways to retweet – adding “RT @Twittername” at the start of the tweet,  and adding “via @Twittername” at the end of the tweet. Either is acceptable, but the latter takes up more valuable “character real estate” — it’s longer, so you can fit less into the retweet. It is acceptable to change spellings of words to make a retweet fit the character limit.

You can also use Twitter’s built-in retweet button, but this is a newer feature has not been adopted by everyone,  and it’s not the ideal way to retweet as it seems less “personal.” You can personalize retweets by adding a comment at the beginning or end.

Here’s a screenshot showing all three types of retweet:

Types of Tweets

There are many ways to tweet. Here are a few examples of different types of tweets to give you some ideas.

Informative with a link

Provocative or timely statement or quote

Questions to start conversations

Referring to someone with an @

Response to an @ message

Retweet

Promotional (with a link or link to image)

How to Use Hashtags

Hashtags are simply words prefixed with a hash sign (#) added to your tweet. Hashtags provide ways to group and find topically-related or conversationally related tweets — readers can click the hashtag in your tweet to find related discussions on the same topic. For example, if you tweet about wine, the best hashtag for wine discussion is “#wine.” By using it strategically, you could attract more followers.

Here are some example tweets using the #wine hashtag:

#FollowFriday or #FF

There is an organic Twitter-wide event called “Follow Friday” where people recognize their favorite Twitterers by referencing them in a list and then adding the hashtag “#followfriday” or “#ff.” Note that you should make sure your Follow Friday list is retweetable.

Follow Friday is a great way to give kudos, help others discover new and relevant Twitterers and to attract the attention of people you follow who may not follow you back yet.

It’s a good idea to thank people when they give you kudos or list you in a Follow Friday tweet, but don’t thank everyone one by one as this can be annoying to your followers.

What are some of your favorite Twitter tips?

Photo by Flickr user Mr_Stein,  licensed under Creative Commons

I just finished listening to Traffic: Why We Drive the Way We Do, appropriately enough, while driving around Boston. The book is probably better read/skimmed than listened to, and/or needs abridgment. There are some worthwhile nuggets of information, however, and I’ve selected a few below.

SUVs are hugely costly to society. Because they are long and sluggish, they spend much more time getting rolling from a stop at a red light. This is one reason that our city traffic has slowed down. SUV drivers sit higher from the optical rush of the road so they are more likely to speed (just as you wouldn’t have a strong sensation of speed in an airplane 1000′ above the ground, though the airplane is moving at 4X the speed of a car). SUV drivers in at least two countries studied are less likely to wear seatbelts, more likely to be talking on a mobile phone, and less likely to have both hands on the wheel. They crash constantly and are statistically less safe than a minivan that is lighter weight.

Low-cost parking meters in cities are a primary cause of traffic jams and accidents. About 12 percent of cars driving around a city are looking for a parking spot. Those folks drive very slowly. When they stop, they tend to get hit by other cars, and traffic comes to a standstill until the accident is cleared. As soon as parking spots are more than 80 percent occupied, city traffic slows down to a crawl. (If we had a nationwide wireless Internet and perhaps an RFID transponder in cars, the solution would presumably be dynamic pricing for parking spaces so that there were always about 20 percent free.)

Intersections with lights are hugely dangerous and have very little capacity compared to roundabouts. The heavyweight control systems and signs don’t ensure driver or pedestrian safety. The intersection is useless during a clearing phase that has to be lengthened every year (now it is about 2 seconds of red in all directions). The intersection is very slow to start up again after a red (see the note about ponderous SUVs above).

Signs are basically useless, especially as they have been layered onto our roads year after year. People drive slower around curves with no “curve warning 30 mph” sign than they do with the sign. Deer crossing signs do nothing to reduce the prevalence of deer-car collisions.

High curbs and crosswalks do not protect pedestrians. In fact, in Dutch cities where all signs, curbs, and markings have been removed, accidents and injuries have gone down. Traffic engineers have spent decades applying techniques that work on highways to city streets and continue doing so though all research shows it doesn’t work. All of the road engineering discourages drivers from paying attention to what is happening around them. [Similar results were found in London.]

Skill does not make for a safer driver; an insurance company study of NASCAR-style racing drivers found that these supremely skilled individuals were more likely to get into accidents when driving on public streets than the average driver.

Contrary to advertisements touting the miracles of airbags, new cars are no safer than old cars, according to studies in Norway and the U.S. Adjusted for miles driven, people in new cars are more likely to be in an accident and more likely to be injured than people in old cars. Quite a few people are killed in new cars while traveling at less than 35 mph. What would reduce deaths and the cost of injuries would be if everyone wore helmets, though this has never been seriously proposed.

I recommend the book in print because each section stands on its own nicely and the reader can pick and choose the most interesting topics.

During a discussion today about potential iPad sales, a friend and I got distracted by reflecting on the tremendous value delivered by a vanilla PC. Over on the Dell Web site right now, a basic 15″ laptop costs $499 and includes 320 GB of hard drive and 3 GB of RAM. So for the same price as an iPad, you’ve got a machine that can do the following:

give each family member his or her own private and personalized set of files, programs, and bookmarks
support the creation of almost any kind of document, plan, or project
run nearly all of the world’s software development tools
function as a videophone
function as a television, DVD player (and burner), radio, and videogame
be used almost anywhere in the world (multi-voltage power supply and standard 802.11 wireless adapter)

The machine exacts a price in terms of learning and administration, but it does a lot for $499!

So… can Apple sell a huge number of machines at the same price that do a lot less? Sure, at least to those who have an extra $499 for an indulgence.

Let’s look at Apple’s track record:

Macintosh: substantially more expensive than competition, 2-5 percent market share
iPod: comparable price/megabyte to competition, 55-70 percent market share
iPhone: slightly higher price than other smart phones, 16 percent market share of smartphones (source), negligible market share of all mobile phones

There have been enough spendthrifts worldwide to keep Steve Jobs in Gulfstreams, even when an Apple product carries a premium price, but Apple has only touched the mass market with products that are close in price to competitors’ products.

The best predictor of iPad world dominance may therefore be how much it costs to buy a similar tablet from a competitor. I.e., if the competition costs about the same (see “Zune”), Apple could end up with most of the market.

Minimum Desirable Product
(if you don’t see the slides, go here to Slideshare)

Recent slides for a talk in Steve Blank / Eric Ries’s class on High-Tech Entrepreneurship
Yesterday I had the pleasure of giving a talk at Steve and Eric‘s class at Haas on the topic of Minimum Desirable Product – if you haven’t read the original article, it provides some useful context. I included an set of slides above on the topic, updated from my talk yesterday, which you can peruse at your convenience.

After you’re done, you can read my extended remarks below on some stuff I learned along the way. Frankly, any of these could probably be its own blog post but I’ve been feeling lazy lately so you get a couple sentences apiece instead

“Viable” means different things to different people – my usage is meant to be pretty specific
Eric noted during my talk that I use a very narrow definition of “viable” within Minimum Viable Product, which is true. I believe in his usage of it, the focus on viability is actually a conglomeration of IDEO’s concept of desirability, feasibility, and viability. It’s frankly a coincidence that IDEO and the Lean Startup use a common term, though I believe they mostly overlap. I prefer IDEO’s framework because it allows a bit more precision in describing the class of issues you’re concerned about, but frankly there’s a ton of gray area. (Is a low-priced X a desirability thing or a viability thing? Honestly, both.)

Viability-first strategies do work, and may be the right thing for you
Many companies have come and gone that make products that aren’t that great, don’t generate a lot of consumer value, and yet still pull in a lot of money. It’s a strategy that can work, and I’m not arguing the opposite. However, I’m convinced that if your goal is to make a mainstream web property that has daily engagement, starting with the goal of creating lots of user value is probably the way to go. Similarly, if you have a highly transactional business like ecommerce, designing for daily engagement is probably overkill – in that case, reducing your cost of customer acquisition might be the right way to go. So it’s all very situational, and frankly, very personal based on how you want to run your product.

Minimum Desirable Product is just a starting point – you still need to figure everything else out
I also want to note that my message isn’t just to build for any random group of users and then the rest will take care of itself. That’s far too idealistic. Instead, it’s just a starting point for how you think of the problem. Ultimately, all your product ideas still need to be filtered through the lens of whether you can market them, that the market is big enough, and that the technology issues aren’t insurmountable. There was a recent Times interview with Steve Jobs on the iPad that illustrates this perspective:

… surely Apple stands at the intersection of liberal arts, technology and commerce? “Sure, what we do has to make commercial sense,” Jobs concedes, “but it’s never the starting point. We start with the product and the user experience.”

Metrics can be oriented towards user value
I’ve written before on some of the short-comings of using metrics-driven product strategies, such as here and here. An analytics dashboard is ultimately just one tool out of many that help you optimize whatever goal you want to set. If you are very focused on validating your business model and spend all your time tracking metrics such as viral factor, ARPU and conversation rates, then you will make those go higher. If you use your metrics to define user benefits and optimize those (I’ve begun calling this “Metrics of Love”) then you’ll make your value proposition go higher. So depending on your perspective and where you want to start, you’ll end up in different places.

Highly desirable consumer products also have minimalist featuresets
In consumer products, unlike some enterprise products, there’s a big focus on simplicity and immediate value. In some ways, the idea of a “minimum desirable product” is kind of misleading because highly desirable products may also have minimum featuresets also, perhaps even more minimal than an MDP. The important part is that they are the right features, and in fact, it often takes a longer time to simplify your product and boil it down to the core value. I think that’s an interesting paradox that exists in consumer products, and one that I didn’t grasp for a long time.

Learning about your business and learning about your product desirability are different things
One of the interesting points that came up yesterday was that if you view your company as a learning machine to validate your business before you run out of money, then you may see that worldview clash with wanting to deliver maximum product desirability. In many cases, shipping a 50% done feature may teach you a ton about the market, and very quickly you will learn what you need and want to move on. The problem is, it may turn out that going from 50% to 100% in user experience actually continues to increase value to the user, by making things more refined and more compelling, even if you stop learning about your business. This is a hard thing to trade off, and requires situational judgement. As Steve noted during yesterday’s discussion, deciding when you stop and just consolidate and refine what you have, versus packing in new features – well that’s the place where entrepreneurship is an art and not a science

OK! Back to blogging vacation See you guys later.

After my initial one-year deal with Verizon FiOS ran out, they sent me shocking near-$200 monthly bills, so I downgraded my service to the bare minimum last fall. I had Internet plus a lifeline phone. Then they started filling my mailbox with an offer to go back to the glorious world of Verizon unlimited phone, Internet, plus some TV for a $65/month teaser rate, escalating to $95/month for a total two-year period. I finally took the bait and upgraded my service via the Web site that they provided.

Five minutes later, my Internet stopped working. I called FiOS tech support. “You placed an order for new service, so of course we had to shut off your old service,” they helpfully explained. When would the Internet that I was already paying for be restored? “You’re scheduled for an install on April 9th.” So the upgrade process has a designed-in two-day service outage? “No. Sometimes people are cut off for a week or more.”

I have drafted a summary/review of The Greatest Trade Ever. Comments/corrections would be appreciated.

An article by John Doerr had a great quote from technology luminary Alan Kay that every entrepreneur needs to remember “The best way to predict the future is to invent it.”

I’m working with a company that at one point had a product that was not only best in class, but also technically far ahead of its competition.  It created a better way of offering its service and customers loved it and paid for it. Then it made a fatal  mistake.  It asked its customers what features they wanted to see in the product and they delivered on those features. Unfortunately for this company, its competitors didn’t ask customers what they wanted. Instead, they had a vision of ways that business could be done differently and as a result better.  Customers didn’t really see the value or need, until they saw the product.  When they tried it , they loved it.

So what did my company do when they saw what their competitor had done ? They repeated their mistake and once again asked their customers what they wanted in the product. Of course the customer responded with features that they now loved from the other product.

They didn’t improve their competitive positioning. They put themselves in a never ending revolving door of trying to respond to customer requests. To make matters worse, resources and brainpower that could be applied to “inventing the future” were instead being used to catch up with features that locked them into the past.

Entrepreneurs always need to be reminded that its not the job of their customers to know what they don’t know. In other words, your customers have a tough enough time doing their jobs. They don’t spend time trying to reinvent their industries or how their jobs are performed. Sure, every now and then you come across an exception. But you can’t bet the company on your finding that person at one of your customers.

Instead, part of every entrepreneurs job is to invent the future. I also call it “kicking your own ass”. Someone is out there looking to put you out of business. Someone is always out there who thinks they have a better idea than you have. A better solution than you have. A better or more efficient product than you have.  If there is someone out there who can “kick your ass” by doing it better, its part of your job as the owner of the company to stay ahead of them and “kick your own ass” before someone else does.

Your customers can tell you the things that are broken and how they want to be made happen. Listen to them. Make them happy. But they won’t create the future roadmap for your product or service. That’s your job.

The best way to predict the future is to invent it.  Words that should always be part of your product or service planning.

ShareThis

jPlayer is a jQuery plugin that allows you to:


play and control audio files in your webpage
create and style an audio player using just HTML and CSS
add sound effects to your jQuery projects
stream faster using HTML5 and alternative ogg format support

All of this with HTML5 <audio> support for compliant browsers that allow mp3 or ogg format, while supporting other browsers using mp3 format with no visible Flash.

I had been meaning to explore PubSubHubbub, a protocol that enables near-realtime consumption of data feeds. Then somebody asked me: “Can OData feeds update through PubSubHubbub?” OData, which recently made a splash at the MIX conference, is based on Atom feeds. And PubSubHubbub works with Atom feeds. So I figured it would be trivial for an OData producer to hook into a PubSubHubbub cloud.

I’ve now done the experiment, and the answer is: Yes, it is trivial. In an earlier post I described how I’m exporting health and performance data from my elmcity service as an OData feed. In theory, enabling that feed for PubSubHubbub should only require me to add a single XML element to that feed. If the hub that connects publishers and subscribers is Google’s own reference implementation of the protocol, at http://pubsubhubbub.appspot.com, then that element is:

<link rel="hub" href="http://pubsubbubbub.appspot.com"/>

So I added that to my OData feed. To verify that it worked, I tried using the publish and subscribe tools at pubsubhubbub.appspot.com, at first with no success. That was OK, because it forced me to implement my own publisher and my own subscriber, which helped me understand the protocol. Once I worked out the kinks, I was able to use my own subscriber to tell Google’s hub that I wanted my subscriber to receive near-realtime updates when the feed was updated. And I was able to use my own publisher to tell Google’s hub that the feed had been updated, thus triggering a push to the subscriber.

In this case, the feed is produced by the Azure Table service. It could also have been produced by the SQL Azure service, or by any other data service — based on SQL or not — that knows how to emit Atom feeds. And in this case, the feed URL (or, as the spec calls it, the topic URL) expresses query syntax that passes through to the underlying Azure Table service. Here’s one variant of that URL:

http://{ODATA HOST}/services/odata?table=monitor

That query asks for the whole table. But even though the service that populates that table only adds a new record every 10 minutes, the total number of records becomes unwieldy after a few days. So the query URL can also restrict the results to just recent records, like so:

http://{ODATA HOST}/services/odata?table=monitor&since_hours_ago=4

The result of that query, however, is different from the result of this one:

http://{ODATA HOST}/services/odata?table=monitor&since_hours_ago=24

Now here’s my question. The service knows, when it updates the table, that any URL referring to that table is now stale. How does it tell the hub that? The spec says that the topic URL “MUST NOT contain an anchor fragment” but “can otherwise be free-form.” If the feed producer is a data service that supports a query language, and the corresponding OData service supports RESTful query, there is a whole family of topic URLs that can be subscribed to. How do publishers and subscribers specify the parent?

Here are four mechanical devices I prefer to their modern counterparts.

French press. It makes better coffee than a typical coffee machine. Also, a French press work without electricity. Next time a hurricane comes through Houston and knocks out our power, I can still make my coffee.

Reel mower. I had gasoline powered lawn mowers until last year. Sometimes they’d start, sometimes they wouldn’t. My reel mower always starts. And it’s quiet.

Rake. I had a leaf blower once. It was obnoxiously loud and a nuisance to my neighbors. I much prefer raking leaves even though it takes longer.

Pencil sharpener. With four children, we sharpen a fair number of pencils. We have owned a couple electric pencil sharpeners. They were noisy, hard to use, and soon wore out. Our mechanical pencil sharpener is cheaper and far more reliable.

I’m no Luddite, but I firmly believe that newer isn’t necessarily better.

Related posts:

Selective use of technology
Tim Bray’s high-tech monastic cell
Software bloat
What’s wrong with paper?

I’m fascinated by this this central limit theorem for infinitely divisible distributions. I knew about two special cases, the usual central limit theorem, and the result that a suitable limit of Bernoulli random variables gives you the Poisson distribution, but I didn’t know that there was a general theory about the phenomenon.

Tomasso Dorigo has a new, non-technical, post explaining muon decay. Tomasso characterized his description as “although not perfectly correct in the physics, was actually not devoid of some didactic power.”

I had a very enjoyable debate with Avner Rosen of Boxee yesterday at SXSW.  We tended to go around in circles defending our positions. His (to paraphase): the internet will do what cable and satellite do, but it will do it better. That MicroSoft and Apple among others will in the future come up with new ways to do it all better than we can now.

Mine: Of course the internet can support video, but it cant do it as well as the current digital cable and satellite distributors can, nor is there a profit model that would ever incent content providers to switch their content from the internet to TV.

I wont rehash it all here, but the most salient comment came from the audience when someone asked “Whats the difference if the gatekeeper MicroSoft/Google/Apple rather than Comcast/Time Warner/Directv/Dish ?”

Of course the questioner was right.

Bits are bits . The economics of content are going to search for the best place to be monetized, regardless of the delivery platform.  I think the current model will be the place content finds, Avner seemed to feel it would be some future internet.

It shouldn’t matter.

We both seemed to agree that one of the biggest future changes for the internet will be an increase in bandwidth. The question the needs to be answered is “how will all the new bandwidth, both on the backbone and in the last mile be used ?”

Will it be used for digital video content, ie, TV ? Why would we want it to be used for TV ?

The internet has the opportunity to continue to be transformative. There are applications out there that can change the quality of our lives.  Im not talking about the ability to update your facebook, or check out what where everyone is at on 4Square and twitpic what is happening to your friends and followers. Social Media is one more amazing communication platform that is impacting our lives. But its not tranformative.

There will be transformative applications that need all the bandwidth they can get. Medical, Transportation, Defense, Gaming, Simulations and who knows what. As computers become more powerful, we need to be able to send more data to the cloud where they can crunch data and return it to us.

That is the value of an open internet. The things we cant imagine today. The applications that are just dreams because  they dont have enough horsepower and bandwidth to work today. I want the internet to be a platform for amazing. Not Gilligan’s Island reruns.

I recognize that with openness comes the risk of the least common denominator dominating.  Porn has just as much right to dominate the net as any other app. But should it ?

Yes it should. The lesson of TV over the internet vs TV over digital cable or satellite isnt about openness. Its about the value of Application Specific Networks. DIgital CAble is an application specific network. Its only job is to serve up content and it works very well.

The FCC needs to recognize this and start working with internet broadband providers to define tunnels,  Private Virtual Networks  or just plain reserved bandwidth that are saved for future applications. When a transformative application presents itself, there should be an opportunity to define the network to optimize that application.

In the event some form of ubiquitous entertainment comes first ,  takes over the net and saturates it, how is the FCC going to solve the bandwidth traffic jam ? They wont ever be able to put the bandwidth genie back in the bottle. Screaming at broadband providers to spend the money wont work any better than screaming to expand highways helps to alleviate rush hour traffic

The FCC wants Internet 2 at universities to flourish and develop new things. But where are they going to roll them out to if all of our bandwidth to the home is being used for some new form of 3D virtual fantasy sports?

I just dont think these things will just manage to take care of themselves.  The reality is that when we run into future internet application roadblocks our politicians will jump in the mix and attempt to “solve” the problems. You know where that will get us.

We need to start the process now of putting some bandwidth to the home away for a rainy day. Not only do we need to preserve bandwidth in the last mile, but the FCC also needs to  figure out a way to create a transparent market or exchange that allows competing applications a means of knowing how and when they will have access to bandwidth when, not if,  it becomes constrained.

And if you really want to make things interesting, the company that is doing the most work on realtime markets and competing for resources that i know of ? ….. Google.

Deal with it today or struggle with it in the future.

ShareThis

In ordinary calculus, you define “integral derivatives” — the first derivative, second derivative, etcetera. If you think of differentiation as an operator D that takes functions to functions, then the higher-order derivatives are just Di for natural numbers i. As far back as Liouville, mathematicians have defined fractional derivatives, extensions of this definition to real numbers, i. There is more than one possible definition, Wikipedia page gives the usual definition, which is in terms of the Laplace transform.

I’ve never known if fractional derivatives were good for anything, or were just a historical curiosity. (They are a special case of singular integral operators, which are useful in PDEs.) This very brief paper discusses an application of fractional derivatives to models of particles in a liquid. This sounds like it should be related to Brownian motion, and it is, but the processes that arise are related to more general Lévy processes than just Brownian motion.

I ran across this interesting post by a historian who was an undergraduate mathematics major. She found her old linear algebra notes, and was surprised to find how little of it she still understood:

It’s not just that I can’t answer this question now, it’s that I can barely comprehend even what it means. The terminology bounces through my brain, stirring vague imprecise echoes… It’s disconcerting enough to come across the ghost of your own self. It would be even more disconcerting to know for sure that part of your intellect is now forever closed off to you.

Like many people, while a graduate student Roger L. Goodwin took a wide variety of courses in applied probability, statistics, and operations research. Unlike the others, though, he compiled lecture notes from all of his classes into one big book, available for download.