Before we dip into the news that caught my eye this week, here's a thought-provoking excerpt from a new interview with Clay Shirky over at the Findings.com blog:

"Publishing is not evolving. Publishing is going away. Because the word 'publishing' means a cadre of professionals who are taking on the incredible difficulty and complexity and expense of making something public. That's not a job anymore. That's a button. There's a button that says 'publish,' and when you press it, it's done ... Institutions will try to preserve the problem for which they are the solution. Now publishers are in the business not of overcoming scarcity but of manufacturing demand. And that means that almost all innovation in creation, consumption, distribution and use of text is coming from outside the traditional publishing industry." (Read the entire interview here.)

Now, here are a few stories that got my attention in the publishing space this week.

All-you-can-eat magazines

Ken Doctor over at Nieman Lab took a look at Next Issue, a newly launched Netflix-like magazine platform. He describes the venture:

"It offers single-priced, all-you-can-eat access to top-shelf magazines, including Time Inc's People, Fortune, Sports Illustrated, and Time; Conde Nast's Vanity Fair, Allure, and Conde Nast Traveler; Hearst's Esquire and Popular Mechanics; and Meredith's Better Homes and Gardens and Fitness. Thirty-two magazines in total, at launch."

Though there is some comparison to be found between magazine and newspaper revenue losses in the digital era, as both so far have failed to fully embrace the web for profit, this platform appears to be disruptive in a big-picture fashion. As Doctor points out in the post, the big difference here with newspapers — and I might add book publishing houses — is the five big magazine companies that together own Next Issue (Time Inc., Conde Nast, Hearst, Meredith, and News Corp.) pooled their efforts to create the platform.

Doctor describes the pricing tiers and offers a nice analysis of how this endeavor might play out — it's well worth the read.

Also in magazine-related news, Zite is expanding its offerings, with the blessing and support of eight publishers (nine if you include its parent company CNN), with its new Publisher Program.

Tom Krazit at GigaOm took a look at the program and explains that though no money is changing hands, publishers will be allowed to place house ads at the bottoms of their sections linking readers back to their websites or apps. And though Zite initially had issues with publishers, the tension is waning. Krazit reports:

"... publishers are starting to realize that they can attract new readers through apps like Zite and build their brands, [said Mark Johnson, CEO of Zite] ... he said that content publishers have the same discoverability problem that small mobile developers have to confront, and that apps like Zite can drive traffic to those publishers that they wouldn't otherwise enjoy."

TOC Latin America — Being held April 20, TOC Latin America will focus on standards, global digital publishing trends, case studies of innovative publishers in Latin America, consumer habits, and much more.

Register to attend TOC Latin America. Save 20% through 4/12/12 with code TOCLAfbTOC

Google's multifaceted ebook approach loses a facet

Google jostled indie booksellers again this week with an announcement that it will discontinue its ebook reseller program come January 2013. Scott Dougall, Google's director of product management for digital publishing, explains the situation in a blog post:

With the launch of Google eBooks in 2010, we introduced a multifaceted approach to selling ebooks: online, on devices, through affiliates and through resellers. One part of that effort — the reseller program — has not gained the traction that we hoped it would, so we have made the difficult decision to discontinue it by the end of January next year.

It's important to note that the separate affiliate program will not be affected. Jeannie Hornung, a spokesperson for Google, told Good E-Reader: "... booksellers will still be highlighted in the 'Buy this book' section of Google Book search, supported with our affiliate program and have access to free Books APIs."


Indies may not be left selling solo, however. American Booksellers Association (ABA) president Oren J. Teicher told The Next Web:

"... we have every confidence that, long before Google's reseller program is discontinued, ABA will be able to offer IndieCommerce users a new alternative e-book product, or choice of products, that will not only replace Google eBooks as it currently works on IndieCommerce sites, but that will be in many ways a better product."

The original letter the ABA sent to booksellers can be found here, and (hat tip to ShelfAwareness) the ABA is offering an FAQ about the situation.

People who e-read buy books

The Pew Research Center's Internet & American Life Project released a new study report on ereading this week. The report findings show a marked increase in the number of people ereading:


"... some 43% of Americans age 16 and older say they have either read an e-book in the past year or have read other long-form content such as magazines, journals, and news articles in digital format on an e-book reader, tablet computer, regular computer, or cell phone."

Findings also show that by February 2012, 21% of adults in the U.S. had read an ebook in the past year — up from 17% in mid-December 2011.

And those numbers are likely to continue to rise in a steep incline. A post on the study over at Reuters notes that "Forrester, a consultancy, has forecast that nearly a quarter of Americans will own an e-book reader by 2016." The post notes Amazon's marketshare as well: "Online retailer Amazon.com Inc has about 65 percent of the e-book market, according to Cowen & Co estimates."


The Pew study, which was funded by a grant from the Bill & Melinda Gates Foundation, generally focused on reading behavior, both print and digital. And the news for publishers looks very positive on one front: According to the study, readers — especially ereaders — prefer to buy books:

A majority of print readers (54%) and readers of e-books (61%) prefer to purchase their own copies of these books.

And some stats for publishers afraid of losing sales via library card holders: 14% of readers reportedly borrowed the last book they read from a library — however, 12% of those who purchased their last read started their search at the library. You can find a nice selection of the study's library statistic highlights at INFOdocket.

One of the more surprising areas of the study looks at the devices on which people are reading. I found the percentages for computers and, in the U.S., for cell phones notable:

42% of readers of e-books in the past 12 months said they consume their books on a computer.
41% of readers of e-books consume their books on an e-book reader like original Kindles or Nooks.
29% of readers of e-books consume their books on their cell phones.
23% of readers of e-books consume their books on a tablet computer.

You can view the report in full here. Lee Rainie, the head of the Pew Internet Project, also will be the featured guest on today's Follow the Reader discussion on Twitter at 4 p.m. EST. You can join in at #followreader.

Related:

Books as a service: How and why it works

What publishers can learn from Netflix's problems

For booksellers, the future is brighter than it seems

Publishing News: Survey says publishers continue to miss out on digital opportunities

More Publishing Week in Review coverage

This post is part of the TOC podcast series. You can also subscribe to the free TOC podcast through iTunes.

I've never consulted an index in an ebook. From a digital content point of view, indexes seem to be an unnecessary relic of the print world. The problem with my logic is that I'm thinking of simply dropping a print index into an ebook, and that's as shortsighted as thinking the future of ebooks in general is nothing more than quick-and-dirty conversions of print books. In this TOC podcast interview, Kevin Broccoli, CEO of BIM Publishing Services, talks about how indexes can and should evolve in the digital world.

Key points from the full video interview (below) include:

Why bother with e-indexes? — Searching for raw text strings completely removes context, which is one of the most valuable attributes of a good index. [Discussed at the 1:05 mark.]

Index mashups are part of the future — In the digital world you should be able to combine indexes from books on common topics in your library. That's exactly what IndexMasher sets out to do. [Discussed at 3:37.]

Indexes with links — It seems simple but almost nobody is doing it. And as Kevin notes, wouldn't it be nice for ebook retailers to offer something like this as part of the browsing experience? [Discussed at 6:24.]

Index as cross-selling tool — The index mashup could be designed to show live links to content you own but also include entries without links to content in ebooks you don't own. Those entries could offer a way to quickly buy the other books, right from within the index. [Discussed at 7:28.]

Making indexes more dynamic — The entry for "Anderson, Chris" in the "Poke The Box" index on IndexMasher shows a simple step in this direction by integrating a Google and Amazon search into the index. [Discussed at 9:42.]

You can view the entire interview in the following video.



Mini TOC Chicago — Being held April 9, Mini TOC Chicago is a one-day event focusing on Chicago's thriving publishing, tech, and bookish-arts community.

Register to attend Mini TOC Chicago

Related:

Searching in ebooks: A unique use case that requires a unique approach

What ebook designers can learn from Bible-reading software

More TOC Podcasts

This post originally appeared on Joe Wikert's Publishing 2020 Blog ("iBooks Author: Appreciating Apple's Intent"). It's republished with permission.

With all the buzz about the agency model, the Justice Department, allegations of collusion, etc., I figure the time is right for a post about ebook pricing. Here are some quick thoughts as both a consumer and a publisher:

Eliminating waste is always a good thing — Walmart has mastered this for years. They squeeze every bit of waste out of the supply chain and generally end up with the lowest prices. I'm a frequent Walmart customer, and I greatly appreciate this. In fact, the only people who don't like this are (a) other retailers who can't match those prices and (b) ecosystem players who are part of the waste that's being eliminated, including suppliers.

Loss leaders are a great retail model — Selling some products at or below cost is a great way to bring customers in the door, regardless of whether that door is physical or virtual. I'm sure I've bought many cartons of milk at a loss for the retailer who made it up by selling me other items at a nice profit. It's a model that works, but have you ever seen a store that sells most of their products at a loss, every day?

Taking loss leadership to a new level — Remember when Amazon first launched the Kindle and pretty much every ebook was $9.99? It's no secret that Amazon was losing money on the majority of those sales. In fact, they still are. Prior to the agency model, Amazon was free to set whatever customer price they wanted for ebooks, even if it meant they were selling every single one of them at a loss. That brings up the razor/blades model, where it's not unusual for the razor to be sold at a loss, but the profit is made on the sale of the blades. So, if ebooks are the razors, what are the blades? The ereader device? According to iSuppli, the Kindle Fire's manufacturing cost is slightly higher than its retail price. How long can a retailer stay in business when they're losing money on both the razors and the blades? Presumably, they're making some money on other products they're selling (e.g., shoes, electronics, etc.). Perhaps. Then again, if they have deep enough pockets they can continue selling all their products at a loss until the cash dries up. In the meantime, competitors will find it difficult, if not impossible, to compete, so they'll disappear. What happens after that? Do prices remain low as products are still sold at a loss? Not if that company wants to stay in business.

The agency model prevents brand erosion — Think of the premium products you've bought or admired. Oftentimes, their prices are higher than most of the competition's. What would happen if those prices were suddenly significantly reduced? Would those products retain the full value of their premium brand? Highly unlikely. And shouldn't the owner of that brand have a say in what price is associated with it? Again, it's OK for a short-term loss-leader model, but I'm talking about selling something at or below cost for years and years, not just for a day or two. Over time, the value of that brand is affected. That's why I think publishers should definitely have the option to go with the agency model so they can manage retail prices and not let their brand lose value. By the way, consumers will ultimately vote with their wallets. If they feel the publisher's prices are too high, they'll stop buying and that publisher will either need to make adjustments or go out of business.

Fixed prices vs. price-fixing — In the U.S., we're so used to competitive retailer discounts that we're surprised to hear of the fixed price models used in other countries. For example, in Germany the price you pay for a book doesn't change from one retailer to the next. They're all required to sell them at the same price. Obviously, there's a huge difference between Germany's fixed price law and the price fixing the Justice Department is alleging. Germany's model doesn't lend itself to squeezing out waste like the U.S. model, but I'll bet it prevents one deep-pocketed retailer from putting its competitors out of business.

I don't work at a big six publisher, but I believe publishers should have the option to choose between the agency and wholesale models. The key issue though is that the Justice Department has suggested that Apple and a number of publishers colluded to keep prices high. I think this article by Gordon Crovitz in The Wall Street Journal sums it up quite nicely, particularly in the closing two paragraphs. Read that piece and ask yourself if the Justice Department's efforts will actually fix or merely add to an existing problem.

What's your opinion of the pricing questions and allegations currently facing the book publishing industry?

TOC Bologna — Being held March 18, TOC Bologna will feature sessions, demos, workshops and keynotes covering the art and business of storytelling in the digital age.

Register to attend TOC Bologna

Related:

Ebook pricing power is undermined by perceived value
Agency model may violate anti-cartel laws in Europe
It's time for a unified ebook format and the end of DRM

Around the globe, the bond between data and journalism is growing stronger. In an age of big data, the growing importance of data journalism lies in the ability of its practitioners to provide context, clarity and, perhaps most important, find truth in the expanding amount of digital content in the world. In that context, data journalism has profound importance for society.

To learn more about the people who are doing this work and, in some cases, building the newsroom stack for the 21st century, I conducted in-person and email interviews during the 2012 NICAR Conference and published a series of data journalist profiles here at Radar.

Sarah Cohen (@sarahduke), the Knight professor of the practice of journalism and public policy at Duke University, and Anthony DeBarros (@AnthonyDB), the senior database editor at USA Today, were both important sources of historical perspective for my feature on how data journalism is evolving from "computer-assisted reporting" (CAR) to a powerful Web-enabled practice that uses cloud computing, machine learning and algorithms to make sense of unstructured data.

The latter halves of our interviews, which focused upon their personal and professional experience, follow.

What data journalism project are you the most proud of working on or creating?

DeBarros: "In 2006, my USA TODAY colleague Robert Davis and I built a database of 620 students killed on or near college campuses and mined it to show how freshmen were uniquely vulnerable. It was a heart-breaking but vitally important story to tell. We won the 2007 Missouri Lifestyle Journalism Awards for the piece, and followed it with an equally wrenching look at student deaths from fires."

Cohen: "I'd have to say the Pulitzer-winning series on child deaths in DC, in which we documented that children were dying in predictable circumstances after key mistakes by people who knew that their
agencies had specific flaws that could let them fall through the cracks.

I liked working on the Post's POTUS Tracker and Head Count. Those were Web projects that were geared at accumulating lots of little bits about Obama's schedule and his appointees, respectively, that we could share with our readers while simultaneously building an important dataset for use down the road. Some of the Post's Solyndra and related stories, I have heard, came partly from studying the president's trips in POTUS Tracker.

There was one story, called "Misplaced Trust," on DC's guardianship
system, that created immediate change in Superior Court, which was
gratifying. "Harvesting Cash," our 18-month project on farm subsidies, also helped point out important problems in that system.

The last one, I'll note, is a piece of a project I worked on,
in which the DC water authority refused to release the results of a
massive lead testing effort, which in turn had shown widespread
contamination. We got the survey from a source, but it was on paper.

After scanning, parsing, and geocoding, we sent out a team of reporters to
neighborhoods to spot check the data, and also do some reporting on the
neighborhoods. We ended up with a story about people who didn't know what
was near them.

We also had an interesting experience: the water
authority called our editor to complain that we were going to put all of
the addresses online -- they felt that it was violating peoples' privacy,
even though we weren't identifyng the owners or the residents. It was more
important to them that we keep people in the dark about their blocks. Our
editor at the time, Len Downie, said, "you're right. We shouldn't just put
it on the Web." He also ordered up a special section to put them all in
print.

Where do you turn to keep your skills updated or learn new things?

Cohen: "It's actually a little harder now that I'm out of the newsroom,
surprisingly. Before, I would just dive into learning something when I'd
heard it was possible and I wanted to use it to get to a story. Now I'm
less driven, and I have to force myself a little more. I'm hoping to start
doing more reporting again soon, and that the Reporters' Lab will help
there too.

Lately, I've been spending more time with people from other
disciplines to understand better what's possible, like machine learning
and speech recognition at Carnegie Mellon and MIT, or natural language
processing at Stanford. I can't DO them, but getting a chance to
understand what's out there is useful. NewsFoo, SparkCamp and NICAR are
the three places that had the best bang this year. I wish I could have
gone to Strata, even if I didn't understand it all."

DeBarros: For surveillance, I follow really smart people on Twitter and have several key Google Reader subscriptions.

To learn, I spend a lot of time training after work hours. I've really been pushing myself in the last couple of years to up my game and stay relevant, particularly by learning Python, Linux and web development. Then I bring it back to the office and use it for web scraping and app building.

Why are data journalism and "news apps" important, in the context of the contemporary digital environment for information?

Cohen: "I think anything that gets more leverage out of fewer people is
important in this age, because fewer people are working full time holding
government accountable. The news apps help get more eyes on what the
government is doing by getting more of what we work with and let them see
it. I also think it helps with credibility -- the 'show your work' ethos --
because it forces newsrooms to be more transparent with readers / viewers.

For instance, now, when I'm judging an investigative prize, I am quite
suspicious of any project that doesn't let you see each item, I.e., when
they say, "there were 300 cases that followed this pattern," I want to see
all 300 cases, or all cases with the 300 marked, so I can see whether I
agree.

DeBarros: "They're important because we're living in a data-driven culture. A data-savvy journalist can use the Twitter API or a spreadsheet to find news as readily as he or she can use the telephone to call a source. Not only that, we serve many readers who are accustomed to dealing with data every day -- accountants, educators, researchers, marketers. If we're going to capture their attention, we need to speak the language of data with authority. And they are smart enough to know whether we've done our research correctly or not.

As for news apps, they're important because -- when done right -- they can make large amounts of data easily understood and relevant to each person using them."

These interviews were edited and condensed for clarity.

Around the globe, the bond between data and journalism is growing stronger. In an age of big data, the growing importance of data journalism lies in the ability of its practitioners to provide context, clarity and, perhaps most important, find truth in the expanding amount of digital content in the world. In that context, data journalism has profound importance for society.

To learn more about the people who are doing this work and, in some cases, building the newsroom stack for the 21st century, I conducted a series of email interviews during the 2012 NICAR Conference. This interview followed the conference and featured a remote participant who diligently used social media and the World Wide Web to document and share the best of NICAR:

The #nicar12 tips, tools & links page has new mapping, info design & election presos & references: j.mp/y5XD7f /cc @kn0wtheory
— Chrys Wu (@MacDiva) February 24, 2012

Chrys Wu (@MacDiva) is a data journalist and user engagement strategist based in New York City. Our interview follows.

Where do you work now? What is a day in your life like?

I work with clients through my company, Matchstrike, which specializes in user engagement strategy. It's a combination of user experience research, design and program planning. Businesses turn to me to figure out how to keep people's attention, create community and tie that back to return on investment.

I also launch Hacks/Hackers chapters around the world and co-organize the group in New York with Al Shaw of ProPublica and Jacqui Cox of The New York Times.

Both things involve seeking out people and ideas, asking questions, reading, wireframing and understanding what motivates people as individuals and as groups.

How did you get started in data journalism? Did you get any special degrees or certificates?

I had a stats class in high school with a really terrific instructor who also happened to be the varsity basketball coach. He was kind of like our John Wooden. Realizing the importance of statistics, being able to organize and interpret data — and learning how to be skeptical of claims (e.g., where "4 out of 5 dentists agree" comes from)— has always stayed with me.

Other than that class and studying journalism at university, what I know has come from exploring (finding what's out there), doing (making something) and working (making something for money). I think that's pretty similar to most journalists and journalist-developers currently in the field.

Though I've spent several years in newsrooms (most notably with the Los Angeles Times and CBS Digital Media Group), most of my journalism and communications career has been as a freelancer. One of my earliest clients specialized in fundraising for Skid Row shelters. I quantified the need cases for her proposals. That involved working closely with the city health and child welfare departments and digging through a lot of data.

Once I figured that out, it was important to balance the data with narrative. Numbers and charts have a much more profound impact on people if they're framed by an idea to latch onto and compelling story to share.

Did you have any mentors? Who? What were the most important resources they shared with you?

I don't have individual mentors, but there's an active community with a huge body of work out there to learn from. It's one of the reasons why I've been collecting things on Delicious and Pinboard, and it's why I try my best to put everything that's taught at NICAR on my blog.

I always try look beyond journalism to see what people are thinking about and doing in other fields. Great ideas can come from everywhere. There are lots of very smart people willing to share what they know.

What does your personal data journalism "stack" look like? What tools could you not live without?

I use Coda and TextMate most often. For wireframing, I'm a big fan of OmniGraffle. I code in Ruby, and a little bit in Python. I'm starting to learn how to use R for dataset manipulation and for its maps library.

For keeping tabs on new but not urgent-to-read material, I use my friend Samuel Clay's RSS reader, Newsblur.


What data journalism project are you the most proud of working on or creating?

I'm most proud of working with the Hacks/Hackers community. Since 2009, we've grown to more than 40 groups worldwide, with each locality bringing journalists, designers and developers together to push what's possible for news.

As I say, talking is good; making is better — and the individual Hacks/Hackers chapters have all done some version of that: presentations, demos, classes and hack days. They're all opportunities to share knowledge, make friends and create new things that help people better understand what's happening around them.

Where do you turn to keep your skills updated or learn new things?

MIT's open courses have been great. There's also blogs, mailing lists, meetups, lectures and conferences. And then there's talking with friends and people they know.

Why are data journalism and "news apps" important, in the context of the contemporary digital environment for information?

I like Amanda Cox's view of the importance of reporting through data. She's a New York Times graphics editor who comes from a statistics background. To paraphrase: Presenting a pile of facts and numbers without directing people toward any avenue of understanding is not useful.

Journalism is fundamentally about fact-finding and opening eyes. One of the best ways to do that, especially when lots of people are affected by something, is to interweave narrative with quantifiable information.

Data journalism and news apps create the lens that shows people the big picture they couldn't see but maybe had a hunch about otherwise. That's important for a greater understanding of the things that matter to us as individuals and as a society.

This interview has been edited and condensed for clarity.

If you've ever seen a child interact with an iPad, you've seen the power of the touch interface in action. Is this a sign of what's to come — will we be touching and swiping screens rather tapping buttons? I reached out to Josh Clark (@globalmoxie), founder of Global Moxie and author of "Tapworthy," to get his thoughts on the future of touch and computer interaction, and whether or not buttons face extinction.

Clark says a touch-based UI is more intuitive to the way we think and act in the world. He also says touch is just the beginning — speech, facial expression, and physical gestures are on they way, and we need to start thinking about content in these contexts.

Clark will expand on these ideas at Mini TOC Austin on March 9 in Austin, Texas.

Our interview follows.

Are we close to seeing the end of buttons?

Josh Clark: I frequently say that buttons are a hack, and people sometimes take that the wrong way. I don't mean it in a particularly negative way. I think buttons are an inspired hack, a workaround that we've needed just to get stuff done. That's true in the real world as well as the virtual: A light switch over here to turn on a light over there isn't especially intuitive, and it's something that has to be learned, and re-learned for every room we walk into. That light switch introduces a middle man, a layer of separation between the action and the thing you really want to work on, which is the light. The switch is a hack, but a brilliant one because it's just not practical to climb up a ladder in a dark room to screw in the light bulb.

Buttons in interfaces are a similar kind of hack — an abstraction we've needed to make the desktop interface work for 30 years. The cursor, the mouse, buttons, tabs, menus ... these are all prosthetics we've been using to wrangle content and information.

With touchscreen interfaces, though, designers can create the illusion of acting on information and content directly, manipulating it like a physical object that you can touch and stretch and drag and nudge. Those interactions tickle our brains in different ways from how traditional interfaces work because we don't have to process that middle layer of UI conventions. We can just touch the content directly in many cases. It's a great way to help cut through complexity.

The result is so much more intuitive, so much more natural to the way we think and act in the world. The proof is how quickly people with no computing experience — people like toddlers and seniors — take so quickly to the iPad. They're actually better with these interfaces than the rest of us because they aren't poisoned by 30 years of desktop interface conventions. Follow the toddlers; they're better at it than we are.

So, yes, in some contexts, buttons and other administrative debris of the traditional interface have run their course. But buttons remain useful in some contexts, especially for more abstract tasks that aren't easily represented physically. The keyboard is a great example, as are actions like "send to Twitter," which don't have readily obvious physical components. And just as important, buttons are labeled with clear calls to action. As we turn the corner into popularizing touch interactions, buttons will still have a place.

Mini TOC Austin — Being held March 9, 2012 — right before SXSW — O'Reilly Tools of Change presents Mini TOC Austin, a one-day event focusing on Austin's thriving publishing, tech, and bookish-arts community.

Register to attend Mini TOC Austin

What kinds of issues do touch- and gesture-oriented interfaces present?

Josh Clark: There are issues for both designers and users. In general, if a touchscreen element looks or behaves like a physical object, people will try to interact with it like one. If your interface looks like a book, people will try to turn its pages. For centuries, designers have dressed up their designs to look like physical objects, but that's always just been eye candy in the past. With touch, users approach those designs very differently; they're promises about how the interface works. So designers have to be careful to deliver on those promises. Don't make your interface look like a book, for example, if it really works through desktop-like buttons. (I'm looking at you, Contacts app for iPad.)

So, you can create really intuitive interfaces by making them look or behave like physical objects. That doesn't mean that everything has to look just like a real-world object. Windows Phone and the forthcoming Windows 8 interface, for example, use a very flat tile-like metaphor. It doesn't look like a 3-D gadget or artifact, but it does behave with real-world physics. It's easy to figure out how to slide and work the content on the screen. People figure that stuff out really quickly.

The next hurdle — and the big opportunity for touch interfaces — is moving to more abstract gestures: two- and three-finger swipes, a full-hand pinch, and so on. In those cases, gestures become the keyboard shortcuts of touch and begin to let you create applications that you play more than you use, almost like an instrument. But wait, here I am talking about abstract gestures; didn't I just say that abstractions — like buttons — are less than ideal? Well, yeah, the trouble is you don't want to have the overhead of processing an interface, of thinking through how it works. The thing about physical abstractions (like gestures) versus visual abstractions (like buttons) is that physical actions can be absorbed into muscle memory. That kind of subconscious knowledge is actually much faster than visual processing — it's why touch typists are so much faster than people who visually peck at the keys. So, once you learn and absorb those physical actions — a two-finger swipe always does this or that — then you can actually move really quickly through an interface in the same way a pianist or a typist moves through a keyboard. Intent fluidly translated to action.

But how do you teach that stuff? Swiping a card, pinching a map, or tapping a photo are all based on actions we know from the physical world. But a two-finger swipe has no prior meaning. It's not something we'll guess. Gestures are invisible with no labels, so that means they have to be taught.

Screenshot from Apple's trackpad tutorial.

In what ways can UI design alleviate these learning issues?

Josh Clark: Designers should approach this by thinking through how we learn any physical action in the real world: observation of visual cues, demonstration, and practice. Too often, designers fall back on instruction manuals (iPad apps that open with a big screen of gesture diagrams) or screencasts. Neither are very effective.

Instead, designers have to do a better job of coaching people in context, showing our audiences how and when to use a gesture in the moment. More of us need to study video game design because games are great at this. In so many video games, you're dropped into a world where you don't even know what your goal is, let alone what you're capable of or what obstacles you might encounter. The game rides along with you, tracking your progress, taking note of what you've encountered and what you haven't, and giving in-context instruction, tips, and demonstrations as you go. That's what more apps and websites should do. Don't wait for people to somehow find a hidden gesture shortcut; tell people about it when they need it. Show an animation of the gesture and wait for them to copy it. Demonstration and practice — that's how we learn all physical actions, from playing an instrument to learning a tennis serve.

How do you see computer interaction evolving?

Josh Clark: It's a really exciting time for interaction design because so many new technologies are becoming mature and affordable. Touch got there a few years ago. Speech is just now arriving. Computer vision with face recognition and gesture recognition like Kinect are coming along. So, we have all these areas where computers are learning to understand our particularly human forms of communication.

In the past, we had to learn to act and think like the machine. At the command line, we had to write in the computer's language, not our own. The desktop graphical user interface was a big step forward in making things more humane through visuals, but it was still oriented around how computers saw the world, not humans. When you consider the additions of touch, speech, facial expression, and physical gesture, you have nearly the whole range of human (and humane) communication tools. As computers learn the subtleties of those expressions, our interfaces can become more human and more intuitive, too.

Touchscreens are leading this charge for now, but touch isn't appropriate in every context. Speech is obviously great for the car, for walking, for any context where you need your eyes elsewhere. We're going to see interfaces that use these different modes of communication in context-appropriate combinations. But that means we have to start thinking hard about how our content works in all these different contexts. So many are struggling just to figure out how to make the content adapt to a smaller screen. How about how your content sounds when spoken? How about when it can be touched, or how it should respond to physical gestures or facial expressions? There's lots of work ahead.

Are Google's rumored heads-up-display glasses a sign of things to come?

Josh Clark: I'm sure that all kinds of new displays will have a role in the digital future. I'm not especially clever about figuring out which technology will be a huge hit. If someone had told me five years ago that the immediate future would be all about a glass phone with no buttons, I'd have said they were nuts. I think both software and context and, above all, human empathy make the difference in how and when a hardware technology becomes truly useful. The stuff I've seen of the heads-up-display glasses se[…]

Around the globe, the bond between data and journalism is growing stronger. In an age of big data, the growing importance of data journalism lies in the ability of its practitioners to provide context, clarity and, perhaps most important, find truth in the expanding amount of digital content in the world. In that context, data journalism has profound importance for society.

To learn more about the people who are doing this work and, in some cases, building the newsroom stack for the 21st century, I conducted a series of email interviews during the 2012 NICAR Conference.

Meghan Hoyer (@MeghanHoyer) is a data editor based in Virginia. Our interview follows.

Where do you work now? What is a day in your life like?

I work in an office within The Virginian Pilot’s newsroom. I’m a one-person team, so there’s no such thing as typical.


What I might do: Help a reporter pull Census data, work with IT on improving our online crime report app, create a DataTable of city property assessment changes, and plan training for a group of co-workers who’d like to grow their online skills. At least, that’s what I’m doing today.

Tomorrow, it’ll be helping with our online election report, planning a strategy to clean a dirty database, and working with a reporter to crunch data for a crime trend story.

How did you get started in data journalism? Did you get any special degrees or certificates?

I have a journalism degree from Northwestern, but I got started the same way most reporters probably got started - I had questions about my community and I wanted quantifiable answers. How had the voting population in a booming suburb changed? Who was the region’s worst landlord? Were our localities going after delinquent taxpayers? Anecdotes are nice, but it’s an amazingly powerful thing to be able to get the true measure of a situation. Numbers and analysis help provide a better focus - and sometimes, they upend entirely your initial theory.

Did you have any mentors? Who? What were the most important resources they shared with you?

I haven’t collected a singular mentor as much as a group of people whose work I keep tabs on, for inspiration and follow-up. The news community is pretty small. A lot of people have offered suggestions, guidance, cheat sheets and help over the years. Data journalism - from analysis to building apps -- is definitely not something you can or need to learn in a bubble all on your own.

What does your personal data journalism "stack" look like? What tools could you not live without?

In terms of daily tools, I keep it basic: Google docs, Fusion Tables and Refine, QGIS, SQLite and Excel are all in use pretty much every day.

I’ve learned some Python and JavaScript for specific projects and to automate some of the newsroom’s daily tasks, but I definitely don’t have the programming or technical background that a lot of people in this field have. That’s left me trying to learn as much as I can as quick as I can.

In terms of a data stack, we keep information such as public employee salaries, land assessment databases and court record databases (among others) updated in a shared drive in our newsroom. It’s amazing how often reporters use them, even if it’s just to find out which properties a candidate owns or how long a police officer caught at a DUI checkpoint has been on the force.

What data journalism project are you the most proud of working on or creating?

I'm proud of using regional records to do an analysis which forced Norfolk to revamp its whole real estate tax collection process.

A few years ago, I combined property ownership records, code enforcement citations, real estate tax records and rental inspection information from all our local cities into "Cashing Blight -- and found a company with hundreds of derelict properties.

Their properties seemed to change hands often, so a partner and I then hand-built a database from thousands of land deeds that proved the company was flipping houses among investors in a $26 million mortgage fraud scheme. None of the cities in our region had any idea this was going on because they were dealing with each parcel as a separate entity.

That’s what combining sets of data can get you - a better overall view of what’s really happening. While government agencies are great at collecting piles of data, it’s that kind of larger analysis that’s missing.

Where do you turn to keep your skills updated or learn new things?

To be honest - Twitter. I get a lot of ideas and updates on new tools there. And the NICAR conference and listserv. Usually when you hit up against a problem - whether it’s dealing with a dirty dataset or figuring out how to best visualize your data -- it’s something that someone else has already faced.

I also learn a lot from the people within our newsroom. We have a talented group of web producers who all are eager to try new things and learn.

Why are data journalism and "news apps" important, in the context of the contemporary digital environment for information?

Data is everywhere, but in most cases it’s just stockpiled and warehoused without a second thought to analysis or using it to solve larger problems.

Journalists are in a unique position to make sense of it, to find the stories in it, to make sure that governments and organizations are considering the larger picture.

I think, too, that people in our field need to truly push for open government in the sense not of government building interfaces for data, but for just releasing raw data streams. Government is still far too stuck in the “Here’s a PDF of a spreadsheet” mentality. That doesn’t create informed citizens, and it doesn’t lead to innovative ways of thinking about government.

I’ve been involved recently in a community effort to create an API and then apps out of the regional transit authority’s live bus GPS stream. It has been a really fun project - and one I hope makes local governments rethink their practices.

Around the globe, the bond between data and journalism is growing stronger. In an age of big data, the growing importance of data journalism lies in the ability of its practitioners to provide context, clarity and, perhaps most important, find truth in the expanding amount of digital content in the world. In that context, data journalism has profound importance for society.

To learn more about the people who are doing this work and, in some cases, building the newsroom stack for the 21st century, I conducted a series of email interviews during the 2012 NICAR Conference.

Matt Stiles (@Stiles) , a data journalist based in Washington, D.C., maintains a popular Daily Visualization blog. Our interview follows.

Where do you work now? What is a day in your life like?

I work at NPR, where I oversee data journalism on the State Impact project, a local-national partnership between us and member stations. My typical day always begins with a morning "scrum" meeting among the D.C. team as part of our agile development process. I spend time acquiring and analyzing data throughout each data, and I typically work directly with reporters, training them on software and data visualization techniques. I also spend time planning news apps and interactives, a process that requires close consultation with reporters, designers and developers.

How did you get started in data journalism? Did you get any special degrees or certificates?

No special training or certificates, though I did attend three NICAR boot camps (databases, mapping, statistics) over the years.

Did you have any mentors? Who? What were the most important resources they shared with you?

I have several mentors, both on the reporting side and the data side. For data, I wouldn't be where I am today without the help of two people: Chase Davis and Jennifer LaFleur. Jen got me interested early, and has helped me with formal and informal training over the years. Chase helped me with day-to-day questions when we worked together at the Houston Chronicle.

What does your personal data journalism "stack" look like? What tools could you not live without?

I have a MacBook that runs Windows 7. I have the basic CAR suite (Excel/Access, ArcGIS, SPSS, etc.) but also plenty of open-source tools, such as R for visualization or MySQL/Postgres for databases. I use Coda and Text Mate for coding. I use BBEdit and Python for text manipulation. I also couldn't live without Photoshop and Illustrator for cleaning up graphics.

What data journalism project are you the most proud of working on or creating?

I'm most proud of the online data library I created (and others have since expanded) at The Texas Tribune, but we're building some sweet apps at NPR. That's only going to expand now that we've created a national news apps team, which I'm joining soon.

Where do you turn to keep your skills updated or learn new things?

I read blogs, subscribe to email lists and attend lots of conferences for inspiration. There's no silver bullet. If you love this stuff, you'll keep up.

Why are data journalism and "news apps" important, in the context of the contemporary digital environment for information?

More and more information is coming at us every day. The deluge is so vast. Data journalism at its core is important because it's about facts, not anecdotes.

Apps are important because Americans are already savvy data consumers, even if they don't know it. We must get them thinking -- or, even better, not thinking -- about news consumption in the same way they think about syncing their iPads or booking flights on Priceline or purchasing items on eBay. These are all "apps" that are familiar to many people. Interactive news should be, too.


This interview has been edited and condensed for clarity.

Around the globe, the bond between data and journalism is growing stronger. In an age of big data, the growing importance of data journalism lies in the ability of its practitioners to provide context, clarity and, perhaps most important, find truth in the expanding amount of digital content in the world. In that context, data journalism has profound importance for society.

To learn more about the people who are doing this work and, in some cases, building the newsroom stack for the 21st century, I conducted a series of email interviews during the 2012 NICAR Conference.

Michelle Minkoff (@MichelleMinkoff ) is an investigative developer/journalist based in Washington, D.C. Our interview follows.

Where do you work now? What is a day in your life like?

I am an Interactive Producer at the Associated Press' Washington DC bureau, where I focus on news applications related to politics and the election, as well as general mapping for our interactives on the Web. While my days pretty much always involve sitting in front of a computer, the actual tasks themselves can vary wildly. I may be chatting with reporters and editors in politics, environment, educational, national security or myriad beats about upcoming stories and how to use data to support reporting or create interactive stories. I might be gathering data, reformatting it or crafting Web applications. I spend a great deal of time creating interactive mapping systems, working a lot with geographic data, and collaborating with cartographers, editors and designers to decide how to best display it.

I split my time between working closely with my colleagues in the Washington bureau on the reporting/editing side, and my fellow interactive team members, only one of whom is also in DC. Our team is global, headquartered in New York, but with members spanning the globe from Phoenix to Bangkok.

It's a question of walking a balance between what needs to be done on daily deadlines for breaking news, longer-term stories which are often investigative, and creating frameworks that help The Associated Press to make the most of the Web's interactive nature in the long run.

How did you get started in data journalism? Did you get any special degrees or certificates?

I caught the bug when I took a computer-assisted reporting class from Derek Willis, a member of the New York Times' Interactive News Team, at Northwestern's journalism school where I was a grad student. I was fascinated by the role that technology could play in journalism for reporting and presentation, and very quickly got hooked. I also quickly discovered that I could lose track of hours playing with these tools, and that what came naturally to me was not as natural to others. I would spend days reporting for class, on and off Capitol Hill, and nights exchanging gchats with Derek and other data journalists he introduced me to. I started to understand SQL, advanced Excel, and fairly quickly thereafter, Python and Django.

I followed this up with an independent study in data visualization back at Medill's Chicago campus, under Rich Gordon. I practiced making Django apps, played with the Processing visualization language. I voraciously read through all the Tufte books. As a final project, I created a package about the persistence of Chicago art galleries that encompasses text, Flash visualization and a searchable database.

I have a concentration in Interactive Journalism, with my Medill masters' degree, but the courses mentioned above are but a partial component of that concentration.

Did you have any mentors? Who? What were the most important resources they shared with you?

The question here is in the wrong tense. I currently "do" have many mentors, and I don't know how I would do my job without what they've shared in the past, and in the present. Derek, mentioned above, was the first. He introduced me to his friend Matt [Waite], and then he told me there was a whole group of people doing this work at NICAR. Literally hundreds of people from that organization have helped me at various places on my journey, and I believe strongly in the mantra of "paying it forward" as they have -- no one can know it all, so we pass on what we've learned, so more people can do even better work.

Other key folks I've had the privilege to work with include all of the Los Angeles Times' Data Desk's members, which includes reporters, editors and Web developers. I worked most closely with Ben Welsh and Ken Schwencke, who answered many questions, and were extremely encouraging when I was at the very beginning of my journey.

At my current job at The Associated Press, I'm lucky to have teammates who mentor me in design, mapping and various Washington-based beats. Each is helpful in his or her own way.


Special attention deserves to be called to Jonathan Stray, who's my official boss, but also a fantastic mentor who enables me to do what I do. He's helping me to learn the appropriate technical skills to execute what I see in my head, as well as learn how to learn. He's not just teaching me the answers to the problems we encounter in our daily work, but also helping me learn how to better solve them, and work this whole "thing I do" into a sustainable career path. And all with more patience than I have for myself.

What does your personal data journalism "stack" look like? What tools could you not live without?

No matter how advanced our tools get, I always find myself coming back to Excel first to do simple work. It helps us an overall handle on a data set. I also will often quickly bring data into SQLite, a Firefox extension that allows a user to run SQL queries, with no database setup. I'm more comfortable asking complicated questions of data that way. I also like to use Google's Chart Tools to create quick visualizations for myself to better understand a story.

When it comes to presentation, since I've been doing a lot with mapping recently, I don't know what I'd do without my favorite open source tools, Tilemill and Leaflet. Building a map stack is hard work, but the work that others have done before it have made it a lot easier.

If we consider programming languages tools (which I do), JavaScript is my new Swiss army knife. Prior to coming to the AP, I did a lot with Python and Django, but I've learned a lot about what I like to call "Really Hard JavaScript." It's not just about manipulating the colors of a background on a Web page, but parsing, analyzing and presenting data. When I need to do more complex work to manipulate data, I use a combination of Ruby and Python -- depending on which has better tools for the job. For XML parsing, I like Ruby more. For simplifying geo data, I prefer Python.

What data journalism project are you the most proud of working on or creating?

That would be " Road to 270", a project we did at the AP that allows users to test out hypothetical "what-if" scenarios for the national election, painting states to define to which candidate a state's delegates could go. It combines demographic and past election data with the ability for users to make a choice and deeply engage with the interactive. It's not just telling the user a story, but informing the user by allowing him or her to be part of the story. That, I believe, is when data journalism becomes its most compelling and informative.

It also uses some advanced technical mapping skills that were new to me. I greatly enjoyed the thrill of learning how to structure a complex application, and add new tools to my toolkit. Now, I don't just have those new tools, but a better understanding of how to add other new tools.

Where do you turn to keep your skills updated or learn new things?

I look at other projects, both within the journalism industry and in general visualization communities. The Web inspector is my best friend. I'm always looking to see how people did things. I read blogs voraciously, and have a fairly robust Google Reader set of people whose work I follow closely. I also use lynda.com frequently (I tend to learn best by video tutorials.) Hanging out on listservs for free tools I use (such as Leaflet), programming languages I care about (Python), or projects whose mission our work is related to (Sunlight Foundation) help me engage with a community that cares about similar issues.

Help sites like Stack Overflow, and pretty much anything I can find on Google, are my other best friends. The not-so-secret secret of data journalism: we're learning as we go. That's part of what makes it so fun.

Really, the learning is not about paper or electronic resources. Like so much of journalism, this is best conquered, I argue, with persistence and stick-to-it-ness. I approach the process of data journalism and Web development as a beat. We attend key meetings. Instead of city council, it's NICAR. We develop vast rolodexes. I know people who have myriad specialties and feel comfortable calling on them. In return, I help people all over the world with this sort of work whenever I can, because it's that important. While we may work for competing places, we're really working toward the same goal: improving the way we inform the public about what's going on in our world. That knowledge matters a great deal.

Why are data journalism and "news apps" important, in the context of the contemporary digital environment for information?

More and more information is coming at us every day. The deluge is so vast that we need to not just say things are true, but prove those truths with verifiable facts. Data journalism allows for great specificity, and truths based in the scientific method. Using computers to commit data journalism allows us to process great amounts of information much more efficiently, and make the world more comprehensible to a user.

Also, while we are working with big data, often only a subset of that data is valuable to a specific user. Data journalism and Web deve[…]

Around the globe, the bond between data and journalism is growing stronger. In an age of big data, the growing importance of data journalism lies in the ability of its practitioners to provide context, clarity and, perhaps most important, find truth in the expanding amount of digital content in the world. In that context, data journalism has profound importance for society.

To learn more about the people who are doing this work and, in some cases, building the newsroom stack for the 21st century, I conducted a series of email interviews during the 2012 NICAR Conference.

Ben Welsh (@palewire) is an Web developer and journalist based in Los Angeles. Our interview follows.


Where do you work now? What is a day in your life like?

I work for the Los Angeles Times, a daily
newspaper and 24-hour Web site based in Southern California. I'm a member
of the Data Desk, a team of reporters and
Web developers that specializes in maps, databases, analysis and
visualization. We both build Web applications and conduct analysis for
reporting projects.

I like to compare The Times to a factory, a factory that makes information.
Metaphorically speaking, it has all sorts of different assembly lines. Just
to list a few, one makes beautifully rendered narratives, another makes battleship-like investigative projects.

A typical day involves juggling work on difference projects, mentally
moving from one assembly line to the other. Today I patched an embryonic open-source release, discussed our next move on a pending public records request, guided the real-time publication of results from the GOP primaries in Michigan and Arizona, and did some preparation for how we'll present a larger dump of results on Super Tuesday.

How did you get started in data journalism? Did you get any special
degrees or certificates?

I'm thrilled to see new-found interest in "data journalism" online. It's
drawing young, bright people into the field and involving people from
different domains. But it should be said that the idea isn't new.

I was initiated into the field as a graduate student at the Missouri School
of Journalism. There I worked at the National Institute for Computer-Assisted Reporting , also known as NICAR. Decades before anyone called it "data journalism," a disparate group of misfit reporters discovered that the data analysis made possible by computers enabled them to do more powerful investigative reporting. In 1989, they founded NICAR, which has, for decades, been training data skills
to journalists and nurtured a tribe of journalism geeks. In the time since, computerized data analysis has become a dominant force in investigative reporting, responsible for a large share of the field's best work.

To underscore my point, here's a 1986 Time magazine article about how
"newsmen are enlisting the machine."

Did you have any mentors? Who? What were the most important resources they
shared with you?

My first journalism job was in Chicago. I got a gig working for two great people there, Carol Marin and Don Moseley, who have spent most of their careers as television journalists. I worked as their assistant. Carol and Don are warm people who are good teachers, but they are also excellent at what they do. There was a moment when I realized, "Hey, I can do this!" It wasn't just something I heard about in class, but I could actually see myself doing.

At Missouri, I had a great classmate named Brian
Hamman, who is now at the New York Times. I remember seeing how invested Brian was in the Web, totally committed to Web development as a career path. When an opportunity opened up to be a graduate assistant at NICAR, Brian encouraged me to pursue it. I learned enough SQL to help do farmed-out investigative work for TV stations. And, more importantly, I learned that if you had technical skills you could get the job to work on a cool story.

After that I got a job doing data analysis at the Center for Public Integrity in Washington DC. I had the opportunity to work on investigative projects, but also the chance to learn a lot of computer programming along the way. I had the guidance of my talented coworkers, Daniel Lathrop, Agustin Armendariz, John Perry, Richard Mullins and Helena Bengtsson. I learned that computer programming wasn't impossible. They taught me that if you have a manageable task, a few friends to help you out and a door you can close, you can figure out a lot.

What does your personal data journalism "stack" look like? What tools
could you not live without?

I do my daily development in gedit text editor, Byobu's slick implementation of the screen terminal and the Chromium browser. And, this part may be hard to believe, but I love Ubuntu
Unity. I don't understand what everybody is complaining about.

I do almost all of my data management in the Python Web development
framework Django and
PostgreSQL's database, even if
the work is an exploratory reporting project that will never be published. I find that the structure of the framework can be useful for organizing just about any data-driven project.

I use GitHub for both version-control and
project management. Without it, I'd be lost.

What data journalism project are you the most proud of working on or
creating?

As we all know, there's a lot of data out there. And, as anyone who works
with it knows, most of it is crap. The projects I'm most proud of have
taken large, ugly data sets and refined them into something worth knowing:
a nut graf in an investigative story, or a
data-driven app that gives the reader some new
insight into the world around them. It's impossible to pick one. I like to
think the best is still, as they say in the newspaper business,
TK.

Where do you turn to keep your skills updated or learn new things?

Twitter is a great way to keep up with what is getting other programmers excited. I know a lot of people find social media overwhelming or distracting, but I feel plugged in and inspired by what I find there. I wouldn't want to live without it.

GitHub is another great source. I've learned so much just exploring other
people's code. It's invaluable.

Why are data journalism and "news apps" important, in the context of the
contemporary digital environment for information?

Computers offer us an opportunity to better master information, better
understand each other and better watchdog those who would govern us. I
tried to talk about some of the ways simply thinking about the process of
journalism as an algorithm can point the way at last week's NICAR
conference in a talk called "Human-Assisted Reporting." In my opinion, we should aspire to write code that embodies the idealistic principles and investigative methods of the previous generation. There's all this data out there now, and journalistic algorithms, "robot
reporters," can help us ask it tougher questions.

The promise of data journalism was a strong theme throughout the National Institute for Computer-Assisted Reporting's (NICAR) 2012 conference. In 2012, making sense of big data through narrative and context, particularly unstructured data, will be a central goal for data scientists around the world, whether they work in newsrooms, Wall Street or Silicon Valley. Notably, that goal will be substantially enabled by a growing set of common tools, whether they're employed by government technologists opening Chicago, healthcare technologists or newsroom developers.

At NICAR 2012, you could literally see the code underpinning the future of journalism written - or at least projected - on the walls.


"The energy level was incredible," said David Herzog, associate professor for
print and digital news at the Missouri School of Journalism, in an email interview after NICAR. "I didn't see participants wringing their hands and worrying about the future of journalism. They're too busy building it."

Just as open civic software is increasingly baked into government, open source is playing a pivotal role in the new data journalism.

"Free and open-source tools dominated," said Herzog. "It's clear from the panels and hands-on classes that free and open source tools have eliminated the barrier to entry in terms of many software costs."

While many developers are agnostic with respect to which tools they use to get a job done, the people who are building and sharing tools for data journalism are often doing it with open source code. As Dan Sinker, the head of the Knight-Mozilla News Technology Partnership for Mozilla, wrote afterwards, journo-coders took NICAR 12 "to a whole new level."

The PANDA Project officially launched in beta in St. Louis, including a provisioning party. You can check out PANDA on Github.
The Associated Press' Overview Project is going to make digging through documents easier.
The LA Times Datadesk shared a tool, Django Bakery, to turn Django applications into flat HTML files.
Jonathan Soma, with help from John Keefe of WNYC, built Tabletop.js as a way to use a public Google Spreadsheet as a source of data for a Web app. Tabletop is on Github too.

While some of that open source development was definitely driven by the requirements of the Knight News Challenge, which funded the PANDA and Overview projects, there's also a collaborative spirit in evidence throughout this community.

This is a group of people who are fiercely committed to "showing your work" -- and for newsroom developers, that means sharing your code. To put it another way, code, don't tell. Sessions on Python, Django, mapping, Google Refine and Google Fusion tables were packed at NICAR 12.

Journalists want to learn python: #nicar12 twitter.com/MeghanHoyer/st…
— Meghan Hoyer (@MeghanHoyer) February 24, 2012

No, this is not your father's computer-assisted reporting.

"I thought this stacked up as the best NICAR conference since the first in 1993," said Herzog. "It's always been tough to choose from the menu of panels, demos and hands-on classes at NICAR conferences. But I thought there was an abundance of great, informative, sessions put on by the participants. Also, I think NICAR offered a good range of options for newbies and experts alike. For instance, attendees could learn how to map using Google Fusion tables on the beginner's end, or PostGIS and qGIS at the advanced level. Harvesting data through web scraping has become an ever bigger deal for data journalists. At the same time, it's getting easier for folks with no or little programming chops to scrape using tools like spreadsheets, Google Refine and ScraperWiki. "

We've scraped a webpage using a simple chrome extension with @michelleminkoff at #nicar12 twitter.com/ChrisLKeller/s…
— Chris Keller (@ChrisLKeller) February 25, 2012

On the history of NICAR

According to IRE, NICAR was founded in 1989. Since its founding, the Institute has trained thousands of journalists how to find, collect and public electronic information.

Today, "the NICAR conference helps journalists, hackers, and developers figure out best practices, best methods,and best digital tools for doing journalism that involves data analysis and classic reporting in the field," said Brant Houston, former executive director of Investigative Reporters and Editors, in an email interview. "The NICAR conference also obviously includes investigative journalism and the standards for data integrity and credibility."

"I believe the first IRE-sponsored [conference] was in 1993 in Raleigh, when a few reporters were trying to acquire and learn to use spreadsheets, database
managers, etc. on newly open electronic records," said Sarah Cohen, the Knight professor of the practice of journalism and public policy at Duke University, in an email interview. "Elliott Jaspin was going around the country teaching reporters how to get data off of 9-track tapes. There really was no public Internet. At the time, it was really, really hard to use the new PC's, and a few reporters were trying to find new stories. The famous ones had been Elliott's school bus drivers who had drunk driving records and the Atlanta Color of Money series on redlining."

"St. Louis was my 10th NICAR conference," said Anthony DeBarros, the senior database editor at USA Today, in an email interview. "My first was in 1999 in Boston. The conference is a place where news nerds can gather and remind themselves that they're not alone in their love of numbers, data analysis, writing code and finding great stories by poring over columns in a spreadsheet. It serves as an important training vehicle for journalists getting started with data in the newsroom, and it's always kept journalists apprised of technological developments that offer new ways of finding and telling stories. At the same time, its connection to IRE keeps it firmly rooted in the best aspects of investigative reporting -- digging up stories that serve the public good.

Baby, you can drive my CAR

Long before we started talking about "data journalism," the practice of computer-assisted reporting (CAR) was growing around the world.

"The practice of CAR has changed over time as the tools and environment in the digital world has changed," said Houston. "So it began in the time of mainframes in the late 60s and then moved onto PCs (which increased speed and flexibility of analysis and presentation) and then moved onto the Web, which accelerated the ability to gather, analyze and present data. The basic goals have remained the same. To sift through data and make sense of it, often with social science methods. CAR tends to be an "umbrella" term - one that includes precision journalism and data driven journalism and any methodology that makes sense of date such as visualization and effective presentations of data."

On one level, CAR is still around because the journalism world hasn't coined a good term to use instead.

"Computer-assisted reporting" is an antiquated term, but most people who practice it have recognized that for years," said DeBarros. "It sticks around because no one has yet to come up with a dynamite replacement. Phil Meyer, the godfather of the movement, wrote a seminal book called "Precision Journalism, and that term is a good one to describe that segment of CAR that deals with statistics and the use of social science methods in newsgathering. As an umbrella term, data journalism seems to be the best description at the moment, probably because it adequately covers most of the areas that CAR has become -- from traditional data-driven reporting to the newer category of news applications."

The most significant shift in CAR may well be when all of those computers being used for reporting were connected through the network of networks in the 1990s.

"It may seem obvious, but of course the Internet changed it all, and for a while it got smushed in with trying to learn how to navigate the Internet for stories, and how to download data," said Cohen. "Then there was a stage when everyone was building internal intranets to deliver public records inside newsrooms to help find people on deadline, etc. So for much of the time, it was focused on reporting, not publishing or presentation. Now the data journalism folks have emerged from the other direction: People who are using data obtained through APIs who often skip the reporting side, and use the same techniques to deliver unfiltered information to their readers in an easier format the the government is giving us. But I think it's starting to come back together -- the so-called data journalists are getting more interested in reporting, and the more traditional CAR reporters are interested in getting their stories on the web in more interesting ways.

Whatever you call it, the goals are still the same.

"CAR has always been about using data to find and tell stories," said DeBarros. "And it still is. What has changed in recent years is more emphasis toward online presentations (interactive maps and applications) and the coding skills required to produce them (JavaScript, HTML/CSS, Django, Ruby on Rails). Earlier NICAR conferences revolved much more around the best stories of the year and how to use data techniques to cover particular topics and beats. That's still in place. But more recently, the conference and the practice has widened to include much more coding and presentation topics. That reflects the state of media -- every newsroom is working overtime to make its content work well on the web, on mobile, and on apps, and data journalists tend to be forward thinkers so it's not surprising that the conference would expand to include those topics."

What stood out at NICAR 2012?

The tools and tactics on display at NICAR were enough to convince Tyler Dukes at Duke to write that NICAR taught me I know nothing. Browse through the tools, slides and links from NICAR 2012 curated by Chrys Wu to get a sense of just h[…]

At TOC, you're as likely to run into media professionals, entrepreneurs and innovators as you are publishers, booksellers and others working in traditional publishing. This, in turn, makes the underlying themes as varying and diverse as the attendees. This is the fourth in a series, taking a look at five themes that permeated interviews, sessions and/or keynotes at this year's show. The complete series will be posted here.

Discovering how readers discover books and other types of content is becoming more and more important as the digital era ushers in a multitude of avenues for readers to find books, blogs, news, websites, etc. At this year's TOC show, the issue of discovery was a major theme, and in many discussions, data and discovery went hand in hand.

Otis Chandler, founder and CEO of Goodreads.com, presented a data-driven analysis of how people find books in the "How Consumers Discover Books Online" session. He defined discovery as "all the touchpoints in the mind of the consumer it takes to get them to decide to purchase and read a book." Using Goodreads' data as a basis, he shared the ways people discover books:

Tied with people adding books during Goodreads registration, "Search" was the number one method of discovery. And Chandler noted, "those users had heard of the books elsewhere." Goodreads conducted a survey to find out exactly how readers found those books:

Chandler also noted discovery differences between types of readers. For instance, avid readers are less likely to find books through friend recommendations than casual readers, but more likely than casual readers to find books through Goodreads' recommendations. Different genres had unique results as well — romance, fantasy, SciFi, and YA readers were more likely find books by browsing genres and lists, for instance.

The slide presentation from the session can be found here.

The "Beyond 'Discovery' — Understanding The True Potential Of An Insight-oriented Publishing Environment" session addressed the issue of discovery from a data standpoint as well, in terms of how data is used to facilitate discovery. Presenters Kristen McLean, founder and CEO of Bookigee, and Kelly Gallagher, vice president of publishing services at RR Bowker, took a look at how data used to be treated and how data needs to be used now to be effective. This is what Data & Information 1.0 looked like:

And this is what Data & Information 2.0 looks like:

Data & Information 2.0 takes the data and analysis, puts it into context and uses it to achieve more effective discovery and sales:

The session also reviewed some tools and insights to help content creators better connect with readers and discussed the importance of engaging customers in real-time. The session slides can be found here as soon as they are available.


In a video interview, Linda Gagnon, senior vice president of Global Digital Markets at Baker & Taylor, addressed the issue of discovery from an ecosystem standpoint. She said "the key is going to be getting consumers to discover they can consume content on a multitude of devices — that's really where the future is: discovery and utilization of these new devices with content that they're used to." As to how less-avid readers can be brought into the ecosystem and how they can discover content, she responded:

"Over 70% of consumers have never bought a digital book. It's that 70% that we need to now attract. They haven't decided to buy a dedicated device because there's no need to expend the money for that because they just don't read that often. So, we need to find them where they live. The OEMs have a way to connect with them with the devices that [consumers] have for multi-purposes. The next step is to provide ways of discovering the content that is most important to them across those devices, so they're not tied into any specific ecosystem ... once they experience content that's relevant to them, I think that will drive additional purchases once they get hooked."

Gagnon's entire interview can be viewed in the following video:



If you couldn't make it to TOC, or you missed a session you wanted to see, sign up for the TOC 2012 Complete Video Compilation and check out our archive of free keynotes and interviews.

Related:

Hooked on context

The search for serendipitous recommendations

Goodreads chases the recommendation Holy Grail

The more you engage, the better the advice

More themes from TOC 2012

Peter Collingridge (@gunzalis), cofounder of Enhanced Editions says digital books are requiring a new style of data-driven marketing and promotion that publishers aren't yet implementing. He also says that book marketing is broken and big data is the solution.

In the following interview, Collingridge talks about how real-time data and analytics can help publishers and he shares insights from the beta period of Bookseer, a market intelligence service for books his company is developing.

What are some key findings from the Bookseer beta?

Peter Collingridge: I think despite the increasing awareness of data as being a critical tool for publishers to compete, it's genuinely hard for people to look at data as a natural addition to the work they are doing, whether that's in PR, marketing, acquisition, or pricing.

Publishing has operated in a well-defined way for a long time, where experience and intuition have dominated decision making and change is hard. What has been really exciting is that when people have the data in front of them, clearly showing the immediate impact of something they did — a link between cause and effect that they couldn't see before — they get really excited. We've had people talking about being "obsessed" and "addicted" to the data.

Some of the most surprising findings: That on some titles, big price changes aren't as relevant to volume as everyone thinks; that big-name glowing reviews of literary fiction don't have anywhere near the impact on sales to merit the effort; and that social media buzz almost never translates into sales.

For me, the key observations so far are around marketing. First, big budget media spending and ostentatious banner ads might impress authors and bookshops, but they deliver very poor return on investment (ROI) for sales. Secondly, the super-smart publishers are behaving like startups and doing tiny little pieces of very focused and cheap marketing — and watching the results like hawks before iterating in direct response to the data. Bookseer is designed to disclose the former and to aid the latter — and that is probably our biggest finding: it works!

Find out more about Bookseer in the following video from the If Book Then conference earlier this year in Milan.

What kinds of data are most important for publishers to track?

Peter Collingridge: Before we built Bookseer, we spoke with 25 people across the industry, including authors big, small and unpublished; editors and publishers; managing directors; digital directors; sales, marketing and PR directors; and literary agents. We asked exactly that question.

For most people, the data they had was pretty basic: Nielsen (which obviously only goes to the granularity of one week) plus the F5 button to manically refresh an Amazon web page for changes in sales rank. Neither of these is particularly helpful in determining the impact of an activity.

Of course, there are loads of data points, but we began with the lowest-hanging fruit. Aggregated sales (print and digital) across multiple sources; Amazon sales rank; price; best-seller charts; social media mentions; buzz; review coverage in mainstream and new media, and on social reading sites; and other factors such as promotion (advertising and other) and merchandising.

We think the most important thing to do is aggregate activity and data points across as many sources as possible, building a picture of what's going on for one title or across a whole retailer, and allowing publishers to draw their own conclusions.

What does real-time data let publishers do?

Peter Collingridge: Publishing has been B2B, about supplying books into bookshops, for forever — combined with working with media to support that. And for that world, weekly aggregated retail sales work, I guess. But when you're in a much faster-paced world, with the industry moving toward being consumer- rather than trade-facing, and with a fragmented retail and media landscape, you need to make decisions based on fact: What is the ROI on a £50,000 marketing campaign? Where do my banner ads have the best CTR? Who are the key influencers here — are they bloggers, mainstream media, or somewhere else? How many of our Twitter followers actually engage? When should we publish, in what format, and at what price?

Data should absolutely inform the answers to these questions. Furthermore, with a disciplined approach to promotion, where activities are separated from each other by a day or a few hours, real-time measurement can identify what works and what doesn't. We can identify the difference between Al Gore tweeting about a book and Tim O'Reilly doing the same; the difference between a Time review and a piece on CNN; the impact of a price drop against an email sent to 200,000 subscribers; and measure the exact ROI on a £300 campaign against a £30,000 one.

Over time, you build up a picture of which tactics work best and which don't. And immediate feedback allows you to hone your activities in real-time to what works best (particularly if you are A/B testing different approaches), or from a more strategic perspective, to plan out campaigns that have historically worked best for comparable titles.

How would you describe the relationship between sales and social media?

Peter Collingridge: Right now, sales drives social — not the other way round. However, I believe there will come a point when that's not the case, and we will be able to identify that.

This interview was edited and condensed.

Strata 2012 — The 2012 Strata Conference, being held Feb. 28-March 1 in Santa Clara, Calif., will offer three full days of hands-on data training and information-rich sessions. Strata brings together the people, tools, and technologies you need to make data work.
Save 20% on registration with the code RADAR20

Related:

Publishers: What are they good for?

Book Publicity and Marketing: How Soon was Yesterday?e

Three reasons why we're in a golden age of publishing entrepreneurship

Social data is an oracle waiting for a question

While reporters and editors have been the traditional vectors for information gathering and dissemination, the flattened information environment of 2012 now has news breaking first online, not on the newsdesk.

That doesn't mean that the integrated media organizations of today don't play a crucial role. Far from it. In the information age, journalists are needed more than ever to curate, verify, analyze and synthesize the wash of data.


To learn more about the shifting world of data journalism, I interviewed Liliana Bounegru (@bb_liliana), project coordinator of SYNC3, the first international Data Journalism Awards, and Data Driven Journalism at
the European Journalism Centre.

What's the difference between the data journalism of today and the computer-assisted reporting (CAR) of the past?

Liliana Bounegru: There is a "continuity and change" debate going on around the label "data journalism" and its relationship with previous journalistic practices that employ
computational techniques to analyze datasets.

Some
argue [PDF] that there is a difference between CAR and data
journalism. They say that CAR is a technique for gathering and analyzing data as a way of enhancing (usually investigative) reportage, whereas data journalism pays attention to the way that data
sits within the whole journalistic workflow. In this sense, data journalism pays equal attention to finding stories and to the data itself. Hence, we find the Guardian Datablog
or the Texas Tribune publishing datasets
alongside stories, or even just datasets by themselves for people to
analyze and explore.

Another difference is that in the past, investigative reporters
would suffer from a poverty of information relating to a question they
were trying to answer or an issue that they were trying to address.
While this is, of course, still the case, there is also an overwhelming
abundance of information that journalists don't necessarily know what
to do with. They don't know how to get value out of data. As Philip Meyer recently wrote to
me: "When information was scarce, most of our efforts were devoted to
hunting and gathering. Now that information is abundant, processing is
more important."

On the other hand, some argue that there is no difference between
data journalism and computer-assisted reporting. It is by now common
sense that even the most recent media practices have histories as
well as something new in them. Rather than debating whether or not
data journalism is completely novel, a more fruitful position would be
to consider it as part of a longer tradition but responding to new
circumstances and conditions. Even if there might not be a difference
in goals and techniques, the emergence of the label "data journalism"
at the beginning of the century indicates a new phase wherein the
sheer volume of data that is freely available online combined with
sophisticated user-centric tools enables more people to work with more
data more easily than ever before. Data journalism is about mass data
literacy.

Strata 2012 — The 2012 Strata Conference, being held Feb. 28-March 1 in Santa Clara, Calif., will offer three full days of hands-on data training and information-rich sessions. Strata brings together the people, tools, and technologies you need to make data work.
Save 20% on registration with the code RADAR20

What does data journalism mean for the future of journalism? Are
there new business models here?

Liliana Bounegru: There are all kinds of
interesting new business models emerging with data journalism. Media
companies are becoming increasingly innovative with the way they
produce revenues, moving away from subscription-based models and
advertising to offering consultancy services, as in the case of the
German award-winning OpenDataCity.

Digital technologies and the web are fundamentally changing the way
we do journalism. Data journalism is one part in the ecosystem of
tools and practices that have sprung up around data sites and
services. Quoting and sharing source materials (structured data) is in
the nature of the hyperlink structure of the web and in the way we are
accustomed to navigating information today. By enabling anyone to
drill down into data sources and find information that is relevant to
them as individuals or to their community, as well as to do fact
checking, data journalism provides a much needed service coming from a
trustworthy source. Quoting and linking to data sources is specific
to data journalism at the moment, but seamless integration of data in
the fabric of media is increasingly the direction journalism is going
in the future. As Tim
Berners-Lee says, "data-driven journalism is the future".

What data-driven journalism initiatives have caught your attention?

Liliana Bounegru: The data journalism project FarmSubsidy.org is one of my
favorites. It addresses a real problem: The European Union (EU) is
spending 48% of its budget on agriculture subsidies, yet the money
doesn't reach those who need it.

Tracking payments and recipients of agriculture subsidies from the
European Union to all member states is a difficult task. The data is
scattered in different places in different formats, with some missing
and some scanned in from paper records. It is hard to piece it
together to form a comprehensive picture of how funds are distributed.
The project not only made the data available to anyone in an easy to
understand way, but it also advocated for policy changes and better
transparency laws.

Another of my favorite examples is the LRA Crisis Tracker, a
real-time crisis mapping platform and data collection system. The
tracker makes information about the attacks and movements of the Lord's
Resistance Army (LRA) in Africa publicly available. It helps to inform
local communities, as well as the organizations that support
the affected communities, about the activities of the LRA through an
early-warning radio network in order to reduce their response time to
incidents.

I am also a big fan of much of the work done by the Guardian Datablog.
You can find lots of other examples featured on datadrivenjournalism.net,
along with interviews, case studies and tutorials.

I've talked to people like Chicago Tribune news app developer
Brian Boyer about the emerging "newsroom
stack." What do you feel are the key tools of the data
journalist?

Liliana Bounegru: Experienced data journalists
list spreadsheets as a top data journalism tool. Open source tools and
web-based applications for data cleaning, analysis and visualization
play very important roles in finding and presenting data stories. I
have been involved in organizing several workshops on ScraperWiki and Google Refine for
data collection and analysis. We found that participants were quite
able to quickly ask and answer new kinds of questions with these
tools.

How does data journalism relate to open data and open government?

Liliana Bounegru: Open government data means that
more people can access and reuse official information published by
government bodies. This in itself is not enough. It is increasingly
important that journalists can keep up and are equipped with skills
and resources to understand open government data. Journalists need to
know what official data means, what it says and what it leaves out.
They need to know what kind of picture is being presented of an
issue.

Public bodies are very experienced in presenting data to the public
in support of official policies and practices. Journalists, however,
will often not have this level of literacy. Only by equipping
journalists with the skills to use data more effectively can we break
the current asymmetry, where our understanding of the information that
matters is mediated by governments, companies and other experts. In a
nutshell, open data advocates push for more data, and data journalists
help the public to use, explore and evaluate it.

This interview has been edited and condensed for clarity.

Photo on associated home and category pages: NYTimes: 365/360 - 1984 (in color) by blprnt_van, on Flickr.

Related:

International Data Journalism Awards Recognize a Crucial Field
The
growing importance of data journalism
Data
journalism and the emerging newsroom stack
The work of data journalism: Find, clean, analyze, create ... repeat

This post originally appeared on Publishers Weekly.

Imagine buying a car that locks you into one brand of fuel. A new BMW, for example, that only runs on BMW gas. There are plenty of BMW gas stations around, even a few in your neighborhood, so convenience isn't an issue. But if one of those other gas stations offers a discount, a membership program, or some other attractive marketing campaign, you can't participate. You're locked in with the BMW gas stations.

This could never happen, right? Consumers are too smart to buy into something like this. Or are they? After all, isn't that exactly what's happening in the ebook world? You buy a dedicated ebook reader like a Kindle or a NOOK and you're locked in to that company's content. Part of this problem has to do with ebook formats (e.g., EPUB or Mobipocket) while another part of it stems from publisher insistence on the use of digital rights management (DRM). Let's look at these issues individually.

Platform lock-in

I've often referred to it as Amazon's not-so-secret formula: Every time I buy another ebook for my Kindle, I'm building a library that makes me that much more loyal to Amazon's platform. If I've invested thousands or even hundreds of dollars in Kindle-formatted content, how could I possibly afford to switch to another reading platform?

It would be too inconvenient to have part of my library in Amazon's Mobipocket format and the rest in EPUB. Even though I could read both on a tablet (e.g., the iPad), I'd be forced to switch between two different apps. The user interface between any two reading apps is similar but not identical, and searching across your entire library becomes a two-step process since there's no way to access all of your content within one app.

This situation isn't unique to Amazon. The same issue exists for all the other dedicated ereader hardware platforms (e.g., Kobo, NOOK, etc.). Google Books initially seemed like a solution to this problem, but it still doesn't offer mobi formats for the Kindle, so it's selling content for every format under the sun — except the one with the largest market share.

EPUB would seem to be the answer. It's a popular format based on web standards, and it's developed and maintained by an organization that's focused on openness and broad industry adoption. It also happens to be the format used by seemingly every ebook vendor except the largest one: Amazon.

Even if we could get Amazon to adopt EPUB, though, we'd still have that other pesky issue to deal with: DRM.

The myth of DRM

I often blame Napster for the typical book publisher's fear of piracy. Publishers saw what happened in the music industry and figured the only way they'd make their book content available digitally was to tightly wrap it with DRM. The irony of this is that some of the most highly pirated books were never released as ebooks. Thanks to the magic of high-speed scanner technology, any print book can easily be converted to an ebook and distributed illegally.

Some publishers don't want to hear this, but the truth is that DRM can be hacked. It does not eliminate piracy. It not only fails as a piracy deterrent, but it also introduces restrictions that make ebooks less attractive than print books. We've all read a print book and passed it along to a friend. Good luck doing that with a DRM'd ebook! What publishers don't seem to understand is that DRM implies a lack of trust. All customers are considered thieves and must be treated accordingly.

The evil of DRM doesn't end there, though. Author Charlie Stross recently wrote a terrific blog post entitled "Cutting Their Own Throats." It's all about how publisher fear has enabled a big ebook player like Amazon to further reinforce its market position, often at the expense of publishers and authors. It's an unintended consequence of DRM that's impacting our entire industry.

Given all these issues, why not eliminate DRM and trust your customers? Even the music industry, the original casualty of the Napster phenomenon, has seen the light and moved on from DRM.

TOC NY 2012 — O'Reilly's TOC Conference, being held Feb. 13-15, 2012, in New York, is where the publishing and tech industries converge. Practitioners and executives from both camps will share what they've learned and join together to navigate publishing's ongoing transformation.

Register to attend TOC 2012

Lessons from the music industry

Several years ago, Steve Jobs posted a letter to the music industry pleading for them to abandon DRM. The letter no longer appears on Apple's website, but community commentary about it lives on. My favorite part of that letter is where Jobs asks why the music industry would allow DRM to go away. The answer is that, "DRMs haven't worked, and may never work, to halt music piracy." In fact, a study last year by Rice University and Duke University contends that removing DRM can actually decrease piracy. Yes, you read that right.

I recently had an experience with my digital music collection that drove this point home for me. I had just switched from an iPhone to an Android phone and wanted to get my music from the old device onto the new one. All I had to do was drag and drop the folder containing my music in iTunes to the SD card in my new phone. It worked perfectly because the music file formats are universal and there was no DRM involved.

Imagine trying to do that with your ebook collection. Try dragging your Kindle ebooks onto your new NOOK, for example. Incompatible file formats and DRM prevent that from happening ... today. At some point in the not-too-distant future, though, I'm optimistic the book publishing industry will get to the same stage as the music industry and offer a universal, DRM-free format for all ebooks. Then customers will be free to use whatever e-reader they prefer without fear of lock-in and incompatibilities.


The music industry made the transition, why can't we?

Related:

On pirates and piracy
Book piracy: Less DRM, more data
Putting Ebook Piracy into Perspective
The Analog Hole: Another Argument Against DRM

Apple updated iBooks Author to version 1.0.1 on Friday afternoon, the only change being an update to the software's controversial end user license agreement. The updated EULA now specifically only applies distribution restrictions to the interactive .ibooks format files generated by the app.






Read the comments on this post

Here are a few of the stories that caught my attention this week in the publishing space.

Barnes & Noble puts its foot down on Amazon

Last week, Amazon teamed up with Houghton Mifflin Harcourt to print and distribute the Amazon Publishing East Coast's adult titles under a new imprint, New Harvest. Some speculated the move might get Amazon through the brick-and-mortar doors of B&N. This week, B&N made it clear that not only would HMH's New Harvest imprint not make it in the door, but that no Amazon Publishing title would. In a post for the New York Times, Julie Bosman quoted from a statement made by Jaime Carey, B&N's chief merchandising officer:

"Our decision is based on Amazon's continued push for exclusivity with publishers, agents and the authors they represent. These exclusives have prohibited us from offering certain e-books to our customers. Their actions have undermined the industry as a whole and have prevented millions of customers from having access to content. It's clear to us that Amazon has proven they would not be a good publishing partner to Barnes & Noble as they continue to pull content off the market for their own self interest."

O'Reilly's general manager and publisher Joe Wikert called on B&N this week to disrupt the industry — maybe this is its first move. Bosman also took a look at B&N's position in the industry and its importance to the publishing ecosystem, especially in the face of a competitor like Amazon. Jordan Weissmann at The Atlantic mulled the prospects of Amazon killing publishing and argued: "In a financial arms race, publishers simply can't beat Amazon's arsenal."

codeMantra collectionPoint 3.0 — Compose it; convert it; package it; distribute it; track it; re-price it; control your digital book workflow and metadata from one platform with collectionPoint 3.0, now available

Breaking up is hard to do

Amazon had issues with a social networking partner this week as well. As of Monday, Goodreads no longer displayed book data from the Amazon Product Advertising API, opting instead to move its data partnership to the Ingram Book Company. A Goodread's representative told Laura Hazard Owen that "the [API license agreement] terms now required by Amazon have become so restrictive that it makes better business sense to work with other data sources." Owen outlined some of the specifics on the restrictions:

"Amazon requires sites that use its API to link that content back to the Amazon site exclusively — so a book page on Goodreads would have to link only to its product page on Amazon and not to any other source or retailer ... Amazon also does not allow any content from its API to be used on mobile sites and apps."

Jon Mitchell at ReadWriteWeb took a deeper look into the situation — and explained why Goodreads will survive its breakup with Amazon.

The news caused some readers to worry about their cultivated Goodreads bookshelves. GalleyCat detailed potential data issues and offered up a Goodreads link that allows users to check on the state of their shelves to see if any tidying up is necessary.

Jonathan Franzen waxes absurd on ebooks

There's no shortage of things slated to be destroying society, and this week, author Jonathan Franzen added ebooks to the list. The Telegraph quoted Franzen speaking at a book festival in Cartagena, Colombia:

"I think, for serious readers, a sense of permanence has always been part of the experience. Everything else in your life is fluid, but here is this text that doesn't change. Will there still be readers 50 years from now who feel that way? Who have that hunger for something permanent and unalterable? I don’t have a crystal ball. But I do fear that it's going to be very hard to make the world work if there's no permanence like that."

Chenda Ngak at CBS's techt@lk took offense at Franzen's remarks, stating: "Even if I agree with him, as a book lover, his statements are too condescending to take seriously." Jonathan Segura at NPR chimed in as well, calling Franzen's comments "absurd" and pleading that we "get past the e-books versus print books thing." Segura's final comment pretty much summed up the overarching sentiment:

"We should worry less about how people get their books and — say it with me now! — just be glad that people are reading."

Photo (top): Kiftsgate Court, Chipping Campden, Gloucestershire - No Entry - sign by ell brown, on Flickr

Photo (bottom): Broken Kindle by kodomut, on Flickr

Related:

We're in the midst of a restructuring of the publishing universe (don't panic)

Hating Amazon is not a strategy

Coming soon to a location near you: The Amazon Store?

Open Question: Is it realistic for publishers to cut Amazon out of the equation?

More Publishing Week in Review coverage

This post originally appeared on Joe Wikert's Publishing 2020 Blog ("iBooks Author: Appreciating Apple's Intent"). It's republished with permission.

Apple's recent announcement and release of its iBooks Author tool was met with plenty of controversy. This HuffPost article pretty well sums things up.

My question is simply this: Why all the fuss? Apple's intent has never been to improve the book publishing industry. Just like Amazon and any other ebook vendor, Apple's goal is to capture share of this rapidly growing segment. In Apple's case, it simply decided to offer an authoring tool that's capable of creating some pretty darned cool products. If Amazon were to do the same thing and create a terrific authoring tool for mobi or KF8 format, would the industry be as upset? I don't think so.

How is this any different from the App Store model itself? Developers are creating apps for the App Store, and they know they'll only run on an iOS device. They also realize they'll have to go through Apple's approval process before getting into the App Store.

Prior to the release of iBooks Author, the content creation and distribution model looked like this:

Author writes material in favorite word processor.
Author/publisher edit and convert that content into mobi format for distribution on Amazon, EPUB format for distribution through iBookstore and others, etc.

The exact same model still exists today, even with the introduction of iBooks Author. That's right. Apple's EULA doesn't really lock you into its distribution channel for your content. That restriction only applies to a "book or other work you generate using [the iBooks Author] software." All Apple's really trying to do is prevent you from tweaking the output of its tool to create content for other distribution channels. OK, that's kind of annoying, but far from the lock-in nightmare so many people are describing it as. Based on my interpretation, you're able to use the same content as input to the iBooks Author tool as you'd use for a mobi-formatted product you want to sell on Amazon.

(I should also point out that I'm far from an Apple fanboy. Anyone who knows me realizes I dumped my iPhone last year for an Android-based Samsung Galaxy S II (and yes, I love it). I also tried to dump my iPad for a Kindle Fire but found the Fire user experience to be very disappointing. I'll probably make the jump to another Android tablet later this year, once key apps like Zite are available. In the meantime though, I want to make it clear I'm not here to shill for Apple. If anything, I'm currently in a stage where I'd prefer to buy devices that aren't made by the content providers. Samsung is high on my list, for example.)

Apple doesn't have an objective to move the publishing industry forward. It sees an opportunity to reinvent this industry, and it feels it can do so within its own, closed ecosystem. It's as simple as that, and it's consistent with everything it has done in the App Store up to now.

Let's also not forget that the iBooks Author tool is free. It's not like we paid Apple $50, $100 or more for some authoring tool that we thought could work for all content formats and distribution channels. If the tool's feature set is compelling enough, I'd like to think the other ebook vendors (e.g., Barnes & Noble, Amazon, Kobo, etc.) will have to come up with something at least as powerful for their own platforms. If not, they get left in the dust and Apple gains share. Seems pretty fair to me.

In the meantime, I plan to do some hands-on testing with iBooks Author. At first, I was discouraged because you can't download iBooks Author unless you're running Lion. I'm still on Snow Leopard, but an O'Reilly colleague sent me this link that shows you how to tweak a couple of settings so you can download and run iBooks Author on a Snow Leopard system. I just tried it, and it works fine. (You just have to carefully read and interpret the steps since it's a translation from French to English.)

TOC NY 2012 — O'Reilly's TOC Conference, being held Feb. 13-15, 2012, in New York City, is where the publishing and tech industries converge. Practitioners and executives from both camps will share what they've learned and join together to navigate publishing's ongoing transformation.

Register to attend TOC 2012

Related:

A few thoughts on iBooks Author and Apple's textbook move
The textbook industry might not be as "reinvented" as Apple hoped

Educators so far seem excited about the potential promise of a learning "revolution" enabled by Apple's new iBooks Author app. However, not everyone is feeling that same level of enthusiasm: e-book publishing experts have concerns about the formatting that iBooks Author can output, which isn't fully ePub 2 or ePub 3 compliant. Furthermore, Apple has added a clause to iBooks Author's end user license agreement that prohibits selling e-books created with iBooks Author anywhere but the iBookstore.

iBooks created by iBooks Author use ePub 2 along with certain HTML5 and JavaScript-based extensions that Apple uses to enable multimedia and interactive features. Those interactive features will only work with Apple's iBooks app, not with other e-reader software or hardware, because only Apple supports those extensions.

Still, there shouldn't be any technical limitation to exporting a strictly ePub 2-compliant ePub document if none of the interactive features are used. Unfortunately, iBooks Author only exports PDFs and text.






Read the comments on this post

Apple may be poised to announce changes coming to iBooks, and perhaps eBook publishing, sometime this month. In particular, we believe the announcement may have important reverberations for textbook publishers and buyers.

According to a report by All Things Digital published Monday, the company is planning a media event in New York to make a "media-related," not hardware-related announcement. Further, sources for TechCrunch claimed the announcement will focus on "improvements to the iBooks platform," and the event will supposedly be more geared towards the publishing industry (not necessarily consumers).

Apple has recently highlighted the ability of its iBooks platform to include sound, video, and other features by offering a free eBook of The Yellow Submarine. And based on information from our own sources, we believe the announcement could likely involve support for the EPUB 3 standard, which enables a wider variety of multimedia and interaction features. Amazon recently announced its own similarly improved eBook standard using HTML5 and CSS3.

Several authors have also told Ars that they long for tools to help transform book text into standards-compliant eBooks. The opportunity certainly seems ripe for Apple to offer such a tool. If Apple created software that could generate standards-compliant EPUB files, it could be a boon to both the publishing industry and independent authors alike.

Incidentally, one source who has worked with Apple to integrate technology in education recently suggested that Apple may have important changes coming to its iBooks platform directed specifically toward the academic set. Digital textbooks represent another nascent market that Apple could potentially upend as it did with music and mobile apps.




Read the comments on this post

This post is part of the TOC podcast series, which we'll be featuring here on Radar in the coming months. You can also subscribe to the free TOC podcast through iTunes.

Book reading has always been considered a solitary activity, but maybe that's just because of the limitations of print. Social reading platforms are sprouting up all around us, and mobNotate is one of the more interesting ones. This TOC podcast features insight from mobNotate's founder, Ricky Wong (@kinwong), as well as their technical advisor, Sean Gerrish. They talk about where they are with the mobNotate platform, why social is an important part of tomorrow's reading experience and what it will look like.

(Listen to this interview via the embedded player or download the MP3 file.)

Key points from the full interview include :

Machine learning makes it happen — Related conversations are already happening on the web, but mobNotate ties them back to the text so you don't have to hunt them down. [Discussed at the 0:45 mark.]

Social reading is not an oxymoron — If social reading is implemented correctly it will feel like an on-topic conversation with a lot of really interesting people. If it's done poorly, of course, it's nothing more than a distraction. [Discussed at 1:38.]

Reader apps & devices don't lend themselves to content creation — And that's where a tool like mobNotate comes in, which makes it extremely easy to add your thoughts to the conversation. Think "tapping and swiping" rather than "typing" as well as "curation" rather than "creation." [Discussed at 6:41.]

Social isn't just for certain genres of content — There are different (and better ways) to implement social features on different types of content. [Discussed at 9:35.]

Community is an important part of the value proposition — Social features can help add to the value of your product and therefore help justify a higher price. [Discussed at 11:35.]

Social features can still result in a clean & simple reading environment — Sean's example here of Google "then and now" is a terrific analogy. Social reading functionality needs to be as important to the user experience as images and videos have become to search results. [Discussed at 15:00.]

The 80/20 rule applies here as well — A small percentage of users will likely create and curate the content that's used by the larger audience. [Discussed at 15:46.]

You can listen to the entire interview here.

TOC NY 2012 — O'Reilly's TOC Conference, being held Feb 13-15, 2012 in New York City, is where the publishing and tech industries converge. Practitioners and executives from both camps will share what they've learned and join together to navigate publishing's ongoing transformation.

Register to attend TOC 2012

Related:

Content is a social creature

How online bookstores should get social

The future of the book

More TOC Podcasts

This post is part of the TOC podcast series, which we'll be featuring here on Radar in the coming months. You can also subscribe to the free TOC podcast through iTunes.

Book reading has always been considered a solitary activity, but maybe that's just because of the limitations of print. Social reading platforms are sprouting up all around us, and mobNotate is one of the more interesting ones. This TOC podcast features insight from mobNotate's founder, Ricky Wong (@kinwong), as well as their technical advisor, Sean Gerrish. They talk about where they are with the mobNotate platform, why social is an important part of tomorrow's reading experience and what it will look like.

(Listen to this interview via the embedded player or download the MP3 file.)

Key points from the full interview include :

Machine learning makes it happen — Related conversations are already happening on the web, but mobNotate ties them back to the text so you don't have to hunt them down. [Discussed at the 0:45 mark.]

Social reading is not an oxymoron — If social reading is implemented correctly it will feel like an on-topic conversation with a lot of really interesting people. If it's done poorly, of course, it's nothing more than a distraction. [Discussed at 1:38.]

Reader apps & devices don't lend themselves to content creation — And that's where a tool like mobNotate comes in, which makes it extremely easy to add your thoughts to the conversation. Think "tapping and swiping" rather than "typing" as well as "curation" rather than "creation." [Discussed at 6:41.]

Social isn't just for certain genres of content — There are different (and better ways) to implement social features on different types of content. [Discussed at 9:35.]

Community is an important part of the value proposition — Social features can help add to the value of your product and therefore help justify a higher price. [Discussed at 11:35.]

Social features can still result in a clean & simple reading environment — Sean's example here of Google "then and now" is a terrific analogy. Social reading functionality needs to be as important to the user experience as images and videos have become to search results. [Discussed at 15:00.]

The 80/20 rule applies here as well — A small percentage of users will likely create and curate the content that's used by the larger audience. [Discussed at 15:46.]

You can listen to the entire interview here.

TOC NY 2012 — O'Reilly's TOC Conference, being held Feb 13-15, 2012 in New York City, is where the publishing and tech industries converge. Practitioners and executives from both camps will share what they've learned and join together to navigate publishing's ongoing transformation.

Register to attend TOC 2012

Related:

Content is a social creature

How online bookstores should get social

The future of the book

More TOC Podcasts

Many of publishing's big developments from 2011 will continue to shape the industry in 2012. So with that in mind, here's a look at five of the most important lessons from last 12 months.

Amazon is, indeed, a disruptive publishing competitor

If it wasn't apparent before, Amazon's publishing intentions became plainly obvious this year. The wave started out small, with a host of expanding self-publishing tools for authors, but it grew to tsunami proportions as Amazon launched imprint after imprint, from romance to science fiction. Amazon also hired industry heavy-hitter Larry Kirshbaum, who "is charged with building something that will look like a general trade publisher.'"

Some of Amazon's publishing projects.

Amazon further extended its reach into publishing when it launched the Kindle Owner's Lending Library. The ebook lending waters already were murky and contentious for publishers — HarperCollins instigated a memorable dustup, as did Penguin — but Amazon's move into the space caused a full-fledged uproar among publishers as well as authors, and may have damaged the publisher-library relationship further.

O'Reilly's Joe Wikert highlighted one of the main problems from the publisher perspective:

As Amazon stated in its press release, "For the vast majority of titles, Amazon has reached agreement with publishers to include titles for a fixed fee." So no matter how popular (or unpopular) the publisher's titles are, they get one flat fee for participation in the library. I strongly believe this type of program needs to compensate publishers and authors on a usage level, not a flat fee. The more a title is borrowed, the higher the fee to the publisher and author. Period.

And Amazon may be encroaching on feature magazines like the Atlantic and the New Yorker as well. In a sign of possible things to come, freelance journalist Marc Herman took his long-form story, "The Shores of Tripoli," and expanded it into a $1.99 Kindle Single. According to his blog, he has plans to expand on the model, which would further sideline traditional publishing avenues.

Publishers aren't necessary to publishing

Authors have figured out they don't need publishers to publish books. The self-publishing book market saw quite a boom this year as the publishing format started becoming more mainstream and the services offered by self-publishing companies became more comprehensive — providing authors with platforms, sales, marketing, editing, etc.

Amazon has a role in this boom as well. The Wall Street Journal reported that "Amazon.com Inc. fueled the growth [in self-publishing] by offering self-published writers as much as 70% of revenue on digital books, depending on the retail price. By comparison, traditional publishers typically pay their authors 25% of net digital sales and even less on print books."

Another trend emerged this year to further sideline the publisher's role: the rise of the agent-publisher. This controversial and contentious business model allows agents to step in to provide expanded publishing services to authors. In an interview, Booksquare's Kassia Krozser explained that the new agent-publisher role emerged because of failings on the part of traditional publishers: "Traditional publishers need to not only rethink how they sell their value to authors and agents, but they also need to rethink the economic structure of their deals." Krozser also expressed concerns that the agent-publisher role carries a conflict of interest — see her interview here.

Readers sure do like ebooks

There good news is that people are still reading and they're embracing the digital transformation. The Book Industry Study Group (BISG) released a report in November that showed that readers are solidly committing to digital books. A couple highlights from the report:

Power buyers are spending more. More than 46% of those who say they acquire e-books at least weekly ... report that they have increased their dollars spent for books in all formats, compared with 30.4% of all survey respondents.

"... nearly 50% of print book consumers who have also acquired an e-book in the past 18 months would wait up to three months for the e-version of a book from a favorite author, rather than immediately read it in print."

The number of devices sold is telling as well. A Pew report found that "ereader ownership growth in the U.S. doubled in six months, from 6% to 12% of adults owning an ebook reader."

Though the new Kindle Fire is selling at a loss, Amazon reported that it is selling Kindles at a clip of "well over one million Kindle devices per week" — at least for the three weeks following Black Friday. Amazon hasn't disclosed the total number of devices it has sold, but one analyst estimates the sales to be 8% of total revenues in 2011 and predicts that amount will rise to 9.9% in 2012. So ... a lot of Kindles. Combine those numbers (vague as they might be) with the 40 million iPads sold, and the conclusion is clear: ereading is now mainstream.

HTML5 is an important publishing technology

HTML5 entered the publishing space in a big way this year — some calling it the "future of digital publishing." From storage to multimedia to content behavior (think shaking the iPhone or automatically sizing for different screen sizes) to geolocation to a host of other interactive features, HTML5 has squared itself up to become an important player in the industry. Amazon (mostly) embraced it in its Kindle Format 8, and HTML5 is supported in EPUB3.

HTML5 is platform agnostic and may even be able to save — or make — publishers money. In an interview early in the year, Google's Marcin Wichary explained:

It's very important to recognize that HTML5 fits all the devices you can think of, from the iPhone in your pocket to Google TV to the tablets to small screens and big screens. It's very easy to take the content you already have and through the "magic" of HTML5, refine it so it works very well within a given context. You don't have to do your work over and over again. Of course, all of these different means come with different monetization opportunities, like ads on the web or on mobile devices.

You can view Wichary's full interview below.



DRM is full of unintended consequences

It turns out DRM does more than provide publishers with a false sense of security — locking the content of books also locks those books into a platform (ahem, Kindle). This point was highlighted by author Charlie Stross in a November blog post in which he argued that DRM had become a strategic tool for Amazon:

... the big six's pig-headed insistence on DRM on ebooks is handing Amazon a stick with which to beat them harder. DRM on ebooks gives Amazon a great tool for locking ebook customers into the Kindle platform. If you buy a book that you can only read on the Kindle, you're naturally going to be reluctant to move to other ebook platforms that can't read those locked Kindle ebooks — and even more reluctant to buy ebooks from rival stores that use incompatible DRM ... If the big six began selling ebooks without DRM, readers would at least be able to buy from other retailers and read their ebooks on whatever platform they wanted, thus eroding Amazon's monopoly position.

So, to recap, we've learned that DRM doesn't stop anyone from pirating, nor does it come with the necessary data to support its impact. But it does give publishers one thing: a longer length of rope with which to hang themselves.

TOC NY 2012 — O'Reilly's TOC Conference, being held Feb. 13-15, 2012, in New York City, is where the publishing and tech industries converge. Practitioners and executives from both camps will share what they've learned and join together to navigate publishing's ongoing transformation.

Register to attend TOC 2012

Related:

"Hating Amazon is not a strategy"

Do agent-publishers carry a conflict of interest?

Publishers: What are they good for?

Book piracy: Less DRM, more data

What if a book is just a URL?

Many of publishing's big developments from 2011 will continue to shape the industry in 2012. So with that in mind, here's a look at five of the most important lessons from last 12 months.

Amazon is, indeed, a disruptive publishing competitor

If it wasn't apparent before, Amazon's publishing intentions became plainly obvious this year. The wave started out small, with a host of expanding self-publishing tools for authors, but it grew to tsunami proportions as Amazon launched imprint after imprint, from romance to science fiction. Amazon also hired industry heavy-hitter Larry Kirshbaum, who "is charged with building something that will look like a general trade publisher.'"

Some of Amazon's publishing projects.

Amazon further extended its reach into publishing when it launched the Kindle Owner's Lending Library. The ebook lending waters already were murky and contentious for publishers — HarperCollins instigated a memorable dustup, as did Penguin — but Amazon's move into the space caused a full-fledged uproar among publishers as well as authors, and may have damaged the publisher-library relationship further.

O'Reilly's Joe Wikert highlighted one of the main problems from the publisher perspective:

As Amazon stated in its press release, "For the vast majority of titles, Amazon has reached agreement with publishers to include titles for a fixed fee." So no matter how popular (or unpopular) the publisher's titles are, they get one flat fee for participation in the library. I strongly believe this type of program needs to compensate publishers and authors on a usage level, not a flat fee. The more a title is borrowed, the higher the fee to the publisher and author. Period.

And Amazon may be encroaching on feature magazines like the Atlantic and the New Yorker as well. In a sign of possible things to come, freelance journalist Marc Herman took his long-form story, "The Shores of Tripoli," and expanded it into a $1.99 Kindle Single. According to his blog, he has plans to expand on the model, which would further sideline traditional publishing avenues.

Publishers aren't necessary to publishing

Authors have figured out they don't need publishers to publish books. The self-publishing book market saw quite a boom this year as the publishing format started becoming more mainstream and the services offered by self-publishing companies became more comprehensive — providing authors with platforms, sales, marketing, editing, etc.

Amazon has a role in this boom as well. The Wall Street Journal reported that "Amazon.com Inc. fueled the growth [in self-publishing] by offering self-published writers as much as 70% of revenue on digital books, depending on the retail price. By comparison, traditional publishers typically pay their authors 25% of net digital sales and even less on print books."

Another trend emerged this year to further sideline the publisher's role: the rise of the agent-publisher. This controversial and contentious business model allows agents to step in to provide expanded publishing services to authors. In an interview, Booksquare's Kassia Krozser explained that the new agent-publisher role emerged because of failings on the part of traditional publishers: "Traditional publishers need to not only rethink how they sell their value to authors and agents, but they also need to rethink the economic structure of their deals." Krozser also expressed concerns that the agent-publisher role carries a conflict of interest — see her interview here.

Readers sure do like ebooks

There good news is that people are still reading and they're embracing the digital transformation. The Book Industry Study Group (BISG) released a report in November that showed that readers are solidly committing to digital books. A couple highlights from the report:

Power buyers are spending more. More than 46% of those who say they acquire e-books at least weekly ... report that they have increased their dollars spent for books in all formats, compared with 30.4% of all survey respondents.

"... nearly 50% of print book consumers who have also acquired an e-book in the past 18 months would wait up to three months for the e-version of a book from a favorite author, rather than immediately read it in print."

The number of devices sold is telling as well. A Pew report found that "ereader ownership growth in the U.S. doubled in six months, from 6% to 12% of adults owning an ebook reader."

Though the new Kindle Fire is selling at a loss, Amazon reported that it is selling Kindles at a clip of "well over one million Kindle devices per week" — at least for the three weeks following Black Friday. Amazon hasn't disclosed the total number of devices it has sold, but one analyst estimates the sales to be 8% of total revenues in 2011 and predicts that amount will rise to 9.9% in 2012. So ... a lot of Kindles. Combine those numbers (vague as they might be) with the 40 million iPads sold, and the conclusion is clear: ereading is now mainstream.

HTML5 is an important publishing technology

HTML5 entered the publishing space in a big way this year — some calling it the "future of digital publishing." From storage to multimedia to content behavior (think shaking the iPhone or automatically sizing for different screen sizes) to geolocation to a host of other interactive features, HTML5 has squared itself up to become an important player in the industry. Amazon (mostly) embraced it in its Kindle Format 8, and HTML5 is supported in EPUB3.

HTML5 is platform agnostic and may even be able to save — or make — publishers money. In an interview early in the year, Google's Marcin Wichary explained:

It's very important to recognize that HTML5 fits all the devices you can think of, from the iPhone in your pocket to Google TV to the tablets to small screens and big screens. It's very easy to take the content you already have and through the "magic" of HTML5, refine it so it works very well within a given context. You don't have to do your work over and over again. Of course, all of these different means come with different monetization opportunities, like ads on the web or on mobile devices.

You can view Wichary's full interview below.



DRM is full of unintended consequences

It turns out DRM does more than provide publishers with a false sense of security — locking the content of books also locks those books into a platform (ahem, Kindle). This point was highlighted by author Charlie Stross in a November blog post in which he argued that DRM had become a strategic tool for Amazon:

... the big six's pig-headed insistence on DRM on ebooks is handing Amazon a stick with which to beat them harder. DRM on ebooks gives Amazon a great tool for locking ebook customers into the Kindle platform. If you buy a book that you can only read on the Kindle, you're naturally going to be reluctant to move to other ebook platforms that can't read those locked Kindle ebooks — and even more reluctant to buy ebooks from rival stores that use incompatible DRM ... If the big six began selling ebooks without DRM, readers would at least be able to buy from other retailers and read their ebooks on whatever platform they wanted, thus eroding Amazon's monopoly position.

So, to recap, we've learned that DRM doesn't stop anyone from pirating, nor does it come with the necessary data to support its impact. But it does give publishers one thing: a longer length of rope with which to hang themselves.

TOC NY 2012 — O'Reilly's TOC Conference, being held Feb. 13-15, 2012, in New York City, is where the publishing and tech industries converge. Practitioners and executives from both camps will share what they've learned and join together to navigate publishing's ongoing transformation.

Register to attend TOC 2012

Related:

"Hating Amazon is not a strategy"

Do agent-publishers carry a conflict of interest?

Publishers: What are they good for?

Book piracy: Less DRM, more data

What if a book is just a URL?

Here are a few stories that caught my eye in the publishing space this week.

The concern of 99 cents

Author Melissa Foster took a look this week at the 99-cent price debate, highlighting the good, the bad and the ugly. The bad and the ugly mostly focused on how the price point affects independent authors:

If an author is self-published through Amazon KDP, he or she earns 34 cents per 99-cent book sold ... If you add up the average cover cost of $350, average editing job of $1,400, then divide by 34 cents, the author would have to sell 5,134 books just to break even, and that's nearly impossible without an additional amount for advertising.

Foster follows this by pointing out that most independent authors don't sell more than 100 copies of a book — that's a whopping $34 — and says independent authors who publish through small presses generally only pocket 12 cents per 99-cent book sold.

Employing this price point doesn't bode well for authors looking to sign with a traditional publisher, either. Foster quoted agent Jenny Bent: "... publishers are increasingly skeptical about how success at 99 cents will translate into success using their very different business model."

Author M.J. Rose also is quoted in the post, arguing that this sort of focus on price is wrong:

Readers may buy you once for 99 cents, but if they are disappointed they will never buy you again or even download you for free. On the other hand a reader will pay $4.99, $5.99 even up to $12 for an ebook of a writer whose work speaks to her. I'm seeing way too much conversation about what to charge for the book instead of how to write the book ... Quality matters more than ever.

Foster's analysis also highlighted some positive aspects of the price point, including using it as a promotional or marketing tool. An author could set the first book in a series at 99 cents, for instance, to help suck in readers — the old "the first one's (almost) free, but you'll be back" routine.

There's a lot more discussion on this debate over in the comment section of Kevin Kelly's blog post on this topic (from earlier this year). But really, the bottom line is this: the 99-cent price point is only financially viable for authors who are able to sell a boatload of books.

SPi Global partners with publishers and information providers to maximize the value of their content online and offline. With escalating costs of production and printing, changing customer preferences, and the need to adapt, SPi Global enables organizations to exploit and invest in new media technology. With a complete suite of digital and publishing services, we help companies gain a competitive advantage through our unique and
innovative solutions. For more information, please visit spi-global.com/content-solutions.

SOPA, meet DeSopa

The Stop Online Piracy Act (SOPA) hearing was delayed, possibly until next year, but anti-SOPA geeks aren't waiting to see what's going to happen. Andy Greenberg reported over at Forbes that "the Internet's communities of coders and free speech advocates" are hard at work building tools to circumvent SOPA's copyright protection measures:

... a developer named Tamer Rizk has been busy building an add-on for Firefox called DeSopa, which aims to give any Firefox user access to sites that SOPA's copyright protection measures have blocked. 'This program is a proof of concept that SOPA will not help prevent piracy,' reads a note included on DeSopa's download page. 'If SOPA is implemented, thousands of similar and more innovative programs and services will sprout up to provide access to the websites that people frequent. SOPA is a mistake. It does not even technically help solve the underlying problem, as this software illustrates.'

(Note: as of publication, the DeSopa add-on had been taken down from Mozilla's site.)

Greenberg also looked at Reddit users who "have been assembling their own lists of IP addresses for key sites that might be blocked under SOPA, what some of them call the 'Emergency List'." He also has a nice discussion of SOPA's unintended consequences and the collateral damage it could cause. The piece is well worth the read.

The future of stories is here

In a post at The Atlantic, senior editor Alexis Madrigal highlighted "The Fantastic Flying Books of Mr. Morris Lessmore" as the perfect gift for kids whose parents have an iPad. The river of book recommendations is hip deep this time of year, but the last line of Madrigal's post prodded me to check out the app: "It's what the future of stories looks like." (Hat tip to @tcarmody.)

Playing with the book/app reminded me of articles predicting that the coffee-table book will make it through the digital transition relatively unscathed. I'm not so sure about that. If the beauty of the art in this book and the way it's integrated into the interactivity are an indication of future stories, print may well be in trouble on the coffee-table front as well. Imagine an iPad coffee-table book that could play music from a foreign country and teach you common phrases in the native tongue; one that could seamlessly integrate video, animation or sound with the content. Print books can't do that.

The screenshot below shows the interactivity options and more of the beautiful art:

I sure hope Madrigal is right — this book app points to a very rich future for stories, and you don't need kids (or to be a kid) to fall in love with it.

Related:

Congress considers anti-piracy bills that could cripple Internet industries

Ebook pricing power is undermined by perceived value

10 innovative digital books you should know about

What publishers can and should learn from "The Elements"

The iPad as a "bedtime computer"

More Publishing Week in Review coverage

Here are a few stories that caught my eye in the publishing space this week.

The concern of 99 cents

Author Melissa Foster took a look this week at the 99-cent price debate, highlighting the good, the bad and the ugly. The bad and the ugly mostly focused on how the price point affects independent authors:

If an author is self-published through Amazon KDP, he or she earns 34 cents per 99-cent book sold ... If you add up the average cover cost of $350, average editing job of $1,400, then divide by 34 cents, the author would have to sell 5,134 books just to break even, and that's nearly impossible without an additional amount for advertising.

Foster follows this by pointing out that most independent authors don't sell more than 100 copies of a book — that's a whopping $34 — and says independent authors who publish through small presses generally only pocket 12 cents per 99-cent book sold.

Employing this price point doesn't bode well for authors looking to sign with a traditional publisher, either. Foster quoted agent Jenny Bent: "... publishers are increasingly skeptical about how success at 99 cents will translate into success using their very different business model."

Author M.J. Rose also is quoted in the post, arguing that this sort of focus on price is wrong:

Readers may buy you once for 99 cents, but if they are disappointed they will never buy you again or even download you for free. On the other hand a reader will pay $4.99, $5.99 even up to $12 for an ebook of a writer whose work speaks to her. I'm seeing way too much conversation about what to charge for the book instead of how to write the book ... Quality matters more than ever.

Foster's analysis also highlighted some positive aspects of the price point, including using it as a promotional or marketing tool. An author could set the first book in a series at 99 cents, for instance, to help suck in readers — the old "the first one's (almost) free, but you'll be back" routine.

There's a lot more discussion on this debate over in the comment section of Kevin Kelly's blog post on this topic (from earlier this year). But really, the bottom line is this: the 99-cent price point is only financially viable for authors who are able to sell a boatload of books.

SPi Global partners with publishers and information providers to maximize the value of their content online and offline. With escalating costs of production and printing, changing customer preferences, and the need to adapt, SPi Global enables organizations to exploit and invest in new media technology. With a complete suite of digital and publishing services, we help companies gain a competitive advantage through our unique and
innovative solutions. For more information, please visit spi-global.com/content-solutions.

SOPA, meet DeSopa

The Stop Online Piracy Act (SOPA) hearing was delayed, possibly until next year, but anti-SOPA geeks aren't waiting to see what's going to happen. Andy Greenberg reported over at Forbes that "the Internet's communities of coders and free speech advocates" are hard at work building tools to circumvent SOPA's copyright protection measures:

... a developer named Tamer Rizk has been busy building an add-on for Firefox called DeSopa, which aims to give any Firefox user access to sites that SOPA's copyright protection measures have blocked. 'This program is a proof of concept that SOPA will not help prevent piracy,' reads a note included on DeSopa's download page. 'If SOPA is implemented, thousands of similar and more innovative programs and services will sprout up to provide access to the websites that people frequent. SOPA is a mistake. It does not even technically help solve the underlying problem, as this software illustrates.'

(Note: as of publication, the DeSopa add-on had been taken down from Mozilla's site.)

Greenberg also looked at Reddit users who "have been assembling their own lists of IP addresses for key sites that might be blocked under SOPA, what some of them call the 'Emergency List'." He also has a nice discussion of SOPA's unintended consequences and the collateral damage it could cause. The piece is well worth the read.

The future of stories is here

In a post at The Atlantic, senior editor Alexis Madrigal highlighted "The Fantastic Flying Books of Mr. Morris Lessmore" as the perfect gift for kids whose parents have an iPad. The river of book recommendations is hip deep this time of year, but the last line of Madrigal's post prodded me to check out the app: "It's what the future of stories looks like." (Hat tip to @tcarmody.)

Playing with the book/app reminded me of articles predicting that the coffee-table book will make it through the digital transition relatively unscathed. I'm not so sure about that. If the beauty of the art in this book and the way it's integrated into the interactivity are an indication of future stories, print may well be in trouble on the coffee-table front as well. Imagine an iPad coffee-table book that could play music from a foreign country and teach you common phrases in the native tongue; one that could seamlessly integrate video, animation or sound with the content. Print books can't do that.

The screenshot below shows the interactivity options and more of the beautiful art:

I sure hope Madrigal is right — this book app points to a very rich future for stories, and you don't need kids (or to be a kid) to fall in love with it.

Related:

Congress considers anti-piracy bills that could cripple Internet industries

Ebook pricing power is undermined by perceived value

10 innovative digital books you should know about

What publishers can and should learn from "The Elements"

The iPad as a "bedtime computer"

More Publishing Week in Review coverage

Stephen Colbert opened his October 25th, 2011, show with his normal exuberance. He bragged about his special early access to the iPhone, the iPad, and the iV (a product that feeds the Internet directly into your veins; he assured us a short wait of six months before its release). The release of Walter Isaacson's "Steve Jobs" would be no different, as Colbert pulled the 600-page biography from behind his desk. But Colbert immediately became perplexed.

The single finger touchscreen swipe on the cover didn't turn pages. When you turned the book upside down, the picture didn't reorient. Colbert complained there was no place to plug in his headphones so he could listen to it. And then he tried to activate the voice recognition by touching the bottom of the cover, "Tell me about Steve Jobs. Where is the nearest church or camera store?" He ended the segment saying that the device would soon be released with "a revolutionary softcover." The jokes played well to the geekish sensibilities of the studio audience, but I am not sure even the show's writers knew how well the sketch described the confused state of book publishing.


"Steve Jobs" will serve as a prominent road marker on the path from atoms to bits. The decision for Simon & Schuster to hold the digital release of the biography for two weeks to match the physical release even after the death of Jobs is worthy of a Harvard Business School case. And at the same time, even as computers now interface with us in almost every aspect of our lives and Jobs' critical role in that proliferation, the majority of people will read his life story on paper.


Colbert poking fun at the Jobs biography repeats, again, a meme that we in the publishing industry should be gravely concerned about — our customers don't know what a book is anymore.


The consequences of book updates

In July 2011, I launched an experimental project with O'Reilly called "Every Book Is a Startup." The project is meant to poke at the boundaries of traditional publishing. The book was created around the idea that new material will be released over time, culminating in a finished work early in 2012. Readers are encouraged to constantly give feedback about the material. The pricing is dynamic, increasing slowly to match the amount of material released, but once purchased, a customer receives all future updates for free.

We are only using one distribution point at the start of the project, oreilly.com, because the distribution system for electronic books is not designed to allow an ebook to be updated and released again. You might remember one of the side effects of Amazon's 2009 recall of "1984" was that after the book was restored, customers found their bookmarks and notes had disappeared.

We, unfortunately, found the same problem with our release strategy. Wonderful publishing startups like Readmill and SocialBook have created the possibility for readers using EPUB files to highlight important passages and share those with others back through the web, but when a reader of "Every Book Is A Startup" loads a new edition, their digital artifacts suffer the same fate as the readers of "1984" — the loss of their old thoughts as I present them with my new ones.

I have been hesitant to call "Every Book Is A Startup" a book because of the expectations people hold for a book: a finished work, written from a position of singular authority, available in some way in a physical form. What I never expected was how strongly the qualities of a book would be brought forward from the physical to the digital. Digital books have been designed to carry forward the same atomic quality of immutability of physical books. As I reached out to my colleagues working in the world of ebooks, the consensus was that no one had considered a reality where an author, given the ability to distribute directly and virtually cost free, would consider updating their work and the consequences that might have.


Bits and atoms don't behave the same way, but we have built the next step forward in publishing as though they do.

Possibilities arise from a new name

The trouble to this point is that a book is a book. Stacey Madden used precisely those words to title an essay in the inaugural issue of "Toronto Review of Books" that describes this predicament. "I do not mean to argue the advantages of paperbound books over their electronic counterparts," wrote Madden. "The contents of both are, for the most part, the same, and the differences lie mainly in medium. I am simply pointing out a semantic fact. E-books are not 'books' but digitized compositions." She firmly believes the book's 550-year-old meaning that connects both form and format should be maintained. "Before a collection of human thoughts is transformed into what we call a 'book,' it is merely a story, a manuscript, a document, or a text." Madden points to the need for more of us to see the difference between a book and its electronic counterparts.

Now, Madden writes further about the poetic qualities of the book and declares the superiority of the bound volume for its weight, smell, and ability to act as apartment furnishing. This judgment undermines the broader point and shows from another perspective the real trouble we are in.

The people who love books for what they are and what they have been are grabbing for their hardcovers and their paperbacks and saying "This word belongs to us." The digerati paving the way with wireless tablets and social networking recommendation services are trying to say, "You don't understand, we have books and we have made them way better." This is messy and leads to confusion.

We are living through a time in book publishing where words fail us, a situation that we should all find some irony in given the products we sell. We need some new language that describes what happens and, more importantly, what is possible when the words are separated from the paper. Those two things need to be separated so we can build systems and infrastructures that support the new capabilities of the technology.

For several decades, what we know today as a "car" was referred to as a "horseless carriage." It was easier to describe this new invention as what it was not, rather than what it was.

Maybe there are books and there are paperless books. I know it is a little awkward, and you want to ask yourself, "What does that mean?" — but when you remove the paper from a book, it becomes so much easier to see the possibilities.

TOC NY 2012 — O'Reilly's TOC Conference, being held Feb. 13-15, 2012, in New York City, is where the publishing and tech industries converge. Practitioners and executives from both camps will share what they've learned and join together to navigate publishing's ongoing transformation.

Register to attend TOC 2012

Photo credit for associated book picture used on home and category pages: Old book (1882) by VanDammeMaarten.be, on Flickr

Related:

Every Book is a Startup (book/project)
Every Book is a Startup (webcast)
What publishing can learn from tech startups
Ebooks and Print Books are Not Mutually Exclusive

This is part of an ongoing series related to Peter Meyers' project "Breaking the Page: Transforming Books and the Reading Experience." We'll be featuring additional material in the weeks ahead. (Note: This post originally appeared on A New Kind of Book. It's republished with permission.)

I've been fiddling with the idea of using multiple displays to give a presentation — putting different slides on different screens. One design sketch — working title: "Documan" — has gotten some chuckles around my office (yes, I work alone):

Man-mounted iPads, plus a nearby monitor. A few possibilities not shown: each iPad could contain images, not just text; objects could move between iPads or from iPad to monitor; and presenter could rotate one or more iPads.

Why on earth does the world need to see a man strap on a half dozen iPads? And, more importantly, what kind of message would benefit from a rig like this?

Beats me. But I do think that content experiments, designed expressly for the screens we all use — rather than our ancestors' print pages or single PowerPoint slides — are the best way to figure out how stories and teaching change when they move onto the touchscreen.

I'll spare you, for now, the words and images I'm testing out to fill those screens. (One teaser, though: think about how easy Keynote for iPad makes it to build an action that exits screen right and enters screen left. Now, if you could just get the timing right when using two iPads ...).

Clearly, I'm not the only guy playing around here. Ahead, I round up a few content confections that span multiple screens. Some involve separate physical displays, others use different virtual windows. Not all of this stuff is new. But I find it thought provoking how creative types are using the small, medium, and large screens that increasingly coexist near each other.

iPad + projector

Joe Sabia calls himself an "iPad storyteller" — love it! He showed off his stuff at a recent TED talk where he uses his tablet and a variety of different apps (iBooks, a drawing app, Google Earth, Photos, and so on) to entertain an audience that is variously fixed on him, the big projector screen which his iPad is attached to, and the iPad's display itself.

iPad + magician

Sleight-of-hand artist and iPad maestro Simon Pierro pulls off some awfully clever tricks with his iPad and a real tennis ball, a glass of milk, and a weather forecaster's hair (she's on a video inside the iPad). I have no idea what's magic, what's video editing trickery, or what he and the iPad are actually doing. And, you know what? It doesn't matter. What he demonstrates here is how man and machine can team up to entertain in really innovative ways. Don't miss his part two, where he — sorta/kinda — sheds light on what he's done.

iPad-powered window displays

Gin Lane Media filled up three of Saks 5th Avenue's storefront windows with 64 iPads and nine 27-inch displays.

iPad/iPhone partnerships

A few apps use the big and small screen of a tablet and a smartphone in tandem. The iOS app Scrabble, for example, lets you conduct group games in which the iPad serves as publicly viewable board and the iPhone is each player's private letter stash. Remote Palette is a painting app where the iPad is the canvas and the iPhone is the paint palette.

Multiple browser windows

The band Arcade Fire worked with director Chris Milk to compose this mind-blowing HTML5-powered interactive video for its song "We Used to Wait." You give this web app the address of the house or building where you grew up in. It then whips together a custom-built video (woven around some stock footage) that incorporates Google Maps footage of your old neighborhood and other graphical magic mashups … all in multiple browser windows of various sizes. (It only works in the Chrome browser.) If you like this one, you'll love sour-mirror.jp, which uses snapshots of you from your laptop's webcam, and your Facebook and Twitter feed, to compose a multi-window extravaganza. It all culminates in a mosaic of your face built out of pix pulled from your social media feeds.

Multi-screen patterns

Here's a pattern-style analysis of different content and interaction designs for multiple displays, from the basic (how Amazon uses Whispersync to keep book location and notes coordinated across a user's different reading devices) to some innovative software that helps end users take an image, chop it up, and display it on their own collection of displays. That's what the next item is about.

Junkyard Jumbotron

Free to use (beta) software from some MITers that automatically splits up an image and displays it on whatever collection of screens (smartphones, tablets, PCs) you assemble. This demo shows it in action.

The multi-screen experience

Here's a five-minute video, with a bunch of TV and consumer electronics execs and analysts. Nothing hugely revelatory, but a nice little brain-tickler about how we are entering an age wherein audience and content producers alike are thinking about how to create and consume stories that play across displays of many different sizes.

Splitscreen: A Love Story

Heartwarming. Winner of a Nokia smartphone video-making contest, this video shows how split-screen stories can add up to more than the sum of their parts.

Google Wave cinema: "Pulp Fiction"

Not really — okay, not at all — safe for work, but a really nifty example of how innovative, multi-pane software (in this case, the soon-to-be late Google Wave), allowed one artist to take a scene from "Pulp Fiction" and render it within this program, weaving in videos, image, text, and maps.

TOC NY 2012 — O'Reilly's TOC Conference, being held Feb. 13-15, 2012, in New York City, is where the publishing and tech industries converge. Practitioners and executives from both camps will share what they've learned and join together to navigate publishing's ongoing transformation.

Register to attend TOC 2012

Related:

What we could do with really big touchscreens
More stories from the "Breaking the Page" project

Amazon started out as a book retailer, a company that was arguably a friend to book publishers, since it expanded the market for many of their books. But increasingly, the web giant is becoming a competitor to those traditional publishers, as the New York Times details in a recent article and as we have noted a number of times. Just as it did with book retailing, Amazon has its sights set on lowering the barriers between authors and readers, both via the Kindle and through its own publishing deals — and in many cases, the biggest barrier between authors and readers is a traditional publisher. Until that changes, Amazon will continue to win.

Although some are just beginning to see the company as a publishing competitor, Amazon has been marshalling its forces for some time now. As GigaOM Pro analyst Mike Wolf has described in a number of posts, the company has been putting together the pieces of a “book industry in a box” for the better part of a year — launching new imprints of its own for various different genres, including one devoted to popular thrillers. Then in May, it hired publishing-industry veteran Larry Kirshbaum, former CEO of the Time Warner Book Group, and opened a New York office.

In the months since then, Amazon has signed deals with a number of prominent authors, including one with popular writer Tim Ferriss, whose books — such as The 4-Hour Workweek and The 4-Hour Body — have sold millions of copies. The terms of the deal with Ferriss weren’t released, but the author said “The opportunity to partner with a technology company that is embracing publishing is very different than partnering with a publisher embracing technology.” Amazon also signed thriller writer Barry Eisler, who gained attention earlier this year when he turned down a $500,000 two-book deal with a traditional publishing house and said he planned to self-publish instead.

It’s not just about the money
Why are authors signing these kinds of deals? In some cases it could be about the money (a deal with former TV star Penny Marshall was reportedly for $800,000 according to the New York Times), but in many cases it seems to be mostly about getting past some of the legacy processes that are typical with traditional publishers, and expanding the potential market for a book. The core of the problem confronting the industry is summed up in a comment by Amazon executive Russell Grandinetii in the NYT piece, in which he says:

The only really necessary people in the publishing process now are the writer and reader. Everyone who stands between those two has both risk and opportunity.

If you look at the comments made by Barry Eisler about why he decided to take a deal with Amazon instead of self-publishing, he says virtually nothing about the money, or about other factors that traditional publishers are used to focusing on. It’s the other terms of the deal that he was swayed by: for example, the fact that Amazon was going to come out with an e-book version within a matter of days after the book was finished, and then follow that quickly with a paperback — and that both were going to be sold at a cheaper price, instead of the traditional industry’s approach of trying to charge print prices for electronic books.

What I care about is readers, because without readers I can’t make a living [and] I want people to read a lot. To that end, if I can find a way to get readers books that cost less and are delivered better and faster, I want that.

Just part of a wave of disruption
And as we’ve described before, Amazon signing deals to publish authors is just part of the bigger wave of disruption that is sweeping through the industry: self-publishing via the Kindle is becoming a larger and larger phenomenon, thanks in part to advocates such as JA Konrath and the kind of success that writers like Amanda Hocking have had by publishing their own books. Authors such as John Locke have shown that selling a million copies of a self-published book is not only possible but entirely feasible — and the fact that he and other writers who do so get to keep 70 percent of the proceeds is yet another wakeup call for the traditional industry.

And what kind of response have mainstream publishers had to all of this? Most have just continued to offer the old deals they are used to — deals that serve the publisher’s needs, but not necessarily those of the author. And in some cases, they have tried to punish authors who try to meet those needs themselves: the NYT piece describes how Hawaiian writer Kiana Davenport, who signed a book deal with Penguin last year, was threatened by the publisher after she packaged some of her short stories into a Kindle e-book. Penguin wanted all copies of the book removed from the internet; when the author refused, the publisher cancelled the deal and is now suing her for breach of contract.

Here’s a hint for book publishers: take a lesson from the music industry, and don’t spend all your time suing people for misusing what you believe is your content — think instead about why they are doing this, and what it says about how your business is changing, and then try to adapt to that. Amazon is giving authors what they want, and as long as it continues to do so, you will be at a disadvantage. Wake up and smell the disruption.

Post and thumbnail photos courtesy of Flickr users Umberto Salvagnin and Marcus Hansson

Related research and analysis from GigaOM Pro:Subscriber content. Sign up for a free trial.
As E-book Sales Grow, So Does DisintermediationDisruptapalooza 2011: how Amazon’s Kindle is changing the portable media gameBuilding a better paywall: strategies for monetizing news content

If the Internet is the public square of the 21st century, the Daily Dot wants to be its town crier. The newly launched online media startup is trying an experiment in community journalism, where the community is the web. It's an interesting vision, and one that looks to capitalize on the amount of time people are spending online.

The Daily Dot wants to tell stories through a mix of data journalism and old-fashioned reporting, where its journalists pick up the phone and chase down the who, what, when, where, how and why of a video, image or story that's burning up the social web. The site's beat writers, who are members of the communities they cover, watch what's happening on Twitter, Facebook, Reddit, YouTube, Tumblr and Etsy, and then cover the issues and people that matter to them.

Even if the newspaper metaphor has some flaws, this focus on original reporting could help distinguish the Daily Dot in a media landscape where attention and quality are both fleeting. In the hurly burly of the tech and new media blogosphere, picking up the phone to chase down a story is too often neglected.

There's something significant about that approach. Former VentureBeat editor Owen Thomas (@OwenThomas), the founding editor of the Daily Dot, has emphasized this angle in interviews with AdWeek and Forbes. Instead of mocking what people do online, as many mainstream media outlets have been doing for decades, the Daily Dot will tell their stories in the same way that a local newspaper might cover a country fair or concert. While Thomas was a well-known master of snark and satire during his tenure at Valleywag, in this context he's changed his style.

Where's the social data?

Whether or not this approach gains traction within the communities the Daily Dot covers remains to be seen. The Daily Dot was co-founded by Nova Spivack, former newspaper executive Nicholas White, and PR consultant Josh Jones-Dilworth, with a reported investment of some $600,000 from friends and family. White has written that he gave up the newspaper to save newspapering. Simply put, the Daily Dot is experimenting with covering the Internet in a way that most newspapers have failed to do.

"I trust that if we keep following people into the places where they gather to trade gossip, argue the issues, seek inspiration, and share lives, then we will also find communities in need of quality journalism," wrote White. "We will be carrying the tradition of local community-based journalism into the digital world, a professional coverage, practice and ethics coupled with the kind of local interaction and engagement required of a relevant and meaningful news source. Yet local to us means the digital communities that are today every bit as vibrant as those geographically defined localities."

To do that, they'll be tapping into an area that Spivack, a long-time technology entrepreneur, has been investing and writing about for years: data. Specifically, applying data journalism to mining and analyzing the social data from two of the web's most vibrant platforms: Tumblr and Reddit.

White himself is unequivocal about the necessity of data journalism in the new digital landscape, whether at the Daily Dot or beyond:

The Daily Dot may be going in this direction now because of our unique coverage area, but if this industry is to flourish in the 21st century, programming journalists should not remain unique. Data, just like the views of experts, men on the street, polls and participants, is a perspective on the world. And in the age of ATMs, automatic doors and customer loyalty cards, it's become just as ubiquitous. But the media isn't so good with data, with actual mathematics. Our stock-in-trade is the anecdote. Despite a complete lack of solid evidence, we've been telling people their cell phones will give them cancer. Our society ping-pongs between eating and not eating carbs, drinking too much coffee and not enough water, getting more Omega-3s — all on the basis of epidemiological research that is far, far, far from definitive. Most reporters do not know how to evaluate research studies, and so they report the authors' conclusions without any critical evaluation — and studies need critical evaluation.

Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science -- from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively.

Save 30% on registration with the code STN11RAD

Marshall Kirkpatrick, a proponent and practitioner of data journalism, dug deep into how data journalism happens at the Daily Dot. While he's similarly unsure of whether the publication will be interesting to a large enough audience to sustain an advertising venture, the way that the Daily Dot is going about hunting down digital stories is notable. Kirkpatrick shared the details over at ReadWriteWeb:

In order to capture and analyze that data from sites like Twitter, YouTube, Reddit, Etsy and more (the team says it's indexing a new community about every six weeks), the Dot has partnered with the mathematicians at Ravel Data. Ravel uses 80Legs for unblockable crawling, then Hadoop, its own open source framework called GoldenOrb and then an Eigenvector centrality algorithm (similar to Pagerank) to index, analyze, rank and discover connections between millions of users across these social networks.

There are a couple of aspects of data journalism to consider here. One is supplementing the traditional "nose for news" that Daily Dot writers apply to finding stories. "The data really begins to serve as our editorial prosthetics of sorts, telling us where to look, with whom to speak, and giving us the basic groundwork of the communities that we can continue to prod in interesting ways and ask questions of," explained Doug Freeman, an associate at Daily Dot investor Josh Jones-Dilworth's PR firm, in an interview. In other words, the editors of the Daily Dot analyze social data to identify the community's best sources for stories and share them on a "Leaderboard" that — in beta — shows a ranked list of members of Tumblr and Reddit.

Another open question is how social data could help with the startup's revenue down the road. "Our data business is a way of creating and funding new value in this regard; we instigated structured crawls of all of the communities we will cover and will continue to do so as we expand into new places," said Freeman. "We started with Reddit (for data and editorial both) because it is small and has a lot of complex properties — a good test balloon. We've now completed data work with Tumblr and YouTube and are continuing." For each community, data provides a view of members, behaviors, and influence dynamics.

That data also relates to how the Daily Dot approaches marketing, branding and advertising. "It's essentially a to-do list of people we need to get reading the Dot, and a list of their behaviors," said Freeman. "From a brand [point of view], it's market and audience intelligence that we can leverage, with services alongside it. From an advertiser [point of view], this data gives resolution and insight that few other outlets can provide. It will get even more exciting over time as we start to tie Leaderboard data to user accounts and instigate CPA-based campaigns with bonuses and bounties for highly influential clicks."

Taken as a whole, what the Daily Dot is doing with social data and digital journalism feels new, or at least like a new evolution. We've seen Facebook and Twitter integration into major media sites, but not Reddit and Tumblr. It could be that the communities of these sites acting as "curation layers" for the web will produce excellent results in terms of popular content, though relevance could still be at issue. Whether this venture in data journalism is successful or not will depend upon it retaining the interest and loyalty of the communities it covers. What is clear, for now, is that the experiment will be fun to watch — cute LOL cats and all.

Related:

Data journalism, data tools, and the newsroom stack
The growing importance of data journalism
Before you interrogate data, you must tame it
Social data is an oracle waiting for a question

Publication of information obviously includes traditional media, such as
books, newspapers, magazines, music, and video. But we can generalize
considerably to include blogs, tagging (e.g., Delicious, Flickr),
commenting systems, Twitter, Facebook, and Myspace.

From a biological
point of view, publishing can expand to encompass all of human
social signaling -- both verbal and non-verbal -- and include the myriad
little acts of information production and consumption we all engage in.

Even seen from this outer limit of generality, it's clear that
digital is ushering in a
rapid convergence in publishing. While some forms are born
digital and online, others are being reinvented there as technological
advance sets old media free. There is massive disruption -- both behind
and ahead of us -- as the convergence continues.

Three convergence trends: smaller, easier, more personal

There are three convergence trends in publishing that are already
apparent.

One clear long-term trend is that smaller pieces of
information are being published. Considering just modern digital forms
of publishing, there is a roughly chronological progression toward smaller
publications: emails, Usenet postings, web pages, blog posts, blog
comments, tweets, tags.

Traditional media are also being fractured into smaller pieces,
particularly where the media packaging existed only to address
physical quirks of the media or the act of publishing. To give one
example: Popular music publishing centered on delivering albums. This
was a by-product of physical equipment -- LPs, CDs, and their
players -- which did not align particularly well with the more natural
unit of popular musical output, the song. Given low-cost flexible
alternatives, it's no wonder that these forms of content are now jumping
the packaging ship and going directly digital in pieces that make more
sense. This leaves traditional publishers scratching their heads and
clinging to increasingly irrelevant and anachronistic packaging
methodologies -- newspapers being another example -- with attendant
declining advertising possibilities. Clay Shirky has written and spoken
with insight and eloquence on these changes (see here
and here).

A second trend is a reduction in
friction. As access to easy-to-use and inexpensive publishing
technology increases, it becomes economically feasible to publish
smaller and less valuable pieces of content. We have reached the point
where anyone with
access to the Internet can easily and cheaply publish trivial, tiny pieces
of information --
even
single words.

The third trend is the rise of publishing
personal information. Our inescapable sociability is driving us to
shape the Internet into a mechanism for publishing information about
ourselves.

These three trends -- smaller, easier, more personal -- provide a
framework to examine the development of online information publishing.

The three trends and the future of books

Over the last few months, interesting discussion has arisen about the
future of books and publishing. One provocative example is Hugh McGuire's
post "The
line between book and Internet will disappear."
Let's consider what the trends of smaller, easier, and more personal might
tell us about Hugh's topic: the future of books.

First, these trends reinforce Hugh's claim that the line between book
and Internet will disappear. The forces of convergence in publishing
are surely strong enough to drag the book across that line. But more
specifically, which of these trends will books succumb to? Which will
books resist?

Books typically have an internal coherence that may prevent their
traditional packaging from fracturing along more natural fault lines the
way it does with newspapers, magazines and albums. But as the difficulties
and costs of publishing continue to fall, and as methods for online billing
evolve, publishers or authors may themselves opt to fracture book packaging
for economic reasons. It was not long ago that novels were routinely
published in serialized form. If it's all digital, why not?

Because modern forms of publishing are giving
end users a voice, it seems a safe bet that books will become living
digital objects and that the traditional distinctions between author and
reader, and between publisher and consumer, will blur considerably.

Conceptually, though perhaps not technologically, there's a
long way to go. Even the most avant-garde online services are only now
contemplating this kind of future. I'm willing to bet that Hugh
is also right that publishers' products will have APIs. The API,
provided that it allows users and applications to write, can be
the vehicle by which a book is alive on the Internet, in the sense that it
will allow the contribution of information to books,
and make that information actionable.

Terry Jones will discuss the writable future of publishing at the next Tools of Change for Publishing Conference (Feb. 14-16, 2011). Save 15% on registration with the code TOC11RAD.

A world of writable containers

Looking at publishing from the broad perspective outlined above, with
its clear general convergence and specific trends, I consider it inevitable
that books and their publishers will be drawn into a digital future along
the lines that Hugh predicts.

You can look at this more widely, though. Publishing will converge
on the usage of underlying information storage that provides for a
world of openly writable containers. You could, for example, build a
Twitter-like system on such a basis, providing seamlessly for user
annotations. At the other end of the spectrum, you could use this type
of writable system to publish customizable living digital objects --
writable containers -- representing
books (or anything else). VC Fred Wilson lends weight to the claim of convergence toward
a more openly writable world in his blog post, "Giving every person a voice":

If I look back at my core investment thesis over the past five years, it is this single idea, that everyone has a voice on the Internet, that is central to it. And as Ev [Williams] said, society has not fully realized what this means. But it's getting there, quickly.

As Brian O'Leary noted in "Context first", mental models and mindset changes are required.
Shifting people from read-only thinking to imagining a computational world
that is by-default writable is something I've been trying to pull off for years. (FluidDB, a database we're building at Fluidinfo, is meant to explicitly prepare for the type of future Hugh envisions. Everything in FluidDB can be added to -- tagged -- by anyone or any application. )

Read-only containers of content are an
inherently limiting form of media, whether physical or digital. APIs that provide controlled access to information are similarly
limited. They prevent the accumulation of
unanticipated or personalized contextual information.

From one perspective, arguing that this kind of convergence is
inevitable may seem like a radical oversimplification or wishful thinking,
but from another it seems deadly simple and obvious. In plainest terms, I
believe the future of publishing is a writable one. One in which
we step beyond the default of read-only publishing via traditional
containers and APIs, to something that's both natural and empowering: a
world in which data itself becomes social, and in which we can personalize
arbitrarily. In other words, a world in which we always have write
permission.

Related:

The
line between book and Internet will disappear
Context
First (Magellan Media Partners)
Dancing
out of time: Thoughts on asynchronous communication
Getting
closer to the Web 2.0 address book

When Mandy Brown, Jason Santa Maria and I formed A Book Apart, one topic burned uppermost in our minds, and there was only one author for the job.

Nothing else, not even “real fonts” or CSS3, has stirred the standards-based design community like the imminent arrival of HTML5. Born out of dissatisfaction with the pacing and politics of the W3C, and conceived for a web of applications (not just documents), this new edition of the web’s lingua franca has in equal measure excited, angered, and confused the web design community.

Win free copies of HTML5 For Web Designers on Gowalla!

Just as he did with the DOM and JavaScript, Jeremy Keith has a unique ability to illuminate HTML5 and cut straight to what matters to accessible, standards-based designer-developers. And he does it in this book, using only as many words and pictures as are needed.

Watch Jeremy Keith discuss HTML5 with Dan Benjamin and me live on The Big Web Show this Thursday at 1:00 PM Eastern.

There are other books about HTML5, and there will be many more. There will be 500 page technical books for application developers, whose needs drove much of HTML5’s development. There will be even longer secret books for browser makers, addressing technical challenges that you and I are blessed never to need to think about.

But this is a book for you—you who create web content, who mark up web pages for sense and semantics, and who design accessible interfaces and experiences. Call it your user guide to HTML5. Its goal—one it will share with every title in the forthcoming A Book Apart catalog—is to shed clear light on a tricky subject, and do it fast, so you can get back to work.

4 May 2010
Jeffrey Zeldman, Publisher
A Book Apart “for people who make websites”
In Association with A List Apart
An imprint of Happy Cog™

The present-day content producer refuses to die.

And don’t miss…

Read Chapter One free in today’s issue of A List Apart!
The author, Mr Jeremy Keith himself, shares his thoughts!
Creative director Jason Santa Maria discusses the design of A Book Apart!
Editor Mandy Brown discusses the business side of A Book Apart!

With new technologies constantly coming on-line, and with states like California, Texas, and Oregon allowing digital curriculum to replace printed curriculum, the question arises: what will textbooks look like in the coming years?

Dale’s post, "A hunger for good learning," featured a fantastic video about teaching math. In a few brief minutes, Dan Meyer showed us a photo of a math problem involving filling a tank of water and calculating how long that would take, then showed us why traditional approaches to teaching this problem stifled student learning. The picture showed a traditional math problem with a line drawing of the tank, a problem set-up written in text (octagonal tank, straight sides, 27oz per second, etc.) followed by short sub-steps that are needed to solve the problem (calculate the surface area of the base, calculate the volume). Then, finally, it asks the question “how long will it take to fill the tank?” Dan’s view is that this spoon-feeding of problem solving in little steps trains students not to think like mathematicians and not to have the patience for solving complex problems. Instead, Dan prefers to show his students a video of the tank filling up, agonizingly slowly, until the students are eager to know “How long until that tank fills up, anyway?” And then they’re off -- discussing, questioning, and, most importantly, formulating the problem on their own, just as good mathematicians do.

It seems that what the textbook looks like in the 21st century is a lot more like Dan's presentation than the bound paper tomes we grew up with. If the 21st century textbook is delivered digitally to students, we can expect it to be far more than a .pdf representation of a traditional text. For example, let's say the textbook publisher chose to experiment with findings from the research community that kids learn better from authentic and difficult problems than they do from bite-sized steps laid out one after the other. The publisher does what Dan Meyer did, recreates the tank problem and updates a version of the textbook for a handful of beta testers. The next morning, Dan’s students walk into class and open the book to chapter 5. The old problem is gone, instead there is just a video of a tank and instructions that say “watch me fill up -- when you know how long it takes, please enter the answer.” Sure, a student might choose to watch the video for seven-plus hours and finally write down the time it took. But when boredom sets in, a more engaging option is to just play with the problem. By staying up to date with new information and practices, this textbook is living.

In this example, the student finds all the needed tools lying around the page. A ruler for measuring the size of the tank, a cup of known size and a stopwatch to measure the rate of water flow, as well as various other tools, leaving it to the student to decide which ones are relevant to solving the problem. This textbook is interactive.

On the opposite page of the book is a chat window where students can share hypotheses, discuss approaches, share results from using the tools (I get 18.4 inches for the height, but you got 18.7). This textbook is participative.

Of course, for the kid who already understands this deeply and finishes quickly, there are better challenges waiting. Similar problems with trickier shapes to the tank, problems where there is more than one pipe filling the tank up, problems where the rate of the water varies. This textbook provides each student with the right level of challenge at any given time -- it is adaptive.

If some of these problems require new tools and concepts, the student has the ability to research on the Internet, connect with tutors in higher grades, chat with other students across the world who happen to be wrestling with this same problem right now, or find and watch a YouTube-sized lecture on a relevant topic. This textbook is connected.

In addition to living, interactive, participative, adaptive, and connected, we can expect the 21st century textbook to be personalized and mashable. Beyond that, though, could the 21st century textbook hold out a unique promise - that the student who uses this kind of textbook no longer needs to wait for high-stakes, anxiety-inducing tests to determine whether he had learned a topic? What if the digital textbook were instrumented to collect and interpret data in such a way that it could tell a student's level of mastery without test-taking, just from how he engaged with the content? Some of these measurements and interpretations are easy to imagine, such as: 'Which digital tool did the student first pick up to make measurements in the tank-filling problem', and 'What keywords did he search for on the internet?' Other kinds of data will be harder to interpret, such as: 'What solutions did he try on his scratch pad', 'What questions did he ask his peers', and 'Which of his peers' questions did he answer?' But to any degree, what would it mean for a textbook to understand a student's level of mastery in real-time from his work in this digital medium? With what information could a teacher know exactly what next challenge would be optimal for each student’s learning on a daily basis?

What if the textbook publishers could see, in aggregate, how effective their content is, learn from that, adapt their textbooks, and redistribute new and improved content in months, weeks, or days rather than the current seven-year adoption cycles -- much in the way that Google measures our interactions with their applications and improve them based on the results. Depending on how well the beta testers in Dan's classroom learned to solve algebra problems, the textbook modifications might become standard for all algebra students. What if the instrumented 21st century textbook were able to measure both a student's learning and its own effectiveness, and that capability moved education innovation itself to Internet time?

Book By Its Cover is a glorious new blog devoted to the beauty of books.