Tweet: Content farms v curating farmers: Deeper insights in Demand Media’s model & finding opportunity in finding quality.
I spent an hour on the phone the other day with Steven Kydd, exec VP of Demand Studios, to understand their model—using algorithms to assign content creation based on search and advertising demand and to minimize cost and maximize revenue—because I wanted to learn a deeper layer of lessons than I think we’re hearing in the discussion of Demand’s allegedly evil genius.
The talk thus far misses their key insight and the opportunities they create. Much of what I see online is fear that Demand Media—with the slightly rechristened “Aol.” following fast behind—will cheapen content and flood the internet—that is, search results—with crap that’s just good enough to fool algorithms. Some also fear that while putting content creators to work they will put better content creators out of work: the dreaded deprofessionalization and deflation of media.
Michael Arrington marks the end of “hand-crafted content” (somewhere I hear Nick Carr and Andrew Keen cackling maniacally). And Read Write Web’s Richard MacManus worries that the web’s quality will suffer.
They may be right. But then again, the internet has always been filled with crap. So the challenge has always been how you find the cream. That’s where opportunities lie. That’s what Google saw. The new question is whether Google can keep ahead of the content farms and continually find new and better ways to find better stuff. I’ll bet on Google over crap-creators. But they better get cracking.
This is why, when I proposed an X prize to solve media’s key problems at Yale symposium, Clay Shirky responded with a call for work on what he called “algorithmic authority.” A few of my students’ proposals in my entrepreneurial journalism class tackled just this problem with discovering and prioritizing content for us: one using humans aided by algorithms, one using algorithms aided by humans; neither operated like a one-size-fits-all search engine (but then, soon, Google won’t either).
I think we may see search fall as the sole or even key means of discovery and filtering of quality content. I see three rings of discovery today: search (Google); algorithms (see: Google News, Daylife); and humans (see: Twitter). Note again that Bit.ly alone causes as many clicks a month—one billion—as Google News. Human power rises again. That’s what Fred Wilson says today when he argues that social beats search, because “it’s a lot harder to spam yourself into a social graph.” As search becomes more personal and no longer universal, SEO as a dark art and as the fertilizer for content farms will diminish and the social graph — our own circles of authority — will become more important in search as well. So I have faith that there are solutions to stem any rising tide of crap.
This is how I put it in a tweet: “algorithm-aided human writing will meet human-aided algorithmic curation; quality will rise.”
In all of this, I caution us not to miss Demand’s key insight: that the public should assign the creators, including journalists. The public often knows what it wants to know. I learned this lesson when I consulted at About.com and saw how they monitor search queries to see where there are questions for which the don’t have answers. When that happens, they go write answers; Demand automates the process. Makes sense.
This is not how we have operated in media: We decided which questions to answer because we asked them. What hubris! Today, I teach my students to find conversations on the internet and add journalism to them in the form of answers, corrections, reporting, explanations. In 2007, my students in a seminar at Burda in Munich and in my class at CUNY asked why the public doesn’t assign us and my entrepreneurial students in two classes have worked on that problem. Jay Rosen just started playing with this notion at ExplainThis.org, creating a platform for the public to ask reporters to report their questions. Demand and About are doing the same thing, only through search queries. Jeff Sonderman compiles some more examples. Where appropriate, reversing the assignment pipe is a good idea.
Demand is also creating a system they say will find the best writer for each assignment. We are free to disagree with their methods and results, but there’s insight here, too. Two students in my entrepreneurial journalism class won a grant to create a platform to do just this with local and hyperlocal news assignments (note that Kydd told me Demand isn’t touching news); I’ll report more on their project as it gets closer to launch. Can’t news organizations learn and steal some lessons from Demand? What if you wanted to create a content asset — say, a complete travel guide — and you opened up the process and its discreet tasks to a marketplace of paid contributors, enabling you to do larger projects at lower cost than before?
I always tell my students: Wherever you see a problem, look for the opportunity. That’s Arrington’s point: The next generation of content creation is here; deal with it. If you don’t like what Demand et al are doing, see the opportunity in it to surface quality content and to create competitive quality stars whose creations rise not just through algorithmic search cynicism but through human recommendation. Dig to the next layer.
I got lots of details from Kydd about the Demand method. In their view, they have combined content-creation and social-media platforms to enable content creators with “spare cycles”—his nerdy words—to earn money.
Kydd says 11 community members contribute to each article by fulfilling the discreet functions Demand identified: writing, copy-editing, copy-chiefing, reviewing titles, managing topic pages, checking facts. That is done by freelancers. The staff directs, edits, curates, and manages them. The algorithm makes all this more effective as it tracks content and ad demand and writes headlines for pieces it says will get traffic and earn money. Editors are 1.5 times more effective in creating assignments that will generate traffic, Kydd said, but the algorithm is 4.9 times better than creators.
Kydd said Demand pays from $0 (with revenue sharing) to $100 per piece; it averages at $20. Copy editors make $2.50-$3 per piece, which works out to $15-20 an hour. He said these people like to wake up and know there’s work they can do—there are 100k assignments waiting for takers right now—while they wait for old, human editors to respond to pitches. He said they also like being paid twice a week. Kydd said Demand employed 4,500 creators (text and video) and 400 copy editors in the last 30 days.
What amazed me most is that Demand uses its method not only for service content but for jokes at Cracked.com. Could an algorithm and social network replace Jay Leno? Easy.
: LATER: See also Doc Searls on junk food and chefs.
: Paul Marcum tweeted today: “Prediction: increasing clutter from algo content farms + mobile app convenience will have even @jeffjarvis paying for news by 2011.” I responded seeing the irony here: that value will come from aggregation and curation of quality content. But imagine then if the aggregators become more valuable than the creators and start charging; the creators (i.e., Murdoch) will go batshit. I’ve argued that in the link economy, there are two creations of value around content: from those who make the content and from those who bring together the public around it. Where is there greater value? We’ll see….
: LATER STILL: See Upendra Shardanand (founder of Daylife, where I’m a partner) on the need for new tools to create new handcrafted content. Problem is, he says, we’re using old text tools. See my related posts on storytelling and post-page media.
As I pointed out to Jay, http://HelpMeInvestigate.com is demand-driven media where people post questions and the site team work to help them answer them. The crucial addition is that we also help them to build or find a community around that question. In other words, to find enough eyeballs to make that bug shallow.
I am glad that you touch on the point about news. Sometimes journalists do need to find news that the audience isn’t aware of. I think that Demand’s model and others can help shape specific content strategy, but not necessarily for news. The way I see that it could work for news is through real-time search, but even then someone has to first break the news for others to know about it. Either way, there are surely lessons to be learned in producing content that is more strategic.
Hi, don’t want to spam about the company I work for that addresses that exact issue. However you could have a look at a blog post written by one of our users: http://www.theotherschoolofeconomics.org/?p=567
Where content is abundant, value predominantly lies in curation, filtering, connecting people with content – if anything, that’s what Google made clear.
Great post – thanks for the shout-out – content farming is hardly evil, just the latest, smartest way of giving people what they want (and, as it follows a long-established tradition of quantitative analysis pissing off writers, going to be a ripe target for some time to come).
On content creation, Demand has obviously dominated round one (and with some of the smarter guys I knew at Yahoo! over there I wouldn’t expect them to falter anytime soon) but I think the new Aol is going to give them a good run that will yield both increased quality and costs – here’s hoping a “content farm bubble” can save the economy!
As for aggregation/discovery, social is the filter to beat and I don’t see that changing. Should that continue to be the course, I think the following are some likely developments:
1. Google is going to have to nail social search. Facebook and Twitter are going to suck up a lot of oxygen here.
2. Quality producers will actually have a shot at more leverage. Maybe not enough to set terms but close.
3. The wide distribution of influence that Twitter has yielded will cause quality producers to actively recruit evangelists, evolving into a program that’s a digital version of the stacking of free papers at the LGA – DCA Shuttle gates.
I think that anybody who tries to “game” Google (ie AOL, Associated Content, Demand Media) is going to lose in the long run. Their strategy should be to beat Google, on their own terms.
In other words, if you play by rules that others have created, you will lose. If you rewrite the rules, you will win. This is the true definition of innovation.
Content companies should be developing new means and platforms for delivering content. If not, then they are ceding control and profits to the next platform creator (ie the next Google, Facebook, Craigslist or Twitter.) Imagine if the NY Times had invented their own version of Craigslist. We’d be having a very different conversation.
Murdoch is taking the Potter approach. Blogs are more like Bailey Building & Loan. The bank wants to make more money and the Building & Loan wants to keep everybody building. The Building & Loan is always fighting Potter and Potter needs more slums to keep people paying. You know where it ends, though it usually depends on where you start. You can go from a Potter slum to Potters field. Potter can bury you. The Building & Loan can help you buy a house and a car. I still have two dollars, so I’m still in business. Potter got a bigger bailout, but in the end Bailey had a better Christmas. This year the Potter bank bonuses are golden handcuffs. The Building & Loan provided tin parachutes. The greater value is in tin, because tin and copper make bronze. That makes war metals. Potter has more gold, but so what.
>>>Today, I teach my students to find conversations on the internet and add journalism to them in the form of answers, corrections, reporting, explanations.
Based on this approach, there would have been no Watergate investigation, since there was little public curiosity about the break-in.
Great point Robert!
While journalists may think that Watergate was a big deal, the regular public now sees it as a disgruntled employee beef.
Deep Throat wasn’t a saint. He was just another guy who thought that he deserved a promotion. Nothing he’s done support his belief.
And, speaking of little public curiosity, MSM is sitting on ClimateGate despite there being significant interest in all things climate. Or, as one wag put it, ClimateQuiddick, as the two responses have been spike and “there’s no story here”. (There’s also the NYT suggesting that leaking is always wrong, but they shut up about that fairly fast.)
Wouldn’t want to disturb the narrative, now would we?
>>>While journalists may think that Watergate was a big deal, the regular public now sees it as a disgruntled employee beef.
What a great comment on the future Jeff envisions for journalism . . .
and robert levine is at it again. i’m still laughing at the brilliant “Let them eat Googlejuice” comment.
anyhow, i’m glad, jeff, that you’re looking into the whole demand thing a bit more. it’s nice to hear that they don’t intend to “touch” news, but by simple virtue of existing on the same platform as news, where search/click are the only units of value, news will always lose. that’s because news isn’t trying to answer questions like, “how do i make papier mache,” but search/demand are. quite simply, search engines /demand and news serve two very different purposes.
i strongly disagree with you that i (we) know what i want to know. THAT’S real hubris. journalists are paid to research the stuff that those with hectic lives don’t have time to look into. responsible citizens acknowledge that there are other people with more specialized knowledge than themselves, and hear them out — with curiosity and skepticism — when they have something to say.
it’s not demand’s fault that news loses — it’s the fact that on our Internet, search is the path to all content. i vote for a search engine-free, ad-free internet (like HBO of the web) where quality content can live and not have to compete with crap. i’d gladly continue to use the crappy, free Internet when i want car part information. for everything else, though, i’m ready to pay!
No, Rob, this is just one method of journalism. By your logic, hiring people to write record reviews and teaching them to do it would also preclude Watergate. That herring’s as red as a Coke can.
Journalists keep saying “save journalism”. The problem is that their answers to “why?” aren’t compelling.
Watergate, Tet, and climate coverage are merely three examples of how journalists’ perception of value and benefit differs significantly from that of their readers.
This is important if journalism’s future involves readers.
> journalists are paid to research the stuff that those with hectic lives don’t have time to look into.
True, but only in the most trivial sense.
> responsible citizens acknowledge that there are other people with more specialized knowledge than themselves, and hear them out — with curiosity and skepticism — when they have something to say.
Also true, however, while journalists are paid, it’s not clear where they fit. They don’t have specialized knowledge. And, they’re pretty much failing as intermediaries.
You’re confusing the difference between types of articles (reviews and investigations) with the difference in approaches to journalism (from expertise and from populism). I proceed from expertise: Whether I’m assigning reviews or investigative articles, I believe I have a knack for figuring out what readers will and should be interested in, although I also take into account what they are interested in.
“What hubris!” you say above, so I’m assuming that you disagree. You proceed from populism, assuming that what people are interested in is what they should be reading about. That’s why you believe in citizen journalism, trust in open-source products and worship Google’s almighty PageRank system. And yet I suspect you’d identify yourself as an expert.
I’d suggest that the stories on the front page of the New York Times provide a more complete and more nuanced understanding of the world than the results of the top searches on Google. Care to compare?
Traditional media could learn more than a thing or two from Demand Media about monetising traffic. It’s seen its traditional areas of classifieds and display advertising nosedive in terms of revenue because of technology and recession. Rather than talk about raising paywalls – it should buy or invest in the likes of DM to create a new income stream to subsidise a free access news operation. It’s kinda obvious – but will they get it? Will they heck.
> I’d suggest that the stories on the front page of the New York Times provide a more complete and more nuanced understanding of the world than the results of the top searches on Google. Care to compare?
That’s an odd comparison.
The top searches, queries actually, show what people are interested in. Since news is a small subset of people’s interests, I wouldn’t expect the top queries to tell me much more about news than I expect from my cat.
Moreover, queries, top or not, don’t even list the results, let show us the content, which is where the meat actually is. Queries contain even less information than headlines.
The NYT’s competition is, wait for it, news services, and google search isn’t a news service, even though it can be used to find news.
Some of these news services deliver on paper, while others use via electronic means, such as “front pages” (msn.com is one), dedicated services (news.google.com is one), e-mail, rss readers/podcasts, blogs (the NYT’s legal news isn’t nearly as good as what I get from a couple of blogs), scroll-bars on screens, etc. Note that the selection process, whether automated and how much, is independent of the delivery means.
Watergate is an interesting case in point regarding the public’s default interest versus a Journalists drive to investigate and inform. How many years and how many political scandals later and those rascal politicians still try to lie, cheat and steal their way toward maintaining power. We need Journalists to continue tilting at the windmill of political corruption, but we don’t have a way to pay for it. I’m scared for the future of our democracy as the business that funded those tasked with pursuing truthful reporting evaporates and no alternative is evidently presenting itself.
> How many years and how many political scandals later and those rascal politicians still try to lie, cheat and steal their way toward maintaining power.
How confused do you have to be to think that anything that happens to a politician will disuade some future politician from trying the same thing?
> We need Journalists to continue tilting at the windmill of political corruption
“continue” implies that they do today. They don’t. They occasionally get their knickers in a bunch over pols who they don’t like, but are basically content to rewrite press releases. And that’s the way it’s been for decades, if not longer.
> I’m scared for the future of our democracy as the business that funded those tasked with pursuing truthful reporting evaporates and no alternative is evidently presenting itself.
If that scares you, why haven’t you been scared by the fact that “those tasked” haven’t bothered to “[pursue] truthful reporting” for decades (at least)?
People aren’t interested in “saving journalism” because journalism hasn’t done anything worth saving. Telling folks what journalism could do doesn’t help – it reminds them of the gap.
If you want journalism to be saved, do things worth saving.
No offense, but I sense a paradox. If blogs are a form of journalism, and if Jeff Jarvis is an intermediary offering commentary on Steven Kydd and Demand Studios, and also if Jarvis has little specialized knowledge of Demand’s codebase, for example, and therefore evidently falling into triviality as an adequate intermediary, then why should you have any interest in responding to comments made about what he is saying?
Although I agree that the current news business has a long list of failings, I think you’re using both too broad a brush and too narrow a definition to paint an accurate picture of what constitutes journalism and what it should be doing. The fact that you’re getting an alternate view of ClimateQuiddick means somebody is doing journalism to provide that view.
Without putting too fine a point on it, it can be argued that your comments are also a form of journalism in the context of Juvenal’s “Who watches the watchers?”
Andy, are you a failed journalist?
>> People aren’t interested in “saving journalism” because journalism hasn’t done anything worth saving. Telling folks what journalism could do doesn’t help – it reminds them of the gap.
(that does sound a little bitter)
Who are the “people” you refer to?
Call it what you will, but I think there’s a lot of interest in “saving journalism.” Jeff Jarvis appears to make his living analyzing and commenting on the demise/salvation/reinvention of journalism (“reminding them of the gap.” as you say).
>> If you want journalism to be saved, do things worth saving.
It sounds like you have some things in mind. What are they?
Jeff – Thanks for another great post and also more balanced than I’ve seen elsewhere on this subject – seems like an awful lot of people are setting themselves up as judge and jury on the “quality” of content.
In fact this whole issue is why I felt driven to comment – on what basis do you judge the quality of an article such as “How to potty train a ferret” (on Demand’s ehow.com) it’s not possible to compare it to an expose on government corruption but it doesn’t mean it has any less value to the individual searcher / reader.
Also I wonder how much the type of content produced by Demand Media and others is cheapening content and flodding the internet – this content only surfaces when relevant searches are put in.
Ultimately won’t we (the users of the web) be the judges of content quality – the good stuff will be linked to and rise to the top.
the old media model was
a) identify content areas we think will be in demand
b) bring in talent to create the media
c) spend a lot of money distributing and marketing it
Demand Studios represents an example of a more efficient new model
a) automatically identify areas of real immediate demand
b) efficiently allocate talent and production resources to create media to fill this demand
c) automatically distribute to the sites where the demand exists
On the internet you can’t spend enough money on distribution and marketing to drive demand for something you are guessing people will watch. There are too many options for people to choose from.
It’s easy to see which model will survive. Spot the internet video company who’s not laying off (or still paying for) talent right now?
Plus quality matters. Why? because a better, higher quality show is simply a click away and viewing is immediately measurable. The audience will decide the quality, not the producer/studio.
The market will set the price directly against quality produced by the talent. Talent that performs well will be able to set higher fees. If Demand wants higher quality, they will have to pay more.
It’s a shame Demand’s platform is still studio-like however, effectively providing financing, and choosing who produces what content. Acting as a middle-man means controlling information on the two sides of the market, which is not good.
I feel they are missing a trick.
Demand should disintermediate themselves. How about reversing the market?
a) automatically identify areas of real immediate demand
b) auction off these areas to the highest bidder with a minimum level of quality [based on past performance]. YES, the talent pays for the rights
c) talent creates the media, controls the story
d) the platform automatically distributes the media to places where demand exists
e) everyone shares in the revenue
It’s more scalable as it’s driven directly by the people that can supply the media and talent is incentivised to produce something compelling and of high quality. And the talent is [always] paid based on performance.
(originally posted 1 year ago on newteevee – http://newteevee.com/2008/10/31/the-state-of-the-media-in-two-acts/#comment-248467)
> Call it what you will, but I think there’s a lot of interest in “saving journalism.” Jeff Jarvis
There’s a lot of interest among journalists, such as Jarvis. Outside of that, not so much.
If saving journalism requires the cooperation of “not journalists”, such as readers, that’s a problem.
>> If you want journalism to be saved, do things worth saving.
> It sounds like you have some things in mind. What are they?
How about never promising something that you don’t deliver, and never saying “you’ll miss us because” when referring to something that you don’t actually do.
I’m an optimist. I believe that when people tell the truth, they do better things that they do when they lie about what they do.
However, even if I’m wrong about that, eliminating the gap between what journalists say that they do and what they actually do is likely to help.
“We read and reprint press releases so you don’t have to” is a value proposition.
> if Jeff Jarvis is an intermediary offering commentary on Steven Kydd and Demand Studios, and also if Jarvis has little specialized knowledge of Demand’s codebase
Jarvis doesn’t have to know Demand’s code to comment usefully on what he does know, advertising models. More to the point, his comments don’t turn on the code. The climate models do.
FWIW, I don’t take Jarvis as an expert on everything he says.
> The fact that you’re getting an alternate view of ClimateQuiddick means somebody is doing journalism to provide that view.
Yes, “somebody”, but, for the most part, not professional journalists. You remember professional journalists, they’re the folks who claim to be informing us on such matters.
I’m not objecting to journalism. I’m pointing out that professional journalists aren’t very good at it.
There’s much good here both in the post and the discussion.
However, speaking of herrings, I think there’s an implication of a false either/or here. In B2B media (yes, the poor stepchildren), before the Web was prevalent we spent a lot of money on reader surveys, and more significantly most interviews began or ended with questions like “So what else are you thinking about? What’s keeping you up at night?” Yes, gasp, this concept predates the Internet.
I think most publications run a mix of what we KNOW readers want and what WE think they will want. Hubris? I guess so. Any act of creation requires a little, wouldn’t you say?
Hi, I recently wrote an article on content farming at Smart Mosch. But what surprises me to find is that these articles are not that rare actually. Searching for “content farms” actually returns very relevant results. So this means..we’re actually moving! Of course, there’s more to wait until “what is given” meets at its highest “what is actually relevant” in terms of content and news.
You can read my article: “Content Farms: To Write or Not To Write for?” at http://www.smartmosch.com/2010/01/13/content-farms/
I do agree with all the ideas you have offered in your post. They’re very convincing and can definitely work. Still, the posts are very brief for starters. Could you please prolong them a little from next time? Thanks for the post.