On Crowdsourcing

I need to temporarily interrupt my regularly scheduled programming to address some remarkable changes and vigorous debates within the field of digital humanities. The following thoughts must be taken with a grain of salt, as I do not profess to be an expert on dh or computing as a whole. However, as a graduate student who is learning more about dh and anxious to see the field progress in a positive manner, I feel it necessary to speak out. I will try my best to address the heart of the matter and cogently summarize my concerns.

Not too long ago, an open access digital publication called The Journal of Digital Humanities was created in an effort to promote “a comprehensive, peer-reviewed, open access journal that features the best scholarship, tools, and conversations produced by the digital humanities community in the previous semester.” However, concerns were raised about inconsistent publishing standards for peer reviewed work and a lack of transparency about the review process. In response to these concerns, a group of scholars has established a new website called DHThis, which aims to cultivate and promote dh content based on the opinions of its membership. More specifically, users will have the power to vote “yes” or “no” as to what content they want to have published on the website’s front page. As Michael Widner explains, DHThis has embraced a belief that “openness and the wisdom of the crowd [within boundaries]” trumps the traditional peer review process for selecting good scholarship. While acknowledging that the peer review process has its problems (especially when the process is ostensibly “blind,” yet the content under review is “post-publication,” aka freely available online), I strongly disagree with the notion of crowdsourcing online content in the form of a yes/no binary.

Jeff Howe, a writer for Wired magazine and one of the first people to coin the term “crowdsourcing” in 2006, defines it as such:

Crowdsourcing is the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call.

Wikipedia may constitute the most popular and successful crowdsourcing operation in human history, but there are many other online projects that also exemplify what I think is the true essence of crowdsourcing. In 2011, the New York Public Library created a project called “What’s On the Menu?” Through this initiative, NYPL made a call for help in transcribing upwards of 45,000 historic restaurant menus, some dating as far back as the 1840s. 45,000 menus is obviously a lot of transcribing to do for a small staff, but not impossible for a group composed of thousands of internet users who want to help out with an ambitious project that could help historians, chefs, and food lovers ask new questions about the history of food. Around the same time, Nicole Saylor of the University of Iowa Libraries embarked on a crowdsourcing project to transcribe Civil War diaries which led to upwards of 70,000 hits a day from interested visitors anxious to help transcribe these important documents.

With each of these projects, crowdsourcing led to several important achievements. For one, previously inaccessible historic documents are now accessible and readable to anyone looking to conduct research or learn more about a given topic. Equally important–if not more important–these projects allow users to take a sense of ownership in these cultural heritage institutions by giving them an opportunity to engage in meaningful work that advances the goals of the institution while also instilling a sense of satisfaction within the user. “At its best,” argues Trevor Owens, “crowdsourcing is not about getting someone to do work for you, it is about offering your users the opportunity to participate in public memory.”

However, it seems to me that the yes/no voting system DHThis has embraced for selecting good dh content represents the kind of crowdsourcing that has the potential to be quite loathsome. I generally avoid Reddit and dislike its yes/no voting system for selecting content, largely because it has perpetuated what Natalia Cecire describes as a “terrifying cesspool of misogyny, racism, and assholery.” I do not believe that DHThis will promote or encourage any sort of behavior like that, nor do I think “the people” are incapable of choosing good content, but there are other problems that a yes/no voting system brings to the table, in my opinion:

Yes/No Crowdsourcing is essentially a popularity contest: It seems to me that those already closely involved with dh (those deep inside the “big tent”) and have a strong digital presence are going to be the ones shaping the content of DHThis (and perhaps even dh as a whole) for the foreseeable future. They are the ones with thousands of Twitter followers, personal blogs that get lots of views, and posts that will get selected for voting in DHThis. Maybe this is the way things should be, but it seems like we should be asking if there are better ways for incorporating new and younger voices into the dh discussion. Who might be left out by leaving curatorial duties soely to the DHThis membership?

Not everything can be whittled down to a yes/no binary: News sites all around the world have gone to the yes/no binary to help determine the most popular comments on a news article, often through programs like Discqus. Thus, a September 11 Washington Times article on the “Million Muslim March/Two Million Bikers to DC” controversy that yielded 5,800 comments had the following as its most popular comment, with more than 2,700 thumbs up votes: “Someone should have set up catering trucks serving pulled pork and BLT sandwiches.” And as mentioned, other sites like Reddit rely on yes/no votes with commentary that is also lacking. As a scholar, historian, and writer, I think I deserve more for my efforts at public writing than crowdsourced yes or not votes. Facebook is actually quite genius if you think about it. There is no ‘dislike’ button on the site, and there will never be such a button. If I post a picture of myself on Facebook having fun or comment on the passing of a loved one, the last thing I want is to have people ‘dislike’ my actions. Similarly, I would hope that if someone doesn’t like my writing, they make an effort to provide transparent and constructive criticism, not an anonymous “no” vote.

Serious thoughts and arguments deserve better than a yes/no choice. If I were to submit an article for DHThis or if it were scooped up for voting, I would feel quite uncomfortable by the idea of someone dismissing my entire argument by simply voting “no,” especially considering the fact that it takes a great deal of courage to even write for a public audience in the first place. A yes/no environment makes me feel unwelcome, plain and simple. I realize that DHThis is an experiement, and I want it and others doing dh projects to succeed. For now, however, I’ll think continue to observe these discussions from afar and attempt to learn more about digital technology in my free time.

Cheers

Advertisements

3 responses

  1. I can empathise with the rejection of the yes/no vote and the simplification it entails. It’s interesting as well it’s actually an up/down vote. (Apparently “upvoting” is a verb). What are the implications of “downvoting” a link submitted by another colleague? It should be noted that the Pligg set up means that what gets voted is not necessarily the article/post/content itself, but the *link*, the submission, and this means by extension the person who submitted it as well. (Often those who submit links are the authors of the content linked to, but under current DH vibes, will that judged as “shameless self-promotion”?)

    This complicates things, as it’s not only the content being judged, but the participation via submission. I suppose the idea is that collegues submit content by other colleagues as well. Will a submission of a link to someone else’s work be interpreted as an endorsement of the content? (as with RTs?) When a link is upvoted, have people read the linked content or what other motivations are in play?

    Something that fascinates me is that as you pointed out it is clear (though it was not made explicit in the project’s About page) that DHThis is a response to the Journal of Digital Humanities and therefore to DHNow. But how different is it from DHNow?

    By the way, I’ve been a participant in both platforms (DHNow and DHThis), and I guess I have supported them by contributing. (I say this to avoid somoene thinking one is only critical in a negative, unproductive sense).

    1. Hi Ernesto,

      Thanks so much for this thoughtful comment, which further clarifies some of my concerns and correctly points out that it’s an up/down vote. I agree that the participation via submission is being judged too, and you present good questions about endorsements and exterior motivations that may influence someone to support a specific piece of content. Finding explanations or answers to these questions is quite tough.

      Regarding DHNow, I will have to admit that I am fairly ignorant of the platform and how it functions, and that is my fault. I was vaguely aware of its existence, but I observed that a lot of the conversation was revolving around JDH (which I am familiar with), so it appeared to me that the much of the impetus for starting DHThis revolved around the perceived shortcomings of JDH. I will have to further analyze DHNow.

      I suggested on Twitter today that DHThis consider the possibility of a tagging system with room for more extensive commentary on pieces up for submission. I also recommended that by incorporating a similar/recommended readings section at the bottom of each front page post, submissions with similar tags that do not have as many upvotes can be more prominently displayed and considered by readers for upvoting. I would keep the upvotes but remove the downvotes.

      Thanks again for the comment, Ernesto.

  2. […] brief addendum to my post yesterday on crowdsourcing and […]

What do you think? Leave a comment here!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: