To Crowdsource or Not To Crowdsource?

Sneaking its way into many of our conversations of the last month and half has been a debate over the value of crowdsourcing.  Should we do it?  Is it useful?  My original intention with this post was to offer a defense of crowdsourcing as a valuable endeavor for academia.  While I still think that, ultimately, I am supportive of crowdsourcing or something similar, the fact that it has taken me two weeks of stops and starts to write this post speaks to my own struggles, doubts, and uncertainty about our project as a crowdsourcing project.  As I understand them, some of the concerns about crowdsourcing from our team are as follows:

  1. Does crowdsourcing (used for interpretive purposes) create any kind of useful data, knowledge, or insights for academic purposes?
  2. Does crowdsourcing treat people like cogs in a system, resulting in their dehumanization?
  3. Assuming that we want our project to be relevant beyond academic walls, can a DH crowdsourcing project actually reach beyond those walls?  Are non-academics interested?  Will people from other walks of life even have the chance to be exposed to such a project?

I think these are legitimate concerns which we need to address.  However, I am going to argue that we should, nevertheless, endeavor to incorporate some sort of crowdsourcing aspect into our project.  Without roaming too far away from the central issues, I will try to explain why:

One of my primary concerns with academic pursuits is a failure to consider the implication of our profession, research, and practices for the society beyond our own institutional borders.  This might be a surprising criticism of a world that often (though not always) hails the insights of feminism and postmodernism, which invite an acknowledgment of subjective meanings and encourage a self-reflective awareness during research and writing.  However, while this approach may be used within the confines a research project, I often the lament the degree to which we fail to ask important self-reflective questions about research and academia in general.  What is the role of a university?  Of a library?  How do our practices matter in the world beyond the ivory tower?  How might our scholarship influence policy, definitions of truth, or identity?   What are the lines of communication and influence between our universities and the rest of society?   Is our scholarship relevant to someone aside from other academics?   As privileged members of the some 30% of the U.S. population who graduate from college and the even smaller enclave who make their living in academic institutions, I feel we are obligated to ask these questions.

Part of what attracted me to the Praxis program was the use of crowdsourcing, which I saw as an opportunity to engage in these big questions by bridging the space between the ivory tower and the world beyond.   Not only could a crowdsourcing tool have research potential, it could also help to make us aware of the world outside our own institutions and to (hopefully) keep an eye toward that world as we pursue our own scholarship.   I am still excited about this potential despite our concerns over crowdsourcing.  So here are my thoughts on the three questions listed above:

Will a crowdsourcing project produce useful information for academic pursuits?  Perhaps not in direct manner.  But, it could clue us into important and relevant questions which we can then address in our work, ensuring a certain degree of relevancy between our work and the nonacademic world.

Does crowdsourcing turn people into dehumanized cogs?  This is such an important question as we endeavor to be responsible and reflective scholars!  The way in which crowdsourcing has been used in the past to harness the energy of the masses makes this a legitimate concern.  However, I think our team is in agreement that we are interested in a project that centers on interpretation rather than using “the crowd” to accomplish a particular project.  Perhaps this implies that we need a new term for this approach—maybe we are not really doing crowdsourcing so much as suggesting that through a collaborative interpretive project we could get a sense of the pulse of a community.   Does this still result in dehumanization of individuals?  I am uncertain—this is definitely a concern which deserves more conversation.

Can a DH project really reach beyond the walls of the academy?  Could it provide a line of communication between professional scholars and others?   I think that the right kind of project with an inviting, playful interface and an approach that is presented in non-specialist language could, in fact, do just that.  However, I agree with some of my team members, that actually reaching beyond the academic community would be challenging.  Reaching beyond the 30% of the population which is college educated might be even more unlikely.  But I am still excited by the potential to do these things, and don’t think that the difficulties they present should prevent us from undertaking them.

I’m sure there are other concerns when it comes to building a tool that depends on crowd participation.   I feel as if I have barely tipped the iceberg on this issue, so comment away.  Should we crowdsource?   What are the benefits?  The pitfalls?  How does this decision relate to the larger mission of our work and the work of academic institutions in general?


  1. I’d like to respond quickly to Brandon’s last point: which texts we upload to Prism matters. Several of us were having a brain storming session today and discussing what kinds of texts might be the most useful or most problematic for Prism. Perhaps texts which we encounter everyday and require less context would be the most provocative for collaborative interpretation. Examples would be things like the constitution and declaration of independence, marriage vows, inform consent documents, religious texts, and popular music lyrics. These things are texts, but not literature. They also are usually shorter and self-contained texts (rather than experts from larger works). So this adds an additional question to keep in mind as we move forward. Does Prism work for some texts in ways that it does not work for others?

  2. These are really great thoughts, Claire. Some responses to your three main questions:

    1. and 2: I think that this sort of crowdsourced interpretation is only useful insofar as we can keep the users from becoming cogs. I’m not sure that the mass of information on an overhead tells us anything self-evidently. It’s up to humans to interpret that information, and the process of facilitating those conversations is only aided by keeping a sense of the individuals that went into the markings. After all – Prism is unique in that its data is not generated by a machine. It comes from people. We need to embrace that humanity. I see two immediate ways to retain this sense of the human:

    a) Retain a sense of each user’s unique set of markings within the aggregate. I talk about this a bit in an upcoming post. Last year’s team looked forward to this, based on some of their visualization drawings. I think it’s crucial for any use.
    b) Recognize that the data is only useful as it is used and interpreted by people. To this end, I think we should craft the tool in such a way that facilitates those conversations, either in person or digitally. I think this is what the visualizations are meant to do ultimately. You could make the argument that a lot of academic research is done in isolation, but I think this is a bit misguided. There are countless articles and books that talk about the first paragraph of Portrait of an Artist as a Young Man in terms of realism vs. modernism. We shouldn’t forget that none of those writings occurs in isolation – they speak back to each other. Writing in academia is always a form of conversation, but it is one that happens on a sometimes incredibly slow and protracted basis. Prism is a different, graphic form of these same sorts of conversation.

    Prism starts with humans. It ends with humans. It’s our job to make sure that the tool doesn’t obscure the connection.

    3. In relation to going beyond the academic community – I think that the uses for a broader community are definitely there. Prism is all about critical readings. Cecilia’s suggestion for photographic Prisming is a great reminder of how we all interpret things every day. Life is an interpretation, filled with smaller, fleeting moments of analysis. Insofar as we all have a stake in more nuanced understandings of how we move through and interact with the world, I think we all have a stake in the kinds of questions that Prism asks. It’s not unlike the sort of analysis that pundits do on transcripts of political speeches everyday; Prism just takes a graphic approach rather than a verbal one. The first step then, might just be to open the tool up to those texts and materials that people outside the academic community interpret everyday and might care about: images, speeches, legal cases, religious documents, etc.