Here is a description of the ASECS Digital Humanities Caucus Roundtable 2 including the participants’ talking points. Please feel free to add comments on the panel or the issues raised.
“Evaluating Digital Work: Projects, Programs, and Peer Review” Session Description:
As new media projects begin to supplement or in some cases replace the print essay, research paper, scholarly article or monograph, what modes of evaluation should we expect or demand of students (undergrads and graduate students), colleagues, and ourselves? In recent times groups like HASTAC and the MLA begin discussion on evaluating digital work — the latter appropriately enough ona wiki — while the president of the MLA has even advised that more digital dissertations be produced. Similarly, the American Historical Association and the Center for History and New Media at George Mason University announced in 2008 a new annual prize for “an innovative and freely available new media project that reflects thoughtful, critical, and rigorous engagement with technology and the practice of history.” Since our students now might ask to produce a YouTube video instead of a paper, and our colleagues go up for tenure with a digital archive instead of a book, it behooves the member of ASECS to join this discussion and reflect on both methods and criteria for judging specifically historical work.
The organizers welcome descriptions of your own or others’ classroom or institutional experiments; policies that you or others have authored; ideas for future criteria, rubrics, methods and standards for evaluation; as well as theoretical examinations of evaluation as a concept. Panelists might reflect on the following, applied to both students and faculty:
- Which standards should we preserve and which reject as we move from evaluating print work to digital projects?
- What are the digital equivalents to (or replacements for) familiar print products such as (for undergrads) the interpretive essay or research paper; (for graduate students) the seminar paper or dissertation; (for faculty) the journal article, chapter or book?
- How do we transfer such criteria as length and depth to innovative projects?
- How do we account for the new skills that may need to be acquired to produce works in new media?
- How do we fairly assess forms requiring multiple team members?
- How can we import crowdsourcing as a method into the classroom or for examining scholarly work?
- How do we allow for citation and account for influence in the digital realm?
- What is the relationship between methods of evaluation for students and that for faculty?
- How do new media projects de- and re-construct print-based concepts we bring to evaluation, such as periodicity, authorship, originality, knowledge, and intellectual property?
- How might digital pedagogy and service be reconceived as valuable and quantifiable forms of scholarly work?
Participants:
Associate Professor and Chair of English
Co-Director, Gender Studies Institute
Co-Editor, Digital Defoe: Studies in Defoe & His Contemporaries
In my presentation, “Digital Scholarly Journals and Peer Review: A Case Study,” I will raise a series of questions and offer some preliminary answers to them in light of my experience as one of the two editors of Digital Defoe: Studies in Defoe & His Contemporaries, an online multi-media, peer-reviewed scholarly journal first published in 2009. These questions include: How are submissions to digital journals evaluated? Are they evaluated any differently than the work submitted to print journals? In particular, how are (and should) mixed-media submissions be peer reviewed? Is only the verbal content of such submissions peer-reviewed or is visual presentation, auditory quality, etc. considered, and, if so, are English professors qualified to engage in this kind of peer review? Should digital journals take advantage of new presentation modes in publishing scholarly submissions or will this make it more difficult for scholars to move smoothly through the process of tenure and promotion? Is concern about how multi-modal works will be evaluated by tenure and promotion committees the reason that few scholars take advantage of what digital journals can offer them? How (if necessary) should those who publish in digital journals explain the “worth” or “weight” of their publication(s) to tenure and promotion committees and how can editors of digital journals assist contributors who are concerned about this issue?
2. Bill BLAKE, University of Wisconsin, Madison
Lecturer of Early Modern Literature
Director of Literary & Linguistic Computing
University of Wisconsin-Madison
3. Allison MURI, University of Saskatchewan
Assistant Professor of English
Muri will discuss teaching, the “Grub Street Project,” and the development of a new undergraduate Minor in Digital Culture and New Media.
Assistant Professor of English
Trico DH Faculty Liaison
The project I will describe arises from my DH work in liberal arts colleges. Over the past two years, I have organized and hosted various activities focused on new media, DH, and undergraduates with my colleague, Katherine Rowe (Bryn Mawr College)—including a Mellon-sponsored national seminar (‘Digital Archives and the Future of the Humanities at Liberal Arts Colleges’), which brought in ODH representatives; and the national liberal arts student conference “Re: Humanities,” which will take place at Haverford this November. Rowe and I are now in the process of spearheading a Trico (Swarthmore, Bryn Mawr, Haverford) DH Regional Center. As we think about the role of DH in the undergraduate intellectual experience, I have created course assignments around the construction of a digital archive in the liberal arts classroom. I will describe the work with reference to an advanced course on Restoration and 18th-Century Print Culture.
Thanks, Lisa and co., for a great panel and roundtable discussion. The thought that I wanted to pull out from our discussion was that we should regard our research as “work process” rather than “work product.” This insight is as important for our students as it is for us. For those (i.e. everyone) working in divided or oblivious departments, initiating more fine grained evaluations of teaching (i.e., peer review, portfolios, etc.) could also wake people up to the potential uses of this stuff for both research and teaching. DH is threatening to scholars to the extent that it makes visible their working assumptions and deeper investments concerning writing, teaching, scholarship, etc. To gain adherents, this like any other form of faculty-initiated change needs to pick manageable projects and goals, collaborate as much as possible, cultivate and grow a learning community among faculty, and see how this could fit in with existing priorities and practices at your institution. Good luck, DM
Pingback: Humanidades digitales. Evaluar lo heterogéneo » Ernesto Priani - Sed fugitare decet simulacra