For my proposal, I’m offering up a roughly formed-slammed together session that is sorta make and sorta talk based. Throughout the last few years, the evaluation of digital scholarship has been of growing focus particularly within the context of scholarly communication and, to my mind, within the digital humanities/digital literatures communities and the new media/media studies communities.
There’s a laundry list of voices being thrown into the mix—whether in media (e.g. the numerous Chronicle, Slate, InsideHigherEd articles), in scholarship (e.g. evaluation as a key consideration in monograph length dh works), in presentations (see the tweetstream from AHA2012/2013/2014 or DH2012/2013), or in policies (see MLA’s guidelines). The Journal of Digital Humanities and others are steadily increasing the review of digital projects. Even JAH has gotten into the game of evaluating digital projects as a key scholarly activity.
Pedagogically, we are seeing evaluation exercises becoming more prominent in both undergraduate and graduate courses. Brian Croxall’s done this with his undergrads, Doug Seefeldt and Will Thomas have done this with their grad students and there are a growing number of Digital History courses that ask students to evaluate digital projects….AHA’s even offered a report (under the rubric of public history) about best practices for review.
Yet, for all of these discussions and resources about the value of digital work within teaching and promotion, there is little consensus for digital historians on the explicit components that projects should be reviewed, assessed, and evaluated on. I’m proposing to lead a group think exercise where we create a checklist, question list, or some other sort of evaluative framework that could be used by non-digital historians to familiarize themselves with how digital history projects should be reviewed and evaluated. I imagine something that reflects a shared value discussion…what matters within digital history and how to we want to be evaluated on our scholarship? If a book review does these certain things to be considered a “good review” then what must we as digital historians look for in digital projects for a project to rated a “good” digital project.
Oddly enough, I logged on this morning prepared to suggest my own version of this session (though I doubt it would have been as well thought out as Jennifer’s proposal). One of the issues we run in to in evaluating digital scholarship is that it comes in so many forms and I would argue the reviewer needs to understand at least the broad outlines of those forms to evaluate the project on its merits.
So, yes, this is definitely a conversation we need to have. Even better if we can emerge from the session with with an “evaluative framework” (or two or three) for historians.
[Full disclosure: I have a vested interest in this conversation since I’m the contributing editor for Digital History Reviews at the JAH.]