Julie and Vicki have published a research report on using audio feedback for peer review on student writing.  They used iPods distributed as part of the Duke Digital Initiative, to test if audio feedback is an effective way for students to offer high-quality comments to each other on their writing.   Their report “Can you hear us now?: A comparison of peer review quality when students give audio versus written feedback” (pdf) is published in the annual 2008 edition of The WAC Journal, a national peer-reviewed journal on writing across the curriculum.

Julie and Vicki noticed that when they gave students feedback on their writing, audio feedback was more time-efficient and seemed to be of higher quality than written comments.  They designed a study to find out if students would experience the same efficiency and effectiveness using audio feedback for peer review.  Students in their classes gave and received peer reviews using both audio and written comments.  Students were surveyed about their preferences and perceptions at the end of the semester.  In addition, Julie and Vicki assessed the quality of the peer reviews using defined criteria and two raters for each review.

They found that audio peer reviews contained more specific and higher order comments than written peer reviews.  They conclude that audio feedback significantly improves the quality of peer reviews.  The paper finishes with concrete suggestions based on their results and experiences with students for effectively using audio feedback in the classroom.

This paper is a great example of the Scholarship of Teaching and Learning; Vicki and Julie have carefully researched “what works” in teaching using audio peer reviews, and offered suggestions for others based on their results.

Project Description:

Vicki Russell, Director of the Duke University Writing Studio, is investigating innovative ways that tutors can work with students on writing assignments, and students can collaborate on writing projects. Russell, using 3d virtual worlds software, has created a virtual Writing Studio – an online, 3d “space” where students, faculty and writing tutors can collaborate in real time.

The goals of the project are to use the extensive resources for writers available on the Writing Studio Web site as a foundation for creating an interactive virtual learning environment, demonstrating for writers that writing is a dynamic rather than static process. Allowing access to these resources during a tutoring session with students in a real-time virtual environment will provide interactive ways to facilitate writing as recursive rather than linear process. In addition, by creating a virtual environment space for collaborative writing that is expandable and reproducible for other units at Duke, the project will focus on helping writers improve individual written texts and become more self-reflective better writers and provide faculty with tools to help their students become more effective writers and critical thinkers.

Promoting a larger “culture of writing” on the Duke campus, the resource will include spaces for exploring writing resources in non-linear ways to help students during the writing process. The online presence will be a meeting place where writing tutors can assist students with writing assignments and spaces where student organizations can collaborate on writing projects for publication.

Russell was awarded a CIT Strategic Initiative Grant to facilitate the early stages of her project. Russell used funding from the CIT grant to learn more about virtual worlds applications, to investigate different virtual worlds tools determine technical suitability for the project and to survey students about current and potential virtual worlds interest and use.  The grant was also used to develop a plan and paper-prototype as a blueprint for implementing the future development of the virtual Writing Studio.

During the grant term, Russell, working with Writing Studio tutors and graduate students Richard Musselwhite and Jen Walsh, compared different virtual worlds tools (Second Life, Protosphere, Cobalt, and Qwak) and their possible suitability for the project.  Based on factors such as the ability to custom program objects in the application, ease of use, suitability for an academic environment and the facilities for text collaboration, the team choose Qwak, a commercial product built on the Croquet/Cobalt engine, for initial development, with a plan to port the Virtual Writing Studio to Cobalt after testing in courses.

Finally, Russell and the team surveyed students about their prior use of virtual worlds software and the types of features they might find useful in a Virtual Writing Studio.  While only a small number of students had tried some type of virtual worlds software in the past, they expressed interest in chat with tutors, access to help materials and collaborative spaces.

The team built some sample spaces in Qwak based on a paper prototype developed during the grant term.  The CIT provided additional funding to license Qwak for creation of spaces for the Virtual Writing Center so that it could be tested in Spring 2009.

During the semester, Holly Ryan and Jennifer Welsh used the virtual world in their course.  Students in Welsh’s class used Qwaq to create learning rooms that students can visit to discover a wide variety of topics.  In Ryan’s course, students revised writing studio handouts into materials that would be effective for visual kinesthetic and aural learners.  The materials included videos, virtual rooms, PowerPoint slides and additional handouts that are housed in the Virtual Studio for use by future students.  In March 2009, Russell presented at the College Conference on Composition and Communication about the project.

The Virtual Writing Studio was used for sessions between students and writing tutors during the semester to a limited extent.  “The handful of experiences we had were well received and successful, with both students and tutors feeling the medium offered a different and ultimately richer way of interacting with the text under consideration,” Russell said.  Welsh noted in that one student “was able to take as much time as needed to go over the paper, and really work through it”; She also noted that “It’s also clear that working in a virtual space contributed to how they were able to work through the paper– reading through it on a panel, scrolling up or down, highlighting, etc.”

While the Virtual Writing Studio is still a “work in progress”, Russell reports that it has been well received by students.  They plan to continue to explore the most effective ways that students can be aware of the resource and ways to encourage them to try it out.  The Virtual Writing Studio will be continued in Qwaq in the fall and spring.  Faculty interested in trying the Virtual Writing Studio are encouraged to contact Vicki Russell via email for more information.

Student survey questions (PDF)
Student survey results summary (PDF)
Comparison of virtual worlds software (PDF)

Project start date: 5/13/2008
Funding awarded: $4,000

Project Description

Cary Moskovitz has been using the concept of “think aloud” responses in his writing courses, getting professionals in various fields at Duke to record comments and feedback on student papers. Students gain valuable insight from a reader familiar with the subject area of the paper and gain a better sense of ownership of their writing in a “real world” situation.

To build on the success of this approach in his own courses, Moskovitz is cooperated with the Duke Alumni Association to introduce this method into Writing in the Disciplines courses, by locating Duke alumni residing around the United States to serve as readers in three classes in Economics, History and Chemistry in Spring 2008.

During Fall 2007, Moskovitz used CIT funding to explore the technologies which will be used to implement the full project in Spring 2008. From fifty alumni applicants, twenty were paired with Duke students.  Readers used a webcam-assisted teleconference to virtually “meet” with their student partners, and web-based audio recording software to record a “think aloud” response on a student paper in the course. Students used the readers’ feedback to edit and improve their writing based on this professional input. The CIT consulted on technology approaches, documentation for the volunteer readers and students, and evaluation of the project.

The main challenges encountered during the project were recruitment of alumni readers with expertise that matched the student papers and with technology problems encountered by some users.  Using Wimba for recording of responses was successful, but some users had difficulty configuring and setting up the webcams or Skype account for the project.  Reactions to use of video for the project were mixed, with some users finding it more efficient to talk with students by phone and others reporting that it added a meaningful dimension to their interactions with the students.

• Moskovitz’s full report on the project (PDF)
• Duke Today feature article about the project

Project Started: 8/15/2007
Funding: $5,000 total (Fall 2007, Spring 2008)