Assessing Writing in the first Year: How do we Measure Success?
By Karen Nulton, Director of Writing Assessment (with Irvin Peckham, Director of the First-Year Writing Program)
Assessment is context-specific: our goals determine how we measure success. Over the past three years Dr. Rebecca Ingalls, the Director of the First-Year Writing Program (FWP: the three writing courses required of students across the university), and I (as Director of Writing Assessment) worked together to create and measure goals for the FWP: we documented rhetorical growth from the first to the last day of the sequence, measured how well students could create an evidence-based argument at the end of the sequence, and evaluated the needs and skills of international students. Our shared goals produced rich conversations and robust data that were useful inside and outside of the program.
This past July the university was fortunate to hire Dr. Irvin Peckham from LSU as the new Director of the First-Year Writing Program to replace Dr. Ingalls, who resigned in June. During our first meeting, Irvin said something that struck me: “If students are writing texts that we don’t want to read,” he said, “We’re probably asking them to write the wrong things.” Authentic writing, Irvin argues, is primarily a tool for exploring, discovering, and communicating. When students are part of a genuine conversation (defined as a speaker with something to say and a reader/listener willing to listen) their motivation to write—and thus what they write—improves, as does our experience as readers. Conversely, when students are uninterested in the writing task and unfamiliar with genre requirements, their errors in grammar, syntax, and coherence increase (Bean 2009). This simple observation has meant a subtle but important internal shift in the writing program. First-year writing faculty have been encouraged to use the writing sequence to create writing opportunities that are genuine conversations (between students; between student and teacher) to help students to get around writing blocks and encourage ease of writing. We work in the program to extend these skills in writing to academic writing situations in other.
As Irvin has worked with his advisory committee to redefine the goals of the program, our assessments have shifted to match the different objectives. We started by asking the incoming class of 2014 what their writing experiences were prior to college and followed this with a survey about their current writing practices. We asked a final open-ended question about what we could do in the first-year writing courses to help students engage with writing; of the 438 first-term students who responded,
281 students (65%) asked to be able to write on topics tied to their interests and lives. They asked for more freedom of choice in writing topics and clearly said that if they are interested in the topic they write.
58 students (13%) asked for clarity in writing topics and in the class. Many students tied this request back to a version of being treated with respect (asking that professors don’t assign additional work at the last minute and but respect their time management by clearly detailing assignments and due dates at the beginning of the course).
50 (11%) students keyed in on emotion and the value of non-graded writing, asking that professors provide supportive guidance rather than just criticism of what was wrong in a paper. Some students also talked of their anxiety around writing and how they need support from professors rather than just criticism. In general, these comments asked for professors to pay attention to the person behind the grade/paper/comment.
Essentially, students said they wrote best with clear directions on topics of interest to them within a collaborative, supportive process. This qualitative data informs slightly different pedagogical goals than the quantitative data we had collected in previous years. Rather than quantitatively evaluate student writing as performances, we looked at what they wrote about their writing experiences both to evaluate their fluency and to discover from their analyses what kinds of teaching practices would help them develop their writing abilities. We shared our results with students and faculty as the feedback loop necessary for effective assessment.
We are designing new assessment and feedback strategies that will encourage FWP teachers to be a part of an ongoing self-assessment, re-seeing their own practices through the prism of what students say about their previous writing experiences in the classroom. We imagine one effective kind of assessment as one in which the administration, the teachers, and the students collaboratively investigate better ways to teach, learn, and write.
We are not looking for a magic bullet. We know that there is no one formula for good writing assessment any more than there is for good writing. Assessment, however, is how we research a topic of interest. It’s a place to pause, reflect, gather evidence and move ahead shaped by any knowledge acquired.