Rubric Development

(Based on Introduction to Rubrics: An Assessment Tool to Save Grading Time, Convey Effective Feedback, and Promote Student Learning by Stevens and Levi 2005; Assessing Academic Proarams in Higher Education by Allen 2004; and Learner-Centered Assessment on College Campuses: shifting the focus from teaching to learning by Huba and Freed 2000)

Format for a Rubric

  • Task Description
  • Scale
  • Attributes
  • Descriptions of Attributes

Components of a Rubric

A rubric involves four components:

Part 1: Task Description

  • Involves a "performance" of some sort by the student
  • The task can take the form of a specific assignment; e.g., a paper, a poster, a presentation
  • The task can take the form of overall behavior; e.g., participation, use of proper lab protocols, behavioral expectations in the classroom

Part 2: Scale

  • Describes how well or poorly any given task has been performed
  • Positive terms which may be used: "Mastery", "Partial Mastery", "Progressing", "Emerging"
  • Nonjudgmental or noncompetitive language: "High level", "Middle level", "Beginning level"
  • Commonly used labels:
    • Sophisticated, competent, partly competent, not yet competent
    • Exemplary, proficient, marginal, unacceptable
    • Advanced, intermediate high, intermediate, novice
    • Distinguished, proficient, intermediate, novice
    • Accomplished, average, developing. Beginning
  • 3-5 levels are typically used
    • the more levels there are, the more difficult it becomes to differentiate between them and to articulate precisely why one student's work falls into the scale level it does
    • but, more specific levels make the task clearer for the student and they reduce the professor's time needed to furnish detailed grading notes

Part 3: Dimensions

  • Lay out the parts of the task simply and completely
  • Should actually represent the type of component skills students must combine in a successful scholarly work
  • Breaking up the assignment into its distinct dimensions leads to a kind of task analysis with the components of the task clearly identified

Example:

Task: Each student will make a 5-minute presentation on the changes in one community over the past 30 years. The student may focus the presentation in any way he or she wishes, but there needs to be a thesis of some sort, not just a chronological exposition. The presentation should include appropriate photographs, maps, graphs, and other visual aids for the audience.

Excellent

Competent

Needs work

Knowledge/understanding
20%

Thinking/inquiry
30%

Communication
20%

Use of visual aids
20%

Presentation skills
10%

Part 4: Description of the Dimensions

  • A rubric should contain at the very least a description or the highest level of performance in that dimension
  • Scoring Guide Rubric = a rubric that contains only the description of the highest level of performance

Example Scoring Guide Rubric: (includes description of dimensions at the highest level of performance)

Task: Each student will make a 5-minute presentation on the changes in one community over the past 30 years. The student may focus the presentation in any way he or she wishes, but there needs to be a thesis of some sort, not just a chronological exposition. The presentation should include appropriate photographs, maps, graphs, and other visual aids for the audience.

Criteria

Comments

Points

Knowledge/understanding
20%

The presentation demonstrates a depth of historical understanding by using relevant and accurate detail. Research is thorough and goes beyond what was presented
in class or in the assigned texts.

Thinking/inquiry
30%

The presentation is centered
around a thesis, which shows a highly developed awareness of historiographic or social issues and a high level of conceptual ability.

Communication
20%

The presentation is imaginative
and effective in conveying ideas to
the audience. The presenter responds effectively to audience reactions and questions.

Use of visual aids
20%

The presentation includes
appropriate and easily understood visual aids,which the presenter refers to and explains at appropriate moments in the presentation.

Presentation skilis
10%

The presenter speaks clearly and loudly enough to be heard, using eye contact, a lively tone,gestures, and body language to engage the audience.

Example Three-level Rubric: (includes description of dimensions with all levels of performance described)

Task: Each student will make a 5-minute presentation on the changes in one community over the past 30 years. The student may focus the presentation in any way he or she wishes, but there needs to be a thesis of some sort, not just a chronological exposition. The presentation should include appropriate photographs, maps, graphs, and other visual aids for the audience.

Excellent

Competent

Needs work

Knowledge/understanding
20%

The presentation demonstrates a depth of historical understanding by using relevant and
accurate detail.
Research is thorough and goes beyond what was presented in class or in the assigned texts.

The presentation uses knowledge that is generally accurate with only minor inaccuracies and that is generally relevant to the
student's thesis.
Research is adequate but does not go much beyond what was presented in class or in the assigned text.

The presentation uses little relevant or accurate information,
not even that which was presented In class or In the assigned texts.
Little or no research is
apparent.

Thinking/inquiry
30%

The presentation is centered around a thesis, which shows a highly developed awareness of historiographic or sodaI issues and a high level of conceptual abilitv.

The presentation shows an analytical structure and a central thesis,but the analysis is not always fully developed or linked to the thesis.

The presentation shows no analytical structure and no central thesis.

Communication
20%

The presentation is
imaginative and effective in conveying ideas to the audience.
The presenter responds effectively to audience reactions and questions.

Presentation techniques
used are effective in conveying main ideas,but they are a bit unimaginative.
Some questions  from the audience remain unanswered.

The presentation fails to
capture the interest of the audience and/or is confusing in what is to be communicated.

Use of visualaids
20%

The presentation includes appropriate and easily
understood visual aids which the presenter refers to and explains at appropriate moments in
the presentation.

The presentation includes appropriate visual aids, but these are too few,are in a format that makes the difficult to use or understand, or the presenter does not refer to or explain them in the presentation.

The presentation includes no visual aids or includes visual aids
that are inappropriate or too small or messy to be understood.
The presenter makes no mention of them in the
presentation.

Presentation skills
10%

The presenter speaks
dearly and loudly enough to be heard,using eye contact, a lively tone, gestures,and body language to engage the audience.

The presenter speaks
dearly and loudly enough to be heard but tends to drone or fails to use eye contact, gestures,and
body language consistently
or effectively at times.

The presenter cannot be heard or speaks so unclearly that she or he cannot be understood. There is no attempt to engage the audience through eye contact, gestures,or body lanquaqe.

Four Stages in Constructing a Rubric

Reflecting

In this stage, we take the time to reflect on what we want from the students, why we created this assignment, what happened the last time we gave it, and what our expectations are.

  1. Why did you create this assignment?
  2. Have you given this assignment or a similar assignment before?
  3. How does this assignment relate to the rest of what you are teaching?
  4. What skills will students need to have or develop to successfully complete this assignment?
  5. What exactly is the task assigned?
  6. What evidence can students provide in this assignment that would show they have accomplished what you hoped they would accomplish when you created the assignment?
  7. What are the highest expectations you have for student performance on this assignment overall?
  8. What is the worst fulfillment of the assignment you can imagine short of simply not turning it in at all?

Listing

In this stage, we focus on the particular details of the assignment and what specific learning objectives we hope to see in the completed assignment.

  • Answers to (d)-(e)-(f) above regarding skills required, the exact nature of the task, and the types of evidence of learning are most often a good starting point to generate this list. Once the learning goals have been listed, you add a description of the highest level of performance you expect for each learning goal. These will later contribute to the "Descriptions of Dimensions" on a finished rubric.

Grouping and Labeling

In this stage, we organize the results of our reflections in Stages 1 and 2, grouping similar expectations together in what will probably become the rubric dimensions. Start with the highest performance expectations completed in Stage 2 and group together items which are related. Once the performance descriptions are in groups of similar skills, read them and start to find out what is common across the group and label it. These labels will ultimately become dimensions on the rubric - it is important to keep them clear and neutral; e.g., "Organization", "Analysis", or "Citations".

Application

In this stage, we apply the dimensions and descriptions from Stage 3 to the final form of the rubric, utilizing the matrix/grid format.

Once you have identified what you are assessing; e.g., critical thinking, here are steps for creating holistic rubrics:

  • Identify the characteristics of what you are assessing; e.g., appropriate use of evidence, recognition of logical fallacies
  • Describe the best work you could expect using these characteristics- this describes the top category
  • Describe the worst acceptable product using these characteristics - this describes the lowest acceptable category
  • Describe an unacceptable product- this describes the lowest category
  • Develop descriptions of intermediate-level products and assign them to intermediate categories. You might decide to develop a scale with five levels; e.g., unacceptable, marginal, acceptable, competent, outstanding, or three levels; e.g., novice, competent, exemplary, or any other set that is meaningful.
  • Ask colleagues who were not involved in the rubric's development to apply it to some products or behaviors and revise as needed to eliminate ambiguities.

Example

HOLISTIC rubric for assessing student essays
(Assessing Academic Programs in Higher Education by Allen 2004)

Inadequate

The essay has at least one serious weakness. It may be unfocused,underdeveloped,or rambling. Problems with the use of language seriously interfere with the reader's ability to understand what is being communicated.

Developing
competence

The essay may be somewhat unfocused,underdeveloped,or rambling,but it does have some coherence.  Problems with the use of language occasionally interfere with the reader's ability to understand what is being communicated.

Acceptable

The essay is generally focused and contains some development of ideas,but the discussion may be simplistic or repetitive. The language lacks syntactic complexity and may contain occasional grammatical errors but the reader is able to understand what is being communicated.

Sophisticated

The essay is focused and dearly organized,and it shows depth of development. The language is precise and shows syntactic variety, and ideas are dearly communicated to the reader.

Example

Developing Useful Rubrics: Questions to Ask and Actions to Implement
(Learner-Centered Assessment on College Campuses: shifting the focus from teaching to learning by Huba and Freed 2000)

Question

Action

What criteria or essential elements must be present in the stude.nt's work to ensure that it is high in quality?

These should be the criteria that distinguish good work from poor work

Include these as rows in your rubric

How many levels of achievement do I wish to illustrate for students?

  • The levels should generally describe a range of
    achievement varying from excellent to unacceptable
    • Example: exemplary, profident,marginal, unacceptable
    • Example: sophisticated,competent, partly competent,not yet competent
    • Example: distinguished,profident, intermediate, novice
      Example: accomplished, average, developing, beginning

Indude  these as columns in your rubric and label them

For each criterion or essential element of quality, what is a clear description of performance at each achievement level?

  • Avoid undefined terms (e.g.,significant", viar, shows considerable thought")
  • Avoid value-laden terms (e.g.,excellent", poor")
  • Use objective descriptions that help provide guidance
    to the students for getting better when needed

Indude descriptions in the appropriate cells of the
rubric

What are the consequences of performing at each level of quality?

Add descriptions of consequences to the commentaries in the rubric

What rating scheme will I use in the rubric?

Add this to the rubric in a way that fits in with your

When I use the rubric, what aspects work well and what aspects need.improvement?

  • Does the rubric help you distinguish among the levels
    of quality in a student sample?
  • Do the criteria seem to be appropriate?
  • Are there too many or too few levels of achievement specified?
  • Are there any descriptions that are incomplete or
    unclear?

Revise the rubric accordingly

Additional questions/actions when developing rubrics for specific assignments

What content must students master in order to complete the task well?

Develop criteria that reflect knowledge and/or use of content and add them to the rubric

Are there any important aspects of the task that are spedfic to the context in which the assessment is set?

Identify skills and abilities that are necessary in this context and add related criteria to the rubric

In the task,is the processof achieving the outcome as important as the outcome itself?

Indude and describe criteria that reflect important aspects of the process

Additional Resources

Association of American Colleges and Universities VALUE Project

The major achievement of the Valid Assessment of Learning in Undergraduate Education Project, completed in fall of 2009, was preparation of institutional-level rubrics for fifteen of the AAC&U Essential Learning Outcomes (e.g., oral communication, critical thinking). All of the VALUE rubrics are available for download in PDF format at the VALUE web site; those most closely related to the IUPUI PULs are linked directly in the Assessing IUPUI PULs section of this website. In addition, the Winter 2009 issue of the AAC&U Peer Review journal focuses entirely on the VALUE  project. Though the entire journal must be purchased (IUPUI is a member, so reduced costs apply), several of the articles are available online for general readership.  In  addition,  the  Project  published  in  January  2010  Assessing  Outcomes  and Improving Achievement: Tips and Tools for Using Rubrics (Terrel L. Rhodes, ed.), which can be ordered online at the reduced member rates.

View Resource

IUPUI University Library

Information about information literacy in general as well as IUPUI standards for competency in each year of study.

View Resource

University of West Florida, Center for University Teaching, Learning, and Assessment

Links to several web sites about rubric development.

View Resource

University of Hawaii at Manoa, Assessment Center How-to Creating and Using Rubrics

Useful tips and how-to's for deciding whether and how to use rubrics, how to develop them, how to use them effectively in classes, and how to orient and calibrate group ratings for reliability.

View Resource

St. John’s University, Online Resources for Higher Education Assessment

Extended list of links to various organizations in the assessment and portfolios fields, plus glossaries, explanations, and papers.

View Resource

North Central College (Illinois), Authentic Assessment Toolbox

Maintained by Professor Jon Mueller, the Toolbox section on Rubrics includes a helpful overview of the kinds and uses of rubrics as well as advice on creating them.

View Resource

Community College of Philadelphia, Viewpoints: a journal of developmental and collegiate teaching, learning and assessment, “Building a Better Mousetrap: The Rubric Debate,” Madeline Marcotte

Journal article provides an extended introduction--both philosophical and practical--to rubrics in higher education.

View Resource

California State University Fresno, Institutional Research, Assessment and Planning, “Using Scoring Rubrics”

Provides a helpful summary of what rubrics are and how to develop them, along with suggestions for using them for both grading and program assessment.

View Resource

“Creating a Rubric for a Given Task”

Though situated on a web site about WebQuests, this information can easily be generalized to other kinds of assignments at different educational levels.

View Resource