Are we stuck with grades (letter or numerical) as the only form of assessment?  What does an A, B, C, D, or F really mean?  How do we communicate to students what constitutes “A” work versus “B” or “D” work?  How do we explain the difference between “good” and “excellent” work without falling into the jargon of assessment research, learning outcomes, and other rubrics that can seem detached and arbitrary?

What if we have a large class and do not have time to write laborious, detailed comments on each paper, every time?  Is there any way to make evaluation clear, formative, helpful–and not too killing on the grader?

These are some of the questions that the distinguished educator and psychologist Barbara Rogoff, Professor of Psychology at the University of California Santa Cruz, researches and thinks about deeply. 

Even better:  she finds ways to translate her research into actual practices. 

She pulls back the veil, so to speak, of how a prof and her Teaching Assistants talk about the grades they assign and shares that dialogue with the students to help them understand better.

Generously, she has given permission to me to share two very useful handouts that she shares with her TA’s and students to explain what a grade means and offer language that can be imported into any assessment and give useful, formative feedback in a relatively easy way.

She has generously given me permission to share both of these with my “Black Listed” open class, the Futures Initiative Fellows, and the HASTAC community.  Borrow anything that is useful to you!

[Apologies for spacing issues in moving this from a pdf to Drupal—CND.]

I.  Evaluation Handout: For Teaching Assistants and Undergraduate Students in  “Human Development as a Cultural Process”

Evaluation Format – Psych 114
To help you understand our criteria for evaluation, here is a draft of the format that we use to
systematize our narrative evaluations. (Individual Evals will be paragraphs tailored to each student’s case.)
Because the evaluation emphasizes improvement, we focus especially on the last half of the course.

Note that the evaluation categories below DO NOT map onto the A-F letter grade scale.

Overall, (Name)’s performance indicated

•impressive
•extremely well-developed
•well-developed
•a good working
•a satisfactory
•uneven
•minimal understanding of the ideas of the course.

[Optional: Over the quarter, we observed impressive progress in ____________________________________
(e.g., understanding the course material, depth of analysis of ideas and evidence, coherence and organization of ideas).]

(Name’s) weekly 1-2 page essays on main ideas of the reading and two 3-4 page lab reports
(involving cultural, historical interview research) were usually
• ideas that would be publishable with further development.
• so productive in extending key ideas that they’re worth presenting to the class to help explore issues and evidence.
• exemplary, connecting and extending ideas and evidence in ways that could serve as models for others.
• mostly on track, reflecting active engagement with the topic, but needing to be pushed further.
• just satisfactory, reflecting some engagement with the topic, but not sufficiently grounded in the main ideas.
• not satisfactory, either without adequate engagement with the topic or not turned in at all.
The assignments were • turned in on time.
• sometimes late or not turned in.
• often late or missing.

(Name) •contributed insightful ideas, handled in-class activities well, and supported other students’
learning and writing in
•listened actively, engaged in the in-class activities, and contributed to the learning dynamics in
•usually attended and participated in the in-class activities in
•was often absent from or not contributing to
the required class and lab meetings.

(Name’s) presentation on cultural aspects of human development
• showed extremely thoughtful engagement with the main ideas.
• showed thoughtful engagement with the main ideas.
• suggested uneven engagement with the main ideas.
• was not satisfactory / was not done.

[Optional: (Name) showed impressive interest in learning, by _____________________.]
 

***
Letter grades at the end of the quarter are based on a comprehensive analysis of the whole picture
provided by the narrative evaluation, especially during the last half of the course.

A passing grade (A to C) requires evidence that the student understands the material well enough to give a
reasonably accurate explanation of the main ideas of the course:
‘A’ work means that across the quarter, the student’s work could often serve as a model for other students.
‘B’ means that the student gave evidence that they basically understood the main ideas, but often without
extending them in thoughtful or deep ways.
‘C’ applies to work that is often just satisfactory.

Refer to this key during the quarter if you get nervous about how you’re doing in terms of letter grades.
Feel free to ask us to go over your record to tell you what it looks like in terms of letter grades.

———————

[Apologies for spacing issues in moving this from a pdf to Drupal—CND.]

II. TA Notes for Psych 114:

Comments & Evaluations of Essays and Labs

Our comments and evaluation focus on the students’ understanding and extension of the main
ideas of the week, their appropriate use of evidence to support their ideas, and the clarity of their
expression. (We don’t comment on spelling/punctuation/grammar sorts of writing problems,
except if they are serious enough to make understanding difficult, and then we mention the
problem in the margin and advise them to get help with it from a tutor. It may be hard at first to
focus on the students’ ideas rather than the technical aspects of their writing; it should get easier.)
Have the book(s) handy to refer to as you read the papers.

In general, the aim of the comments is not to ‘correct’ the papers but to give feedback to
improve the students’ next assignment (rather than trying to “fix” this one). Tell them
especially what they are doing well, so they’ll know what to build on. Former TAs advise that the
more you give helpful feedback early in the quarter, the easier the rest of the quarter is; the
students catch on and then we don’t have so much work to do. If we give them good feedback at
the beginning, they get on track and comments can be simpler.

Please use the margins to make conversational comments as you read, following my model. This
helps students see how they’re communicating, and helps you point to areas that work well or
poorly. For problematic papers, try to point out what works well (or goes in a good direction)
along with identifying the problem sections. If there are many problems, focus on a few of the
most important ones that will help the student the next time (and don’t fill the paper with
comments on the rest of their problems — there’s only so much they can take in at one time).

Please pencil your comments for the first few assignments, until we get well calibrated, so we can
change any if need be. Please consult with me on any papers you can’t figure out what to do with.
Put a summary comment at the end of the paper in your own words, along with the evaluation
sticker I provide (see below).

The summary comment focuses on issues related to the evaluation:

• Ideas that would be publishable with further development.
• So productive in extending key ideas that it’s worth presenting to class to help explore issues and evidence.
• Exemplary, connecting and extending ideas and evidence in ways that could serve as a model for others.
• Mostly on track, reflecting active engagement with the topic, but needing to be pushed further.
• Just satisfactory, reflecting some engagement with the topic, but not sufficiently grounded in the main ideas.
• Not satisfactory, without adequate engagement with the topic.

Please refer to the work (“This paper was…”) rather than the student (i.e., not “You…”) to help
students see that we’re evaluating the paper, not the person. This makes it easier to give
straightforward feedback.

Please use their name to address your summary comments to them
personally and put your name on your comments, so they’ll know the comments come from you.

Be straightforward about where the papers need to improve. State critique constructively and
clearly — this is what will help them see what we’re looking for. (Being ‘soft’ on them doesn’t
do them a favor. But you don’t need to be tough either. More like a helpful coach.)

Note the summary evaluation on your record grid for each student, following our scale.

Please give me a count of the number of each evaluation category, each time (combining your two
sections).

I’ll be reading papers along with you all quarter, to help you with the kind of comments we want
for this class, and to help us calibrate among ourselves. Please also look at each others’
comments and consult with each other, especially in the early weeks of the class.

Evaluate the essays holistically: If they are very thoughtful in one area, but skimp on another, it
can often be OK. The main idea is to see if they seem to be thinking about what they’re reading,
and have caught on to the prior weeks’ ideas.

 

The summary evaluation categories:

Most Essays and Labs usually get a ‘mostly on track’ evaluation; they need to stand out one way
or another for the other evaluation categories:

• ideas that would be publishable with further development.
• so productive in extending key ideas that they’re worth presenting to the class to help explore issues and evidence.
• exemplary, connecting and extending ideas and evidence in ways that could serve as a model for others.  Some part of the essay goes beyond the reading in a very interesting way, and the rest is fine. The paper doesn’t have to be beautifully written to be ‘exemplary’ (or higher categories) — it’s the ideas we focus on. If the ideas are EXM but the exposition is so confusing you couldn’t actually show it to another student as a model, you can write, “If you took care of ___, the paper would be” and check the EXM box and record EXM.
• almost exemplary, can’t quite be used as a model for others, but it’s close and has no big conceptual problem. This category is to be avoided if you can choose EXM or OT, just use it when it’s really needed, by writing almost in front of the EXM box.
• mostly on track, reflecting active engagement with the topic, but needing to be pushed further.  They did the essay fine, engaging with some ideas in the reading. They might not understand the ideas entirely, but they’re thinking things through even if they show confusion. The main criterion is that the student seems to be thinking about a main idea or two in the readings. They don’t have to have this week’s deas exactly ‘right’ but they give evidence that they’re exploring a central idea, and they are on top of the ideas of prior weeks. If they have a problem in understanding the current week’s reading, you can point out to them that they should look at the reading again, but that you’re glad they’re pondering the ideas. (Take into account where we are in the quarter. If they don’t fully ‘get’ ethnocentrism for Essay #1, that’s OK, but if they don’t get it in the next one, then it’s not OK.)  Often this category can be detected if you see strengths in the essay but you also see a big BUT. So “It’s good….but…..”   If you use this category, be sure to write what the essay needs – e.g., more depth to get beyond the chapter.
• just satisfactory, reflecting some engagement with the topic, but not sufficiently grounded in the main ideas. Something’s really missing or wrong, without strong compensating excellence. The work seems shoddy and superficial, as if they haven’t been trying to understand the material.
• not satisfactory, without adequate engagement with the topic. Be alert to BS papers that could be done without opening the books. If they could have written what they did without reading the main chapter, it is probably NS. If the essay addresses the wrong chapter, it’s NS.  In your record grid, if it is not turned in, note both NS and missing.
 

***
The evaluation categories are not absolute, but rather are relative to what we expect that they
should have learned at that particular point in the course. For example, the ideas they write
during the first of the course might get a ‘mostly on track’ evaluation, but the same ideas would
not show enough learning if they wrote them later in the course.

The writing can be exploratory regarding main ideas in the reading, rather than being absolutely
correct, if they show good general understanding and sincere, thoughtful exploration.

On occasion, an essay won’t fit any of these categories, or will be between two. Then you can
note on the paper that it falls between two categories (and note the same on your record grid).
But try to fit the evaluation into one category so mixed categories are very rare.
If papers are late unexcused, write LATE on the top of the first page. (Do this when they hand
them to you late, and indicate how late – e.g., 1 day late, 2 days late).

If they are late during class, write LATE IN CLASS and add, “Please come to class on time, it is helpful
if we can get going without disruptions.”

You can accept one unexcused slightly late paper per student; after that, consult with me and
we may not accept the papers. (Consider the rest of this student’s way of handling the class.)

Use a stiffer standard to evaluate any papers that are later than the class they are due in — the
revision should show the learning that should occur by the current point in the course.

If we ask them to write comments on classmates’ papers and a paper has none, write at the top:
“Where are your classmate’s comments? This is an important part of this assignment.”

If students ask you to give them feedback in advance, you can, but only if they sit with you as
you go over it. (So – no editing over email.) It’s more likely they’ll get helpful feedback this
way, and less likely that you’ll be put in the position of co-author….

[NB:  Other notes for TAs go on to give specific guidance for the specific assignments]

 

III.  PROFESSIONAL BIOGRAPHY

Dr. Barbara Rogoff is Professor of Psychology at UCSC.  She investigates cultural variations in learning process and settings, with a special interest in communities where formal schooling has not be prevalent.  She analyzes “collaboration, learning through observation, children’s interest and keen attention to ongoing events, roles of adults as guides or as instructors, and children’s opportunities to participate in cultural activities or in age-specific child-focused settings.”

Also see www.intentcommunityparticipation.com