Grading vocabulary: How correct is necessary?

Question

When grading vocabulary (either in a traditional classroom/test environment, or spaced repetition), is there any research into how to most effectively grade responses?

Examples of possible answers (not to be taken as an exhaustive list!), using the word "apple":

• Answers should be counted correct only if they are 100% correct.

  Spelling "aple", pronunciation /ɑpəl/, vague description "a type of fruit" are counted incorrect

• Answers should be counted correct if they get the substantial idea across, are intelligible, would be understood by a typical speaker, etc.

  Spelling "aple", pronunciation /ɑpəl/, vague description "a type of fruit" are counted correct

• Answers earn partial credit (meaningful on tests, perhaps not so much for SRS) if they are partially correct.

  Spelling "aple", pronunciation /ɑpəl/, vague description "a type of fruit" are counted for partial credit.

Which of these, or some other grading standard, leads to best results in students?

On the one hand, aiming for perfection seems ideal, but does it lead to student fatigue and demotivation with concepts that are hard to master?

What research has been done into this topic, in particular as it relates to language teaching?

Answer 1

How vocabulary tests are graded depends on the teacher's (implicit or explicit) language learning or teaching theory. There are many such theories (see e.g. What are the main foreign language teaching methods?) with different views on how vocabulary should be taught (and tested).

In addition, there can be individual differences between teachers that subscribe to the same language learning or teaching theory. (Even in the context of standardised language tests, evaluators should regularly attend seminars in order to make sure that they all use the same evaluation criteria. See Bausch, Christ & Krumm (eds.): Handbuch Fremdsprachenunterricht, 5th ed. A. Francke, 2007, page 376.)

Language teaching theories that prefer a direct method teach vocabulary in context, while theories that allow the use of L1 let learners study associated pairs (L2 word - L1 translation). These approaches also determine how vocabulary is tested, e.g. through translations or by other means (cloze tests, building sentences with a given word, ...).

In both cases, the evaluator or teacher needs to weight the errors. For example, Englische Fachdidaktik. Theorien, Praxis, Forschendes Lernen by Wolfgang Gehring (Erich Schimdt Verlag, 1999/2010) lists the following types of minor versus major errors:

Major errors:

• Errors against general language rules.
• Errors that seriously hamper understandability.
• Errors against the current language learning goal (i.e. something that was taught very recently).
• Errors against "common construction".
• Strong deviations from language rules and stylistic norms.

Minor errors:

• Errors against specific language rules.
• Errors that do not seriously hamper understandability.
• Minor deviations from "common construction".
• Minor deviations from language rules and stylistic norms.

The above error types apply to language tests generally (i.e. essay composition, oral skills tests, ...) and do not apply well to vocabulary that require translations (L1 to L2, or L2 to L1). In the current communicative approaches, simple word translation tests are often replaced by other types of tests, e.g. given a specific word, build a sentence that shows that you know the word's meaning. Errors in such sentences would need to be weighted according to the criteria listed above.

However, there are also vocabulary learning techniques where the only criterion is correctness. One example is SAFMEDS (Say All Fast Minute Each Day Shuffle). SAFMEDS is a technique that is part of precision teaching, a method of teaching and evaluating developed by Ogden Lindsley in the 1960s (and inspired by behaviourism). In SAFMEDS, you get through a deck of flashcards using the following method:

• You say the word on the other side of the card.
• You try to get through the deck within a minute.
• You put aside the flashcards where you made errors. (The number of errors is tracked in a "celeration chart".)
• You shuffle the deck after each run through the deck, so you don't memorise the sequence of cards.
• You do this every day.

(To get an idea of what SAFMEDS is, see e.g. SAFMEDS Tutorial by Kelly Byrne on YouTube, Vocabulary Aquisition with SAFMEDS: How to learn words quickly by Dr. C. A. Young-Pelton, and Surviving on SAFMEDS by Danielle Costa.)

Another teaching method where frequent formative tests play an important role is mastery learning, an instruction approach first formulated by Benjamin Bloom in the late 1960s. In mastery learning, all learners need to achieve a specific level of mastery before moving on to the next "lesson". Mastery learning has also been used in language teaching. However, I have not yet found literature that discusses grading of errors. For example, "Mastery Learning in Modern Languages - a Case Study" (Parkinson, B. L.; Mitchell, R. F.; Johnstone, R. M., 1983) discusses the use of mastery learning for teaching French in two schools in Scotland. The article does not mention the weighting of errors.

So "how correct is necessary" is highly dependent on the learning theory. There are vocabulary learning methods that are more effective than others, but some of the existing methods have probably not been compared yet (e.g. SAFMEDS versus the SuperMemo algorithm).

Answer 2

I'm not citing any studies here, so it's a partial answer, but I'm trying to provide arguments for what I say, so I hope it can be helpful.

Note that grading vocabulary has different purposes in different settings you mentioned. In case of SRS, if you mark something as incorrect, you'll review that soon. In a classroom, when the teacher corrects a student, it is only meant as a feedback, and it doesn't mean that the item that got corrected will be reviewed soon. Finally, tests are tools for estimating the student's level, not for learning.

Let's start with SRS. I don't think "100% correctness" is a reasonable goal in most cases. In case of pronunciation, it would mean that you consider the answer good until you can reliably pass in your answer as a native speaker of the language. That's obviously very hard, and most learners don't even care about passing as natives, so aiming for 100% correctness is just a waste of time. And even if you do want to pass as a native, I think it's better to distinguish pronunciation learning from vocabulary learning. Since the question is about the former, I'd say that in this context you only should care whether your pronunciation is understandable. You don't want to endlessly repeat words that you can use very well just because your pronunciation is slightly off.

As for the meaning, again, 100% correctness is not possible in most cases. You gave an example of a concrete noun, but how about more abstract ones, or verbs? Most of them won't have exact equivalents in other languages you know, so there is simply so such thing as perfect correctness when you learn such words out of context. You can only decide if you have a general idea about their meaning.

Spelling is probably the easiest area where you could aim at 100% correctness. But then again, it depends on your goals and abilities, how transparent the writing system is, whether you want to get the writing abilities of an educated native speaker. For example, if you have a dyslexia and have spelling problems in other languages you know, you may want to relax the correctness criteria for the language you're learning. Unless, of course, you're specifically aiming at getting rid of spelling problems.

In case of tests, you simply need to make an arbitrary decision of what is the level of correctness, based on what you want to measure: communicative ability of the student, his or her ability to pass as a native speaker, or whatever else.

Let me finally answer the question in the title: "how correct is necessary?". Children and people learning by immersion get little explicit feedback, and if they do, it doesn't cause automatic repetitions like in an SRS. Still, they manage to learn the language, so that shows that the emphasis on correctness is not necessary. However, it doesn't mean that it can't make learning more effective. But, due to the reasons I discussed above, aiming for 100% correctness is either impossible, or may slow down the learning.