Quick Start for Knowledge and Judgment Scoring

Home Page Index

Quick Start

INTRODUCTION AND INSTRUCTIONS

“Listen up! We have a new way to do multiple-choice today.

You have a choice of marking the best answer for each question, as you always have,

or using the questions to report what you actually know and can trust.

1. Read the question and see if you can use it to report what you know or can do.

2. If yes, then compare the answer you have in mind with the printed answers.

3. If you find a match, you are probably right. Mark it.

4. If there is no match, you may want to omit (don't know).

5. You get one point for a right mark and one point for knowing what you can trust.”

“If I mark all, will I get the same score as before?”

“Yes, as you are marking and letting me be responsible for scoring the test instead of

you taking the responsibility to report what you actually know and understand.

“How do I know what I know?

“By practicing reporting what you know and by fully learning, understanding, part or

all of the lesson or assignment rather than memorizing unrelated bits.”

“Can I take the test both ways?”

“On this test, there will be time to do that.”

“Why are we doing this?”

“By developing your self-judgment, to know what you can trust, you will become a better,

more successful student. You will not need to spend time figuring out a “best answer”

to a question you can not read or understand, or you have no answer of your own.”

SCORING

The instructions give knowledge and judgment equal value in the dynamic testing

environment. Students at all levels can understand a 1:1 ratio for knowledge to

good judgment (to know what they can trust as the basis for further learning).

Scoring, after the test is finished, when no changes can be made, is a static

environment. Now a right mark also counts as not making a wrong mark. Scoring

can use a ratio of 0:1:2 for wrong, omit, and right.

(Essays, reports, and projects can also be scored in this fashion, to promote student

development, using information bits and rubrics instead of multiple-choice marks.)

GRADING

Grades must reflect some measure of accomplishment. In a positive environment with

high expectations and high performance, the traditional ten point grade spread can be

used: 90, 80, 70, 60 for A, B, C, and D. There will be few if any Ds.

Schools in California have dropped the letter grade D as students passing at this level

were not prepared for future coursework. Standardized tests tend also to require 70-

75% for passing.

This leaves two options, a ten point grade spread and no D or a 7.5 point grade

spread with a D, when a significant number of students earn scores less than 70%. Or, if

permitted, rescale the raw scores from “good questions” to a traditional ten point grade

spread distribution.

All of the above is dependent upon the difficulty and validity of the test questions.

ANALYSIS

No analysis replaces the judgment of a good teacher. Analysis helps make your

judgment better when grading, counseling students, and during course revision.

Knowledge and Judgment Scoring produces a student view of a test not available from

right mark scoring. Questions are placed into four groups:

Expected                     Most students mark and are mostly right.

Misconception             Most students mark and are mostly wrong.

Discriminating             Few students mark and are mostly right.

Guessing                     Few students mark and are mostly wrong.

The teacher view of the test, from the item analysis, places questions into three groups:

Mastery/Easy               Ninety percent or more mark right.

Unfinished                   Neither mastery nor biased (discriminating).

Discriminating             Consistent portions of the class mark right and wrong.

Test reliability is derived from the above analysis for the test and for a test with 50 and

100 questions for unfinished and discriminating questions. Only discriminating

questions (without known age, sex, race, etc. biases) are used on standardized tests.

A value of 0.7 is a good classroom test and over 0.9 for a standardized test.

A student counseling matrix, based on each of these analyses, relates student,

question, class, and test performance. An effective learning experience is discussing

with your class the topic or skill related to each problem question.

Then, keep, drop, or give each student one point for each bad question (on the PUP

Edit sheet). Then click Score again for an accurate report on which to set grades.

Assign grades on the “good test” within the full test.

AN EFFECTIVE TEACHER

An effective teacher will observe a number of things happening over time:

1. More students shift from guessing to reporting.

2. Quality scores rise.

3. A more positive classroom climate develops.

4. Study habits change from rote memory to understanding.

5. Students enjoy meaningful masterly learning (question, answers, verify).

6. Students help other students to break out of old study habits.

7. Empowered students learn for their own satisfaction rather than just for a test.

8. Students make fewer frivolous test question challenges.