Quick Start
INTRODUCTION AND INSTRUCTIONS
“Listen up! We have a new way to do multiple-choice today.
You have a choice of marking the best answer for each question, as you always have,
or using the questions to report what you actually know and can trust.
1. Read the question and see if you can use it to report what you know or can do.
2. If yes, then compare the answer you have in mind with the printed answers.
3. If you find a match, you are probably right. Mark it.
4. If there is no match, you may want to omit, to avoid making a wrong mark.
5. You get one point for a right mark and one point for not making a wrong mark.”
“Does that mean the test is worth 50% when we start?”
“Yes, for perfect judgment in having made no wrong marks.”
“How do I know what I know?
“By practicing reporting what you know and by fully learning part or all of the lesson or
assignment rather than memorizing unrelated bits.”
“Can I take the test both ways?”
“On this test, there will be time to do that.”
“Why are we doing this?”
“By measuring and rewarding your self-judgment, you will become a better, more
successful student. You also do not need to spend time figuring out a “best answer”
to a question you can not read or understand or you have no answer of your own.”
SCORING
The instructions give knowledge and judgment equal value in the dynamic testing
environment. Students at all levels of thinking can understand a 1:1 ratio for knowledge
to good judgment (omit to avoid making a wrong mark).
Scoring, after the test is finished, when no changes can be made, is a static environment.
Now a right mark also counts as not making a wrong mark. Scoring can use a ratio of
0:1:2 for wrong, omit, and right.
GRADING
Grades must reflect some measure of accomplishment. In a positive environment with
high expectations and high performance, the traditional ten point grade spread can be
used: 90, 80, 70, 60 for A, B, C, and D. There will be few if any Ds.
Schools in California have dropped the letter grade D as students passing at this level
were not prepared for future coursework. Standardized tests tend also to require 70-
75% for passing.
This leaves two options, a ten point grade spread and no D or a 7.5 point grade
spread with a D, when a significant number of students earn scores less than 70%. Or, if
permitted, rescale the raw scores from “good questions” to a traditional ten point grade
spread distribution.
All of the above is dependent upon the difficulty and validity of the test questions.
ANALYSIS
No analysis replaces the judgment of a good teacher. Analysis helps make your
judgment better when grading, counseling students, and during course revision.
Knowledge and Judgment Scoring produces a student view of a test not available from
right mark scoring. Questions are placed into four groups:
The teacher view of the test, from the item analysis, places questions into three groups:
Test reliability is derived from the above analysis for the test and for a test with 50 and
100 questions for unfinished and discriminating questions. Only discriminating
questions (without known age, sex, race, etc. biases) are used on standardized tests.
A value of 0.7 is a good classroom test and over 0.9 for a standardized test.
A student counseling matrix, based on each of these analyses, relates student,
question, class, and test performance. A good learning experience is discussing
problem questions with your class.
Then, keep, drop, or give each student one point for each bad question. Rescore the
test for an accurate report on which to set grades. Assign grades on the “good test”
within the full test.