Criteria for Constructing a Question Paper

Schools evaluate students to find out how much they have learnt. Evaluation and examination are not the same thing. Evaluation is the wider process, and the examination is its main tool. The examination comes first: students sit it, and the teacher then uses the results to evaluate their learning. In the same way, the main tool of a written examination is the question paper.

Examinations come in different forms. A written examination is given through a question paper, and students answer in writing. In an oral examination the teacher asks and the student answers aloud. In a practical examination the student shows the skill by doing a task. Oral and practical examinations are different in nature from written ones. How good a written examination is depends largely on how good its question paper is.

Students are evaluated through the examinations that question papers make possible, so a question paper has to be right. If the paper itself is faulty, the results it produces are faulty too, and the evaluation of the student cannot be correct. Imagine a question that uses a word with two meanings. A student who understood the lesson well may read it the other way and write a different answer. The mark that student gets no longer shows what they actually learnt.

In one academic session a school holds many examinations, such as unit tests, terminal examinations and the final examination. For each kind of examination the questions have to be prepared according to a plan. The questions must be able to measure the objectives written in the curriculum, so they need to be objective-based and achievement-based. Teachers may hold different examinations during the year to reach the objectives the curriculum sets. When such an examination is given, with its time fixed according to the weight of marks, it should have these qualities:

• Valid: it measures exactly what it is meant to measure.
• Reliable: it gives the same result whoever marks it and whenever it is marked.
• Objective: the examiner's personal likes and feelings do not affect it.
• Practical: it can be carried out with the time, resources and conditions available.
• Purposeful: it is tied to the aims of the curriculum.

To keep examinations at a good standard, questions can be written well in advance and kept in a question bank. Questions stored this way can be taken out and used at once whenever they are needed. Suppose a teacher, after finishing each lesson, writes some easy, some medium and some difficult questions from it and files them. When the examination arrives, the paper is put together from questions that were thought through carefully, not written in a hurry the night before.

A question paper is an essential tool for conducting an examination. Without one there is no written examination at all, which is why it matters so much. Because its questions are written to suit the student's age, level and standard, it helps the teacher evaluate the student correctly. An examination also brings out a student's weaknesses. Once these are known, the teacher can give feedback that helps remove them. A good question paper therefore does more than hand out marks. It shows the way to improve learning.

A question paper cannot be put together carelessly. Many things have to be considered while making one, and there are definite criteria for doing it well. The questions must stay on the set subject and must be valid and reliable. It works best to collect questions that cover the content in a way that meets the objectives the curriculum sets. Then, before the examination is even held, the teacher can look over the paper and make sure of three things:

• Level of difficulty: how easy or hard the questions are.
• Discrimination: whether the questions can tell apart students who know more from those who know less.
• Validity: whether the questions really measure what they are meant to measure.

The criteria to keep in mind when constructing a question paper are these:

• (a) Meeting the aims of the curriculum
• (b) The student's age, level and standard
• (c) Covering all the content
• (d) Variety
• (e) Reliability and validity
• (f) Simplicity and clarity
• (g) Practicality
• (h) Confidentiality
• (i) Balance
• (j) Using a specification grid
• (k) Preparing an answer key

The questions in a paper must be written so that they meet both the general and the specific objectives of the curriculum. An examination is held to check whether certain objectives have been reached, so each question must measure exactly those objectives. Objectives differ with the nature of the content: one lesson aims at knowledge, another at a skill, another at understanding, and another at the ability to apply what was learnt. The question has to measure that particular ability. If the aim is that students can read a map, asking "What is a map?" misses the point. Giving them a map and asking them to find the direction of a place from it measures the real aim.

Because questions are based on the curriculum rather than on one textbook, some may come from outside the textbook, as long as they stay within what the curriculum covers. This encourages students to build the habit of reading beyond their textbook.

Questions must be written with the student's age, level and standard in mind. The textbook is prepared according to the curriculum, so the teacher uses it as the base and then considers how much the students can understand and how well they can write. Asking a grade 4 child to write a two-page essay is wrong, and so is giving grade 10 students nothing but very easy questions. A question written without understanding the students' age, level and standard cannot evaluate them correctly. Some questions can also draw on the students' outside experience, meaning what they have seen and lived through at home and in their community.

A question paper must represent all of the content. It should cover the content set for the examination, with different kinds of questions drawn from every lesson. Questions should not keep returning to the same topic or lesson. Suppose ten lessons are set for an examination, but every question comes from the first three. A student who studied the other seven lessons carefully gets no chance to show it. The questions should also cover all three kinds of learning a lesson expects: knowledge, skill and attitude.

A question paper also needs variety. An examination can mix objective questions, short-answer questions and long-answer questions. Questions of only one type cannot evaluate a student properly. If every question asked students to fill in a blank, the teacher would learn how much they remember, but not whether they can reason or put their own ideas into writing. With a variety of questions, all of these abilities can be tested:

• the skill of explaining
• the ability to reason
• the ability to remember
• knowledge of the content

A question paper must be both reliable and valid. Each question must measure its objective, and it must also be worded so that everyone takes the same meaning from it. For a question to be reliable, the marks given for it must be consistent, so the same answer earns the same mark whenever it is checked. A good paper has both of these qualities.

Two examples make the difference clear. A question that says "Write about water" might lead one student to write about the uses of water, another about water pollution and another about the water cycle. Which answer counts as right depends on the examiner, and the marks change with it, so the question is not reliable. Now take a question meant to test calculation in mathematics, written in language so difficult that a student cannot work out what is being asked. The question ends up measuring reading ability instead of mathematics, so it is not valid.

A question paper must be simple and clear. Questions should be arranged from simple to complex, with the easier ones first and the harder ones later, which lets students settle in with confidence. Each question should be written in language students understand and should carry clear instructions. Clear instructions keep students from being confused about what to do. An instruction such as "Answer any three of the following five questions" leaves no doubt at all.

A question paper must be practical and useful, which means it suits the time and the conditions it is used in. It should not include questions that make the examination hard to run, such as a question needing laboratory equipment in a school that has no laboratory. While answering, students should be able to connect the question to their own lives, and questions should include current issues in society. Time matters too. The number and length of questions must fit the time allowed. If a two-hour examination holds three hours of work, even students who know everything cannot finish.

Confidentiality must be protected while a question paper is made. Typing, printing and storing the paper all have to be done securely. Keeping it secret is essential if the examination is to be fair and free from bias. If a paper leaks before the examination, the few students who have seen it gain an unfair advantage, and those who studied honestly are treated unjustly. When confidentiality is kept:

• the examination keeps its value;
• healthy competition among candidates is kept;
• the purpose for which the examination was held can be fulfilled.

A question paper should hold easy, medium and difficult questions together. A class has students at every level, so the paper needs questions that an average student can solve. At the same time, it must be able to separate the students who know a lot from those who know less. If every question is easy, everyone scores full marks and nobody can tell who knows how much. If every question is hard, many students cannot answer at all, and even what they do know stays hidden.

A specification grid should be used when preparing a question paper. The grid is drawn up according to the objectives of the curriculum.

A specification grid can be built so that the questions measure knowledge, skill and attitude. When questions are written from it, every kind of content and every objective is represented, and the paper becomes scientific. It should also be used to make the questions reliable and valid. Here is a small sample grid for a unit test of 25 full marks lasting 50 minutes:

A teacher reading this grid knows that the Evaluation lesson must carry 9 marks of questions in all: 4 marks testing knowledge, 3 testing skill and 2 testing attitude, with 18 minutes set aside for them. The grid makes sure no lesson is left out and no single lesson takes over the paper.

An answer key must be prepared so that answer sheets are marked in a uniform way. When different examiners mark the same answer sheet, each with their own feelings about it, one question can end up with very different marks. The answer key exists to stop that from happening.

Marking objective questions causes no doubt, because an answer is either right or wrong. Subjective questions are different: the examiner has to decide how many marks an answer earns depending on which topics or points it covers. Take a 4-mark question, "Write the importance of a question paper." If the answer key lists four points worth 1 mark each, any examiner will arrive at the same mark. Marking with an answer key in this way makes the checking of answer sheets fair and unbiased.

A question paper built on these criteria becomes practical and useful. If the paper is poor, the examination cannot serve the purpose of evaluation. Yet a question paper cannot show everything about a student. A habit of helping others or the skill of working in a group does not appear in a written answer. So other tools of evaluation, besides examinations, should be used too. They are:

• classroom observation
• rating scale
• anecdotal record
• self-evaluation
• personal profile
• interview
• practical work
• extracurricular activities
• class participation
• homework