If you give the same test to the same people twice, do you get similar results? That simple question is at the heart of test-retest reliability — one of the most practical checks a researcher can run on a questionnaire, scale, or test. If scores jump around for no good reason, you cannot trust what the instrument is telling you. If they stay stable, you have good reason to believe your measurement is dependable.
In this guide, you will learn what test-retest reliability means in plain English, how it differs from other kinds of reliability, and how to measure it step by step, including a worked example, interpretation guidance, and the mistakes that most often trip students up.
What Is Test-Retest Reliability?
Test-retest reliability is a form of reliability that shows whether a measurement gives consistent results over time. The procedure is straightforward: administer the same test to the same group of people on two occasions, then compare the two sets of scores. If the instrument is reliable, people who scored high the first time should generally score high the second time, and people who scored low should generally score low again.
The key word here is “generally.” Nobody expects identical scores. People have good days and bad days, they may remember a question, or they may genuinely change a little between the two occasions. Test-retest reliability does not demand perfection. It asks whether the pattern of scores is stable enough that the test can be trusted.
This kind of reliability matters most when you are measuring something that should not change quickly: a personality trait, an attitude, general anxiety, job satisfaction, or a stable ability. If you are measuring something that changes fast, such as mood after an intervention or knowledge after a training course, test-retest reliability is the wrong tool — change is exactly what you expect to see, and a low stability coefficient may simply mean your intervention worked.
A Simple Example
Imagine you develop a short questionnaire measuring statistics anxiety among undergraduate students. You give it to a class on a Monday and again two weeks later, with no statistics teaching in between. If the students who were most anxious the first time are still the most anxious the second time, and the least anxious students are still the least anxious, your questionnaire shows good test-retest reliability. If the rankings shuffle completely, something is wrong: either the questions are unclear, the trait is unstable, or outside factors interfered.
Notice what this example reveals. Test-retest reliability is not about whether scores are high or low. It is about whether the relative standing of people stays similar across time. That is why researchers usually report a correlation between the two sets of scores rather than simply comparing averages.
Test-Retest Reliability vs Other Types of Reliability
Reliability has several faces, and students often mix them up. The table below keeps them straight.
| Type of reliability | What it checks | How it is usually tested |
|---|---|---|
| Test-retest reliability | Stability over time | Same test, same people, two occasions. |
| Internal consistency | Whether items on one test hang together | Cronbach’s alpha on a single administration. |
| Inter-rater reliability | Agreement between different scorers or observers | Two raters score the same responses and their agreement is compared. |
| Parallel-forms reliability | Whether two different versions of a test give similar results | Two equivalent forms given to the same group. |
These types answer different questions, so they complement rather than replace each other. A questionnaire can have excellent internal consistency but poor test-retest reliability, for instance, if its items agree with each other on one day but the whole construct drifts over time. Good research reports the type of reliability that matches the claim being made.
How to Measure Test-Retest Reliability: Step by Step
- Step 1: Choose a suitable sample. Aim for a group that resembles the population your test is designed for. Small samples make reliability estimates very wobbly, so most researchers aim for at least 30 participants, and larger samples give more stable estimates. Keep the sample as similar as possible on both occasions.
- Step 2: Pick an appropriate time interval. This is the hardest decision in the whole process. Too short an interval, and participants may simply remember their previous answers, which inflates reliability. Too long an interval, and real change in the trait itself will lower it. For many questionnaires, an interval of two to four weeks is a common compromise, but the right interval depends on what you are measuring. A mood scale may need days; a personality scale can tolerate months. Whatever you choose, state the interval clearly when you report your results, because a reliability coefficient without its interval is hard to interpret.
- Step 3: Administer the test under similar conditions. Use the same instructions, the same format, and as far as possible the same setting on both occasions. If the first administration is a quiet classroom test and the second is a rushed online form completed on a phone, differences in scores may reflect the conditions, not the test.
- Step 4: Score both administrations in exactly the same way. Apply the same scoring rules, including any reverse-scored items, to both sets of responses. A scoring slip between the two occasions can destroy an otherwise sound reliability study.
- Step 5: Calculate the correlation between the two sets of scores. The usual statistic is the Pearson correlation coefficient (r) between Time 1 and Time 2 scores. Many researchers now prefer the intraclass correlation coefficient (ICC), because it also takes account of any systematic shift in scores between occasions, not just whether the ranking is preserved. If you report an ICC, say which form you used, since different ICC forms answer slightly different questions. Most statistical software, including SPSS, can compute both.
- Step 6: Report and interpret your result honestly. Report the coefficient, the sample size, the time interval, and the statistic used. Also check for obvious problems: did many participants drop out between occasions, did scores shift noticeably in one direction, or did a few extreme scores drive the correlation? A reliability coefficient is only as trustworthy as the study behind it.
A Worked Example (Simplified and Illustrative)
Suppose a researcher pilots a 10-item statistics anxiety scale with 40 students. Each student completes the scale at the start of the semester and again three weeks later, before any statistics teaching begins. Scores can range from 10 to 50.
The researcher enters both sets of scores into SPSS, computes the Pearson correlation, and finds r = 0.84. The means are almost identical on both occasions, and no student changes by more than a few points. The conclusion is reasonable: over a three-week interval, the scale gives stable results for this group, and test-retest reliability looks good.
Now imagine a different outcome: r = 0.45, with several students shifting by 15 points or more. That would be a warning sign. Before blaming the scale, the researcher should ask what changed. Did an exam fall between the two occasions? Did some students discuss the questions afterwards? Did the administration conditions differ? Only after ruling out these explanations should the scale itself be revised, for example by rewording ambiguous items or clarifying the response format.
Note that this example is deliberately simplified to show the logic. Real reliability studies report more detail, including confidence intervals for the coefficient, but the reasoning is exactly the same.
How to Interpret a Test-Retest Coefficient
There is no single magic cut-off that applies to every situation, but researchers commonly use rough rules of thumb. As a general guide, a coefficient below 0.70 is often seen as questionable for research use, 0.70 to 0.79 as acceptable, 0.80 to 0.89 as good, and 0.90 and above as excellent. Treat these as guidelines, not laws. A screening tool used for high-stakes decisions needs higher reliability than a short scale used in an exploratory classroom survey. Context, sample size, and the purpose of the test all matter, and a coefficient should always be reported with its interval and sample so readers can judge it for themselves.
Also remember that a reliability coefficient describes a test used with a particular group on a particular occasion. It is not a permanent property stamped on the questionnaire. The same scale can show strong test-retest reliability with university students and weaker reliability with a very different group, or over a much longer interval. That is why published scales report reliability evidence from several studies, not just one.
What Can Go Wrong: Factors That Affect Test-Retest Reliability
Several factors can distort your result, and examiners love asking about them. Memory effects occur when participants recall their earlier answers, which artificially inflates stability, especially with short intervals. Practice effects happen when taking the test once changes performance the second time, a particular problem with ability and performance tests. Genuine change in the trait itself lowers stability, but that is a property of what you are measuring, not a flaw in the test. Finally, changes in testing conditions, illness, fatigue, or major events between occasions add noise that has nothing to do with the instrument. A thoughtful reliability study tries to control what it can and honestly reports what it could not.
Common Mistakes to Avoid
The first mistake is choosing the wrong reliability type: reporting test-retest reliability for something that is supposed to change, or reporting internal consistency when the research question is really about stability over time. The second is using a tiny sample and treating the resulting coefficient as precise. The third is hiding the interval: a coefficient of 0.85 over two days is a very different achievement from 0.85 over six months, so always state it. The fourth is reporting only the coefficient and nothing else — no sample size, no statistic, no note about dropouts. And the fifth is assuming reliability proves validity. A test can give beautifully stable scores and still measure the wrong thing. Reliability is necessary for validity, but it is not the same as validity.
Frequently Asked Questions
What is the difference between test-retest reliability and internal consistency? Test-retest reliability checks stability over time using two administrations of the same test. Internal consistency checks whether the items of a single test agree with each other at one point in time, usually with Cronbach’s alpha. You need the first when stability matters and the second when you want to know whether a scale hangs together as one measure.
How long should the interval between the two tests be? It depends on the trait. Two to four weeks is a common choice for many questionnaires because it is long enough to reduce simple recall of answers but short enough that the trait should not have changed much. For fast-changing states, a shorter interval may be appropriate; for stable traits, a longer one is defensible. Whatever you choose, justify it and report it.
Should I use Pearson’s r or the intraclass correlation coefficient? Pearson’s r is the traditional and simplest choice and is widely understood. The intraclass correlation coefficient is increasingly preferred because it also penalises systematic changes in scores between occasions, such as everyone scoring higher the second time. If you are writing a thesis, check what your supervisor or department expects, and state clearly which statistic you used.
What sample size do I need for a test-retest study? Larger is better, but a common practical minimum is around 30 participants, with 50 or more giving a more stable estimate. Very small samples can produce coefficients that look impressive or terrible largely by chance, so treat results from tiny samples with caution.
Can a test have good test-retest reliability but still be a bad test? Yes. Reliability only means scores are consistent, not that they are meaningful. A bathroom scale that always reads five kilograms too heavy is perfectly reliable and completely wrong. Always check validity alongside reliability before adopting an instrument.
Conclusion
Test-retest reliability answers a question every researcher should ask before trusting a measurement: if I measure the same people again, do I get a similar picture? By choosing a sensible interval, keeping conditions constant, and reporting your coefficient honestly with its sample size and interval, you give your readers real evidence that your instrument is dependable. Pair that evidence with validity checks, and you have a measurement foundation your whole study can stand on.