Statistical Testing (AQA A-Level Psychology): Revision Notes
Statistical Testing
Introduction to statistical testing
Statistical testing provides researchers with methods to determine whether their findings represent genuine effects or simply occurred by chance. In psychology, these tests help us understand whether observed differences or relationships between variables are statistically significant or merely coincidental.
Statistical testing is fundamental to psychological research because it provides an objective way to distinguish between meaningful findings and random variation in data.
The sign test represents one of the simpler statistical tests available to researchers. It examines differences in scores when the same participants are tested under two different conditions, making it particularly useful for repeated measures designs.
Key concepts in statistical testing
Statistical testing
Statistical testing offers a systematic approach for deciding whether to accept or reject research hypotheses. These tests reveal whether observed differences or relationships between variables in psychological research are statistically meaningful or have simply occurred through chance variation.
The sign test
The sign test is a straightforward statistical procedure used to analyse differences in scores between related measurements. This test works when the same participants are tested twice under different conditions, requiring data to be organised into categories (nominal data or better).
Understanding significance
When researchers discover differences between experimental conditions, they must determine whether these represent genuine effects. A significant difference means the results are unlikely to have occurred purely by chance. Without statistical testing, researchers cannot distinguish between meaningful findings and random variation.
The concept of probability
Psychology adopts a significance level of (5%) as the accepted threshold for determining statistical significance. This means that even when researchers find significant differences or relationships, there remains up to a 5% probability that these results occurred by chance alone.
Different Significance Levels
For research involving higher stakes (such as medical trials), more stringent significance levels like (1%) may be employed to reduce the chance of false positive results.
Critical values
After calculating a statistical test, researchers obtain a calculated value that must be compared against a critical value to determine significance. The critical value represents the threshold that must be reached for results to be considered statistically significant.
For the sign test, researchers need three pieces of information to find the appropriate critical value:
- The desired significance level (typically or 5%)
- The number of participants (N value)
- Whether the hypothesis is directional (one-tailed) or non-directional (two-tailed)
Requirements for using the sign test
Essential Conditions for the Sign Test
The sign test can only be applied when specific conditions are met:
- Looking for differences: The research must investigate differences rather than associations or correlations between variables
- Repeated measures design: The same participants must be tested under both experimental conditions
- Nominal data: Data should be organised into categories, though higher levels of measurement can be converted to nominal format for this test
Conducting the sign test
The sign test follows a systematic procedure that transforms data into a simple comparison format, making it accessible even for researchers new to statistical analysis.
Worked Example: Step-by-Step Sign Test Procedure
Step 1: Convert data to nominal format Calculate the difference between each participant's scores in the two conditions. Record whether each participant showed improvement (+), decline (-), or no change (ignored in analysis).
Step 2: Count the signs Add up the total number of positive signs and negative signs. Ignore any participants who showed no change between conditions.
Step 3: Identify the calculated value (S) The calculated value S equals the smaller of the two totals (positive or negative signs). This becomes the value to compare against the critical value.
Step 4: Compare with critical value Using the table of critical values, locate the appropriate critical value based on:
- Sample size (N)
- Significance level ()
- Whether the hypothesis is one-tailed or two-tailed
Step 5: Determine significance For the sign test, the calculated value (S) must be equal to or less than the critical value for results to be considered statistically significant.
Interpreting results
- If critical value: Results are statistically significant
- If critical value: Results are not statistically significant
Table of critical values
The sign test uses a specific table showing critical values for different sample sizes and significance levels. For significance to be demonstrated, the calculated S value must equal or fall below the critical value shown in the table.
Key Features of Critical Values Table
- Different values for one-tailed versus two-tailed tests
- Lower critical values for smaller sample sizes
- More stringent criteria (lower critical values) for higher significance levels
Worked example interpretation
Worked Example: Interpreting Sign Test Results
When researchers apply the sign test to their data, they may find that despite observing differences between conditions, these differences do not reach statistical significance. This occurred in the energy drink example where the calculated value () exceeded the critical value (), indicating the observed differences could reasonably be attributed to chance rather than the experimental manipulation.
Such findings remind researchers that observed differences in mean scores do not automatically indicate statistically significant effects. Proper statistical testing remains essential for distinguishing genuine experimental effects from random variation.
Key Points to Remember:
- Statistical testing determines whether research findings represent genuine effects or chance occurrences
- The sign test analyses differences in repeated measures designs using nominal data
- Psychology typically uses 0.05 significance level, meaning 5% chance results occurred by chance
- For the sign test, the calculated value (S) must be equal to or less than the critical value for significance
- Results require proper statistical analysis before drawing conclusions about experimental effects