Introduction to Statistical Testing (AQA A-Level Psychology): Revision Notes
Introduction to Statistical Testing
What are statistical tests?
Statistical tests provide a method for analysing data collected from psychological research. These tests help researchers determine whether their findings are meaningful or could have occurred by chance alone.
Statistical testing involves comparing an observed value (calculated from your data) with a critical value (found in statistical tables). This comparison tells you whether your results are statistically significant - meaning they are unlikely to have occurred by chance and suggest a genuine effect or difference.
Statistical significance is the foundation of evidence-based research in psychology. It helps distinguish between genuine effects and random variation in data, ensuring that conclusions drawn from research are reliable and meaningful.
The sign test
The sign test is a non-parametric statistical test used when researchers want to analyse the difference between two sets of related data. This test is particularly useful in repeated measures designs where the same participants are tested under two different conditions.
When to use the sign test
Conditions for Using the Sign Test:
The sign test is appropriate when:
- You predict a difference between two sets of data (such as in an experiment)
- Your data is at least nominal level (can be categorised)
- You have used a repeated measures design (RMD)
- You want to assess the direction of any difference between pairs of scores
How statistical significance works
When you run a statistical test, you get an observed value from your calculations. This observed value must then be compared to a critical value from statistical tables.
Critical Comparison Rule for Sign Test:
For the sign test specifically:
- The observed value must be equal to or less than the critical value to be considered statistically significant
- If your result is significant, you can reject the null hypothesis
- If your result is not significant, you must accept the null hypothesis
Worked example: breakfast cereal preferences
Let's examine how to conduct a sign test using a practical example.
Worked Example: Breakfast Cereal Preference Study
Scenario: A food manufacturer wants to know if their new breakfast cereal 'Fizz-Buzz' will be as popular as their existing product 'Kiddy-Slop'. Ten participants try both cereals and indicate their preference.
Step 1: Organise your data
Create a table showing each participant's preference and assign a direction sign:
- Positive (+) sign for participants who prefer the new product (Fizz-Buzz)
- Negative (-) sign for participants who prefer the existing product (Kiddy-Slop)
- Omit participants who show no preference
From the example data:
- 7 participants preferred Fizz-Buzz (+)
- 1 participant preferred Kiddy-Slop (-)
- 2 participants showed no difference (omitted)
Step 2: Calculate the observed value
To find your observed value (s), count the number of times the less frequent sign appears in your data.
In this example:
- Positive signs: 7
- Negative signs: 1
- Observed value (s) = 1 (the less frequent sign)
Step 3: Determine your critical value
You need several pieces of information to find the critical value:
- N = 8 (number of participants with a preference, excluding those with no difference)
- Two-tailed hypothesis (testing for any difference, not a specific direction)
- Significance level p ≤ 0.05 (standard level used in psychology)
Using these parameters in the critical values table: Critical value (cv) = 0
Step 4: Interpret your results
Compare your observed value to the critical value:
- Observed value (s) = 1
- Critical value (cv) = 0
- Since 1 > 0, the result is not significant
Conclusion: Accept the null hypothesis. There is no statistically significant difference in preference between the two breakfast cereals.
Understanding the result
You might find it surprising that seven preferences for one product versus one preference for another doesn't represent a statistically significant difference. This occurs because the sample size was relatively small - with only 8 valid responses, there weren't enough participants to demonstrate a difference that goes beyond the boundaries of chance.
Sample Size Impact:
Small sample sizes can prevent detection of genuine effects. Even when results appear convincing (like a 7:1 preference ratio), statistical tests may still indicate non-significance due to insufficient data to rule out chance variation.
Critical values table for the sign test
Using Critical Values Tables:
The critical values depend on:
- Sample size (N): Number of participants with valid preferences
- Type of hypothesis: One-tailed (directional) or two-tailed (non-directional)
- Significance level: Usually p ≤ 0.05, p ≤ 0.025, p ≤ 0.01, or p ≤ 0.005
For a two-tailed test at p ≤ 0.05:
- N = 8: Critical value = 0
- N = 9: Critical value = 1
- N = 10: Critical value = 1
- N = 12: Critical value = 2
- N = 15: Critical value = 3
- N = 20: Critical value = 5
Remember the Rule: Your observed value must be equal to or less than the critical value to achieve significance.
Analysis and interpretation of correlation
Correlation analysis examines the relationship between two co-variables in correlational studies. The result is a correlation coefficient - a numerical value that indicates both the strength and direction of the relationship between variables.
Understanding correlation coefficients
Correlation coefficients range from:
- +1.0: Perfect positive correlation (as one variable increases, the other increases)
- 0.0: No correlation (no relationship between variables)
- -1.0: Perfect negative correlation (as one variable increases, the other decreases)
The closer the coefficient is to +1 or -1, the stronger the relationship. Values near zero indicate little or no correlation between the variables.
Interpreting Correlation Strength:
Generally, correlation coefficients can be interpreted as:
- 0.7 to 1.0 (or -0.7 to -1.0): Strong correlation
- 0.3 to 0.7 (or -0.3 to -0.7): Moderate correlation
- 0.0 to 0.3 (or 0.0 to -0.3): Weak correlation
Statistical tests for correlational data
Correlational data can be analysed using two main statistical tests:
- Spearman's rho: Used for non-parametric data or when assumptions for parametric tests aren't met
- Pearson's product moment correlation: Used for parametric data that meets specific assumptions
Both tests help determine whether an observed correlation is statistically significant or could have occurred by chance.
Key Points to Remember:
- Statistical tests compare observed values from your data with critical values from tables to determine significance
- The sign test is used for repeated measures designs with nominal-level data to test for differences between conditions
- For significance in the sign test, your observed value must be equal to or less than the critical value
- Sample size matters - small samples may not show significance even with apparent differences
- Correlation coefficients range from +1 to -1, indicating the strength and direction of relationships between variables