Self-Report Data and Sampling Techniques (Edexcel A-Level Psychology): Revision Notes
Self-Report Data and Sampling Techniques
Self-report data
Self-report methods are techniques used to collect data on people's feelings, attitudes, opinions, personality traits, and related characteristics. These methods typically involve surveys, which can be administered through questionnaires or large-scale interviews. The primary advantage of self-report methods is their ability to gather substantial amounts of information efficiently.
Questionnaire
Questionnaires are designed to collect large amounts of data by accessing large samples of participants. They can be administered through various means including post, email, face-to-face contact, or online platforms. Questions typically require participants to provide information about their attitudes, opinions, lifestyles, and other aspects of their lives.
Types of question
Questionnaires can gather different types of information or data. Quantitative data is numerical and can be described as information that is or can be converted into numbers. Qualitative data is defined as non-numerical prose. The type of question asked determines whether the data collected is quantitative or qualitative.
Closed questions have preset fixed answers that respondents select from by circling or ticking the option that most closely matches their opinion. These questions yield quantitative data and can be structured as yes/no responses or as lists of options.
Closed Question Examples:
- Are you male or female?
- Do you have a pet? Yes/No
- Age: 18-40 years/41-60 years/60+ years (please circle)
Whilst respondents find closed questions quick and easy to answer, and researchers can easily analyse the data they produce, they have limitations. Respondents may find them frustrating if the available answers do not match what they would like to express. Closed questions are also limited in the amount of information that can be analysed; researchers cannot determine why respondents answered in a particular way, so the level of detail obtained is restricted.
Attitude scales have been developed to increase the level of detail achieved in questionnaires. These involve more than a yes/no response, instead offering respondents a range of different options so that their strength of opinion can be measured.
A Likert-type scale involves respondents selecting from a fixed set of choices to rate agreement to a series of statements. For example, the Adorno et al. survey on authoritarianism used a Likert scale (see Section 1.1 Content, Factors affecting prejudice).
Likert Scale Question Example:
| Likert Scale Question | Strongly Agree | Agree | Neutral | Disagree | Strongly Disagree |
|---|---|---|---|---|---|
| Naughty children should be smacked | 5 | 4 | 3 | 2 | 1 |
Likert scales involve respondents rating their opinion, and ranked scales involve respondents ranking their choices relative to other options. To score ranked questions, each preference should be given a weighting. For example, if respondents are asked to rank animals according to how fearful they are of them, the most feared animal would have to be given a higher weighting than the least feared. The animal with the highest score will represent the most feared animal.
Ranked Scale Question Example:
| Ranked Scale Question | Rank |
|---|---|
| Horse | |
| Rat | |
| Spider | |
| Cat | |
| Rabbit | |
| Dog | |
| Cow |
Open questions do not involve preset answers but instead allow respondents to answer freely. This allows them to elaborate on their answers and justify their opinions. However, open questions require more time and effort from respondents, and open-ended question responses require qualitative analysis, which can lead to subjective interpretation by the researcher.
Key distinction: Objective vs subjective
Objective means not open to interpretation, whilst subjective means open to interpretation. For example, if you ask someone to estimate the length of a classroom table, it is likely that their estimation will differ from someone else's. This is a subjective interpretation. However, if you ask someone to use a ruler to measure the length of the classroom table, it is likely that their measurement will be exactly the same as someone else's. This is an objective measure.
Quantitative data is objective because numbers are numbers and therefore not open to interpretation. Qualitative data can be subjective because the meanings found in written prose can be open to interpretation.
Issues with questionnaire design
Social desirability
Social desirability occurs when respondents do not give genuine answers but ones which depict them in a more favourable light. That is, they respond to a question in a way that is seen as desirable according to prevailing social norms. For example, few respondents would say they agree with segregation or that heterosexuality was the only natural sexuality, because their responses would be contrary to current social norms and they would be seen in a negative light.
Social desirability can be particularly problematic for research investigating socially sensitive issues or attitudes that go against social norms. However, many questionnaires have built-in lie detectors that can detect socially desirable responses. If too many of these lie-detecting questions are answered in a socially desirable way, the respondent's questionnaire can be excluded from further analysis.
Question construction
Designing questions for a questionnaire can be tricky. However, as a researcher is not present when the questionnaire is being completed, it is essential that questions are not too technical, ambiguous, or complex. It is also important that questions do not lead or mislead respondents into giving a particular answer or ask personal questions, as this violates the right to privacy.
When designing Likert scale statements, it is important to consider the number of options provided, as an odd number of answers to a scale means that the middle value may be selected more frequently. Using an even number of answers on a scale forces respondents to make a choice rather than select a 'neutral' or 'neither agree nor disagree' option.
Response bias or response acquiescence can occur when using Likert-style scales. If all the statements in a set of statements are worded favourably or unfavourably, respondents can slip into just agreeing or just disagreeing with all of them. To resolve this, statements should be reversed and mixed up.
Avoiding Response Bias
Response bias can be avoided by reversing statements and mixing them up in the questionnaire. Examples of statement reversal:
- Marriage helps society to function. / Society does not help marriage to function.
- Pets make people happy. / Pets do not make people happy.
- Politicians help the economy thrive. / The economy can thrive without politicians.
Questionnaire reliability
Reliability refers to the consistency of a measure or finding. External reliability refers to the consistency of a measure or finding over time. Internal reliability refers to the consistency of a measure within itself. Some questionnaires and scales lose their external reliability if respondents repeat them on different occasions, so it is important to establish whether this is the case by using the test-retest method. This literally means that the same people are given the same questionnaire to complete again on a different occasion. If their responses are the same or very similar, external reliability can be established.
Internal reliability is a problem for questionnaires because often several different questions are used to measure the same trait or attitude. The various scales used by Adorno et al. on conservatism, ethnocentrism etc. contained many items intended to measure these concepts. But did they all equally measure the same concept?
In order to establish internal reliability, a split-half method can be employed. This involves splitting the questions into two halves and comparing the findings from both halves during analysis. If all of the questions are measuring the same concept, both halves should achieve the same score. If they do not, it suggests that some of the questions may be measuring a different concept.
Questionnaire validity
Validity refers to the extent to which something is measuring what it intends to measure. If you design a questionnaire intending to measure attitudes about education, then you need to be sure that you design your questions so that they measure this attitude and nothing else.
Sometimes this can be established by simply looking at each question and deciding whether it makes sense in terms of the construct being measured. This is known as face validity. This can also be confirmed by asking an expert in the field to review the questions.
If a questionnaire is a valid measure of a construct, such as intelligence, then it should have predictive validity. This means that it is able to accurately predict the same construct in the future. If an intelligence test has predictive validity, a high intelligence score should correlate with educational success, such as A-level or degree grading.
Another way of establishing whether the questions in a questionnaire are valid is by comparing it to another test measuring the same construct. This is known as concurrent validity.
Interviews
An interview can be used in a survey if it can be administered to a large sample of people relatively easily. This is more likely if it is a structured interview.
Structured interview
Structured interviews are defined by the nature of the questions and the way in which they are asked. Typically, structured interviews are standardised so that all respondents are asked the same questions in the same way, often using closed questions that gather quantitative data. Structured interviews tend to be easy to administer and do not need to establish a rapport between the researcher and respondent. However, the data gathered can be superficial and lack depth, and the respondent may feel stifled and not be able to express their opinions fully, which can be as frustrating as answering closed questions in a questionnaire.
Semi-structured interview
To avoid some of the problems with structured interviews, semi-structured interviews are more conversational and dynamic. A researcher has a set of questions that they aim to be answered, but do not have a standardised format to follow. This means that the conversation can flow a little bit better, whilst still achieving the research aim and getting relevant information from respondents. This type of interview can gather both quantitative and qualitative data.
Unstructured interview
This type of interview begins with a loose research aim and gathers qualitative information from respondents. Unlike structured interviews, the interviewer needs to be analytical during the interview so that they can probe and seek meaning from respondents. An unstructured interviewer needs to be skilled at achieving a good rapport with respondents and responsive to the information offered; they need good listening skills and should use non-judgemental language.
Ethical Considerations in Unstructured Interviews
Ethical issues are important when conducting any type of questionnaire or interview, but are particularly critical when using an unstructured interview because the qualitative data gathered can make direct reference to quotes from respondents. It is important that all respondent details are anonymised and personal details disguised. Due to the reflexive nature of an unstructured interview, the interviewer must deal sensitively when asking for personal information to ensure they do not breach the respondent's right to privacy.
Researcher effects
When asking people questions, there are many interviewer characteristics that can influence the respondent; the sex, age, manner, and personality of the interviewer can all affect how a person responds, whether they are truthful, and whether they disclose information at all.
It is important to predict what characteristics might influence respondents and control them. For example, you can predict that a male interviewer will be unlikely to obtain detailed information from a female participant about their view of marriage. This can be controlled by employing a female interviewer.
Alternative hypotheses
In addition to an overall research aim, a study might also make a prediction about what is likely to occur. This prediction is known as an alternative hypothesis. A hypothesis should contain the variables under investigation and be a clear, testable, and precise statement at the beginning of a report. This prediction is often guided by previous research in the topic area, but if there is limited previous research or mixed findings, the prediction may have to just state that a difference or relationship might be found between the variables under investigation, but not what direction the difference/relationship may take. You will learn more about hypothesis construction in Topic 2: Cognitive psychology (see Section 2.2 Methods).
Sampling techniques
In psychological research, it is necessary to recruit participants or respondents to study. The way in which these participants are selected is known as sampling. It is unlikely that a whole population can be studied, so a sample of the population needs to be gathered using a sampling technique. The technique used will depend on the type of research being conducted and the availability of the participants, but the aim of a sampling method is to select a representative sample of participants; that is, a sample that represents the characteristics of the population well. This will ensure that any conclusions drawn from the research can be successfully generalised back to explain the behaviour of the target population as a whole.
Target population
A target population is the population of people being investigated by a study. For example, if you are investigating attitudes about the NHS at a local hospital, the target population will be people at the hospital. A sample will be recruited from this target population using a sampling technique.
Identifying Target Populations:
- Asking teachers at a school about their views on canteen food → Target population: Teachers at the school
- Using a gym membership list to recruit participants for a survey about exercise → Target population: Gym members
- Stopping people in a street to take part in research → Target population: People in that street/area
- Investigating stress at work in an office building → Target population: Workers in the office building
If the sample gathered is not representative, because of an over- or underrepresentation of a particular type of participant in the sample use, a sampling bias will occur.
Sampling Bias Examples:
Consider these scenarios and their potential impact:
- A researcher recruits student participants in the canteen at lunchtime to find out about attitudes towards A-level reforms
- Participants volunteer to take part in a study about eating habits by responding to an advert in a woman's magazine
- A researcher uses the telephone directory to gather a sample of participants for a study on health-related behaviour
- Attitudes towards a no-smoking policy are gathered from participants recruited from an outside office smoking area
Each of these examples demonstrates how the sampling method can introduce bias that affects the representativeness of the sample and limits the generalisability of conclusions.
Random sampling
The most likely way to recruit a representative sample is by using a random sampling technique. This should ensure that everyone has an equal chance of being selected. A random sample can be achieved in a number of ways. Computers are capable of producing random sequences of numbers, so every person in a target population can be assigned a number, and the computer-generated numbers can be used to select a sample if the numbers correspond. A simpler way would be to place the names of every member of the target population into a hat, shuffle them and draw at random.
Random sampling should result in a representative sample, although this may not always be the case because you can select an unrepresentative sample at random, too. Even if your random sample is representative of the target population, you still need to obtain consent from each participant selected. If they decide to not take part, you may be left with an unrepresentative sample in the end.
Stratified sampling
If the target population has salient characteristics that need to be proportionately represented in the sample recruited, a stratified sampling technique can be used. For example, if you are investigating stress in the workplace in a company, you can find out how many staff occupy different roles within the company, for example office clerks, managers, canteen staff, cleaners, etc. As there may be more clerks than managers, more clerks need to be recruited for the study than managers in order to represent the company staff more fairly. Each subgroup within the company can be randomly sampled by placing the names of all the clerks in a hat, for example, and drawing out a proportionate number.
Opportunity sampling
An opportunity sample makes use of participants who are available. This can involve a researcher going to a student common room and asking people to take part, or investigating passers-by in a high street. Either way, the researcher has limited control over who is recruited and not everyone in a target population has an equal chance of being selected.
Volunteer sampling
Self-selected participants can be recruited by placing an advert in a newspaper or a student common room. Volunteers are self-selecting because they choose to take part; they are not approached and asked by a researcher. The researcher has no control over who volunteers and often a certain type of participant may choose to take part. This can result in a sample bias. However, a researcher may pre-test volunteers before the main study and exclude those with characteristics they feel may not represent the target population.
Key Points to Remember:
- Self-report methods include questionnaires and interviews, used to collect data on attitudes, opinions, and feelings.
- Closed questions yield quantitative data, whilst open questions yield qualitative data.
- Social desirability bias occurs when respondents answer in a way that depicts them favourably rather than truthfully.
- Reliability (external and internal) and validity (face, predictive, concurrent) are essential for ensuring questionnaires measure consistently and accurately.
- Sampling techniques (random, stratified, opportunity, volunteer) aim to select representative samples from target populations to allow generalisation of findings.