Why Research Design Matters (And Why You'll Actually Use This)
You know that feeling when you're scrolling through social media and see yet another article claiming "Scientists discover that coffee/chocolate/wine is good/bad for you"? Ever wonder why these findings seem to contradict each other every few months? The answer lies in research design.
Understanding research designs isn't just about passing the EPPP. When you're in practice, you'll need to evaluate whether that new therapy approach really works, determine if your client's progress is actually due to your intervention, and critically read research that could change how you treat people. Plus, if you ever want to conduct your own research or even track a single client's progress systematically, you'll need this knowledge in your toolkit.
Let's break down the main types of research designs in a way that actually makes sense.
The Big Picture: Qualitative vs. Quantitative
Before diving into specific designs, understand that research falls into two broad camps:
Qualitative research is about understanding the "what" and "why" of human experience. It produces rich, descriptive information that you interpret rather than count. Think of it like having an in-depth conversation with a friend about their breakup – you're gathering detailed stories, emotions, and meanings, not tracking how many times they cried.
Quantitative research is about measuring and counting. It produces numbers you can analyze statistically. It's like tracking your daily step count and sleep hours in your fitness app – concrete data you can graph and compare.
Qualitative Approaches You Should Know
Grounded Theory: Researchers develop a theory by deeply listening to participants' experiences. Imagine you're trying to understand how people decide to switch careers. Instead of testing a pre-existing theory, you interview dozens of career-changers and let the patterns emerge naturally from their stories. The theory is "grounded" in what people actually say, not what textbooks predict.
Phenomenology: This approach focuses on the lived experience of a phenomenon. If grounded theory is about finding patterns, phenomenology is about understanding what something feels like from the inside. For example, what's it really like to experience your first panic attack? How do people actually perceive and make sense of that moment?
Ethnography: Researchers immerse themselves in a culture or setting. Picture a psychologist who wants to understand the culture of an online gaming community dealing with depression, so they join the community, participate in discussions, and observe interactions over months. They're not just asking questions from outside – they're living it.
Thematic Analysis: This is the method where researchers identify recurring patterns or themes in their data. It can stand alone or support other methods. If you recorded ten therapy sessions and noticed clients repeatedly mentioning feelings of being "stuck" or "trapped," you'd be doing thematic analysis.
Making Research More Credible: Triangulation
Triangulation strengthens research by approaching a question from multiple angles. Four types exist:
| Type | What It Means | Example |
|---|---|---|
| Methodological | Use multiple methods | Combine interviews, surveys, and observations of workplace stress |
| Data | Same method, different contexts | Interview therapists in private practice, clinics, and hospitals |
| Investigator | Multiple researchers | Three researchers independently analyze therapy transcripts |
| Theory | Multiple theoretical lenses | Interpret depression data through cognitive, behavioral, and biological frameworks |
Think of it like checking your bank balance. You could just trust one source, but you feel more confident when the app, the ATM, and your monthly statement all show the same number.
Quantitative Research: Three Main Types
Descriptive Research: Taking a Snapshot
Descriptive research describes what exists without manipulating anything. You're simply documenting reality.
Surveys gather information from many people using questionnaires. Like when Netflix asks what you watched and uses that data to understand viewing patterns.
Case studies provide in-depth information about one person, group, or situation. Think of true crime documentaries that exhaustively examine one case from every angle – that's essentially a case study format.
Observational studies involve watching and recording behavior as it naturally occurs. Two key techniques:
Interval Recording (also called time sampling): You divide observation time into chunks and note whether the behavior occurred in each chunk. Imagine watching a coworker during a meeting and checking every two minutes whether they're on their phone. You're not counting every single time they look at it, just marking yes/no for each two-minute window. Best for frequent behaviors without clear starts and stops (like "seems distracted" or "participates in discussion"). There are three kinds, and each has a known bias. Partial interval scores a chunk if the behavior happens at any point in it, so it overestimates. Whole interval scores a chunk only if the behavior lasts the entire chunk, so it underestimates. Momentary time sampling scores only whether the behavior is happening at the moment the chunk ends; its errors run both ways about equally often (Powell et al., 1975).
Worked example: You watch one child for 10 minutes, split into ten 1-minute intervals (interval 1 is 0:00 to 1:00, interval 2 is 1:00 to 2:00, and so on). The child is out of their seat three times: 0:30 to 1:30, 4:10 to 4:40, and 6:50 to 8:20.
- Find the truth first. The episodes last 1 minute, 30 seconds, and 1.5 minutes. That is 3 minutes out of 10, so the child is out of seat 30% of the time.
- Partial interval: score every interval that has any out-of-seat time. That is intervals 1, 2, 5, 7, 8, and 9. Six of ten is 60%. Too high.
- Whole interval: score only intervals where the child is out of seat for the entire minute. Only interval 8 (7:00 to 8:00) qualifies. One of ten is 10%. Too low.
- Momentary time sampling: check only the last instant of each interval (1:00, 2:00, and so on). The child is out of seat at 1:00, 7:00, and 8:00. Three of ten is 30%. Here it lands right on the truth. On another day it could miss a little high or a little low, which is what "errors run both ways" means.
- For comparison, event recording would count 3 episodes, and duration recording would total 3 minutes.
So on the exam: partial interval inflates, whole interval shrinks, and momentary time sampling has no built-in direction. One more wrinkle: partial interval inflates how much of the time a behavior happens, but for a very frequent behavior it can undercount how many times it happens, because five quick episodes inside one interval still score as one interval.
Event Recording (also called event sampling): You count every single occurrence of the behavior. Best for infrequent behaviors with clear beginnings and endings. Noting when each occurrence starts and stops is a different method, duration recording, which measures how long the behavior lasts: "Left desk at 10:15am, returned 10:23am" is a duration record.
Correlational Research: Finding Relationships
Correlational research examines whether two or more variables are related. The key point: correlation doesn't equal causation. You're measuring things as they exist, not manipulating them.
You might find that people who drink more coffee score higher on anxiety measures. But you can't say coffee causes anxiety – maybe anxious people seek out coffee, or maybe stressed jobs lead to both coffee consumption and anxiety.
The data from correlational studies often feeds into regression analysis, letting you predict one variable (criterion/Y variable) from another (predictor/X variable).
Experimental Research: Establishing Cause and Effect
This is where you can actually say "X causes Y" – but only if your design is solid.
Two terms first. The independent variable (IV) is the thing the researcher changes or compares, like which treatment a person gets. The dependent variable (DV) is the outcome being measured, like a depression score. The core question is whether changes in the IV cause changes in the DV.
True Experimental Design: You randomly assign participants to different groups (different levels of your independent variable). Random assignment is crucial because it's like shuffling a deck of cards – it helps ensure groups are similar at the start, so any differences at the end are likely due to your treatment, not pre-existing differences.
Quasi-Experimental Design: You can't randomly assign participants (maybe you're comparing existing groups like "people with PTSD vs. people without PTSD," or you only have one group to work with). Without random assignment, you can't be as confident that your independent variable caused changes in your dependent variable.
Single-Subject Designs: When One Person Is Enough
Single-subject designs are powerful tools for clinical practice. They share key features:
- At least two phases: baseline (A) and treatment (B)
- Multiple measurements during each phase
- You don't start treatment until baseline is stable
Think of these like A/B testing that tech companies do, but with one person.
AB Design: The Basic Version
You measure behavior without treatment (baseline), then apply treatment and keep measuring.
The AB design is the weakest single-subject design. It does not rule out history (a one-time outside event that lines up with treatment) or maturation (slow natural changes like getting tired or growing up). If behavior changes when you start treatment, you still cannot be sure the treatment caused it. That is why the ABAB and multiple baseline designs are stronger.
Imagine you start meditating and notice your anxiety drops. But did the meditation help, or did you also just finally resolve that conflict with your roommate the same week? That's the history problem.
ABAB Design: More Convincing
Add another baseline phase and another treatment phase (baseline-treatment-baseline-treatment).
You may also see this called a reversal or withdrawal design, because you take the treatment away on purpose to see whether the behavior goes back.
It's like checking if your wireless earbuds are really causing that connection issue by disconnecting them (first baseline), connecting them (first treatment), disconnecting again (second baseline), and connecting again (second treatment). If the problem consistently appears and disappears with the earbuds, you know they're the cause.
When behavior returns to baseline after treatment withdrawal and improves again when treatment is reintroduced, you have strong evidence the treatment works.
Multiple Baseline Design: No Need to Withdraw Treatment
Instead of withdrawing treatment, you apply it sequentially to different baselines (different behaviors, settings, or people).
Here's how it works: Suppose you're helping someone reduce three problematic behaviors: interrupting others, arriving late, and forgetting commitments. You track all three behaviors. After establishing baseline for all three, you apply your intervention to just interrupting while continuing baseline for the other two. Once that improves, you add the intervention for lateness while maintaining it for interrupting and continuing baseline for forgetting. Finally, you apply it to all three.
If each behavior only improves when the intervention is applied to it specifically, you've demonstrated effectiveness without ever having to withdraw treatment (which is both more ethical and more practical).
Worked example: reading and picking a single-subject design.
- A teacher tracks a student's tantrums. Average per day: first baseline 8, first treatment 3, treatment removed 7, treatment back 2.
- Did behavior change when treatment started? Yes, 8 dropped to 3.
- Did it go back when treatment stopped? Yes, it climbed to 7. It is unlikely that an outside event (history) happened to start and stop at the same times as your phases.
- Did it improve again when treatment returned? Yes, down to 2. That is the ABAB pattern, and it is strong evidence the treatment works.
- Now change the case. The target is head banging, or the treatment teaches a skill like reading that cannot be unlearned. Taking treatment away would be unsafe, or the behavior would not go back anyway. Pick a multiple baseline design instead, and start treatment on one behavior, setting, or person at a time.
So on the exam: if the stem says treatment cannot or should not be withdrawn, the answer is multiple baseline. If it describes baseline, treatment, baseline, treatment, the answer is ABAB.
Group Designs: Between, Within, and Mixed
Between-Subjects Design
Different groups get different treatments. Each person experiences only one condition.
Like testing three different study methods by assigning different students to each method – flashcards group, practice-testing group, and re-reading group.
Randomized Controlled Trials (RCTs) are the gold standard here. They randomly assign people to treatment or control groups in controlled conditions. The random assignment increases internal validity (confidence that the treatment caused the effect) but the strict conditions can limit external validity (generalizability to real-world settings).
Within-Subjects Design
Each person experiences all (or multiple) conditions at different times.
Instead of having different students try different study methods, you have the same students try all three methods for different exam units. Everyone is their own control group.
Time-series design is essentially a group version of the AB design – you measure everyone repeatedly before and after an intervention.
Mixed Design
You have at least two independent variables: one between-subjects and one within-subjects.
You're comparing three workout programs (between-subjects: each person does only one program), but you're measuring fitness weekly for eight weeks (within-subjects: everyone gets measured repeatedly over time).
Program is between-subjects (you do either yoga, running, or weightlifting). Time is within-subjects (everyone gets measured each week).
Worked example: naming the design.
- 60 students are randomly split into three groups of 20. Each group uses one study method, and everyone takes one test at the end. Each person gets one condition, so this is between-subjects.
- 20 students use all three methods, one per exam unit, with the order shuffled across students (this is called counterbalancing, and it keeps practice or fatigue from piling up on one method). Each person gets every condition, so this is within-subjects. You need fewer people (20 instead of 60) because each person serves as their own comparison.
- 60 students are split into three method groups, and everyone is tested at weeks 1, 2, and 3. Method is between-subjects and time is within-subjects, so this is mixed.
- The design in step 3 has two independent variables, method (3 levels) and time (3 levels). That makes it a 3 x 3 factorial design with 3 times 3 = 9 cells.
So on the exam: for each independent variable, ask "does each person get one level or all of them?" One of each kind means mixed.
Factorial Designs: Testing Multiple Variables at Once
When you have two or more independent variables, you have a factorial design. The major advantage: you can examine main effects and interaction effects.
In plain words: a factor is just an independent variable, and its levels are its versions (therapy type with two levels, CBT and IPT, for example). A main effect asks, "Ignoring the other factor, does this one matter on average?" An interaction asks, "Does this factor's effect change depending on the other factor?" Designs are named by their levels. A 2 x 3 design has two factors, one with 2 levels and one with 3, so it has 2 times 3 = 6 conditions (called cells).
Main effect: The effect of one factor averaged across the levels of the other factor. A nonsignificant main effect does not prove the averaged effect is zero (Greenland et al., 2016; Langenberg et al., 2023). Interaction effect: When the effect of one variable depends on the level of another variable.
| Effect Pattern | Example |
|---|---|
| Main effects only | Therapy type matters; medication dose matters; no interaction |
| Significant interaction, nonsignificant main effects | The therapy effect depends on medication dose; the averaged main-effect tests are not significant |
| Both main and interaction | Therapy matters, medication matters, AND certain combinations work especially well |
Here's a real-world scenario: You're testing whether caffeine and sleep affect test performance. You might find a main effect of sleep (more sleep = better scores), a main effect of caffeine (caffeine = better scores), but also an interaction: caffeine barely helps well-rested people but dramatically helps sleep-deprived people. That interaction is crucial information you'd miss with simpler designs.
When there's a significant interaction, be cautious interpreting main effects – the interaction might tell the more important story.
Worked example: reading main effects and an interaction from four group averages. Say a study crosses sleep (rested or sleep deprived) with caffeine (none or some). The average test scores are:
| No caffeine | Caffeine | |
|---|---|---|
| Rested | 80 | 82 |
| Sleep deprived | 60 | 74 |
- Main effect of sleep: average each row. Rested is (80 + 82) / 2 = 81. Sleep deprived is (60 + 74) / 2 = 67. That is a 14-point gap.
- Main effect of caffeine: average each column. No caffeine is (80 + 60) / 2 = 70. Caffeine is (82 + 74) / 2 = 78. That is an 8-point gap.
- Interaction: compare the caffeine effect inside each row. For rested people it is 82 minus 80, or 2 points. For sleep-deprived people it is 74 minus 60, or 14 points. The effect of caffeine depends on sleep, so the pattern shows an interaction. On a graph, the two lines would not be parallel.
- Now a crossover case. Therapy A scores 70 with a low dose and 50 with a high dose. Therapy B scores 50 with a low dose and 70 with a high dose. Each therapy averages 60, and each dose averages 60, so neither main effect shows up. But the better dose flips depending on the therapy. That is a strong interaction with no main effects, which is why you read the interaction first.
- These numbers show the pattern only. Whether a gap is statistically significant depends on the test (here, a factorial ANOVA), the sample size, and how spread out the scores are.
So on the exam: main effects come from row and column averages. An interaction means the gap in one row differs from the gap in the other row, which shows up as nonparallel lines on a graph.
Special Research Approaches
Analogue Research
This involves studying situations that approximate but don't replicate real-life conditions.
Like how flight simulators train pilots – not exactly like flying a real plane, but close enough to be useful and much safer for learning.
Common examples: using college students instead of clinical populations, or conducting therapy studies in laboratories instead of actual therapy offices.
Advantage: Better internal validity (more control over variables) Disadvantage: Worse external validity (findings may not generalize)
Developmental Research
These designs study change over time.
Longitudinal: Follow the same people over time. Like having yearly check-ins with the same friend group and watching how everyone's careers evolve over a decade.
| Advantages | Disadvantages |
|---|---|
| See actual developmental changes | Time-consuming and expensive |
| Track individual patterns | Attrition bias (dropouts may differ from completers) |
Cross-sectional: Compare different age groups at one time point. Instead of following one group for ten years, you interview 25-year-olds, 35-year-olds, and 45-year-olds all this year.
| Advantages | Disadvantages |
|---|---|
| Quick and relatively inexpensive | Cohort effects (groups differ in more than age) |
| No attrition problems | Can't track individual change |
Cross-sequential: Combine both approaches. You interview 30-year-olds, 40-year-olds, and 50-year-olds today, then follow all three groups and interview them again in ten and twenty years.
More expensive than cross-sectional, less than longitudinal. Helps separate age effects from cohort effects.
Worked example: spotting a cohort effect.
- In 2026, a cross-sectional study tests 30-year-olds and 70-year-olds on a smartphone skills task. The 70-year-olds score lower.
- Is that aging? Maybe not. The 70-year-olds were born around 1956 and the 30-year-olds around 1996. They grew up with very different technology. A difference that comes from when people were born, not how old they are, is a cohort effect, and a cross-sectional design cannot pull it apart from age.
- A longitudinal fix would follow the 30-year-olds until they turn 70. That takes 40 years, and people drop out along the way (attrition).
- A cross-sequential fix tests 30-, 40-, and 50-year-olds in 2026, then retests all three groups in 2036. In 2036, the first group is 40. Compare them with the group that was 40 back in 2026. Same age, different birth years (1996 vs. 1986). If their scores differ, the gap comes from cohort (or from the year they were tested), not from age.
So on the exam: cross-sectional is confounded by cohort, longitudinal is slow and threatened by attrition, and cross-sequential helps separate age from cohort.
Sampling: Who's in Your Study?
Probability Sampling (Random Selection)
Everyone in the population has an equal (or known) chance of selection. This helps ensure your sample represents the population.
Simple random sampling: Like drawing names from a hat – pure chance determines who's selected.
Systematic random sampling: Select every nth person from a list (every 10th person, every 25th person).
Stratified random sampling: Imagine you're selecting participants and want to ensure various age groups are represented. You divide your population into age categories (strata), then randomly select from each category.
Cluster random sampling: Instead of randomly selecting individual users from all of Twitter (impossible), you randomly select specific hashtag communities, then sample from within those communities.
Even with random sampling, sampling error can occur – your sample isn't perfectly representative just due to chance, especially with small samples.
Non-Probability Sampling (Non-Random Selection)
Not everyone has an equal chance of selection. This introduces sampling bias (also called selection bias or systematic error).
Convenience sampling: You use whoever's easily available. Like surveying people in your apartment building – easy, but hardly representative.
Voluntary response sampling: People volunteer to participate. Think of online polls where people choose to respond – the most motivated (or most extreme) opinions are overrepresented.
Purposive/judgmental sampling: You deliberately select people who fit your needs. If you're studying therapist burnout, you intentionally recruit therapists, not random healthcare workers.
Snowball sampling: You ask participants to recommend others. Like when you ask a freelancer you hired for recommendations of other freelancers in their network. Especially useful for hard-to-reach populations.
Sampling Error vs. Sampling Bias, in Plain Words
Sampling error is bad luck. Even a fair random draw can, by chance, pick a few more older people than the population has. It shrinks as the sample gets bigger. Sampling bias is a tilted process. If only motivated people answer your poll, a bigger sample just gives you more motivated people. More people do not fix bias. A better method does.
Also keep two kinds of "random" apart. Random selection (sampling) decides who gets into the study, and it supports generalizing to the population (external validity). Random assignment decides which group each participant joins, and it supports causal claims (internal validity). A study can have one without the other.
Worked example: name the sampling method.
- A clinic lists all 2,000 patients, picks a random starting point, then takes every 20th name. That gives 2,000 / 20 = 100 people. This is systematic random sampling.
- A researcher splits licensed psychologists by license type, then randomly picks 50 from each type. This is stratified random sampling (random picks of people inside every subgroup).
- A researcher randomly picks 5 of a district's 40 schools, then surveys every student in those 5 schools. This is cluster random sampling (random picks of whole groups).
- A researcher posts a survey link on social media and uses whoever answers. This is voluntary response sampling.
- A researcher interviews one person who left a high-control religious group, then asks for names of others who left. This is snowball sampling.
So on the exam: stratified randomly picks people from every subgroup, and cluster randomly picks whole subgroups. Ask what was chosen at random, the individuals or the groups.
Community-Based Participatory Research (CBPR)
CBPR is action research that aims to improve social problems by involving community members as equal partners throughout the research process.
Instead of researchers studying a community from outside, community members help design the study, collect data, interpret findings, and implement changes.
Key principles include:
- Recognize community identity and strengths
- Share power equally among all partners
- Focus on problems the community actually cares about
- Commit to long-term sustainability
- Ensure all partners learn from each other
Rather than a university researcher studying homelessness by interviewing homeless individuals, CBPR would involve homeless individuals as research partners who help shape research questions, interpret findings, and develop solutions.
Where CBPR Comes From
CBPR sits inside a larger family called participatory action research (PAR). The defining idea is research done "with" or "by" people rather than "on" or "about" them, paired with a commitment to act on what is learned. Three roots are worth knowing: Kurt Lewin, who coined the term "action research"; Paulo Freire, whose Pedagogy of the Oppressed (1970) rejected the expert-over-subject hierarchy; and feminist scholarship on whose knowledge counts. The W.K. Kellogg Foundation definition is the one most often quoted: a collaborative approach that equitably involves all partners, starts with a topic the community cares about, and combines knowledge with action to reduce health disparities. Collins and colleagues (2018) brought CBPR to a general psychology audience in American Psychologist and argued that it lines up naturally with psychology's ethical principles of beneficence, justice, and respect for people.
Cousins you may see named on the exam: patient and public involvement (PPI), integrated knowledge translation (IKT), co-production, and citizen science. All share the same move, which is treating the people a study is about as partners with real decision-making power.
How Much Participation Is "Participation"?
Involvement runs on a continuum, and the exam likes frameworks that name the rungs.
Arnstein's Ladder of Citizen Participation (1969) has eight rungs in three bands. The bottom band is non-participation (manipulation and therapy). The middle band is tokenism (informing, consultation, placation). The top band is citizen power (partnership, delegated power, citizen control). Arnstein's line to remember: participation without a redistribution of power is an empty and frustrating process for the powerless. Hart (1992) adapted the ladder for youth participation.
The IAP2 Public Participation Spectrum is the shorter version: inform, consult, involve, collaborate, empower. Only the last two rungs share decisions.
In plain words, the next model is a flow chart: the setting shapes the partnership, the partnership shapes the study, and the study shapes the results. Two terms help. Structural equation modeling is a statistical method that tests a whole diagram of arrows (A leads to B, B leads to C) at once. A mediator is the middle link, the B that carries the effect from A to C.
The CBPR Conceptual Model (Wallerstein, Oetzel, Duran, and colleagues) links four boxes in order: contexts, partnership processes, intervention and research, and outcomes. Tests of the model with structural equation modeling found that partnership synergy and genuine community involvement are the mediators that carry a partnership toward health equity outcomes. Newer frameworks include the Bidirectional Engagement and Equity (BEE) framework (Cunningham-Erves et al., 2024), which tracks equity across relationship building, forming the partnership, forming the research team, and conducting the research.
Think of the ladder like a mixing desk. Informing the community is turning up the monitor speakers so they can hear the track. Partnership is handing them a fader. Citizen control is letting them produce the record.
Sharing Power Across the Research Cycle
Power sharing is the defining feature and the hardest one. Formal involvement that leaves the researchers holding every decision is not truly participatory. At each stage, partners can do real work:
- Setting the question: the community names the problem and its priorities ("nothing about us without us"), which counters researcher- or funder-driven agendas.
- Design: partners co-develop protocols, culturally adapt measures, and choose outcomes that matter locally.
- Recruitment: partners design culturally tailored materials, serve as peer ambassadors, and translate consent tools.
- Analysis and interpretation: lived experience contextualizes the numbers ("multiple ways of knowing").
- Dissemination: findings go back to the community first, and partners co-author and co-own the products.
Hopkins and colleagues (2024) describe power in mental health co-production as operating at three levels at once: structural, interpersonal, and individual. Values mapping and power mapping tools (Littman et al., 2021) exist to make those dynamics explicit at the start rather than discovering them at the end.
Community Advisory Boards
The community advisory board (CAB) is the most common operating mechanism. Members are community residents, people with lived experience, and representatives of local organizations. A CAB works as a two-way liaison: it builds trust, educates the community about the study, brings community concerns back to the team, supplies cultural context, reviews protocols and consent forms, promotes recruitment, and helps disseminate results. Common failure modes are poor management, no formal participation structure, language and literacy barriers, and no budget or operating guidelines.
What the Evidence Shows
Plain version: research suggests CBPR helps with trust, recruitment, and keeping people in studies, but few studies test it head to head against other approaches. (A meta-synthesis, cited below, is a review that pulls together findings from many studies.)
The evidence base is strongest for process and relationship outcomes and weaker for head-to-head effectiveness comparisons.
- Recruitment and retention: in a meta-synthesis of strategies for improving the representation of underrepresented groups (Peters et al., 2024), CBPR was the most frequently used strategy, appearing in roughly 63 percent of studies, with cultural humility in about 40 percent. Wieland and colleagues (2021) reviewed 66 studies and concluded that community engagement likely promotes minority recruitment and retention, while noting that comparative effectiveness studies are scarce.
- Trust: community relationships and peer ambassador models rebuild trust among groups with a history of being excluded or harmed by research.
- Relevance and sustainability: grounding a study in community priorities improves acceptability and the durability of interventions after the grant ends.
- Capacity: recurring themes across the literature are empowerment, transparency, and community skill-building.
The caveat the exam may test: most of this literature is descriptive. Very few studies directly compare one engagement strategy against another.
Risks and Tensions
- Tokenism: involvement limited to late-stage consultation, or symbolic seats at the table with no shared authority. Researchers frequently keep control of the agenda, the methods, and the ethics paperwork.
- Burden on partners: a rapid review by Wearn and colleagues (2025) documented emotional burden, frustration, exposure to negative attitudes, and even further marginalization when involvement is poorly supported. Fair compensation and a real budget line for partners are the fix.
- Timelines and incentives: CBPR takes longer and costs more, academic tenure clocks and grant cycles run faster than community pace, and promotion systems often undervalue engaged scholarship.
- Data ownership: disputes over who owns the data and the products are recurrent, especially when the team and the community differ in power, language, or race and ethnicity.
Ethics, Consent, IRBs, and Data Governance
Ethics guidance now treats communities as stakeholders, not just as pools of individual subjects. The practical expectations are that research responds to community needs, minimizes exploitation, shares benefits (returning health information, investing in local infrastructure), and approaches culture with humility, meaning an ongoing stance of learning rather than a checklist of competencies.
Indigenous and marginalized communities need extra protections because of documented research harms (misuse of genetic samples, harmful publications, inadequate consent). Expect tribal sovereignty, tribal or area IRB review, flexible timelines, and community-first dissemination. Two data governance frameworks are high-yield: OCAP (Ownership, Control, Access, Possession, from First Nations in Canada) and the CARE principles (Collective benefit, Authority to control, Responsibility, Ethics). CARE is meant to complement FAIR (Findable, Accessible, Interoperable, Reusable), which is a data-centric open-science standard and says nothing about the people the data describe.
Informed consent shifts from a one-time individual transaction toward community or collective consent alongside individual consent, plain-language and translated tools, and ongoing communication. Broad consent and open-data reuse are ethically problematic for Indigenous data unless the community agreed to them.
IRB review is a known friction point. Flicker and colleagues (2007) found that IRB forms are built around a biomedical frame of individual risk and rarely account for community-level risk or community co-ownership of knowledge. Brown and colleagues (2010) documented IRBs that were unfamiliar with CBPR and resisted reporting individual results back to participants and ongoing researcher-participant contact, positions that can contradict beneficence and justice. Community members also hold dual roles as participants and as co-creators of data, which review boards need to disentangle.
Data governance moves toward shared or community ownership, data-sharing agreements negotiated up front, and governance structures built by the community rather than only data collected for the community.
Dissemination and Presentation of Research Findings
A study that nobody reads changes nothing. The EPPP treats getting findings into practice as its own knowledge area, and the core distinction is between diffusion, the passive, uncontrolled spread of an idea, and dissemination, the active, planned, tailored delivery of evidence to a target audience. Passive spread (publish and hope) is largely ineffective at changing what practitioners do. Active, tailored approaches work better.
Frameworks Worth Naming
Plain version: these models answer different questions. Rogers explains how a new idea spreads. RE-AIM scores whether a program works in the real world, not just in a trial. Knowledge-to-Action is a step-by-step plan for getting evidence used. Brownson lists the parts of a message you can tune.
- Diffusion of Innovations (Rogers, 1962; fifth edition 2003): the classic theory of how innovations spread over time through communication channels within a social system, with adopter categories from innovators and early adopters through the late majority and laggards. Most later dissemination models build on it.
- RE-AIM (Glasgow, Vogt, and Boles, 1999): an evaluation framework with five dimensions, Reach, Effectiveness, Adoption, Implementation, and Maintenance, designed to judge real-world public health impact and external validity, not just whether a treatment worked in a trial.
- Knowledge-to-Action (Graham et al., 2006): a process model with a knowledge-creation funnel (inquiry, synthesis, tools) feeding an action cycle: adapt knowledge to the local context, assess barriers and facilitators, select and tailor interventions, monitor use, evaluate outcomes, sustain use.
- Brownson's model for dissemination of research: built on communication theory and Rogers, it names four components that determine impact: source, message, channel, and audience. In experimental dissemination studies the channel is by far the most manipulated component (about 86 percent of studies), which leaves source, message, and audience tailoring understudied.
- Nilsen (2015) sorted the crowd of theories, models, and frameworks into five categories: process models, determinant frameworks, classic theories, implementation theories, and evaluation frameworks. Other names that appear: PARiHS, the Consolidated Framework for Implementation Research (CFIR), EPIS, and Kingdon's multiple streams model of policy windows.
Brownson's four components are a mailing: who sends it (source), what it says (message), how it travels (channel), and who opens it (audience). Most researchers obsess over the envelope and forget the reader.
Matching Strategy to Audience
The strategy should follow the goal (awareness only, or actual uptake) and the audience.
- Researchers: journal articles and conference presentations.
- Practitioners: educational meetings and outreach, audit and feedback, decision support tools, and tailored educational materials. These are the active behavioral tools that move practice.
- Policymakers: policy briefs, knowledge exchange events, knowledge brokers, and one-on-one engagement timed to policy windows.
- Communities and the public: plain-language summaries, infographics, comics, podcasts, videos, press releases, and social media.
Multi-channel, multi-audience plans are the norm. One channel rarely reaches everyone who needs the finding.
Returning Results to Participants
Most participants want to know what a study found, and most are never told. Sharing results builds trust and future participation, and community-engaged research treats returning findings to the community first as an obligation. Head-to-head evidence on formats is thin: a UK trial found no difference in understanding between a plain-language summary and a standard press release, and a Croatian trial found no knowledge difference between a plain-language summary and an infographic, although readers rated the infographic as more readable and user-friendly.
Reporting Standards
The EQUATOR Network (2006) catalogs reporting guidelines, and each major one maps to a study design:
- CONSORT (1996, updated 2001, 2010, and 2025) for randomized controlled trials: a checklist plus a participant flow diagram. CONSORT-SPI (2018) extends it to social and psychological interventions, and a nonpharmacologic-treatment extension (2017) covers psychotherapy trials.
- PRISMA for systematic reviews and meta-analyses.
- STROBE for observational studies (cohort, case-control, cross-sectional).
- SPIRIT for trial protocols.
- APA Journal Article Reporting Standards (JARS) (2008, revised 2018) for APA journals. JARS has four modules: JARS-Quant (Appelbaum and colleagues) for quantitative studies, JARS-Qual for qualitative studies, JARS-Mixed for mixed methods, and JARS-REC, which applies to all studies and covers reporting on race, ethnicity, and culture. The 2018 revision splits hypotheses, analyses, and conclusions into primary, secondary, and exploratory, and adds modules for clinical trials, longitudinal, replication, N-of-1, structural equation modeling, and Bayesian designs.
Why these exist: reporting in psychology and social science trials has historically been poor. In a review of 239 trials, only 20 percent identified the study as randomized in the title, blinding was reported in 15 percent, and allocation concealment in 17 percent.
Open Science
The Transparency and Openness Promotion (TOP) guidelines (Nosek et al., 2015, in Science) set the standard, and the TOP Factor scores journals on it. Core practices:
- Preregistration: stating hypotheses, primary and secondary outcomes, and the analysis plan before seeing the data. It is the main defense against HARKing (hypothesizing after results are known) and optional stopping.
- Registered reports: peer review and in-principle acceptance happen before the results exist, so publication no longer depends on which way the findings went.
- Preprints: posting to a server such as PsyArXiv or the Open Science Framework before peer review, for speed.
- Open data and code: de-identified data and analysis scripts in a repository so others can reuse and reproduce the work.
- Open access: the paper itself is free to read.
Research on these practices largely supports better reproducibility and reliability. The costs are less flexibility, more time, and publishing fees.
Publication and Presentation Ethics
- Authorship (ICMJE criteria): an author must meet all four criteria: substantial contribution to the conception or design, or to acquiring, analyzing, or interpreting the data; drafting or critically revising the work; final approval; and accountability for the work. Everyone who meets all four should be listed, and nobody who does not. Getting the funding, collecting data, or supervising in general does not qualify on its own; those people go in the acknowledgments.
- Duplicate publication: publishing the same data twice as if new is a breach; ICMJE allows a secondary publication only under specific stated conditions.
- Conflicts of interest: the ICMJE disclosure form asks authors to list all relevant relationships and activities and lets readers judge relevance, instead of asking authors to decide what counts as a conflict.
- Questionable research practices: flawed peer review, undeserved authorship, selective citation, and deceptive reporting sit on a spectrum below fabrication, falsification, and plagiarism, but they are still integrity failures.
What Actually Changes Practice
Plain version: handing out guidelines does little by itself, and tailoring the push to what is actually blocking people helps a bit more. Three terms in the next paragraph. An odds ratio compares the odds of an outcome between two groups (1 means no difference, above 1 favors the first group). Moderate certainty means the true effect is probably close to the estimate but could be meaningfully different. High heterogeneity means the results varied a lot from study to study.
Passive dissemination (mailing guidelines, publishing and waiting) produces little or no change on its own, though it can work when end users are already motivated, the organization supports the change, and a local champion pushes it. Active strategies do better. A 2026 Cochrane review of tailored implementation strategies (79 studies, more than 25,000 professionals) found that strategies tailored to local barriers probably produce a slight improvement over untailored ones, with an odds ratio of about 1.5, moderate certainty, and high heterogeneity. The components that show up most often in effective packages are educational materials, educational meetings, and audit and feedback, often combined with outside facilitation.
An odds ratio of 1.5 does not mean 1.5 times as many people. Say 40% of clinicians follow a guideline under untailored strategies. Their odds are 40 to 60, or about 0.67. Multiply by 1.5 and the odds become 1.0, which is 50%. So 40% rises to 50%, not to 60%.
Pitfalls When Presenting Results
Plain version: these are ways honest-looking numbers can mislead. A few terms first. Relative risk compares two risks as a ratio or a percent change. Absolute risk difference is the plain gap in percentage points. p-hacking means running the analysis different ways until one version crosses p < .05. Optional stopping means checking the data again and again and stopping as soon as the result is significant. Publication bias means significant results are more likely to get published than null ones.
- Spin: reporting that makes results look better than the data support (Boutron and Ravaud, 2018). A 2026 systematic review found spin in roughly 59 percent of trial abstracts and 65 percent of main texts. Spin travels: spin in an abstract conclusion was the strongest predictor of spin in the press release, and randomized experiments show that spin in news stories causes readers to overestimate treatment benefit.
- Misrepresenting effect sizes: emphasizing relative over absolute risk, dropping confidence intervals, and burying a nonsignificant primary outcome under a significant secondary one.
- p-hacking and related practices: optional stopping, flexible covariates and subgroups, selective outcome reporting, and HARKing. Combined with publication bias, these inflate meta-analytic effect estimates, worst when the true effect is small. They are the leading explanation offered for the replication crisis; the 2015 Open Science Collaboration reproduced only about a third of 100 published psychology findings, and the effects that did replicate were roughly half their original size.
- Overgeneralization: extrapolating past the studied population or design, such as causal claims from observational data or animal-to-human leaps, which is one of the most common distortions in news coverage.
Worked example: relative vs. absolute risk.
- In a trial, 2 of every 100 people on placebo relapse (2%). On the new drug, 1 of every 100 relapse (1%).
- Relative risk reduction: the risk fell from 2% to 1%, which is half. A headline can say "cuts relapse risk by 50%."
- Absolute risk reduction: 2% minus 1% is 1 percentage point.
- Number needed to treat (NNT): 1 divided by the absolute reduction, so 1 / 0.01 = 100. You treat 100 people to prevent one relapse.
- Same data, very different feel. Reporting only the 50% is the "relative over absolute risk" problem listed above.
So on the exam: a big relative number with no absolute number is a red flag. Absolute risk and NNT show the real-world size of the effect.
Common Misconceptions to Avoid
-
"Quasi-experimental means low quality": Not true. Sometimes random assignment is impossible or unethical. Quasi-experimental designs can be rigorous and valuable.
-
"Single-subject designs aren't real research": Wrong. They provide strong evidence of treatment effectiveness and are incredibly practical for clinical work.
-
"You need huge samples for good research": Sample size depends on your design and goals. Single-subject designs work with one person. Small qualitative studies can provide deep insights.
-
"Qualitative research is just opinions": No. Rigorous qualitative research follows systematic methods and uses strategies like triangulation to ensure credibility.
-
"Correlational research is useless because it doesn't show causation": Correlational research is valuable for prediction, identifying relationships worth studying experimentally, and studying variables you can't ethically manipulate.
-
"Community involvement means better recruitment": Recruitment help is a byproduct. CBPR means shared decision-making power over the questions, the design, the interpretation, and the products. A community that only signs people up has been consulted, not partnered with.
-
"Publishing the paper is dissemination": Publication is passive diffusion. Dissemination is the active, planned, tailored delivery of the finding to the people who can use it, and passive spread on its own rarely changes practice.
Memory Aids for the EPPP
For single-subject designs: A = Away from treatment, B = Bringing in treatment. More phases = more confidence.
For developmental designs:
- Longitudinal = Long-term with same people
- Cross-sectional = Cross-section of different ages now
- Cross-sequential = Crosses both approaches
For sampling: If it has "random" in the name, it's probability sampling. If not, it's non-probability.
For factorial designs: "Main" effects are simple (one variable). "Interaction" effects are complex (variables depend on each other).
For observational recording:
- Interval = In chunks of time
- Event = Every occurrence
- Partial interval = Pumps up (overestimates), Whole interval = Whittles down (underestimates), Momentary = errors both ways
For participation ladders: Arnstein's bottom rungs are Manipulation and Therapy (no participation), the middle is Informing, Consulting, Placating (tokenism), and the top is Partnership, Delegated power, Citizen control (real power). If the community cannot change a decision, it is still tokenism.
For Indigenous data: OCAP = Ownership, Control, Access, Possession. CARE = Collective benefit, Authority to control, Responsibility, Ethics. FAIR is about the data (Findable, Accessible, Interoperable, Reusable); CARE is about the people.
For dissemination models: RE-AIM = Reach, Effectiveness, Adoption, Implementation, Maintenance. Brownson = Source, Message, Channel, Audience ("who says what, how, to whom").
For reporting standards: CONSORT = trials, PRISMA = reviews, STROBE = observational, SPIRIT = protocols, JARS = APA journals.
For authorship: four ICMJE criteria, all four required: contribute, write or revise, approve, and answer for it.
Key Takeaways
- Qualitative research provides rich description and understanding; quantitative research provides measurable data
- Triangulation strengthens research credibility by approaching questions from multiple angles
- True experimental designs allow causal claims through random assignment; quasi-experimental designs lack random assignment, so their causal claims are weaker
- Single-subject designs (AB, ABAB, multiple baseline) are powerful for evaluating individual treatment effects
- Between-subjects = different people in different conditions; within-subjects = same people in all conditions
- Factorial designs reveal both main effects and interactions between variables
- RCTs maximize internal validity but may sacrifice external validity
- Longitudinal research tracks change in the same people; cross-sectional compares different age groups at once
- Probability sampling allows generalization; non-probability sampling is vulnerable to bias but useful for exploration
- CBPR involves community members as equal partners throughout research
- Participation is a continuum (Arnstein's ladder, IAP2 spectrum); involvement without shared decision-making power is tokenism
- Community involvement changes the ethics: collective consent alongside individual consent, IRBs that weigh community-level risk, benefit sharing, and data governance frameworks like OCAP and CARE
- Dissemination is active and tailored; diffusion is passive, and passive spread rarely changes practice
- Rogers, RE-AIM, Knowledge-to-Action, and Brownson's source-message-channel-audience are the dissemination frameworks to know by name
- Reporting standards (CONSORT, PRISMA, STROBE, APA JARS) and open science (preregistration, registered reports, open data) exist because selective reporting, spin, and p-hacking inflate the literature
- Authorship requires all four ICMJE criteria; funding, data collection, or supervision alone do not earn a byline
Remember: Research design isn't just academic busy-work. Every time you read a study claiming a treatment works, evaluate if someone's progress is real, or track your own clinical outcomes, you're using these concepts. Master them, and you'll be a more critical consumer and producer of psychological knowledge.
Idiographic questions and replication
Idiographic means studying one person in depth. It is the opposite of nomothetic research, which looks for general rules across many people. A person's trajectory is just the path their scores take over time.
An idiographic analysis focuses on patterns within an individual. Repeated observations across baseline, treatment, and follow-up can describe that person's trajectory. Replicating the design across people lets researchers examine whether similar patterns recur; one case alone does not establish a population-wide effect (Aschieri et al., 2024).
Flag-resolution sources
- Benedikt Langenberg, Markus Janczyk, Valentin Koob, et al. A tutorial on using the paired t test for power calculations in repeated measures ANOVA with interactions. Behavior research methods. 2023 Aug;55(5):2467-2484. doi:10.3758/s13428-022-01902-8. PMID:36002625. https://pubmed.ncbi.nlm.nih.gov/36002625/
