trustme.bro/r/…
✓ checked
trust me, bro:
here is the receipt.
the claim
Online providers of paid participants yield high data quality and reliability in psychological experiments
the verdict
CONTESTED
contested - evenly split
refutedsupported
the weight of evidence
3 sources for · 4 against

While online participant providers can yield reliable data when using strict screening filters and platforms like Prolific or CloudResearch, overall data quality is heavily contested due to prevalent issues with inattentive respondents, speeders, and non-naivety.

Evidence for · 3
2021 · cited by 1,422
The study finds that specific platforms like Prolific and CloudResearch can yield high data quality when proper attention and screening filters are utilized.
Evidence against · 4
2021 · cited by 80
The guide highlights frequent data quality issues on crowdsourcing platforms, such as speeders, cheaters, and deliberate falsification.
See more details
The analysis

The evidence regarding online participant providers is mixed and heavily platform-dependent. While papers like [0], [1], and [5] demonstrate that platforms such as Prolific can yield high data quality and performance comparable to traditional pools under specific conditions, papers like [2], [4], [7], and [10] emphasize pervasive concerns over careless responding, bots, speeders, and low overall proportions of high-quality data. Because data reliability depends significantly on active researcher filtering and varies drastically by platform (with platforms like MTurk performing poorly), the claim cannot be universally supported and is best classified as contested.

More for · 2
2023 · cited by 1,347
Results indicate that Prolific and CloudResearch participants frequently pass attention checks, follow instructions carefully, and provide meaningful responses.
2024 · cited by 8
Data from problem-solving tasks show that Prolific participants perform comparably to traditional in-person student samples.
More against · 3
2025 · cited by 11
Research points out that participants highly motivated by income or who frequently use platforms may provide lower-quality, inattentive responses.
2025 · cited by 3
A direct comparison found that only a minority of participants on platforms like MTurk and Prolific produced high-quality data, emphasizing that data quality is often questionable.
2022 · cited by 0
Concerns regarding participant non-naivety, low motivation, and careless responding can severely compromise the reliability of online participant crowdsourcing.
The paper trail · every fact has a biography
first checked31 Jul 2026
judged → CONTESTED · 3531 Jul 2026
This receipt carries no identity, shared or not. Sharing publishes your connection to it, not your data.
Check your own claim
Challenge the receipt
trust me, bro: win the argument, pass the class, survive peer review.
This receipt is an automated verdict against our published method · not an opinion about any author or publication.
Terms · Privacy · How verdicts work · Dispute this receipt