While online participant providers can yield reliable data when using strict screening filters and platforms like Prolific or CloudResearch, overall data quality is heavily contested due to prevalent issues with inattentive respondents, speeders, and non-naivety.
The evidence regarding online participant providers is mixed and heavily platform-dependent. While papers like [0], [1], and [5] demonstrate that platforms such as Prolific can yield high data quality and performance comparable to traditional pools under specific conditions, papers like [2], [4], [7], and [10] emphasize pervasive concerns over careless responding, bots, speeders, and low overall proportions of high-quality data. Because data reliability depends significantly on active researcher filtering and varies drastically by platform (with platforms like MTurk performing poorly), the claim cannot be universally supported and is best classified as contested.