In this paper, we present three retrospective observational studies that investigate the relation between data sharing and statistical reporting inconsistencies. Previous research found that reluctance to share data was related to a higher prevalence of statistical errors, often in the direction of statistical significance (Wicherts, Bakker, & Molenaar, 2011). We therefore hypothesized that journal policies about data sharing and data sharing itself would reduce these inconsistencies. In Study 1, we compared the prevalence of reporting inconsistencies in two similar journals on decision making with different data sharing policies. In Study 2, we compared reporting inconsistencies in psychology articles published in PLOS journals (with a data sharing policy) and Frontiers in Psychology (without a stipulated data sharing policy). In Study 3, we looked at papers published in the journal Psychological Science to check whether papers with or without an Open Practice Badge differed in the prevalence of reporting errors. Overall, we found no relationship between data sharing and reporting inconsistencies. We did find that journal policies on data sharing seem extremely effective in promoting data sharing. We argue that open data is essential in improving the quality of psychological science, and we discuss ways to detect and reduce reporting inconsistencies in the literature.

The practices for if and how scholarly journals instruct research data for published research to be shared is an area where a lot of changes have been happening as science policy moves towards facilitating open science, and subject-specific repositories and practices are established. This study provides an analysis of the research data sharing policies of highly-cited journals in the fields of neuroscience, physics, and operations research as of May 2019. For these 120 journals, 40 journals per subject category, a unified policy coding framework was developed to capture the most central elements of each policy, i.e. what, when, and where research data is instructed to be shared. The results affirm that considerable differences between research fields remain when it comes to policy existence, strength, and specificity. The findings revealed that one of the most important factors influencing the dimensions of what, where and when of research data policies was whether the journal’s scope included specific data types related to life sciences which have established methods of sharing through community-endorsed public repositories. The findings surface the future research potential of approaching policy analysis on the publisher-level as well as on the journal-level. The collected data and coding framework is provided as open data to facilitate future research and journal policy monitoring.

Many journals are moving towards a ‘Mandatory Inclusion of Raw Data’ (MIRD) model of data sharing, where it is expected that raw data be publicly accessible at article submission. While open data sharing is beneficial for some research topics and methodologies within health psychology, in other cases it may be ethically and epistemologically questionable. Here, we outline several questions that qualitative researchers might consider surrounding the ethics of open data sharing. Overall, we argue that universal open raw data mandates cannot adequately represent the diversity of qualitative research, and that MIRD may harm rigorous and ethical research practice within health psychology and beyond. Researchers should instead find ways to demonstrate rigour thorough engagement with questions surrounding data sharing. We propose that all researchers utilise the increasingly common ‘data availability statement’ to demonstrate reflexive engagement with issues of ethics, epistemology and participant protection when considering whether to open data.

Preregistration is the act of formally documenting a research plan before collecting (or at least before analysing) the data. It allows those reading a final research report to know which aspects of a study were decided before sight of the data, and which were added later. This enables informed evaluation of the severity with which scientific claims have been tested. We, as the British Neuropsychological Society Open Research Group, conducted a survey to explore awareness and adoption of open research practices within our field. Neuropsychology involves the study of relatively rare or hard-to-access participants, creating practical challenges that, according to our survey, are perceived as barriers to preregistration. We survey the available routes to preregistration, and suggest that the barriers are all surmountable in one way or another. However, there is a tension, in that higher levels of bias control require greater restriction over the flexibility of preregistered studies, but such flexibility is often essential for neuropsychological research. Researchers must therefore consider which route provides the right balance of rigour and pragmatic flexibility to render a preregistered project viable for them. By mapping out the issues and potential solutions, and by signposting relevant resources and publication routes, we hope to facilitate well-reasoned decision-making and empower neuropsychologists to enhance the transparency and rigour of their research. Although we focus neuropsychology, our guidance is applicable to any field that studies hard-to-access human samples, or involves arduous or expensive means of data collection.

concerns in psychology about publication bias and data dredging, more than 140 psychology journals have adopted result-blind peer review. In this approach The replication crisis, also known as the reproducibility or replicability crisis, refers to widespread failures to reproduce published scientific results. Because the reproducibility of empirical results is the cornerstone of the scientific method, such failures undermine the credibility of theories and challenge substantial parts of scientific knowledge. Psychology and medicine have been focal p In… In response to concerns in psychology about publication bias and data dredging, more than 140 psychology journals have adopted result-blind peer review. In this approach, studies are accepted not on the basis of their findings and after the studies are completed, but before they are conducted and on the basis of the methodological rigor of their experimental designs, and the theoretical justifications for their statistical analysis techniques before data collection or analysis is done. Early analysis of this procedure has estimated that 61% of result-blind studies have led to null results, in contrast to an estimated 5% to 20% in earlier research. In addition, large-scale collaborations between researchers working in multiple labs in different countries that regularly make their data openly available for different researchers to assess have become much more common in psychology.

Representational drift of fixed stimuli, learned tasks and familiar environments is observed in many brain areas, leading to reconfiguration of population codes over days to weeks. This raises the question of whether downstream brain regions employ mechanisms to track changes in population activity and thus preserve the fidelity of the information they extract. We show that the statistical properties of drift have a significant impact on such mechanisms. Over an extended period, a net change in population tuning due to drift can arise from an accumulation of small changes distributed across the population, or via abrupt jumps that affect smaller subsets of cells at each time point. We demonstrate that an adaptive readout can exploit the heavy-tailed statistics of abrupt jumps to maintain a more stable readout using a simple inference mechanism. Using experimental data, we investigate the extent to which heavy-tailed drift statistics are observed during representational drift in the posterior parietal cortex and visual cortex. We find that experimentally measured drift does not conform to a Gaussian random walk. Instead, we find sudden jumps in neural tuning that would be advantageous for a downstream observer adapting to changes in representation. These observations motivate future study to determine whether adaptive decoding mechanisms exist in the brain and to determine the physiological mechanisms that shape the statistics of representational drift.