Overview
The credibility of research findings is crucial for advancing knowledge and societal progress. Some believe that high-status universities, like Ivy League schools, produce more reliable research due to their selectivity and resources. In theory, an organization that's able to attract top talent and money should be able to produce better research.
However, elite universities have come under fire for simply increasing the quantity of research without regard to quality or relevance. As the recent replication crisis shows, faculty have been incentivized to simply publish in "A-level" journals to meet tenure requirements and advance their own careers, rather than serve the general public through reproduceable findings.
Irrespective of their actual credibility, the public’s perceptions of Ivy League universities and their research is an interesting topic. Do people trust research from an “elite” school more or less than research from state universities?
We put this question to the test with a novel experiment comparing the perceived credibility of research from both types of schools.
The Experiment
We recruited 400 participants from Amazon Mechanical Turk, a platform commonly used for research studies, to take part in a survey discreetly designed as an experiment. Participants were provided with a research study summary and informed that it was from either Harvard University or Salem State University (randomly assigned). Participants then answered a survey question to rate their perceptions of the research’s believability.
Participants were presented with the following statement:
The following claim may or may not be true:
A research study from [Harvard University / Salem State University] suggests that internet search engines reduce our ability to remember information.
A research study from [Harvard University / Salem State University] suggests that internet search engines reduce our ability to remember information.
We measured our outcome of interest by asking participants the following survey question, "How strongly do you believe this research finding is true?” Responses were provided on a 1-7 scale (1 = Not at all, 7 = Very much).
Results
We found no significant difference in the believability of the research finding when it was attributed to Harvard University (avg. = 4.62) relative to Salem State University (avg. = 4.58) (p = 0.813). Our 95% confidence interval of -0.28 to 0.36 on a 1-7 scale indicates that any difference, if one actually exists, would likely be trivial, at least in this context.
Conclusion
Contrary to the assumption that Ivy League research is viewed as more credible, our findings suggest that the public does not give a credibility boost to research from high-status universities. While this may challenge some beliefs, it opens the door to further investigation into the actual credibility of research from different types of institutions. With a growing dataset of research study replications, we hope to someday delve deeper into this intriguing question.
Methods Note
We used Ordinary Least Squares (OLS) regression analysis to test for significant differences in believability between our ivy league and state university research conditions. A small p-value (less than 0.05) indicates a statistically significant difference. The survey materials and data for this study are available upon request.