Plain English with Derek Thompson: Megapod – Why Is There So Much BS in Psychology?

Release Date: November 27, 2024
Host: Derek Thompson, The Ringer
Episode: Megapod: Why Is There So Much BS in Psychology?

Introduction

In this compelling episode of Plain English with Derek Thompson, Thompson delves deep into the pervasive issues plaguing the field of psychology. Through insightful conversations with social psychologist Adam Mastroianni (introduced as Brian Phillips in the transcript) and Atlantic journalist Dan Engber, the episode dissects the replication crisis, scientific fraud, and the broader implications these challenges have on both academia and real-world applications.

The Replication Crisis and Its Implications

Host: Derek Thompson
Guest: Adam Mastroianni (Brian Phillips)

Admiration and Concern for Psychology

Thompson begins by expressing his deep appreciation for psychology, revealing his admiration for the field while simultaneously grappling with a sense of disillusionment. He acknowledges the impressive strides psychology has made in overturning "folk psychology" — the intuitive, often inaccurate beliefs people hold about human behavior.

Key Contributions of Psychology

Adam Mastroianni:
"I think psychology has been really successful in taking our folk intuitions about human behavior and putting them to the test. For example, discovering that our feelings after bad events aren't as permanent as we thought has been a revelation."
(04:52)

Two Proto-Paradigms in Social Psychology

Mastroianni outlines two foundational concepts that psychology has contributed over the past five decades:

1. Cognitive Biases: Originating from the pioneering work of Kahneman and Tversky in the 1970s, cognitive biases like the availability heuristic have become widely recognized, even earning Nobel Prizes in Economics.

   Mastroianni:
   "People are familiar with terms like the availability heuristic. These are concepts that, if you discuss them, people recognize them because they've become part of the common discourse."
   (10:15)

2. Situational Influence: Psychology has demonstrated that environments and situations significantly influence human behavior, challenging the notion that certain behaviors are solely rooted in individual traits.

   Mastroianni:
   "The Milgram shock experiments showed that ordinary people could perform extraordinary acts under certain situational pressures. This shifted our understanding of human capacity for both good and evil."
   (12:05)

The Replication Crisis

Thompson introduces the replication crisis, highlighting that numerous psychological studies, including seminal findings on power poses and ego depletion, have failed to replicate when redone.

Mastroianni:
"The replication crisis has shown that many findings in psychology don't hold up under repeated scrutiny. This doesn't mean all of psychology is flawed, but it does highlight significant issues."
(14:14)

Consequences of the Replication Crisis

Mastroianni discusses both positive and negative outcomes stemming from the crisis:

• Positive:
  "We're now much better at identifying and avoiding questionable research practices. There's increased transparency and methodological rigor."
  (16:04)

• Negative:
  "However, we've also adopted the flawed approach of questioning the validity of all research without considering its practical significance. Many non-replicated studies weren't particularly impactful to begin with."
  (16:04)

Big Five Theory vs. Non-Scientific Models

A critical examination of the Big Five personality traits reveals that when compared to non-scientific personality tests like the Enneagram, the Big Five performs no better in predicting real-world outcomes.

Mastroianni:
"When the Big Five was pitted against the Enneagram in predicting behaviors like marriage or job types, they performed similarly. This calls into question the empirical superiority of the Big Five."
(24:17)

Nudges and Their Efficacy

The discussion shifts to nudges, small environmental changes intended to influence behavior. Mastroianni cites mega studies indicating that many nudges fail to produce the desired effects, and even experts struggle to predict which interventions will work.

Mastroianni:
"In mega studies testing 53 different nudges to increase gym attendance, only about half had any effect. Experts couldn't reliably predict which ones would work, and non-experts sometimes outperformed them."
(26:46)

Technological Applications of Psychology

Contrasting psychology with hard sciences like physics, Mastroianni argues that psychology hasn't effectively translated its findings into reliable technologies or interventions.

Mastroianni:
"Unlike physics, where theories can lead directly to technologies like transistors, psychology hasn’t produced tools that unequivocally work in the real world. This highlights the field’s struggles with replicability and applicability."
(25:31)

Optimism and the Future of Psychology

Despite the challenges, Mastroianni remains hopeful that psychology is still in its infancy and that continued critical examination can lead to meaningful progress.

Mastroianni:
"We are just getting started in psychology. If we remain open to paradigm shifts and critical self-reflection, the field can evolve and become more robust."
(37:14)

Scientific Fraud in Business School Psychology

Host: Derek Thompson
Guest: Dan Engber

Francesca Gino's Scandal

Journalist Dan Engber recounts the scandal involving Francesca Gino, a renowned Harvard Business School professor, accused of scientific fraud across multiple studies. The Data Colada group first identified potential fraud in four of her papers, leading to an internal investigation by Harvard.

Engber:
"Francesca Gino was a superstar at Harvard Business School, focusing on dishonesty and creativity. In summer 2023, Data Colada highlighted fraud in four of her papers, prompting Harvard to place her on administrative leave."
(46:08)

The Complexity of the Fraud Turducken

Engber describes the multifaceted nature of the fraud, likening it to a "turducken" — a turkey stuffed with a duck stuffed with a chicken — to illustrate the layers of deceit involving multiple researchers, including Dan Ariely.

Engber:
"The scandal is so layered that one paper contained fraudulent data attributed to both Gino and Dan Ariely, creating an unprecedented situation in academic psychology."
(47:14)

The Many Co-Authors Project

In response to the fraud accusations, Juliana Schroeder, a respected business school psychologist, initiated the Many Co-Authors Project to audit the extensive body of work co-authored with Gino. This project aimed to assess the integrity of not just the four flagged papers but all 138 co-authored works.

Engber:
"Juliana Schroeder took a proactive stance, auditing every paper she co-authored with Gino to determine the prevalence of data issues. This initiative sought to uncover whether the fraud was isolated or indicative of a broader problem in the field."
(55:41)

Discovery of Additional Issues

During her audit, Schroeder identified major data problems in a 2018 study she co-authored, which initially appeared to demonstrate that ritualistic behavior could aid in weight loss. Upon correcting data misattributions, the study's positive findings evaporated, suggesting deliberate data manipulation.

Engber:
"In a 2018 study on rituals and weight loss, Schroeder found that flipping certain data points nullified the original findings. The improbability of such a coding error occurring by chance pointed towards intentional fraud."
(62:03)

Schroeder's Confrontation and Denials

Engber details his tense conversation with Schroeder, where he questions whether she might have been complicit in data tampering. Schroeder denies altering data personally, suggesting instead that research assistants may have made inadvertent errors. However, the specificity and improbability of these errors leave the situation unresolved.

Engber:
"When confronted, Schroeder calmly denied any personal involvement in data tampering, attributing the errors to research assistants. However, the nature of the mistakes makes this explanation highly questionable."
(76:06)

Systemic Issues and Ethical Dilemmas

The scandal underscores systemic problems within business school psychology, including questionable research practices (QRPs) like p-hacking and the high-stakes environment that pressures researchers to produce publishable, high-impact findings. Engber argues that these issues create an environment where both intentional fraud and inadvertent errors can flourish.

Engber:
"The pressure to publish sensational findings leads to ethical compromises, whether through QRPs or outright fraud. The current system lacks effective safeguards, making it difficult to maintain research integrity."
(87:36)

Potential for Reform and Future Directions

Despite the grim portrayal, Engber offers a glimmer of hope by suggesting that increased transparency and technological solutions, such as tracking data provenance, could mitigate fraud. However, he cautions that implementing such reforms involves significant costs and logistical challenges, particularly for emerging researchers.

Engber:
"Technological interventions like data provenance tracking on platforms like OSF could help, but they come with high costs and bureaucratic hurdles. Reform requires balancing the need for integrity with practical feasibility."
(100:36)

Conclusion: Reflecting on the State of Psychology

Thompson wraps up the episode by reflecting on the inherent difficulties in applying psychological findings to real-world scenarios, contrasting it with the tangible and testable nature of hard sciences like physics and chemistry. He posits that the absence of hard feedback mechanisms in psychology allows flawed research to persist longer, highlighting the necessity for internal mechanisms, such as the Many Co-Authors Project, to enforce scientific rigor.

Thompson:
"Psychology struggles to self-correct in the way hard sciences do because the real-world implications aren't as immediately testable or observable. This makes the role of audits and internal reviews even more critical."
(103:02)

Engber:
"Falsification loops in psychology aren't as robust as in fields like chemistry, allowing fraudulent or flawed studies to have lasting impacts without immediate real-world repercussions."
(105:26)

Key Takeaways

1. Replication Crisis: A significant portion of psychological research, including well-known theories like power poses and ego depletion, fail to replicate, calling into question their validity.

2. Scientific Fraud: High-profile cases, such as Francesca Gino's, reveal systemic issues within business school psychology, highlighting the prevalence of both intentional fraud and questionable research practices.

3. Methodological Weaknesses: QRPs, including p-hacking and selective reporting, exacerbate the replication crisis, undermining the credibility of psychological research.

4. Technological Gaps: Unlike hard sciences, psychology lacks efficient mechanisms for real-time validation and technological applications, making it harder to self-correct flawed research.

5. Reform Necessity: Initiatives like the Many Co-Authors Project are essential for auditing and restoring integrity within the field, though they face significant practical and cultural barriers.

6. Future Outlook: While the challenges are daunting, increased transparency, technological interventions, and cultural shifts towards valuing methodological rigor over sensational findings offer pathways towards a more robust and reliable psychological science.

Notable Quotes

• Adam Mastroianni:
  "Most of the work that doesn't replicate didn't matter in the first place. We should ask whether it would matter if it were true, rather than just if it's true."
  (16:04)

• Dan Engber:
  "The fraud in science, especially in psychology, is widespread and often goes undetected until someone takes the time to investigate meticulously."
  (74:58)

• Dan Engber:
  "There is no room whatsoever for grace in dealing with fraud in science. The stakes are so high that any suspected wrongdoing leads to total career destruction."
  (84:58)

• Derek Thompson:
  "In psychology, the physical world doesn’t serve as a robust feedback mechanism to falsify theories, unlike in hard sciences like physics or chemistry."
  (103:02)

Conclusion

This episode of Plain English provides a critical exploration of the inherent challenges within psychological research, emphasizing the urgent need for systemic reforms to enhance credibility and applicability. By spotlighting both individual misconduct and broader methodological flaws, Thompson and his guests illuminate the path forward for psychology to evolve into a more reliable and scientifically rigorous discipline.