last update: 2024 09 25
Professor of Behavioral Science
ESADE Business School, Barcelona-Spain
email: urisohn@gmail.com
twitter & blueSky: @uri_sohn
Elsewhere online
-
- groundhog the simplest to use solution for version-control of R packages
- AsPredicted the simplest to use pre-registration site in social science.
- ResearchBox the simplest to use plattform for sharing pre-registrations, data, code, and materials.
- DataColada, a blog I co-host on thinking about evidence and vice versa.
- GRAN, Microsoft used to backup all R packages but stopped in 2023; I took over
- p-curve.com, an app that we created that assesses if a set of studies have evidential value.
- SHFT-CTRL-V. Copy “c:\dropbox\temp” paste “c:/dropbox/temp”
- I explain why I use a treadmill desk (.htm).
Editorial work
– Management Science: Associate Editor (2011-present)
– Psychological Science: incoming “STAR” editor (new team on stats, transparency & rigor)
– Ad hoc reviewing: too much
Ongoing work
48) Banki & Simonsohn “Mega-analysis: Dealing with multiple comparisons in Mega-Studies (and beyond)”
Working Papers
47) Simonsohn, Montealegre, & Evangelidis “The Wisdom of Plots: Stimulus Plots Tell You Whether Average Effects Are Interpretable, Evidence From Wisdom of Crowds Experiments” (.pdf)
46) Montealegre & Simonsohn, “GAMify Spotlight & Floodlight: A More Robust and Informative Approach to Probing Interactions” (.pdf)
Four examples from recent marketing papers where linear simple slopes (spotlight) or linear Johnson-Neyman (floodlight) get it wrong, and how GAM gets it right.
44) Simonsohn, Montealegre, & Evangelidis “Stimulus Sampling Reimagined: Designing Experiments with Mix-and-Match, Analyzing Results with Stimulus Plots ” (.pdf) [this is what I currently present when invited to give a talk]
Focusing on confound management, this paper proposes “Match-and-Mix” as a new (the first?) framework for choosing stimuli for psychology experiments, and “Stimuli Plots” a tool for analyzing data from those experiments. We argue against mixed-models (at least against using them for ‘generalizability’ purposes).
Publications
42) Simonsohn (2024) “Interacting With Curves: How to Validly Test and Probe Interactions in the Real (Nonlinear) World ” (.pdf), AMPPS.
A new toolbox to test interactions which is robust to nonlinearities. This article was previously titled “interactiongate”.
43) Simonsohn, Simmons, & Nelson (2022) “Above Averaging: Scientific Literature Reviews Should Be Selective, Design-Focused, and Evaluative” Nature Psychology Reviews (.pdf)
41) Simmons, Nelson, Simonsohn (2021) “Pre-Registration is a Game Changer: But, Like Random Assignment, It Is Neither Necessary Nor Sufficient For Credible Science”, Journal of Consumer Psychology (.pdf)
40) Simmons, Nelson, Simonsohn (2021) “Pre-Registration: Why and How” Journal of Consumer Psychology
37) Simonsohn, Simmons, Nelson (2020) “Specification Curve Analysis Descriptive and Inferential Statistics for all Plausible Specifications” (.pdf| STATA files .zip) Nature Human Behaviour
note: STATA files are heavily commented, useful for R users too.
Other researchers have developed packages for implementing specification curves in R, Python and STATA. See more.
39) Mislavsky, Dietvorst, & Simonsohn (2019) “The Minimum Mean Paradox: A Mechanical Explanation for Apparent Experiment Aversion” (.pdf) PNAS (letter), V116(48), p23882-23884
Explains a statistical artifact that arises when comparing means across conditions. Covered in DataColada[79].
36) Vosgerau, Simonsohn, Nelson, & Simmons (2019) ” 99% Impossible: A Valid, or Falsifiable, Internal Meta-Analysis” Journal of Experimental Psychology: General V148(9) (.pdf | SSRN )
Internal meta-analysis is only valid if all studies were pre-registered, and the study inclusion rule was set before any studies were run. DataColada[73] is a blogpost summary.
35) Mislavsky, Dietvorst, & Simonsohn (in press; still) “Critical Condition: People Don’t Dislike A Corporate Experiment More than They Dislike Its Worst Condition” Marketing Science (.pdf)
Why do people hate corporate experiments? They don’t.
34) Dietvorst & Simonsohn (2019) “Intentionally Biased: People Purposely Use To-Be-Ignored Information, But Can Be Persuaded Not To” (.pdf) Journal of Experimental Psychology: General, 148(7), p.1228-1238. doi: 10.1037/xge0000541
In hindsight-bias, curse-of-knowledge, and mock-jury-trial paradigms, participants disagree they should ignore the information paying attention to it on purpose, believing they will be more accurate.
33) Simonsohn (2018) “Two Lines: A Valid Alternative to the Invalid Testing of U-Shaped Relationships with Quadratic Regressions” (.pdf) – APP to use test, Advances in Methods and Practices in Psychological Science (AMPPS), V1(4), 538-555 doi: 10.1177/2515245918805755
Everyone uses quadratic regressions to test for u-shapes. That approach is invalid. This paper introduces an easy alternative. Colada[62] is a blogpost summary.
32) Mislavsky & Simonsohn (2018) “When Risk is Weird: Unexplained Transaction Features Lower Valuations” Management Science (.pdf)
We define weird transactions as those having unexplained features, and show such weirdness accounts for effects previously attributed to risk.
31) Simonsohn, Nelson, Simmons (in press for a few years now(!) ) “P-Curve Won’t Do Your Laundry, But it Will Distinguish Replicable from non-Replicable Findings in Observational Research: Comment on Bruns & Ioannidis (2016)” PLOS ONE (SSRN)
Bruns & Ioannidis point out that p-curve cannot distinguish correlation from causation; duh!
30) Nelson, Simmons, & Simonsohn (2018) “Psychology’s Renaissance” Annual Review of Psychology, V69, p.511-534 (.pdf)
Psychology got a lot more credible in the last 7 years, we review how and why.
29) Simmons, Nelson, & Simonsohn (2018) “False-Positive Citations” Perspectives on Psychological Science, V13(2) p.255-259 (.pdf)
Invited piece telling story of “False-Positive Psychology.” We also discuss what we’d done differently, and analyze the citations it has gotten. (Special issue on top-10 most cited APS articles)
28) Munafo et al (2017) “A Manifesto for Reproducible Science” Nature: Human Behavior, V1(21) (.htm)
Review of multiple approaches to making more replicable research. There are boxes through the article summarizing it all; you can just read those.
27) Simmons & Simonsohn (2017) “Power Posing: P-Curving the Evidence” Psychological Science , V28(5) 687-693 (.pdf)
Carney, Cuddy & Yap (2016, .pdf) review 33 published studies to identify moderators that may explain a failure to replicate the original power posing study. But, the 33 studies lack evidential value.
26) Simonsohn (2016) “Each Reader Decides if a Replication Counts; Reply to Schwarz and Clore (2016)” Psychological Science (pdf), V27(10), 1410-1412
Schwartz & Clore (.htm) argue the failure to replicate their sunny–>happy finding doesn’t count. I discuss how to think about which replications count
25) Simonsohn, Simmons, Nelson (2015) “Better p-curves” Journal of Experimental Psychology: General (pdf)
We modified p-curve to be more robust to fraud, honest errors, and p-curvers mistakes
24) Simonsohn (2015) “Small Telescopes: Detectability and the Evaluation of Replication Results” Psychological Science V26(5) p.559-569 (.pdf | Supplement | R Programs)
A new approach to evaluating replications that combines hypothesis testing with effect size estimation
23) Simonsohn, Simmons, Nelson (2014) “P-Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results” Perspectives on Psychological Science V9(6) p.666-681 (.pdf | p-curve.com)
How to estimate effect size, if you must.
22) Silberzahn, Simonsohn, Uhlmann (2014) “Matched-Names Analysis Reveals No Evidence of Name-Meaning Effects: A Collaborative Commentary on Silberzahn and Uhlmann (2013), Psychological Science V25(7), p.1504-1505 (.pdf) (data)
Collaborating with original authors we show Herr Kayser is not disproportionately likely to work as a manager after all.
21) Simonsohn, Nelson, Simmons, (2014) “P-curve: A Key to the File Drawer,” Journal of Experimental Psychology: General, V143(2), p.534-547 (.pdf | Supplement | p-curve.com)
How to analyze the distribution of significant p-values for set of findings to undo impact of selective reporting, of both studies and analyses, on hypothesis testing.
In other words: p-curve helps tell replicable findings apart.
20) Simonsohn (2013) “It Really Just Does Not Follow, Comments on Francis (2013)”, invited commentary for the Journal of Mathematical Psychology, V57(5) p.174-176 (.pdf)
Francis misuses and misinterprets the publication-bias test. Lesson: tool developers should anticipate misuse and take safeguards to prevent it.
19) Simonsohn (2013) “Just Post it: The Lesson from Two Cases of Fabricated Data Detected by Statistics Alone,” Psychological Science, V24(10), p.1875-1888, (pdf| Data & code )
The analysis of Sanna’s and Smeesters’ raw data show they are fake. On top of many other advantages, posting raw data will reduce academic fraud.
18) Simonsohn, Gino (2013) “Daily Horizons: Evidence of Narrow Bracketing in Judgment from 10 years of MBA-admission Interviews”, Psychological Science, V24(2), 219-224 (.pdf)
Interviewers avoid giving too many high/low scores on the same day.
17) Nelson, Simmons, Simonsohn (2012) “Let’s Publish Fewer Papers,” Psychological Inquiry, V23(3), 291-293 (.pdf)
16) Simonsohn (2012) “It Does Not Follow: Evaluating the One-Off Publication Bias Critiques by Francis (2012a,b,c,d,e,f), Perspectives on Psychological Science, V7(6), 597-599 (.pdf)
The critiques are cherry picked, and ignoring evidence is not a justified conclusion from the presence of publication bias.
15) Simmons, Nelson, Simonsohn (2011) “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allow Presenting Anything as Significant”, Psychological Science, V22(11), 1359-1366 (pdf) (ResearchBox page)
Logical argument, experimental demonstration, and simulations showing that if a set of disclosure requirements we propose are not followed, results in experiments are uninterpretable.
14) Saiz & Simonsohn (2013) “Proxying for Unobservable Variables with Internet Document Frequency” Journal of the European Economic Association V11(1) 137-165 (.pdf) (supplement) (Data)
Frequency of Internet documents about X proxies for frequency of X; using insight we replicate published studies predicting corruption.
13) Simonsohn (2011) “Spurious Also? Name Similarity Effects (Implicit Egotism) in Employer Decisions”, Psychological Science, V22(8) 1087-1089 (.pdf)
People disproportionately work for companies with which they share an initial. Probably a spurious correlation.
12) Simonsohn (2011) “Spurious? Name Similarity Effects (Implicit Egotism) in Marriage, Job, and Moving Decisions”, Journal of Personality and Social Psychology, V101(1) 1-24 (.pdf)
Three JPSP papers find that people disproportionately choose spouses, places to live and occupations with names similar to their own. Analyzing the same and additional data I find that the existing evidence is spurious.
note: Pelham and Carvallo wrote a rebuttal to this paper. Here is my 5-page rejoinder titled “In Defense of Diligence”.
11) Pope, Simonsohn (2011) “Round Numbers as Goals: Evidence from Baseball, SAT Takers, and the Lab”, Psychological Science, January, V22(1), 71-79 (.pdf)
When performance is measured numerically, round numbers become implicit goals that strongly influence behavior around them.
10) Simonsohn (2011) “Lessons from an Oops at Consumer Reports: Consumer Follow Experts; Ignore Invalid Information”, Journal of Marketing Research, February V48(1) 1-12 (.pdf)
Consumer Reports released & then retracted info on carseat safety. Surprisingly, people successfully ignored the retracted information.
9) Simonsohn, (2010) “eBay’s Crowded Evenings: Competition Neglect in Market Entry Decisions”, Management Science, V56(7), 1060-1073 (.pdf)
Too many sellers end their auctions at peak time, so they lose money.
8) Simonsohn, (2010) “Weather to Go to College”, Economic Journal (.pdf)
More prospective college students enroll after visiting campus on cloudy day.
7) Simonsohn, (2009) “Direct-Risk-Aversion: Evidence from Risky Prospects Valued Below Their Worst Outcome” Psychological Science, V20(6) 686-692 (.pdf)
People value lotteries less than their worst outcome due to uncertainty; not confusion or “joint-evaluation.” BUT: actually, with Rob Mislavsky we figured out it was not about uncertainty, see paper #32 above.
6) Small & Simonsohn (2008) “Friends of Victims: Personal Experience and Prosocial Behavior.” Journal of Consumer Research, V35 532-542 (.pdf) [raw data]
Donors give more to charities helping the misfortune of someone they know.
5) Simonsohn, & Ariely (2008) “When Rational Sellers Face Non-Rational Consumers: Evidence from Herding on eBay,” Management Science V54(9) 1624-1637 (.pdf)
eBay bidders choose auctions with more bids, so sellers start them cheap.
4) Simonsohn, Karlsson, Loewenstein, and Ariely (2008) “The Tree of Experience in the Forest of Information: Overweighing Experienced Relative to Observed Information” Games and Economic Behavior, V62, 263-286 (.pdf)
People respond more to information that affected them directly.
3) Simonsohn, (2007) “Clouds Make Nerds Look Good: Field Evidence of the Influence of Incidental Factors on Decision Making”, Journal of Behavioral Decision Making, V20(2) 143-152 (.pdf)
College applicants’ academic attributes are weighted more if evaluated on cloudy days.
2) Simonsohn & Loewenstein (2006) “Mistake #37: The Impact of Previously Faced Prices on Housing Demand,” Economic Journal, V116(1) 175-199 (.pdf)
Movers from more expensive cities rent more expensive apartments, at first.
1) Simonsohn (2006) “New-Yorkers Commute More Everywhere: Contrast Effects in the Field,” Review of Economics and Statistics, V88(1) 1-9 (.pdf)
Movers from cities with longer commutes live further from work, at first.
Inactive working papers
Simonsohn “Posterior-Hacking: Selective Reporting Invalidates Bayesian Results Also” (SSRN)
p-hacking misleads Bayesians as much as it misleads other mortals.
Simonsohn, Simmons, Nelson “Anchoring is Not a False-Positive: Maniadis, Tufano and List (2014) ‘Failure-to-Replicate’ is Actually Entirely Consistent with the Original” (SSRN)
The anchoring replication is not significantly different from 0, but also not significantly different from large. The AER rejected the paper combining two positive reviews with an anonymous letter from an unsolicited 3rd reviewer. That letter was not shared with us.
Lewis, Rees-Jones, Simonsohn, & Simmons, “Diminishing Sensitivity to Outcomes: What Prospect Theory Gets Wrong about Diminishing Sensitivity to Price” (.pdf)