Uri Simonsohn @uri_sohn Twitter profile

Last Seen Profiles

@ScrumSportsNY

@reyuchuujin

@tuppyhoroko

@ChelsDingman

@esmaeilion

@TomBeaumont

@dustinlynch

@2001RJT

@zkutalik

@_jlevinson

@MichaelD1729

@AmbitionAcres

@Tnjoroge_

@hakancelenkk

@TorelloSports

@SomersetGrockle

@HayleyNJones

@yoshinorim15475

@FANCYUTD

@MinnowWalsh

@OwenandLouis

@VisualCrea2010

@itsfolknow

@radier7

@RebeccaNoecker

@Build_A_Bow

@DeclanPeer

@max_heaver

@OurCofELike

@Comeniusmuseum

@jcpcol

@gavingeorge88

@TheCaptain_03

@CoachJeremyCase

@6i3gCqLLfkjrNXs

@Aqeel_Glass4

Uri Simonsohn

@uri_sohn

3 years

Too many conspiracy theories, too few conspiracy predictions.

4

24

235

Uri Simonsohn

@uri_sohn

6 years

Authors say X Critics say "just regression to the mean" Authors say "if you are right, then this should happen" Authors run 3 studies. It does happen Authors say "turn out critics are right" @PsychScience

3

52

211

Uri Simonsohn

@uri_sohn

11 months

Post 2 in the series on Data Falsificada. Plus, a message to Gino's 148 co-authors (which include me)

6

44

221

Uri Simonsohn

@uri_sohn

5 years

After years of using it, the 'rio' library in R still feels like a superpower. Read "any" data format by just writing: import(data.txt) or import(data.dta) or import(data.sav) or import(data.xls) etc. Thanks @thosjleeper

2

28

197

Uri Simonsohn

@uri_sohn

2 years

If a book made you want to study economics (undergrad or PhD) . What book was it?

98

24

198

Uri Simonsohn

@uri_sohn

6 years

'Anyone who observed us would conclude the purpose of all academic discussions was to become further entrenched in our original positions' From Richard Russo's book "straight man"

2

20

142

Uri Simonsohn

@uri_sohn

7 years

They kept deleting Wikipedia links to Colada[37] on powerposing "not peer-reviewed" So we published it @jpsimmon

5

53

101

Uri Simonsohn

@uri_sohn

6 months

Psych Science's incoming editor Simine Vizire ( @siminevazire ) is shaking things up The new board includes 16 editors focusing just on methods ...

Journals

APS publishes six of the field’s most influential journals.

www.psychologicalscience.org

2

17

118

Uri Simonsohn

@uri_sohn

8 years

I hope we are as bad at forecasting how people govern as we are at who does

2

59

98

Uri Simonsohn

@uri_sohn

3 years

It's now easier/possible to read the post again. Long delays loading page (due to higher than usual traffic) have been resolved. (this happens every time we write about the Calibri vs Cambria culture war)

Data Colada

@DataColada

3 years

Colada98 - Evidence of Fraud in an Influential Field Experiment About Dishonesty

22

195

802

1

108

Uri Simonsohn

@uri_sohn

5 years

An easy to follow resolution for academics in 2019: use 'eternal' links in papers

[34] My Links Will Outlive You - Data Colada

If you are like me, from time to time your papers include links to online references. Because the internet changes so often, by the time readers follow those links, who knows if the cited content...

datacolada.org

1

51

103

Uri Simonsohn

@uri_sohn

10 months

Save some characters for your next tweet. Don't need to use "alleged"

4

20

106

Uri Simonsohn

@uri_sohn

3 years

Researchers were asked about pre-registration's cost & benefits. - Top benefit (79%) is an unintended one: 'clarify ideas before running study' - Only 8% think it curtails exploration Logg & Dorison (OBHDP 2021)

3

40

98

Uri Simonsohn

@uri_sohn

5 years

I wished there were a twitter button "unfollow for 3 months" Useful when people publish books

4

3

93

Uri Simonsohn

@uri_sohn

7 years

In a second round review it is almost never appropriate to raise new concerns. Stick to assessing "did they address previous concerns?"

4

15

76

Uri Simonsohn

@uri_sohn

6 years

Hi @PardisSabeti Could you please identify the most objectionable choice-of-words or arguments in our posts I share your concern w/lack of civility Don't think we are guilty Prove me wrong with quote

[66] Outliers: Evaluating A New P-Curve Of Power Poses - Data Colada

In a forthcoming Psych Science paper, Cuddy, Schultz, & Fosse, hereafter referred to as CSF, p-curved 55 power-posing studies (.pdf | SSRN), concluding that they contain evidential value [1]....

datacolada.org

3

14

72

Uri Simonsohn

@uri_sohn

3 years

Is there an easy way to find top-5 econ papers that actually post data? In many papers you follow the links only to find explanations of why the data are not posted.

5

3

79

Uri Simonsohn

@uri_sohn

7 years

If I could magically persuade everyone of only one idea I've had I would choose: "The Effect Size Does Not Exist"

[33] "The" Effect Size Does Not Exist - Data Colada

Consider the robust phenomenon of anchoring, where people’s numerical estimates are biased towards arbitrary starting points. What does it mean to say “the” effect size of anchoring? It surely...

datacolada.org

2

21

70

Uri Simonsohn

@uri_sohn

10 months

Seems people are misinterpreting this chart by "Rethinking" bookk author (hope he isn't) p-value distribution are *fine* for discrete tests, with only 2nd order issues. Below I contrast his chart with more diagnostic ones (see p-curve (2014); supplement #3 )

Richard McElreath 🦔

@rlmcelreath

10 months

Since I am talking about p-values today (only day this year I promise), the common claim that p-vals are uniform under the null is not in general true. Even in theory. Here is the dist of p-values under null for logistic regression, two groups:

27

70

575

4

13

76

Uri Simonsohn

@uri_sohn

5 years

Am reviewing for SJDM (the judgment decision-making conference; ). Most submissions include pre-registrations. If you are not pre-registering your experiments, you are unusual.

2

14

71

Uri Simonsohn

@uri_sohn

6 years

*Job announcement* I now live in Barcelona and work at ESADE, with appointments in 2 depts. 1) OIDS (Data Science) 2) Marketing Both are hiring tenure track researchers. Teaching is in English. These and other job listings:

3

22

63

Uri Simonsohn

@uri_sohn

11 months

Some vigilante from the self appointed data police downloaded the posted data, tried to reproduce our figure and found an error. Arrow was pointing at the observation in wrong condition for 'harvard' lower case. We have corrected it Very sorry.

2

1

62

Uri Simonsohn

@uri_sohn

8 years

Out in Psych Science. Debate with Schwarz & Clore on replications.

1

39

49

Uri Simonsohn

@uri_sohn

9 years

I admired Don Green before. Now I admire him more. http://t.co/GFINfDC8Lj http://t.co/w5MyaxwnnY

4

47

51

Uri Simonsohn

@uri_sohn

6 years

DataColada[66] p-curve, power posing, and closely reading papers with very low p-values (e.g., p<1 in a quadrillion)

[66] Outliers: Evaluating A New P-Curve Of Power Poses - Data Colada

In a forthcoming Psych Science paper, Cuddy, Schultz, & Fosse, hereafter referred to as CSF, p-curved 55 power-posing studies (.pdf | SSRN), concluding that they contain evidential value [1]....

datacolada.org

3

26

47

Uri Simonsohn

@uri_sohn

10 months

This is in Luxemburg's airport. Any ideas what it means?

8

2

54

Uri Simonsohn

@uri_sohn

6 years

Please tell @SSRN that if they continue requiring people to login to download papers, you will stop posting with them. Their email address: CommentsSuggestions @ssrn .com

4

15

49

Uri Simonsohn

@uri_sohn

7 years

Colada 63: Many Labs over-estimated importance of hidden moderators

[63] "Many Labs" Overestimated The Importance of Hidden Moderators - Data Colada

Are hidden moderators a thing? Do experiments intended to be identical lead to inexplicably different results? Back in 2014, the "Many Labs" project (htm) reported an ambitious attempt to answer...

datacolada.org

2

23

44

Uri Simonsohn

@uri_sohn

8 years

Trump voters are not all racist But they are all insufficiently anti-racist Our kids are learning the difference isn't consequential enough

1

12

41

Uri Simonsohn

@uri_sohn

5 years

Want to learn R? Just gave this advice to someone: 1) Read this paper: 2) Browse DataColada for posts that interest you, all got heavily commented code

2

16

45

Uri Simonsohn

@uri_sohn

4 years

Would you rather learn the vaccine is >90% effective based on the confidence interval or a sophisticated bayesian model that adds assumed structure and drug testers' beliefs?

10

2

50

Uri Simonsohn

@uri_sohn

11 months

@Devin_G_Pope I will be making a pitch to qualtrics along these lines. An easy solution circumventing data privacy (you often shouldnot post all data) would be MD5 values for columns and column combinations. If people have other ideas DM or email me

3

2

50

Uri Simonsohn

@uri_sohn

6 years

Number 6 will surprise you.

3

21

48

Uri Simonsohn

@uri_sohn

6 years

Can't decide which feature is dumber at Manuscript Central: 1. One password for each journal 2. They render a tiny sized .pdf of paper you have to evaluate (useless) 3. No "download paper" button

3

7

45

Uri Simonsohn

@uri_sohn

9 years

"Better P-curves" Forthcoming in JEPG More robust to fraud, ambitious p-hacking and misguided p-curvers. http://t.co/YtOTDzdXQ1

1

25

44

Uri Simonsohn

@uri_sohn

6 years

Pilot-dropping is possibly the most counterproductive form of p-hacking

[68] Pilot-Dropping Backfires (So Daryl Bem Probably Did Not Do It) - Data Colada

Uli Schimmack recently identified an interesting pattern in the data from Daryl Bem’s infamous “Feeling the Future” JPSP paper, in which he reported evidence for the existence of extrasensory...

datacolada.org

4

19

44

Uri Simonsohn

@uri_sohn

4 years

Tomorrow at Data Colada Seminar: Elisabeth Bik She will tell us about looking for fraud across 20,000 Biomedical research papers. That's our 1st season finale. Second seasons starts September 11th.

1

11

41

Uri Simonsohn

@uri_sohn

4 years

Why doesn't R include version into package names? That would dramatically enhance reproducibility. library(tidyverse) ---> library(tidyverse.1.1.0) They are, after all, actually different packages.

4

39

Uri Simonsohn

@uri_sohn

5 years

re: how to share slides one at a time in real time I found Just tested. Seems very good. Addon for PowerPoint Students get URL, see slides only once all animations are done. They can go back but not forward. With: Polling & feedback @SlideLizard

Success through Presentations

SlideLizard is a PowerPoint based platform for creating, conducting and analyzing your presentations. It includes a company wide collaborative slide library, live Audience Interaction with polls, Q&A...

slidelizard.com

1

4

41

Uri Simonsohn

@uri_sohn

7 years

Go ahead, use Fixed Effect meta-analysis (says creator of I2 index) Unusually conceptually rich & clear stats paper

2

24

42

Uri Simonsohn

@uri_sohn

9 years

P-curve and Effect Size paper is out. Demonstrations include choice overload, Many Labs Replication. http://t.co/8Q8DlcaEae

2

41

43

Uri Simonsohn

@uri_sohn

3 years

@MaartenvSmeden And it's worse. Weighted average gives the hope of 'noise cancelling', but it exacerbates bias. (paper's pdf: )

0

8

39

Uri Simonsohn

@uri_sohn

5 years

"If you think p-values are problematic, wait till you understand Bayes factors"

5

10

40

Uri Simonsohn

@uri_sohn

3 years

Another older post relevant these days Colada[29] "Help! Someone thinks I p-hacked" Spoiler: "my analysis is justifiable" is the wrong answer.

[29] Help! Someone Thinks I p-hacked - Data Colada

It has become more common to publicly speculate, upon noticing a paper with unusual analyses, that a reported finding was obtained via p-hacking. This post discusses how authors can persuasively...

datacolada.org

2

8

39

Uri Simonsohn

@uri_sohn

7 years

You should subscribe to DataColada. We deliver on both

1

0

35

Uri Simonsohn

@uri_sohn

1 year

@jbakcoleman The probability that the lowest of 4 significant p-values is as high as .029 is LOW (for true effects) With 33% power the probability is 0.5%. With 50% power it's effectively 0%. Using math to evaluate papers seems reasonable to me. What's the argument for putting math to rest?

2

5

44

Uri Simonsohn

@uri_sohn

4 years

Looking for a #rstats #R programmer for one-time gig writing package to improve reproducibility of academic research done in R. If interested:

4

44

35

Uri Simonsohn

@uri_sohn

7 years

Conservative argument against refugees:very low but non-0 risk of deadly attack-->ban refugees. If you apply it to guns ownership...

2

16

34

Uri Simonsohn

@uri_sohn

8 months

@lluaces @stephaniemlee It's intuitively compelling, but wrong.

1

3

40

Uri Simonsohn

@uri_sohn

2 years

I'm giving the Colada seminar tomorrow, 12 PM ET 'Interactiongate: Testing and Probing Interactions with Linear Models in the Real (Non-Linear) World is Scandalously Invalid' Panelists: Deborah Mayo, Anne Scheel, Pam Davis-Kean & Gary McClelland

2

11

39

Uri Simonsohn

@uri_sohn

4 years

Feature request for R Studio Button that turns "\" into "/" in selected text "c:\temp\example\" --> "c:/temp/example/"

7

1

38

Uri Simonsohn

@uri_sohn

4 years

@EJWagenmakers @stephensenn @f2harrell Difference scores can (greatly) reduce power

[39] Power Naps: When do Within-Subject Comparisons Help vs Hurt (yes, hurt) Power? - Data Colada

A recent Science-paper (.html) used a total sample size of N=40 to arrive at the conclusion that implicit racial and gender stereotypes can be reduced while napping. N=40 is a small sample for a...

datacolada.org

0

7

37

Uri Simonsohn

@uri_sohn

10 months

I am talking to some people with some power about reducing fraud this coming days. Are there examples of other disciplines asking for data receipts? Name of person who ran the study, exact time/location, unique code for survey on server, etc? DM is ok, public reply is OK.

3

13

38

Uri Simonsohn

@uri_sohn

6 months

Figure 1 of 3 in

2

15

38

Uri Simonsohn

@uri_sohn

3 years

New participant privacy feature at Datasets are scanned (variable names and cell contents) for identity disclosures. Flagging: prolific id, mturk id, ip address, email address, latitude & longitude. Thanks @maltoesermalte for the idea.

2

10

33

Uri Simonsohn

@uri_sohn

6 months

The Many Co-Authors platform went live yesterday. One main goal is to help junior co-authors credibly signal who was involved in data collection in their studies. Added today: figures summarizing the data.

0

5

36

Uri Simonsohn

@uri_sohn

2 years

Looking for good articles giving guidance on how to run simulations for behavioral science. Suggestions? (self-nominations encouraged)

8

34

Uri Simonsohn

@uri_sohn

4 years

Has the contagion rate of workers at supermarkets been examined? Seems easier to compute accurately than aggregate contagion rates, and more relevant for policy decisions. Could examine covariates to guide policy further.

1

31

Uri Simonsohn

@uri_sohn

7 years

We should deal more often with referee concerns by just sharing concerns with readers."Referee 2 thought x about this study, but we think y"

2

4

33

Uri Simonsohn

@uri_sohn

4 years

Unreasonably proud of this.

1

0

32

Uri Simonsohn

@uri_sohn

7 years

First tweet by Goldman Sachs CEO

Lloyd Blankfein

@lloydblankfein

7 years

Today's decision is a setback for the environment and for the U.S.'s leadership position in the world. #ParisAgreement

2K

9K

24K

0

16

28

Uri Simonsohn

@uri_sohn

10 months

@alexoimas @CFCamerer Why do kahneman and tversky count as economists?

4

1

33

Uri Simonsohn

@uri_sohn

6 years

Beating MTurks bots sounds easy, they are probably dumb bots: 1) Captcha (can be home-made: draw 1 of 30 random images of numbers you made) 2) Require typing some fact: "Capital of Chile?", "7 plus fourteen"

10

6

30

Uri Simonsohn

@uri_sohn

8 years

Agree with @lakens on Meehl lectures. I turned them into MP3 files:

Daniël Lakens

@lakens

8 years

@briandavidearp @deevybee Have you watched this lecture series by Meehl? Might be the best thing on the internet.

4

13

32

1

11

28

Uri Simonsohn

@uri_sohn

3 years

An investigative journalism follow-up to the Colada Post on Fraud in a Dishonesty Field Experiment

0

12

31

Uri Simonsohn

@uri_sohn

2 years

Mastodon is kind of spectacularly unintuitive. Has the feel of a college programming homework abandoned in 2008

4

2

30

Uri Simonsohn

@uri_sohn

1 year

@leeflang_m I often use this one: p-hacked antidepressant study to treat bipolar disorder in teens, later tied to higher suicidality, with outcome switching of primary and secondary outcomes

1

4

31

Uri Simonsohn

@uri_sohn

1 year

Was reminded today of my favorite quote by Danny Kahneman: "Nothing is as important as you think it is, when you are thinking about it"

1

6

30

Uri Simonsohn

@uri_sohn

5 years

BDRM 2020 - Barcelona - June 16th-18th (behavioral decision research and management conference)

0

13

27

Uri Simonsohn

@uri_sohn

5 years

I think R Markdown is not for me. Worry it induces a "report writing" mindset, too focused on code run, unfocused on thinking critically and writing persuasively. Also worry collaborative writing is harder. *If you once shard concerns* and now use R Markdown please reply

19

4

26

Uri Simonsohn

@uri_sohn

5 years

It is too hard to get papers retracted. Evidence from Criminology.

2

6

28

Uri Simonsohn

@uri_sohn

8 years

When writing critiques of others' work,pretend their parents are coming for dinner.When reading of yours, pretend it's someone else's work

1

16

26

Uri Simonsohn

@uri_sohn

4 years

Maybe this is obvious. But, instead of researchers doing this: 'Analysis -version17-final-submitted-revised-final.R' They can just do this: 'Analysis 2020 11 11.R'

8

1

26

Uri Simonsohn

@uri_sohn

9 years

P-Curve app 3.0: Power estimate | robustness to dropping extreme results | detailed explanations | R Code | and more. http://t.co/QlVJlbibRw

0

32

27

Uri Simonsohn

@uri_sohn

6 months

@paulnovosad It's not possible to catch fraud in confirmatory way. Protections against file-drawer you mention: 1)Very low p-values (impossibly big file drawers ) 2)Multiple flags for same suspected fraudster Here p < 1/10k. But lower actually as it assumes no single valid obs was dropped.

1

2

29

Uri Simonsohn

@uri_sohn

8 months

@MeganTStevenson This is addressed in e.g., all the observations we flagged were among the 11 that the forensic firm documented mismatched the RA's data (which we did not have). The probability of getting that right by chance is 1 in a billion. Rules out cherry picking.

[114] Exhibits 3, 4, and 5 - Data Colada

We recently presented evidence of data tampering in four retracted papers co-authored by Harvard Business School professor Francesca Gino. She is now suing the three of us (and Harvard University)....

datacolada.org

1

29

Uri Simonsohn

@uri_sohn

5 months

Authors writing "To the best of our knowledge, we are the first to..." --> please include a footnote showing us how hard you tried to find earlier work.

0

4

32

Uri Simonsohn

@uri_sohn

4 years

Is it plausible/probable that if you download someone's R Code that uses packages. And you download all the newest versions of those packages. The code won't run due to lack of backwards compatibility?

14

4

28

Uri Simonsohn

@uri_sohn

8 years

Browsing for new running shoes came across this.

0

2

24

Uri Simonsohn

@uri_sohn

7 years

Colada[57]:10th most cited econ paper this century "sign of interaction in logit is wrong" Why?When to ignore that?

[57] Interactions in Logit Regressions: Why Positive May Mean Negative - Data Colada

Of all economics papers published this century, the 10th most cited appeared in Economics Letters , a journal with an impact factor of 0.5. It makes an inconvenient and counterintuitive point: the...

datacolada.org

1

10

23

Uri Simonsohn

@uri_sohn

9 years

Not following the 99% invisible podcast is simply a mistake. Highest mean and lowest variance quality podcast. http://t.co/m8gG7F3lBw

1

13

25

Uri Simonsohn

@uri_sohn

5 years

You can WhatsApp yourself by creating a group with 1 other person and deleting them. Super handy.

4

2

26

Uri Simonsohn

@uri_sohn

3 years

@OlegUrminsky @hardsci Just the tone used by Jason, and his confidence in other's intentions which he obviously has no info on, (bonus: he incorrectly states Tversky won the Nobel prize), suggests this is one of those tweets justifying the Mute rather than Retweet button.

1

24

Uri Simonsohn

@uri_sohn

5 years

I don't think out-of-office replies are needed this time of year

5

1

22

Uri Simonsohn

@uri_sohn

11 months

"On my left is Duncan Luce and on my right is Amos Tversky" Promising opening line for lecture series on measurement from 1981, amazingly on youtube.

Lecture 1 - An Introduction to Measurement with Patrick Suppes, R....

Lecture 1 - An Introduction to Measurement with Patrick Suppes, R. Duncan Luce, and Amos TverskyThis video is part of a lecture series on measurement from 19...

www.youtube.com

1

3

25

Uri Simonsohn

@uri_sohn

8 years

Being open to alternative views is simply not being a fundamentalist. Being open to one's view being wrong, that's open minded.

0

12

20

Uri Simonsohn

@uri_sohn

7 years

Cheatsheet for journos deciding whether to cover an interesting psych experiment 1) Ask "Can I see your pre-registration?" <end of message>

0

4

24

Uri Simonsohn

@uri_sohn

9 years

replication crisis... more like methods renaissance

0

17

21

Uri Simonsohn

@uri_sohn

8 years

- Pre-Registration made easy. In 7 months: 400 authors 347 pre-registrations 89 already public

2

15

21

Uri Simonsohn

@uri_sohn

8 years

First 4 months for AsPredicted 149 Authors 121 Pre-registrations 35 Already public Now with shorter URLs e.g.,

0

12

23

Uri Simonsohn

@uri_sohn

4 years

I think its' better to save data used in R in raw form (say .csv) not as part of, or in a stand alone, .RData. Please explain if you disagree.

12

0

21

Uri Simonsohn

@uri_sohn

6 years

Looking for discussions of how statisticians/methodologists "p-hack", making their simulations or new methods look better than they are.

10

12

22

Uri Simonsohn

@uri_sohn

6 years

To those interested in the Elsevier boycott. These are the psych journals affected: cc: @EikoFried

1

9

22

Uri Simonsohn

@uri_sohn

6 years

@lakens The poll, though, is on twitter

4

0

23

Uri Simonsohn

@uri_sohn

2 years

@MaartenvSmeden I have used this in class and some students have (unintentionally?) used the term 'Jelly-picking results' (instead of cherry-picking)

1

0

23

Uri Simonsohn

@uri_sohn

8 years

Power posing lacks evidential value. Direct replications not expected to succeed. In press at Psych Science

5

31

20

Uri Simonsohn

@uri_sohn

4 years

We are trying to fix R scripts not running because 'packages' change. For this, we want to learn about variation in installed versions. If you are a researcher, please run 4 lines of R code & share your list anonymously with us.

2

13

21

Uri Simonsohn

@uri_sohn

5 years

@lakens You were supposed to find another study on the same topic with a big (if flawed) effect, average it, and call the whole thing meta analytically true.

0

1

18

Uri Simonsohn

@uri_sohn

6 years

I've noticed that STATA forums are angrier than R forums Could be selection (economists) Could be STATA has more predictable syntax ("just google x" more often actually an option)

3

2

20

Uri Simonsohn

@uri_sohn

5 years

The problem with Elsevier 1. Journals' value = reputation 2. Publishers didn't create it 3. But do own it 4. Sell it too expensively 5. Trans.+coordination costs prevent true value creators reclaiming it Seems Rationally Speaking missed it. cc: @ceptional

2

20