Data from
@OpenTable
on restaurant demand in Seattle.
Stunning trend break the day the first
#covid_19
death was announced.
We held steady around -30% for about a week.
It's all downhill since.
Much respect for
@DLeonhardt
's journalism, but this thread is going on my quant syllabus as an example of how to mislead (not lie! but mislead) with statistics. SAT/ACT are way less of a deal than is implied here.
Strap in, because we're going into the weeds on this thread...
Since Covid, many colleges have dropped their SAT and ACT requirements. But now that decision is creating problems – because the tests contain real information about students’ likelihood of doing well in college.
The evidence is extensive and growing. 🧵
In light of the recent suicides of Alan Krueger and Martin Weitzman, economists & other professionals at risk of aging might find insight in this recent letter penned by Princeton professor emeritus Avinash Dixit.
Thread follows.
@DLeonhardt
@OppInsights
And sure, the SAT can occasionally promote equity. But if it’s an equity agenda you want to pursue, standardized tests can’t hold a candle to:
1) Ending legacy admissions
2) Ending donor child admissions
3) Ending most athlete recruitment
“When you don’t have test scores, the students who suffer most are those with high grades at relatively unknown high schools, the kind that rarely send kids to the Ivy League,” says
@ProfDavidDeming
, a Harvard economist. “The SAT is their lifeline.”
For the small number of people who are not totally sick of
#minimumwage
threads, let me walk you through some of the
@UW
team's findings for
#Seattle
now that they've cleared peer review (conditionally accepted, AEJ:EP).
A lot of them are packed into this one picture.
Here's some data on 2 PhD programs
@DukeU
.
Roughly equal in size.
@usnews
rankings and yield rates (% of admitted students who accept the offer) suggest the bio program is more prestigious.
The econ program receives nearly 10x more applications.
The Econ Bottleneck: a 🧵
What's the largest,
fastest-growing,
most diverse,
least insular,
and (arguably) best governed
social science of them all?
Psychology.
#EconTwitter
, let's review the hard data and then look at some stunning differences when it comes to leadership.
Economics has become the laughingstock of academic publication. Common question asked by scholars in almost any other field: "It takes you *how long* to get a paper published?"
How do we fix it?
Some suggestions...
The attention to seminar culture in econ brought to mind wise words that
@LizAnanat
used to say at the beginning of student workshops
@DukeSanford
.
"For the next 90 minutes, this is all of our paper"
As an audience member you have one job: to help improve the paper.
@DLeonhardt
You might look at this graph and think “those data points are awfully close.” But they aren’t the real data points. They're binned means.
Given information in the
@OppInsights
technical appendix, the actual relationship looks more like the second image here (r=0.44).
To appreciate
@nellgluckman
’s article about
@OppInsights
and elitism, it’s helpful to understand an important feature of the econ ecosystem:
There’s incredibly little 💰💰 in it, and a large share of what’s there is doled out by oligarchs. A 🧵...
Hello everyone arguing about the
#minimumwage
! Someday I'll post a thread about the evidence our
@UW
team compiled in Seattle.
Today let's talk about hours.
Raise the wage on most low-paying jobs & workers still don't have enough to live on. Because they can't get enough hours.
@DLeonhardt
@OppInsights
Where did the applicant go to HS?
This information, by itself, has twice the predictive power of SAT scores. Students from “elite” high schools get better grades than lesser-resourced ones, controlling for all the other student characteristics in the earlier analysis.
Six weeks ago, I invited the 9 candidates for AEA leadership positions to offer statements on pressing issues in the economics profession, including diversity/inclusion & the publication process.
Response rate, as of today: zero.
There is, of course, an economic explanation...
"Brilliant answers to irrelevant questions: a game for anti-social social scientists"
Is that what economists are playing now?
An epistemological concern with roots as old as the discipline itself.
And our thread for today.
Here's a sentence you *won't* read in
@ProfDavidDeming
's pro-testing
@TheAtlantic
essay:
"After making the SAT optional, the number of disadvantaged students attending Ivy League schools declined."
For a good reason: it isn't true. Data in the 🧵...
Rank-and-file economists don't know it yet, but the next president-elect of the American Economic Association has been selected.
Or at least the only individual whose name will be printed on the ballot has been selected.
Here's the story,
#econtwitter
...
@DLeonhardt
@OppInsights
There's a deeper question of whether elite colleges should lavish their ample resources on "sure things," the students who will do fine in college and life with or without those resources, or the students on the margin, the ones for whom resources could be transformational.
If you started a Ph.D. program in economics this fall, your chances of proceeding directly to an academic job (excluding postdocs, but including non-tenure track positions such as lecturers) is approximately...
13%
Is that a market failure? That's our thread topic for today.
Four years ago I sat poolside in San Diego and tweeted out this thread about the state of the AEA, as revealed at the annual business meeting. Things didn't look great then.
They look worse now. In fact, they prompt a question: have we passed "peak economics?" This year's 🧵...
Here,
#econtwitter
, is a photo (replete with San Diego Marriott poolside lounge chair backdrop) of the AEA budget for FY 2020. I picked it up at the sparsely attended business meeting at
#ASSA2020
.
Although printed in black, there's some serious red ink. Let's explore.
@DLeonhardt
@OppInsights
According to the
@OppInsights
analysis standardized test scores explain less than 1/5 of the variation in college GPA. So over 80% of the variation in college grades exists among students with similar test scores. Does that sound like strong prediction to you?
@DLeonhardt
@OppInsights
Good predictions are key to admissions strategy in either case, but if you think that colleges have just gotta have SAT scores to make decent predictions...
think twice.
/end.
.
@nberpubs
has meant a lot to me, but it is an organization built upon a foundation of exclusion.
A more inclusive NBER would run Summer Inst. like most organizations run conferences: open to all, with a common registration fee, rather than the current 2-tier (0/infinity) model.
@SandyDarity
Yup. For instance, I’d love to hear people with more career protection than me talk about how the in-group gets to workshop their papers in front of top journal editors at invitation-only Econ summer camps... at the Royal Sonesta... That’s how privilege begets privilege. Ijs
There are very good jobs outside of academia. I often advise students that the more important distinction for a job is whether it involves self-directed research.
But students do dream of academic jobs, and those dreams die sometimes. Mine nearly did.
My junior market story:
Fourcade, Ollion and Algan (2015) document the peculiar economic practice of restricting professional leadership to the top departments. Our leaders are nominated by a committee, themselves drawn predominantly from top departments. Our peer institutions behave very differently.
@DLeonhardt
@OppInsights
That question is completely irrelevant to our purpose. The real question is, “compared to a prediction that ignores SAT scores, how much better is a prediction that uses them?” And neither
@DLeonhardt
nor
@OppInsights
present the analysis required to address that question.
48 hours into a search for a $95k/year asst. prof. on a 3-year contract: "You can expect the dean and provost to sign off on the job ad within a month. Assuming it conforms to regulations."
48 hours into a search for a $7.75M/year coach on a 7-year contract: "OK here's our guy!"
The key to the future of economics does not lie in the offices of tenured faculty jockeying over departmental bragging rights.
It lies on the campus of a less-selective college or university somewhere, where today a promising young student is choosing what to study.
The AEJ:Micro annual editor's report prompts a philosophical question: if a journal has standards so high that it never accepts a manuscript, does it exist?
Certainly if all it manages to do is waste authors’ time (and submission fees), it shouldn’t.
@DLeonhardt
@OppInsights
It gets cut. A lot. This gets at the question “who the heck are those students with sub-1200 SATs at elite colleges anyway?” Well, it appears a lot of them are legacies, athletes, etc. We don’t need a test score to know that some of these students are lesser academic performers.
@DLeonhardt
@OppInsights
So where
@DLeonhardt
’s thread might lead you to believe elite colleges are forced to take shots in the dark when deprived of test scores, admitting students whom they have no idea about, in reality they retain the clear majority of their predictive power.
So competition is good, sayeth economists, except when that competition risks making a Great Economist feel as though he does not rank
#1
in the hearts or minds of lesser economists.
Here's a radical one: allow parallel submission. Journals currently demand monopoly power over a manuscript for as many years as it takes to render a decision. Require authors to disclose multiple submission & to withdraw from one journal as soon as another accepts it.
@DLeonhardt
@OppInsights
While
@OppInsights
never shows us the regression we need – the one that has everything BUT the SAT scores – we can infer that, on the margin, test scores are definitely improving predictive power. But the clear majority of potential predictive power comes from other factors.
Every year thousands of students fill out the FAFSA even though gov't already knows they qualify for aid--because their family receives means-tested support.
Today, Washington's legislature passed a bill to automatically qualify SNAP recipients for state financial aid!
#waleg
The presumption of innocence, subtly referenced in this new
@briq_institute
release, is morally defensible when the accuser (e.g., the state) is powerful and the accused powerless.
In harassment, however, the roles are reversed. Almost always, the accused is the powerful one.
Does your favorite American college practice legacy admissions?
Google [college name] Common Data Set. Find the answer to question C7.
Here is one data point:
@UW
. Which for the record does not consider legacy status.
My undergrad alma mater,
@Cornell
, does. It shouldn't.
Like many, I'm really impressed by
@florianederer
,
@paulgp
, & Jensen's forensic work to shed light on EJMR. There's no excuse for the pervasive toxicity on that site. The "what do we do about it" question is tough. Here's a suggestion.
#econtwitter
@DLeonhardt
Here David says “the relationship between test scores and college grades… is strong.” And golly what a steep line!
The problem is that the strength of the relationship isn’t measured by the steepness of the line, but how closely data points hew to it.
Consider this new data, from prof's at Dartmouth and Brown, based on data from elite colleges.... There’s only a modest relationship between high school grades and college grades. The relationship between test scores and college grades, though, is strong.
@DLeonhardt
@OppInsights
With these controls, the predictive power of the SAT is cut more than a third. The predictive power of HS GPA barely budges. Our explanatory power, measured by R-squared, is higher than it was. But it goes up a LOT more if we control for one more factor…
Junior economists chase an ever-more-elusive brass ring, an accolade that their senior colleagues obtained with three times higher probability and in one-tenth the amount of time.
What is your message to these junior economists, candidates?
Continuing notes on the state of economics publishing, I’ve received a couple of suggestions to examine
@AEAjournals
semi-flagship outlets. These are the four “AEJ” journals in micro, macro, applied economics, and economic policy. Do they serve authors well? Stats & commentary:
@DLeonhardt
Using statistical jargon, the correct measure of the strength-and predictive value-of the relationship is r-squared (degree to which data points hew to the line), not the regression coefficient (slope of the line).
A few (more) thoughts on the evolution of scholarly publishing, abetted by the metrics computed by the
@eigenfactor
project.
Authors supply content, journals demand it. Journals publish when quality exceeds a certain threshold.
What happens when supply increases?
ICYMI
@NickKristof
found "reason for hope" in a groundbreaking experimental study by
@OppInsights
. Go read if you haven't, then come back here for a contrarian thread: in broader context, this is a desert island of hope in a rising sea of despair.
@DLeonhardt
@OppInsights
But admissions offices have access to way more than these two measures. When we build a model that includes some of them--gender, whether a student is a legacy, athlete, first gen, under-represented minority, early decision, etc.--a funny thing happens to SAT predictive power.
Harvard's economics department did not see fit to release a statement today. Here's what they might have said:
"We are well aware of allegations involving a tenured member of our faculty, Roland Fryer.
@Harvard
policy forbids us to comment on an active investigation. However...
Meanwhile, if you're a chemist, there's a competition to see if your article ranks among the top 3,000 or so. You just need to submit to one journal (JACS), and typically the time to acceptance is under two months.
The economics system is incredibly inefficient.
The American Economic Association's bylaws do not permit direct election of presidents. The nominating committee (chosen by president-elect) and executive committee (chosen by nominating committee) serve as an "electoral college" and place a single name on the ballot...
Over 70% of psychology doctorates awarded in 2016 went to women. More than twice as many women earned psychology Ph.D.'s than econ, sociology, and political science combined.
I've opined before about the problematic, anachronistic nature of academic publication in economics. Many of my critiques could also be applied to other social sciences.
I've decided to try doing something about it.
Meet
Here,
#econtwitter
, is a photo (replete with San Diego Marriott poolside lounge chair backdrop) of the AEA budget for FY 2020. I picked it up at the sparsely attended business meeting at
#ASSA2020
.
Although printed in black, there's some serious red ink. Let's explore.
@DLeonhardt
@OppInsights
Problem
#2
: a strawman. Evidence is presented to answer the question “if you used only SAT or only HS GPA to predict college grades, which would be better?” And by either steepness of line or r-squared, SAT does better. However…
I fundamentally disagree with John M. Clark. The presidency should not be awarded to the economist with the best CV, but rather as the prize in a competition of ideas for what will yield the brightest future for our profession.
As it is in our peer disciplines.
@NickKristof
@OppInsights
@Boeing
@Microsoft
@amazon
Rather than dream up ways to rearrange which families get to live in the limited set of places where the "special sauce" used to be...
We need to find the recipe for the "special sauce" and start cooking it everywhere. That would be an undeniably positive-sum intervention.
@DLeonhardt
@OppInsights
We can, however, make inferences based on the
@OppInsights
technical appendix. Lesson one: predicting with BOTH SAT and HS GPA gives you 4x more predictive power that using HS GPA alone. So far so good!
Some journals require authors to disclose conflicts of interest. That's good.
I've never been asked to disclose conflicts of interest as a reviewer, or as an editor. Journals should ask, and should inform authors if their MS was reviewed by a conflicted referee. Readers too.
You may have read
@BCAppelbaum
's critique of the postwar economics profession yesterday. You may have been shocked to learn that economists attach a dollar value to human life.
At the risk of further shock: you do too!
Consider the deadly...paring knife.
So I'm sitting in my first
@zoom_us
office hours of spring quarter.
No visitors.
So let me extend an invitation to grad students/junior faculty -- anybody have an idea they want to bounce off me? I'll be here 'til 5 PDT, DM for meeting ID number!
Failing that, why not have AER co-editors *automatically* ship reviews of promising-but-not-quite-AER-worthy manuscripts to the relevant AEJ, where a co-editor could immediately render a (non-binding) invitation to R&R based on their read of the manuscript & reviews?
As the world remembers G.H.W. Bush, who lived long and accomplished much, my thoughts turn to Shauna Saunders, a
@DukeEcon
doctoral student who passed away 14 years ago at the age of 29. Her work wasn't in my field, but she conducted it in a secure facility where I also worked.
Alan Krueger's legacy will long outlive all of us.
Like many, he was skeptical about the
@UW
minimum wage findings but unlike many he expressed that skepticism directly to us rather than the press. I'm grateful to have interacted with him.
I know I'm late to the Banerjee/Duflo/Kremer Nobel party, but wanted to offer a comment for junior faculty who may be wondering:
"I've been told I shouldn't co-author with my former mentor(s). But Esther Duflo did so & won the Nobel. What gives?"
Generalizable lesson follows:
My thoughts on academic publishing are heavily influenced by the four years I spent co-editing the Berkeley Electronic Journal in Economic Analysis & Policy (BEJEAP).
Theoretically it still exists today, but it bears little resemblance to what it once was. A brief tale.
Presidents of the American Sociological Association have represented 20 institutions over the past 28 years. Presidents of the American Political Science Association have represented 19.
Economics is far and away the most oligarchic of the social sciences.
This pattern of exclusivity reflects a deep seated worldview, encapsulated in this passage from the work of
@undercoverhist
regarding MIT in the 1960s, that only the work of a handful of economists truly matters.
Ever wanted to change state law? As
@UW
's faculty legislative rep, that's part of my job description. Let me walk you through a case study.
Here's the text of Washington's pass-the-harasser law. Can you see the blind spot? Let's shine a light on it and talk about how to fix it.
@causalinf
@GivingTools
@yudapearl
@KetchupEconomi1
@sterndavidi
The honoraria paid to AEA editors are not available in the association's annual report (papers & proceedings). However as a 501(c)(3) AEA must file IRS 990, which is a public document.
For calendar year 2018, lead editors at the AEA journals were paid $83k-123k.
By revealed preference, economics believes not only that the best research is dominated by a handful of scholars in a small number of departments but also that the correlation between research ability and ability to lead the discipline is 1.
There is this belief in economics, I think widespread, that if you’re not in a top institution, it’s because somehow you’re not good enough.
You’re not good enough with math, you’re not good enough with econometrics, with this, or that.
That’s non-sense.
1/n
New
@USCensusBureau
data released today allows an extension of residential segregation measures
@Cutler_econ
, Ed Glaeser, and I first published in 1999. From 2010 to 2020 segregation continued a decline that first began after passage of the Fair Housing Act. Details follow...
@LetoC
@Google
Here's one more thought that occurred to me as I perused the budget.
Top departments -- where the editors work, where the faculty & students almost certainly cash more referee checks per capita -- get more money from the AEA than they contribute.
Everybody else...
The most prestigious award conferred by the AEA is the John Bates Clark medal, for the best American economist under the age of 40. It was last awarded to an economist outside the 10-institution "cartel" in 1959. (Lawrence Klein, Penn)
Dixit's letter was written in response to
@ArthurBrooks
' recent
@TheAtlantic
essay, which is itself a must read for any professional at risk of aging:
Competition is good, right? Incentives matter, right?
Parallel submission is the norm for academic book manuscripts and let's face it, economics articles are looking more like books all the time.
The thousands upon thousands of dues-paying economists, from the perspective of those atop the hierarchy, busy themselves with work of no consequence and are chiefly useful as potential employers for disappointing students.
I've been on/chaired public policy Ph.D. admissions committees about ten times. Here's some comments for those thinking of applying or more generally about what doctoral admissions should prioritize.
Five factors. Sometimes I have trouble remembering them all, but here goes:
@arthurbrooks
@TheAtlantic
The nature of the trap is as follows: your happiness is tied to your professional success. But for every level of success you attain, there is a higher one that eludes you.
Moreover, when you attain success you immediately begin to worry about becoming a "has-been."
On January 1st, 1972 the American Economic Review imposed a submission fee ($10) for the first time.
From 1972 to the present:
The Consumer Price Index ⬆️ by a factor of 6.3.
The AER submission fee ⬆️ by a factor of 20.
AEA dues for the highest-earners ⬆️ by a factor of 2.2.
@katestarbird
Hello
@UW
🧵 readers! As your faculty legislative rep I'm hearing a lot of concern about the public records act (RCW 42.56). To express yours, please use this link:
While responses are themselves public records, you can submit anonymously.
Less radical option: limit referees to one shot at the paper. Set expectations that R&R requests will involve if...then statements making the future transparent:
e.g., "if the results are robust to adding controls for x, y, z, the revision will be accepted."
Is it any wonder that this discipline has an inclusivity problem? Any wonder that it has chosen to ratchet up barriers to publication in its most prestigious journals rather than expand to accommodate growth in the profession?
Resolved:
To use the majority of unstructured time at my next conference to seek out unfamiliar names & faces, to hear what they've been working on, to offer advice I may have, introduce them to others who should know them. As certain role models do!
Revise and Resubmit: conversations that should be going on in published literature are instead taking place in private, between authors and reviewers.
Ellison (2002) notes that R&Rs were once used sparingly and were in fact a mark of shame.
One option: eliminate R&Rs entirely.
Now let's talk governance. The current president of the American Psychological Association is Dr. Rosie Phillips Davis. She is the first woman of color to lead the organization.
Dr. Ben Bernanke is the current president of the AEA. He is the 117th white male to hold the office.
Among the 18 urban regions with homeless count > 5000 in 2007:
All 8 with 1BR rents above $1200 have seen homelessness worsen over the past decade.
All 10 with 1BR rents below $1200 have seen homelessness improve over the past decade.
r=0.62.
@HUDgov
data.
Moral of the story: the job market is humbling because there are so many factors beyond your control. I've had applicants not offered a job ask what they might have done better. The most common answer:
"not a single thing."
So why does this matter? Can't we just happily talk amongst ourselves?
Scholars in all disciplines are primarily rewarded by communicating within the discipline.
The fate of discipline, however, depends critically on its ability to communicate with the rest of the world.
Having worked on Ph.D. admissions for the past 10 years, in policy programs with acceptance rates <10%, I understand credential inflation as what happens when you have too many candidates chasing too few slots for admission. It's hard to rise to the top of the applicant pool.
For all the angst about the rise of the predoc and increasing time to complete the econ Ph.D., data from
@NSF
show the average time from BA to PhD for 2019 recipients in econ (8.5 years) is tied for lowest in the social sciences AND lower than the comparable stat (9.7) for 1994!