In 2020,
@JamesADiao
and I began studying race adjustment in lung-function equations. With amazing coauthors, today we published the past years of work in
@NEJM
, estimating the many clinical, financial, and occupational implications.
#ATS2024
Full paper:
Introducing our new course at Harvard, BMI 704: Data Science for Medical
Decision Making. If you're interested in data science and machine learning
applied to medicine, have a look below. Guest lectures by
@VPrasadMDMPH
,
@MauSantillana
!
Data and readings:
In our new preprint, we evaluated GPT-4V on 934 challenging
@NEJM
medical image cases and 69 clinicopathological conferences. GPT-4V outperformed human respondents overall and across difficulty levels, skin tones, and image types except radiology, where it matched humans. GPT-4V
Billions of lab tests are performed in the U.S. each year, and most are not stratified by demographic variables (e.g. race, sex, age). Should they be? John Ioannidis,
@chiragjp
, and I explore some of the thorny challenges in our new
@JAMA_current
piece:
Coming soon: NEJM AI, a new journal from NEJM Group.
NEJM AI aims to identify and evaluate state-of-the-art applications of artificial intelligence to clinical medicine. Learn more about the new journal:
Excited to share our new analysis in
@NatureGenet
led by brilliant colleagues
@chiragjp
and
@cmlakhan
of
@HarvardDBMI
: Repurposing large health insurance claims data to estimate genetic and environmental contributions in 560 phenotypes:
Will new genomics and machine learning applications in medicine yield useful biomarkers or expensive "biomarkup"? Excited to share my new article with
@mandl
in
@JAMA_Current
:
Excited to announce that I’ve joined the core faculty of
@HarvardDBMI
and that my lab has moved to
@HarvardDBMI
@HarvardMed
. I’m thrilled to return to where I trained with
@zakkohane
& to join colleagues transforming medicine w/ computation. We’re hiring!
(Thread) The paper by Evangelia et al. in
@JClinEpi
on 'logistic regression = machine learning' for medicine has generated many reactions. This paper may be misinterpreted by
#MachineLearning
cynics and enthusiasts alike (). Some thoughts below👇 1/n
What comes through in the
@NatureMedicine
article by
@EricTopol
is that one of the biggest contributions of
#ArtificialIntelligence
to medicine is highlighting how often physicians disagree with one another. A few examples from the article below.
Excited to announce a new version of our course at Harvard, BMI704: Data Science for Medical Decision Making, and to be teaching with
@chiragjp
. We'll release lectures, data, and an interactive text of data science methods at link below.
Sched + Readings:
We started
@NEJM_AI
to publish clinical-grade evidence needed to improve patient care with AI.
In our new editorial, we draw inspiration from the first year of the
@NEJM_AI
Grand Rounds podcast to illustrate what we are eager to publish going forward:
A few years ago, my mom saw her nephrologist for staging her kidney disease. An Indian American woman, she was given test results that *averaged* those of "African Americans" and "Everyone else". Honored that her story is included in this
@NPR
story:
Can synthetic data produced by latent diffusion models improve medical AI? We studied this question using skin disease classifiers in our new preprint led by
@luke_sagers
@JamesADiao
@lukemelas
. Takeaways:
• Synthetic images can enhance model performance in data-limited
Huge congratulations to new **Associate Prof**
@chiragjp
of
@harvardmed
@HarvardDBMI
. Of course the first thing he wants to do to celebrate is hold group meeting as planned and learn about new research.
🧵1/ I’m beyond proud of my student
@JamesADiao
who today became the 22nd student in the >200-year history of Harvard Medical School to receive his MD Summa Cum Laude.
No better way to celebrate this by noting the uncanny similarities between James Diao and … Taylor Swift.
Could an AI second opinion service improve medical diagnosis and reduce medical errors **today**? We believe the answer is yes with proper oversight.
@AdamRodmanMD
and I reflect on the urgency in our new
@statnews
essay:
How will GPT-4 change medicine? I lost sleep after this long and fascinating conversation with
@peteratmsr
of Microsoft. He took us behind the scenes of the Microsoft/OpenAI partnership and predicted how GPT-4 & beyond will impact healthcare. Listen here:
It's a privilege to work alongside this creative & hard-working team. It was a great 2023 for our lab at
@HarvardDBMI
and we are looking forward to 2024!
If you're a researcher worried you can't compete with the resources of the big companies, this is a stunning defense of the disruptive power of small team science:
Are you looking for a post-doc position in machine learning applied to fundamental questions in medical diagnosis?
@chiragjp
and I are recruiting for an exciting new position across our labs at
@HarvardDBMI
/
@Bos_CHIP
-- reach out if interested!
Reproducing neural architecture search for some ML models can cost 3 R01 grants, but do most medical applications need this? Our take on the top challenges to reproducibility of machine learning in medicine in the latest
@JAMA_current
:
Honoring
@MIT_CSAIL
's Pete Szolovits at
@harvardmed
,
@zakkohane
shares 3 articles by Pete that were decades ahead of their time and are being recapitulated today.
We are supporting and encouraging the use of large language models in submissions to
@NEJM_AI
. Read our rationale in this new editorial led by the inimitable
@DaphneKoller
:
and see the helpful thread by
@AndrewLBeam
below
1/n: We're excited to share a new policy on the responsible use of LLMs in drafting manuscripts for
@NEJM_AI
🔗:
We believe strongly that LLMs provide a net benefit to science writing, so we encourage their use.
Rationale and caveats for this policy 👇
Medical centers across the US have recently removed race from estimates of kidney function. How might this impact care? In our new
@JAMA_Current
study led by superstar
@harvardmed
student
@JamesADiao
, we quantify the broad impact on Black Americans:
Many people believe that 21st century neural networks will soon transform doctors' approaches to medical reasoning; the 18th century Bayes' rule has yet to.
Congrats to **Dr.**
@luke_sagers
who brilliantly defended his PhD dissertation! Luke rigorously evaluated whether synthetic data can improve medical image classifiers across populations. It's been a privilege to work w/ Luke from when he was a summer undergrad at
@HarvardDBMI
We have long known about statistical reasoning challenges in medicine. But where along the diagnostic pathway should we target efforts and for which patient population? My comments on an important new study by
@dr_dmorgan
et al. in
@JAMA_IM
:
The best part of being a prof. is working alongside passionate students over years. There is nothing quite as rewarding as when they are recognized for their hard work and contributions. Congrats to
@JamesADiao
, new
@PDSoros
Fellow:
#SAIL2023
last week was one of the best conferences I've attended. Nuanced and in-depth discussions about medical AI with an amazing community, and I'm already missing the energy. Some of my highlights were
@JamesADiao
giving a spotlight talk on our work on improving
Thrilled to share new work from my group led by
@Bos_CHIP
/
@HarvardDBMI
student Luke Sagers in collab. w/ Luke Melas-Kyriazi and
@chiragjp
, now out in
@AgingJrnl
:
Prediction of chronological and biological age from laboratory data
Full text:
Thread👇 1/n
We had an amazing conversation with
@mcuban
about AI, large language models, trust in medicine, and Skip Bayless (not a sentence I ever thought I would say). Out now on the latest
@NEJM_AI
Grand Rounds:
Key line from
@EricTopol
's recent review of medical
#AI
in
@NatureMedicine
: "the ‘AI chasm’—that is, an algorithm with an AUC of 0.99 is not worth very much if it is not proven to improve clinical outcomes."
A seminal 1978 paper by Szolovits and Pauker described two extremes of medical reasoning: purely categorical & purely probabilistic. Explore the "voracious demand for data" of pure probabilistic reasoning below. Created by James Diao for Harvard BMI 704:
Deep learning approaches for semantic image segmentation are often data hungry or difficult to train. Thrilled to share our new approach called PixMatch, our
#CVPR21
paper now on arXiv. Led by the singular
@lukemelas
:
Paper:
Code:
What a pleasure to chat with the inimitable
@atulbutte
today for
@NEJM_AI
Grand Rounds. Sneak preview below with Atul holding a quote with his attitude towards credit assignment; episode coming soon!
🗣️ New episode of AI Grand Rounds!
Dr.
@RoxanaDaneshjou
shares her journey from a childhood influenced by early exposure to science to her current role as an assistant professor
@StanfordMed
.
Listen to the full episode now:
Our conversation with the extraordinary
@atulbutte
is out now on
@NEJM_AI
Grand Rounds! Atul shares his career journey, some of his secrets in giving legendary talks, and his mission to use massive data to improve health.
Episode link:
Excited to 'debate' Prof. Stultz tmrw about the role of LLMs and AI in medical diagnosis at the annual
@ImproveDX
#SIDM2023
meeting in Cleveland. What role will LLMs and AI play in improving diagnostic accuracy? How will LLMs shape medical education incl. the excellent training I
Grassroots activism has prompted a long-overdue reconsideration of race across medicine. We can achieve both performance and equity with race-free equations. Here are our thoughts on eGFR in the latest
@NEJM
:
A few years ago, we (cc
@sacjai
@zakkohane
) published a small study on medicine's uncomfortable relationship with math (). I wonder if our results would have improved if stats lectures in med school looked a little more like this:
(Thread) 2019 problem: you have a big dataset and everything is significant after Bonferroni. What's spurious; what's real? How do you prioritize? Happy to share my take w/ John Ioannidis &
@chiragjp
out now in the American Journal of Epidemiology:
1/n 👇
Same trial, two very different conclusions. This study in
@JAMA_current
exposes a fundamental tension in interpreting RCTs: should we summarize them as Bayesians using priors informed by previous studies/meta-analyses or should we keep them isolated?
In sum, this paper is generating what I hope will be a useful discussion around reporting and bias. The less comfortable question it leads to is how much time/resources are spent on methods innovation that would be better spent on less glamorous data tasks. 12/12
Incredibly proud of my student Luke Melas-Kyriazi for being selected as a Rhodes Scholar. Keep an eye out for his machine learning+medicine work at
@UniofOxford
!
Diving deep into LLMs,
@AndrewLBeam
and I interviewed
@alan_karthi
and
@vivnat
about Google's Med-PaLM model on the latest
@NEJM_AI
Grand Rounds. They reflect on the need for careful evaluation, the breakthroughs that led us here, and where we are headed.
Excited to share our new
@NEJM_AI
Grand Rounds episode, an enlightening conversation with
@pranavrajpurkar
of
@HarvardDBMI
who takes us behind the scenes of a few seminal medical AI papers over the past decade and also gives us a preview of what's next:
Estimating the Prevalence of COVID-19 in the United States: Three Complementary Approaches. From the brilliant
@MauSantillana
of
@Bos_CHIP
and
@harvardmed
: Full paper:
We are rediscovering much of the conversation about machine learning in medicine from the 1970s and 1980s. HINT: tons of good research project ideas available by mapping that literature onto today's methods & data. Read & re-read Pete Szolovits and others!
Really looking forward to joining
@EricTopol
& Dr. James to discuss AI's role in medical diagnosis and treatment and sharing some of work happening at
@HarvardDBMI
and
@NEJM_AI
AI is here. Is your practice ready? Join us for a live panel featuring insights from
@EricTopol
,
@arjunmanrai
, and Dr. Ted A. James as they delve into AI's role in diagnosis, treatment, precision medicine, and related areas. Register:
#MedTwitter
Much confusion remains about appropriate clinical uses of genetic testing with thorny questions around 'normal' variation. Engage with leaders at the
@HarvardDBMI
@harvardmed
precision medicine conference this fall.
Speaker lineup & registration:
"Data about the timing of when laboratory tests were ordered were more accurate than the test results in predicting survival in 118 of 174 tests (68%)."
We had a really fun conversation with
@erichorvitz
for
@NEJM_AI
Grand Rounds. It was a wide-ranging discussion on AI in medicine, from Eric’s early pioneering efforts to the latest advancements in prompting LLMs. We even snuck in some decision theory.
Listen here:
📢 New episode of AI Grand Rounds!
Dr.
@erichorvitz
describes his career evolution from an interest in neurobiology to significant contributions in AI, particularly in understanding complex systems and applying AI in medicine.
Listen to the full episode:
It takes real grit to transform a medical AI idea into clinical impact. Hear
@MichaelAbramoff
's path-breaking story developing the first autonomous (no doc req'd) medical device to receive FDA authorization on the latest
@NEJM_AI
Grand Rounds:
We know LLMs can ace multiple-choice exams. Taking us deeper, an important new study led by
@stephcabral_
and
@AdamRodmanMD
conducts a nuanced evaluation of the clinical reasoning abilities of GPT-4 wrt physicians. Guess who wins?
Need more of these!
The Manrai Lab at
@Bos_CHIP
/
@harvardmed
is recruiting a funded postdoc in stat gen and/or comp genomics. 2+ years, very competitive salary, and intellectual freedom. Relevant lab papers:
If interested:
Check out the author list h/t
@AndrewLBeam
Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models | medRxiv
Thrilled to announce
@NEJM_AI
Grand Rounds, a new podcast I'm co-hosting with
@AndrewLBeam
. We'll feature informal conversations with experts exploring deep issues at the intersection of artificial intelligence, machine learning, and medicine.
Launching later this month, NEJM AI Grand Rounds is a new podcast exploring how
#ArtificialIntelligence
will change clinical practice and healthcare. Listen to the podcast trailer and subscribe:
#AIinMedicine
Excited to speak about artificial intelligence and pulmonary medicine next week ... will include some of our lab's work at
@HarvardDBMI
and also highlight papers we've published over the past few months at
@NEJM_AI
THREAD -
#AI
is awesome, but will new algorithms reach patients? Shortliffe & Sepulveda in
@JAMA_current
remind us about the decades-long "challenges of credibility and adoption" that have long faced clinical decision support systems 1/n
This was a lot of fun this morning. Especially enjoyed the best cases presented by the fellows and discussed by faculty after my talk (and trialing GPT-4 as discussant!)
Just had a great conversation about radiology, artificial intelligence, and algorithmic bias with
@judywawira
for
@NEJM_AI
Grand Rounds. Episode coming soon
I did not look at this slide for a long time. But now I am concerned. We are about to complete four years of ZERO change in WGS costs, the longest stagnation in the past 20 years.
#stagnation
The question I get most often from students new to data analysis is: What can I do to improve my practical skills? My answer: read and work through . And then do it again in a few months. And then again.
It was really great for me and
@AndrewLBeam
to chat with
@AdamRodmanMD
and/or his digital simulacrum about LLMs, diagnostic reasoning, and the history of medicine for
@NEJM_AI
Grand Rounds. Episode coming soon!
New episode of
@NEJM_AI
Grand Rounds is out! A conversation with the amazing
@oziadias
on AI bias, safety, and generalizability. Ziad illustrated how AI is a double-edged sword that can both exacerbate and reduce health disparities. Link here:
The search for illness is becoming increasingly proactive in the digital and genomic age. What are the consequences? My new piece with
@mandl
in STAT
@statnews
: about our recent
@JAMA_current
piece on Biomarkup:
@JamesADiao
giving a great talk on the opportunities and limitations of using synthetic images produced by generative AI to improve the performance of dermatological classifiers
#SAIL2023