Ali Madani @thisismadani Twitter profile | Pikagi

Pikagi

Ali Madani

@thisismadani

6,523

Followers

1,035

Following

34

Media

368

Statuses

Founder & CEO of Profluent (, we're hiring!). AI+Biology to cure disease. Berkeley PhD. formerly Research @ Salesforce AI.

San Francisco, CA

https://t.co/dcDVwtYDmI

Joined July 2014

Don't wanna be here? Send us removal request.

Pinned Tweet

@thisismadani

Ali Madani

21 days

A wonderful article by @CadeMetz on our OpenCRISPR project @ProfluentBio . Perfectly captures the science, origination, and implications of our recent announcement+release in an approachable and accurate manner. Highly recommend a read through!

@nytimes

The New York Times

22 days

Generative AI technologies can write poetry, computer programs and more. Now, new AI technology is generating blueprints for microscopic biological mechanisms that can edit your DNA.

20

39

85

0

2

14

Last Seen Profiles

@digi_threads

@WarmAudio

@BrookeKilg38482

@0F6bQM6lJqMxF01

@lay_studio

@OanaDumitrascu5

@HYCO_Official

@iQZNOe6mY1Gy9ll

@ManizalesM

@tattumcr

@ialirazaabbas

@mmss_knknhnk

@Chrisboole

@Ansip_EU

@WalkRabbitWalk

@thighstarve

@Mubzzz_

@BryantFootball

@8vlkWeRQWtxsnrq

@l1lmaa

@Rackscapital

@dsasd2169783589

@chrisbkim

@yamakawa_y46530

@rudydesouza

@NoLookMan_Oki37

@ZiggyYag

@KodyStrayhorn

@FeirlyGab

@mikepanayi

@KaanTurna

@isukobone742

@JC_MAGA10

@amohandboll

@Angychess5

@FovarosiN

@thisismadani

Ali Madani

22 days

Can AI rewrite our human genome? ⌨️🧬 Today, we announce the successful editing of DNA in human cells with gene editors fully designed with AI. Not only that, we've decided to freely release the molecules under the @ProfluentBio OpenCRISPR initiative. Lots to unpack👇

282

1K

4K

@thisismadani

Ali Madani

1 year

ChatGPT for biology? Excited to share our work on LLMs for protein design out today @NatureBiotech + Proud to publicly announce @ProfluentBio with a $9M seed round to tackle meaningful challenges in biology with AI. Join us!

38

352

2K

@thisismadani

Ali Madani

3 years

This twirling work of science is special ✨ AFAIK, it's the first crystal structure of a functional #protein fully designed by #AI A milestone in our quest to use language models to generate proteins that are unseen in nature & can function well in the real-world. Read below👇

10

209

681

@thisismadani

Ali Madani

3 years

#Alphafold by #deepmind used solid interdisciplinary intuitions for algorithm/model design. It wasn't just a rinse-and-repeat machine learning exercise. Details on methods are limited, but here's my best interpretation (+some predictions) so far: [1/n]

Tweet media one

8

106

573

@thisismadani

Ali Madani

2 years

📢Excited to share our exploration of billion-scale model and dataset sizes to examine the boundaries of protein language models with ProGen2 Paper: Code: >6B params & >1B seqs from genomic, metagenomic, immune repertoire studies

9

93

397

@thisismadani

Ali Madani

3 months

Published in @NatureBiotech today: @jeffruffolo and I demystify how to design proteins with LMs. Reach out if you have any Qs on how to apply the power of LMs for your protein design goals-- which we do routinely at @ProfluentBio

Tweet card media

Designing proteins with language models

Nature Biotechnology - Protein language models learn from diverse sequences spanning the evolutionary tree and have proven to be powerful tools for sequence design, variant effect prediction and...

4

58

331

@thisismadani

Ali Madani

22 days

Our LLMs were trained on massive scale sequence and biological context to generate millions of diverse CRISPR-like proteins that do not occur in nature, thereby exponentially expanding virtually all known CRISPR families at-will.

Tweet media one

5

33

312

@thisismadani

Ali Madani

22 days

The results point to a future where AI precisely designs what is needed to create a range of bespoke cures for disease. There is still much to build to achieve this vision. To spur innovation and democratization, we are freely releasing OpenCRISPR-1. Try it out!

Tweet media one

7

28

266

@thisismadani

Ali Madani

22 days

Paper: Blog: NYTimes: Press Release: Access OpenCRISPR-1:

Tweet card media

Please express interest in accessing OpenCRISPR using the form below. After clicking "Submit", do not close your browser window! The completion page will have links to 3 resources - our OpenCRISPR...

docs.google.com

9

33

178

@thisismadani

Ali Madani

22 days

AI has become increasingly pervasive in our daily lives from how we sift through information, produce content, and interact with the world. This marks a new chapter where AI is used to alter the fundamental blueprint of who we are - our DNA.

5

8

144

@thisismadani

Ali Madani

22 days

We then focus on type II effector complexes, generating cas9-like proteins and gRNAs. These proteins are hundreds of mutations away from anything in nature.

Tweet media one

Tweet media two

2

5

143

@thisismadani

Ali Madani

22 days

We then characterized our generations in the wet lab and found that the AI-designed gene editors show comparable or improved activity and specificity relative to SpCas9, the prototypical gene editing effector. More characterization is underway but we're already impressed.

Tweet media one

Tweet media two

2

7

141

@thisismadani

Ali Madani

22 days

We were immediately drawn to gene editing due to the pressing societal needs, potential for one-and-done cures to disease, and the scientific challenge + complex biology involving protein, RNA, and DNA.

4

8

129

@thisismadani

Ali Madani

2 months

🥳🚨Combo announcement today: new $35M funding to accelerate our mission, @Fraser (fmr Head of Product at OpenAI) joins our board, & first vertical in gene editing w/ @HilaryEaton14 @PeterCameron4 . Ecstatic to build with a phenomenal team 🚀🚀

@ProfluentBio

Profluent

2 months

We’re excited to announce $35M in additional funding led by @sparkcapital to bring the total raised to $44M from investors including @insightpartners , @airstreet , @aixventureshq , @convergencevc , and a syndicate of angel investors across AI & biotech.

2

10

53

13

2

113

@thisismadani

Ali Madani

3 years

Can generative AI learn to extrapolate? We explore how to generate sequences that enhance desired attributes-- beyond what was seen in training. Works pretty well in #NLP and #proteins ! Blog: Paper: Code:

4

24

112

@thisismadani

Ali Madani

22 days

We also created an AI-designed base editor which exhibited really exciting performance in precise A->G edits.

Tweet media one

2

4

110

@thisismadani

Ali Madani

2 years

More to come on this shortly - if you want to work on the *most meaningful* application of generative AI, send me a DM. Slide directly borrowed from @stateofaireport like #StabilityAI :)

Tweet media one

1

9

99

@thisismadani

Ali Madani

2 years

#hiring for a Head of Data Science position. Hybrid role but based in SF Bay Area. Needs an extensive bioinformatics background / machine learning experience not necessary. Really exciting opportunity to be at the forefront of protein design w/ AI. Pls DM me for details!

3

31

98

@thisismadani

Ali Madani

22 days

This was truly a team effort across all disciplines of the company. @jeffruffolo SNayfach JGallagher @AadyotB JBeazer RHussain JRuss JYip EHill @MartinPacesa @alexjmeeske PCameron and the broader Profluent team. If you want to build with us, join. We’re hiring.

5

5

97

@thisismadani

Ali Madani

1 year

Derek Lowe perspective on our work, Making Up Proteins @ProfluentBio @Dereklowe

Tweet card media

Making Up Proteins

Making Up Proteins

www.science.org

1

9

56

@thisismadani

Ali Madani

3 years

We have a lot of finer details in our preprint: And check out our blog:

Tweet card media

Deep neural language modeling enables functional protein generation across families

Bypassing nature’s evolutionary trajectory, de novo protein generation—defined as creating artificial protein sequences from scratch—could enable breakthrough solutions for biomedical and environme...

www.biorxiv.org

3

8

55

@thisismadani

Ali Madani

3 years

📢 Join us for the "Representation Learning in Biology" Special Session on June 25 as part of 2021 ISMB/ECCB. We've got a great speaker lineup (below), tailored for Bio or ML backgrounds! We're accepting abstracts for talks/posters. DEADLINE: May 30

Tweet media one

1

12

53

@thisismadani

Ali Madani

1 year

Had a great conversation w/ @_KarenHao about protein design and AI, and thrilled to see this piece out in the @WSJ ! Watch this space for more Profluent updates coming soon…

Tweet card media

How AI That Powers Chatbots and Search Queries Could Discover New Drugs

Natural language processing algorithms like the ones used in Google searches and OpenAI’s ChatGPT promise to slash the time required to bring medications to market

3

11

52

@thisismadani

Ali Madani

2 years

okay… this is a protein paper: from database retrieval (ie MSAs) to dataset leakage concerns (ie homology filtering)

@JayAlammar

Jay Alammar

2 years

A 🧵looking at DeepMind's Retro Transformer, which at 7.5B parameters is on par with GPT3 and models 25X its size in knowledge-intensive tasks. A big moment for Large Language Models (LLMs) for reasons I'll mention in this thread.

Tweet media one

10

197

965

2

4

51

@thisismadani

Ali Madani

3 years

Last time, #alphafold made headlines by beating 2nd place by 18.5%. In #CASP14 , #alphafold2 essentially beat SOTA by 165.2%. Results are astounding: demonstrate the power of large sequence databases (+retrieval) and attention-based learning.

Tweet media one

Tweet media two

Tweet media three

1

11

48

@thisismadani

Ali Madani

6 months

🚀 Excited to be a part of the inaugural @FortuneMagazine top 50 AI companies. Can't wait to share more news soon ✌

@ProfluentBio

Profluent

6 months

🥳 @ProfluentBio has been chosen by @FortuneMagazine in their inaugural #AI Innovators list which highlights the top 50 AI companies! Stay tuned for more updates on how we're pushing the boundaries of AI and Biology to benefit society.

Tweet media one

0

1

11

0

5

47

@thisismadani

Ali Madani

2 years

@pastramimachine Two best options: 1) . 2) AlphaFold followed by FoldSeek

Tweet card media

ProteInfer, deep networks for protein functional inference

google-research.github.io

0

2

46

@thisismadani

Ali Madani

1 month

This energy needs to go away 🤮. I'm increasingly seeing stories around 'overhyping AI' - from consumer to biotech. It's a lazy take. There are real breakthroughs happening now. Go closer to the source ... speak with the researchers on the ground enabling the future. Energy=LFG

@AndrewE_Dunn

Andrew Dunn

1 month

NEW: I talked with Daphne Koller, an OG AI person, about insitro's biology research, the need for more data, and the possible impact of "potentially destructive" hype around AI:

0

4

17

1

3

45

@thisismadani

Ali Madani

14 days

Great article on AI x Gene Editing in @Nature describing recent work from us @ProfluentBio and groups such as @BrianHie @arcinstitute . Learn more about the positive impact this field could have on patient lives

@ProfluentBio

Profluent

14 days

Check out @EwenCallaway 's article in @nature about how we’re using protein language design to create highly performant #CRISPR gene editors, and what this could mean for medical applications 👉

2

12

67

0

3

46

@thisismadani

Ali Madani

15 days

an exciting moment for the gene editing field and genomic medicines to find lasting cures to disease

@PrimeMedicine

Prime Medicine

16 days

We are pleased to announce that #FDA has cleared our IND application for PM359 for the treatment of chronic granulomatous disease (CGD), enabling Prime to initiate its Phase 1/2 clinical trial in the US. Details: $PRME

4

21

101

0

8

41

@thisismadani

Ali Madani

4 years

[my take] In the next few years, *large-scale* language modeling will enable the most impactful leaps in protein science/eng. We take an interpretability lens to understand how attention operates via MLM. Check out /use the visualization tool for your protein language models

1

6

39

@thisismadani

Ali Madani

3 years

In the metaverse of last year’s NeurIPS, @KevinKaichuang and I virtually huddled to commiserate re: the lack easy-to-use/standardized ML benchmarks tailored for protein engineering. Now to appear at this year’s NeurIPS, we have FLIP led by @sacdallago @Jody_Mou et al.

@KevinKaichuang

Kevin K. Yang 楊凱筌

@KevinKaichuang

3 years

A set of benchmark tasks for predicting protein fitness or function from sequence. @sacdallago @Jody_Mou @kadinaj Bruce Wittmann, Nicholas Bhattacharya, @samgoldman19 @thisismadani and me ( @KevinKaichuang )

Tweet media one

6

67

314

1

2

35

@thisismadani

Ali Madani

1 year

Profluent is backed by top investors in AI and Biotech. Led by @insightpartners w/ @airstreet @aixventureshq @ PVP We have an awesome team @alexjmeeske @dylanmorris @nathanbenaich @fraser_lab @RichardSocher and more! Join us! ->

Tweet card media

Profluent debuts to design proteins with machine learning in bid to move past 'AI sprinkled on top'

While OpenAI's Microsoft-allied ChatGPT takes the world by storm, a fledgling startup in Berkeley, CA is debuting to take a similar language-learning model approach, but with the goal of designing...

4

4

33

@thisismadani

Ali Madani

3 years

Proteins do everything in life. They're complex molecules and the workhorses for almost all of biology. Nature has evolved proteins over billions of years. But instead of relying on natural evolution, what if we could take control and design proteins ourselves from scratch?

1

0

33

@thisismadani

Ali Madani

3 years

To prove this really works, we had to go to the wet laboratory to synthesize and evaluate our proteins. We partnered with @tierrabio and @fraser_lab at @UCSF to evaluate our AI-generated artificial antibacterial protein sequences across 5 families of lysozymes

1

5

32

@thisismadani

Ali Madani

1 year

For the paper: check out the News & Views article Massive shout-out to our rockstar lineup of researchers @nikhil_ai @fraser_lab @RichardSocher @benwkrause @mohrbp EricGreene ZacharySun @CaimingXiong JamesHolton SubuSubramanian

Tweet card media

Hallucinating functional protein sequences

Nature Biotechnology - Functional proteins with limited homology to natural proteins are designed using a large language model.

1

7

28

@thisismadani

Ali Madani

4 years

Excited about the future of AI and protein engineering! Proud to finally get this project out and to work with such a great team @BMarcusMcCann @nikhil_ai @StrongDuality @RichardSocher

@RichardSocher

Richard Socher

4 years

Introducing ProGen, a large language model trained on 280 million protein sequences that can generate viable proteins based on user specifications. A step towards AI & #nlpproc helping cure disease and clean our planet. Paper: Blog:

Tweet media one

9

106

345

0

8

28

@thisismadani

Ali Madani

3 years

The artificial proteins work just as well as natural proteins across multiple families in a high-throughput assay! With more rigorous kinetics characterization, we observe that the artificial proteins have comparable catalytic efficiencies to a highly-evolved natural protein!

Tweet media one

2

0

29

@thisismadani

Ali Madani

3 years

Data fuels life to this whole field. Love seeing these reminders

@thesteinegger

Martin Steinegger 🇺🇦

3 years

#AlphaFold 2 Colab has processed >10k queries. We now also search against BFD, Mgnify, SMAG( @tomodelmont ), MetaEuk in addition to UniRef. SMAG&MetaEuk have >20M eukaryotic environmental proteins that were not used in AF2 before. @sokrypton @milot_mirdita

Tweet media one

7

69

268

0

1

26

@thisismadani

Ali Madani

3 years

We look to artificial intelligence (AI) for help. In particular, we've seen the usage of language models to controllably generate realistic text in #NLProc . In our work, we've developed some powerful language models to learn from evolution to generate protein sequences.

Tweet media one

1

4

27

@thisismadani

Ali Madani

3 years

I'm genuinely ecstatic about this field and the momentum built by so many labs now. On our end, we'll have more to come... so stay tuned!

4

0

25

@thisismadani

Ali Madani

3 years

happening now at the @RepLearningBio ISMB special session. come check out our great lineup of speakers, panels, and talks. @alexrives currently speaking on the multiple ESM projects at FAIR

Tweet media one

1

1

23

@thisismadani

Ali Madani

2 years

2 wks x 2000 GPUs = 225M "proper" structures w/ ESM2. 12.6% without a match to experimentally-determined structures. have we already uncovered most of the structural motifs at this point?

@alexrives

Alex Rives

2 years

Today we're releasing the structures of over 600M metagenomic proteins—the least understood proteins on earth. A view into the dark matter of the protein universe. Explore the ESM Metagenomic Atlas here:

39

410

2K

1

2

25

@thisismadani

Ali Madani

5 months

I'm at #NeurIPS2023 ! Particularly MLSB and ENLSP workshops + others. Come say hi or DM me. If you want to be at the frontier of impactful science, learn from world-class leaders across disciplines, and build out moonshot ideas: now is the time. Join us. @ProfluentBio 🤯🚀

1

1

23

@thisismadani

Ali Madani

3 years

To examine how similar these artificial sequences are to nature, we search for the closest protein in publicly-available databases to calculate a max identity. Artificial proteins were still functional even at 44% max identity to any known naturally-evolved protein.

Tweet media one

2

0

21

@thisismadani

Ali Madani

5 years

Just finished @EricTopol ’s brilliant book on #DeepMedicine . As someone in the field who is in the last stretch of PhD dissertation writing, it’s inspiring to see such thought leadership and synthesis. Highly recommend to all! Our work was mentioned too! @mofrad @mofradlab

Tweet media one

Tweet media two

1

1

20

@thisismadani

Ali Madani

3 years

Some cool historical context: lysozymes were the first antibiotic discovered (via Alexander Fleming) and were the first enzymes ever to have a structure determined. It's a neat space to push the frontiers of generative AI for protein design.

1

0

21

@thisismadani

Ali Madani

3 years

There's a neat principle/intuition called coevolution that can help explain. The mutational variance observed can give clues to protein structure and function. Read more here:

Tweet media one

1

0

20

@thisismadani

Ali Madani

4 years

We're hiring @SFResearch interns for 2021 in AI research. Msg me directly if you're interested in advancing ML/NLP techniques for proteins. ProGen was just the beginning ;) Application:

3

6

18

@thisismadani

Ali Madani

3 years

Join me for two exciting #ICLR2021 events Today: Check out our updated BERTology meets Biology paper at the 5pm PT poster session! Tomorrow: I'm hosting a Roundtable session to chat about AI for Biology at @SFResearch at 3:30pm PT.

Tweet media one

0

4

19

@thisismadani

Ali Madani

3 years

creative work by @BrianHie @KevinKaichuang ! very neat problem formulation and utilization of open-sourced black-box models. i still firmly believe we've only scratched the surface with protein language models

@BrianHie

Brian Hie

3 years

In some fun recent work with @KevinKaichuang and Peter Kim, we show that by using masked language models to predict local mutational effects, we can construct an evolutionary "vector field" -- kind of like RNA velocity, but for protein evolution!

Tweet media one

9

68

304

0

2

18

@thisismadani

Ali Madani

3 years

The retrieval step is critical to the success of #alphafold and it relies on decades of scientific advances in cost-effective protein sequencing, curation of protein databases (including metagenomics in BFD), and efficient search software [such as developed by @thesteinegger ]

2

2

19

@thisismadani

Ali Madani

3 years

microbes, metagenomics, and language! @y_bromberg goes further in-depth at @RepLearningBio

Tweet media one

0

0

17

@thisismadani

Ali Madani

3 years

On Monday, my dear friend had 4 generations (aged 6mo-90yrs old) massacred by Israel. On Thursday, a @berkeleyMCB professor and @macfound “genius” @PolinaLishko tweets this. There should be no more space for unabashed racists in science.

Tweet media one

2

0

17

@thisismadani

Ali Madani

3 years

Protein sequence databases provide us samples that have defacto passed the fitness test of evolution and are information-rich. "Genetics search" is a retrieval step to find nearest-neighbors as defined by sequence alignment. Why do we need nearest-neighbors (NNs), you ask?

1

0

17

@thisismadani

Ali Madani

3 years

Check out our breast cancer AI model in @NatureComms . It was really neat to work on a recognition task that pathologists are not trained on. Can machine learning be used to find patterns in clinical imaging data that are too difficult for the human eye?

Tweet card media

Deep learning-enabled breast cancer hormonal receptor status determination from base-level H&E...

Nature Communications - Determination of estrogen receptor status (ERS) in breast cancer tissue requires immunohistochemistry, which is sensitive to the vagaries of sample processing and the...

1

1

16

@thisismadani

Ali Madani

3 years

It's been a pleasure working with such fantastic people: @benwkrause @EricRGreene1 @subramaniansk7 BenMohr JamesHolton @CaimingXiong @RichardSocher ZacharySun @fraser_lab @nikhil_ai Truly a team effort across multiple disciplines!

1

0

17

@thisismadani

Ali Madani

3 years

Can we go beyond directly borrowing NLP techniques for proteins? In this work, we utilize intuitions from biology to design a novel pretraining objective. In a controlled study, we show that it can outperform masked language modeling. Great work led by @PascalSturmfels

@KevinKaichuang

Kevin K. Yang 楊凱筌

@KevinKaichuang

3 years

Pretraining by predicting the profile HMM from a representative sequence. @PascalSturmfels , @jesse_vig , @thisismadani @nazneenrajani

Tweet media one

3

6

54

1

4

16

@thisismadani

Ali Madani

2 years

For narrow fitness landscapes, (1) scaling our model size leads to degrading performance (2) our smallest model outperforms an order-of-magnitude larger published model (3) the specific modeling choices utilized by baselines *seem* to not matter too much b/c we achieve SOTA

Tweet media one

1

1

16

@thisismadani

Ali Madani

2 years

The above suggests we need more focus on *data* and *evaluation*. In particular, the alignment of data distributions toward functional protein engineering [personal conjecture] The days of throwing raw (or at best seq-id downsampled) protein sequences to a model may be over ❌

2

2

16

@thisismadani

Ali Madani

21 days

Great thread by @jeffruffolo on OpenCRISPR!

@jeffruffolo

Jeff Ruffolo

22 days

One of the first projects we took on at @ProfluentBio was designing novel gene editing proteins with language models. This grew into an initiative called OpenCRISPR, and today I’m excited to share that work (including the sequences we designed)!

2

55

242

1

0

16

@thisismadani

Ali Madani

14 days

Join @jeffruffolo and Stephen Nayfach for their virtual talk and discussion next Tuesday

@ml4proteins

Machine learning for protein engineering seminar

14 days

Next Tuesday, 5/07 @ 4 pm EST, we're very excited to have @jeffruffolo @stephennaybach from @ProfluentBio present OpenCRISPR! Read the paper here: Sign up on our website to get the Zoom link via email next week!

0

9

43

0

2

15

@thisismadani

Ali Madani

3 years

Next up, the "embed" step. Essentially we need to transform all the protein sequences into vectors in a useful embedding space. #deepmind hasn't provided any details. But it's worth mentioning as an aside that this is extensively studied in ML and more recently in protein ML too

1

0

15

@thisismadani

Ali Madani

3 years

An attention-based technique was used, which has shown promise across ML in language/vision/etc. This allows for efficient learning (ie capturing relations between elements) and uncovering broader principles:

Attention Attention

lilianweng.github.io

1

0

14

@thisismadani

Ali Madani

3 years

GENhance was accepted to #NeurIPS2021 ! It's a really interesting task and technique for extrapolation in generative AI. The data and code are all available:

Tweet card media

GitHub - salesforce/genhance

Contribute to salesforce/genhance development by creating an account on GitHub.

@thisismadani

Ali Madani

3 years

Can generative AI learn to extrapolate? We explore how to generate sequences that enhance desired attributes-- beyond what was seen in training. Works pretty well in #NLP and #proteins ! Blog: Paper: Code:

4

24

112

0

0

13

@thisismadani

Ali Madani

1 year

There needs to be an award for transformative/counter-culture papers that ended up prevailing @BMarcusMcCann @RichardSocher

@_florianmai

Florian Mai 🇺🇳

1 year

Do people remember when @RichardSocher and his team proposed DecaNLP, a benchmark for testing the ability of a single model to solve a variety of NLP tasks? Many ICLR reviewers didn't see the value in "one model for everything" approaches and rejected it.

4

18

128

1

3

14

@thisismadani

Ali Madani

3 years

Truly impactful modern science/eng is a thankless, slow, accumulative process only achieved through a collective. There are so many discoveries by countless individuals that enabled the vaccine- explained below! Yet, we are conditioned to sensationalize Musk/Jobs forms of success

@bert_hu_bert

Bert Hubert 🇺🇦

3 years

In this post, we'll reverse-engineer the actual mRNA code of the @BioNTech_Group / @pfizer SARS-CoV-2 vaccine, character for character. And along the way, this will also explain how the vaccine works. Surprisingly, there are some fun mysteries in there!

357

6K

13K

0

1

12

@thisismadani

Ali Madani

2 months

Grateful for investors and angels who have been part of this journey: @sparkcapital @insightpartners @airstreet @aixventureshq @Fraser @nabeel @dylanmorris @nathanbenaich @JeffDean @RichardSocher @rhomsany @nlarusstone @BMarcusMcCann @LiamFedus and many more!

0

0

12

@thisismadani

Ali Madani

1 year

Do you have protein engineering needs and want to know if/how AI can help? At @SynBioBeta , get your questions answered in a brief 1-1 format. Speak directly with an AI scientist with practical experience in protein design. Ping me to schedule. Limited spots available.

2

1

13

@thisismadani

Ali Madani

3 years

Lastly, the model is differentiable and trained end-to-end (as opposed to a sequential pipeline of specialized models) ... allowing for loss propagation "holistically" through all internal representations.

1

0

13

@thisismadani

Ali Madani

3 years

LMK if there's something I missed or if you have any further interpretations! Even with the limited detail provided, it's a great case study into building cool models with solid intuitions. There's so much more work to be done in proteins, this is the first step! [n/n]

2

0

13

@thisismadani

Ali Madani

2 years

We start with the expected for LLMs. As models scale from 151M to 6.4B parameters, we are able to better capture the distribution of protein sequences drawn from observed evolutionary data

Tweet media one

1

1

13

@thisismadani

Ali Madani

4 months

Paper and open-source code from @MoreheadAlex 's internship at @ProfluentBio , presented at NeurIPS MLSB. Contact us if you're interested in an internship in ML and/or protein design!

@MoreheadAlex

Alex Morehead (何聪)

4 months

I'm excited to announce the results of my summer internship at @ProfluentBio : MMDiff, the first sequence-structure generative model of nucleic acid and protein complexes, presented at NeurIPS MLSB 2023.🧬 Paper: Code:

2

20

117

0

0

13

@thisismadani

Ali Madani

2 years

But hey scale-seekers, don’t despair. We see some evidence of LLMs utility for wider fitness landscapes (>3 mutations). We *may* also see emergent behavior requiring very large model size to identify high-fitness variants for low-homology, highly epistatic landscapes

Tweet media one

1

2

11

@thisismadani

Ali Madani

4 years

Researchers (especially reviewers) in the field still underestimate/discount the power of scale + proper data curation

@alexrives

Alex Rives

4 years

9/9 A first answer to the question about scaling laws. Relationship between language modeling fidelity and downstream performance is linear over the course of training! Suggests results will continue to improve with scale.

Tweet media one

3

0

11

1

1

11

@thisismadani

Ali Madani

2 years

Great work by the protein team at Meta! Genuinely interested in what nature has provided us and how much of it we have mapped out

1

0

12

@thisismadani

Ali Madani

2 years

Also for antibody fitness prediction, training on samples from immune repertoire sequencing (OAS) in theory sounds like a good idea, but in practice performs poorly

Tweet media one

1

1

12

@thisismadani

Ali Madani

3 years

Husam is one of my closest friends. I wish I could tweet about science only, but this is personal for me. Please DM me if you’ve never gotten a chance to learn about Palestine or open to recs of where to visit when in Israel. I promise it’s not complicated/awkward to speak about

0

0

11

@thisismadani

Ali Madani

9 days

I’ll be stopping by @SynBioBeta on Wednesday. Message me if you’d like to chat about OpenCRISPR or protein design more broadly.

3

2

29

@thisismadani

Ali Madani

2 years

We also assess fold- and antibody-specific generation ability for more focused generation of protein libraries

1

2

12

@thisismadani

Ali Madani

2 years

As always with research, this just scratches the surface but points to some exciting directions. It was a pleasure to work with @erik_nijkamp @jeffruffolo @EliWeinstein6 @nikhil_ai to finally get this project out!

5

0

10

@thisismadani

Ali Madani

8 months

Exciting and massive congrats! Hands down, I can’t recommend @airstreet enough. @nathanbenaich is genuinely one-of-a-kind and the best partner in your early journey as a founder, particularly in AI. You need him early on. This dude gets it 🚀🚀🚀

@nathanbenaich

Nathan Benaich

8 months

👋 I'm excited to unveil @airstreet ’s second fund of $121,212,121 as we accelerate our mission to back ambitious AI-first companies in North America and Europe! 🧵 My reflections on the journey, opportunity and what this means for our founders and community:

Tweet media one

91

56

704

1

0

11

@thisismadani

Ali Madani

3 years

Great discussion on equivariance and how a SE(3) Transformer could play a role for #alphafold2 . Wish Deepmind would just release a method’s preprint instead of this slow drip of info

@FabianFuchsML

Fabian Fuchs

3 years

There is still quite a bit of mystery around the details of @DeepMind 's AlphaFold 2, but equivariance & symmetries may have played a significant role in their success. This is @JustasDauparas 's and my take 🧐:

7

83

329

3

0

10

@thisismadani

Ali Madani

3 years

Check out our review of AI and Computer Vision for Medicine...a collaboration b/w @SFResearch @GoogleAI @StanfordAILab @scrippsresearch

@AndreEsteva

Andre Esteva, PhD

3 years

Just published in @Nature_NPJ DM, we review the impact computer vision and #AI have had on medicine, and recommend future directions. w/ @JeffDean , @EricTopol , @RichardSocher , @nikhil_ai , @thisismadani , @syeung10 , @samottaghi , @yun_liu , @katherinechou

Tweet media one

1

8

44

0

1

10

@thisismadani

Ali Madani

1 year

Really neat— it’s surprisingly difficult to find deep learning enabled predictive tasks in healthcare that would be largely impactful Wondering if there’s any indication/intuition that (2) is possible. Can you resolve that in ECG?

@oziadias

Ziad Obermeyer

1 year

This is: 1: A brilliant study— I would never have thought to do CTs on 10,000 healthy people; and never have predicted half have heart disease + mortality risk and 2: A great ML use case— predict CT results with their ECGs to create a cheap, population screening tool!

3

6

95

2

0

11

@thisismadani

Ali Madani

3 years

@jueseph Structure is overrated for most practical applications. Sequence is enough

5

0

9

@thisismadani

Ali Madani

2 years

This leads to exciting generation abilities. With large-scale language models, we can access a larger sequence and structural space for protein engineering

1

1

10

@thisismadani

Ali Madani

1 year

absolutely.

@gdb

Greg Brockman

1 year

Manual inspection of data has probably the highest value-to-prestige ratio of any activity in machine learning.

51

197

2K

0

0

10

@thisismadani

Ali Madani

2 years

🧘Patience✊ I'm just going to wait for the CASP15 results before making any conclusions re: any class of structure-prediction models. CASP is pretty awesome and we'd have no AlphaFold breakthrough without it. So wait we shall...

@MoAlQuraishi

Mohammed AlQuraishi

2 years

Last week’s OmegaFold () and ESMFold () present contrasting takes on how to fuse language models (LMs) with structure prediction. A short 🧵1/9

1

42

285

1

0

8

@thisismadani

Ali Madani

1 month

Just in the past month, we've had Devin from @cognition_labs , Claude 3 Opus from @AnthropicAI , Evo from @arcinstitute . Profluent has a couple drops chartered for this year in the pipeline as well. It's a pure thrill to build right now.

1

0

9

@thisismadani

Ali Madani

3 years

so what's the next big moonshot after #alphafold2 ? probably several directions.. would love to hear everyone's raw thoughts. tweet/RT below

@demishassabis

Demis Hassabis

3 years

Last year we presented #AlphaFold v2 which predicts 3D structures of proteins down to atomic accuracy. Today we’re proud to share the methods in @Nature w/open source code. Excited to see the research this enables. More very soon!

Tweet media one

89

2K

6K

4

0

8

@thisismadani

Ali Madani

3 years

#rosetta #foldx folk, is there a protein system that shows high correlation between simulated ddG values and experimentally-verified stability (e.g. a deep mutational scan dataset)?

2

1

9

@thisismadani

Ali Madani

2 years

this paper will last the test of time. the narrative and intuitions driving the research are solid. its backed by hard work in model experiments and data wrangling. congrats @NotinPascal and team!

@NotinPascal

Pascal Notin

2 years

Very pleased to announce that our paper “Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval” was accepted at ICML! Paper: Repo: (1/10)

Tweet media one

3

60

263

0

0

8

@thisismadani

Ali Madani

2 years

I'll be at 2 #NeurIPS posters. Let's chat together about these Qs: 1) GENhance: How do we extrapolate in sequence generation? Today 4:30p PT 2) FLIP: How do we evaluate models for protein fitness prediction? Fri 8:30a PT

0

1

9

@thisismadani

Ali Madani

3 years

@WedPants @KevinKaichuang Wherever @KevinKaichuang goes is _the_ place to do protein research

1

0

9

@thisismadani

Ali Madani

3 years

Deadline for the Salesforce 12-month AI Residency program is coming up soon! Looking for people from non-traditional backgrounds+experiences. If you've got experience/interest in Biology research areas, reach out to me directly.

Tweet card media

Blaze your trail with us at Salesforce.

careers.salesforce.com

0

2

9

@thisismadani

Ali Madani

3 years

Likely more exciting implications for protein language modeling as we have a definitive/real-world structure for a sequence

@koustuvsinha

Koustuv Sinha

3 years

[1/7] Excited to announce “Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little”. BERT gets high task scores due to its distributional prior rather than its ability to “discover the NLP pipeline”. #NLProc

Tweet media one

9

89

418

1

1

7

@thisismadani

Ali Madani

3 years

The residue-residue edges/interactions can then be transformed into distance matrices which describe the pairwise distances between each building block of a protein... essentially the crux of the protein structure prediction problem

1

0

8

@thisismadani

Ali Madani

3 years

Pushing Biomedical AI forward with a collaboration between @SFResearch and @UscEllison . @nikhil_ai @AndreEsteva @StrongDuality @DLRuderman @DavidAgus @RichardSocher

1

1

8