Ruiqi Gao @RuiqiGao Twitter profile

Pinned Tweet

Ruiqi Gao

1 year

📢 Check our new work (w/ @dpkingma ) on theoretical understanding of weighted diffusion objectives! See details in this thread 👇

Durk Kingma

@dpkingma

1 year

New theoretical work on diffusion objectives: We e.g. show that under a simple condition (monotonic weighting, satisfied by e.g. the v-prediction loss), diffusion objectives equal the ELBO with data augmentation, namely additive noise. 1/2

3

106

643

3

18

150

Last Seen Profiles

@KatyBentz

@HospitalRecords

@MattGiampoala

@ThuleanVibe

@TannerBarb16347

@meihuadang

@davidhoang

@hey_huevember

@VALINOTIRES

@rayssaleal

@breakfast_candy

@Sage19Kyle

@doxelangevin

@bannapafe5

@hina09652238758

@kumooo_oo

@LeMotLeJeu

@masflowchi

@PoetryDayUK

@Yotsubafour4

@EUEW_org

@barrelcats

@visiterlebhutan

@Madison_Loggins

@YGeriAn

@HitchcockLady

@LukeGrden

@z_i_b1

@AdrianeKru67729

@CBItxTourney

@JBsportscaster

@fu99okkakka

@igo_ok

@ekathimerini

@MichiganBucks

@SGUYBRAY

Ruiqi Gao

@RuiqiGao

3 years

Just finish my Ph.D. thesis filing! I am thrilled to announce that I’ll join Google Brain @GoogleAI as a research scientist and work with @dpkingma soon I am quite excited about it!

21

10

709

Ruiqi Gao

@RuiqiGao

2 years

🥳Thrilled to share Imagen Video: our new text-to-video diffusion model generating 1280x768 24fps HD videos! #ImagenVideo Website:

Jonathan Ho

@hojonathanho

2 years

Excited to announce Imagen Video, our new text-conditioned video diffusion model that generates 1280x768 24fps HD videos! #ImagenVideo Work w/ @wchan212 @Chitwan_Saharia @jaywhang_ @RuiqiGao @agritsenko @dpkingma @poolio @mo_norouzi @fleet_dj @TimSalimans

58

734

3K

10

66

409

Ruiqi Gao

@RuiqiGao

2 years

Impressed by the amazing progress of diffusion models? 📢 Check our #CVPR2022 tutorial on #diffusion_models next Sunday morning! w/ @karsten_kreis & @ArashVahdat Agenda: An #Imagen (awesome diffusion models) preview of what to expect 😉:

6

66

378

Ruiqi Gao

@RuiqiGao

5 months

🔥Latent Diffusion Models Tutorial🔥 #NeurIPS2023 @karsten_kreis , @ArashVahdat and I will present the tutorial on latent diffusion models tomorrow! Save the location & time, and excited to see you all there! Tomorrow, Monday, 9:45am-12:15pm, Hall E2.

Latent Diffusion Models: Is the Generative AI Revolution Happening in Latent Space?

Tutorial in Conjunction with NeurIPS 2023

neurips2023-ldm-tutorial.github.io

3

58

344

Ruiqi Gao

@RuiqiGao

1 year

📢 📢 We are looking for a strongly motivated student researcher for this summer in my team ( @GoogleAI brain), working on diffusion models, EBMs and related topics! Email me with your CV or website if you’re interested: ruiqig @google .com

9

26

328

Ruiqi Gao

@RuiqiGao

5 months

Looking for diffusion model advancements at #NeurIPS2023 ? Come to check our oral work "Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation" w/ @dpkingma . New theoretical understanding, SOTA empirical results, and more! Arxiv:

6

30

311

Ruiqi Gao

@RuiqiGao

2 years

The preliminary slides of our #CVPR2022 tutorial on #diffusion_models are available on our website: Looking forward to seeing you, w/ @karsten_kreis & @ArashVahdat , at 8:30 CDT on Sunday in New Orleans or virtually!

3

60

266

Ruiqi Gao

@RuiqiGao

3 years

Pleased to share our new work on learning energy-based models: By maximizing recovery likelihoods on increasingly noisy data, the MCMC becomes more tractable. We achieve (1)high quality samples (2)stable long-run chains (3)estimated likelihoods. (1/n)

2

43

231

Ruiqi Gao

@RuiqiGao

2 years

🥳Check the re-recording of our tutorial on denoising diffusion models on YouTube: The tutorial was originally presented at #CVPR2022 with @karsten_kreis @ArashVahdat . Slides and additional info: Hope you enjoy the learning!!😃

1

35

221

Ruiqi Gao

@RuiqiGao

6 months

📢 Booking your NeurIPS trip? Wondering the advancement of latent diffusion models, and how they have been applied to various domains? @karsten_kreis , @ArashVahdat and I will present the #NeurIPS2023 tutorial on "Latent Diffusion Models" on Mon, Dec 11.

Latent Diffusion Models: Is the Generative AI Revolution Happening in Latent Space?

Tutorial in Conjunction with NeurIPS 2023

neurips2023-ldm-tutorial.github.io

4

28

198

Ruiqi Gao

@RuiqiGao

5 months

Excited to see people coming for our #NeurIPS23 tutorial on “latent diffusion models” at Hall E2. This happening. Hope you’d enjoy it! ☺️

4

10

169

Ruiqi Gao

@RuiqiGao

4 years

(1/2) Interested in understanding towards grid cells? Check our preprint "A Representational Model of Grid Cells Based on Matrix Lie Algebras" (with @jianwen_xie , SC Zhu, YN Wu), with learned regular hexagon grid patterns!

6

33

148

Ruiqi Gao

@RuiqiGao

4 years

Check out our work "Flow Contrastive Estimation of Energy-Based Models" at #CVPR2020 (with @erik_nijkamp , @dpkingma , Z Xu, @iamandrewdai , YN Wu): , a scalable method for learning EBM, w/o MCMC. Oral session: 2.3B, 2-4pm (PDT) Wed. Glad to get in touch!

1

17

125

Ruiqi Gao

@RuiqiGao

2 years

Excited to get this tutorial proposal accepted to #CVPR2022 . Stay tuned! 😄

Arash Vahdat

@ArashVahdat

2 years

🎉 Our tutorial proposal on 𝗗𝗲𝗻𝗼𝗶𝘀𝗶𝗻𝗴 𝗗𝗶𝗳𝗳𝘂𝘀𝗶𝗼𝗻-𝗯𝗮𝘀𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝘃𝗲 𝗠𝗼𝗱𝗲𝗹𝗶𝗻𝗴 has been accepted to #CVPR2022 . Stay tuned for some cool lectures from @karsten_kreis , @RuiqiGao and me at @CVPR 2022.

3

73

513

2

12

109

Ruiqi Gao

@RuiqiGao

10 months

Spending 1/4 of my birthday flying towards Honolulu. Looking forward to catching up with people! 😊

12

2

106

Ruiqi Gao

@RuiqiGao

2 years

Finally this work has been accepted to #ICLR2022 . MCMC could mix, as long as it is in the latent space :).

Ruiqi Gao

@RuiqiGao

4 years

Check our work "Learning Energy-based Model with Flow-based Backbone by Neural Transport MCMC"(w/ @erik_nijkamp , etc.): . We learn EBM by exp. tilting of flow, with mixing MCMC chains for 2000+ steps! Very excited that long run MCMC finally works for EBM.

0

3

30

1

3

76

Ruiqi Gao

@RuiqiGao

1 year

Super excited to be at #NeurIPS2022 from Wed to Sat this week! Looking forward to catching up with people. DM me if you're around and I'd love to chat! 😃

1

0

50

Ruiqi Gao

@RuiqiGao

11 months

Come to check our award candidate paper at #CVPR2023 about distillation of guided diffusion models in the afternoon poster session tomorrow (poster #186 )! Excited to chat about how to make sampling of diffusion models faster 🥳

Chenlin Meng

@chenlin_meng

11 months

We will be presenting our @CVPR Award Candidate paper On Distillation of Guided Diffusion Models tomorrow Wed 21 Jun 4:30-6 p.m. at West Building Exhibit Halls ABC 186! Super honored that our paper is selected as one of the 12 award candidates! 🙏 #CVPR2023

7

23

144

1

6

51

Ruiqi Gao

@RuiqiGao

9 months

If you missed Durk’s talk at SPIGM @ICML2023 , check the recording, slides, and the latest version of our paper under this thread! 😃👇

Durk Kingma

@dpkingma

9 months

Thanks to the SPIGM organizers and those who attended! If you're an ICML registree but missed my talk, you can find it at 2:21:41 under this link: Our updated paper (including new results) will appear on arXiv on Aug 1. I'll also post the slides then.

1

12

105

0

7

47

Ruiqi Gao

@RuiqiGao

4 years

Our work "Flow Contrastive Estimation of Energy-Based Models" with @erik_nijkamp , @dpkingma , Z Xu, @iamandrewdai , YN Wu at #NeurIPS2019 : . Glad to get in touch!

Erik Nijkamp

@erik_nijkamp

4 years

Our work "Flow Contrastive Estimation of Energy-Based Models" ( @RuiqiGao , @erik_nijkamp , @dpkingma , Z. Xu @iamandrewdai , YN Wu) at #NeurIPS2019 : We show (1) joint learning of EBM & Glow, (2) correction of over-dispersion, (3) competitive semi-super cls.

1

17

82

2

4

41

Ruiqi Gao

@RuiqiGao

3 years

⏰Check our poster later today at #ICLR2021 ! A tractable MCMC based learning method of energy-based models. time: 5-7pm PDT link: paper:

Learning Energy-Based Models by Diffusion Recovery Likelihood

While energy-based models (EBMs) exhibit a number of desirable properties, training and sampling on high-dimensional datasets remains challenging. Inspired by recent progress on diffusion...

openreview.net

Ruiqi Gao

@RuiqiGao

3 years

Pleased to share our new work on learning energy-based models: By maximizing recovery likelihoods on increasingly noisy data, the MCMC becomes more tractable. We achieve (1)high quality samples (2)stable long-run chains (3)estimated likelihoods. (1/n)

2

43

231

2

6

36

Ruiqi Gao

@RuiqiGao

5 months

Looking forward to seeing you there and discuss diffusion models!

Bahjat Kawar

@bahjat_kawar

5 months

Join us on Friday at Hall B1 for the #NeurIPS2023 workshop on diffusion models! We are honored to host a panel of experts @ArashVahdat @RuiqiGao @robrombach @sedielem Have questions for the panel? Use this link to submit:

0

4

35

1

2

31

Ruiqi Gao

@RuiqiGao

4 years

Check our work "Learning Energy-based Model with Flow-based Backbone by Neural Transport MCMC"(w/ @erik_nijkamp , etc.): . We learn EBM by exp. tilting of flow, with mixing MCMC chains for 2000+ steps! Very excited that long run MCMC finally works for EBM.

Erik Nijkamp

@erik_nijkamp

4 years

(1/2) "Learning Energy-based Model with Flow-based Backbone by Neural Transport MCMC" ( @erik_nijkamp *, @RuiqiGao *, P. Sountsov, S. Vasudevan, @bo_pang0 , S.-C. Zhu, Y. N. Wu): We show (1) exponential tilting of Glow, (2) mixing MCMC, (3) improved synthesis.

6

12

82

0

3

30

Ruiqi Gao

@RuiqiGao

2 years

In our new work led by brilliant @chenlin_meng , guided diffusion models can be distilled to very few sampling steps with no loss on perceptual quality! It is also adopted by #ImagenVideo and works pretty well. Check the details in Chenlin’s thread 👇.

Chenlin Meng

@chenlin_meng

2 years

Excited to share our work "On distillation of guided diffusion models"! Our distillation approach allows classifier-free guided diffusion models to generate high-quality samples using as few as 1-4 sampling steps😮

10

89

585

0

4

20

Ruiqi Gao

@RuiqiGao

5 months

📢 Check our latest work ReconFusion, 3D reconstruction of realistic scenes with 2D diffusion priors!

Aleksander Holynski

@holynski_

5 months

Excited to share ReconFusion! 3D reconstruction of real-world scenes from only a few photos, powered by diffusion priors: w/ amazing team @ChrisWu6080 @BenMildenhall @philipphenzler @KeunhongP @RuiqiGao @watson_nn @_pratul_ @dorverbin @jon_barron @poolio

9

59

357

0

1

15

Ruiqi Gao

@RuiqiGao

1 year

Come to check Chenlin’s presentation on our paper “On distillation of Guided Diffusion Models” at #NeurIPS22 workshop on score-based models this afternoon! 👇

Chenlin Meng

@chenlin_meng

1 year

Interested in making diffusion models fast? I’ll present our paper On Distillation of Guided Diffusion Models and other interesting works at #NeurIPS workshop on score-based methods this afternoon. 📍/🕜: room 293-294 / 2:30pm - 2:40pm and 3:00pm-4:30pm

2

3

51

0

1

15

Ruiqi Gao

@RuiqiGao

3 months

Exciting!

Jon Barron

@jon_barron

3 months

We just finished a joint code release for CamP () and Zip-NeRF (). As far as I know, this code is SOTA in terms of image quality (but not speed) among all the radiance field techniques out there. Have fun!

23

156

952

0

12

Ruiqi Gao

@RuiqiGao

5 months

🥳🥳

Google DeepMind

@GoogleDeepMind

5 months

We’re excited to announce 𝗚𝗲𝗺𝗶𝗻𝗶: @Google ’s largest and most capable AI model. Built to be natively multimodal, it can understand and operate across text, code, audio, image and video - and achieves state-of-the-art performance across many tasks. 🧵

174

2K

6K

0

11

Ruiqi Gao

@RuiqiGao

5 months

@poolio ’s talk is happening in 12mins at Room 214!

Ben Poole

@poolio

5 months

Drop by the workshop today at #NeurIPS2023 to learn about the intersection of deep learning and inverse problems! I'll be presenting our work on diffusion priors for 3D generation and reconstruction at 11:30.

2

4

58

1

0

11

Ruiqi Gao

@RuiqiGao

3 years

A joint work with my amazing collaborators @YSongStanford @poolio Ying Nian Wu and @dpkingma ! We will release the code soon. (8/n)

0

10

Ruiqi Gao

@RuiqiGao

3 years

Deepest gratitude to my amazing advisors Song-Chun Zhu and Ying Nian Wu! This would not have been possible without your guidance and support. Thanks to all my colleagues and collaborators!

0

10

Ruiqi Gao

@RuiqiGao

5 months

For practitioners, based on this understanding, we propose new weighting functions, that lead to SOTA FID scores on high-res image generation. Ready for use on your own diffusion models with a minimal change of code and get performance boost!

0

2

9

Ruiqi Gao

@RuiqiGao

3 years

Key to our method is that we maximize the recovery likelihood: conditional probability of data at a noise level given their versions at a higher noise level. Compared to marginal likelihood, the MCMC is more tractable for the relatively concentrated conditional distribution.(2/n)

1

0

9

Ruiqi Gao

@RuiqiGao

2 years

Our #NeurIPS2021 paper "On Path Integration of Grid Cells: Group Representation and Isotropic Scaling" ☑️Theoretical analysis of general RNN models ☑️Algebraic & geometric structure of linear models ☑️Clear learned hexagon patterns Session today 4:30PT:

1

0

8

Ruiqi Gao

@RuiqiGao

1 year

Just to clarify, the student researcher position is targeted at current PhD students and it’s short-term (very similar to research intern), not for full-time job after graduation.

0

8

Ruiqi Gao

@RuiqiGao

4 years

Very interesting and solid work!

Durk Kingma

@dpkingma

4 years

New theory paper: . We show rigorously that EBMs of the type E(x|y) = f(x)•g(y) are, under fairly mild conditions, (1) identifiable in functions f and g, and (2) universal conditional density approximators, generalizing previous forms of nonlinear ICA.

4

32

255

0

1

8

Ruiqi Gao

@RuiqiGao

3 years

Although learning from conditional distributions, this estimation method is theoretically consistent in terms of estimating marginal distributions at each noise level, given sufficient data. This is also verified on 2D examples. (4/n)

1

0

7

Ruiqi Gao

@RuiqiGao

5 months

Highly recommend to check this thread for more insightful thoughts about ReconFusion!

Ben Mildenhall

@BenMildenhall

5 months

ReconFusion = standard single-scene optimized 3D reconstruction, additionally guided by a multi-view diffusion prior to allow for decent outputs from significantly fewer input views. some thoughts below for view synthesis fanatics... 1/

6

49

419

0

7

Ruiqi Gao

@RuiqiGao

5 months

For theory enthusiasts, we show that existing diffusion training objectives can be understood as the ELBO with data augmentation, under mild assumptions. It allows for apple-to-apple comparison of DMs with other likelihood-based models, e.g., autoregressive models.

1

2

7

Ruiqi Gao

@RuiqiGao

3 years

We learn a sequence of EBMs trained on increasingly noisy versions of a dataset. After training, samples are generated by progressively sampling from the conditional distributions at decreasingly lower noise levels. (3/n)

1

0

6

Ruiqi Gao

@RuiqiGao

3 years

We make connections of our method to denoising score matching (Song & Ermon, 2019, 2020) and denoising diffusion probabilistic models (Ho et al. 2020). They correspond to the normal approximation of the recovery likelihood. (7/n)

1

0

6

Ruiqi Gao

@RuiqiGao

3 years

Moreover, the long-run MCMC samples do not diverge and still represent realistic images, allowing us to accurately estimate the normalized density of data. Our model obtains 3.18 bits/dim, the first competitive estimated log-likelihood using EBMs. (6/n)

1

0

5

Ruiqi Gao

@RuiqiGao

2 years

@BAristimunha @Warvito @karsten_kreis @ArashVahdat Yes it will be recorded! It will also be livestreamed via zoom that is available to all cvpr attendees.

2

0

5

Ruiqi Gao

@RuiqiGao

3 years

On CIFAR-10, our methods achieve FID 9.60, outperforming existing methods of learning pure energy-based models by a large margin. (5/n)

1

0

4

Ruiqi Gao

@RuiqiGao

2 years

@DocXavi @ArashVahdat @karsten_kreis Thanks for your interest in our tutorial! ☺️

1

0

4

Ruiqi Gao

@RuiqiGao

4 years

@erik_nijkamp @dpkingma @iamandrewdai Welcome to our poster & spotlight sessions at BDL workshop on Friday at #NeurIPS2019 :

0

4

Ruiqi Gao

@RuiqiGao

5 months

@ArashVahdat Wow wow whose cat is so cute ☺️

0

4

Ruiqi Gao

@RuiqiGao

2 years

@zhoubolei I tested negative for the first three days after coming back and with symptoms, but now positive 🤒️. Hope it’s not the case for you 🙏

0

4

Ruiqi Gao

@RuiqiGao

6 months

It also includes a panel discussion with an awesome group of panelists: @dpkingma , @talidekel , @robrombach , @chenlin_meng , and Ying Nian Wu. Looking forward to seeing you there in New Orleans! 🥳🎷

1

0

3

Ruiqi Gao

@RuiqiGao

3 years

Interesting work that combines VAE with neuroscience and improves identifiability.

Xue-Xin Wei

@weixx2

3 years

NeurIPS time: Check our new paper on exacting identifiable neural manifolds by leveraging both neural and behavioral data, “Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE”, with @DingZhou2 .

3

16

0

3

Ruiqi Gao

@RuiqiGao

2 years

Amazing work by colleagues at Brain! Diffusion models are becoming incredibly powerful.

Chitwan Saharia

@Chitwan_Saharia

2 years

We are thrilled to announce Imagen, a text-to-image model with unprecedented photorealism and deep language understanding. Explore and Imagen! A large rusted ship stuck in a frozen lake. Snowy mountains and beautiful sunset in the background. #imagen

61

316

2K

0

3

Ruiqi Gao

@RuiqiGao

5 months

@BenMildenhall Wow such an insightful summary! 🤩

0

3

Ruiqi Gao

@RuiqiGao

2 years

@iPrabhavKaula @karsten_kreis @ArashVahdat We’ll post the video on our website soon. Stay tuned! ☺️

2

0

1

Ruiqi Gao

@RuiqiGao

5 months

Don't miss this great opportunity to work with brilliant @wgrathwohl !

will grathwohl

@wgrathwohl

5 months

Very excited to be hosting a student researcher on my team at Google Deepmind next year. I'm looking to work with a PhD student who has a background in generative models and sampling. If you think that describes you, please apply here!

1

7

78

0

2

Ruiqi Gao

@RuiqiGao

4 years

@arjunreddy2613 @schangpi @GoogleAI Congrats Arjun! Very well done. Haven’t received that hat this year...

0

2

Ruiqi Gao

@RuiqiGao

4 months

@msalbergo @KyleCranmer Woohoo big congrats Michael!!

1

0

2

Ruiqi Gao

@RuiqiGao

2 years

@XPe79361163 @DocXavi @ArashVahdat @karsten_kreis Hi Xiong, thanks for your interest in our tutorial! We will post the video on our website soon. Stay tuned!

0

2

Ruiqi Gao

@RuiqiGao

10 months

@JeffDean @ayirpelle Thank you!!

0

2

Ruiqi Gao

@RuiqiGao

2 years

@nahidalam @karsten_kreis @ArashVahdat We’ll post the video on our website soon. Stay tuned! ☺️

0

2

Ruiqi Gao

@RuiqiGao

4 years

(2/2) We show (1) a representational model of grid cells with two coupled matrix Lie algebras, (2) connection of path integral and dim reduction models via group rep. theory , (3) learned hexagon patterns & faithful path integral.

1

0

2

Ruiqi Gao

@RuiqiGao

1 year

@poolio 🙏🙏🙏

0

2

Ruiqi Gao

@RuiqiGao

3 years

@dpkingma @GoogleAI Thanks, Durk!

0

1

Ruiqi Gao

@RuiqiGao

2 years

@wgrathwohl @liyzhen2 👏👏👏

0

1

Ruiqi Gao

@RuiqiGao

10 months

@hyungjin_chung Thanks Hyungjin!

0

1

Ruiqi Gao

@RuiqiGao

4 years

@weixx2 @jianwen_xie Thanks! We are greatly inspired by your pioneering work on grid cells. Thanks for sharing your recent work, which we shall cite and discuss in the revision. I think your results of HD cells are quite meaningful. It seems to resemble place cells.

0

1

Ruiqi Gao

@RuiqiGao

5 months

@chenlin_meng @dpkingma Thanks Chenlin!!♥️

0

1

Ruiqi Gao

@RuiqiGao

1 year

@shaneguML @GoogleAI @OpenAI @johnschulman2 Congrats on your new journey Shane!!

0

1

Ruiqi Gao

@RuiqiGao

3 years

@YSongStanford @jaschasd @dpkingma @studentofml @StefanoErmon @poolio Woohoo congrats Yang!

0

1

Ruiqi Gao

@RuiqiGao

5 months

@dpkingma lol looks like a very nice piece of bio

0

1

Ruiqi Gao

@RuiqiGao

2 years

☑️accurate path integration paper: code: with wonderful collaborators @jianwen_xie @weixx2 S.C. Zhu, Y.N. Wu

0

Ruiqi Gao

@RuiqiGao

3 years

@douglas_eck @GoogleAI @dpkingma Thanks, Douglas!

0

1

Ruiqi Gao

@RuiqiGao

1 year

Sorry didn’t open DM previously 😅, now it’s open! Feel free to shoot me a message ;).

0

1

Ruiqi Gao

@RuiqiGao

2 years

@_NicT_ @karsten_kreis @ArashVahdat Hi Nicholas, thanks for your interest and the summary for our tutorial! Yes you definitely have our permission to use the slides with credit ☺️.

0

1

Ruiqi Gao

@RuiqiGao

3 years

@wgrathwohl Congrats!!

0

1

Ruiqi Gao

@RuiqiGao

5 months

@robrombach @dpkingma Haha thanks Robin!