Animesh Garg @animesh_garg Twitter profile | Pikagi

Pikagi

Animesh Garg

@animesh_garg

21,109

Followers

1,025

Following

462

Media

1,955

Statuses

Foundation Models for Generalizable Autonomy. Assistant Professor in AI Robotics @GeorgiaTech + @NvidiaAI . prev @Stanford @berkeley_ai @UofTCompSci

Atlanta / San Francisco

https://t.co/iB4qyjcrKd

Joined January 2010

Don't wanna be here? Send us removal request.

Pinned Tweet

@animesh_garg

Animesh Garg

2 months

The Path to Autonomy is paved with Simulation! The focus on building the best data engines for Embodied AI & Robotics has been a core mission at @NVIDIARobotics I am so excited to share that the simulation effort we have been working on for the last 2+ years will be part of…

3

9

90

Last Seen Profiles

@Chanceliga

@footprint

@NeoBSTx2

@igiworldwide

@astrifera__

@nikhildwivedi11

@matchabuns

@useGrow

@RefMichal

@IIIllOlo

@glaciel0602

@_onigriman

@ZUSB10

@oitebal

@gobricker

@DemandezaISDE

@mochaswisss

@ophency

@GSeason6

@tractorbeeem

@KevinSVieira

@drshaunnn

@T_HighAthletics

@gusbourne

@ZSKberlin

@makunouchi4mark

@haunanidoll

@LonelyWriterPod

@NafathRE

@mulaahoneyy

@chapojr18

@grupomexico

@Komethahzine

@Veracitii

@ITrinny

@arjpnrvw

@animesh_garg

Animesh Garg

3 years

As @NeurIPSConf #NeurIPS2021 deadline approaches, I've consolidated some technical writing tips. Succinct answers to each of the questions, Voila, you have a crisp Introduction! Expand on prior work, intuition/tech details, & expts for the rest ensuring the why is always clear

Tweet media one

12

202

971

@animesh_garg

Animesh Garg

3 years

We organized a seminar-style course @UofTCompSci on 3D and Geometric Deep Learning. Here is the reading list (Videos + Slides): Each paper comes with a 10-min tutorial: Hope it helps folks looking to get up to speed on the topic!

Tweet media one

9

180

777

@animesh_garg

Animesh Garg

1 year

Career update: I will be moving to @ICatGT next year. I look forward to working alongside the exceptional researchers at @GTrobotics and @mlatgt

Tweet media one

54

25

753

@animesh_garg

Animesh Garg

3 years

The standards of what ⁦ @Twitter ⁩ thinks is machine learning have dropped somewhat. Yet for some reason 1k+ people like a thread on definition of log. ...and here I am tweeting "excited to share our latest paper on....." I prolly need education on HowTo Twitter

Tweet media one

63

37

530

@animesh_garg

Animesh Garg

3 years

Ever wondered if we solved RL in continuous time, would it be better for robotics. We find in many cases, Yes! Value Iteration in Continuous Actions, States, and Time @_mlutter @MannorShie @Jan_R_Peters Dieter Fox @icmlconf #ICML2021 @NVIDIAAI

5

57

378

@animesh_garg

Animesh Garg

3 years

Kinda definitive guide to FAQs for students evaluating Ph.D. offers! Always ask: Are there snacks in the lab? Advisors: need to invest in munchies! thx to @igilitschenski for sharing this! Original:

Tweet media one

Tweet media two

Tweet media three

6

66

345

@animesh_garg

Animesh Garg

2 years

My thoughts on the Optimus Effort at AI day @Tesla I made some notes on my takeaways.

@animesh_garg

Animesh Garg

2 years

front row seats at AI day at Tesla remark 1 setting expectations for optimus follow along..for more

Tweet media one

5

9

104

29

47

331

@animesh_garg

Animesh Garg

6 months

Nvidia Robotics Research internships for summer 2024 are now open If you are interested in learning for control, RL, and especially Foundation Models: VLM and LLMs for decision making. DM me for more and apply here BTW, these are located in both Santa…

13

35

329

@animesh_garg

Animesh Garg

4 years

"Physically Embedded Planning Problems: New Challenges for Reinforcement Learning" the lack of reference to robotics folks working on this for decades and reinventing the problem as your own! gotta do better @DeepMind How "New" is this?

3

48

259

@animesh_garg

Animesh Garg

4 years

Isaac Gym - @NVIDIAAI physics simulation environment for reinforcement learning research (preview Release) - End-to-End GPU accelerated - Isaac Gym tensor-based APIs fo massively parallel sim Also get in touch for potential internships to flex in Gym!

8

44

243

@animesh_garg

Animesh Garg

3 years

Reviewers in ML continue the trend! S4RL is shot down for being too simple and obvious! All Reviewers "great work, strong empirically in many domains, but there is little fancy-pants theory!" reject🚫 they missed the memo: Surprisingly Simple!

Tweet card media

S4RL: Surprisingly Simple Self-Supervision for Offline...

Offline reinforcement learning proposes to learn policies from large collected datasets without interacting with the physical environment. These algorithms have made it possible to learn useful...

@jachiam0

Joshua Achiam ⚗️

3 years

Hot take: deep RL research has stagnated because conferences have created bad incentives, rewarding researchers for vacuous claims of novelty, tenuous-at-best theoretical connections, or SOTA, while punishing boring analysis of the empirical tricks that actually make things work.

16

99

841

8

16

214

@animesh_garg

Animesh Garg

4 years

Semi-supervised Learning to generate high-resolution images with disentangled latent codes using less <2% labelled data Paper: w\ @wn8_nie , T.Karras, @shoubhikdn , A.Patney, A.Patel, @AnimaAnandkumar Real win: can make your advisor happier ( @drfeifei )

Tweet media one

Tweet media two

2

54

213

@animesh_garg

Animesh Garg

5 years

Our work on Neural Task Graphs is accepted at CVPR 2019 as Oral 😁 ⁦ @drfeifei ⁩ ⁦ @deanh619 ⁩ ⁦ @danfei_xu ⁩ ⁦ @yukez ⁩ @jcniebles ⁩ ⁦ @StanfordSVL ⁩

Tweet card media

Neural Task Graphs: Generalizing to Unseen Tasks from a Single...

Our goal is to generate a policy to complete an unseen task given just a single video demonstration of the task in a given domain. We hypothesize that to successfully generalize to unseen complex...

5

27

205

@animesh_garg

Animesh Garg

7 years

Our new paper on Neural Task Programming: Learning to Generalize Across Hierarchical Tasks

Tweet media one

3

68

204

@animesh_garg

Animesh Garg

6 years

Super cool new paper from the @NvidiaAI Robotics group in Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience [Paper , video: ]

Tweet media one

2

78

204

@animesh_garg

Animesh Garg

1 year

Object-oriented world models is *the* key for reasoning. But, unsupervised task-agnostic methods are hard! SlotFormer, at ICLR2023, is an unsupervised video prediction model that also works for tasks: VQA and model-based planning Read on for more!

7

28

191

@animesh_garg

Animesh Garg

3 years

We are co-organizing #NeurIPS2021 workshop on Differential Equations and PDEs. Promises to be a very exciting agenda on a topic increasingly getting popular among ML folks. Sept 17 deadline for contributed papers stay tuned for more!

@PatrickKidger

Patrick Kidger

3 years

Our 'The Symbiosis of Deep Learning and Differential Equations' workshop has been accepted for #NeurIPS2021 ! Send us your work on data-driven dynamical systems, neural differential equations, solving PDEs with deep learning etc. Tentative submission deadline Sept. 17.

4

51

307

1

29

172

@animesh_garg

Animesh Garg

3 years

We just won the Best student paper at #RSS2021 @RoboticsSciSys 🥳🎉🍾 Check it out here: Video: Time to cut some cake!

@eric_heiden

Eric Heiden

3 years

Very honored that DiSECt won a Best Student Paper Award at #RSS2021 ! Congrats to my co-authors @milesmacklin , Yashraj Narang, Dieter Fox, @animesh_garg and Fabio Ramos, and thanks for this great collaboration @NVIDIAAI ! 🎉

12

6

88

11

10

170

@animesh_garg

Animesh Garg

4 years

Deep RL is not really using deep networks! We found that dense connections and deeper networks help improve learning performance. Results hold across various manipulation and locomotion tasks, for both proprioceptive and image observations!

Samarth Sinha*, Homanga Bharadhwaj*, Aravind Srinivas, and Animesh Garg University of Toronto, Vector Institute, University of Toronto Robotics Institute University of California Berkeley

sites.google.com

@_sam_sinha_

Samarth Sinha

4 years

Deep learning has seen huge gains when you increase the number of layers, but what about Deep RL? Introducing D2RL! Changing how you parameterize your policy + Q function boosts performance Co-led with @mangahomanga . @AravSrinivas @animesh_garg Link:

Tweet media one

Tweet media two

Tweet media three

8

27

160

1

32

165

@animesh_garg

Animesh Garg

4 years

32 for a single conference! And 13 accepts 😱 @svlevine you are making everyone else look like they are not even working

@shaohua0116

Shao-Hua Sun

4 years

#ICLR2020 authors with > 5 submissions: 32 Sergey Levine 20 Yoshua Bengio 16 Cho-jui Hsieh 14 Pieter Abbeel 13 Liwei Wang, Tom Goldstein, Chelsea Finn, Bo Li, Jun Zhu # of accepted papers: 13 Sergey Levine 7 Le Song, Jun Zhu 6 Cho-jui Hsieh, Jimmy Ba, Liwei Wang, Pushmeet Kohli

Tweet media one

15

47

350

3

17

163

@animesh_garg

Animesh Garg

4 years

Sometimes you gotta give it to TAs. The guy below has a plan for doing a discussion section in case of a nuclear attack, COVID19 seems less harrowing! "No matter what happens, I will not give up on you" 🤯

@jennafranke

Jenna ⌬

4 years

Sounds like Berkeley instructors are handling #COVID19 well 😂

Tweet media one

260

11K

46K

4

10

164

@animesh_garg

Animesh Garg

4 years

This needs to be said out loud, particularly by gatekeepers of AI/ML academia (Profs, advisors, mentors, reviewers, SACs, ACs) Novelty for the sake of it is not a virtue! Many students trying to be different for the sake of novelty and often sacrificing utility in the process!

@jbhuang0604

Jia-Bin Huang

4 years

I still remember I marveled at how simple and elegant the Lucas-Kanade tracker is when I first learned about it as a grad student. Here is a fun story about it. "Newness itself is not a virtue, usefulness is." - Takeo Kanade

7

97

513

5

23

158

@animesh_garg

Animesh Garg

4 years

Interested in how "Deep Learning meets Differential equations"? Join us for @iclr_conf workshop on Apr 26. Streaming and discussion platform details to be posted on workshop website. @AnimaAnandkumar @TanNguyen689

Tweet media one

Tweet media two

2

28

155

@animesh_garg

Animesh Garg

5 years

Excited to be at #ICRA2019 Best Paper Award talk Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks Paper: Video:

Tweet card media

Making Sense of Vision and Touch (Old)

Newer Video found here: https://www.youtube.com/watch?v=usFQ8hNtE8c&feature=youtu.be

www.youtube.com

5

33

154

@animesh_garg

Animesh Garg

3 years

Every once in a while authors are bold enough to break the mold and surprise reviewers. a 4 page paper on the effectiveness of patches in deep learning! tl;dr: we had an idea and it works damn well! grabbing🍿to see reviews!!

Tweet media one

Tweet media two

@cHHillee

Horace He

3 years

A very ... interesting 4 page paper at ICLR. I'm curious to see the reviewers' reactions.

Tweet media one

19

93

656

1

17

147

@animesh_garg

Animesh Garg

5 years

We won the best paper award @michellearning @yukez @leto__jean @drfeifei @silviocinguetta

Tweet media one

11

12

149

@animesh_garg

Animesh Garg

3 years

Exciting week in robot manipulation! After the Berkshire Grey IPO, Google deems that its effort is ready to move out of X. Intrinsic, a new alphabet entity to focus on industrial robotics. Rather exciting time to be working in robot manipulation. 🦾

Tweet card media

Introducing Intrinsic

Unlocking the creative and economic potential of industrial robotics for millions more businesses, entrepreneurs, and developers

2

13

142

@animesh_garg

Animesh Garg

4 years

Scaling crowdsourcing robotics with Human Reasoning & Dexterity for Large-Scale Dataset Creation Blog: Paper: Webpage: @AjayMandlekar @yukez @silviocinguetta @drfeifei J. Booher, M. Spero, A. Tung, A. Gupta

Tweet card media

Scaling Robot Supervision to Hundreds of Hours with RoboTurk:...

Large, richly annotated datasets have accelerated progress in fields such as computer vision and natural language processing, but replicating these successes in robotics has been challenging....

1

41

142

@animesh_garg

Animesh Garg

5 years

NeurIPS 2019 Paper IDs are already over 8900! 😱

6

24

140

@animesh_garg

Animesh Garg

4 years

Learning Causal Graphs that capture Physical Systems has high potential yet challenging! Check out End-to-End Causal Discovery from videos Site: Paper: w\ @YunzhuLiYZ @AnimaAnandkumar , A.Torralba, D. Fox

2

32

139

@animesh_garg

Animesh Garg

26 days

Robot learning from BC based models has really come a long way, and continues to surprise me. The complexity of real world tasks achievable with BC based methods on real data sidesteps the challenges in RL, particularly the sim2real aspects! The new release of ALOHA Unleashed…

2

18

135

@animesh_garg

Animesh Garg

5 years

Perks of working at @NvidiaAI ;) arguably the most kick-ass simulator is in-house and now for everyone else as well. Going to be at = #NeurIPS2018 in case you are interested in chatting about opportunities in ML, Vision, and Robotics.

@NVIDIARobotics

NVIDIA Robotics

@NVIDIARobotics

5 years

At #NeurIPS2018 we announced that PhysX, the world’s most popular physics simulation engine, is now open source. Robotics researchers can easily train machines in realistic environments.

6

163

457

4

24

128

@animesh_garg

Animesh Garg

2 years

Value Gradient weighted Model-Based Reinforcement Learning at #ICLR2022 Blog: Paper: 📢 Poster: Wed Apr 27, 10:30 am PST @c_voelcker V. Liao @SoloGen @UofTCompSci @VectorInst for more 👇

Tweet media one

4

22

128

@animesh_garg

Animesh Garg

4 years

CNNs are biased towards high-frequency textural information. New work on fixing CNN's over-reliance on texture through a curriculum that exposes texture slowly. Results in better features that generalize both to new datasets and to new tasks. Paper:

Tweet card media

Curriculum By Smoothing

Convolutional Neural Networks (CNNs) have shown impressive performance in computer vision tasks such as image classification, detection, and segmentation. Moreover, recent work in Generative...

@_sam_sinha_

Samarth Sinha

4 years

If you’re looking for a paper for your Friday readings, maybe check out our new work! Joint work with @hugo_larochelle & @animesh_garg ! If you want to improve your CNN but don’t want to add trainable parameters or add any regularization loss, give our paper a try! 🙂

Tweet media one

Tweet media two

8

48

156

1

23

128

@animesh_garg

Animesh Garg

2 years

A leading car company makes a humanoid robot that appears to be ahead of its times, and hopefully rallies many researchers to work on the problem. This is ground breaking ...only that this tweet is a few years too late

Tweet media one

10

8

122

@animesh_garg

Animesh Garg

15 days

I met Jensen in my first week at Stanford, and admittedly, did not realize the long game he was driving Nvidia towards! Indeed the world has come a long way since but Jensen maintains his conviction to drive AI at @nvidia and broadly.

@drfeifei

Fei-Fei Li

8 years

@nvidia CEO JensenHuang delivered the world's 1st AI supercomputer DGX-1 today to SAIL! @jcniebles @silviocinguetta

Tweet media one

Tweet media two

Tweet media three

2

52

228

2

5

123

@animesh_garg

Animesh Garg

3 years

What does the video remind you of the most? oh! I know... (deep) Reinforcement Learning papers proposing new algorithms #SOTA

@Alex_Verbeek

Alexander Verbeek 🌍

3 years

🥅 Goal!!! 😂

66

156

724

4

7

109

@animesh_garg

Animesh Garg

4 years

Proud moment for members (and alums) of ⁦ @StanfordSVL ⁩ upon the election of ⁦ @drfeifei ⁩ to National Academy of Engineering Congrats Fei-Fei!

0

12

106

@animesh_garg

Animesh Garg

2 years

front row seats at AI day at Tesla remark 1 setting expectations for optimus follow along..for more

Tweet media one

5

9

104

@animesh_garg

Animesh Garg

3 years

Teaching is hard, they said! have to prep exams, they said! then this....

@ritvijtiwary

Ritvij Tiwari

3 years

Tweet media one

75

341

3K

5

1

105

@animesh_garg

Animesh Garg

8 months

I am beyond excited to be building the PAIR group at @ICatGT working on ML and Robotics. @mlatgt & @GTrobotics is a nexus that will help us create the next wave in AI Robotics. Thanks Nathan Deen for the kind praise.

@ICatGT

Georgia Tech School of Interactive Computing

9 months

The next 10 to 12 years will see a boom for advancement in robot manipulation, says @animesh_garg . The new assistant professor has chosen @GeorgiaTech as the place he wants to be during this crucial time. @GTrobotics @mlatgt

0

5

32

2

10

105

@animesh_garg

Animesh Garg

2 years

I talked about Structure in Reinforcement Learning for Robotics at #ICRA2022 workshop on Behavior Priors tl;dr: Structured Biases in DL improve both efficiency & generalization. Robot Learning / RL needs new ones! Slides: Video:

Tweet media one

Tweet media two

1

16

100

@animesh_garg

Animesh Garg

6 years

Exciting new work on Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration @deanh619 @YukeZhu @danfei_xu @silviocinguetta @drfeifei

Tweet media one

2

20

102

@animesh_garg

Animesh Garg

5 years

Our new work at #Corl2019 will present RL with Ensemble of Suboptimal Teachers -aka- specify as much as you can easily, let learning handle the rest. Blog: Paper: w\ @andrey_kurenkov , A. Mandlekar, @RobobertoMM , @silviocinguetta

1

26

100

@animesh_garg

Animesh Garg

5 years

Official announcement of the @UofTCompSci Toronto robotics research cluster at @UTM and part of @UofTRobotics . @BurgnerKahrs (Continuum Robots), @florian_shkurti (Mobility), @animesh_garg (Manipulation) Hiring students, postdocs, engineers - reach out!

Tweet media one

Tweet media two

5

18

97

@animesh_garg

Animesh Garg

3 years

We have been running an AI in Robotics (AIR) Reading Group. PIs have so many talks, while students are in background. AIR is a platform for students to network across their immediate network and get speaking experience. For the students, by the students

2

19

99

@animesh_garg

Animesh Garg

2 years

Thesis defense of my mentee, and then close friend @AjayMandlekar Take home: Robot intelligence needs supervision at scale. And it's not just an algorithms problem, equally a systems problem! ⭐️-committee: @drfeifei @silviocinguetta @EmmaBrunskill @chelseabfinn @DorsaSadigh

Tweet media one

Tweet media two

2

13

97

@animesh_garg

Animesh Garg

3 years

please can we have this in Vision ( @CVPRConf , @ICCV_2021 ), ML ( @NeurIPSConf , @iclr_conf , @icmlconf ), & Robotics ( @corl_conf @RSS_Foundation ) Ideas where to start the petition, @gneubig

@ReviewAcl

ACLRollingReview

3 years

ARR is now accepting submissions! Please see for an overview of the submission form and link to the submission site. Submit by 5/15 to be eligible for @emnlpmeeting ! #NLProc

3

49

112

3

12

94

@animesh_garg

Animesh Garg

2 years

We have been working on conditional video generation for a while and even have a paper in the upcoming CVPR however the results in this paper are just amazing! Now I can go revive my dream of being a TikTok celeb!

@_akhaliq

AK

2 years

CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers github:

49

362

1K

2

6

89

@animesh_garg

Animesh Garg

3 years

I am very excited about the potential of high perf simulation in enabling AI Robotics. Looking forward to providing an overview of things we have been developing in the last couple of years, and a preview of things to come! Come check it out: Nov 8 2:30pm EST.

@UofTRobotics

University of Toronto Robotics Institute

3 years

Could simulation frameworks provide faster more accurate learning for industrial robots? Find out from Prof Animesh Garg on Nov 8. Full schedule & free registration #Robotics #AI #ML @animesh_garg @angelaschoellig @domo_mr_roboto

Tweet media one

0

3

21

1

12

91

@animesh_garg

Animesh Garg

4 months

Need a helping hand in the lab? Tired of manually doing tedious chemistry experiments over and over? Meet ORGANA 🤖 🧪 – a modular and user-friendly robotic lab assistant that can interact, plan and automate a number of chemistry experiments!

2

18

87

@animesh_garg

Animesh Garg

3 years

This work is accepted at @NeurIPSConf #NeurIPS2020 Come talk to us: Wed Dec 09 09:00 AM -- 11:00 AM (PST) @ Poster Session 3 #874 Paper and Talk at Code is now out as well

Tweet card media

GitHub - pairlab/v-cdn: [NeurIPS 2020] Causal Discovery in Physical Systems from Videos

[NeurIPS 2020] Causal Discovery in Physical Systems from Videos - pairlab/v-cdn

@animesh_garg

Animesh Garg

4 years

Learning Causal Graphs that capture Physical Systems has high potential yet challenging! Check out End-to-End Causal Discovery from videos Site: Paper: w\ @YunzhuLiYZ @AnimaAnandkumar , A.Torralba, D. Fox

2

32

139

2

17

88

@animesh_garg

Animesh Garg

2 years

Call for Papers for #Neurips2022 is out! 📅 May 19, 2022 1PM PDT @SGhalebikesabi & I will be Comms Chairs managing @NeurIPSConf among other things! This is going to be a fun year!

3

10

85

@animesh_garg

Animesh Garg

4 years

Are you interested in emerging questions in perception for manipulation? Consider our workshop on "Visual Learning and Reasoning for Robotic Manipulation" at #RSS2020 @RoboticsSciSys . Work-in-progress ideas equally welcome as are polished ones!

Tweet media one

0

12

85

@animesh_garg

Animesh Garg

2 years

Accelerated Policy Learning with Parallel Differentiable Simulation at #ICLR2022 Project: Paper: Poster: Tue Apr 26 6:30 PM PDT read more 🧵👇

2

15

82

@animesh_garg

Animesh Garg

4 years

Interested in Exploration in RL with image based inputs? Self-supervised goal reaching requires committed exploration for long-horizon tasks. LEAF: Latent Exploration Along the Frontier with reachability estimation Paper: @florian_shkurti @therealhomanga

Tweet media one

Tweet media two

2

13

82

@animesh_garg

Animesh Garg

2 years

Grasping with multi-finger hands is so hard! Sadly, sampling-based data scaling doesn't work 😭 Our @eccvconf paper presents Grasp'D! Diff Sim for exploring in full grasp space with surface contacts. It works for both robot and human hands without imitation data Thread 👇

@dylanturpin

Dylan Turpin

2 years

Grasp'D: Differentiable Contact-rich Grasp Synthesis for Multi-fingered Hands at #ECCV2022 Project: Paper: Video: [Thread below]

1

7

33

1

11

80

@animesh_garg

Animesh Garg

5 years

This is mind-blowing that this patent application is even possible and passed internal checks at @GoogleAI . Neither would this truly be novel nor unique. And yet just because someone else did not patent this, someone tries to patent not an idea but an entire field 🤣 🤦🏻‍♂️

@danfei_xu

Danfei Xu@ICRA24

5 years

Google files patent “Deep Reinforcement Learning for Robotic Manipulation”

1

5

19

6

24

80

@animesh_garg

Animesh Garg

3 years

Long term reasoning needs an understanding ofcontinuous changes in the world. "What happens if I open the door" Action Concept Grounding Networks learn these semantics @GnosisYu W.Chen @SMEasterbrook @UofTRobotics @VectorInst

3

14

81

@animesh_garg

Animesh Garg

5 months

Must feel like end of an era at FAIR with Ross ( @inkynumbers ) going to AI2 and Kaiming going to MIT Both Ross and Kaiming have taught the community the need for good systems and simplicity in research, especially at scale.

@anikembhavi

Ani Kembhavi @ ICLR 2024

5 months

Incredibly excited to announce that Ross Girshick ( @inkynumbers ) will be joining the PRIOR team @allen_ai ! Ross is one of the most influential and impactful researchers in AI. I'm so honored that he is joining us, and I'm really looking forward to working with him.

6

22

304

1

5

80

@animesh_garg

Animesh Garg

4 years

Learning Keypoints for robust state representations. "Unsupervised Disentanglement of Pose, Appearance, and Background from Images and Videos" Paper: Code: @aysegl_dndr , Kevin Shih, @abstract_ai , @drewtao , @ctnzr @NvidiaAI

Tweet media one

0

19

77

@animesh_garg

Animesh Garg

1 year

Using LLMs for code generation allows a very intuitive and effective way to perform feedback guided multistage planning for robotics. Come chat with me and @Ishika_S_ about ProgPrompt on Thurs Jun 1st - Poster Hall, 3-4.40pm, Pod 10 at #ICRA2023

@imankitgoyal

Ankit Goyal

2 years

Super excited to share our work ProgPromt! We show how LLMs can be used for situated robot planning by prompting them with pythonic code. abs: project page: [1/9]

1

4

47

0

21

77

@animesh_garg

Animesh Garg

2 years

Great research, great food, and you get to stay after graduation! most tech jobs that one may want exist in Toronto. (3rd after SF and NYC) students shopping for grad school and postdocs should take note 😉 @CadeMetz does a profile in @nytimes

2

5

76

@animesh_garg

Animesh Garg

3 years

Isaac Gym blog @NVIDIAAI Using just one A100 GPU, Isaac Gym achieves same perf. in ~10 hours — as compared to 30 hours on 6000+ CPUs. A single GPU outperforming an entire cluster by a factor of 3x Reel of current envs in IG:

Tweet card media

Introducing NVIDIA Isaac Gym: End-to-End Reinforcement Learning for...

Announcing a preview release of Isaac Gym - NVIDIA’s physics simulation environment for reinforcement learning research. RL-based training is now more access...

www.youtube.com

@animesh_garg

Animesh Garg

4 years

Isaac Gym - @NVIDIAAI physics simulation environment for reinforcement learning research (preview Release) - End-to-End GPU accelerated - Isaac Gym tensor-based APIs fo massively parallel sim Also get in touch for potential internships to flex in Gym!

8

44

243

2

7

76

@animesh_garg

Animesh Garg

3 years

We are hiring new students in robotics in various area at the newly formed robotics cluster. @florian_shkurti @BurgnerKahrs @igilitschenski @UofTRobotics @UofTCompSci And btw we have a farm of (real) robots (>20), in case you are looking to play during grad school

Tweet media one

@igilitschenski

Igor Gilitschenski

@igilitschenski

3 years

Passionate about Math 🧮, CS 💻, AI 🧠, and Robots 🦾? This season, I am looking for my first cohort of graduate students 🎓 at the fantastic @UofTCompSci 🇨🇦 ❤️💻. Find out more at and apply by Dec 1st.

4

19

103

1

10

76

@animesh_garg

Animesh Garg

3 years

ha! i was (and still am) advised by @drfeifei could not have asked for a better postdoc experience.

1

1

76

@animesh_garg

Animesh Garg

4 years

Our new paper on hierarchical framework that combines model-based control and RL for learning robust quadruped controllers Paper: Video: X. Da @zhaomingxie @HoellerDavid B. Boots @AnimaAnandkumar @yukez @BuckBabich at @NVIDIAAI

@NVIDIAAI

NVIDIA AI

4 years

Researchers at NVIDIA have built a robot that automatically adapts to different terrains helping delivery robots and other autonomous machines function more effectively in environments. See this technology in action. #GTC20

2

18

50

2

18

76

@animesh_garg

Animesh Garg

4 years

Many schools have dropped (or made optional) the GRE requirement -- a step in the right direction. btw GRE is optional at @UofTCompSci as well! It is an undue burden for non-native speakers to memorize words far too recondite for prosaic usage!

@paul_pearce

Paul Pearce

4 years

UC Berkeley MIT Stanford CMU UIUC U. of Washington Cornell Georgia Tech Princeton UT Austin Michigan Wisconsin UCSD Harvard UMD UPenn Purdue UMass Amherst NYU NEU UChicago And I hear more are coming. I'll add more if people reply.

43

78

379

6

6

75

@animesh_garg

Animesh Garg

3 years

Interested in Robotics Engineering in research? We are hiring an Engineering Tech in Robotics to help with robotics education content + open-sourcing robotics frameworks + robot learning algorithms! DM for info & retweet🙏 @UofTCompSci @UofTRobotics @UTM

Robotics Laboratory Technician

Robotics Laboratory Technician

jobs.utoronto.ca

3

22

73

@animesh_garg

Animesh Garg

2 months

Simulation is the data factory for robotics. Yet, we seem to only use it for scale! Scale is not all you need!, or atleast not the only ingredient. Algorithmic innovation matters🛠️ So what is beyond vectorized physics? I provide a perspective on using additional information…

Tweet media one

Tweet media two

0

13

74

@animesh_garg

Animesh Garg

4 years

A fresh look at low cost mobile manipulation which simplifies the mechanism get seems to achieve a broad set of tasks Ex-Googler's Startup Comes Out of Stealth With Beautifully Simple, Clever Robot Design.

Tweet card media

Ex-Googler's Startup Comes Out of Stealth With Beautifully Simple, Clever Robot Design

Hello Robot's Stretch wants to reinvent how mobile manipulators perform tasks in home environments

spectrum.ieee.org

0

8

71

@animesh_garg

Animesh Garg

3 years

Looking forward to interacting with friends from MILA, and talking about recent work on my series on Generalizable Autonomy Part- I was last year at MIT Deep Learning Claas. Hopefully this would be a worthy part-II same old problem, all new methods!

Tweet card media

MIT 6.S191 (2020): Generalizable Autonomy for Robot Manipulation

MIT Introduction to Deep Learning 6.S191: Lecture 8Generalizable Autonomy for Robot ManipulationLecturer: Animesh Garg (NVIDIA & University of Toronto)Janua...

www.youtube.com

@MontrealRobots

REAL - Robotics and Embodied AI Lab

@MontrealRobots

3 years

The second speaker in the robot learning seminar series is @animesh_garg from @UofTRobotics Tune in at 4 pm ET tomorrow (5 Feb) Seminar series schedule: @Mila_Quebec

Tweet media one

0

7

36

1

11

69

@animesh_garg

Animesh Garg

5 months

Come talk to us about SlotDiffusion - an object-centric Latent Diffusion Model (LDM) designed for both image and video data 🗓️ Wed, Dec 13, 10:45 📌 Poster #611 Hall B1+B2 (level 1) The students couldn't be here but the advisors ( @igilitschenski ) are equally fun to chat with!

Tweet media one

1

10

71

@animesh_garg

Animesh Garg

5 years

One-shot imitation learning with a continuous relaxation of the discrete symbolic planner that directly plans on the probabilistic outputs of the symbol grounding model. #IROS2019 @deanh_tw @danfei_xu @yukez @silviocinguetta @drfeifei @jcniebles

Tweet card media

Continuous Relaxation of Symbolic Planner for One-Shot Imitation Learning

We address one-shot imitation learning, where the goal is to execute a previously unseen task based on a single demonstration. While there has been exciting progress in this direction, most of the...

2

14

69

@animesh_garg

Animesh Garg

3 years

Want to know about the future of Robotics in Retail & Manufacturing. Meet us on: Nov 8 10am-6pm EST Keynotes: @pabbeel of @CovariantAI + @berkeley_ai Matt Mason of @BerkshireGrey + @CMU_Robotics Free to attend, Register now! Stay tuned for more!

Tweet media one

@UofTRobotics

University of Toronto Robotics Institute

3 years

Join us on Nov 8 for the UofT free 1-day Robotics Institute AI virtual symposium. Strategize with academic leaders closing the gap between shop floor reality & practical robot & data platforms in retail & manufacturing. Details & Registration:

Tweet media one

2

10

18

0

15

67

@animesh_garg

Animesh Garg

4 years

Unsupervised disentanglement is very difficult to learn. However, with ~1% labels, semantic alignment can be achieved. Check out our paper on semi-supervised disentanglement tomorrow at #ICML2020

@wn8_nie

Weili Nie

4 years

Excited to present our paper at #ICML2020 (Jul 16 7 a.m. PDT and Jul 16 8 p.m. PDT). Welcome to our discussions. Semi-Supervised StyleGAN for Disentanglement Learning with Tero Karras, @animesh_garg , Shoubhik Debnath, Anjul Patney, @abp4_ankit , @AnimaAnandkumar @NVIDIAAI

1

6

37

0

10

70

@animesh_garg

Animesh Garg

2 years

are you a student from Ukraine facing program interruption, apply to @UofTCompSci Summer Program for Students from Ukraine. ( @VectorInst , @UofT ) All expenses covered and quick turnaround. 📅 Apply by Apr 8, a decision by Apr 13. 🙏spread the word🙏

Tweet card media

Summer Program for Students from Ukraine — Department of Computer Science, University of Toronto

web.cs.toronto.edu

1

27

68

@animesh_garg

Animesh Garg

3 months

The intricacy, image quality and spatiotemporal consistency of the SORA generated videos is mesmerizing. The problem of conditional video generation has been around, but this feels like a leap! The video below is generated with a prompt "Reflections in…

3

7

68

@animesh_garg

Animesh Garg

4 years

@thienan496 @causalinf @_MiguelHernan @yudapearl @EpiEllie I would also suggest starting with 4 part Jonas Peters's lectures on youtube

Tweet card media

Lectures on Causality: Jonas Peters, Part 1

May 10, 2017MITMachine learning expert Jonas Peters of the University of Copenhagen presents “Four Lectures on Causality”. Produced by the Laboratory for Inf...

www.youtube.com

0

7

67

@animesh_garg

Animesh Garg

1 year

This is so reminiscent of India winning cricket WC in 2011 for @sachin_rt , so it goes for Messi in football ( @TeamMessi ) The little master could not have not have retired without a WC title

Tweet media one

Tweet media two

3

1

67

@animesh_garg

Animesh Garg

5 years

NVidia Robotics Research lab is now open for business. Talented team of folk having fun with all kinds of robots. No more a roboticist can ask for! Thx Jensen for the new venture in @NvidiaAI

@geekwire

GeekWire

5 years

Here’s what’s cooking inside Nvidia’s new AI robotics research lab in Seattle

0

10

16

1

7

67

@animesh_garg

Animesh Garg

3 years

we have a great group of AI/ML in {robotics + vision} consider applying and do mention the specific names in your application before Dec 1 @FidlerSanja @RaquelUrtasun @florian_shkurti @igilitschenski and myself

@RaquelUrtasun

Raquel Urtasun

3 years

If you are interested in joining my group as a Masters or PhD student, apply here . Deadline December 1st. I recommend you mention this fact in your research statement. Join the #selfdriving revolution! #gradschool #AI

5

43

173

2

2

66

@animesh_garg

Animesh Garg

6 years

Pumped to share our work on large scale learning from people as described in 'Robots learn tasks from people' 1/3

Tweet card media

Robots learn tasks from people | Stanford News

With a smartphone and a browser, people will be able to teach robots how to do basic tasks.

news.stanford.edu

3

19

64

@animesh_garg

Animesh Garg

4 years

This is an very exciting direction of my research in injecting latent inductive bias to do better meta learning. Go check out the poster and if interested in working on it apply to work with me at @UofTCompSci , @VectorInst or @NvidiaAI

0

12

66

@animesh_garg

Animesh Garg

1 year

Telling a good story is as important as doing the research! 🎥🎙️🎬 Research communication requires effort and planning. It is hard to expect students to do also learn all the media editing tools. An amazing HowTo make a Research Video by @martoskreto 👏

Tweet media one

2

27

66

@animesh_garg

Animesh Garg

4 years

Ever wondered if choosing better action spaces can speed up learning in manipulation. But how? Consider joining the discussion "Action Representations for Learning in Continuous Control" at #RSS2020 @RoboticsSciSys . @RobobertoMM @leto__jean @Mvandepanne

Tweet media one

2

19

65

@animesh_garg

Animesh Garg

5 years

Which control spaces work well for RL in contact-rich robot manipulation -- 'Variable Impedance Control in End-Effector Space' #IROS2019 Paper: Video: w\ Roberto M-M., @michellearning , R. Gardener, @silviocinguetta , @leto__jean

Tweet media one

0

15

64

@animesh_garg

Animesh Garg

4 years

New paper on Goal-Based Imitation from Third Person Videos. Motion reasoning that combines task & motion planning to resolve semantic ambiguity demonstrator intent and outputs symbolic goal representations from video Paper:

Tweet card media

Motion Reasoning for Goal-Based Imitation Learning

https://arxiv.org/abs/1911.05864

www.youtube.com

1

17

65

@animesh_garg

Animesh Garg

3 years

When you try to open a new door, do you try to yank it up? Likely, no. Why should your robot continue to do so! Check out our #ICRA2021 paper on learning action spaces for efficient contact-rich manipulation. paper:

Tweet card media

LASER: Learning a Latent Action Space for Efficient Reinforcement...

LASER. More details at http://www.pair.toronto.edu/laser/The process of learning a manipulation task depends strongly on the action space used for exploratio...

www.youtube.com

1

10

65

@animesh_garg

Animesh Garg

3 months

Google just released Gemma, a new set of open LLMs (2B & 7B) Reported perf tops Llama2 (both at 7B & 13B scales) and Mistral-7B! 🎉 (Comparisons with Mixtral 8x7B would be valuable both with and w/o MoE for Gemma) Importantly, they are free to use…

Tweet media one

2

4

64

@animesh_garg

Animesh Garg

4 years

It’s always great to visit MIT This time for a intro to robot learning lecture in deep learning short course Thanks to Alexandre Amini and @apsoleimany for inviting me

Tweet media one

Tweet media two

3

4

64

@animesh_garg

Animesh Garg

4 years

Multi-modal state representation learning through self-supervised cross-modal prediction enables generalization to new shapes and robustness Paper @michellearning @yukez @silviocinguetta @drfeifei @leto__jean K. Srinivasan P. Shah

Tweet card media

Making Sense of Vision and Touch: Multimodal Representations for Contact-Rich Tasks

We learn a multimodal representation that combines vision and touch through self-supervision.

ai.stanford.edu

1

12

64

@animesh_garg

Animesh Garg

4 years

This a such a cool new application in vision in generative video models

@jbhuang0604

Jia-Bin Huang

4 years

Super excited to share our work on video completion at #ECCV2020 !🤩Our method seamlessly removes objects, watermarks, or expands field-of-view from casually captured videos. Paper: Project: With @gaochen315 , Ayush, and @JPKopf

71

942

4K

0

10

62

@animesh_garg

Animesh Garg

5 years

Latest paper with collaborators at @NvidiaAI on High-quality, long-range video interpolation, and extrapolation through unsupervised latent structure inference followed by a temporal prediction. Paper: K. Shih, @aysegl_dndr , R. Pottorff, A. Tao, @ctnzr

0

13

63

@animesh_garg

Animesh Garg

3 years

@PrasoonPratham @Twitter I did not tag you because I did not want to poke fun at you, and my own reach is smaller and perhaps disjoint from yours. Broadly I appreciate you being vocal about learning ML early on and you being an example. although basics math should not be equated to ML

4

0

61

@animesh_garg

Animesh Garg

4 years

I am deeply saddened by the fact that so many students from @UofT and other Canadian institutions were on the ill-fated flight. I am honored that the University has come out publicly in support of these tragic times.

0

2

61

@animesh_garg

Animesh Garg

4 years

Many dynamic processes, including many in robotics & RL, involve a set of interacting subprocesses, which can be decomposed into locally independent causal mechanisms. Solution: Counterfactual Data Augmentation (CoDA) Paper: w\ @SilviuPitis E. Creager

Tweet media one

Tweet media two

@anirudhg9119

Anirudh Goyal

4 years

Counterfactual Data Augmentation using Locally Factored Dynamics This is an interesting work done by Silviu Pitis, Elliot Creager, @animesh_garg .

1

2

35

1

10

61

@animesh_garg

Animesh Garg

3 years

want to build the next generation in general purpose object manipulation to help with lab automation? @A_Aspuru_Guzik and I are hiring a joint postdoc in robotic manipulation and 3D object perception @acceleration_c @VectorInst @UofTRobotics + email us

1

15

61