Danfei Xu@ICRA24 @danfei_xu Twitter profile

Last Seen Profiles

@BartChmielowiec

@CousinJahh

@jfudem

@Yunaafps

@anilkjain61

@SumitDefence

@utddenvor

@chescaleigh

@latrellhoover99

@yaban334

@nourran_amgad

@fordoglunk

@LeTour

@JDenman9

@hina_nanac

@Clubearr

@kinokochan99

@10DowningStreet

@Viktorshly

@oveitcom

@pikaramagazine

@WestwoodDNA

@Leah713Austin

@gino_gt_daigo

@up_q_i

@bari5

@NuriaVarela

@edendownes

@UCD_Research

@Mikito_PonPon7

@kwasbeb

@maydayterraneo

@buzzle_bysizzle

@ha__2202

@finbarmcd

@osakaracing

Danfei Xu@ICRA24

@danfei_xu

4 years

How to do Research At the MIT AI Lab (1988). Almost all advices are still valid more than three decades later. Highly recommended.

8

196

878

Danfei Xu@ICRA24

@danfei_xu

3 years

I’ll be joining GaTech @gtcomputing @ICatGT as an Assistant Professor in Fall 2022! Looking forward to continuing my work in Robot Learning as a faculty and collaborating with researchers & students at GTCS @GTrobotics @mlatgt . Reach out for collaborations / joining the lab!

47

12

522

Danfei Xu@ICRA24

@danfei_xu

3 years

I defended!

Fei-Fei Li

@drfeifei

3 years

Very proud of my student @danfei_xu (co-advised with @silviocinguetta ) for his wonderful PhD thesis defense today! Danfei’s work in computer vision and robotic learning pushes the field forward towards enabling robots to do long horizon tasks of the real world. 1/2

4

8

223

19

4

296

Danfei Xu@ICRA24

@danfei_xu

2 years

I'm recruiting! If you are excited about teaching robots to perceive, reason about, manipulate, and move around everyday environments, apply the CS Ph.D program at GT (Interactive Computing) and mention my name. Apps from underrepresented groups in AI&Robo are especially welcome!

5

53

275

Danfei Xu@ICRA24

@danfei_xu

2 years

A bit more formally: I'm hiring Ph.D. students in Robot Learning this year! If you are excited about the future of data-driven approaches to robotics, apply through the School of Interactive Computing at @gtcomputing by Dec 15th.

Danfei Xu@ICRA24

@danfei_xu

3 years

I’ll be joining GaTech @gtcomputing @ICatGT as an Assistant Professor in Fall 2022! Looking forward to continuing my work in Robot Learning as a faculty and collaborating with researchers & students at GTCS @GTrobotics @mlatgt . Reach out for collaborations / joining the lab!

47

12

522

0

40

186

Danfei Xu@ICRA24

@danfei_xu

1 year

In case anyone is curious about large-scale RLHF for robotics. This is probably the first paper you should read:

Data-Driven Robotics

Large-scale machine learning is one of the greatest triumphs of artificial intelligence in the last decade. The recipe for success typically involves the following ingredients: a large deep network,...

sites.google.com

2

41

178

Danfei Xu@ICRA24

@danfei_xu

4 years

Greetings Twitterverse! Excited to share that I'm going on the academic job market this year! Check out my research at

7

22

156

Danfei Xu@ICRA24

@danfei_xu

5 years

Excited to share my internship project @DeepMindAI with Misha Denil @notmisha ! Positive-Unlabeled Reward Learning arXiv:

3

17

107

Danfei Xu@ICRA24

@danfei_xu

7 years

@elonmusk No it's not a more complex game. It only requires optimizing immediate reward like health and money. Go requires longer horizon planning.

5

7

100

Danfei Xu@ICRA24

@danfei_xu

4 months

Since we are entering the "BC is all you need" phase of Robot Learning😜 --- Robomimic () allows you to play with SOTA algorithms (BC-Transformer, DiffusionPolicy, etc.) on challenging tasks. Also easy to integration with physical robots!

2

19

100

Danfei Xu@ICRA24

@danfei_xu

23 days

I often get this question: Is LLM all you need for robot planning? I'd go: "obviously not, because you need to consider physical constraints, dynamics, ... ", which then turn into a non-stop rant. Now I'll just point them to this paper 😎

LIDAR@GT

@GT_LIDAR

23 days

If you're interested in learning SOTA of optimization-based task and motion planning, please give it a read of our recent survey paper, ranging from classical to learning methods. @ZhaoZhigen @ShuoCheng94 @yding25 @ZiyiZhou2 @ShiqiZhang7 @danfei_xu

1

14

60

0

9

100

Danfei Xu@ICRA24

@danfei_xu

28 days

This is clearly going to benefit the privileged. Even the info that this conference/track existed probably will only circulate in a small group with direct tie to academia/tech (parents etc). How about we flip this into a track for creating accessible tutorials, lectures,…

NeurIPS Conference

@NeurIPSConf

29 days

This year, we invite high school students to submit research papers on the topic of machine learning for social impact! See our call for high school research project submissions below.

21

48

203

1

8

96

Danfei Xu@ICRA24

@danfei_xu

4 years

Excited to share Generalization Through Imitation (GTI)! GTI learns visuomotor control from human demos and generalizes to new long-horizon tasks by leveraging latent compositional structures. Joint w/ @AjayMandlekar @RobobertoMM @silviocinguetta @drfeifei

2

27

93

Danfei Xu@ICRA24

@danfei_xu

16 days

Super neat system! It seems that Chinese robotics startups have everything they need to quickly iterate on capable & low-cost hardware. Will US startups be able to compete? Chaining together dynamixals/off-the-shelf motors likely won’t cut it…

Simon Kalouche

@simonkalouche

16 days

Nice smooth hardware

18

63

397

6

11

94

Danfei Xu@ICRA24

@danfei_xu

7 months

Can't believe that I just came across this insanely cool paper. 3D gaussian seems to be such an intuitive representation to model large & dynamic scenes (Lagrangian vs. Eularian). Expect it to drive a whole new wave of dense/obj-centric representation w/ self-supervision.

Jonathon Luiten

@JonathonLuiten

9 months

Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis We model the world as a set of 3D Gaussians that move & rotate over time. This extends Gaussian Splatting to dynamic scenes, with accurate novel-view synthesis and dense 3D trajectories.

24

368

2K

3

10

88

Danfei Xu@ICRA24

@danfei_xu

1 month

160 H100 for GT Makerspace!

Georgia Tech College of Engineering

@gatechengineers

1 month

Putting the promise of AI directly in students’ hands: We’re powering up the Georgia Tech AI Makerspace - a student-focused AI supercomputer hub. Proud to work with @nvidia and @WeAre_Penguin to make this a reality on campus for our students.

5

34

151

4

5

84

Danfei Xu@ICRA24

@danfei_xu

3 years

New preprint! Affordance is a versatile repr. to reason about interactions in a complex world. But it is also *myopic*, because it only means that an action is feasible instead of leading to a long-term goal. How can we use affordances to plan for long-horizon tasks? 1/

4

6

73

Danfei Xu@ICRA24

@danfei_xu

4 years

Accepted to RSS 2020!

Danfei Xu@ICRA24

@danfei_xu

4 years

Excited to share Generalization Through Imitation (GTI)! GTI learns visuomotor control from human demos and generalizes to new long-horizon tasks by leveraging latent compositional structures. Joint w/ @AjayMandlekar @RobobertoMM @silviocinguetta @drfeifei

2

27

93

2

6

72

Danfei Xu@ICRA24

@danfei_xu

5 years

We present Regression Planning Network (RPN), a type of recursive network architecture that learns to perform high-level task planning from video demonstrations. #NeurIPS2019 (1/3)

1

21

67

Danfei Xu@ICRA24

@danfei_xu

2 years

Applying imitation learning to real world problems takes more than new algorithms. We are organizing a workshop "Overlooked Aspects of Imitation Learning: Systems, Data, Tasks, and Beyond” at RSS22! Exciting speakers & more to come. Submit by May 7th!

RSS 2022 IL Workshop

News

sites.google.com

1

8

65

Danfei Xu@ICRA24

@danfei_xu

18 days

🤖 Inspiring the Next Generation of Roboticists! 🎓 Our lab had an incredible opportunity to demo our robot learning systems to local K-12 students for the National Robotics Week program @GTrobotics . A big shout-out to @saxenavaibhav11 @simar_kareer @pranay_mathur17 for hosting…

1

11

64

Danfei Xu@ICRA24

@danfei_xu

4 years

It's a strange time to share this but I'll be co-instructing the Stanford CS231n course next quarter! Now that all courses are pass/fail, we might experiment w/ something new😃 Suggestions / tips on online lecturing are appreciated!

2

63

Danfei Xu@ICRA24

@danfei_xu

1 year

One of the most impressive CV works I've seen recently. Also huge kudos to Meta AI for sticking to open sourcing despite the trend increasingly going towards the opposite direction.

AI at Meta

@AIatMeta

1 year

Today we're releasing the Segment Anything Model (SAM) — a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from any photo or video + zero-shot transfer to other segmentation tasks ➡️

144

2K

7K

0

3

59

Danfei Xu@ICRA24

@danfei_xu

2 months

We also made a similar transition to ROS-free. The non-obvious thing is that modern NN models (BC policies, VLMs, LLMs) breaks the abstraction of ROS modules. Raw sensory stream instead of state estimation, actions instead of plans, etc. Need new ROS for the next-gen modules.

Chris Paxton

@chris_j_paxton

2 months

Interesting (and sad) result here; I really had hoped more people would be able to just run with ROS2. But it seems like it's not quite there, if this is in any way worth doing for a small company/fast-moving startup that should be the target audience.

6

3

44

1

4

59

Danfei Xu@ICRA24

@danfei_xu

4 years

Presenting two papers at #NeurIPS2019 ! Come say hi if you are around. 1. Regression Planning Networks We combine classic symbolic planning and recursive neural network to plan for long-horizon tasks end-to-end from image input. Paper & Code: 1/

1

9

59

Danfei Xu@ICRA24

@danfei_xu

2 years

Our group headed by @MarcoPavoneSU at NVIDIA Research is hiring fulltime RS and interns! Tons of cool problems in planning, control, imitation, and RL. Job posting 👇 Intern: Full-time:

0

5

54

Danfei Xu@ICRA24

@danfei_xu

4 years

In 2010 I was a high school senior in Shanghai. I cold-called a company making educational robots and started my first internship in robotics. Almost a decade later, I’m doing a Ph.D. at Stanford, still in robotics, still happy. Let’s see where the next decade leads me.

1

0

51

Danfei Xu@ICRA24

@danfei_xu

1 year

Annnnd that's a wrap! First semester teaching at GT and it's been an absolute blast. Really happy to see the progression of the student projects and the final poster session joined by ~170 students. Couldn't have made it without my awesome TAs. Thanks @mlatgt for the sponsorship!

1

50

Danfei Xu@ICRA24

@danfei_xu

2 months

Detail: 10hz image -> 200hz EEF control. I'm guessing keep the same image token for 20 steps while updating proprio state? Also given how smooth the motion looks --- high-quality OSC implementation?

Corey Lynch

@coreylynch

2 months

Finally, let's talk about the learned low-level bimanual manipulation. All behaviors are driven by neural network visuomotor transformer policies, mapping pixels directly to actions. These networks take in onboard images at 10hz, and generate 24-DOF actions (wrist poses and…

19

59

407

2

49

Danfei Xu@ICRA24

@danfei_xu

1 year

First work coming out of my lab at GT! LEAGUE is a "virtuous cycle" system that combines the merit of Task and Motion Planning and RL. The result is continually-learning and generalizable agents that can carry their knowledge to new task and even environments.

Shuo Cheng

@ShuoCheng94

1 year

Introducing LEAGUE - Learning and Abstraction with Guidance! LEAGUE is a new framework that uses symbolic skill operators to guide skill learning and state abstraction, allowing it to solve long-horizon tasks and generalize to new tasks and domains. Joint work with @danfei_xu 1/6

1

7

31

1

10

47

Danfei Xu@ICRA24

@danfei_xu

8 months

Super excited about this new #CoRL2023 work on compositional planning! We introduce a new generative planner (GSC) to compose skill-level diffusion models to solve long-horizon manipulation problem, without ever training on long-horizon tasks. @ICatGT @GTrobotics @mlatgt

Utkarsh Mishra @ICRA24

@utkarshm0410

8 months

How to enable robots to plan and compositionally generalize over long-horizon tasks? At #CoRL2023 , we introduce Generative Skill Chaining (GSC), a diffusion-based, generalizable and scalable approach to compose skill-level transition models into a task-level plan generator.(1/7)

3

29

160

1

3

44

Danfei Xu@ICRA24

@danfei_xu

3 months

This blew my mind 100x more than GPTs did.

OpenAI

@OpenAI

3 months

Introducing Sora, our text-to-video model. Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions. Prompt: “Beautiful, snowy…

10K

33K

141K

3

1

45

Danfei Xu@ICRA24

@danfei_xu

8 months

Among so many thoughtful & nuanced discussions on regulating AI, the EU chooses to "mitigate the risk of extinction from AI"... This is some sort of joke, right?

European Commission

@EU_Commission

8 months

Mitigating the risk of extinction from AI should be a global priority. And Europe should lead the way, building a new global AI framework built on three pillars: guardrails, governance and guiding innovation ↓

435

483

2K

1

6

42

Danfei Xu@ICRA24

@danfei_xu

4 years

We are organizing the Deep Representation and Estimation of State tutorial at the virtual IROS2020! Fantastic speaker line-up: @leto__jean , Yunfei Bai, and @ChrisChoy208 . Co-organized with @KuanFang and @deanh_tw . A short thread about each session👇

1

9

43

Danfei Xu@ICRA24

@danfei_xu

2 months

Agreed. Humanoid is a problem, not a solution.

Watney Robotics

@watneyrobotics

2 months

Do you really need legs? We don't think so. As much we love anthropomorphic humanoids (our co-founder built one in 9th grade), we believe virtually all menial tasks can be done with two robot arms, mounted on wheels. In our view, @1x_tech 's Eve robot is the optimal form factor…

8

11

115

2

4

41

Danfei Xu@ICRA24

@danfei_xu

2 years

Honored to be selected a DARPA Riser and giving a talk about our robot learning works!

DARPA

@DARPA

2 years

T minus 2 hours until we begin our next #DARPAForward event @GeorgiaTech . @DoDCTO Heidi Shyu will kick off a packed agenda featuring experts on pandemic preparedness, cybersecurity, and more. Visit our page for more on how you can join future events:

7

15

50

4

1

39

Danfei Xu@ICRA24

@danfei_xu

6 years

Dr. Pedro Domingos to Lead the Independent R&D Effort at D. E. Shaw's new machine learning research group.

D. E. Shaw Group Forms New Machine Learning Research Group

/PRNewswire/ -- The D. E. Shaw group, a global investment and technology development firm and a pioneer in quantitative approaches to trading and investment,...

www.prnewswire.com

1

10

39

Danfei Xu@ICRA24

@danfei_xu

3 months

modular -> end to end -> modular

Matei Zaharia

@matei_zaharia

3 months

Interesting trend in AI: the best results are increasingly obtained by compound systems, not monolithic models. AlphaCode, ChatGPT+, Gemini are examples. In this post, we discuss why this is and emerging research on designing & optimizing such systems.

30

259

1K

4

2

40

Danfei Xu@ICRA24

@danfei_xu

6 months

Robotics dataset is expanding at an unprecedented pace. How do we control the quality of the collected data? Our #CoRL2023 work presents an offline imitation learning method that learns to discern (L2D) data from expert in a mixed-quality demonstration dataset. Code coming soon!

Sachit Kuhar

@SachitKuhar

6 months

Introducing our #CoRL2023 work Learning to Discern (L2D)! As robotics datasets grow, quality control becomes ever more important. L2D is our solution for handling mixed-quality demo data for offline imitation learning. (1/6)

3

7

26

0

10

40

Danfei Xu@ICRA24

@danfei_xu

4 years

Used ACME for my summer internship project @DeepMind , can confirm it's an amazing framework.

Google DeepMind

@GoogleDeepMind

4 years

Interested in playing around with RL? We’re happy to announce the release of Acme, a light-weight framework for building and running novel RL algorithms. We also include a range of pre-built, state-of-the-art agents to get you started. Enjoy!

14

401

1K

0

2

38

Danfei Xu@ICRA24

@danfei_xu

1 year

An open-source playground for training generative agents from real-world driving data! Work led by @Yuxiao_Chen_ @iamborisi @drmapavone and myself from our team at @NVIDIAAI , in close collaboration with @NVIDIADRIVE .

Yuxiao Chen

@Yuxiao_Chen_

1 year

We are excited to announce the release of Traffic Behavior Simulation (TBSIM) developed by the Nvidia Autonomous Vehicle research group (), which is our software infrastructure for closed-loop simulation with data-driven traffic agents. (1/7)

3

32

195

2

38

Danfei Xu@ICRA24

@danfei_xu

4 years

We are organizing a workshop on imitation learning at RSS2020 ()! The workshop will bring together well-known researchers in field. CfP includes short-length, full-length, and position papers. Tentative submission deadline Apr 9th. RT and spread the word!

RSS IL Workshop

Live Session Recording

sites.google.com

0

12

35

Danfei Xu@ICRA24

@danfei_xu

7 months

Need a fully automated sim-to-real pipeline to train locomotion policies for arbitrary robot (with URDF) in ~1hr

Davide Scaramuzza

@davsca1

7 months

@DisneyResearch introduces their new robot at #IROS2023 ! Trained in simulation with #reinforcementlearning ! @ieeeiros

106

2K

6K

0

37

Danfei Xu@ICRA24

@danfei_xu

6 months

Object representation is a fundamental problem for robotic manipulation. Our #CoRL2023 work found that *density field* can efficiently represent state and dynamics of non-rigid objects such as granular material. To be presented as a spotlight&poster on Thursday!

Shangjie Xue

@ShangjieXue

6 months

How to represent granular materials for robot manipulation? Introducing our #CoRL2023 project: Neural Field Dynamics Model for Granular Object Piles Manipulation, a field-based dynamics model for granular object piles manipulation. 🌐 👇 Thread

1

6

19

0

5

35

Danfei Xu@ICRA24

@danfei_xu

4 years

Join us on Sunday at 9:00-1:30pm PT for the Advances & Challenges in Imitation Learning for Robotics #RSS2020 Workshop: with an exciting list of speakers! Live streaming at

1

3

34

Danfei Xu@ICRA24

@danfei_xu

3 years

A thread by my awesome co-instructor @RanjayKrishna recapping @cs231n for the past quarter. It happens to be the *largest* class on campus for the quarter! Thanks all the teaching staff, especially our head TA @kevin_zakka for making this course possible!

Ranjay Krishna

@RanjayKrishna

3 years

Academic quarter recap: here's a staff photo after the last lecture of @cs231n . It's crazy that we were the largest course at Stanford this quarter. This year, we added new lectures and assignments (open sourced) on attention, transformers, and self-supervised learning.

2

128

1

34

Danfei Xu@ICRA24

@danfei_xu

6 months

Looking forward to welcome everyone to the beautiful ATL! @GTrobotics @ICatGT @gtcomputing

Conference on Robot Learning

@corl_conf

6 months

Gearing up for the conference next week, check this interactive feature as you prep for your time at the conference. Discover cool papers and insights. Did you know that we have 199 contributed papers from 873 authors originating in 25 countries! 🤯

1

15

71

0

2

32

Danfei Xu@ICRA24

@danfei_xu

4 years

Accepted to ICRA 2020! Paper & code:

6-PACK

Chen Wang, Roberto Martín-Martín, Danfei Xu, Jun Lv Cewu Lu, Li Fei-Fei, Silvio Savarese, Yuke Zhu

sites.google.com

Roberto

@RobobertoMM

5 years

We present 6-PACK, an RGB-D category-level 6D pose tracker that generalizes between instances of classes based on a set of anchors and keypoints. No 3D models required! Code+Paper: w/ Chen Wang @danfei_xu Jun Lv @cewu_lu @silviocinguetta @drfeifei @yukez

3

66

283

2

7

33

Danfei Xu@ICRA24

@danfei_xu

1 month

It was an honor to have been part of this epic journey!

Fei-Fei Li

@drfeifei

1 month

It’s that time of the year - first lecture of @cs231n !! It’s the 9th year since @karpathy and I started this journey in 2015, what an incredible decade of AI and computer vision! Am so excited to this new crop of students in CS231n! (Co-instructing with @eadeli this year 😍🤩)

20

78

974

0

1

31

Danfei Xu@ICRA24

@danfei_xu

3 years

Incredibly excited to be able to attend an academic event in person!

Dorsa Sadigh

@DorsaSadigh

3 years

Bay Area Robotics Symposium (BARS) will be happening in person this Friday on October 29! The registration will close on October 27th, 5 p.m. Register here: Program:

2

18

74

0

30

Danfei Xu@ICRA24

@danfei_xu

2 years

Whoa this is huge! No more wrangling w/ python2 compatibility & root access issues.

Tobias Fischer

@TobiasRobotics

2 years

Incredibly happy that our @RoboStack paper has been accepted to the @ieeeras Robotics & Automation Magazine 🥳. @RoboStack brings together #ROS @rosorg with @condaforge and @ProjectJupyter . Preprint: . Find out some key benefits in this 🧵: 1/n

6

106

371

1

5

29

Danfei Xu@ICRA24

@danfei_xu

3 years

Check out our new work on imitation learning from human demos! We released a set of sim&real tasks, demo datasets, and a modular codebase & clean APIs to help you develop new algorithms!

Ajay Mandlekar

@AjayMandlekar

3 years

Robot learning from human demos is powerful yet difficult due to a lack of standardized, high-quality datasets. We present the robomimic framework: a suite of tasks, large human datasets, and policy learning algorithms. Website: 1/

4

57

263

0

2

29

Danfei Xu@ICRA24

@danfei_xu

5 years

Check out our #ICCV2019 work on harnessing mid-level representations in training interactive agents.

Yuke Zhu @ ICRA2024

@yukez

5 years

We are releasing our #ICCV2019 work on goal-directed visual navigation. We introduced a method that harnesses different perception skills based on situational awareness. It makes a robot reach its goals more robustly and efficiently in new environments.

2

33

141

0

3

29

Danfei Xu@ICRA24

@danfei_xu

4 years

Blog post by @deanh_tw and I summarizing our line of work on generalizable imitation of long-horizon tasks: Neural Task Programming, Neural Task Graphs, and Continuous Relaxation of Symbolic Planner. Enjoy!

Stanford AI Lab

@StanfordAILab

4 years

What if we can teach robots to do new task just by showing them one demonstration? In our newest blog post, @deanh_tw and @danfei_xu show us three approaches that leverage compositionality to solve long-horizon one-shot imitation learning problems.

0

39

140

1

4

28

Danfei Xu@ICRA24

@danfei_xu

3 years

Our new work on training competent robot collaborators from human-human collaboration demonstrations! @corl_conf @stanfordsvl @StanfordAILab

0

4

27

Danfei Xu@ICRA24

@danfei_xu

3 years

@gtcomputing @ICatGT @GTrobotics @mlatgt In the meantime, I will spend my gap year at @nvidia Research. I couldn’t be more excited and I’m immensely grateful for my advisors @silviocinguetta @drfeifei and many collaborators and friends who helped me to get here.

0

1

27

Danfei Xu@ICRA24

@danfei_xu

2 years

To carry out long-horizon tasks, robots must plan far and wide into the future. What state space should the robot plan with, and how can they plan for objects & scenes that they have never seen before? See 👇for our new work on Generalizable Task Planning (GenTP).

Chen Wang

@chenwang_j

2 years

1/ Can we improve the generalization capability of a vision-based task planner with representation pretraining? Check out our RAL paper on learning to plan with pre-trained object-level representation. Website:

2

13

72

0

26

Danfei Xu@ICRA24

@danfei_xu

7 months

Excited to share our milestone in building generalizable long-horizon task solvers at #CoRL2023 ! As part of our long-term vision for a never-ending data engine for everyday tasks, HITL-TAMP combines the best of structured reasoning (TAMP) and end-to-end imitation learning.

Ajay Mandlekar

@AjayMandlekar

7 months

How can humans help robots improve? Introducing Human-In-The-Loop Task and Motion Planning (HITL-TAMP), a perpetually-evolving TAMP system that learns visuomotor skills from human demos for contact-rich, long-horizon tasks. #CoRL2023 Website: 1/

3

32

214

0

2

27

Danfei Xu@ICRA24

@danfei_xu

6 months

Learning for high-precision manipulation is critical to bridge *intelligence* to repeatable *automation*. C3DM is a diffusion model that learns to remove noise from the input by "fixating" on the target object. To be presented at the Deployable Robot workshop at #CoRL2023 today!

Vaibhav Saxena

@saxenavaibhav11

6 months

Introducing C3DM 🤖 - a Constrained-Context Conditional Diffusion Model that solves robotic manipulation tasks with: ✅ high precision and ✅ robustness to distractions! 👇 Thread

1

9

35

0

2

27

Danfei Xu@ICRA24

@danfei_xu

1 year

Robot auction sale by Intrinsic (Alphabet’s industrial automation/robotics startup). Sad to see this.

Chris Anderson

@chr1sa

1 year

If you're a hardware biz or R&D lab in Silicon Valley, you should definitely be keeping your eye on the liquidation auctions, which are on fire right now This one is auctioning off more than 100 new and used Kuka robot arms:

13

31

181

2

26

Danfei Xu@ICRA24

@danfei_xu

1 year

Fantastic research led by Chen! Continuing our work on hierarchical imitation starting for real-world long-horizon manipulation. It turns out that we can train high-level planner directly from *human video*. This greatly reduces need for on-robot data and improves robustness 1/2

Chen Wang

@chenwang_j

1 year

How to teach robots to perform long-horizon tasks efficiently and robustly🦾? Introducing MimicPlay - an imitation learning algorithm that uses "cheap human play data". Our approach unlocks both real-time planning through raw perception and strong robustness to disturbances!🧵👇

20

144

742

1

2

26

Danfei Xu@ICRA24

@danfei_xu

4 years

It's today! DeepRL workshop

Danfei Xu@ICRA24

@danfei_xu

4 years

@RobobertoMM @deanh_tw @yukez @silviocinguetta @drfeifei 2. Positive-Unlabeled Reward Learning Deep Reinforcement Learning Workshop Joint work with @notmisha 3/

0

3

0

8

27

Danfei Xu@ICRA24

@danfei_xu

24 days

Data fuels the progress in robotics, whether it's sim, real teleoperated, or auto-generated. Our workshop at #RSS2024 will bring together researchers from academia, industry, and startups around the world to share insights🧐 and hot takes 🔥.

Ajay Mandlekar

@AjayMandlekar

24 days

Data is the key driving force behind success in robot learning. Our upcoming RSS 2024 workshop "Data Generation for Robotics” will feature exciting speakers, timely debates, and more! Submit by May 20th.

6

25

96

0

25

Danfei Xu@ICRA24

@danfei_xu

5 years

I admire people who can explain complex things clearly.

1

0

24

Danfei Xu@ICRA24

@danfei_xu

5 years

So on a whim I decided that I want to know more about the reward hypothesis of RL and found this page. Quite an interesting read

1

8

25

Danfei Xu@ICRA24

@danfei_xu

6 months

Very nice post! Slightly different take: Scaling up should be the **question**, not the answer. Yes we need to scale up to more task, envs, robots, but there should be many possible answers to this question. Training on lots of data may be an answer but should not the only one.

Nishanth Kumar

@nishanthkumar23

6 months

There was a lot of good and interesting debate on "is scaling all we need to solve robotics?" at #CoRL23 . I spent some time writing up a blog post about all the points I heard on both sides:

22

47

257

1

24

Danfei Xu@ICRA24

@danfei_xu

6 years

Our paper on learning generalizable neural programs for complex robot tasks will appear in #icra2018 ! See you soon. Arxiv: Two minutes paper: , Video:

Neural Task Programming: Learning to Generalize Across Hierarchical...

Stanford Vision and Learning LabMore info: https://stanfordvl.github.io/ntp/*******Abstract*******In this work, we propose a novel robot learning framework c...

www.youtube.com

0

7

22

Danfei Xu@ICRA24

@danfei_xu

1 year

Ah yes the familiar feeling of "everything that can go wrong goes wrong" end-of-the-semester chaos.

0

22

Danfei Xu@ICRA24

@danfei_xu

7 months

We! are! hiring! 👏

Mark Riedl

@mark_riedl

7 months

Yo! Georgia Tech School of Interactive Computing @ICatGT is live! … Come be part of the coolest computing science department in the world

3

26

103

0

1

21

Danfei Xu@ICRA24

@danfei_xu

3 years

I got my first robotics research experience through CMU RISS. Fantastic program and mentors!

CMU Center for Perceptual Computing and Learning

@roboVisionCMU

3 years

Fully funded undergraduate research internships at CMU’s Robotics Institute! Apply by Jan 15, 2021!

3

12

41

0

21

Danfei Xu@ICRA24

@danfei_xu

3 years

TFW you know enough to understand that something is really hard but don't know enough to make meaningful progress.

1

0

21

Danfei Xu@ICRA24

@danfei_xu

11 months

This year’s EECS rising star will be hosted by GaTech! Submit your materials by July 10th. Please RT and spread the word!

Kexin Rong

@kexinrong

11 months

The EECS Rising Stars 2023 Workshop is now accepting applications 🎉Check out for more details. Deadline is July 10th. Help spread the word!

2

51

78

0

3

20

Danfei Xu@ICRA24

@danfei_xu

5 years

Google files patent “Deep Reinforcement Learning for Robotic Manipulation”

1

5

19

Danfei Xu@ICRA24

@danfei_xu

4 years

Trying to better understand contrastive learning: Intuitively, contrastive learning relies on dense pos/neg sample coverage. SimCLR & others increase coverage using image augmentation. But how dense does the space have to be & what about spaces that cannot be augmented easily?

2

0

19

Danfei Xu@ICRA24

@danfei_xu

5 months

As the Deep Learning course at GT draws to a close this semester, I'd like to extend a heartfelt thanks to @WilliamBarrHeld . His exceptional lecture and programming assignment on Transformers and LLMs were truly enlightening. Don't miss out on these incredible resources!

Will Held

@WilliamBarrHeld

5 months

For @danfei_xu 's Deep Learning course this semester, I made a homework for Transformers and gave a lecture on LLMs. I'm sharing resources I made for both in hopes they are useful for others! Lecture Slides: HW Colab:

1

13

44

1

0

17

Danfei Xu@ICRA24

@danfei_xu

1 year

Excited about hierarchy, abstraction, model learning, skill learning, planning with LLMs, and benchmarking long-horizon manipulation tasks? Submit a paper to our learning for Task and Motion Planning (L4TAMP) workshop at RSS'23!

Jeannette Bohg

@leto__jean

1 year

We are organizing the RSS’23 Workshop on Learning for Task and Motion Planning Contributions of short papers or Blue Sky papers are due May 19th, 2023.

2

8

42

1

2

18

Danfei Xu@ICRA24

@danfei_xu

3 years

A neat extension of our Regression Planning Networks to 3D scene graph and more fine-grained skills!

Yuke Zhu @ ICRA2024

@yukez

3 years

Delighted to present our recent work on hierarchical Scene Graphs for neuro-symbolic manipulation planning. We use 3D Scene Graphs as an object-centric abstraction to reason about long-horizon tasks. w/ @yifengzhu_ut , Jonathan Tremblay, Stan Birchfield

0

15

98

0

1

18

Danfei Xu@ICRA24

@danfei_xu

5 years

Our new work on learning real-time 6DoF tracking from RGB-D data.

Roberto

@RobobertoMM

5 years

We present 6-PACK, an RGB-D category-level 6D pose tracker that generalizes between instances of classes based on a set of anchors and keypoints. No 3D models required! Code+Paper: w/ Chen Wang @danfei_xu Jun Lv @cewu_lu @silviocinguetta @drfeifei @yukez

3

66

283

0

1

18

Danfei Xu@ICRA24

@danfei_xu

5 years

OPT is practically the only pathway for international students like me to legally work in the U.S. after graduation. This is beyond short-sighted.

Kai Sheng Tai

@kaishengtai

5 years

The OPT program is crucial for retaining talented international students in the US. I relied on the OPT myself for summer internships during college and for full-time work after graduation.

3

42

158

1

17

Danfei Xu@ICRA24

@danfei_xu

1 year

Looking forward to welcoming the robot learning community to ATL!

Conference on Robot Learning

@corl_conf

1 year

The cat is out of the bag! We'll be in Atlanta next year. #CoRL2022 #CoRL2023

0

7

87

0

1

17

Danfei Xu@ICRA24

@danfei_xu

2 months

This is a great effort to collect large robot dataset on standardized hardware setup! Also happy to see that Robomimic is adopted as the core policy learning infrastructure.

Alexander Khazatsky

@SashaKhazatsky

2 months

After two years, it is my pleasure to introduce “DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset” DROID is the most diverse robotic interaction dataset ever released, including 385 hours of data collected across 564 diverse scenes in real-world households and offices

5

77

298

0

2

14

Danfei Xu@ICRA24

@danfei_xu

7 months

Make sure to check out our workshop on NeRF + robotics happening tomorrow at #ICCV2023 , Paris and virtually!

Yue Wang

@yuewang314

7 months

#ICCV2023 Join us for the “Neural Fields for Autonomous Driving and Robotics” () workshop 8:55-17:00 on 10/3 at S03! We have a great lineup of speakers @vincesitzmann @jon_barron @AjdDavison @LingjieLiu1 @jiajunwu_cs @lucacarlone1 @Jamie_Shotton .

3

14

75

1

0

16

Danfei Xu@ICRA24

@danfei_xu

4 years

Why is learning object-centric representation important for RL/robot learning? If it is merely a form of state dim reduction, and the only useful info it provides is 3D pose / 2d bbox & object appearance, then shouldn't ppl focus on better pose estimator / detector?

4

3

16

Danfei Xu@ICRA24

@danfei_xu

3 years

Grateful for the amazing team! Also go check out our codebase & dataset at

GitHub - ARISE-Initiative/robomimic: robomimic: A Modular Framework for Robot Learning from...

robomimic: A Modular Framework for Robot Learning from Demonstration - ARISE-Initiative/robomimic

github.com

Ajay Mandlekar

@AjayMandlekar

3 years

Our robomimic study was accepted as an oral at CoRL 2021! Extremely grateful to the reviewers and our amazing team @danfei_xu @josiah_is_wong @snasiriany @chenwang_j Rohun K. @drfeifei @silviocinguetta @yukez @RobobertoMM Check out the codebase and datasets if you haven't yet!

1

11

71

0

16

Danfei Xu@ICRA24

@danfei_xu

4 years

The original Perceptual Symbol Systems article and its commentary & author responses are truly goldmines of refs on neural symbolic research & its cogsci background. Also intrigued to find hints of many modern DL ideas in the article such as ...

1

16

Danfei Xu@ICRA24

@danfei_xu

4 years

cool idea: learning feature embeddings via multi-view contrastive loss, similar to DenseObjectNet by Florence et al., 2018

Adam W. Harley

@AdamWHarley

4 years

Our #ECCV2020 paper is now on arXiv. We show that 3D object tracking emerges automatically when you train for multi-view correspondence. No object labels necessary! Video: results from KITTI. Bottom right shows a bird's eye view of the learned 3D features.

0

45

203

0

1

15

Danfei Xu@ICRA24

@danfei_xu

5 years

Huge congrats!

Yuke Zhu @ ICRA2024

@yukez

5 years

Our #ICRA2019 paper received the Best Conference Paper Award w/ @michellearning @leto__jean @animesh_garg @drfeifei @silviocinguetta

7

5

117

0

16

Danfei Xu@ICRA24

@danfei_xu

6 years

Met & chat with @elonmusk about autonomous driving and AGI. Thanks @russelljkaplan @karpathy @shivon for making it happen!

Russell Kaplan

@russelljkaplan

6 years

Fun chat last night at the Tesla party with Elon and some @StanfordCVGL folks #CVPR2018

2

3

49

0

15

Danfei Xu@ICRA24

@danfei_xu

4 years

Tomorrow at 8AM PST!

Danfei Xu@ICRA24

@danfei_xu

4 years

Can we train visuomotor policies for real-world long-horizon tasks AND generalize across tasks? Join us Wed 8-10am virtually at #RSS2020 (Paper #61 ) for a live discussion with @AjayMandlekar , @RobobertoMM , and myself. See 👇 for a short thread about our paper.

0

3

13

0

14

Danfei Xu@ICRA24

@danfei_xu

4 years

A bit overdue but here! A blog post about our RSS work on learning long-horizon motor policies from human demos and generalizing to new tasks.

Stanford AI Lab

@StanfordAILab

4 years

Human demonstrations are often used to teach robots new tasks, but how can we achieve generalization when learning by imitation? Check out our latest blog post about Generalization Through Imitation (GTI) courtesy of @danfei_xu and @AjayMandlekar :

0

17

34

0

2

14

Danfei Xu@ICRA24

@danfei_xu

5 years

It's a good time to read more books on world history & politics instead of tweets & news.

0

2

14

Danfei Xu@ICRA24

@danfei_xu

5 years

Check out our latest work on end-to-end object pose estimation! Code is available on Github:

DenseFusion

Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martín-Martín, Cewu Lu, Li Fei-Fei, Silvio Savarese

sites.google.com

Yuke Zhu @ ICRA2024

@yukez

5 years

We have just released our new work on 6D pose estimation from RGB-D data -- real-time inference with end-to-end deep models for real-world robot grasping and manipulation! Paper: Code: w/ @danfei_xu @drfeifei @silviocinguetta

3

23

99

0

5

14

Danfei Xu@ICRA24

@danfei_xu

1 year

Somehow I'm more impressed by the throw motion than the 540 flip. The whole thing is so freakin' awesome!

Boston Dynamics

@BostonDynamics

1 year

It’s time for Atlas to pick up a new set of skills and get hands on.

2K

13K

59K

2

1

14

Danfei Xu@ICRA24

@danfei_xu

3 years

Humans heavily rely on visual attention to guide hand movements to perform everyday tasks like reaching and grasping. Can we teach robots similar hand-eye coordination abilities without direct supervision? 👇Our #IROS2021 work on generalizable imitation via hand-eye coordination.

Chen Wang

@chenwang_j

3 years

Can robots learn hand-eye coordination simply from teleoperated human demonstrations? Our new #IROS2021 paper presents a novel action space to enable this! Website: 1/9

3

37

107

0

14

Danfei Xu@ICRA24

@danfei_xu

1 year

Welcome to ATL!🐝

Animesh Garg

@animesh_garg

1 year

Career update: I will be moving to @ICatGT next year. I look forward to working alongside the exceptional researchers at @GTrobotics and @mlatgt

54

25

753

2

0

13

Danfei Xu@ICRA24

@danfei_xu

4 years

Can we train visuomotor policies for real-world long-horizon tasks AND generalize across tasks? Join us Wed 8-10am virtually at #RSS2020 (Paper #61 ) for a live discussion with @AjayMandlekar , @RobobertoMM , and myself. See 👇 for a short thread about our paper.

Danfei Xu@ICRA24

@danfei_xu

4 years

Excited to share Generalization Through Imitation (GTI)! GTI learns visuomotor control from human demos and generalizes to new long-horizon tasks by leveraging latent compositional structures. Joint w/ @AjayMandlekar @RobobertoMM @silviocinguetta @drfeifei

2

27

93

0

3

13

Danfei Xu@ICRA24

@danfei_xu

1 year

@hausman_k Agreed that representation is the gap! But I don’t think anyone throught we could solve robotics w/o proper perception, systems modeling, and optimization, and none of which is simply “scripting” (at least not the kind that LLM can solve).

1

13

Danfei Xu@ICRA24

@danfei_xu

21 days

Not an expert in gaiting, but pretty cool to see fast-ish, natural-looking straight leg gait.

Mentee Robotics

@MenteeBot

24 days

Introducing Menteebot: Groundbreaking Humanoid Robot. We're proud to unveil Menteebot, the culmination of a two-year journey by our brilliant team! Menteebot is a groundbreaking humanoid robot designed for versatility. Visit our website for more demos.

13

37

124

4

1

13

Danfei Xu@ICRA24

@danfei_xu

6 years

“See your publications as liability, not assets.” — Vladlen Koltun on doing good research.

0

3

13

Danfei Xu@ICRA24

@danfei_xu

4 years

@karpathy Adaptive Computation Time for Recurrent Neural Networks (Graves, 2016)

Adaptive Computation Time for Recurrent Neural Networks

This paper introduces Adaptive Computation Time (ACT), an algorithm that allows recurrent neural networks to learn how many computational steps to take between receiving an input and emitting an...

arxiv.org

0

11