Yuke Zhu @ ICRA2024 @yukez Twitter profile

Last Seen Profiles

@MollyKummel

@Garrido1971

@to_me92008

@drumcodeuk

@AkumuDragonz

@CoachMikeAllen

@CSIR_NEIST

@MantaExchange_

@yukiiarts45

@CGTMadrideduc

@Alythium

@CallHimGoat

@goonerdudewoo

@jazzset

@ch_sokai

@penguinlibrosmx

@mAtHwsi

@JAVICHICAT

@DavidGogina

@JeffSonnenfeld

@TKMattya_pixiv

@DaytonPolice

@mybabatweets

@gruairportsp

@OfficialMojflix

@Medway_NHS_FT

@ordinarywillard

@Flaminiesta

@determinantfin

@HospitalCivil

@horidouraku

@isogai_guitar

@HannZiq

@KylenMills

@domm

@JJVILLARD

Yuke Zhu @ ICRA2024

@yukez

27 days

Got a taste of @Tesla 's FSD v12.3.4 last night. By no means flawless, but the human-like driving maneuvers (with no interventions) delivered a magical experience. Excited to witness the recipe of scaling law and data flywheel for full autonomy show signs of life in real products.…

32

207

2K

Yuke Zhu @ ICRA2024

@yukez

3 years

The game of tenure-track faculty job: ℍ𝕒𝕣𝕕 𝕞𝕠𝕕𝕖: 1st year ℍ𝕖𝕝𝕝 𝕞𝕠𝕕𝕖: 1st year + COVID-19 𝕀𝕟𝕗𝕖𝕣𝕟𝕠 𝕞𝕠𝕕𝕖: 1st year + COVID-19 + No Power/Internet in freezing Texas P.S. It has been great fun to play. What's next?

26

27

995

Yuke Zhu @ ICRA2024

@yukez

3 years

My Robot Learning class @UTCompSci is updated with the latest advances and trends, such as implicit representations, attention architectures, offline RL, human-in-the-loop, and synthetic data for AI. All materials will be public. Enjoy! #RobotLearning

13

106

558

Yuke Zhu @ ICRA2024

@yukez

5 years

New work: we built a meta-learning algorithm for an agent to discover the causal and effect relations from its visual observations and to use such causal knowledge to perform goal-directed tasks. Paper: Joint work w/ @SurajNair_1 @drfeifei @silviocinguetta

5

108

463

Yuke Zhu @ ICRA2024

@yukez

2 months

📢Update announced in today’s #GTC2024 Keynote📢 We are working on Project GR00T, a general-purpose foundation model for humanoid robots. GR00T will enable the robots to follow natural language instructions and learn new skills from human videos and demonstrations. Generalist…

12

64

427

Yuke Zhu @ ICRA2024

@yukez

4 years

Heard students say WFH lowers productivity. In 1665, a Cambridge college student had to WFH during a pandemic. He got away from professors and worked on math alone. When he returned, the world knew him as Issac Newton! Good time to think hard in pajamas.

During a pandemic, Isaac Newton had to work from home, too. He used the time wisely.

With neither older scholars to guide him nor modern media distractions, student Isaac Newton came up with his most amazing works while sheltering from the plague.

www.washingtonpost.com

8

116

416

Yuke Zhu @ ICRA2024

@yukez

3 months

Thrilled to co-lead this new team with my long-time collaborator @DrJimFan . We are on a mission to build transformative breakthroughs in the landscape of Robotics and Embodied Agents. Come join us and shape the future together!

Jim Fan

@DrJimFan

3 months

Career update: I am co-founding a new research group called "GEAR" at NVIDIA, with my long-time friend and collaborator Prof. @yukez . GEAR stands for Generalist Embodied Agent Research. We believe in a future where every machine that moves will be autonomous, and robots and…

241

471

4K

16

21

310

Yuke Zhu @ ICRA2024

@yukez

5 years

Life update: I will be joining @UTAustin as an Assitant Professor in @UTCompSci starting Fall 2020. I am thrilled to continue my research on robot learning and perception as a faculty and look forward to collaborating with the exceptional faculty, researchers, and students at UT.

22

15

378

Yuke Zhu @ ICRA2024

@yukez

2 years

Honored to receive the NSF CAREER award titled "Intelligent Manipulation in the Real World via Modularity and Abstraction" to advance our lab's research on building autonomy stack for general-purpose robot manipulation in the wild!

26

15

346

Yuke Zhu @ ICRA2024

@yukez

4 years

Number 50 is a role model for PhD advisors.

Rex Chapman🏇🏼

@RexChapman

4 years

Cleansing the timeline with this one. Be a number 50 in someone’s life...

2K

41K

190K

2

22

344

Yuke Zhu @ ICRA2024

@yukez

1 year

Excited to share our latest progress on legged manipulation with humanoids. We created a VR interface to remote control the Draco-3 robot 🤖, which cooks ramen for hungry graduate students at night. We can't wait for the day it will help us at home in the real world! #humanoid

7

67

325

Yuke Zhu @ ICRA2024

@yukez

5 years

Releasing my Stanford Ph.D. dissertation and talk slides "Closing the Perception-Action Loop: Towards Building General-Purpose Robot Autonomy", a summary of my work on robot perception and control @StanfordSVL Slides: Dissertation:

3

55

310

Yuke Zhu @ ICRA2024

@yukez

2 years

Congratulations to @snasiriany and @huihan_liu on winning the #ICRA2022 Outstanding Learning Paper award for their first paper @UTCompSci “Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks”!

24

20

310

Yuke Zhu @ ICRA2024

@yukez

4 years

Today we are on the road to Austin, TX. I have a pleasant melancholy moving out of the Bay Area, a place where we have lived for seven years, leaving behind many fond memories and long-time friends. Meanwhile, thrilled to start out a new life. Tons of exiting things to come!

22

3

270

Yuke Zhu @ ICRA2024

@yukez

4 years

Taught my first (online) class @UTCompSci . Super pumped to teach a grad-level Robot Learning seminar this fall. Great to see UT students from all kinds of backgrounds passionate about learning what’s going on at the forefront of AI + Robotics🤘Syllabus:

6

21

255

Yuke Zhu @ ICRA2024

@yukez

1 year

Just learned that our MineDojo paper won the Outstanding Paper award at #NeurIPS2022 See you in New Orleans next week!

Jim Fan

@DrJimFan

2 years

Introducing MineDojo for building open-ended generalist agents! ✅Massive benchmark: 1000s of tasks in Minecraft ✅Open access to internet-scale knowledge base of 730K YouTube videos, 7K Wiki pages, 340K Reddit posts ✅First step towards a general agent 🧵

12

129

483

5

31

247

Yuke Zhu @ ICRA2024

@yukez

8 months

Very impressed by the new @Tesla_Optimus end2end skill learning video! Our TRILL work () spills some secret sauce: 1. VR teleoperation, 2. deep imitation learning, 3. real-time whole-body control. It's all open-source! Dive in if you're into humanoids! 👾

1

54

242

Yuke Zhu @ ICRA2024

@yukez

4 years

Some of my proudest memories of my PhD are working with people from different countries and being advised by a stellar all-women thesis committee. I encourage students from diverse backgrounds to apply for my future lab @UTCompSci where diversity and inclusion will be valued.

3

20

239

Yuke Zhu @ ICRA2024

@yukez

3 months

We are unlikely to create an “ImageNet for Robotics”. In retrospect, ImageNet is such a homogeneous dataset. Labeled images w/ boxes. Generalist robot models will be fueled by the Data Pyramid, blending diverse data sources from web and synthetic data to real-world experiences.

6

31

239

Yuke Zhu @ ICRA2024

@yukez

5 years

Excited to start my gap year in @NvidiaAI ! Looking forward to a lot of new research opportunities with the brilliant minds!

7

5

192

Yuke Zhu @ ICRA2024

@yukez

5 years

Dear academics, check out our 6 pack!! 💪 Ok... I meant 6-PACK, our new 6DoF Pose Anchor-based Category-level Keypoint tracker, real-time tracking of novel objects without known 3D models!

Roberto

@RobobertoMM

5 years

We present 6-PACK, an RGB-D category-level 6D pose tracker that generalizes between instances of classes based on a set of anchors and keypoints. No 3D models required! Code+Paper: w/ Chen Wang @danfei_xu Jun Lv @cewu_lu @silviocinguetta @drfeifei @yukez

3

66

283

1

36

190

Yuke Zhu @ ICRA2024

@yukez

3 months

One of RL's most future-proof ideas is that adaptation is just a memory problem in disguise. Simple in theory, scaling is hard! Our #ICLR2024 spotlight work AMAGO shows the path to training long-context Transformer models with pure RL. Open-source here:

1

34

188

Yuke Zhu @ ICRA2024

@yukez

6 months

Just wrapped up my #CoRL2023 early-career keynote on 𝐏𝐚𝐭𝐡𝐰𝐚𝐲 𝐭𝐨 𝐆𝐞𝐧𝐞𝐫𝐚𝐥𝐢𝐬𝐭 𝐑𝐨𝐛𝐨𝐭𝐬 on Wed. In case you missed it, here's a brief summary. Check out the slide deck for more detail: 🧵1/N

4

24

179

Yuke Zhu @ ICRA2024

@yukez

2 years

Future driverless cars will talk with each other! We introduce Coopernaut, a cooperative driving model that uses vehicle-to-vehicle (V2V) communication for robust driving in challenging traffic conditions. #CVPR2022 Paper: Project:

4

19

180

Yuke Zhu @ ICRA2024

@yukez

4 years

We are organizing a (virtual) workshop on Visual Learning and Reasoning for Robotic Manipulation at #RSS2020 . We invite extended abstract submissions that address the research problems at the intersection of perception and manipulation:

1

31

175

Yuke Zhu @ ICRA2024

@yukez

4 years

Loved the Slow Science Manifesto (). We were told, "slow down to go faster." Oh boy, this is so much easier said than done. As a young academic, seeing fellow scholars churning out dozens of papers a year, it takes guts to hit the pause button and think!

1

21

173

Yuke Zhu @ ICRA2024

@yukez

4 years

As much as I'd like to tweet positivity and focus on #AcademicChatter , I know how difficult this moment is for the Asian community when my wife and I feel anxious about going out for shopping & errands, hearing recent news about hate crimes. Hatred is NOT a solution to a virus.

5

6

170

Yuke Zhu @ ICRA2024

@yukez

3 years

Implicit neural representations have pushed the envelope of 3D Vision and Graphics in recent years. How will they be useful for Robot Manipulation? Our work GIGA demonstrated that they can bridge geometry reasoning and affordance learning for 6-DoF grasping in cluttered scenes.

2

18

167

Yuke Zhu @ ICRA2024

@yukez

3 years

🔥robosuite updates🦾After eight months of dev effort, excited to release our v1.3 version! We integrate advanced graphics renderers with our simulation framework and provide vision APIs to bridge robot perception and decision-making research. Try it out!

2

29

163

Yuke Zhu @ ICRA2024

@yukez

6 months

Can't wait to attend #CoRL2023 for the next two days and give an early career keynote titled "𝐏𝐚𝐭𝐡𝐰𝐚𝐲 𝐭𝐨 𝐆𝐞𝐧𝐞𝐫𝐚𝐥𝐢𝐬𝐭 𝐑𝐨𝐛𝐨𝐭𝐬: 𝐒𝐜𝐚𝐥𝐢𝐧𝐠 𝐋𝐚𝐰, 𝐃𝐚𝐭𝐚 𝐅𝐥𝐲𝐰𝐡𝐞𝐞𝐥, 𝐚𝐧𝐝 𝐇𝐮𝐦𝐚𝐧𝐥𝐢𝐤𝐞 𝐄𝐦𝐛𝐨𝐝𝐢𝐦𝐞𝐧𝐭" on Wed!

Speakers — CoRL 2023

www.corl2023.org

3

10

165

Yuke Zhu @ ICRA2024

@yukez

5 years

Thanks Fei-Fei @drfeifei for being such an amazing advisor, mentor, role model, and friend! Finishing a PhD is the end of the beginning. And greater things have yet to come!

Fei-Fei Li

@drfeifei

5 years

Very proud of my PhD student @yukez for passing his PhD thesis defense with flying colors! His work is on perception, learning and robotics. Thank you thesis committee members @leto__jean @EmmaBrunskill @silviocinguetta & Dan Yamins!

10

19

286

4

2

156

Yuke Zhu @ ICRA2024

@yukez

4 years

Research used randomized controlled trial to show "tweeting improves citations". What improves the long-term impact of a paper?

8

31

154

Yuke Zhu @ ICRA2024

@yukez

5 months

First time attending @HumanoidsConf (on the @UTAustin campus!) Feels pumped to see the lightning-fast progress in this space. I expect this community to proliferate in the next few years --- Generalist robot intelligence can't be achieved without general-purpose hardware!

1

14

153

Yuke Zhu @ ICRA2024

@yukez

2 months

My #CoRL2023 keynote talk on Pathway to Generalist Robots is on YouTube now: I discussed the three key ingredients for building general-purpose robot autonomy: scaling law, data flywheel, and human-like embodiment. If you want to learn more about our…

Pathway to Generalist Robots: Scaling Law, Data Flywheel, and...

CoRL2023 Early Career Keynote - Yuke Zhu (November 8th, 2023)Title: Pathway to Generalist Robots: Scaling Law, Data Flywheel, and Humanlike EmbodimentAbstrac...

www.youtube.com

1

33

154

Yuke Zhu @ ICRA2024

@yukez

5 months

I won't be at NeurIPS next week. But our team is seeking interns to work on exciting and ambitious new projects on Large Language Models for Agents (starting early next year). Please fill out the Application Form below if you're interested.

1

26

78

Yuke Zhu @ ICRA2024

@yukez

1 year

📢Release note📢 We are pleased to release *robosuite* v1.4 and migrate its backend to @DeepMind 's MuJoCo binding for long-term support and feature extensibility, solidifying our commitment to building open-source research software. Try it out at

2

21

146

Yuke Zhu @ ICRA2024

@yukez

5 years

We are releasing our #ICCV2019 work on goal-directed visual navigation. We introduced a method that harnesses different perception skills based on situational awareness. It makes a robot reach its goals more robustly and efficiently in new environments.

2

33

141

Yuke Zhu @ ICRA2024

@yukez

4 years

100% agreed! I also felt extremely lucky to have some kindest and smartest advisors @Stanford and colleagues @UTCompSci "We're all smart. Distinguish yourself by being kind." This quote is one of the first principles I will teach to my students as a scholar.

Vijay Chidambaram

@vj_chidambaram

4 years

All the technically strongest people I know are *kind* people. My advisors/profs at @WisconsinCS , my colleagues at @UTCompSci , they are all competent, caring, empathetic human beings. Sure, there are some jerks, but they are the minority -- there is no need to hire them.

6

20

224

3

14

139

Yuke Zhu @ ICRA2024

@yukez

4 years

Check out a new blog post of our work on long-horizon planning for robot manipulation. We also released RoboVat, our learning framework that unifies #BulletPhysics simulation and Sawyer robot control interfaces. Sim2real has never been easier.

Stanford AI Lab

@StanfordAILab

4 years

How can a robot solve complex sequential problems? In our newest blog post, @KuanFang introduces CAVIN, an algorithm that hierarchically generate plans in learned latent spaces.

0

22

89

1

23

133

Yuke Zhu @ ICRA2024

@yukez

2 years

ICML deadline tonight, RSS deadline tomorrow, and CVPR rebuttals due next Monday. For researchers working on robot learning and perception, life is goooood 😌

4

3

126

Yuke Zhu @ ICRA2024

@yukez

1 year

Roomba builds a static map of your home by moving around. Can a robot create articulated models of indoor scenes through its physical interaction? Ditto in the House builds digital twins of articulated objects in everyday environments. #ICRA2023 Website:

0

16

126

Yuke Zhu @ ICRA2024

@yukez

2 years

Our robot can now make morning coffee for you… The secret recipe: 1⃣ Object-centric representation 2⃣ Transformer-based policy architecture 3⃣ Data-efficient imitation learning algorithm 4⃣ Robust impedance controller. Enjoy ☕️! #CoRL2022 #VIOLA

Yifeng Zhu 朱毅枫

@yifengzhu_ut

2 years

How can robot manipulators perform in-home tasks such as making coffee for us? We introduce VIOLA, an imitation learning model for end-to-end visuomotor policies that leverages object-centric priors to learn from only 50 demonstrations!

9

57

320

2

13

126

Yuke Zhu @ ICRA2024

@yukez

4 years

Before the coronavirus outbreak, I almost decided to name my new lab VIRAL, which stands for Visual Intelligence & Robot Autonomy Lab. Now I have to change it 😅 Epidemics make us think harder.

5

1

121

Yuke Zhu @ ICRA2024

@yukez

5 months

Check out our new survey paper on Foundation Models in Robotics!

AK

@_akhaliq

5 months

Foundation Models in Robotics: Applications, Challenges, and the Future paper page: We survey applications of pretrained foundation models in robotics. Traditional deep learning models in robotics are trained on small datasets tailored for specific…

6

156

606

4

26

118

Yuke Zhu @ ICRA2024

@yukez

11 months

Heading to @CVPR today! We are organizing a 3D Vision and Robotics workshop tomorrow with a great line-up of speakers: Also, I am recruiting a postdoc on vision + robotics for my group. Come to chat with me if interested - DMs are open!

3DVR 2023

3D perception is critical in robotic applications, such as manipulation and navigation. Understanding the visual world is critical for robots to operate in the real world. In recent years, we have...

sites.google.com

0

17

115

Yuke Zhu @ ICRA2024

@yukez

5 years

Our #ICRA2019 paper received the Best Conference Paper Award w/ @michellearning @leto__jean @animesh_garg @drfeifei @silviocinguetta

7

5

117

Yuke Zhu @ ICRA2024

@yukez

4 months

2 years ago I was shopping for a coffee machine at Target. I found a perfect Keurig not for me but for my robot: - Round tray to insert a K-cup; - Lid open/close w/ weak forces; - Coffee out w/ one button click. There's no magic. Human ingenuity is behind every robot's success.

Yifeng Zhu 朱毅枫

@yifengzhu_ut

4 months

If you want to learn more about how the task has motivated a line of research in manipulation, see the list: - VIOLA: - HYDRA: - AWE: - HITL-TAMP: - MimicGen:

0

13

70

3

13

92

Yuke Zhu @ ICRA2024

@yukez

3 years

We are organizing a #CVPR2021 Workshop on 3D Vision and Robotics to promote the cross-pollination of ideas between these two research fields. CfP is open. We look forward to your contributions!

0

21

114

Yuke Zhu @ ICRA2024

@yukez

3 years

Rewriting classical robot controller with physics-informed neural network, plugging it as learnable module into data-driven autonomy stack, trained with large-scale GPU-accelerated simulation ➡️ Adaptivity & Robustness to the next level💡

Josiah Wong

@josiah_is_wong

3 years

How can we enable robot controllers to better adapt to changing dynamics? Idea: learn a data-driven controller implemented with physics-informed neural networks, and finetune on task-specific dynamics. Website: Paper:

3

17

85

0

19

107

Yuke Zhu @ ICRA2024

@yukez

2 years

Excited to share my recent talk at the Stanford Robotics Seminar on “Objects, Skills, and the Quest for Compositional Robot Autonomy” featuring projects from my first year @UTCompSci and our lab’s vision of building the next generation of autonomy stack.

Stanford Seminar - Objects, Skills, and the Quest for Compositional...

Yuke ZhuUT AustinFebruary 18, 2022Recent years have witnessed great strides in deep learning for robotics. Yet, state-of-the-art robot learning algorithms st...

www.youtube.com

1

12

100

Yuke Zhu @ ICRA2024

@yukez

3 years

Delighted to present our recent work on hierarchical Scene Graphs for neuro-symbolic manipulation planning. We use 3D Scene Graphs as an object-centric abstraction to reason about long-horizon tasks. w/ @yifengzhu_ut , Jonathan Tremblay, Stan Birchfield

0

15

98

Yuke Zhu @ ICRA2024

@yukez

5 years

We have just released our new work on 6D pose estimation from RGB-D data -- real-time inference with end-to-end deep models for real-world robot grasping and manipulation! Paper: Code: w/ @danfei_xu @drfeifei @silviocinguetta

3

23

99

Yuke Zhu @ ICRA2024

@yukez

2 years

Excited to share VIMA, our latest work on building generalist robot manipulation agents with multimodal prompts. Massive transformer model + unified task specification interface for the win!

Jim Fan

@DrJimFan

2 years

We trained a transformer called VIMA that ingests *multimodal* prompt and outputs controls for a robot arm. A single agent is able to solve visual goal, one-shot imitation from video, novel concept grounding, visual constraint, etc. Strong scaling with model capacity and data!🧵

18

147

870

1

14

97

Yuke Zhu @ ICRA2024

@yukez

2 years

🎉Public release🎉 Thrilled to kickstart our new embodied AI moonshot: building general-purpose open-ended agents with Internet-scale knowledge!

Jim Fan

@DrJimFan

2 years

Introducing MineDojo for building open-ended generalist agents! ✅Massive benchmark: 1000s of tasks in Minecraft ✅Open access to internet-scale knowledge base of 730K YouTube videos, 7K Wiki pages, 340K Reddit posts ✅First step towards a general agent 🧵

12

129

483

1

10

97

Yuke Zhu @ ICRA2024

@yukez

3 years

I felt fortunate to attend all four CoRL conferences in the past and served as an AC the first time. @corl_conf is hands down my favorite conference - focused Robot Learning community, high-quality (<200) papers, YouTube live stream, inclusion events. I couldn't ask for more!

0

3

94

Yuke Zhu @ ICRA2024

@yukez

2 years

Texas is a booming state for robotics research and industry. We are bringing together robotics researchers across the state this Friday at Texas Regional Robotics Symposium (TEROS) 2022. Great line-up of speakers and live steam for all talks. Join us at !

0

16

93

Yuke Zhu @ ICRA2024

@yukez

4 months

We'll witness more and more demos of humanoid robots doing the same tasks the robotics community has mastered with simpler systems. Yet people will still be awed. It speaks more about human psychology than technology. Humanoids make sense in domains requiring social interaction.

7

3

93

Yuke Zhu @ ICRA2024

@yukez

2 years

We've released an updated version of ACID, our #RSS2022 paper on volumetric deformable manipulation, with real-robot experiments. ACID predicts dynamics, 3d geometry, and point-wise correspondence from partial observations. It learns to maneuver a cute teddy bear into any pose.

1

10

92

Yuke Zhu @ ICRA2024

@yukez

11 months

All talk recordings of our #CVPR2023 3D Vision and Robotics Workshop are now available on the YouTube playlist: . Check them out in case you missed the event!

CVPR23 3D Vision and Robotics Workshop

https://sites.google.com/view/cvpr2023-3d-vision-robotics

www.youtube.com

Yuke Zhu @ ICRA2024

@yukez

11 months

Heading to @CVPR today! We are organizing a 3D Vision and Robotics workshop tomorrow with a great line-up of speakers: Also, I am recruiting a postdoc on vision + robotics for my group. Come to chat with me if interested - DMs are open!

0

17

115

0

13

88

Yuke Zhu @ ICRA2024

@yukez

2 years

Spot-on! Top AI researchers and institutes have the magic power of pushing a research field years back, simply by publishing initial papers and inadvertently creating a vicious cycle of worthless publications. With great power comes great responsibility.

Christoph Molnar

@ChristophMolnar

2 years

A lot of machine learning research has detached itself from solving real problems, and created their own "benchmark-islands". How does this happen? And why are researchers not escaping this pattern? A thread 🧵

54

722

3K

0

8

87

Yuke Zhu @ ICRA2024

@yukez

4 years

Sharing the slides of my talk "Learning Keypoint Representations for Robot Manipulation" presented at the Workshop on Learning Representations for Planning and Control @IROS2019MACAU Slides: Workshop:

IROS 2019 Workshop

News: Room Location: LG-R8 Instructions for accepted papers presentations have been updated. All accepted papers will have oral presentations! Accepted papers are now available on our website!...

sites.google.com

1

18

84

Yuke Zhu @ ICRA2024

@yukez

3 years

We have six papers to be presented at #ICRA2021 this week, spanning the topics of imitation learning for manipulation, neuro-symbolic planning, multimodal perception, uncertainty quantification, and morphological computation. A thread /5

1

2

83

Yuke Zhu @ ICRA2024

@yukez

10 months

I will attend ICML in Hawaii next week to present VIMA () and meet friends. Our NVIDIA team is seeking new talent for AI Agents, LLMs, and Robotics. Reach out via DMs if interested!

Jim Fan

@DrJimFan

10 months

I'm going to ICML in Hawaii! My team pushes the research frontier in AI agents, multimodal LLMs, game AI, and robotics. If you're interested in joining NVIDIA or collaborating with me, please reach out by email! My contact info is at If applicable,…

10

23

366

0

6

81

Yuke Zhu @ ICRA2024

@yukez

5 years

Ajay gave a great talk on our RoboTurk project #IROS2019 , nominated for Best Paper on Cognitive Robotics. Large-scale real robot dataset through crowd teleportation! More information can be found at

0

20

82

Yuke Zhu @ ICRA2024

@yukez

2 years

Uploading physical objects to the virtual world (metaverse) by observing and interacting with them in the real world. Exciting new work on sim2real via real2sim with articulated objects #CVPR2022 #Ditto

1

8

79

Yuke Zhu @ ICRA2024

@yukez

3 years

robosuite v1.2 released: new sensor simulation APIs, visual/dynamics/sensor randomization for sim2real, enhanced operational space controllers, and human demonstrations! Check it out from here:

0

7

76

Yuke Zhu @ ICRA2024

@yukez

4 years

Pleased to be invited by @SamsungUS to talk about my research on robot perception and learning. Covered our latest work on self-supervised sensorimotor learning, hierarchical planning, and cognitive learning and reasoning in the open world. Video:

Towards General-Purpose Robot Autonomy: A Progressive Roadmap, Yuke...

Robots and autonomous systems have been playing a significant role in the modern economy. Although custom-built robots have remarkably improved productivity,...

www.youtube.com

1

10

75

Yuke Zhu @ ICRA2024

@yukez

4 years

Our department @UTCompSci @UTAustin is recruiting new Robotics faculty this year. Come join us in the booming city of Austin!

0

21

73

Yuke Zhu @ ICRA2024

@yukez

3 years

Excited to introduce 𝚛𝚘𝚋𝚘𝚖𝚒𝚖𝚒𝚌, a new framework for Robot Learning from Demonstration. This open-source library is a sister project of 𝚛𝚘𝚋𝚘𝚜𝚞𝚒𝚝𝚎 in our ARISE Initiative. Try it out!

Ajay Mandlekar

@AjayMandlekar

3 years

Robot learning from human demos is powerful yet difficult due to a lack of standardized, high-quality datasets. We present the robomimic framework: a suite of tasks, large human datasets, and policy learning algorithms. Website: 1/

4

57

263

0

6

73

Yuke Zhu @ ICRA2024

@yukez

4 years

A nice summary of our recent works on imitation learning from visual demonstration. Compositionality and abstraction are key to scaling up IL algorithms to long-horizon manipulation tasks.

Stanford AI Lab

@StanfordAILab

4 years

What if we can teach robots to do new task just by showing them one demonstration? In our newest blog post, @deanh_tw and @danfei_xu show us three approaches that leverage compositionality to solve long-horizon one-shot imitation learning problems.

0

39

140

1

13

72

Yuke Zhu @ ICRA2024

@yukez

10 months

Pleased to see our Sirius paper nominated for the Best Paper Award #RSS2023 : Join our presentation in Daegu, Korea on July 11th! Exciting times ahead as our lab explores the new frontier of 𝗥𝗟𝗢𝗽𝘀 (Robot Learning + Operations) in long-term deployment.

Yuke Zhu @ ICRA2024

@yukez

1 year

Like the best chess players are human-AI teams (centaurs), trustworthy deployment of robot learning models needs such a partnership! Sirius is our first milestone toward Continuous Integration and Continuous Deployment (CI/CD) for robot autonomy during long-term deployments👇

0

12

54

3

70

Yuke Zhu @ ICRA2024

@yukez

3 years

Looking forward to sharing our latest progress on GPU-accelerated robotics simulation in the Isaac Gym tutorial @RoboticsSciSys 2021 next Monday.

NVIDIA Robotics

@NVIDIARobotics

3 years

Join us on July 12th at #RSS . This workshop will introduce the end-to-end GPU accelerated training pipeline in #NVIDIA Isaac Gym, demonstrate #robotics applications, and answer questions in breakout sessions. Register here: #AI #robots #nvidiaisaac

0

24

72

0

5

67

Yuke Zhu @ ICRA2024

@yukez

2 months

I will give a talk at #SXSW2024 on How to Train a Humanoid Robot tomorrow from 10 to 11:30 a.m. Come to check out our ramen-cooking DRACO 3 robot developed @texas_robotics and learn the technical stories behind it!

2

6

63

Yuke Zhu @ ICRA2024

@yukez

4 years

Check out our recent work on building agents that self-generate training tasks to facilitate the learning of harder tasks.

Kuan Fang

@KuanFang

4 years

We introduce APT-Gen to procedurally generate tasks of rich variations as curricula for reinforcement learning in hard-exploration problems. Webpage: Paper: w/ @yukez @silviocinguetta @drfeifei

0

22

58

0

12

62

Yuke Zhu @ ICRA2024

@yukez

5 years

At #IROS2019 next week, I will be giving invited talks: 1. "Learning How-To Knowledge from the Web" at AnSWeR (Mon) 2. "Learning Keypoint Representations for Robot Manipulation" at LRPC (Fri) Come to learn about our recent work!

IROS 2019 Workshop

News: Room Location: LG-R8 Instructions for accepted papers presentations have been updated. All accepted papers will have oral presentations! Accepted papers are now available on our website!...

sites.google.com

0

16

61

Yuke Zhu @ ICRA2024

@yukez

5 years

Two papers are accepted in CVPR 2019: Neural Task Graph () and DenseFusion ().

DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion

A key technical challenge in performing 6D object pose estimation from RGB-D image is to fully leverage the two complementary data sources. Prior works either extract information from the RGB...

arxiv.org

0

10

57

Yuke Zhu @ ICRA2024

@yukez

1 month

Revisiting @DavidEpstein 's book Range: Why Generalists Triumph in a Specialized World as a roboticist unveils a compelling insight: Elite athletes usually start broad and embrace diverse experiences as a generalist prior to delayed specialization. Given this prevailing pathway…

1

5

60

Yuke Zhu @ ICRA2024

@yukez

7 months

This is a fantastic initiative and exciting collaboration between industry and academia toward unleashing the future of Robot Learning as a Big Science! We must join forces in the quest for the north-star goal of generalist robot autonomy.

Google DeepMind

@GoogleDeepMind

7 months

Introducing 𝗥𝗧-𝗫: a generalist AI model to help advance how robots can learn new skills. 🤖 To train it, we partnered with 33 academic labs across the world to build a new dataset with experiences gained from 22 different robot types. Find out more:

25

311

1K

0

6

59

Yuke Zhu @ ICRA2024

@yukez

4 years

Excited about coming back to Vancouver, the beautiful city where I attended college, for #NeurIPS2019 Looking forward to catching up with the latest research and hanging out with friends!

1

57

Yuke Zhu @ ICRA2024

@yukez

1 year

Like the best chess players are human-AI teams (centaurs), trustworthy deployment of robot learning models needs such a partnership! Sirius is our first milestone toward Continuous Integration and Continuous Deployment (CI/CD) for robot autonomy during long-term deployments👇

Huihan Liu @ ICRA 2024

@huihan_liu

1 year

Deep learning for robotics is hard to perfect. How do we harness existing models for trustworthy deployment, and make them continue to learn and adapt? Presenting Sirius🌟, a human-in-the-loop framework for continuous policy learning & deployment! 🌐:

5

59

253

0

12

54

Yuke Zhu @ ICRA2024

@yukez

6 months

MimicPlay is selected for an oral presentation next Thursday at #CoRL2023 . Come check out this exciting work led by @chenwang_j

Jim Fan

@DrJimFan

6 months

What's the best way for humans to teach robots? I'm excited to announce MimicPlay, an imitation learning algorithm that extracts the most signals from unlabeled human motions. MimicPlay combines the best of 2 data sources: 1) Human "play data": a person uses their hands to…

17

153

657

1

7

52

Yuke Zhu @ ICRA2024

@yukez

3 months

Debates abound on the necessity of a human-like form factor for generalist robots. Humanoid robots are overkill now, but they make sense from first principles. A deep tech will always be overkill until we put hard work into it. Why not stop debating and shape the future together?

7

4

54

Yuke Zhu @ ICRA2024

@yukez

8 days

Our Eureka follow-up work is out!

Jason Ma

@JasonMa2020

8 days

Introducing DrEureka🎓, our latest effort pushing the frontier of robot learning using LLMs! DrEureka uses LLMs to automatically design reward functions and tune physics parameters to enable sim-to-real robot learning. DrEureka can propose effective sim-to-real configurations…

24

109

579

1

6

54

Yuke Zhu @ ICRA2024

@yukez

2 years

Yet another manifestation of the power of hybrid Imitation + Reinforcement learning! Imitating high-level cognitive reasoning 🧠 from humans + reinforcing agile motor actions 🦿 in parallel simulation = quadrupedal locomotion in dynamic, human-centered environments #PRELUDE

Mingyo Seo

@kiwi_sherbet

2 years

Introducing PRELUDE, a hierarchical learning framework that allows a quadruped to traverse across dynamically moving crowds. The robot learns gaits from trial and error in simulation and decision-making from human demonstration. Paper, Code, Videos:

1

10

39

2

7

54

Yuke Zhu @ ICRA2024

@yukez

3 years

In case you missed our 3D Vision and Robotics (3DVR) Workshop at #CVPR2021 , we have released the invited talks, contributed papers, and panel discussions online. Check them out to learn about the latest progress in this booming research area. Website:

3DVR 2021

In recent years, there has been tremendous progress on 3D vision for analysis and understanding of 3D data, such as 3D semantic segmentation, 3D object detection and tracking. These advances have...

sites.google.com

0

7

52

Yuke Zhu @ ICRA2024

@yukez

1 year

We're excited to announce that our 3D Vision and Robotics workshop will be returning to @CVPR 2023! We can't wait to connect with researchers from both communities and exchange ideas and insights.

0

4

52

Yuke Zhu @ ICRA2024

@yukez

2 months

Check out our work on data-efficient imitation learning with behavior primitives led by @TianGao_19 !

Tian Gao

@TianGao_19

2 months

Imitation learning often involves significant human effort to collect a large dataset for robust policy learning. How can we train robust policies in low-data regimes? Our imitation learning framework PRIME scaffolds manipulation tasks with behavior primitives, breaking down…

1

17

94

0

3

46

Yuke Zhu @ ICRA2024

@yukez

3 years

Fun project with @NVIDIAAI collaborators on visual cognition and concept learning turned into a @NeurIPSConf spotlight. Looking forward to seeing more works on meta-learning and neuro-symbolic AI in this space.

1

4

51

Yuke Zhu @ ICRA2024

@yukez

2 years

Finally got a chance to watch @NandoDF 's #CoRL2021 keynote on Data-Driven Robotics. Loved the 10 lessons from his robot learning research @DeepMind and great to see the research progress over the last few years! Check it out in case you missed it.

Conference of Robotic Learning 2021 - Live

null

www.youtube.com

2

7

48

Yuke Zhu @ ICRA2024

@yukez

2 years

I see the word “foundation” in “foundation models” as a closer analog of foundation in cosmetics (a base for the rest of the makeup) than an underlying principle. The problem is that people mix it up with "foundational," and the name is so catchy that I can’t stop saying it...

1

5

46

Yuke Zhu @ ICRA2024

@yukez

3 years

Learning vision-based manipulation on a shoestring budget of time and human efforts. Bottom-up principle to discover skills from demonstrations + Skills as building blocks to scaffold long-horizon behaviors = Solving multi-stage tasks within 30min data collection on a real robot.

Yifeng Zhu 朱毅枫

@yifengzhu_ut

3 years

Vision-based long-horizon manipulation is challenging. We introduce BUDS, a bottom-up method to extract reusable sensorimotor skills from hierarchical task structures. It can compose these skills to solve multi-stage manipulation tasks with demonstrations collected in 30min.

2

4

47

0

7

46

Yuke Zhu @ ICRA2024

@yukez

4 years

Brilliant performance art of hacking Google Maps illustrating why we should keep a critical eye on our reliance on technology and algorithmic governance in our daily lives

Simon Weckert

@simon_deliver

4 years

99 smartphones are transported in a handcart to generate virtual traffic jam in Google Maps. Through this activity, it is possible to turn a green street red which has an impact in the physical world by navigating cars on another route! #googlemapshacks

375

13K

31K

1

4

46

Yuke Zhu @ ICRA2024

@yukez

5 years

Great moment to share and celebrate with family, friends, mentors, and everyone who has accompanied me throughout the amazing PhD journey @Stanford !

Fei-Fei Li

@drfeifei

5 years

It’s Commencement @Stanford today, one of my favorite days of the year! I’m a proud advisor of many wonderful students receiving undergrad, master or PhD degrees! Best wishes to you all - make all of us proud! And be a technologist with both smart and heart! <3

0

10

246

1

46

Yuke Zhu @ ICRA2024

@yukez

5 years

I am super excited to attend #CoRL2019 in Osaka this week and #IROS2019 in Macau next week, presenting our recent works on Hierarchical Planning and RoboTurk. Looking forward to catching up with friends and colleagues! Drop me a message if you'd like to have coffee and chat!

0

4

45

Yuke Zhu @ ICRA2024

@yukez

4 years

We have two papers on learning keypoint representations for robot manipulation accepted in #ICRA2020 . 6-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypoints KETO: Learning Keypoint Representations for Tool Manipulation

KETO: Learning Keypoint Representations for Tool Manipulation

Abstract We aim to develop an algorithm for robots to manipulate novel objects as tools for completing different task goals. An efficient and informative representation would facilitate the effecti...

sites.google.com

0

6

43

Yuke Zhu @ ICRA2024

@yukez

2 years

I will be at #CVPR2022 next week and #RSS2022 the week after. Looking forward to catching up with vision and robotics friends in the Big Easy and the Big Apple. Please reach out if you want to chat😃

0

41

Yuke Zhu @ ICRA2024

@yukez

3 years

robosuite v1.1 released: Highlights: 1. Major refactoring to simplify procedural generation of objects and environments for fast prototyping 2. Contributing guideline: Try it: pip install robosuite

Release v1.1 - Improved Interfaces and Bug Fixes · ARISE-Initiative/robosuite

robosuite 1.1.0 Release Notes Highlights New Features Improvements Critical Bug Fixes Other Bug Fixes Highlights While most surface-level functionality hasn't changed, the underlying infrastr...

github.com

1

4

42

Yuke Zhu @ ICRA2024

@yukez

1 year

Finally on my way to #CoRL2022 in Auckland, NZ! Super pumped to have my first conference in Oceania and catch up with robot learning friends. Reach out if you'd like to chat!

2

1

37

Yuke Zhu @ ICRA2024

@yukez

5 years

Attending #ICRA2019 in Montreal for the next three days. Will be presenting our best paper-nominated work on multimodal perception for contact-rich manipulation. Looking forward to syncing up with friends and colleagues!

1

39