Senior Research Scientist
@allen_ai
, ex-Ph.D.
@uwcse
, Interested in computer vision, robotics and deep learning, Climber on the weekends.
Opinions are my own
🚀 Imitating shortest paths in simulation enables effective navigation and manipulation in the real world. Our findings fly in the face of conventional wisdom!
This is a big joint effort from PRIOR
@allen_ai
(6 first authors!).
As of a few hours ago, I'm officially a Research Scientist
@allen_ai
. So excited to start my career working with the amazing researchers at AI2 and PRIOR team.
I feel awful working on the CVPR deadline and having first-world problems (like OMG my learning curves are ugly!), while my parents, who are both healthcare workers in Iran, keep getting re-infected and getting hospitalized over and over again!
The work was done during
@RchalYang
's internship @ PRIOR
@allen_ai
. You know what else we do during our internships after doing cool projects? We go on boat rides :D
Introducing our new approach for building robust robots: Phone2Proc!
@CVPR
23
Want to boost your sim-to-real agent’s performance from 34% to 70% WITHOUT any real-world finetuning? Are you tired of agents’ poor performance in complex real-world environments?This thread is for you!
At PRIOR
@allen_ai
, we are working at the X of robotics & CV to build the next generation of household robots. We work on ambitious projects and push boundaries in this field. If you’re interested in being a part of this team, join us! Reach out to me directly!
#hiring
#tech
Since I moved to the US 7 years ago, I have been going through a lot. Not seeing my family for years, having to leave friends behind, going through an identity crisis as an immigrant,...
All this time, I tried to cheer myself by remembering why I immigrated.
Received an email today from the NZ embassy after 5 months of waiting! They are asking if I'm still going to
@corl_conf
(happened mid-December last year). Always fun to have an Iranian passport!
P.S. I still managed to co-organize a workshop, give a talk & join a panel remotely!
Last week at
#ECCV2022
, I published a first-author paper. Unfortunately, I didn't get to present it in person. With everything going on in life, I forgot to let Twitter know about our project. The idea is to relax the sensory information for mobile object manipulation.
🤖
#RoboticsAI
#RobotLearning
! Are we targeting the right robotic tasks? Industrial or home application? Let's dive deep at the CoRL’23 Workshop, Nov 6, 8:30am-12:30pm EST, Hub 1
@corl_conf
, Atlanta. Don't miss the flaming debate at 11:50am! 🔥
Checkout our
#cvpr22
paper.
Embodied agents need to have a rich representation of the environment they are interacting with. This can be in the form of a scene representation encoding the relationships of the entities in the room.
Very happy to announce the Embodied AI Lecture Series @ PRIOR
A live lecture and discussion series in Embodied AI with a focus on cross-disciplinary work.
Lectures held biweekly Fri 11am PST. Open to all!
#embodiedai
@allenai
Details:
In PNW we do our brainstorming sessions in remote cabins with the view of the cascades. How do you do yours?
#humblebrag
An amazing 15-hour day off-site with PRIOR
@allen_ai
.
(Don't worry it wasn't all working. We also had fun activities and games.)
The first season of Embodied AI Lecture Series came to an end last Friday, with a total of 12 amazing speakers, talking about different perspectives towards embodied AI. Message me if you have any suggestions on potential speakers for the next chapter :)
Check out our new work (with Daniel Gordon, Dieter Fox and Ali Farhadi) on unsupervised representation learning from unlabeled YouTube videos. We demonstrate improvements over recent unsupervised techniques and fully supervised ImageNet pretraining.
Have you been looking for a framework for visual object manipulation and navigation? Do you need diverse cluttered photo-realistic environments for your manipulation experiments? Search no more! Introducing ManipulaTHOR, our virtual agent with a highly articulated robotic arm.
@ Computer Vision Community:
What is the RSS of Computer Vision? (i.e., a small conference where all the talks are high quality, limited partying involved and you actually get to have meaningful productive research conversations and initialize collaborations)
I was supposed to be on vacation enjoying my parents' company, but the
@CVPR
deadline got extended. At least I'm grateful for having fun colleagues who make this unexpected change of plans (aka one more week of zero-sleep-ten-coffee-a-day) enjoyable!
Deadline week
@allen_ai
for
#CVPR2024
.
While the last few days always add a bit of stress, working with this incredible team is so enjoyable and rewarding!
Where is the rest of the team ? At their desks, refusing to join us for dinner 😀
We are back with a new chapter of Embodied AI Lecture Series @ PRIOR
@allen_ai
! Our first lecture, by
@Ken_Goldberg
, is happening next week on Friday, Feb 18th at 11 am. Subscribe to our mailing list to receive more information.
Do you want to know how we trained end-to-end RGB-Only robots that navigate remarkably efficient in indoor environments without any real-world fine-tuning? Come to our poster
@CVPR
#137
#CVPR2023
! Starting Now! W/
@anikembhavi
@LucaWeihs
@mattdeitke
Rose Hendrix and Ali Farhadi
#CVPR2022
Discussing the "potential negative impacts" of one of our submissions:
Me: I really can't think of anything! What could this work possibly negatively impact?
Co-author: My mental health!
Me [thinking]: Oh, in that case... I have so much to write...!
One of the biggest goals was for more freedom, especially as a woman in a country with extremely sexist lawmakers. Seeing what US lawmakers did on Friday makes me reconsider my decision. This was truly a sad day for America. A new low...
Most representation learning works these days focus on contrastive approaches. What if we considered human attention and muscle movements as a supervisory signal? Check out our new large-scale dataset and representation learning approach.
Paper:
It's difficult to send your daughter to school every day without knowing if she'll make it back home safely or if you'll end up rushing to the hospital because her entire class got poisoned. Unfortunately, this is the reality in Iran right now.
Find yourself a robot that stares at you affectionately, just like mine does!
I was cleaning the random snapshots my robot had saved and found this photo! It's cute/creepy that it's staring at me even when I'm not paying attention to it!
The work was done during
@RchalYang
's internship @ PRIOR
@allen_ai
. You know what else we do during our internships after doing cool projects? We go on boat rides :D
Before every conference deadline, I want to quit research forever and abandon coding, writing papers, making figures. And after of them, (Not hours after! Just milliseconds after we submit.) I fall back in love.
#ICCV2021
#AdrenalineRush
One of the things I really enjoy is listening to Persian Dance Music while working close to deadlines. It truly boosts my morale! Even if the world's collapsing and nothing's going according to the plan! So if you see me slightly dancing while writing papers, that's why!
It just makes me sad. Makes me remember my privileges and how it's only out of LUCK that I have what I have today. Makes me think about the classmates who are smarter and more qualified than me and are protesting in Iran and risking their lives right now.
Excited to see that our work on ProcTHOR has won a
#NeurIPS2022
Outstanding Paper award!
This was a team effort but a huge shout out to Matt Deitke who really drove the work ()!
Jetting across the continent for a 24-hour adventure to
@corl_conf
✈️. In Atlanta on Monday? Let's connect! 🤝
#CoRL2023
P.S. Hiring interns for summer 2024! Curious about what we've been cooking up? 😁🤫🔥🍳
#PunIntended
#HiringNow
Wait, WHAT? How did this happen? It has been 1 year already??? Am I just enjoying my time at
@allen_ai
and that's why I can't feel the time anymore :D? Seriously though, how is that possible?
As of a few hours ago, I'm officially a Research Scientist
@allen_ai
. So excited to start my career working with the amazing researchers at AI2 and PRIOR team.
Reach for the stars! 🌟 Our 800K-object Objaverse unlocked new horizons. Now, we're aiming higher with Objaverse-XL: the largest known 3D dataset with 10M assets!
Introducing Objaverse-XL, an open dataset of over 10 million 3D objects!
With it, we train Zero123-XL, a foundation model for 3D, observing incredible 3D generalization abilities: 🧵👇
📝 Paper:
CVPR's deadline nights are not fun anymore when everything is virtual! I appreciate pre-covid era even more now! Good old days when we could all work from the same place!
Excited to share our work (with
@shubhtuls
, Saurabh Gupta, Ali Farhadi, and Abhinav Gupta), on predicting the physical forces from RGB videos. The work was done during my internship at FAIR.
Don't miss out our talk at CVPR2020 (if it happens😀)
PDF:
It was also a reminder of how HARD it is to be a girl! Equal pay? Heh! We're still dealing with more basic things here! Like whether I am forced to have another human being grow inside me and change my life and career forever!
It felt weird being on the other side of the mentoring table at
@WiCVworkshop
@cvpr
. Joining as a mentee 5 years ago helped me a lot to find myself as a researcher and to become who I am today.
#WiCV
Snapshots from our social dinner event
@CVPR
We would like to thank our keynote speakers, all mentors and mentees for attending and for sharing their experiences!
This is by far one of the coolest demos I have ever seen! I personally witnessed some of the video recordings and I can attest it is NOT cherry picked!
@SongShuran
and her team doing amazing work like always!
Can we collect robot data without any robots?
Introducing Universal Manipulation Interface (UMI)
An open-source $400 system from
@Stanford
designed to democratize robot data collection
0 teleop -> autonomously wash dishes (precise), toss (dynamic), and fold clothes (bimanual)
#CVPR2020
is over! And yes, there was no "real" networking event, and sure, not many people showed up for Q&As. But this was my most productive CVPR as an attendee. I loved that I could teleport between workshops and/or watch the recorded videos whenever I want.
We are starting the new series of Embodied AI Lecture Series @ PRIOR
@allen_ai
. Message me if you have any suggestions for the invited speakers or the topics you'd like us to cover.
Her crime? Not completely wearing her hair with Hijab. Read the last sentence again. It’s not “not wearing her hair”, it’s “not completely wearing her hair”. It’s not “was forced to pay a fine”, it’s “was murdered”.
I gave a talk on one of my research projects last night in an improv comedy show! This was one of the coolest events I've ever been a part of. Super creative comedians do an improv show on the presented works. Thank you for hosting me
@BanditScience
@CaroDuncombe
Presenting our work at virtual
#CVPR2020
(with
@shubhtuls
, S. Gupta, A. Farhadi, and A. Gupta), on predicting the grasping points and physical forces from videos.
Q&A: Tomorrow at 10am and 10pm PST
Video:
Project page:
#cvpr20
@rdesh26
F1 process for Iranians:
- filling application : ~ 1 hour
- trying to book interview slot: 2-3 months (if lucky)
- standing in line at embassy: ~ 2 hours
- interviewing: < 3 minutes
- AP: 3-12 months* (sometimes years...)
*Can be rejected with no explanation!
Then I open my academic twitter. Someone just published a paper on a new topic. Someone just found a new model for a task. Someone just got their PhD. Someone released a new framework.
Nothing wrong with it and I'm happy for/proud of all of them/you. But...
Sim2Real! Sim2Real! Sim2Real! After years of seeing push back against sim2real in my robotics journey, it's awesome to see more and more research proving how cool Sim2Real is! Big shoutout to
@xiaolonw
for their amazing work showing off Sim2Real in humanoid robotics! 🤖
#sim2real
Let’s think about humanoid robots outside carrying the box. How about having the humanoid come out the door, interact with humans, and even dance?
Introducing Expressive Whole-Body Control for Humanoid Robots:
See how our robot performs rich, diverse,
Instead of scrolling twitter up and down, searching for papers on
@SemanticScholar
or Google, or searching for most discussed papers, for this CVPR
#CVPR2021
, I'm trying CVPR Buzz
by Matt Deitke.
And I LOVE IT!
My colleague (
@Mitchnw
) and I made an astonishing scientific discovery a year ago but we never published it. I guess it's time to share it with the community. This phenomenon happens every year from November 8th-15th.
Tomorrow (Monday) at 9am PST, I’ll be presenting our recent work “What Can You Learn from Your Muscles? Learning Visual Representation from Human Interactions”
@iclr_conf
#iclr2021
. Stop by our poster zoom for a chat about this work (or how virtual conferences are not fun! 😒).
Now I'm totally tempted to do another Ph.D.! Ana is gonna be the best advisor one can wish for. And not just in doing amazing research but also being a fabulous cool energetic kind 😎 human being! Can't wait to explore Utah's ⛷️🧗♀️🗻🏔️ with you :)
Some good news! I'm joining the University of Utah
@UUtah
@UtahSoC
as an Assistant Professor this summer. I'm excited I'll be part of the U's NLP group and work alongside
@viveksrikumar
&
@EllenRiloff
!
🤖 Navigation and Manipulation are inseparable!
🚪Opening doors & 🧹cleaning tables are possible using ONLY RGB & simulation-based training, w/ NO real-world fine-tuning 🌍 or data collection. [Done during
@RchalYang
's internship
@allen_ai
using
@hellorobotinc
RE1 🤖]
How to tackle complex household tasks like opening doors, and cleaning tables in real?
Introducing HarmonicMM: Our latest model seamlessly combines navigation and manipulation, enabling robots to tackle household tasks using only RGB visual observation and robot proprioception.
ProcTHOR doesn't need advertising! It speaks for itself. Just check out the demo, or train a model with it. You'll see why we are very proud of this work! Unlimited rooms, unlimited configurations!
Massive datasets have enabled many recent advancements in computer vision and NLP. ProcTHOR presents a platform to enable similar success stories in Robotics and Embodied AI in general.
ProcTHOR website:
Details:
"I'm not fearlessly brave, but I don't have time to let fear affect my life. We only get one shot. It's a gift to live." - Ida Kaller-Vincent, an awe-inspiring researcher and climber who battled lung cancer twice.
Check out our recent work on evaluating self-supervised representation models. We examine over 700 training experiments including 30 encoders, 4 pre-training datasets, and 20 diverse downstream tasks.
Are we using the right metrics to measure progress on self-supervised representation learning? Should we discard an encoder that doesn't provide a good initialization for ImageNet classification?
We try to answer these types of questions:
Kotar, et al.
Today I bumped up my comfort speed for listening to audiobooks/podcasts/videos to 2.5x (from 2x). Fascinated by how our brains can adapt if we make slow changes!
This point is never mentioned: There are *so many* international grad students "stuck" in the US because they might be denied re-entrance / renewing their travel visa would take forever. Their only chance of presenting in person is that occasionally conferences are in the US.
Our planners have full observability. Exploration is just an emergent behavior! (Really surprising, I know!) Remarkably, the agent learns to explore and backtrack (see website examples). Here is a sample training trajectory.
An amazing blog post by
@jeffbigham
about Mechanical Turk. Highly recommended, especially if you are planning on doing an MTurk experiment anytime soon.
Another death, ... One year later, same reason, same place!
By the way, do you know what happened to the two girls who photographed the parents mourning their daughter's loss last year?
Today they got their sentence! Collectively 25 years of jail.
Such a sad day for Iran...
Looking to scale up in 3D vision, embodied AI, or generative models? Objaverse has got your back! Access high-quality 3D objects for free and with ease via
@huggingface
.
#scaling
#AI
#CVPR2023
Introducing Objaverse, a massive open dataset of text-paired 3D objects!
Nearly 1 million annotated 3D objects to pave the way to build incredible large-scale 3D generative models: 🧵👇
🤗 Hugging Face:
📝ArXiv:
#CVPR2023
When they talk about the lines of code one's written in a programming language do they only count the lines written conciously? If not, I might have the world record for python! Close to deadlines I rewrite my projects in my dreams over and over and over...
Applications are now open for summer 2021 internships with the PRIOR (computer vision) team at AI2! Join a creative team working at the forefront of
#computervision
research and tech for the common good.
Deadline to apply: Nov 20, 2020
Apply here:
Tomorrow, we are honored to host
@DoctorJosh
@ Embodied AI Lecture Series @ PRIOR
@allen_ai
.
He is going to present a method that designs biological machines from the ground up: designing new machines in simulation and building them using biological tissues.
This week on Friday (at 11 am PST) we are hosting Luca Carlone
@lucacarlone1
in our Embodied AI Lecture Series @ PRIOR
@allen_ai
. Subscribe here to receive the meeting link.
What can embodied AI agents learn from gameplay? Our
#ICLR2021
paper () shows that by playing hide-and-seek, agents learn representations rivaling (self-)supervised approaches. We test our agent's with a suite of tasks inspired by experiments for infants.
Join us for Summer'22 internships. Message me if you are interested in hearing more about our Embodied AI Research. We have exciting new research directions!
Applications now open for summer '22 internships with the PRIOR (computer vision) team
@allen_ai
We are looking for interns to work in several areas including: Embodied AI, Vision-&-Language and AI for the Common Good.
Deadline: Nov 8, 2021
Apply here:
We are back with our Embodied AI Lecture Series @ PRIOR
@allen_ai
. Tomorrow, we are honored to host Nick Cheney. This session might be different from what you normally expect of an E-AI talk. Nick is going to explain how an "agent's body" contributes to its "intelligent behavior".
Look who's finally here!
@mattdeitke
The cloud button is for expressing objections (comments), the heart button is for saying you approve (like), and the arrows are for promoting other people's ideas (retweet). Here goes your first tutorial! :D You're welcome!
Different scenarios, different behaviors, right? 🤔 So why not have one model that adapts to what you need instead of building a new one each time? 🚀 That's what we did! Had a blast working with Minyoung on this 👩🔬👨💻. Oh, and she's hunting for a Ph.D. program now! 🎓
We only train once: How can we effectively customize a robot for multiple users?
We propose 'Promptable Behaviors', a novel personalization framework that deals with diverse preferences without any retraining.
website:
paper:
🧵👇
If you notice any of us, Iranian colleagues, seeming emotional at work, don't ask anything unless you're prepared for a waterfall of tears. Sharing the pain with the women in Iran...
Checkout our
#cvpr22
paper.
Embodied agents need to have a rich representation of the environment they are interacting with. This can be in the form of a scene representation encoding the relationships of the entities in the room.
You might consider rotating a vector by 90 degrees as a super easy calculation for an AI agent (w/o explicit supervision w/ deepnets)! The news is it's not as easy as you'd think! And yet some people are worried AI might overtake us? LOL!
#DebuggingAIModelsHurts
Just a reminder that if you'd like to receive the link for this Friday's meeting, today is the last day to subscribe to our mailing list ().
Speaker:
@DhruvBatraDB
Title: Do Blind AI Navigation Agents Build Maps?
Time: Friday May14th 11am PST
Very happy to announce the Embodied AI Lecture Series @ PRIOR
A live lecture and discussion series in Embodied AI with a focus on cross-disciplinary work.
Lectures held biweekly Fri 11am PST. Open to all!
#embodiedai
@allenai
Details:
🌟What I’m offering: A fantastic internship @ PRIOR
@allen_ai
🤩. Experience a beautiful summer in Seattle ☀️, get hands-on mentorship, work on cool project and meet awesome people at AI2 and UW! 🤝
Applications open for a 1-2 yr Computer Vision residency
@allen_ai
-Partner with strong mentors
-Engage in impactful research
-Author papers at top venues
-Boost your grad school application
Importantly, enjoy a collaborative and supportive environment.
As one of the first users of this framework, I can say it is very easy to use and you should definitely give it a try if you are working on embodied AI.
AI2 is proud to announce our new
#embodiedAI
framework AllenAct! This library offers free, open, first-class support for a growing collection of embodied environments, tasks, and algorithms, plus reproductions of state-of-the-art
#AI
models.
Learn more:
It's just time-consuming to check the snow fall / weather trends every time you wanna go touring and you might miss some important weather history that has resulted in unstable snow layers beneath the surface.
I started my talk by acknowledging the fight for freedom in Iran. That's what my Tshirt says as well. Be the voice of people in Iran, especially our brave women. No matter what you do or where you work, every single voice matters!
Just watched the keynote talk from
@iclr_conf
. Not only a great informative talk on interesting topics but also changed my whole perspective on online talks. I'm re-evaluating how I think about making a conference talk! Amazing lecture by
@mmbronstein
Have you ever thought of turning mobile phones into robots? If yes, join us TOMORROW (FRIDAY 23rd) at 11am PST
@allen_ai
PRIOR E-AI Lecture Series. We are honored to host Matthias Müller from Intel. He will talk about leveraging smartphones to equip robots with extensive sensors.
🧑Popular Opinion: Simulation is not enough and real-world fine-tuning is a must.
🔭Our Finding: SPOC, trained in simulation only transfers remarkably well to the real world. No adaptation, No fine tuning.
-"After AlexNet took first place in the ImageNet Challenge, everybody of a certain age went through the five stages of grief."
I constantly try to think about what is the "AlexNet of the day" and what I might be grieving through denial :)
I've loved
#ICCV
since my first one in 1990. In this blog post, I reflect on the last 31 years of ICCV and the field of computer vision. Hopefully you enjoy this on the last day of
#ICCV2021
. See you in Paris in 2023!
excited to share Act the Part (AtP), a framework to learn how to interact with articulated objects to discover and segment their parts!
arxiv:
website w/ demo:
joint work w/ .
@ehsanik
and .
@SongShuran
1/5