Deep learning isn't hitting a wall but my ability to keep track of it is. So I started compiling links to milestones from the past few years at . I hope it'll serve as a useful reference for you! Or that you'll at least the enjoy the Dall-E illustrations
This is a compilation of the advice I've found most useful while navigating my CS PhD . It's what I wish I had starting out at CMU, not only because it has some of the best free wisdom out there, but also because it's ꧁ ✦ p i n k ✦ ꧂
I’m excited to announce that I’ve started a new role at
@openai
as Voice Actor
#6
!
Thanks
@joannejang
for giving me my big break and for bringing this delightful feature into the world
Use your voice to engage in a back-and-forth conversation with ChatGPT. Speak with it on the go, request a bedtime story, or settle a dinner table debate.
Sound on 🔊
Yesterday, I defended my PhD thesis! 🤗🥳🎓 Something I hadn't realized about grad school is that cultivating a PhD student is a massive group effort. THANK YOU to everyone who invested in me and our work. Hoping to make you all proud and to pay it forward.
Excited to share what our team has been working on! You can now use GPT-3 to edit existing text or insert in the middle of text. Available in the API today!
Prompt: “A movie trailer featuring the adventures of the 30 year old space man wearing a red wool knitted motorcycle helmet, blue sky, salt desert, cinematic style, shot on 35mm film, vivid colors.”
We have reached an agreement in principle for Sam Altman to return to OpenAI as CEO with a new initial board of Bret Taylor (Chair), Larry Summers, and Adam D'Angelo.
We are collaborating to figure out the details. Thank you so much for your patience through this.
At the end of the day, the greatest privilege of my job is working with people who are driven by mission. These last 5 days, I saw people across OpenAI remaining calm and resolute in driving their mission despite all that was happening around them. And I saw people across…
Jokes aside, I actually DID start a new gig
@OpenAI
to work on public policy in our Global Affairs team! Kicked it off by following
@sGianella
around Europe and trying my best to be helpful to EU policymakers. Super fun and a bit surreal tbh!
I deeply regret my participation in the board's actions. I never intended to harm OpenAI. I love everything we've built together and I will do everything I can to reunite the company.
Governance of an institution is critical for oversight, stability, and continuity. I am happy that the independent review has concluded and we can all move forward united.
It has been disheartening to witness the previous board’s efforts to scapegoat me with anonymous and…
So. About
@OpenAI
. I've been bombarded by sales calls from rival LLM companies seeking some opportunistic business wins. That's fair game and business is a contact sport so I get it. And we were already evaluating backup providers as a hedge before the drama. But the reason we're…
Throwback to distributing my thesis committee survival kits. Equipped with red pens, espresso beans, Advil, tea, and a mug for student tears. Everything you need to get through a dissertation!
It's my first day at
@DeterminedAI
! Super excited to help build out the DL training platform that I wish I had during the last 5 years. Wish me luck! Or better yet, head over to Github and check us out!
Happy birthday GPT4 ❤️
The story continues to be the friends we made along the way, from
@jiangelaa
(and so many others) to working with Khan Academy, Duolingo, BeMyEyes, to many more who are just as passionate about the potential future.
It's been a humbling year.
Now…
We are adding support for plugins to ChatGPT — extensions which integrate it with third-party services or allow it to access up-to-date information. We’re starting small to study real-world use, impact, and safety and alignment challenges:
Was just introduced to the concept of new years ~themes~ and I love it. Resolutions tend to be all-or-nothing and rigid (go to the gym 8x a week), whereas a theme (the year of health!) rewards incremental progress and encourages using a variety of approaches to improve.
A personal tutor, powered by GPT-4!
@khanacademy
was one of our first collaborators in discovering what GPT-4 was capable of. It's been so fun working with the team and
@JessicaShieh
to make the vision of an AI tutor come to life!
Did you hear? 👀 Khan Academy is using GPT-4 from
@OpenAI
to shape the future of learning.
Starting today, you can sign up to test our AI-powered guide, Khanmigo. A tutor for learners. An assistant for teachers.
Come explore with us! ✨
🌟GPT-4⃣is here!🌟
It gets human-level perf on lots of benchmarks (passes the bar!), has 32K context (~50 pgs of text), and processes images too! 🖼️
Join the waitlist:
Or skip the line by contributing to
💡 - The OpenAI Global Affairs team continues to grow
Looking for a creative policy pro to shape and drive our work, advocacy and partnerships at Member States level in the EU.
The role can be based in 🇮🇪 Dublin, 🇧🇪 Brussels, 🇪🇸 Madrid or 🇫🇷 Paris.
Liam and I wrote about how Determined would have saved us from wasting countless hours writing boilerplate code and hacking together infrastructure during our PhDs. It's one of the reasons I joined
@DeterminedAI
and why I'm so excited to share our product!
Excited about this one! We've adapted canonical
@huggingface
examples to train on multiple machines and/or spot instances. No need to change the code. You can just configure the # of GPUs or even an HP search by tweaking the config. All examples here:
1 week from today: We're hosting a lunch-and-learn on scaling NLP model training ‼️ 📅
Learn how to build a chatbot using a
@huggingface
Transformer language model with distributed training & spot instances - 𝘢𝘯𝘥 𝘨𝘦𝘵 𝘧𝘳𝘦𝘦 𝘭𝘶𝘯𝘤𝘩 𝘰𝘯 𝘶𝘴!
Love this idea for
@SCSatCMU
😍. I don't know where I'd be without
@bustamantefe
graciously reviewing my apps 6 years ago. Feedback from researchers on grad school applications is a big plus, but is hard to come by if you're not already plugged in to the community.
We’ve started a Student-Applicant Support program for underrepresented students. If you’re considering applying to the Computer Science PhD program at Stanford, we will do our best to give you one round of feedback on your application. Apply by October 31
I couldn't have said it better myself! If you're getting started with deep learning, check out
@CShorten30
series on distributed training and HP search with
@DeterminedAI
, as well as his other instructional content from
@labs_henry
!
This video explains how to use the Determined Training Platform for CIFAR-10 hyperparameter optimization!
- Configuration yaml files ⚙️
- Data and Model definitions 🏛️
- Experiment Tracking Viz. 👀
- Hyperparameter Optimization Gains 🎉
#100DaysOfMLCode
@mcoblenz
I stood up during the talk because it helped with my presentation energy. And remember to get a picture with your committee :)
Good luck! I'll be there! 2/2
@DanielWongLK
Omg you're gonna make me cry! 😭 I'm beyond grateful for our time together and your support through all the craziness. I couldn't have done this without you and I'm so excited to witness the rest of your journey!
@5trobl
@dave_andersen
@eladhoffer
@arXiv_Daily
@zacharylipton
Thanks for your interest! Intuitively, SelectiveBackprop is the most effective during later stages of training because early on most examples are useful. So, I would expect that starting with traditional will improve accuracy moreso than ending with traditional.
@mcoblenz
Thanks! I thought doing a zoom defense was a surprisingly smooth experience. If you have a waiting room, I'd suggest adding 2 co-hosts to monitor it, where at least one is a committee member. That way there's a host present when the meeting is committee members only. 1/2