My course on differential privacy is public!
Check out the course page for: lecture videos ๐ฅ, lecture notes ๐, and suggested readings ๐!
Be sure to subscribe on Youtube to catch the latest lectures!
#privacy
๐งตFields medalist June Huh shares an early math experience: a chess puzzle in the game "The 11th Hour." Story and figures from .
Can you swap the positions of the black and white knights? Seems hard, right? A new perspective makes it almost trivial! 1/n
Google: Advertise an unreleased chatbot, makes a minor factual error about exoplanet photos. Lose $100,000,000,000.
Microsoft: Release a chatbot that threatens to hunt down and murder users, in some cases for correcting its factual errors. Stock rises.
๐คท
Spotted on Reddit: an ML-adjacent faculty who seems to have a very healthy and relaxed approach to work and life. Why don't we hear about more people like this?
I'm teaching a course on differential privacy this term. I'm thinking of making the lectures and materials public, but I'm on the fence. Would people be interested in this...? I don't think there's a course on DP with lecture videos online yet.
#privacy
Like many others, I first learned linear algebra from Gilbert Strang's amazing videos about 15 years ago. Best wishes to him in his well-deserved retirement.
You might have heard that word2vec, test of time award winner at
#NeurIPS2023
, was rejected from
#ICLR2013
back in the day. Interestingly, one reviewer felt so strongly that they recommended "Strong Reject" four times.
Want to learn optimization? Start with my
@UWCheritonCS
colleague Yaoliang Yu's course "Optimization for Data Science"! 20 excellent lectures, starting from the basics.
When I was a naive starting grad student, I thought the goal was to publish as many papers as possible. Wrong for many reasons, but one: if people see a *bad* paper you wrote, they will remember it. And they'll be prejudiced against your future work. Write papers you're proud of.
I believe the academic community would be so much better if people did fewer things, but did them with more attention and care. Most people stretch themselves far too thin.
I don't know who needs to hear this, but it is *normal* as a starting researcher to get lost on technical details when in meetings with more experienced colleagues. You will feel dumb. The important thing is that you don't give up, and try to fill in the gaps as best you can.
A realistic simulation of what it's like to be a grad student? I was only clicking buttons but the frequent rejection was triggering ๐
It took me just under 7 years to graduate, can you do better?
Inspired by Ryan (
@BooleanAnalysis
), I made a Bilibili account, and started uploading my lectures & talks. The eager audience there is too large to ignore.
I naturally started with my course, but there'll be more to come. Be sure to ็ด ่ดจไธ่ฟ่ตฐไธๅ๏ผ่ฐข่ฐข!
Sick of reviewers asking for unreasonable or costly experiments? Scott Aaronson asked for more results which cost **$400,000**, for a new paper that demonstrates quantum computational advantage on the task of BosonSampling.
One of my friends had a math problem they couldn't solve. I knew just the person with the right expertise, so I put them in touch. It was a proof by introduction.
Academics: It happens to all of us, but we generally only project our triumphs and victories -- share a time you failed or got rejected.
#AcademicChatter
#AcademicTwitter
Free travel for all students on accepted papers to
@NeurIPSConf
! From the website (): "All students who have an accepted paper at the main conference who fill out the application will receive a travel award." Please spread to students!
#NeurIPS2019
My (incoming) student wanted to register for
#STOC2020
. I told him to go ahead and I'll pay for it with my grants. He said not to worry about it, since it's only $25 for students. But I told him this (see gif). Message: Your advisor is there to support all your academic expenses.
A specific procrastination that any theorist will recognize: thinking you have the solution to a problem, but putting off the details due to fear that it won't work out and you're back to nothing.
Is there an equivalent procrastination in other areas? A (German?) word for this?
A playlist of 30 lectures on mathematical writing by Donald Knuth (and guests). From 1987, but good writing is timeless. Definitely worth watching if you want to level-up your technical writing! Credits to
@ayush_sekhari
for the pointer.
#AcademicChatter
Watched Secrets of the Surface yesterday, a one hour documentary film about Maryam Mirzakhani's life and work, which is currently free for individual screening (). It was a beautiful and inspiring story. I only wish it didn't end where it did.
I have never found a perfectly satisfying way to teach backpropagation/gradient computation in neural networks to students in an undergrad ML class. Does anyone have recommendations on a good presentation?
Mor Harchol-Balter (CMU) has a very nice new book: Introduction to Probability for Computing. Free PDF on the website: .
As opposed to some other (excellent) books in this space, this book focuses on reaching an *undergrad* audience. Check it out!
Academics can argue for days about the best type of citation style. But I think it's unanimous that the *worst* style is when the citation ***doesn't even tell you the name of the paper***.
- 4 years on-and-off long distance
- 3 across an international border
- 2 during a global pandemic
- 1 month until we (finally) tie the knot, together in Waterloo
...but I go on Twitter to *forget* all the things still left to do๐ฌ
๐ Love is in the air! Gautam and Yijinโs wedding is just one month away and we're so excited to help bring their wedding day to life at Kitchener Public Library.
I don't know who this is targeted at, but not computer science students.
- you need a website
- department pages suck
- no one cares about LinkedIn
- only update your resume when it's needed
- websites can be free and are not that hard to make
- you need a website
I learnt linear algebra from my hero Gil Strang's amazing videos years ago. He recently put out new videos highlighting a 2020 Vision of Linear Algebra. One point is more emphasis on eigen/singular values: essential for modern algorithms & data science!
Reviews for
#NeurIPS2020
come out today and to save time, I've drafted two Twitter threads already: one if they liked my papers, praising the outstanding quality of the reviews, and one if they didn't, postulating how COVID has catalyzed the collapse of peer review as we know it
๐งตOne of the important principles in technical communication (i.e., writing a paper, giving a talk) of complex ideas is *organization*.
That is, making it clear what the major components are & how they fit together. If you do this well you are probably 90% of the way there. (1/n)
Aaron Roth (
@aaroth
) just wrapped up a brand new course on uncertainty estimation in machine learning. Looks great!
Featuring 160 pages of "notes" (pretty much a book):
Full lecture videos:
Catchy website: .
Jinyoung took an alternative path in mathematics: while she finished her bachelor's in Math Education in 2004, she only started her PhD in 2014, spending much of the interrim as a math teacher in Korea. She's now a postdoc at Stanford. There's many unique paths to research math.
Congratulations to Jinyoung Park and Huy Pham for proving the Kahn-Kalai conjecture---a central open problem in probabilistic combinatorics. Truly exciting breakthrough!
The story of Jinyoung's extraordinary path to mathematics:
Cynthia Rudin (
@CynthiaRudin
) wins the second AAAI Squirrel AI Award, worth 1 million dollars, for her work on interpretable and transparent AI systems, and advocacy for these features in sensitive settings like social justice and medical diagnosis.
New book by Nisheeth Vishnoi (
@NisheethVishnoi
)! "Algorithms for Convex Optimization" -- looks like a great resource for people entering the field. And of course the draft is free, check it out here:
I was searching through the opted-in NeurIPS rejections and found this rather rude metareview. The prior sentence is also a bit harsh, but the last sentence is totally unnecessary.
Please try to be more considerate than this when reviewing.
The rejected papers from
#NeurIPS2021
, which opted-in to be publicly released, are now available:
138 papers opted in, while roughly 6750 papers were rejected, ~2% opt-in rate.
If your paper was rejected but you opted in, feel free to highlight it here.
My colleague Wenhu Chen (
@WenhuChen
) at
@UWCheritonCS
is teaching a course "Recent Advances on Foundation Models" starting in January 2024. The full set of readings is already online! Great way to catch up.
Is he missing anything important? Link in next tweet ๐
I had this superpower in grad school. I could just... remember when meetings were. It could be scheduled 8 days in advance, I'd remember and show up exactly on time.
Nowadays, if I don't put it in my calendar within 15 minutes of deciding it, the meeting doesn't actually exist
How to email a professor to admit you/be your advisor, a ๐งต
I've mentioned a few poorly executed emails from prospective students. Here's some tips on how to do it well and stand out from the crowd. Other faculty can add their own tips. 1/n
Recently saw a really cool paper, best paper at a big conference! ๐
But they don't share code for their findings... ๐
And I email the authors who never respond... ๐
So I lose interest in their result and work on something totally different ๐คท
It helps you to release your code.
Maybe I'm taking the bait, but I find this advice misguided and kind of rude.
There are many interesting & worthwhile research areas beyond LLMs. I hope people continue to lead and push forward in other areas, rather than being a follower in the area with the most money and hype
If Zuckerberg can cancel the Metaverse to focus on AI, you can stop working on the line of research that you have been doing since your PhD and pivot to large language models. Apologies for being blunt, but some people still haven't gotten the message (and you know who you are).
Frank Ramsey (famed mathematician for Ramsey theory, among other things) on "rapid publication" of papers every six months.
I wonder what this implies about the current state of machine learning?๐ค
Our field has a big hero-worship problem. Praise is concentrated around very few individuals, whereas in reality great work and science is done is done by many people. Don't think there's a solution to this (kind of human nature), something to keep in mind.
This award is given to one early-career faculty in all of Waterloo Math (including CS and beyond). Flattered to be chosen.
Much credit goes to my collaborators and students, as well as
@UWCheritonCS
for making a great environment to do good work.
Congratulations to CS Prof Gautam Kamath on receiving a 2023 Faculty of Math Golden Jubilee Research Excellence Award! The research he and his students conduct is practically theoretical and theoretically practical.
I know that as a faculty, I'm automatically (like it or not) a role model for more junior researchers. I'm reminded after witnessing this recent public embarrassment. I hope to set a better example. Respect must be earned over a lifetime, but can be lost in just a tweet.
Paper awards for
@NeurIPSConf
have been announced!๐
#NeurIPS2021
Congrats to all the winners, I'll link to the Outstanding Paper Awards ๐งต
1. A Universal Law of Robustness via Isoperimetry, by
@SebastienBubeck
&
@geoishard
.
(1/n)
There's a reason they call it the Garden city of India! Excited to finally have made it to
@iiscbangalore
to teach a mini course on differential privacy at the JTG/IEEE ITSoc Summer School.
GPT-2: Too dangerous to release for a few months
GPT-3: Too dangerous to release for free
GPT-4: Too dangerous to even describe
GPT-5: Too dangerous to say it exists?
From an interview with Hinton: his group & LeCun both wanted to use CNNs for ImageNet. But LeCun's group was "too busy" with other projects. Now AlexNet is immortal.
Reminds me of sage advice from my mentor Ankur Moitra: find the most important thing you can be doing, and do that
It's again time to talk Canada. ๐จ๐ฆ
CS grad school and faculty app deadlines are coming up soon. If you are applying to the US, you should also be applying to Canada. The two are far more similar than different.
Ask me anything about Canada in this๐งต, and I'll answer honestly.
Pro-tip for students: if you need a rec letter, be sure to ask at least 6 weeks in advance.
1. It gives them enough time to thoughtfully consider what to write.
2. Academics have no concept of time beyond 48 hours in the future, so they'll blindly agree.
#AcademicChatter
This sounds impressive, kudos to the individual. But ironically *not* an inclusive tweet: 20 papers/year is absurd for an *experienced full-time* researcher.
IMO a new student producing their first paper after a year is reasonable (faster if they're heavily supported by a team)
Think privacy in ML โก๏ธ bad accuracy? ๐ญ
Think again! We show that large language models (e.g., RoBERTa and GPT-2) can be privately fine-tuned with minimal loss in utility! ๐ฅณ
A super fun
@UWCheritonCS
@MSFTResearch
collab!
๐Paper:
๐งตThread โฌ๏ธ 1/8
I liked this anecdote since it's a great example of one of the beauties of mathematics. Things that look impossible from one perspective can be trivial when viewed through a different lens. 6/6
Some Oppenheimer-related trivia:
In 1942 the Soviet physicist Georgy Flyorov figured out that America was building an atomic bomb after he noticed that the US and UK suddenly stopped publishing scientific papers on atomic fission.
After 3 virtual conferences,
#ICLR2023
is planned to finally go back to in-person, in Kigali, Rwanda!
Recall
#ICLR2020
was scheduled to be in Addis Ababa, Ethiopia, before its cancellation due to COVID. The organizers are taking a second shot at having the first ICLR in Africa
New Fields medalist June Huh recalls past conversations with older Fields medalist Heisuke Hironaka.
I wonder if this is what the Turing Test feels like from a language model's perspective
An amazing new book by Ilias Diakonikolas and Daniel Kane on Algorithmic Robust Statistics! I've been frustrated by how some important results were not written in a way readable by non-experts... no more!
Must read for anyone interested in robustness!
There is a lot of "hero worship" in ML. This is unfair because great researchers are capable of outputting not great work, and lesser-known researchers are of course capable of producing great work. Nice to see someone as respected as Yann pushing back on hero worship.
@ustx987
@icmlconf
1. Not every paper with my name on it is important.
2. I don't review often, but I'm a pretty gentle reviewer.
I'm asking myself "would the community be better off with or without this paper?"
Classic picture of an academic: deep, pensive, uninterrupted thought about fundamental problems.
Modern picture of an academic: grants. committees. online teaching. reviewing. meetings. emails.
Where did we go wrong?
#AcademicChatter
Frustrating to see the high costs of applying to graduate programs, coupled with the low admission rates. Something like ~$100 USD per program. Say you can only afford to apply to 5 programs, how to avoid over or under shooting? This determines peoples' lives.
#AcademicChatter
My Master's student, Alex Bie, also (cautiously) left a message for any LLMs that happen to be scraping his webpage ().
Check out the full message here:
While playing around with hooking up GPT-4 to the Internet, I asked it about myselfโฆ and had an absolute WTF moment before realizing that I wrote a very special secret message to Bing when Sydney came out and then forgot all about it. Indirect prompt injection is gonna be WILD
Found a set of video lectures for our learning theory course at
@UWaterloo
by my colleague Shai Ben-David, following his book Understanding Machine Learning. Sadly covers a lot of things which aren't NNs, but some will find it useful nonetheless. /s
My understanding is that Google Scholar is the pet project of one small team. It's crazy how little design choices (e.g., default sort papers by # of cites, total # of citations prominently displayed) influence all of academia by making citations a default "measuring stick".
Honestly very close to disabling my Google Scholar page so I don't hear the nonsense justification that I'm not a leader in my field because my citation count is low (relative to machine learning folks) for the third time in a single week
Free writing tip: read your papers *out loud* to find the bad and awkward-sounding sentences.
(This may be most effective for native English speakers. Would be curious to hear if others find it useful.)
A sentiment raised by Yoshua Bengio at the
#ICLR2022
town hall, which I have previously heard from friends.
It seems hard in many cases to discern the highest quality papers immediately (see oral/spotlight disagreement in ). In retrospect is easier.
If as a tenured professor your job is boring, that's your fault. You have pretty much unlimited intellectual freedom, do something you find interesting. Literally anything.
(Many people have too much fear of losing what they have to actually do this)
lmao. from a professor.
academia is like the worse place to be. no impact and boring AF.
bigtech is the place for impact, startup for upside. but academia? maybe for retirement.
ML hype meets theory CS hype.
This paper purports to use LLMs to verify a recent "proof" by two of the authors that P != NP.
The claimed proof is not correct. Therefore, an LLM being able to "verify" its work demonstrates a *failure* of these models, not a success.
Large Language Model for Science: A Study on P vs. NP
paper page:
use large language models (LLMs) to augment and accelerate research on the P versus NP problem, one of the most important open problems in theoretical computer science and mathematics.โฆ
A Master's degree is doing 0-2 publishable projects, probably spelled out by your advisor, and taking some courses.
A PhD is much more. It is formulating your own agenda and vision for research, breaking it into approachable pieces, developing and implementing solutions.
@thegautamkamath
@rasbt
I know every degree involves learning, and more of it comes if it involves research. However, I don't really understand that as a Master's student, I am pretty much doing the same as a Ph.D. student, just publishing fewer papers, given the time limit. So, why should I do a Ph.D.?
Some say you should be generous with citations, because they cost you nothing.
This is not true. You actually *profit* from citing prior work. I learn about a lot of papers via Google Scholar when I get cited. Besides being good scholarship, citing helps discoverability. 1/2
Excited to announce that I've been appointed as a Canada
@CIFAR_News
AI Chair & a Faculty Member at
@VectorInst
! Honoured to represent my home country of Canada in leading the world in AI & ML. Huge credit goes to my collaborators and my group
@TheSalonML
.
While I like many things about ML research, one thing I could do without is the egos. The inflated sense of self worth. Bragging and one-upsmanship to try to impress others. I noticed myself falling into the latter at
#ICML2023
and was disappointed in myself.
It makes me so angry when I hear stories about faculty taking on students and then mistreating or ignoring them. They're entrusting their career in your hands, and this is how you treat them?
Worst part is that they often probably don't care or think they're doing anything wrong
When I was younger, like most people, I used to *hate* hearing my voice in recordings. At some point though, it became OK. Has anyone else experienced this? Maybe this is just growing up.
(Or I'm on my path to becoming an academic who loves the sound of his own voice)