Kyunghyun Cho Profile
Kyunghyun Cho

@kchonyc

61,607
Followers
2,247
Following
1,369
Media
12,488
Statuses

a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity ( @CILVRatNYU ) & @genentech ( @PrescientDesign ).

Manhattan, NY
Joined June 2009
Don't wanna be here? Send us removal request.
@kchonyc
Kyunghyun Cho
3 years
i was honored to receive the inaugural samsung AI researcher award. i’ve donated prize money to @Mila_Quebec to support newly arriving female students from latin america, africa, south asia, south east asia and korea. easiest decision ever: read more at
@Mila_Quebec
MilaQuebec
3 years
Mila alumnus Kyunghyun Cho @kchonyc makes a financial contribution to support female PhD and postdoctoral fellows in order to promote equity and diversity at Mila and within the AI community.
1
11
223
36
83
1K
@kchonyc
Kyunghyun Cho
3 years
i've received the Ho-Am Prize (Engineering) which comes with a pretty substantial cash prize. i plan to spend it for a few causes close to my heart in the next few weeks. here goes the first one:
24
41
669
@kchonyc
Kyunghyun Cho
6 years
“We’re machines,” says Hinton. “We’re just produced biologically. Most people doing AI don’t have doubt that...
8
253
623
@kchonyc
Kyunghyun Cho
6 months
ok here you go. my prediction: 1. ilya sutskever becomes CEO. 2. geoff hinton joins the OpenAI board.
32
24
613
@kchonyc
Kyunghyun Cho
26 days
once @ylecun told me (heavily paraphrased), it's not F=ma but \min (F-ma)^2. i didn't realize its importance, but it is perhaps the most enlightning perspective i've ever heard.
44
42
604
@kchonyc
Kyunghyun Cho
1 year
so.. i guess they haven’t heard the news yet… should i break it to them?
Tweet media one
Tweet media two
13
10
579
@kchonyc
Kyunghyun Cho
2 years
some mathematicians will shortly write papers explaining her behaviour in a principled and rigorous way.
@bremen79
Francesco Orabona
2 years
When you debug a machine learning model
35
207
1K
16
45
541
@kchonyc
Kyunghyun Cho
3 years
students are curious, and their questions are great: a few select questions from this semester, and my (incomplete) answers at
11
107
526
@kchonyc
Kyunghyun Cho
5 years
re "Why not consider other models? such as XLNet": I agree with the reviewer on the importance of time travel research, but it's slightly out of the scope of this paper.
10
69
522
@kchonyc
Kyunghyun Cho
5 years
ahahahahahahahahahaha
Tweet media one
8
84
496
@kchonyc
Kyunghyun Cho
3 years
OMG! my alma mater chose me as the Alumnus of the Year! Kiitos paljon!
@CSAalto
Computer Science
3 years
Aalto School of Science has just selected our alumnus Kyunghyun Cho as the Alumnus of the Year! 🥳He defended his doctoral thesis at Aalto in 2014 and is now associate professor of computer science and data science at New York University. Big congratulations Kuynghyun Cho!
1
7
101
25
14
471
@kchonyc
Kyunghyun Cho
6 months
if you trust @GoogleDeepMind Gemini about itself, it has 1.56 trillion parameters and cost Google $1-2billion (as opposed to GPT-4 which cost OpenAI $500M.) there were more than 100 engineers in the team who worked on Gemini. jailbreak by 한국어 🤣🤣🤣
21
71
465
@kchonyc
Kyunghyun Cho
2 years
i don’t know why anyone, including myself; spends any time working on ML when everything was invented early 90s.
@SchmidhuberAI
Jürgen Schmidhuber
2 years
30 years ago: Transformers with linearized self-attention in NECO 1992, equivalent to fast weight programmers (apart from normalization), separating storage and control. Key/value was called FROM/TO. The attention terminology was introduced at ICANN 1993
Tweet media one
26
140
1K
21
28
438
@kchonyc
Kyunghyun Cho
4 years
We must now realize the promise of AI by trusting Turing, unifying our framework and building our algorithm. I am running for president of the United States!
9
16
441
@kchonyc
Kyunghyun Cho
4 years
Do you and your org have ML/NLP/DS questions but not have anyone to talk with about them? I’d like to listen to your problems, talk about them and brainstorm ways forward with you. See
17
80
439
@kchonyc
Kyunghyun Cho
2 months
machine learning without model selection is biology without evolution. any claim of state of the art is vacuous without model selection, and indeed some were in continual learning.
@_sungmin_cha
Sungmin Cha
2 months
Exciting news! Our recent research paper, "Hyperparameters in Continual Learning: a Reality Check," is now available on arXiv! Check it out at .
1
6
54
3
11
88
@kchonyc
Kyunghyun Cho
2 years
let me help them out: “:q”
Tweet media one
8
18
436
@kchonyc
Kyunghyun Cho
5 years
ICML 2021 in SEOUL, KOREA!
9
52
390
@kchonyc
Kyunghyun Cho
4 years
even more counterintuitive is that we are still using "inches"
@fermatslibrary
Fermat's Library
5 years
Here's a useful counterintuitive fact: one 18 inch pizza has more 'pizza' than two 12 inch pizzas
Tweet media one
1K
27K
65K
4
27
386
@kchonyc
Kyunghyun Cho
5 years
77
15
378
@kchonyc
Kyunghyun Cho
4 years
doesn't everyone check out my homepage regularly? 🤪 apparently not 😅 to save some people's time (and my time), i've quit FAIR a couple of months ago: i.e. sorry, i can't take you in as an intern at FAIR. it's me, not you.
10
7
366
@kchonyc
Kyunghyun Cho
1 year
so, elon thinks he can catch up with OpenAI in 6 months.
20
11
360
@kchonyc
Kyunghyun Cho
4 years
NYC cheering was faster than any mobile notification!
4
9
361
@kchonyc
Kyunghyun Cho
8 months
so, how many of your papers are about to become irrelevant?
Tweet media one
16
29
348
@kchonyc
Kyunghyun Cho
5 years
thinking about spending 1/2 lecture next spring for ug intro to ml on kalman filter by teaching it as an extension of PCA and showing them how to estimate posterior and/or parameters by backprop (e.g., ) any thoughts?
7
61
349
@kchonyc
Kyunghyun Cho
4 years
i’ve experienced how elections happen in a few countries, _but_ when it comes to the complexity and ridiculousness of the rules and system, US beats the hell out of all the others.
6
8
347
@kchonyc
Kyunghyun Cho
4 years
i'm quite embarrassed and wanted to sweep it under a rug, but let me share what happened behind this, largely for my own record/reminder and for a small hope this might raise awareness.
@VectorInst
Vector Institute
4 years
@tarfandy @KKofahi @kchonyc @SPOClab We humbly accept your comment. In this case our marketing and keynote lineup did not reflect the diversity of the full program and we thank @kchonyc and @hhexiy for graciously stepping up to do the right thing. Our new, full lineup can be found here:
0
0
19
14
16
352
@kchonyc
Kyunghyun Cho
3 years
about 90% of what i often teach in <Intro to ML> is in this paper from 1962:
4
43
349
@kchonyc
Kyunghyun Cho
5 years
slides from my keynote talk at #emnlp2019
4
83
338
@kchonyc
Kyunghyun Cho
6 years
thanks, @NvidiaAI , for the shiny new Titan V!
Tweet media one
11
26
340
@kchonyc
Kyunghyun Cho
6 years
OMG *geoff hinton* nominated me for this
@CIFAR_News
CIFAR
6 years
CIFAR Azrieli Global Scholar @kchonyc @NYUDataScience nominated by Geoffrey Hinton as one of @Bloomberg 's people to watch in 2018:
0
10
50
17
23
336
@kchonyc
Kyunghyun Cho
2 years
until @gmail figures out how to block SEO spam mails that all look the same, i will completely ignore every single paper from @GoogleAI on few-shot learning for NLP.
10
21
327
@kchonyc
Kyunghyun Cho
1 year
best neurips ever
11
10
318
@kchonyc
Kyunghyun Cho
5 years
"Due to our concerns about malicious applications of [Our model ... trained simply to predict the next word], we are not releasing the trained model" for the humanity, i feel now obliged to remove all the pretrained model weights i've made public so far. 😅
@OpenAI
OpenAI
5 years
We've trained an unsupervised language model that can generate coherent paragraphs and perform rudimentary reading comprehension, machine translation, question answering, and summarization — all without task-specific training:
172
3K
6K
9
63
308
@kchonyc
Kyunghyun Cho
1 year
the highlight of #iclr2023
Tweet media one
11
1
304
@kchonyc
Kyunghyun Cho
1 year
i just can't watch _that_ senate hearing beyond some select excerpts. i can't believe the discourse on AI is about "oh AI may kill all of us unless i get to dictate wtf should be done" by a few clueless dudes, without any discussion on some immediate benefits & harms. (1/2)
11
40
285
@kchonyc
Kyunghyun Cho
5 years
super nervous
@emnlp2020
emnlp2020
5 years
We'd like to announce the keynote speakers for EMNLP-IJCNLP 2019: Meeyoung Cha (KAIST), Kyunghyun Cho @kchonyc (NYU & FAIR), and Noam Slonim @noamslonim (IBM).
0
15
103
10
6
280
@kchonyc
Kyunghyun Cho
4 years
since everyone loves/hates ranking authors based on their submissions/acceptances to #ICLR2020 , here's another rank i just created based on the # of letters in their names. @svlevine does not show up in top-50 (one with all names, and the other with the first name segments only)
Tweet media one
Tweet media two
5
14
275
@kchonyc
Kyunghyun Cho
10 months
if i ever leave @nyuniversity , you know which university i will be at
Tweet media one
5
0
262
@kchonyc
Kyunghyun Cho
6 years
aftr numerous rejections and improvrments, the paper's finally accepted and published officially. great work by Zhengping Che, Sanjay Purushotham, @david_sontag and @yanliu_usc
13
52
261
@kchonyc
Kyunghyun Cho
5 months
btw, it took 12 years, but i did end up having a dinner with CL last year
Tweet media one
@kchonyc
Kyunghyun Cho
5 months
some absolutely not-plagiarised text snippets in my dissertations: * "it would not have been possible for me to survive long, freezing, snowy winter of Finland without songs from four girls of 2NE1 (especially, the leader CL)."
Tweet media one
1
1
59
9
7
257
@kchonyc
Kyunghyun Cho
3 years
"... NL[P] ... scientists themselves completely forget what natural language is like." - Ernest Davis (2021) it's only April, and we've got the best quote of 2021 candidate.
4
29
254
@kchonyc
Kyunghyun Cho
1 year
before and after #iclr2023
Tweet media one
Tweet media two
7
2
255
@kchonyc
Kyunghyun Cho
2 years
enjoying this awesome tutorial on conformal prediction by @ml_angelopoulos & @stats_stephen : . what a nice, straightforward framework to think of UQ in practical terms. there’s an associated tutorial paper as well: .
9
43
249
@kchonyc
Kyunghyun Cho
3 years
David Blei has an interesting signature
Tweet media one
3
8
249
@kchonyc
Kyunghyun Cho
2 years
another motivational AI tweet drop 💣 prompt engineering is a symptom not a cure and must be treated not encouraged.
14
23
243
@kchonyc
Kyunghyun Cho
5 years
congratulations to @ylecun for the turing award. we, CILVRies, couldn't resist but present him with a physical obligatory Le Cake. (photo credit Y-Lan Boureau)
Tweet media one
7
21
242
@kchonyc
Kyunghyun Cho
3 years
which graduate program would admit me? no UG research experience+very low GPA+no GRE ever+expired but reasonable toefl hmm..
@deliprao
Delip Rao e/σ
3 years
Every few years, I feel like I need re-education. What graduate program you would go to today if you were math/quantitatively inclined?
29
5
106
11
5
242
@kchonyc
Kyunghyun Cho
4 months
not a single commercial LLM can sort the list of ~100 references according to the last name of the first author. so disappointing.
34
10
238
@kchonyc
Kyunghyun Cho
3 years
what a simple yet effective idea! :) looking at it from the architectural depth perspective ( by zheng et al.,) the depth (# of layers between a particular input at time t' and output at time t) is now (t-t') x L rather than (t-t') + L.
Tweet media one
@tesatory
Sainbayar Sukhbaatar
3 years
We updated our Feedback Transformer paper with new experiments. Transformers fail on very simple algorithmic tasks as it is a feedforward model. A simple fix is to attend to higher-level representations (it's like remembering our past thoughts)
11
89
545
1
48
241
@kchonyc
Kyunghyun Cho
7 months
very much enjoyed this paper (), but reading the paper makes this field look more psychology than machine learning; not sure if i'll enjoy the field as much as i did this paper when this trend goes further.
8
33
241
@kchonyc
Kyunghyun Cho
5 years
NYC the city of machine translation
Tweet media one
5
15
233
@kchonyc
Kyunghyun Cho
4 years
ahahahaha @geoffreyhinton hasn’t managed to “publish” the idea in his talk from 1973 yet but plans to do so “soon”
Tweet media one
13
12
226
@kchonyc
Kyunghyun Cho
2 years
now @YejinChoinka can't deny she's a genius
@macfound
MacArthur Foundation
2 years
Computer Scientist @YejinChoinka uses natural language processing to develop AI systems that can understand language and make inferences about the world. Learn more about the 2022 MacArthur Fellow #MacFellow
17
81
579
5
18
220
@kchonyc
Kyunghyun Cho
8 months
prediction: @OpenAI will announce within six months that they are from there on exclusively using sourced (and often paid for) data and stop using freely crawled data for their language models.
28
12
223
@kchonyc
Kyunghyun Cho
3 years
i endorse Fig. 1
@jaseweston
Jason Weston
3 years
(1/2) 🚨 Our new work! 🚨 "Retrieval Augmentation Reduces Hallucination in Conversation" @shtruk @spencerpoff @moyapchen @douwekiela @jaseweston We infuse dialogue models with knowledge, significantly reducing hallucinated facts during conversation.
Tweet media one
Tweet media two
3
50
269
3
14
223
@kchonyc
Kyunghyun Cho
2 years
well :) 5 years too late but still happy to receive the best research paper award cc ⁦ @orf_bnw ⁩ ⁦ @caglarml ⁩ ⁦ @imkelvinxu
Tweet media one
18
5
219
@kchonyc
Kyunghyun Cho
1 year
not everything we learn needs to be for the next paper 😩
5
7
220
@kchonyc
Kyunghyun Cho
4 years
it has arrived! @jacobeisenstein
Tweet media one
5
17
217
@kchonyc
Kyunghyun Cho
3 years
it's a bit embarrassing, but we've come to learn the RTRL-based online hparam opt in our was proposed earlier by Luca Franceschi et al. (S3.2 in ).
Tweet media one
5
8
218
@kchonyc
Kyunghyun Cho
1 year
Bing suspended itself from using the Bing Image Creator while trying to help me out ... i'm sorry
Tweet media one
8
25
214
@kchonyc
Kyunghyun Cho
2 years
wait... hold on... this is the panel i signed up for...? why are my hands and eyes shaking?
@agarwl_
Rishabh Agarwal
2 years
Acknowledging that there is no consensus on best evaluation practices for ML, the workshop would also have 3 panel discussions. The 1st panel discussion would be about "Incentives for Better Evaluation", featuring researchers who have seen the field of ML explode. [3/N]
Tweet media one
2
4
26
4
7
216
@kchonyc
Kyunghyun Cho
6 years
the paper behind BERT is now online: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova
0
67
213
@kchonyc
Kyunghyun Cho
3 years
all, may i suggest we stop blindly averaging scores from different tasks? if i average the (normalized) scores from a game of baseball, a game of soccer and a game of basketball played by each city, rank the cities and claim one city is better than others, what would you call me?
20
9
213
@kchonyc
Kyunghyun Cho
7 years
DeepMind aims at solving AGI, but AV doesn't work.. :(
Tweet media one
14
33
207
@kchonyc
Kyunghyun Cho
4 years
I got my degrees from the top university in the world (sorted alphabetically)
@togelius
Julian Togelius
4 years
This is a good post. It's true that top universities get an avalanche of applications from people wanting to do AI/ML PhDs. And published research is a very good indicator of research potential. But there are other universities than the top ones and other conferences than NeurIPS
6
51
257
5
5
204
@kchonyc
Kyunghyun Cho
2 years
one of them leads 25M followers and the other has 30k followers on twitter
Tweet media one
Tweet media two
8
0
203
@kchonyc
Kyunghyun Cho
23 days
a new blog post, because it is Saturday. <Fixing DPO but I have a dinner reservation …>
5
32
204
@kchonyc
Kyunghyun Cho
7 months
a number of weird definitions and weirdly specific points, but overall, worth reading it to see which areas are considered as priorities by WH. in this 🧵, let me copy-paste those few weird/interesting/specific points i found reading it.
5
43
199
@kchonyc
Kyunghyun Cho
4 years
I WON. I WON THE ELECTION.
13
4
200
@kchonyc
Kyunghyun Cho
1 year
. @MinaLee__ starts her talk at @nyuniversity an hour after GPT-4 was released. both exciting and nervous 😂😂😂
Tweet media one
2
7
198
@kchonyc
Kyunghyun Cho
4 years
#acl2020nlp bonnie webber walked up to me and told me after one of my talks quietly* "i don't think you have any idea what discourse is" which i could only nod in agreement. congratulations! (*) probably to save me from embarrassment
2
8
195
@kchonyc
Kyunghyun Cho
1 year
i may be as well
Tweet media one
17
6
194
@kchonyc
Kyunghyun Cho
3 years
A Lecture on NLP from Big Ideas in Artificial Intelligence (): This is the NLP section of the course organized by NYU in Spring 2021. These are preliminary recordings which were edited and polished for the final versions.
3
34
192
@kchonyc
Kyunghyun Cho
3 years
🤯 from the NeurIPS tutorial by @ChengSoonOng and @mpd37
Tweet media one
9
14
189
@kchonyc
Kyunghyun Cho
2 years
ICML 2023 will happen in Hawaii not in Seoul. yes, i understand your frustration (think of my own frustration as well ...) i've seen how this decision was made and can tell you it wasn't a light decision. thanks for your understanding! see for more info.
Tweet media one
4
27
191
@kchonyc
Kyunghyun Cho
3 years
""" Notable individual investors in the round include ... NBA star Kevin Durant ... """ !?
@VentureBeat
VentureBeat
3 years
Hugging Face triples investment in open source machine learning models by @kharijohnson
1
28
83
9
11
187
@kchonyc
Kyunghyun Cho
7 years
My lecture note for the undergrad course <Intro to Machine Learning> from this Spring (2017) is available online...
0
50
186
@kchonyc
Kyunghyun Cho
10 months
#ICML2023 begins!
Tweet media one
4
1
185
@kchonyc
Kyunghyun Cho
3 years
the breiman lecture on causal learning by Marloes Maathuis is clear and well-delivered to the degree that i am having a false sense that i actually understand causal inference.
5
14
183
@kchonyc
Kyunghyun Cho
3 years
is it because it’s easier developing an AI driver than an AI car?
@engadget
Engadget
3 years
Tesla is working on an AI-powered humanoid robot
Tweet media one
61
150
820
7
17
179
@kchonyc
Kyunghyun Cho
3 years
my friends, how do you feel to have become meta employees? ;)
10
3
180
@kchonyc
Kyunghyun Cho
1 year
disappointed we plan to release reviews hopefully tomorrow but hey what can we do at this rate? #ICML2023 ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░ 92.41%
26
7
183
@kchonyc
Kyunghyun Cho
2 years
quite a few people told me earlier LM & MT were just applications and they wanted to do "core" ML research. i wonder what they are thinking and doing now 🤣 are you all scaling up?
9
11
180
@kchonyc
Kyunghyun Cho
6 months
the only concrete take-away i got over the past few days is that i can't rely on a single service provider of LM to build any LM-powered applications. too fragile and too risky.
11
25
178
@kchonyc
Kyunghyun Cho
5 months
i mean it isn’t that surprising, is it? here’s a prescient whatsapp convo from 2021.
Tweet media one
@ChombaBupe
Chomba Bupe
5 months
An explanation for why GPT-4 is degrading: "... we find that on datasets released before the LLM training data creation date, LLMs perform surprisingly better than on datasets released after" New tasks are difting away from what GPT-4 was trained on.
Tweet media one
49
422
2K
6
13
175
@kchonyc
Kyunghyun Cho
4 years
before too late, let me thank the volunteers without whom we couldn't have run ICLR'20 virtually: #ICLR2020 @iclr_conf
3
14
171
@kchonyc
Kyunghyun Cho
3 years
it's a double-embarrassment day. NaturalProof in , which i loved working on, turned out also to be a repetition of the work by Deborah Ferreira .
Tweet media one
6
10
173
@kchonyc
Kyunghyun Cho
5 years
HYPE: a @PyTorch based library for embedding a graph in a hyperbolic space @mnick
Tweet media one
Tweet media two
3
37
167
@kchonyc
Kyunghyun Cho
5 years
my strategy is to wait for Sam to share some of those on our group slack
6
5
169
@kchonyc
Kyunghyun Cho
4 months
i wrote this proposal back in 2017. NSF rejected it & the panel found it "Low Competitive". grant rejection is an everyday affair, but i felt particularly bitter about this one. its title was <End-to-End Search Engine-Guided Query-Driven Summarization>.
Tweet media one
Tweet media two
7
12
165
@kchonyc
Kyunghyun Cho
4 years
i wanted to briefly share why i joined @NYUDataScience 5yrs ago and how i've liked it so far:
2
22
166
@kchonyc
Kyunghyun Cho
3 years
Bong Joon-Ho (yes, that Bong Joon-Ho) is the recipient in the area of art. i haven't complained much about the on-going pandemic myself, but man.. i can't believe i'm missing this 1-in-a-lifetime oppt to meet Bong Joon-Ho because of the pandemic. i'm so furious at covid-19.
@NYUDataScience
NYU Data Science
3 years
Let's all give a HUGE congrats to CDS faculty @kchonyc for being awarded a 2021 Samsung Ho-Am Prize in the field of Engineering! His award recognizes his work in developing a neural machine translation algorithm that can provide high-quality translations.
6
14
200
8
1
166