i was honored to receive the inaugural samsung AI researcher award. i’ve donated prize money to
@Mila_Quebec
to support newly arriving female students from latin america, africa, south asia, south east asia and korea. easiest decision ever: read more at
Mila alumnus Kyunghyun Cho
@kchonyc
makes a financial contribution to support female PhD and postdoctoral fellows in order to promote equity and diversity at Mila and within the AI community.
i've received the Ho-Am Prize (Engineering) which comes with a pretty substantial cash prize. i plan to spend it for a few causes close to my heart in the next few weeks.
here goes the first one:
once
@ylecun
told me (heavily paraphrased), it's not F=ma but \min (F-ma)^2. i didn't realize its importance, but it is perhaps the most enlightning perspective i've ever heard.
re "Why not consider other models? such as XLNet": I agree with the reviewer on the importance of time travel research, but it's slightly out of the scope of this paper.
Aalto School of Science has just selected our alumnus Kyunghyun Cho as the Alumnus of the Year! 🥳He defended his doctoral thesis at Aalto in 2014 and is now associate professor of computer science and data science at New York University. Big congratulations Kuynghyun Cho!
if you trust
@GoogleDeepMind
Gemini about itself, it has 1.56 trillion parameters and cost Google $1-2billion (as opposed to GPT-4 which cost OpenAI $500M.) there were more than 100 engineers in the team who worked on Gemini.
jailbreak by 한국어
🤣🤣🤣
30 years ago: Transformers with linearized self-attention in NECO 1992, equivalent to fast weight programmers (apart from normalization), separating storage and control. Key/value was called FROM/TO. The attention terminology was introduced at ICANN 1993
We must now realize the promise of AI by trusting Turing, unifying our framework and building our algorithm. I am running for president of the United States!
Do you and your org have ML/NLP/DS questions but not have anyone to talk with about them? I’d like to listen to your problems, talk about them and brainstorm ways forward with you. See
machine learning without model selection is biology without evolution.
any claim of state of the art is vacuous without model selection, and indeed some were in continual learning.
doesn't everyone check out my homepage regularly? 🤪 apparently not 😅 to save some people's time (and my time), i've quit FAIR a couple of months ago: i.e. sorry, i can't take you in as an intern at FAIR. it's me, not you.
thinking about spending 1/2 lecture next spring for ug intro to ml on kalman filter by teaching it as an extension of PCA and showing them how to estimate posterior and/or parameters by backprop (e.g., ) any thoughts?
i’ve experienced how elections happen in a few countries, _but_ when it comes to the complexity and ridiculousness of the rules and system, US beats the hell out of all the others.
i'm quite embarrassed and wanted to sweep it under a rug, but let me share what happened behind this, largely for my own record/reminder and for a small hope this might raise awareness.
@tarfandy
@KKofahi
@kchonyc
@SPOClab
We humbly accept your comment. In this case our marketing and keynote lineup did not reflect the diversity of the full program and we thank
@kchonyc
and
@hhexiy
for graciously stepping up to do the right thing. Our new, full lineup can be found here:
until
@gmail
figures out how to block SEO spam mails that all look the same, i will completely ignore every single paper from
@GoogleAI
on few-shot learning for NLP.
"Due to our concerns about malicious applications of [Our model ... trained simply to predict the next word], we are not releasing the trained model" for the humanity, i feel now obliged to remove all the pretrained model weights i've made public so far. 😅
We've trained an unsupervised language model that can generate coherent paragraphs and perform rudimentary reading comprehension, machine translation, question answering, and summarization — all without task-specific training:
earlier we open-sourced the code and also trained models for breast cancer screening from (led by
@NanWu__
,
@kjgeras
et al.) based on our own data () at .
i just can't watch _that_ senate hearing beyond some select excerpts. i can't believe the discourse on AI is about "oh AI may kill all of us unless i get to dictate wtf should be done" by a few clueless dudes, without any discussion on some immediate benefits & harms.
(1/2)
We'd like to announce the keynote speakers for EMNLP-IJCNLP 2019: Meeyoung Cha (KAIST), Kyunghyun Cho
@kchonyc
(NYU & FAIR), and Noam Slonim
@noamslonim
(IBM).
since everyone loves/hates ranking authors based on their submissions/acceptances to
#ICLR2020
, here's another rank i just created based on the # of letters in their names.
@svlevine
does not show up in top-50 (one with all names, and the other with the first name segments only)
aftr numerous rejections and improvrments, the paper's finally accepted and published officially. great work by Zhengping Che, Sanjay Purushotham,
@david_sontag
and
@yanliu_usc
some absolutely not-plagiarised text snippets in my dissertations:
* "it would not have been possible for me to survive long, freezing, snowy winter of Finland without songs from four girls of 2NE1 (especially, the leader CL)."
"... NL[P] ... scientists themselves completely forget what natural language is like." - Ernest Davis (2021)
it's only April, and we've got the best quote of 2021 candidate.
enjoying this awesome tutorial on conformal prediction by
@ml_angelopoulos
&
@stats_stephen
: . what a nice, straightforward framework to think of UQ in practical terms.
there’s an associated tutorial paper as well: .
congratulations to
@ylecun
for the turing award. we, CILVRies, couldn't resist but present him with a physical obligatory Le Cake. (photo credit Y-Lan Boureau)
what a simple yet effective idea! :)
looking at it from the architectural depth perspective ( by zheng et al.,) the depth (# of layers between a particular input at time t' and output at time t) is now (t-t') x L rather than (t-t') + L.
We updated our Feedback Transformer paper with new experiments. Transformers fail on very simple algorithmic tasks as it is a feedforward model. A simple fix is to attend to higher-level representations (it's like remembering our past thoughts)
very much enjoyed this paper (), but reading the paper makes this field look more psychology than machine learning; not sure if i'll enjoy the field as much as i did this paper when this trend goes further.
Computer Scientist
@YejinChoinka
uses natural language processing to develop AI systems that can understand language and make inferences about the world.
Learn more about the 2022 MacArthur Fellow
#MacFellow
prediction:
@OpenAI
will announce within six months that they are from there on exclusively using sourced (and often paid for) data and stop using freely crawled data for their language models.
Acknowledging that there is no consensus on best
evaluation practices for ML, the workshop would also have 3 panel discussions. The 1st panel discussion would be about "Incentives for Better Evaluation", featuring researchers who have seen the field of ML explode. [3/N]
the paper behind BERT is now online:
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova
all, may i suggest we stop blindly averaging scores from different tasks? if i average the (normalized) scores from a game of baseball, a game of soccer and a game of basketball played by each city, rank the cities and claim one city is better than others, what would you call me?
This is a good post. It's true that top universities get an avalanche of applications from people wanting to do AI/ML PhDs. And published research is a very good indicator of research potential. But there are other universities than the top ones and other conferences than NeurIPS
a number of weird definitions and weirdly specific points, but overall, worth reading it to see which areas are considered as priorities by WH. in this 🧵, let me copy-paste those few weird/interesting/specific points i found reading it.
#acl2020nlp
bonnie webber walked up to me and told me after one of my talks quietly* "i don't think you have any idea what discourse is" which i could only nod in agreement.
congratulations!
(*) probably to save me from embarrassment
A Lecture on NLP from Big Ideas in Artificial Intelligence (): This is the NLP section of the course organized by NYU in Spring 2021. These are preliminary recordings which were edited and polished for the final versions.
ICML 2023 will happen in Hawaii not in Seoul. yes, i understand your frustration (think of my own frustration as well ...) i've seen how this decision was made and can tell you it wasn't a light decision. thanks for your understanding!
see for more info.
the breiman lecture on causal learning by Marloes Maathuis is clear and well-delivered to the degree that i am having a false sense that i actually understand causal inference.
quite a few people told me earlier LM & MT were just applications and they wanted to do "core" ML research. i wonder what they are thinking and doing now 🤣
are you all scaling up?
the only concrete take-away i got over the past few days is that i can't rely on a single service provider of LM to build any LM-powered applications. too fragile and too risky.
An explanation for why GPT-4 is degrading:
"... we find that on datasets released before the LLM
training data creation date, LLMs perform surprisingly better than on datasets released after"
New tasks are difting away from what GPT-4 was trained on.
i wrote this proposal back in 2017. NSF rejected it & the panel found it "Low Competitive". grant rejection is an everyday affair, but i felt particularly bitter about this one.
its title was <End-to-End Search Engine-Guided Query-Driven Summarization>.
Bong Joon-Ho (yes, that Bong Joon-Ho) is the recipient in the area of art.
i haven't complained much about the on-going pandemic myself, but man.. i can't believe i'm missing this 1-in-a-lifetime oppt to meet Bong Joon-Ho because of the pandemic. i'm so furious at covid-19.
Let's all give a HUGE congrats to CDS faculty
@kchonyc
for being awarded a 2021 Samsung Ho-Am Prize in the field of Engineering! His award recognizes his work in developing a neural machine translation algorithm that can provide high-quality translations.