The Ubiquitous Knowledge Processing Lab researches Natural Language Processing, Text Mining, eLearning, and Digital Humanities ·
@CS_TUDarmstadt
·
@TUDarmstadt
Are Emergent Abilities in Large Language Models just In-Context Learning?
Spoiler: YES 🤯
Through a series of over 1,000 experiments, we provide compelling evidence:
Our results allay safety concerns regarding latent hazardous abilities.
A🧵👇
#NLProc
Prof. Iryna Gurevych has been just awarded the first "LOEWE Spitzenprofessur" (Distinguished Hessian Chair) by the Hessian government. With a funding of 2.5 Million Euros the program aims to provide excellent research and infrastructural conditions. Congratulations, Iryna!
Excited to announce the 1st ELLIS NLP Workshop "Open Challenges and Future Directions of NLP" Feb 24-25 2021
Keynotes by Yoshua Bengio
@YejinChoinka
@HinrichSchuetze
@sameer_
Yulia Tsvetkov will be broadcast on Youtube
War has never been a solution & never will be
Unbelievable such a war in
#Ukraine
is still possible in 21 century, in the middle of
#Europe
We are one family.This is an attack not by people but by politicians against
#values
#democracy
& life
Unite for a better world
#NotoWar
We are thrilled to announce that Prof. Iryna Gurevych, Head of
@UKPLab
, was elected as VP-Elect and the future president of ACL in 2023! On behalf of all UKP, we would like to congratulate to this great honor!
#NLProc
#aclnlp
PhD students wanted! 🎓 The European Laboratory for Learning & Intelligent Systems
@ELLISforEurope
is a pan-European AI network of excellence. The 2023 Call for Applications of the ELLIS PhD Program is now online & can be found here ➡️
#JoinELLISforEurope
🤔 Can Diverse Counterfactuals help in improving model performance on unseen question answering datasets?
🚀 Discover how smaller language models are supercharged with counterfactual instances generated by LLMs in our study! 📚
#NLProc
#AI
#Research
📝
An amazing new class of ACL fellows — congratulations to all, and thank you for all your groundbreaking work in
#NLProc
! 👏👏
Philip Cohen, Pascale Fung, Iryna Gurevych, Dan Jurafsky, Hermann Ney, Philip Resnik, Donia Scott, Noah Smith, Kam-Fai Wong
@_jasonwei
Does Instruction Tuning (including on Code) lead to »reasoning«?
Turns out: No 🚫
Without in-context learning the tasks solvable by T5 770M and GPT-3 175B are comparable! 🤯😯 (4/)
#NLProc
A warm welcome to Anmol Goel (
@anmgoel
), who has just started at UKP Lab in the
@ELLISforEurope
PhD program! 👋 Anmol's main research topics are
#NLProc
,
#Privacy
and
#Causality
. You can find out more about him on his personal website:
@_jasonwei
If you're interested in our research: We provide open access to our code and results, including model-generated responses so researchers can build on these results:
Code ➡️
Data ➡️
(5/)
UKP is pleased to announce the second central call for applications for the ELLIS PhD Program. Want to earn your PhD while working with the best talent in machine learning in Europe? Take a look:
@_jasonwei
We perform the first set of experiments to test models without in-context learning and show that in this setting only 2 of the 22 tasks originally reported to be emergent are »truly emergent«.
Of these 2, one tests linguistic skills and the other tests memory.
No »Reasoning« 🤔
It was previously reported by
@_jasonwei
et al. that Large Language Models have Emergent Abilities.
The »unpredictable emergence« of such a large number of abilities, it has been claimed, is indicative of the possibility that there are latent hazardous abilities. (2/)
A team from
@UKPLab
will work the next two years in collaboration with
@Amazon
Alexa on a »virtual research assistant«. The assistant is intended to simplify the work of scientists in
#MachineLearning
and
#NLProc
by aggregating current research results.
@IGurevych
wins the highly-coveted ERC Advanced Grants & receives € 2.5 million for
“InterText – Modeling Text as a Living Object in a Cross-Document Context”
#ERCAdG
#NLP
#AI
.
@TUDarmstadt
(1/2)
UKP is hiring again! We are accepting applications for Associate Research Scientists (PostDoc- or PhD-level) in Machine Learning and Natural Language Processing. More details here:
#NLProc
#CompLing
Excited to share our
@emnlp2020
paper about a simple method to extend existing sentence embedding methods to new languages, by Nils Reimers and Iryna Gurevych
A warm welcome to Hovhannes Tamoyan ., who has just started his Ph.D. at
@ukplab
/
@TUDarmstadt
!
@tmynnnn
's main research interest is theory of NLP. You can find him on Medium: and on his personal webpage:
Many thanks to our great invited speakers at our UKP Day so far, with
@GabiStanovsky
about low resource multilingual NLP and
@dirk_hovy
about biases in NLP! Looking forward to our last invited talk by
@yufanghou
later this afternoon, about automatic leaderboard construction.
Congratulations to Thomas Otmar Arnold, who was awarded a teaching prize for the best exercise in the summer semester 2022 at our department
@CS_TUDarmstadt
's and
@d120de
's colloquium last week! He received a certificate (and a toucan) for his course »Information Management« 📜🦜
Excited to announce the 1st ELLIS NLP Workshop "Open Challenges and Future Directions of NLP" Feb 24-25 2021
Keynotes by Yoshua Bengio
@YejinChoinka
@HinrichSchuetze
@sameer_
Yulia Tsvetkov will be broadcast on Youtube
A warm welcome to
@HendrikSchuff
, who has just started his postdoc at UKP Lab! 👋 Hendrik's research focuses on the explainability and human-centred evaluation of
#NLProc
systems. You can find out more about him on his personal website:
Every year, the Carlo and Karin Giersch Foundation at
@TUDarmstadt
awards the Athena Prize for Good Teaching to scientists with outstanding contributions in education. We congratulate our colleague Dr. Thomas Arnold, who was one of this year's laureates!
We are pleased to welcome Aishik Mandal (
@jit_aishik_13
), who has just started his PhD at the Ubiquitous Knowledge Processing Lab! 👋 Aishik’s research concerns
#Privacy
-aware
#Multimodality
and
#DialogueSystems
. You can find out more about him here:
We are pleased to welcome Seolhwa Lee PhD at
@ukplab
/
@TUDarmstadt
!
@Seolhwa_Lee_
is a postdoctoral researcher, her research interests are privacy-aware NLP, dialogue systems, and meeting summarization. You can also find her on her personal website:
#NLProc
Six papers authored or co-authored by members of
@UKPLab
have been accepted for publication at ACL 2023.
Congratulations to all involved!
#NLProc
#ACL2023NLP
4 papers (co-)authored by UKP colleagues have been accepted in the Findings section at
@emnlpmeeting
! Congratulations to all involved! 💐 (1/🧵)
#EMNLP2023
We are proud to announce that UKP Lab's ArgumenText is one of the Hessen Ideen Challenge participants nominated by the
#tudarmstadt
. We would really appreciate your support for our team by casting your vote at .
#HIW2020
It's an honor to announce that
@IGurevych
is elected as a member of Berlin-Brandenburg Academy of Sciences & Humanities
She is appointed due to her excellent scientific achievements & belongs to the Academy technical science class
@TUDarmstadt
#Leibniztag
Herzlich willkommen! Zum heutigen
#Leibniztag
hat die BBAW 13 Wissenschaftler:innen als Ordentliche Mitglieder neu in ihre Reihen aufgenommen. Nähere Infos zu den neuen Akademiemitgliedern:
Since the launch of
#GPT4
in March, researchers worldwide have been exploring its capabilities. So have our colleagues Tilman Beck (
@devnull90
) and Martin Tutek (
@mtutek
) at UKP Lab, generously supported by a
@Microsoft
@Azure4Research
Grant which has just been extended! 🎉🎉🎉
Congratulations and a fond farewell to UKP's Tristan Miller, who now
leads his own research project on computational humour at the Austrian
Research Institute for
#ArtificialIntelligence
(OFAI) in Vienna:
#NLProc
#CompLing
How do we teach the most important
#NLProc
skills to a new generation? Last month, our colleague Dr. Thomas Arnold took part in the
@gscl_org
's »Teaching for NLP« workshop, part of
#KONVENS2023
in Ingolstadt – and he brought back this wonderful poster.
On September 17, 2023, the Teaching for NLP Workshop took place as part of the satellite program of KONVENS 2023 in Ingolstadt. Thanks for the great exchange on teaching heterogenous groups and on what should be part of a modern NLP curriculum.
"UKP-SQuARE v3: A Platform for Multi-Agent QA Research" was accepted at
#acl_2023
🇨🇦! Read more about
@UKP_SQuARE
's amazing pre-print in the following 🧵 or here:
#NLP
#NLProc
We are excited to announce that UKP-SQuARE v3: A Platform for Multi-Agent QA Research was accepted at
#acl_2023
🇨🇦 as a demo paper 🤩 You can read our pre-print 📜 here
ArgSciChat: A new argumentative dialogue dataset. It consists of 498 messages collected from 41 dialogues on 20 NLP papers. It can be used to evaluate conversational agents and further encourage research on argumentative scientific agents.
Pointwise V-usable information (PVI) excels in many
#NLProc
tasks. But fine-tuning
#LLMs
with it is very time-consuming 🐌 Is in-context PVI the necessary next step?
Yes! 🎉Check out our empirical analysis accepted at
#EMNLP2023
– and this 🧵 (1/7)
📄
We are glad to welcome Jonathan Tonglet (
@TongletJ
), who just began as an
@ELLISforEurope
PhD student at the Ubiquitous Knowledge Processing Lab 👋 His main research topic is Multimodal Fact-checking. Here is where you may learn more about him:
#NLProc
The UKP Lab is hiring Postdoc Research Fellows in the field of
#AI
/
#NLProc
. Areas of work include
#ConvAI
, Multimodal fact-checking, Security of LLMs,
#QA
& Natural Language Processing for mental health.
⬇️ Apply until Jan 15th, 2024! 🚀
We are hiring again! UKP is accepting applications for an Associate Research Scientist ("NLP for the Social Good") and Associate Research Scientist ("Secure and Robust NLP") position. Details at and
#NLProc
#CompLing
Ever faced difficulties to write good rebuttals for
#PeerReview
? Jiu-Jitsu argumentation comes to the rescue 🥋 !
In our
#EMNLP2023
paper we draw inspiration from Jiu-Jitsu argumentation to generate rebuttals that align with a reviewer’s attitude roots and themes 📚 (1/🧵)
Less than a month to go until
#ACL2023NLP
kicks off in
#Toronto
🇨🇦! Time to present to you the papers authored or co-authored by UKP members that have been accepted for this year's
@aclmeeting
– in front of a magnificent Torontonian skyline.
#NLProc
The Ubiquitous Knowledge Processing Lab is happy to welcome Toru Sasaki (
@tsskiam
), who has joined us via the
@ELLISforEurope
PhD program 👋 In his research Toru focuses on Generative
#NLProc
and NLP for education. You can also find him on Linkedin:
Congratulations to our colleague Thomas Arnold (second from left), who won the first place at the
@TUDarmstadt
#ScienceSlam
in September! 💐 At the opening of the
#Wissenschaftsschloss
Darmstadt, he explained to 400 interested guests how Large Language Models work.
Strahlender Sonnenschein 🌞, rund 10.000 Gäste 👨👩👦👦 und ein reichhaltiges Programm 🎉: Mit einem großen Fest hat die
@TUDarmstadt
am Samstag das „
#Wissenschaftsschloss
“ 🏰 nach 15-jähriger Sanierung wieder eröffnet und der Stadtgesellschaft zurückgegeben.
We are glad to welcome Justus-Jonas Erker (
@j_st_s
) at
@UKPLab
, who worked before at the
@DFKI
&
@MaastrichtU
👋 In his PhD, Justus researches representation learning for long-dependency context modeling in dialog systems. You can find him here:
#NLProc
Six papers authored or co-authored by members of
@UKPLab
have been accepted for publication at
@eaclmeeting
2023, May 2-6, in Dubrovnik, HRV.
Congratulations to all involved!
#EACL2023
How can
#NLProc
be used in psychotherapy to estimate a client’s depression severity and well-being? On Friday 9 June, 10AM BST, Iryna Gurevych
@IGurevych
will give the keynote at the NLP for Social Good (NSG) Symposium 2023 at
@LivUni
.
Find out more here:
A warm welcome to Falko Helm, who has just started as a PhD candidate at the UKP Lab! 👋 Falko researches
#Multimodality
& Structure for
#TransformerModels
. He is also interested in
#GraphTheory
to handle linked documents. Find more about Falko here:
Are Emergent Abilities in Large Language Models just In-Context Learning? This is the topic of
@IGurevych
's talk at the Symposium on Large Language & Foundation Models of
@ELLISforEurope
,
@ai4mediaproject
&
@AIDAdocAcademy
on October 17/18, Amsterdam:
➡️
We are pleased to welcome Federico Tiblias, who has just started his PhD at the Ubiquitous Knowledge Processing Lab! Federicos interests lie at the interface between
#NLProc
and
#QuantumComputing
. You can find out more about him here:
Can we combine integer linear programming with exemplar selection to improve In-Context Learning?
Yes! All you need is to optimize your Knapsack 🎒
The paper by
@TongletJ
et al. on
#SEER
was just accepted to
#EMNLP2023
– learn more in this 🧵 (1/8)
📰
On Oct 17,
@Harish
Tayyar Madabushi presented the talk »Are Emergent Abilities in Large Language Models just In-Context Learning?« at the Symposium on LLMs & Foundation Models of
@ELLISforEurope
,
@ai4mediaproject
&
@AIDAdocAcademy
. Watch the video here ➡️
We're hiring! Join UKP as an Associate Research Scientist in
#NLProc
and
#MachineLearning
(topics like human-in-the-loop ML, conversational AI, claim validation, multimodal commonsense reasoning). Details at
We are happy to announce the release of 🧮 MathDial, a dataset of one-to-one teacher-student tutoring dialogues grounded in multi-step math reasoning problems.
Discover more about our latest
#EMNLP2023
Findings paper in this 🧵 (1/8)
#dialogue
#NLProcessing
#MathWordProblems
Update: A total of eight submissions with contributions by
@UKPLab
colleagues have been accepted for the
#ACL2023NLP
! 🎉 Plus two submissions to this year's
@_starsem
, which takes place co-located with the
@ACLmeeting
.
This afternoon at 5:30 pm,
@IGurevych
from
@UKPLab
will give a talk titled "InterText: Modeling Text as a Living Object in Cross-Document Context" at
@BilkentUniv
's Distinguished Seminar Series.
The Zoom Link can be found here:
#KI
im Klassenzimmer: Auf der gleichnamigen Veranstaltung der
@CAISnrw
in Bochum wurde eifrig diskutiert, wie und wo KI sinnvoll im Unterricht eingesetzt werden kann - und wo auch Risiken bestehen. Es gibt viele interessante Ansätze und Ideen - doch Zustand und Ausstattung
Human feedback on poor dialog system performance is important for the development of
#ConvAI
. 👍👎
But datasets with free-text human feedback are scarce. Can data augmentation come to the rescue?
A 🧵 on one of our
#EMNLP2023
papers (1/11).
📰
#NLProc
New preprint: How can we make life easier for annotators? By providing them with instances in an ordering that helps them to better understand the task!
Joint effort with
@anhklie
and
@IGurevych
!
Code (+ data):
We are proud that one of our recent papers was quoted in the discussion paper »Capabilities and risks from frontier AI« for the 2023
#AISafetySummit
, held on 1-2 Nov. in Bletchley Park!
Find the discussion paper here:
& more info on our paper in the 🧵⬇️
Are Emergent Abilities in Large Language Models just In-Context Learning?
Spoiler: YES 🤯
Through a series of over 1,000 experiments, we provide compelling evidence:
Our results allay safety concerns regarding latent hazardous abilities.
A🧵👇
#NLProc
Ever wondered how to model code generation when having abstract (under-specified) inputs? Check out this thread by
@UKPLab
/
@ELLISforEurope
PhD student
@LHaausing
on their paper “Python Code Generation by Asking Clarification Questions” which was accepted for
#ACL2023NLP
🇨🇦:
1/ Paper alert 🚨 Our paper “Python Code Generation by Asking Clarification Questions” has been accepted to
#ACL2023
🎉🥽👀🫵!
This work is a collaboration with
@MohsenMes
,
@andre_t_martins
and
@IGurevych
. Codegen country, let’s ride🚗~
Next Tuesday, 10 a.m. CEST, there will be a public event organised by
@Hessian_AI
in our building. Paolo Papotti
@paolopapotti
(
@EURECOM
) will talk about »Querying Large Language Models with SQL«. More info and a zoom link can be found here:
New job openings at our lab with
@IGurevych
:
We are looking for several Associate Research Scientists (PhD or PostDoc level) in NLP and ML, and an Independent Research Group Leader in NLP.
More about the openings:
and
We are excited to announce that “
@UKP_SQuARE
: An Interactive Tool for Teaching Question Answering” has been accepted at the Workshop on Innovative Use of NLP for Building Educational Applications
@AclSigedu
held at
#ACL2023
#ACL2023NLP
! 🎉 1/x
»Are the machines taking over now?« Several experts from
@CS_TUDarmstadt
and
@TUDarmstadt
were interviewed this year for a TV documentary with this title – one of them Iryna Gurevych
@IGurevych
. The feature can now be found in the
@ARDde
Mediathek.
➡️
Tomorrow at
@ACL2023NLP
: The demo session for »CARE: Collaborative AI-assisted Reading Environment«! 🎉 CARE is a sleek interface for
- collaborative PDF reading and commenting
- seamless data collection
- quick model deployment and testing.
Find out more about CARE in this 🧵.
"Model to debunk AI-written texts?": Our colleagues
@IGurevych
and
@JiUngLee1
were asked by
@smc_germany
for statements on a model that is supposed to identify whether scientific texts were created by AI. You can find their assessment (in German) here:
#SavetheDate
: Next week Iryna Gurevych (
@IGurevych
) of
@UKPLab
will give a short German-language input at the online workshop »GPT for archives and museums? Understanding and Using Large Language Models« of
@BLM_Karlsruhe
.
Registration is possible here:
On Thursday, Sriraam Natarajan
@Sriraam_UTD
(
@ut_dallas
) will give a talk at
@Hessian_AI
on »Neurosymbolic learning via Integration of (Relational) Planning and (Deep) RL«. The event takes place off- and online, more info and a zoom link can be found here:
We are hiring several short-term student assistants for a chatbot annotation study, conducted by
@PetrakDominic
and
@thy2512
in the context of
@SERMAS_eu
(Socially-acceptable Extended Reality Models and Systems).
#XR
Interested? More infos here ➡️
Good morning Toronto 🇨🇦! Both events today at the
@aclmeeting
with participation of
@UKPLab
colleagues will take place at the Frontenac Ballroom and Queen's Quay and start at 11:00 and 14:00 respectively (CET 17:00 and 20:00).
#ACL2023NLP
AI is getting better and better at processing texts. But what about "living texts" - texts that dynamically evolve through revisions over time?
The
@ERC_Research
has produced a short video about
@IGurevych
's work at
@TUDarmstadt
. Read the interview here:
#AI
dealing with language has so far mostly focused on static texts.
ERC grantee Iryna Gurevych aims to create a new generation of AI that could help people to follow information through different sources, or collaborate more efficiently on documents👉
Check our new preprint on collecting a large and high-quality dialogue tutoring dataset on teaching math word problems!
Ongoing work with
@dmacjam
Sankalan Pal Chowdhury
@TanmaySinha655
Manu Kapur
@IGurevych
and
@mrinmayasachan