Pasquale Minervini 🚀 looking for postdocs! @PMinervini Twitter profile

Pinned Tweet

Pasquale Minervini 🚀 looking for postdocs!

2 years

Interested in a PhD on NLP/ML-related topics with me at @EdinburghUni ? 🧑‍🎓🚀 Apply to either the UKRI CDT in NLP ( @Edin_CDT_NLP ) or to the ILCC PhD Program - the deadline is around the corner! More at ⏰ International - Nov 25th, 2022 ⏰ UK - Jan 27th, 2023

UKRI CDT in Natural Language Processing

@Edin_CDT_NLP

2 years

📢Applications welcome & encouraged from individuals relating to groups under-represented in technology for the FINAL cohort of UKRI CDT in NLP, @EdinburghUni ! 🥹 Programme/Application Details: Deadlines: ⏰Non UK – 25.11.22 ⏰ UK – 27.01.23 @InfAtEd

0

13

14

1

22

63

Last Seen Profiles

@floraegy

@DavidPepper

@kamalojasv181

@Succeder_O

@bourboli

@EvanWorkman14

@valkentine

@petepennanen

@3OqZzOAJi63LoWr

@pavan_pappula

@hijabjilbab1

@floraegy

@PrincesSyahrini

@YiqiLuo

@ogimrash

@Michellevi65112

@DrTurleyTalks

@Sanjaychennai13

@ibvxlogxhodob

@Firstkpp

@SSSF_edu

@ctchildrens

@FL1PZT3RZ

@ooc_big

@JennieLuci72243

@oofos

@EndymionYT

@drc000000

@jdurran

@ezecrypts

@AnishaMiah_

@seb_mrts

@Gofblug

@BlackHatEvents

@CCA_Activities

@ToldYouThusly

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

how it started / how it's going

12

245

1K

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

Based on our NeurIPS 2021 paper (), we developed a PyTorch library that easily allows you to include discrete combinatorial solvers into larger neural architectures, and back-prop through them, in one line of code! 🚀 Check it out!

18

182

1K

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

Hey everyone! I'm joining the School of Informatics at the University of Edinburgh ( @EdinburghNLP , @InfAtEd , @EdinburghUni ) as a faculty member in September 2022, and have funding for PhD students in NLP and ML! 🧑‍🎓🚀 (🧵1/5)

62

137

709

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

For the folks working on reasoning with neural models -- just a reminder that if you train a ResNet to produce the shortest path in a map, accuracy will be ~0%, while if you incorporate a shortest path algorithm into the model, you quickly go above 90%! ()

15

51

365

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

Conditional Theorem Provers are scalable neuro-symbolic reasoning models that learn to recursively select and generate rules on-the-fly conditioned on the goal via gradient-based optimisation! To appear at #ICML2020 , Arxiv Slide 1/N

6

91

355

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

@ilyasut the fact that GPUs are much better than all other NN methods suggests that the brain might be using GPUs too

7

15

344

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

Am I doing this right? Or should I use torch.einsum to achieve consciousness instead?

Ilya Sutskever

@ilyasut

2 years

it may be that today's large neural networks are slightly conscious

453

557

3K

16

13

271

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

5 years

Our paper “Differentiable Reasoning on Large Knowledge Bases and Natural Language” will appear at #AAAI2020 as an oral! We scale neural theorem provers to massively large KBs and corpora. Paper and TF/Py🔥 code will follow :) W/ @backprop2seed , @_rockt , @riedelcastro , @egrefen

15

61

255

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

OMG our @NeurIPSConf paper () got featured on @ykilcher 's YouTube channel!! 🚀🚀🚀 This is quite a life goal for me -- Yannic's paper explanations are extremely clear (and fun!), and I always listen to them when I'm digging into a new topic 🤗

Implicit MLE: Backpropagating Through Discrete Exponential Family...

Combining discrete probability distributions and combinatorial optimization problems with neural network components has numerous applications but poses several challenges. We propose Implicit...

arxiv.org

Yannic Kilcher 🇸🇨

@ykilcher

2 years

🔥New Video🔥How to backpropagate through an algorithm? Seems crazy, but this paper shows it's actually possible for a large class of algorithms, such as k-subset, ILP, and many graph algorithms. Watch my (amateur 🙃) attempt at an explanation here:

3

68

361

6

31

242

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

Our #AAAI2020 paper "Differentiable Reasoning on Large Knowledge Bases and Natural Language" is online! -- Together with an open-source neuro-symbolic reasoning framework, in TF: , w/ @backprop2seed @_rockt @riedelcastro @egrefen

3

54

235

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

1 year

💥KG Embedding Models are GNNs! 💥 We found that their training dynamics can be described via message-passing operations, enabling us to define a new type of GNNs that achieves SOTA results on inductive and transductive link prediction tasks! #NeurIPS22 🧵

9

46

229

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

A wide and deep tutorial on everything about Graph Neural Nets -- theoretical foundations, efficiency and scalability, adversarial attacks, Graph2Seq models, and much more:

2

52

224

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

wish I woke up to a NeurIPS paper & an AAAI tutorial acceptance notification a bit more often 😅 will share more details soon!

12

3

225

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 months

We are working on an open leaderboard for measuring hallucinations in LLMs! 🚀🚀🚀 Read more about it here: Leaderboard address (and discussion forum, if you'd like to provide feedback or help 🙂):

Clémentine Fourrier 🍊 is off

@clefourrier

4 months

Another day, another leaderboard on the hub! ✨ @PMinervini and his team used 14 different datasets to create a leaderboard measuring hallucinations in LLMs 🤯 Read their ool intro blog with an in depth analysis of the results:

0

16

71

5

49

207

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

6 years

Neural theorem provers combine the strengths of rule-based and neural models, but that comes at a great computational cost. In we overcome such difficulties by dynamically considering only the most promising proof paths! @matthecroat @_rockt @riedelcastro

1

54

180

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

29 days

Llama 3 was trained using intra-document causal masking, as suggested by @yuzhaouoe 's paper "Analysing The Impact of Sequence Composition on Language Model Pre-Training"! 🚀🚀🚀

7

26

180

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

A while ago, Erik ( @_kire_kara_ ) got in touch with me to work on his MSc thesis project at @ucl_nlp . Then, one thing led to another, and now this happened! Thank you @_kire_kara_ @danieldazac @michaelcochez , collaborating with you is a blast and I enjoy every second 🙂

ICLR 2024

@iclr_conf

3 years

We are thrilled to announce the #ICLR2021 Outstanding Paper Awards! Out of 860 excellent papers, the award committee identified 8 that are especially noteworthy: Congratulations to the authors!! @shakir_za @iatitov @aliceoh @NailaMurray @katjahofmann

1

122

547

17

19

172

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

8 months

I am keeping a list of datasets and metrics for Hallucination Detection in LLMs, you can find it here: -- Please feel free to contribute! 🧑‍🎓🧑‍💻

1

25

142

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

Do we need deep black-box models to answer complex logical queries in KGs? We show how a neural link predictor can be used to produce more accurate results than SOTA models while improving explanations! w/ @_kire_kara_ @danieldazac @michaelcochez , arxiv:

2

36

141

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

8 months

My amazing collaborators and I will be presenting two papers at @NeurIPSConf on LLM pre-training (, w/ @jeankaddour @oscar__key @p_nawrot ) and complex query answering (, w/ @_kire_kara_ @danieldazac @michaelcochez @IAugenstein )! 🚀🚀

6

16

136

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

Turns out you can scale neuro-symbolic reasoning even further by dynamically generating rules on-the-fly conditioned on a goal, rather than iterating over a possibly very large knowledge base 🙂 To appear at #ICML2020 , stay tuned! (SOTA on the CLUTRR relational reasoning dataset)

5

21

130

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

@choldgraf let's not forget our whole data science stack relies on ~15 people spending their time contributing to open source software -- criticising these seems sub-optimal in many ways to me

4

38

122

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

To appear at #ICLR2021 / @iclr_conf as an Oral - top 2% of all submissions! 🙂 - Massive congrats to @_kire_kara_ for his first paper as an @UCL_NLP MSc student 🙌 and to our amazing collaborators @danieldazac and @michaelcochez 🚀

Complex Query Answering with Neural Link Predictors

Neural link predictors are immensely useful for identifying missing edges in large scale Knowledge Graphs. However, it is still not clear how to use these models for answering more complex queries...

arxiv.org

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

Do we need deep black-box models to answer complex logical queries in KGs? We show how a neural link predictor can be used to produce more accurate results than SOTA models while improving explanations! w/ @_kire_kara_ @danieldazac @michaelcochez , arxiv:

2

36

141

2

19

121

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

Today: - Got to see the view from the Edinburgh office for the very first time (thanks @tetraduzione for the plant! 🥰) - NeurIPS went okay! (stay tuned!) Now probably the universe will decide to compensate in some way -- I'm expecting a trail of grant rejections 😬😂

8

2

118

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

We just integrated our state-of-the-art model for Complex Query Answering on large-scale Knowledge Graphs (, ICLR 2021, Outstanding Paper Award) in the SNAP KGReasoning library: -- check it out! 🚀🧑‍💻 Any feedback is very welcome 🙂

3

23

117

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

"Recent work by Minervini et al (Minervini et al., 2019) gives me hope that a neuro-symbolic hybrid approach could break new ground. [..] The very fact that people are trying gives me more hope; reasoning and knowledge need to be first-class citizens if we are to move forward" ❤️

MONTREAL.AI

@Montreal_AI

4 years

The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence "absolutely brilliant" —Nobel Laureate Danny Kahneman @GaryMarcus : #AIDebate #ArtificialIntelligence #CognitiveModels #Reasoning #RobustAI

1

48

86

1

14

110

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

1 month

Excited to share our latest work on improving LLM pre-training! 🚀 The amazing @yuzhaouoe et al. found that focusing on how pre-training sequences are composed and attended over can significantly improve the generalisation properties of LLMs on a wide array of downstream tasks,

1

26

107

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

1 year

Here's a friendly introduction to Implicit MLE (, NeurIPS 2021) and its adaptive version (, AAAI 2023) for back-propagating through structured discrete distributions and combinatorial solvers: , check it out! 🚀

1

20

103

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

5 years

"Generating Textual Adversarial Examples for Deep Learning Models: A Survey" - - an extremely on point survey on adversarial machine learning for NLP

Adversarial Attacks on Deep Learning Models in Natural Language...

With the development of high computational devices, deep neural networks (DNNs), in recent years, have gained significant popularity in many Artificial Intelligence (AI) applications. However,...

arxiv.org

0

30

103

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

If you want to know how to turn arbitrary black-box combinatorial solvers, such as many graph algorithms, into exponential family distributions and back-propagate through them, check out our NeurIPS 2021 paper! and PyTorch library!

GitHub - uclnlp/torch-imle: Implicit MLE: Backpropagating Through Discrete Exponential Family...

Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions - uclnlp/torch-imle

github.com

Gabriel Peyré

@gabrielpeyre

2 years

Exponential families hybridize convex analysis and statistics.

2

104

547

3

19

103

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

6 months

For a project, we needed to directly optimise a non-differentiable ranking metric -- with Implicit MLE (, NeurIPS 2021 & AAAI 2023) implementing a differentiable Mean Reciprocal Rank took us ~10 minutes 🚀🚀 Sample code available at

3

22

101

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

6 years

At #emnlp18 there was a lot of interest in adversarial examples - inputs designed to cause your model to make a mistake. I think that most people in NLP only see them as slightly perturbed inputs where predictions are different (as in vision), but this is just a special case. 1/3

1

24

100

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

8 months

Why not incorporate a planning algorithm in your neural model, then? 🙃 All it takes with Implicit MLE (, NeurIPS 2021; , AAAI 2023) is a one-line decorator!

Yann LeCun

@ylecun

8 months

Auto-Regressive LLMs can't plan (and can't really reason). "While our own limited experiments didn't show any significant improvement [in planning abilities] through fine tuning, it is possible that with even more fine tuning data and effort, the empirical performance may well

59

153

908

3

17

97

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

🤗🤗🤗

0

95

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

1 year

Hi folks, I just joined @ELLISforEurope ! 🚀 Let me know if you want to collaborate on some ELLIS-related initiatives!

2

3

92

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

Here are some intersting posters I found while attending NeurIPS 🙂 (🧵, 0/N)

2

26

89

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

5 months

Is this guy seriously bragging for making the life of foreign students in the UK harder? 🤦🤦🤦

Rishi Sunak

@RishiSunak

5 months

From today, the majority of foreign university students cannot bring family members to the UK. In 2024, we’re already delivering for the British people.

12K

2K

7K

2

9

87

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

@percyliang For anyone wondering, here's a link to the video describing the project:

GPT-4chan: This is the worst AI ever

#gpt4chan #4chan #ai GPT-4chan was trained on over 3 years of posts from 4chan's "politically incorrect" (/pol/) board.(and no, this is not GPT-4)EXTRA VIDEO...

www.youtube.com

1

3

84

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

1 year

Folks, assume you had to build a fairly large GPU cluster (100+ A100s) for academic use. Which workload manager would you use and why?

Slurm

1224

Kubernetes

804

Sun/Oracle Grid Engine

82

Other (in comments)

370

26

7

85

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

1 year

I'll be presenting "Adaptive Perturbation-Based Gradient Estimation for Discrete Latent Variable Models" at AAAI ( @RealAAAI ) in the next few days! 🚀 If you are interested, get in touch! Paper: Code: Poster:

3

12

79

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

CLARIFY is an European project I'm PI-ing on the UCL side. For the next three years I'll be working on using eXplainable AI methods and algorithms for improving the lives of cancer survivors. Our latest work on neuro-symbolic reasoning fits in this vision.

4

77

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

Consider submitting your work at the @CIKM2021 Workshop on Knowledge Injection in Neural Networks! We have some stellar panellists ( @VeredShwartz , @GaryMarcus , @partha_p_t , and @luislamb ) and speakers ( @ABosselut , and @rao2z ) 🚀 Check it out:

KINN2021

Workshop Schedule (8.00 am PST to 12 noon PST Nov 1st 2021) Workshop Opening Remarks – 8.00 am PST Keynote Talk 1 by Subbaro Kambhampati – 8.05 am PST Invited paper talk: Facts As Experts (Google...

sites.google.com

1

26

76

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

9 months

@HannesStaerk an extremely helpful tool for detecting nans is torch.autograd.set_detect_anomaly(check_nan=True):

1

6

72

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

NLP models that can jointly produce predictions and natural language explanations are pretty cool to work with until you realise you can invert them and make them produce mutually inconsistent explanations! Preprint at , to appear at #ACL2020 @aclmeeting

Make Up Your Mind! Adversarial Generation of Inconsistent Natural...

To increase trust in artificial intelligence systems, a promising research direction consists of designing neural models capable of generating natural language explanations for their predictions....

arxiv.org

Oana-Maria Camburu

@oanacamb

4 years

Our #acl2020nlp paper is now online: ! An adversarial framework for finding inconsistent natural language explanations. -- @BrendanShilling , @PMinervini , Thomas Lukasiewicz, and Phil Blunsom

2

23

61

1

12

71

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

@fchollet "ok the first topic we will cover in organic chemistry 101 will be Dijkstra's algorithm"

4

2

67

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

11 months

@miniapeur @oanacamb 's thesis is great:

Explaining Deep Neural Networks

Deep neural networks are becoming more and more popular due to their revolutionary success in diverse areas, such as computer vision, natural language processing, and speech recognition. However,...

arxiv.org

2

9

69

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

Attending @icmlconf remotely was amazing -- I could present our work and have interesting chats, attend all the talks I wanted (even multiple times!) without having to run from one session to another, and still be able to take the doggo for walkies while camping

4

0

71

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

@saintjavelin the real reason why Russia tried to invade Ukraine:

2

3

67

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

Registration to the Conference on Uncertainty in Artificial Intelligence (UAI, #UAI2021 @UncertaintyInAI ) is *free for all students*, consider attending!!

antonio vergari - hiring postdocs & PhD students

@tetraduzione

3 years

Is there a reason why a BSc, MSc or PhD Student in #AI or #ML should not attend #UAI2021 @UncertaintyInAI on July 27-30? The conference is top tier, will be virtual and registration will be free for students!!! 👇👇👇

5

67

259

1

17

67

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

The @RealAAAI crowd getting ready to hear from Y. Bengio @geoffreyhinton @ylecun

3

17

67

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

How is "Respect for authority" a way to assess candidate grad students @penn_state ? 🤦

8

2

68

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

7 years

Our @uai2017 paper on Adversarial Sets for injecting prior knowledge in neural link predictors @_rockt @riedelcastro

1

15

65

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

Join us in our quest to design deep neural models that can learn discrete structured representations suitable for generating explanations and solving reasoning tasks! 🚀🤖 We have funding for a 2-year postdoc at @EdinburghUni – Apply by Jan 16th, 2023! 🧵

2

21

64

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

1 year

"And is this co-author of yours, Matrix-Vector Multiplication, in the room with us right now?"

🔥Kareem Carr | Statistician 🔥

@kareem_carr

1 year

so i guess we’re doing this now

197

879

6K

2

3

65

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

All recordings from the 1st International Joint Conference on Learning & Reasoning (IJCLR 2021) are now online!

1

15

64

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

Yeah, let's see what Abu Dhabi has to offer! From

EMNLP 2024

@emnlpmeeting

2 years

Discover what Abu Dhabi has to offer. #NLProc

1

6

24

4

7

64

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

9 months

To appear at @cikm2023 !🚀 "Combining Inductive and Deductive Reasoning for Query Answering over Incomplete KGs" with @medinaandresel , Kiên, Csaba, and @DariaStepanov10 from @Bosch_AI ! We provide datasets and methods for complex QA with logical constraints!

0

12

60

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

Q: What adversarial failure mode do reading comprehension chopsticks suffer from? A: Undersensitivity! (confidence: 99.7%) 🙂🥢 -- more in our "Undersensitivity in Neural Reading Comprehension" (), by @Johannes_Welbl @max_nlp Pontus and @riedelcastro

3

15

61

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

5 years

Full room for Ian Goodfellow's ( @goodfellow_ian ) invited talk on Adversarial Machine Learning! @RealAAAI #AAAI19 #AAAI2019

0

4

60

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

8 months

Working with both LLMs and KGs? Consider submitting your work to our newly minted #EMNLP workshop "Knowledge Graphs and Language Models" -- more information soon! 🚀🚀🚀 @emnlpmeeting

Russa Biswas

@russa_biswas

8 months

Thrilled to share that our workshop on Knowledge Graphs and Language Models (KaLLM) has been accepted for @emnlpmeeting 2024! Dive into the world of #KGs and #LLMs with us. Stay tuned for details! #KaLLMWorkshop #EMNLP @frimelle @gdm3000 @PMinervini @sameer_ @agarwal_oshin

0

10

52

1

18

62

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

If you would like to work with me during your PhD (or postdoc -- more on this later!), make sure to apply to the ILCC PhD or the NLP CDT program at the University of Edinburgh! Here is a list of research topics (), and I'm happy to explore new directions!

EdinburghNLP

@EdinburghNLP

2 years

The Institute for Language, Cognition and Computation at the University of Edinburgh invites applications for three-year PhD studentships starting in 2023. These can be in any area of computational linguistics/NLP, speech technology, and cognitive science:

0

12

35

0

26

61

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

Our poster session is starting in ~20 mins, at 10am CEST 💬! Meet us ( @_kire_kara_ , @danieldazac , and yours truly) on GatherTown: @iclr_conf #ICLR2021

0

12

61

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

9 months

Our chapter's preprint "Approximate Answering of Graph Queries" is online! , with the amazing @michaelcochez @dimitrisalivas @_kire_kara_ @berrendorfmax @danieldazac @michael_galkin @Mniepert @ren_hongyu On answering complex queries on incomplete/noisy KGs!

0

14

60

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

8 months

Our "No Train, No Gain" paper on LLM pre-training received some (very well-deserved🙃) media coverage before appearing in the #NeurIPS2023 proceedings; this would have been unthinkable with EMNLP/*CL due to their self-harming arxiv embargo policy 🤕 Paper:

No Train No Gain: Revisiting Efficient Training Algorithms For...

The computation necessary for training Transformer-based language models has skyrocketed in recent years. This trend has motivated research on efficient training algorithms designed to improve...

arxiv.org

TechXplore

@TechXplore_com

9 months

Machine- #training workarounds found to yield little advantage @arxiv

0

1

6

1

8

59

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

If this paper had a Related Work section, it would probably be easier to compare their approach to all the work on Adaptive Computation Time and Conditional Computation being done to this day

Google DeepMind

@GoogleDeepMind

3 years

Introducing PonderNet, a new algorithm that allows artificial neural networks to learn to “think for a while” before answering, like humans do. This improves the ability of neural networks to generalize outside of the training distribution:

24

297

1K

1

6

56

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

A very biased selection of posters from #ICLR2021 🧵 1/n

1

11

59

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

5 months

Bam! 😍😍 One of my fav papers from this year's NeurIPS () mentions our work on learning to calibrate neural link predictors (, also NeurIPS) by back-propagating through the solution search process! Thanks @loreloc_ @tetraduzione !❤️

0

6

58

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

To appear at @emnlp2020 ! I'm really proud of having been Joe's thesis supervisor at UCL: it's impressive to get an MSc thesis project accepted at one of the top NLP conferences. This is one of the 4 accepted papers out of 4 submissions from @UCL_NLP members! Stay tuned 🚀

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

Joe's thesis work “There is Strength in Numbers” is online! He found that adversarial training can be ineffective at removing biases from representations, whereas using ensembles of adversaries produces more robust models, w. Joe Haim @rockt_ @riedelcastro

2

9

54

2

10

57

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

2021 is coming to an end, and I need to sincerely thank all my collaborators (from @UCL_NLP , @UCL_DARK , @MetaAI , @NEC_EMEA , @VUamsterdam , @InfAtEd , and many more) for the ground-breaking work they were able to pull off this year: it was truly an honour working with you. [🧵, 1/N]

1

7

54

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

Joe's thesis work “There is Strength in Numbers” is online! He found that adversarial training can be ineffective at removing biases from representations, whereas using ensembles of adversaries produces more robust models, w. Joe Haim @rockt_ @riedelcastro

2

9

54

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 months

Our "AI Hub in Generative Models" is happening!! 🚀🚀 -- we will focus on developing tools that industry, researchers, and gov can use to build responsible generative models to benefit the economy and society! Thank you, @davidobarber , for leading this 🙂

David Barber

@davidobarber

3 months

I'm delighted to announce that the UCL AI Centre @ai_ucl will be leading a new UK Hub on Generative Models with a fantastic academic team from Edinburgh, Manchester, Cardiff, Surrey, Oxford, Imperial and Cambridge and deep industry support

11

36

205

1

6

53

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

Looking forward to discussing our work at the ICLR 2021 Outstanding Paper session next week! @iclr_conf #ICLR2021

1

3

54

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

Hey @emnlpmeeting , given that homosexuality is a crime in the UAE, what are the guidelines for non-binary folks planning to attend EMNLP? "Please attend remotely or be prepared for some jail time, lashing, or both" @QueerinAI

EMNLP 2024

@emnlpmeeting

2 years

Abu Dhabi is among the safest cities in the world (ranked #31 overall by the 2021 Safe Cities Index) with a culture that embraces diversity while emphasizing mutual tolerance and privacy --- an umbrella under which many lifestyles thrive. (5/8)

5

3

9

3

13

54

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

6 years

We built on this for and - the idea is: just declaratively specify some knowledge, let the adversary use it for finding where the model is wrong (without labeled data), and fix it via adversarial training. 3/3

Adversarially Regularising Neural NLI Models to Integrate Logical...

Adversarial examples are inputs to machine learning models designed to cause the model to make a mistake. They are useful for understanding the shortcomings of machine learning models,...

arxiv.org

0

5

53

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

"Deep Adversarial Learning for NLP" - , - a very rich set of pointers on adversarial examples, training, and generation in NLP. A bit perturbation oriented, but very cool nevertheless! From @WilliamWangNLP , @sameer_ et al./1

1

20

53

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

1 year

Bam! Massive congrats @mindjimmy , @zhaoyuhitsz et al. 🚀🚀🚀 The @emnlpmeeting paper is available here:

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

Here it is! An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks, #emnlp2022 @emnlpmeeting

0

2

19

6

8

53

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

We have an opening for a Research Fellow (Postdoc) position to work on improving the generalisation properties of Question Answering systems, funded by @Cisco ! 🚀 If you would like to join our amazing group ( @UCL_NLP ) at @AI_UCL , apply by April 6th! 📆

1

27

52

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

5 years

Leon and I will be at #ACL2019 ! If you like the work, or the idea of end-to-end differentiable query reformulation, come chat with us! :) #acl2019nlp

Leon Weber-Genzel

@weberple

5 years

I'm really excited about our ACL 2019 paper on End-to-end Differentiable Reasoning for explainable Multi-Hop Question Answering: - joint work with @PMinervini , Jannes Münchmeyer, Ulf Leser, @_rockt

1

12

80

1

4

52

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

5 months

Looking for the best LLM pre-training regimen on an academic hardware budget? Check out Jean's ( @jeankaddour ) amazing NeurIPS poster! 🚀🚀🚀 👇 Paper:

No Train No Gain: Revisiting Efficient Training Algorithms For...

The computation necessary for training Transformer-based language models has skyrocketed in recent years. This trend has motivated research on efficient training algorithms designed to improve...

arxiv.org

Jean Kaddour

@jeankaddour

5 months

Thank you to everyone who came to see our poster! #NeurIPS2023

1

19

90

0

11

51

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

how it started / how it's going

4

0

51

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

In his MSc project, @AndrewJWren proposed a new method based on Implicit Maximum Likelihood Estimation for learning latent discrete DAG-structured representations! To appear at the #CML4Impact and #nCSI workshops at @NeurIPSConf !

Andrew J. Wren

@AndrewJWren

2 years

Out today Discrete Directed Acyclic Graphs via Backpropagation takes a probabilistic approach to learn DAGs from data. Grateful to my excellent co-authors & UCL MSc ML supervisors @PMinervini , Luca Franceschi & Valentina Zantedeschi! #NeurIPS2022

0

1

10

2

12

51

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

Can neural link predictors invent new concepts and leverage them to improve generalisation? 🚀 Check out @agaa_d 's @akbc_conf paper "Neural Concept Formation in Knowledge Graphs", and catch up with her at the conf! , Today 19:10 UTC!

0

10

50

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

28 days

I just migrated some code from @huggingface text-generation pipelines to vLLM ( @vllm_project ) and got a ~6x speed improvement right away -- what's going on? 🤯

5

2

50

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

1 year

@togelius We won an ICLR Outstanding Paper Award on an academic compute budget, with an MSc project, nevertheless! 🙂 -- In our case, the lack of compute forced us to come up with clever ideas to solve the problem.

Complex Query Answering with Neural Link Predictors

Neural link predictors are immensely useful for identifying missing edges in large scale Knowledge Graphs. However, it is still not clear how to use these models for answering more complex queries...

arxiv.org

0

3

49

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

@DarceyNLP About PyTorch -- my quality of life improved by several orders of magnitude when I started using einsum! Check it out!

3

6

48

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

6 years

I'll be in Brussels for presenting our work on using logic for generating adversarial examples for NLI models, and making models more robust via adversarial training! If you are attending @conll2018 or @emnlp2018 , let's catch up!

0

11

48

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

1 year

"winter break" 😅

4

0

46

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

1 year

Happening today! Poster: Paper:

EdinburghNLP

@EdinburghNLP

1 year

An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks. Yuxiang Wu, Yu Zhao, Baotian Hu, Pasquale Minervini, Pontus Stenetorp and Sebastian Riedel Paper: Poster: Poster Session 9, Dec 10th, 11am Local time 4/N

1

8

1

4

48

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

4 years

@SchmidhuberAI another favourite paper of mine, from @ZoubinGhahrama1 et al. got rejected from NIPS before becoming one of the most influential papers in ML ever! never give up!

0

47

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

5 months

@hbouammar The fact that something is simple doesn't imply it's not worthy of sharing/publishing --

3

48

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 months

Amazing work, @soheeyang_ et al.!! Thank you, @KassnerNora , for talking about this work last week at @InfAtEd 🙂 Paper: Do Large Language Models Latently Perform Multi-Hop Reasoning?

AK

@_akhaliq

3 months

Google announces Do Large Language Models Latently Perform Multi-Hop Reasoning? study whether Large Language Models (LLMs) latently perform multi-hop reasoning with complex prompts such as "The mother of the singer of 'Superstition' is". We look for evidence of a latent

1

50

267

3

8

48

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

If you would like to work with me, please get in touch!

Yuge Shi (Jimmy)

@YugeTen

2 years

The school of informatics at the University of Edinburgh and DeepMind are offering an ML PhD scholarship for students who identify as gender/racial/ethnic minorities in 2022/23. See thread for details. (1/n)

4

77

236

1

10

47

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

1 month

@miniapeur "cool work! seems super related to this paper of mine, check it out: <arxiv link>"

1

0

46

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

We got featured in @seb_ruder 's amazing NLP newsletter!

Sebastian Ruder

@seb_ruder

3 years

NLP News: ICLR 2021 Outstanding Papers, Char Wars, Speech-first NLP, Virtual conference ideas Featuring a round-up of @iclr_conf best papers, ideas for fun things to do at virtual conferences, Star Wars references 🛸, and more...

8

61

332

0

2

47

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

3 years

I'm happy that, for a very talented MSc student I was lucky enough to supervise, the first impact with peer review looked like this 🤗🤍

Sergey Ivanov

@SergeyI49013776

3 years

#ICLR2021 reviews are finalized and below is a final table with scores. Here is a teaser of how rebuttal helps the papers. 🧵1/n

3

20

98

1

47

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

2 years

To whoever wrote a blog post about our work and submitted it to the ICLR 2022 Blog Track () -- thank you! 🙏 I'm really looking forward to reading it!

3

4

46

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

1 year

Check out Joe's latest work, "Logical Reasoning for Natural Language Inference Using Generated Facts as Atoms" It uses a LLM to decompose a premise into atomic facts and trains a model with a strong logical inductive bias to take instance-level decisions!

Joe Stacey

@_joestacey_

1 year

Really happy to share our new paper on logical reasoning with NLI! Its been a really exciting project to work on, so thanks so much to my amazing collaborators and supervisor @PMinervini @oanacamb Haim and @MarekRei !

3

4

28

0

10

44

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

7 months

🚀🚀🚀🚀 More on our Implicit MLE (links to our NeurIPS and AAAI papers, videos, slides, code, etc.) here:

GitHub - EdinburghNLP/torch-adaptive-imle

Contribute to EdinburghNLP/torch-adaptive-imle development by creating an account on GitHub.

github.com

AI for Global Goals

@GlobalGoalsAI

7 months

Dr. Pasquale Minervini ( @PMinervini ) uncovers the power of blending algorithms with neural networks in AI. Boosting problem-solving and reasoning skills in AI just got real! 💡 #OXML23 #AI4GG #SDGs #Machinelearning #AI #NeuralNetworks #AlgorithmicAI #AIInnovation

0

1

11

0

4

45

Pasquale Minervini 🚀 looking for postdocs!

@PMinervini

1 year

To appear at AAAI 2023! @RealAAAI 🚀 The camera-ready will contain more experiments and analyses, and we will release a plug-and-play library for adaptively back-propagating through complex discrete distributions and combinatorial solvers -- stay tuned! 🙂

Adaptive Perturbation-Based Gradient Estimation for Discrete...

The integration of discrete algorithmic components in deep learning architectures has numerous applications. Recently, Implicit Maximum Likelihood Estimation (IMLE, Niepert, Minervini, and...

arxiv.org

fly51fly

@fly51fly

2 years

[LG] Adaptive Perturbation-Based Gradient Estimation for Discrete Latent Variable Models P Minervini, L Franceschi, M Niepert [UCL Centre for Artificial Intelligence & University of Stuttgart] (2022) #MachineLearning #ML #AI

1

3

5

0

8

44