Edward Grefenstette @egrefen Twitter profile

Pinned Tweet

Edward Grefenstette

5 months

Can't refuse a direct request from the Cho-sen one. 🧵 TIME FOR A YEAR END RETROSPECTIVE 🧵 wherein I get to brag about having the honour of having been included on my students' papers who, by now, are generally more clued in than me about what's what. Let's go! [1/24]

Kyunghyun Cho

@kchonyc

5 months

the end of the year is upon us; please share your objective and highly certain merit with all of us. ho ho ho

2

0

10

3

6

33

Last Seen Profiles

@NeoWang_CVN

@nickrheinberger

@ICI

@ssishuwa

@AHOY_Pirates

@godwinjohn33

@renj1i

@GuylaineLemair5

@LT91CB97

@WeaverBask

@StampOutSpiking

@brendonurie

@wave_mjy

@ofsaytadustum

@FranceJerusalem

@Run_Forefoot

@TransitoPolicia

@siyarion

@CurrySandwiches

@stantradingman

@Titouss01

@HancoxWitch

@StoneGrag

@drishtiias

@cannibalxing

@delta_downfall_

@rstaynings

@liviodilo

@MarthaKelly3

@swiftie_awardss

@KIMWOOSEOK_SG

@RepTedDeutch

@Dhayt0

@Full_Metal_Ray

@DLanceBlack

@MixtheSkele

Edward Grefenstette

@egrefen

8 months

I will be posting (probably next week) some job listings for a new team I’m hiring into at @GoogleDeepMind . I’ll be looking for some research scientists and engineers with a strong engineering background to help build increasingly autonomous language agents. Watch this space.

30

70

817

Edward Grefenstette

@egrefen

4 years

Baguette-propagation.

14

58

755

Edward Grefenstette

@egrefen

3 years

Sorry, @DeepMind , it looks like we got there first! Proud to announce that our Jointly Optimized Kernel Evaluator agent achieved AGI this morning around 20 mins past 4am GMT. The solution was delightfully simple and yet elegant. It won’t fit in this tweet so THREAD (1/70)

11

44

758

Edward Grefenstette

@egrefen

5 years

I'm happy to announce I've joined @facebookai (FAIR) as as research scientist, working out of the London office. Get in touch if you want to chat about internships, PhDs, working together, etc.

46

31

602

Edward Grefenstette

@egrefen

3 years

I don’t want to brag, but I wrote (basically almost published) some homework when I was 5 that used multiplication (basically almost linear algebra), which was used in DL methods like LSTM, GANs, etc so you can thank me later for basically inspiring the most cited papers in ML.

10

20

578

Edward Grefenstette

@egrefen

11 months

Pleased as punch (the drinky kind, not the hurty kind) to be returning to Google @DeepMind as Director of Research today. It's an exciting time to be helping develop general agents that can adapt to open-ended environments, communicate with us, and help us in novel ways!

51

15

567

Edward Grefenstette

@egrefen

1 year

@maosbot Not defending the view you’re objecting to, but Japan had a good pre-pandemic attitude towards mask-wearing e.g. when ill, during other epidemics, etc. I was rather hoping we’d develop some of this attitude in the west after the pandemic, but here we are 😔

19

20

557

Edward Grefenstette

@egrefen

3 years

Want to do a PhD with me at @facebookai (FAIR) + @uclcs ? I am recruiting up to one student this year for a special FAIR-UCL studentship. FAIR will pay your fees (inc. international fees) + an extremely generous stipend, plus access to FAIR compute. Interested? Read on... (1/5)

16

108

502

Edward Grefenstette

@egrefen

2 years

Today I’m delighted to announce that I have joined @CohereAI as Head of ML, to assist the further developing of ML R+D across the company, and help build up the new London office! 🎉 Excited to start a new journey with this brilliant team 🥰

cohere

@cohere

2 years

Today we’re welcoming aboard @egrefen and @pblunsom to lead the grand opening of our new London office! Interested in tagging along? We’re currently hiring in London and across the 🌍! Check out open positions at , or drop us a line at talent @cohere .com

9

23

188

45

12

494

Edward Grefenstette

@egrefen

5 years

I think that the string "Pre-AGI" is the shortest number of characters that gives me an instant rage aneurism. I despair for the future of our once approximately scientific field, now become the world's most well-financed techno-cult.

13

75

485

Edward Grefenstette

@egrefen

3 years

🧵THREAD 🧵 Are you looking to do a 4 year Industry/Academia PhD? I am looking for 1 student to pioneer our new FAIR-Oxford PhD programme, spending 50% if their time at @UniofOxford , and 50% at @facebookai (FAIR) while completing a DPhil (Oxford PhD). Interested? Read on… 1/9

10

114

484

Edward Grefenstette

@egrefen

3 years

Humans are powerful generative models. This is such an amazing example, just shake your device a little.

Bence Nanay

@BenceNanay

3 years

Recognise the painting?

31

71

419

14

98

484

Edward Grefenstette

@egrefen

7 years

Probably one of the more important DL papers of the last 5 years: it shows that the DL community has been good at rushing to flag-plant by creating flashy new models, but terrible at evaluating them by training good baselines. Can you trust YOUR model’s results in <insert task>?

Gábor Melis (@[email protected])

@GaborMelis

7 years

Give your baselines some love and they might surprise you:

3

127

309

7

160

434

Edward Grefenstette

@egrefen

5 years

Strongly (and respectfully) disagree with @NandoDF here. ML has a deep reproducibility problem. "Successful" methods are often unstable and require tricks not described in paper. Rarely have I been able to repro results without looking at assumptions in code but not in paper.

Nando de Freitas 🏳️‍🌈

@NandoDF

5 years

Research is not about re-running code, adding many comparisons where we know who wins, using citations purely for credit assignment, writing papers that no one remembers. We don’t want Neurips2020 to reproduce Neurips2019. We want it to be different and full of new ideas.

8

40

191

9

63

397

Edward Grefenstette

@egrefen

5 years

Announcing TorchBeast, an IMPALA-inspired @pytorch platform for distributed RL research. Used in a growing number of projects here at @facebookai . Project lead by Heinrich Küttler, with major effort by @nntsn et al. Paper: Code:

GitHub - facebookresearch/torchbeast: A PyTorch Platform for Distributed RL

A PyTorch Platform for Distributed RL. Contribute to facebookresearch/torchbeast development by creating an account on GitHub.

github.com

6

107

348

Edward Grefenstette

@egrefen

5 years

Clearly @DeepMindAI is primarily about fundamental research so of course it's loss making. Why is anyone (triumphantly) surprised about this? Do we look at profit margins of academic groups? Would the world be a better place if groups like DM changed focus to generate income?

Gary Marcus

@GaryMarcus

5 years

“Alphabet’s DeepMind Losses Soared To $570 Million In 2018” ⁦⁦ @Forbes ⁩

5

18

38

13

42

350

Edward Grefenstette

@egrefen

8 months

🚨 JOB ALERT 🚨 We're hiring research scientists/engineers to conduct research on next-generation assistant technologies to power increasingly autonomous agents which strive to support humans Research Scientist: Research Engineer:

15

83

343

Edward Grefenstette

@egrefen

1 year

@NathanpmYoung I’m about to blow your mind then because here’s an efficient way to pack 25 squares into a square.

6

1

315

Edward Grefenstette

@egrefen

4 years

Got a complicated RL exploration problem? Sparse/no reward? It's dangerous to go alone: bring an AMIGo! This thread introduces work done by Andres Campero, with @robertarail , Josh B. Tenenbaum, @HeinrichKuttler , @_rockt and me during Andres' internship at FAIR London. [1/5]

4

57

286

Edward Grefenstette

@egrefen

3 years

Happy to announce that I’ve been “promoted” to Honorary Professor of @UCL . I will continue to support research at @ucl_dark and within @ai_ucl in general.

22

7

284

Edward Grefenstette

@egrefen

5 years

I think there's something fundamentally wrong and unscientific with @AndrewYNg 's "heroes of deep learning" series. It's a terrible culture to assume a field has a few superstars rather than building on the work of many. Encourages West Coast-style self-promotion above rigour.

14

23

282

Edward Grefenstette

@egrefen

5 years

Happy to announce our paper on Generalized Inner Loop Meta Learning, aka Gimli (), with @brandondamos , @denisyarats , Phu Mon Htut, Artem Molchanov, Franziska Meier, @douwekiela , @kchonyc , and @soumithchintala . THREAD [1/6]

5

83

282

Edward Grefenstette

@egrefen

4 years

I’ll have whatever ⁦ @fchollet ⁩ is having 😉

6

8

268

Edward Grefenstette

@egrefen

4 years

I cannot believe my eyes, as @NeurIPSConf 2020 AC, that these are suitable reasons for desk rejection. The first and third points are things that should be evaluated by several reviewers. The third point (esp re presentation) is not a good sole reason for rejection. But… (cont.)

8

52

263

Edward Grefenstette

@egrefen

8 months

The idea that an ML paper should be written with a plot twist really tickles me. Like “oh yeah this method sucks but let’s do it lip service and talk about some evals and WAIT WHAT’S THIS?! ITS SOTA ENTERING THE RING WITH A STEEL CHAIR!!”.

Dan Roy

@roydanroy

8 months

No no no no no no no no no. Thankfully, this advise was ignored by the authors. But this wide spread but unspoken belief is why NeurIPS/ICML/ICLR reviewing for empirical papers is totally broken.

26

379

9

13

253

Edward Grefenstette

@egrefen

5 years

@EylonALevy This is the sort of shit you get to say if you've had the *privilege* of not being affected by her racist, xenophobic, and classist policies as both homesec and PM. Not everyone has that luxury. Yours, A citizen of nowhere.

1

5

231

Edward Grefenstette

@egrefen

5 years

To wrap up 2018, I can now announce that after four interesting years there, I left @DeepMindAI back in November. I am thankful for the friendships and collaborations I formed along the way. I will miss many, many of my former colleagues, but am excited for my next adventure! 😀

9

10

244

Edward Grefenstette

@egrefen

4 years

In case you were wondering...

5

34

244

Edward Grefenstette

@egrefen

5 years

In parallel with this paper, @facebookai has released higher, a library for bypassing limitations to taking higher-order gradients over an optimization process. Library: Docs: Contributions very welcome.

GitHub - facebookresearch/higher: higher is a pytorch library allowing users to obtain higher order...

higher is a pytorch library allowing users to obtain higher order gradients over losses spanning training loops rather than individual training steps. - facebookresearch/higher

github.com

Edward Grefenstette

@egrefen

5 years

Happy to announce our paper on Generalized Inner Loop Meta Learning, aka Gimli (), with @brandondamos , @denisyarats , Phu Mon Htut, Artem Molchanov, Franziska Meier, @douwekiela , @kchonyc , and @soumithchintala . THREAD [1/6]

5

83

282

1

76

239

Edward Grefenstette

@egrefen

5 years

Doing a PhD in a CS/ML related field? We're looking for several(!) interns to come work with likes of @_rockt , @riedelcastro , and me (+ others) on NLU, RL+Language, Program Induction/Synthesis, and Metalearning at FAIR London in 2020. Get in touch!

22

59

240

Edward Grefenstette

@egrefen

6 years

Many machine reading datasets only require extracting a short span/entity. To drive research on systems that can read and understand complex narratives, we introduce NarrativeQA: human questions & answers about entire books/plays/movies. Upcoming in TACL.

The NarrativeQA Reading Comprehension Challenge

Reading comprehension (RC)---in contrast to information retrieval---requires integrating information and reasoning about events, entities, and their relations across a full document. Question...

arxiv.org

4

77

234

Edward Grefenstette

@egrefen

4 years

@archer_rs The more I read this, the more I think it must be some sort of indirect poe, but ultimately I don’t care: If it’s all true, it’s a beautiful schadenfreude-inducing story, and if it’s not, this is some top class writing. Either way, I await the next tweet with bated breath.

4

9

222

Edward Grefenstette

@egrefen

4 years

Lots of interest in meta-learning/differentiable optimization at #ICLR2020 . We're happy to announce v0.2 of higher, a @PyTorch library for writing meta-learning research code in near-native pytorch. This is a fairly big update addressing some key blockers.

4

46

216

Edward Grefenstette

@egrefen

6 years

Happy to share our new @DeepMindAI paper on AGILE, a method for training agents to follow language instructions by jointly learning a reward model from examples. No more template languages, or problems with hard/impossible to code reward functions!

Learning to Understand Goal Specifications by Modelling Reward

Recent work has shown that deep reinforcement-learning agents can learn to follow language-like instructions from infrequent environment rewards. However, this places on environment designers the...

arxiv.org

2

50

210

Edward Grefenstette

@egrefen

8 months

Congratulations Dr @MinqiJiang ! @_rockt and I are so proud of the first PhD graduate from @UCL_DARK 🥰

24

9

213

Edward Grefenstette

@egrefen

1 year

Oof. What a read!

The AI Founder Taking Credit For Stable Diffusion’s Success Has A History Of Exaggeration

Stability AI became a $1 billion company with the help of a viral AI text-to-image generator and some misleading claims from founder Emad Mostaque.

www.forbes.com

12

23

211

Edward Grefenstette

@egrefen

4 years

For the next conf, I’m contemplating taking LSD and engaging in an hour of Dadaist automatic writing. I’ll throw in some figures with a few buzz words, some unparsable maths with Greek letters you haven’t even HEARD of, TeX it all up, and submit. 100% novelty. Strong accept.

8

10

209

Edward Grefenstette

@egrefen

3 years

Paging @AndrewYNg , or whoever actually writes The Batch: the headline you were looking for was "Wreck-ognition". That is all.

4

12

206

Edward Grefenstette

@egrefen

2 years

This surprising result should serve as a moment of reckoning for RL research. Reward may be enough in theory (if only) but an astounding amount of domain knowledge can, and probably must, be exploited in order to tractably solve complex problems.

The NetHack Learning Environment

@NetHack_LE

2 years

Proud to present this short report on the outcomes of and learnings from #NetHackChallenge21 , held at @NeurIPSConf . Did DeepRL win the day, or did symbolic challengers surprise us all? What do the results tell us about next steps for AI? Read to find out!

10

73

302

5

32

202

Edward Grefenstette

@egrefen

9 months

At AMLD GenAI, @armandjoulin is telling us how the building of custom language models is increasingly going to be within the reach of smaller teams and orgs. Paired with Angela Fan’s and @jefrankle ’s talks yesterday, this paints a picture of a future where LLMs proliferate.

12

22

189

Edward Grefenstette

@egrefen

1 year

I just don’t get this attitude of saying something won’t work until you’re red in the face. Conversational search is a cool idea. If the tech isn’t ready, or the idea was cool but not useful, people just won’t flock to it. Chill out and let some notion of utility be the judge.

@[email protected] on Mastodon

@emilymbender

1 year

Chatbots are not a good UI design for information access needs

5

48

207

9

15

192

Edward Grefenstette

@egrefen

5 years

Good (deep) RL work shows stddev and mean across many seeds go demonstrate reliability of the method, rather than top-k (out of ???) runs. Most papers I've read do not do anything nearly as sound. Maybe I'm reading the wrong papers...

Dan Luu

@danluu

5 years

"actually the seed is also a hyper-parameter"

9

231

988

8

35

186

Edward Grefenstette

@egrefen

4 years

I think what a lot of senior people posting meta about NeurIPS acceptance/rejection/excitement might be forgetting is that it was a lot easier (and dare I say more fun) to get speculative/exploratory (or just any) work published circa 2010-2014 than now. (1/4)

2

15

188

Edward Grefenstette

@egrefen

6 years

Final version of our ICLR paper is out. Key points: * ConvNets good for vision, not so good for tree structure. * Explicit conditioning on syntax helps. * Nets can learn model-theoretic solution to entailment. * Attention is not (always) all you need.

Can Neural Networks Understand Logical Entailment?

We introduce a new dataset of logical entailments for the purpose of measuring models' ability to capture and exploit the structure of logical expressions against an entailment prediction task. We...

arxiv.org

0

67

178

Edward Grefenstette

@egrefen

7 years

Yup. Definitely the best #ICLR2017 slides.

3

79

175

Edward Grefenstette

@egrefen

4 years

Want to help push the boundaries of RL research? Need a rich, difficult, and procedurally-generated environment with loads of structure and intricacy? An astounding amount of human play data? Sophisticated strategies and documentation? We got you (and it's faster than ALE!) [1/6]

2

49

172

Edward Grefenstette

@egrefen

5 years

A brilliant @PyTorch implementation of continuous stacks, extending work we did on unbounded neural memory at @DeepMindAI , and also inspired by related work on algorithm induction by @armandjoulin and Mikolov of @facebookai .

Will Merrill

@lambdaviking

5 years

Better, faster stack neural networks (in PyTorch)!

1

32

157

0

44

169

Edward Grefenstette

@egrefen

6 months

I guess the “open” part referred to the way corporate drama unfolds, rather than code or model release policies.

2

4

167

Edward Grefenstette

@egrefen

6 years

Super cool @PyTorch reimplementation (+ new stuff) of our @DeepMindAI differentiable stacks/queues/etc (NIPS'15) by @Yale undergrad(!) Will Merrill. Check it out!

GitHub - viking-sudo-rm/StackNN: Experiments with differentiable stacks and queues in PyTorch

Experiments with differentiable stacks and queues in PyTorch - viking-sudo-rm/StackNN

github.com

0

47

166

Edward Grefenstette

@egrefen

7 years

What f***ing trashy excuse of a journalist writes this sh***y sensationalist s***? DO YOU GUYS NOT HAVE EDITORS??

22

50

165

Edward Grefenstette

@egrefen

4 years

lol guess I’ll never make it to A(G)I stardom via that podcast, I guess. Was it something I said? 😃

14

1

163

Edward Grefenstette

@egrefen

5 years

ICLR2020 CFP says "Author names are revealed at the end of the conference." Love the mystery of it all! Should we wear masks while presenting? 🙂

5

161

Edward Grefenstette

@egrefen

3 years

I should probably be deeply ashamed by admitting this, but I've only *just* realised that the gradient of MSE loss (with 1/2 coef) is equivalent to taking the gradient of the KLD of two gaussians with the prediction and target values as means (and variance 1), wrt the prediction.

6

11

154

Edward Grefenstette

@egrefen

4 years

@TheBabyExpert @piersmorgan It’s not clear from the article you linked why or how the use of the term in this context would be racist.

6

0

137

Edward Grefenstette

@egrefen

3 years

∂ln(🙂)/∂🙂 = 🙃 Am I doing this right?

3

148

Edward Grefenstette

@egrefen

4 years

THREAD (application process at bottom) We are looking for research interns to work with me, @_rockt , @HeinrichKuttler et al. at @facebookai (FAIR) London. Applicants should be doing a PhD, and ideally be interested in a project aligned with the topics of our recent pubs. [1/4]

7

26

149

Edward Grefenstette

@egrefen

2 years

On this most auspicious day, I am happy to end the speculation and announce that I will be setting up the world’s first AI-first beet farm in rural Pennsylvania. Looking forward to producing some fresh beets using the best gradients money can buy.

Sam Shead

@Sam_L_Shead

2 years

Those who have left the company include Edward Grefenstette, a research scientist that led Meta’s efforts on a branch of AI known as reinforcement learning, who departed in February.

1

6

19

1

147

Edward Grefenstette

@egrefen

2 months

I say this lovingly, because I have nothing but respect for Andrej, however… Silicon Valley is one hell of a drug if *human communication* now receives this sort of characterisation.

Andrej Karpathy

@karpathy

2 months

Reading a tweet is a bit like downloading an (attacker-controlled) executable that you instantly run on your brain. Each one elicits emotions, suggests knowledge, nudges world-view. In the future it might feel surprising that we allowed direct, untrusted information to brain.

792

1K

11K

16

5

141

Edward Grefenstette

@egrefen

8 years

Montreal DLSS talks now online. Slide-aligned videos coming soon. Awesome set of lectures!

0

85

134

Edward Grefenstette

@egrefen

3 years

Pleased to have been awarded, with @LittleBimble , the 2021 IJCAI-JAIR Best paper prize. Through a series of unfortunate events and miscommunications, we were sadly unable to be at IJCAI to receive the prize or present the work, but thank the committee for recognising our work.

6

10

140

Edward Grefenstette

@egrefen

5 months

Work-life balance is a recipe for a kind of achievement Pedro doesn’t understand.

Pedro Domingos

@pmddomingos

5 months

Work-life balance is a recipe for underachievement.

158

79

846

2

4

137

Edward Grefenstette

@egrefen

5 years

BECOMING A PARENT IS A MAGICAL TIME OF YOUR LIFE THEY SAID

9

3

137

Edward Grefenstette

@egrefen

2 years

Man these @OpenAI DALL·E 2 samples are getting more and more impressive. Note how despite the absurdity of the prompt, the photorealism is almost lifelike. Amazing…

No Context Brits

@NoContextBrits

2 years

A subway under a Subway.

78

402

10K

2

4

136

Edward Grefenstette

@egrefen

7 years

And now, for something a little different… We show that robust, interpretable latent rules can be synthesised by backprop. Data efficient, good generalisation. Can be trained end-to-end within a larger neural network. Upcoming in JAIR. w/ @LittleBimble

Learning Explanatory Rules from Noisy Data

Artificial Neural Networks are powerful function approximators capable of modelling solutions to a wide variety of problems, both supervised and unsupervised. As their size and expressivity...

arxiv.org

3

37

133

Edward Grefenstette

@egrefen

4 years

One thing to highlight is the FiLM² layer introduced in §4.1 of the paper, which is a particularly simple-yet-powerful way of cross-conditioning from 2+ modalities. We did text/vision but in principle this works for anything… @pytorch code for this layer:

Edward Grefenstette

@egrefen

4 years

The code for our RTFM task suite and text2π architecture (in @PyTorch ) is now available at ! Great work by @hllo_wrld ! Read the paper: and a blog post about the work:

0

12

47

1

25

135

Edward Grefenstette

@egrefen

4 years

"Going" to @icmlconf ? Come hear about the future of language+RL at the #LaReL2020 workshop on Language in Reinforcement Learning, held July 18. Here's a short thread introducing some of the highlights. [1/9]

1

47

129

Edward Grefenstette

@egrefen

3 years

I am honoured to have been named an @ELLISforEurope Fellow. Together with @_rockt , we are committed to advancing the cause of European AI/ML Research within our work at @ai_ucl 's @ucl_dark , and @facebookai .

5

4

131

Edward Grefenstette

@egrefen

6 years

Delighted that our paper on neural nets and logic was accepted to ICLR'18. Summary: on highly and heterogeneously structured tasks (detecting logical entailment), models are ranked: semantically-aware > syntactically-aware > LSTM > ConvNet.

1

37

130

Edward Grefenstette

@egrefen

5 years

"AI is the new electricity" "AI = gradients + pray + love" Etc. Is there something in the water in silicon valley (and occasionally elsewhere) which compels people to write what are, at best, empty ML platitudes, and at worst (to paraphrase Pauli) "not even wrong"?

Greg Brockman

@gdb

5 years

For differentiable problems, there’s backpropagation. For everything else, there’s RL.

17

57

448

8

15

129

Edward Grefenstette

@egrefen

5 years

Awesome new "learning to reason" (mathematically) dataset and task suite (which I helped out with a bit in my last months at @DeepMindAI ), in a project envisioned, lead, and primarily executed by the brilliant @dwsaxton . Give it a look!

Google DeepMind

@GoogleDeepMind

5 years

Today we're releasing a large-scale extendable dataset of mathematical questions, for training (and evaluating the abilities of) neural models that can reason algebraically. Paper: Code and data:

18

788

2K

2

25

128

Edward Grefenstette

@egrefen

5 years

Say it with me: meta-learning is not the same the same thing as few shot learning.

7

15

128

Edward Grefenstette

@egrefen

7 years

#acl2017nlp Semantic Compositionality DL Slides:

4

38

124

Edward Grefenstette

@egrefen

5 years

Grefenstette_count += 1

13

1

124

Edward Grefenstette

@egrefen

5 years

Predictably, the useless leeches at @ElsevierConnect show how little they understand or care about science and scholarship. I can't wait for this industry to die out. Your taxes paid for the research, so why let these muppets profit from it? Support open access publication.

Elsevier

@ElsevierConnect

5 years

Want a higher h-index? Maybe you should be spending more time on Twitter. #Altmetrics reveal insights into the impact of scientific knowledge:

16

142

325

3

31

128

Edward Grefenstette

@egrefen

3 years

This is a weird take. By the same token, is CS a branch of logic or of statistics? Is physics a branch of maths, or chemistry a branch of physics? Just because a field evolves to use the methods of another doesn’t make it a branch of it, if the focus of the field is different.

François Chollet

@fchollet

3 years

Within 10-20 years, nearly every branch of science will be, for all intents and purposes, a branch of computer science. Computational physics, comp chemistry, comp biology, comp medicine... Even comp archeology. Realistic simulations, big data analysis, and ML everywhere

298

1K

5K

7

1

125

Edward Grefenstette

@egrefen

6 years

Excellent article by @andrey_kurenkov . A great and healthy step back from recent successes of DeepRL to ask "can we do even better?" by questioning whether methods unbiased by prior experience and instruction can scale.

RL’s foundational flaw

RL as classically formulated has lately accomplished many things - but that formulation is unlikely to tackle problems beyond games. Read on to see why!

thegradient.pub

1

50

125

Edward Grefenstette

@egrefen

4 years

Happy to have played a part in the design of @facebookai 's NetHack Learning Environment, a project lead by Heinrich Küttler and @_rockt , with significant contributions by @nntsn and many others. This env will help push the boundaries of RL research. [1/7]

2

39

123

Edward Grefenstette

@egrefen

3 years

There's some weapons-grade auto-back-patting coming out of @OpenAI this week. Sure, you *often* need dedication/effort/rigour to make progress in any domain. But (a) it's toxic to suggest this is the only way to do so, (b) it's just wrong to equate effort with working a 90h week.

2

5

123

Edward Grefenstette

@egrefen

6 years

The NarrativeQA dataset is now online:

GitHub - google-deepmind/narrativeqa: This repository contains the NarrativeQA dataset. It includes...

This repository contains the NarrativeQA dataset. It includes the list of documents with Wikipedia summaries, links to full stories, and questions and answers. - google-deepmind/narrativeqa

github.com

Edward Grefenstette

@egrefen

6 years

Many machine reading datasets only require extracting a short span/entity. To drive research on systems that can read and understand complex narratives, we introduce NarrativeQA: human questions & answers about entire books/plays/movies. Upcoming in TACL.

4

77

234

1

57

123

Edward Grefenstette

@egrefen

3 years

... and this joke is somewhat ruined by the fact that Twitter won’t let me suppress the video preview when I link to a video explaining our fabulous method. Oh well, here you go anyway...

3

121

Edward Grefenstette

@egrefen

3 years

One of the greatest moments of disappointment in my life was when I discovered that the expression “balls to the wall” refers to the operation of levers in a train or plane, rather than to testicles. It immediately made saying it 80% less funny.

15

6

118

Edward Grefenstette

@egrefen

17 days

For the last year or so, I’ve been saying (to anyone willing to listen), that—modulo best eng practices and appropriate scale—most research and progress in AI is going to come from rethinking how we evaluate models and use data. A short 🧵

2

10

119

Edward Grefenstette

@egrefen

4 years

The second point *really* gets my goat. Sure, we don’t want to send some mad rambling to reviewers, but who will be most affected by desk rejections on this point? ESL folk. Non-traditional submitters. Small labs in countries with few native English speakers. (cont.)

5

11

114

Edward Grefenstette

@egrefen

6 years

Thanks to everyone who came to our #ICLR2018 poster on neural networks and logical entailment. Here's copy of the poster for those who missed it.

1

21

114

Edward Grefenstette

@egrefen

7 years

It’s not an ACL tutorial on vector representations of meaning if there isn’t at least one Ray Mooney quote.

4

18

116

Edward Grefenstette

@egrefen

4 years

More of this behaviour, and less of the other stuff, please.

1

22

116

Edward Grefenstette

@egrefen

5 years

Disappointing to see so little discussion happening on @iclr_conf 2020 papers. What's the point of a rebuttal phase if reviewers dump their opinion and have no intention of changing it or discussing it?

6

10

115

Edward Grefenstette

@egrefen

1 year

There should be a term like “dunning-kruger by proxy” to reference people who somehow still think Elon Musk is smart despite everything that’s happened in the last few months.

26

3

112

Edward Grefenstette

@egrefen

5 years

Not expecting a Turing award off the back of this one, but here's a neat little study—done with @DeepMindAI colleagues—of the effect of naive ensembling *during* adversarial training on the robustness of neural networks to adversarial examples. Enjoy.

Strength in Numbers: Trading-off Robustness and Computation via...

While deep learning has led to remarkable results on a number of challenging problems, researchers have discovered a vulnerability of neural networks in adversarial settings, where small but...

arxiv.org

1

18

115

Edward Grefenstette

@egrefen

4 years

Love this. Take a look if you're doing a PhD in NLP and thinking about what to work on next. Take a look if you think "BeRt WiLl SoLvE lAnGuAgE lol!11" and need someone to point you to actual research topics.

(((ل()(ل() 'yoav))))👾

@yoavgo

4 years

so, here are a bunch of stuff i find interesting. no particular order. and definitely not comprehenssive. - creative ways to apply massive LMs. Sure we can fine-tune them with extra supervision. What else can we do with them?

3

33

181

1

13

113

Edward Grefenstette

@egrefen

1 year

@idavidrein @NathanpmYoung True. Although the absolute pinnacle of mathematical beauty is the square packing singularity: the optimal way of packing a square in a square.

4

1

108

Edward Grefenstette

@egrefen

4 years

Holy bad statistics, Batman!

Preethi Kasireddy

@iam_preethi

4 years

Hong Kong: - Life expectancy: 84 years - Meat consumption per capita: 153 kg (world's highest) India: - Life expectancy: 68 years - Meat consumption per capita: 4 kg (world's lowest)

2K

3K

11K

5

3

111

Edward Grefenstette

@egrefen

3 years

Awesome result: @facebookai (FAIR), in collaboration with @ucl_nlp , takes the top (and often second as well) place in *every* competition in EfficientQA. Great job, everyone!

Colin Raffel

@colinraffel

3 years

Mind-boggling results on the final EfficientQA leaderboard: The best system beat the REALM baseline by almost 20 points, and a 30 megabyte model got > 25% accuracy! Looking forward to hearing more about these systems at NeurIPS.

0

31

222

2

13

109

Edward Grefenstette

@egrefen

3 years

After giving it a try for a few weeks, I’ve deleted clubhouse. One of the main perks of working in tech in London is I *don’t* have to deal with inane Silicon Valley navel-gazing on the daily, so I have no idea why I thought it was a good idea to invite it into my living room 🤮

3

4

107

Edward Grefenstette

@egrefen

3 years

Want to work with me at FAIR London? I'm looking to take up to 2 interns in 2022 (flexible start) for 4 months. You must be actively pursuing a PhD, and ideally in the last 2 years. Email me your research interests with [Internship 2022] in the subject line, at egrefen @fb .com.

4

12

107

Edward Grefenstette

@egrefen

4 years

Haaaave you met the AI community?

Susan David, Ph.D.

@SusanDavid_PhD

4 years

In general, experts—or people who are highly regarded in any field—are often hooked on their own self-importance. But sometimes status or accomplishment in one realm has no relevance in another.

9

19

85

4

104

Edward Grefenstette

@egrefen

4 years

Things I planned on doing post @iclr_conf deadline: * catch up on life admin * sleep more * clean my office Things I actually did: * play Among Us

2

106

Edward Grefenstette

@egrefen

6 years

#nips2018 submissions be like...

0

8

102

Edward Grefenstette

@egrefen

2 years

Toy Semantic Search (1/4) We frequently want to search through documents to find the answer to a question. Examples include technical documentation, like docs for a programming language, or a company wiki to find out how to set up our laptops ↓

2

11

100

Edward Grefenstette

@egrefen

6 years

The datasets (and code to generate more data) for our ICLR paper “Can Neural Networks Understand Logical Entailment?” is now online. Enjoy (and test your sequence models on it).

GitHub - google-deepmind/logical-entailment-dataset: This repository contains an entailment dataset...

This repository contains an entailment dataset for propositional logic, and code for generating that dataset. It also contains code for parsing the dataset in python. - google-deepmind/logical-enta...

github.com

2

24

105