Samuel Albanie @SamuelAlbanie Twitter profile

Last Seen Profiles

@stw_pdg

@sylvieaprile

@KasihLudah

@Ruby

@showthekidspod

@YouTubeTV

@jandakembangstw

@ToonkinsNFT

@alivyavfree

@honeybeaa_

@ddonws

@UwUKero

@std_Liege

@JerryBeach73

@Kidney_Int

@HnaC4cTLgm49044

@Wata_Ridley

@nagafujiriku

@Madeline_Raves

@_ScottJonesy

@joshsmiler

@niinomiwak82684

@DinEppin

@T4ko0_

@themb0w

@yamayama_nae

@uiuc_nlp

@tim_j_stowe

@Kishtar

@JURISTnews

@0Gupu

@ShamaJunejo

@MiNimmm2006

@bamr69

@MetroRodUK

@whkgseo

Samuel Albanie

@SamuelAlbanie

2 years

How can we reduce the computational cost of training neural networks? Bo Zhao, Hakan Bilen and collaborators have produced a creative body of work developing a technique known as "dataset condensation". 1/7

6

95

471

Samuel Albanie

@SamuelAlbanie

1 year

The latest competitor to GPT-4: A biological large language model Available via a text completion API

Samuel-API

A biological competitor to GPT-4.

samuel-api.com

24

42

442

Samuel Albanie

@SamuelAlbanie

2 years

Just how striking are the recent language model results with Flan-PaLM? Here's a plot. Across 57 tasks on mathematics, US history, computer science etc., Flan-PaLM surpasses **both** the June 2023 and June 2024 SotA forecasts from this summer by competitive forecasters. 1/3

9

60

423

Samuel Albanie

@SamuelAlbanie

2 years

Finetuning language models on instructions increasingly seems a compute-efficient way to gain performance. Recent work from @hwchung27 , @_jasonwei , @JeffDean , @quocleix & others scales this up to new regimes. TLDR: Even for big models (540B params), gains are substantial. 1/12

1

49

263

Samuel Albanie

@SamuelAlbanie

2 years

There has been an explosion of NLP research in prompting techniques for communicating tasks to language models. But writing and sharing good prompts is awkward. PromptSource is a tool that was developed as part of @BigscienceW to tackle this challenge. 🧵1/11

4

35

243

Samuel Albanie

@SamuelAlbanie

3 months

A small personal update: - Excited to join Google DeepMind 🚀 - Grateful for the wonderful humans I've had the pleasure of working with on my journey so far at @Cambridge_Eng and @Oxford_VGG ❤️

21

5

237

Samuel Albanie

@SamuelAlbanie

4 months

TLDR: RLHF - improves generalisation, but - reduces diversity relative to supervised fine-tuning (SFT). Interesting work from @_robertkirk et al.

5

40

230

Samuel Albanie

@SamuelAlbanie

2 years

I have a YouTube channel that aims to provide slow, technical explanations of some developments in Machine Learning:

3

27

201

Samuel Albanie

@SamuelAlbanie

1 year

1/ 🚀🔬 Introducing our groundbreaking research paper: "Large Language Models are Few-shot Publication Scoopers" We've discovered the secret to achieving personal glory and a lifetime supply of Cheerios Joint work with @LiliMomeni and J. F. Henriques Appears @sigbovik today

4

28

192

Samuel Albanie

@SamuelAlbanie

1 year

BLOOM. A large language model trained by researchers from around the world by @BigscienceW . How did they do it? Why did they do it? Let's dive in. 1/21 🧵

7

43

188

Samuel Albanie

@SamuelAlbanie

1 year

GPT4Geo - studies GPT-4's geographic knowledge & reasoning - suggests GPT-4 can plan complex journeys, describe the global semiconductor supply chain and roughly reconstruct the Hong Kong MTR map With @J_Roberts_1 , Timo, Sowmen, @kaihan_vis

5

40

143

Samuel Albanie

@SamuelAlbanie

4 months

TLDR: An LLM agent based on GPT-4 can autonomously hack websites Work by R. Fang, @daniel_d_kang and others at @IllinoisCS Paper:

3

29

137

Samuel Albanie

@SamuelAlbanie

3 months

TLDR: Human feedback is key to LLMs, but it is not a panacea - it under-values some aspects (e.g. factuality) - is biased (e.g. assertive text is judged more factual) A nice example of the empirical science of annotation By @tomhosking Blunsom @max_nlp

0

28

134

Samuel Albanie

@SamuelAlbanie

1 year

LLMs as Tool Makers - uses LLMs to create their own reusable tools (Python functions) for problem-solving - allows a lighter model to use tools built by a heavier model relatively cheaply By @tianle_cai , X. Wang, @tengyuma , @xinyun_chen_ , @denny_zhou

1

25

123

Samuel Albanie

@SamuelAlbanie

4 months

TLDR: Unsupervised knowledge discovery in LLMs is hard Intriguing theoretical and empirical results from @seb_far et al. Paper: And for those who enjoy video summaries:

1

29

125

Samuel Albanie

@SamuelAlbanie

1 year

VisionLLM - Key idea: treat images as a foreign language for a generalist LLM decoder - Strong performance on object detection (60 mAP on COCO) - paper: by W. Wang, @PKUCXK et al.

0

21

100

Samuel Albanie

@SamuelAlbanie

6 months

What is Mamba? Here's a short video about the architecture Proposed by @_albertgu and @tri_dao

Mamba - a replacement for Transformers?

Mamba is a new neural network architecture proposed by Albert Gu and Tri Dao.Timestamps:00:00 - Mamba - a replacement for Transformers?00:19 - The Long Range...

www.youtube.com

6

12

88

Samuel Albanie

@SamuelAlbanie

4 months

TLDR: An LLM can improve by providing its own rewards during training Work by @WeizheY et al. Paper:

1

12

82

Samuel Albanie

@SamuelAlbanie

4 months

TLDR: Emergent capabilities appear due to the choice of - nonlinear, or - discontinuous metrics Work by @RylanSchaeffer et al. (Outstanding paper, NeurIPS '23) Paper: Also recommended - some nuances by @boazbaraktcs :

0

10

80

Samuel Albanie

@SamuelAlbanie

4 months

*TLDR* Major gains in pretraining efficiency/quality by - filtering data with an LLM judge and - asking the judge to only keep the "informative" stuff Work by @noveens97 et al. Paper:

2

9

78

Samuel Albanie

@SamuelAlbanie

2 years

Semantic segmentation is valuable, but it remains costly and painful to scale up. ReCo (NeurIPS 2022) aims to tackle this problem by using: - the retrieval abilities of CLIP - the co-segmentation abilities of vision transformers Here's how it works. 🧵1/9

2

14

77

Samuel Albanie

@SamuelAlbanie

4 months

TLDR: Using an LLM to rephrase text documents to be "in high quality English language as in sentences on Wikipedia" can achieve ~3x faster LLM pretraining Work by @pratyushmaini et al. Paper:

1

16

76

Samuel Albanie

@SamuelAlbanie

3 months

Today I'll give my final lecture on data structures & algorithms @Cambridge_Eng @Cambridge_Uni 😢 But, for those keen to study: - re-recorded videos - slides - and code are all available online: (the fun Red-Black Tree vis. is based on work by @lsbardel )

1

11

73

Samuel Albanie

@SamuelAlbanie

4 months

TLDR: Scaling does not seem to improve visual data-type understanding Joint work with @vishaal_urao , @maxburg , @MatthiasBethge (ICLR 2024) Paper:

1

15

71

Samuel Albanie

@SamuelAlbanie

1 year

FactScore - evaluating factual precision of LM outputs is costly - LMs (+ retrieval) can help - Find big diffs in factual prec. of LMs (e.g. GPT-4 vs StableLM) By @sewon__min , @kalpeshk2011 , @ml_perception @LukeZettlemoyer , @HannaHajishirzi et al.

0

9

66

Samuel Albanie

@SamuelAlbanie

1 year

GPT-4 Out-performs RL Algorithms by Studying Papers and Reasoning - RL agents have low sample efficiency on open-ended games - GPT-4 works better by: (i) reading instructions (ii) selecting next action By @yw_yuewu , @shrimai_ @rsalakhu , @ybisk et al.

0

17

64

Samuel Albanie

@SamuelAlbanie

1 year

Using ChatGPT to explore a Computer Vision/ML research project - a mini-collaboration. Investigator: How can SENet ideas improve ViT? ChatGPT: Plug the SENet module into the ViT architecture. OK... reasonable enough. So down the rabbit hole we go... 1/9

2

7

61

Samuel Albanie

@SamuelAlbanie

1 year

What is GPT-4? And just how good is it at exams? Let's take a look at some of what we know. 🧵 1/22

1

11

61

Samuel Albanie

@SamuelAlbanie

1 year

AlignScore Motivation: checking factual consistency is hard work Key idea: train general text alignment function, then use as building block to assess factual consistency By @yzha_zha , @ZhitingHu et al.

0

13

54

Samuel Albanie

@SamuelAlbanie

2 years

Multitask prompted finetuning (aka instruction finetuning) can boost language model performance. But how can we make progress beyond English (esp. on languages with limited finetuning data)? Work by @Muennighoff & others in @BigscienceW studies this in detail. 1/17 🧵

1

14

54

Samuel Albanie

@SamuelAlbanie

4 months

TLDR: MMMU is a massive, difficult image & text benchmark for LLMs Work by @xiangyue96 and collaborators. Paper:

3

9

52

Samuel Albanie

@SamuelAlbanie

1 year

Gorilla - 7B LLaMA finetuned on (instruction, API call) pairs - Achieves solid performance vs strong (untuned) commercial models at writing API calls Paper: By @shishirpatil_ , @tianjun_zhang , @xinw_ai , & @profjoeyg 1/2

1

7

50

Samuel Albanie

@SamuelAlbanie

1 year

The False Promise of Imitating Proprietary LLMs - imitation improves "style, persona & instruction adherence of open-source LMs" - but "falls short... on more challenging aces such as factuality, coding & problem solving" Paper: By @arnavg_ ,

0

8

49

Samuel Albanie

@SamuelAlbanie

1 year

QLoRA - can finetune 65B model on a 48GB GPU & retain 16-bit finetuning performance Key ideas: (1) 4-bit NormalFloat data type (2) Double Quantization (3) Paged Optimizers Paper: By @Tim_Dettmers , @ArtidoroPagnoni , @universeinanegg , @LukeZettlemoyer

1

9

49

Samuel Albanie

@SamuelAlbanie

1 year

Let’s Verify Step by Step - finds process-supervision outperforms outcome-supervision on maths problems - potential example of a "negative alignment tax" (good for alignment + capabilities) By @HunterLightman et al.

1

11

46

Samuel Albanie

@SamuelAlbanie

8 months

Do you like morning jogs? Do you enjoy speculating about the future of AI? Are you attending @ICCVConference ? If you answered yes to all three, meet at 8 am Wed, Thur, Fri @ OKKO Hotels Porte De Versailles entrance. All welcome. 1/2

5

4

45

Samuel Albanie

@SamuelAlbanie

2 years

Flan-PaLM was part of a study on scaling up instruction finetuning by @hwchung27 , @_jasonwei & others at @Google Gains from: - bigger models... - more tasks (but diminishing returns) - chain-of-thought finetuning - chain-of-thought prompting with self-consistency 2/3

1

2

44

Samuel Albanie

@SamuelAlbanie

11 months

Are Multimodal LLMs the future for Computer Vision? Kosmos-2 is a new model from Microsoft Research It has quite a broad range of tricks up its sleeve (including grounding) An overview of the work 👇

What is KOSMOS-2?

"KOSMOS-2: Grounding Multimodal Large Language Models to the World" is a new preprint from Microsoft research that illustrates multimodal grounding abilities...

www.youtube.com

1

11

43

Samuel Albanie

@SamuelAlbanie

2 years

Links: Flan-PaLM: MMLU benchmark of 57 tasks (explains human baselines): Forecasts: (relevant forecasts updated Aug 15th 2022) Useful context for forecasts by @JacobSteinhardt : 3/3

Updates and Lessons from AI Forecasting

Earlier this year, my research group commissioned 6 questions [https://prod.hypermind.com/ngdp/en/showcase2/showcase.html?sc=JSAI] for professional forecasters to predict about AI. Broadly speaking,...

bounded-regret.ghost.io

2

4

41

Samuel Albanie

@SamuelAlbanie

1 year

Scaling Data-Constrained Language Models - a new data-constrained scaling law - generalizes Chinchilla scaling to repeated data regime By @Muennighoff @srush_nlp @boazbaraktcs @Fluke_Ellington @olapiktus @Nouamanetazi @TurkuNLP @Thom_Wolf @colinraffel

0

11

40

Samuel Albanie

@SamuelAlbanie

1 year

Orca: Progressive Learning from Complex Explanation Traces of GPT-4 - goes big on imitation learning (includes 1M GPT-4 responses) - outperforms Vicuna-13B "by more than 100% in complex zero-shot reasoning benchmarks" By @subho_mpi et al.

1

10

40

Samuel Albanie

@SamuelAlbanie

11 months

Does CoT really reveal the reasoning process of an LLM? Perhaps.. But then again, perhaps not New work from Anthropic studies this question empirically: "Measuring Faithfulness in Chain-of-Thought Reasoning" by T. Lanham et al. An overview👇

Is Chain of Thought faithful?

Do LLMs really "show their work" when they perform chain of thought reasoning? "Measuring Faithfulness in Chain-of-Thought Reasoning" is a new paper from An...

www.youtube.com

2

12

40

Samuel Albanie

@SamuelAlbanie

1 year

Hallucination snowballing LMs can over-commit to early mistakes, leading to later mistakes they would not otherwise make Hypothesis: LMs strive for consistency with earlier hallucinations By @zhang_muru @OfirPress @lambdaviking @alisawuffles @nlpnoah

2

15

40

Samuel Albanie

@SamuelAlbanie

1 year

🧪🔬 1/ Can GPT-4 assist with scientific hypothesis generation? 🤖 "Conversations with GPT-4" is a preliminary exploration of this topic

3

8

37

Samuel Albanie

@SamuelAlbanie

5 years

What’s a simple way to improve video retrieval? Use more modalities & deal with noise! Excited to announce latest work with Yang Liu, @NagraniArsha & Andrew Zisserman. SoTA on five video benchmarks. Paper: Code/Models: #bmvc2019

0

11

33

Samuel Albanie

@SamuelAlbanie

1 year

Struggling to keep up with recent AI developments? Try **AI News with Samuel Albanie** A weekly dose of research papers, tools & resources The #1 AI news show with Samuel Albanie, as voted by me

AI News

Research, tools, news relating to AI

www.youtube.com

1

5

34

Samuel Albanie

@SamuelAlbanie

2 years

🤗 Datasets: A community library for natural language processing (and other fields too) From @qlhoest and a wide range of contributors across @huggingface and beyond

🤗 Datasets: A community library for natural language processing

A fast description of "🤗 Datasets: A community library for natural language processing" by Q. Lhoest et al., published as an EMNLP demo in 2021.The library ...

www.youtube.com

1

8

33

Samuel Albanie

@SamuelAlbanie

4 months

TLDR: - Widely used benchmarks like HumanEval lack test coverage - EvalPlus synthesises new test-cases to cover gaps - Consequence: HumanEval ranking changes for some models Work by @JiaweiLiu_ et al. Paper:

0

5

32

Samuel Albanie

@SamuelAlbanie

4 years

This is an amazing piece of work on continual learning from @xu__ji and collaborators @Oxford_VGG , using a single unified model to synthesize artificial replay samples on the fly during training. The benefits of experience replay, but without a buffer

0

7

31

Samuel Albanie

@SamuelAlbanie

1 year

Do LMs Know When They're Hallucinating References? - finds many fabrications can be identified using only black-box queries. - most useful on more powerful models like GPT-4 By A. Agrawal, @LesterMackey , @adamfungi

0

3

31

Samuel Albanie

@SamuelAlbanie

4 months

Here's one reason I think longer context windows (e.g. 10M tokens for Gemini 1.5) are a big deal for software dev: the whole codebase can be in context The original HN comment responds to the question "How are some people exceptionally productive?":

1

7

31

Samuel Albanie

@SamuelAlbanie

1 year

PaLM-2 vs other LLMs - Comparison made in Chatbot arena by @lmsysorg - Major gap in Elo Rating (GPT-4 vs PaLM-2) - Some caveats in the thread below 1/2

3

4

30

Samuel Albanie

@SamuelAlbanie

1 year

Exploring the State of Instruction Tuning on Open Resources - Compares instruction resources - Finds base model is key By @yizhongwyz @hamishivi @pdasigi @jmhessel @tusharkhot @khyathi_chandu @davidjwadden @nlpnoah @i_beltagy @HannaHajishirzi

2

3

29

Samuel Albanie

@SamuelAlbanie

1 year

In 1960, Norbert Wiener made an observation. - if we make a machine that's hard to stop - it would be good to make sure it does what we want 1/3 🧵

2

4

27

Samuel Albanie

@SamuelAlbanie

1 year

Another week, another full bucket of AI news. Some highlights... 🧵1/25

3

5

22

Samuel Albanie

@SamuelAlbanie

4 months

TLDR: A new family of lightweight LLMs (2B and 7B params) - 7B model is trained 6T tokens on 4096 TPUv5e - weights available for commercial use Work from Google DeepMind Paper:

0

6

25

Samuel Albanie

@SamuelAlbanie

11 months

Can you prove which data was used to train an AI? New techniques from @damichoi95 , @yonashav and @DavidDuvenaud suggest the answer may be "yes" An overview of the work 👇

Can we verify training data?

How can you prove that a training set was used to train a neural network?That's the question considered by Choi, Shavit and Duvenaud in the work "Tools for V...

www.youtube.com

0

4

23

Samuel Albanie

@SamuelAlbanie

1 year

ToolkenGPT Key idea: represent tools as tokens for LLMs Strong performance vs in-context learning on question answering Paper: by @Ber18791531 , @ZhitingHu and others

0

12

24

Samuel Albanie

@SamuelAlbanie

1 year

**Seeking feedback** - I'd like to improve my AI news YouTube videos - I'd greatly appreciate any constructive criticism - the feedback is anonymous Give feedback here: The news videos can be found here:

Samuel Albanie - AI Research videos (anonymous feedback)

I make YouTube videos about AI News (https://www.youtube.com/playlist?list=PL9t0xVFP90GC17cY2_bOZRphNQhscmYf9) and AI research in general: https://www.youtube.com/channel/UCMtxPy2z1qzwvP7wiarkoaw I'm...

docs.google.com

2

4

24

Samuel Albanie

@SamuelAlbanie

4 years

We're excited to announce that the Video Pentathlon is now live! Test out your video retrieval skills on five challenging benchmarks: MSRVTT, MSVD, YouCook2, ActivityNet and DiDeMo. More here: Baselines and features provided! #CVPR2020 #video

1

9

22

Samuel Albanie

@SamuelAlbanie

1 year

The legend.

0

22

Samuel Albanie

@SamuelAlbanie

4 months

*TLDR:* Can you get Chain-of-Thought without prompting? Yes. How? By altering the LLM decoding process... Work by X. Wang and @denny_zhou Paper:

0

1

21

Samuel Albanie

@SamuelAlbanie

2 years

Papers/code links for Dataset Condensation: - Gradient Matching (ICLR '21) - DSA (ICML '21) - CAFE (CVPR '22) - Distribution Matching (WACV '23) Code: 3/7

GitHub - VICO-UoE/DatasetCondensation: Dataset Condensation (ICLR21 and ICML21)

Dataset Condensation (ICLR21 and ICML21). Contribute to VICO-UoE/DatasetCondensation development by creating an account on GitHub.

github.com

1

7

22

Samuel Albanie

@SamuelAlbanie

1 year

Did you wake up today with a strange desire to learn more about B-trees and memory hierarchies? If so, this may be the video for you. 1/2

B-trees: Samuel's tutorial

Samuel's tutorial on B-trees (memory hierarchy, disk accesses, search, insertion and deletion).Timestamps:00:00 - B-Trees: Samuel's Guide01:57 - Precursor: M...

www.youtube.com

1

4

22

Samuel Albanie

@SamuelAlbanie

1 year

A lot has been happening in AI over the last few weeks Here are a few highlights 1/15 🧵

1

9

21

Samuel Albanie

@SamuelAlbanie

1 year

Nothing to see here...

1

2

21

Samuel Albanie

@SamuelAlbanie

2 years

Related in this space includes: - Dataset distillation @TongzhouWang et al. (arxiv '18) - Label distillation @OBohdal et al. (NeuriPS workshop '20) - KIP by @IAmTimNguyen et al. (ICLR '21) 4/7

Dataset Meta-Learning from Kernel Ridge-Regression

One of the most fundamental aspects of any machine learning algorithm is the training data used by the algorithm. We introduce the novel concept of $ε$-approximation of datasets, obtaining...

arxiv.org

1

3

21

Samuel Albanie

@SamuelAlbanie

2 years

For those who prefer a narrated version: 2/12

Scaling Instruction-Finetuned Language Models (Flan-PaLM)

Flan-PaLM was introduced in the work "Scaling Instruction-Finetuned Language Models" by Chung et al. which appeared on arxiv in October 2022.This video descr...

www.youtube.com

2

20

Samuel Albanie

@SamuelAlbanie

4 months

TLDR: Getting LLMs to debate options helps humans choose the right answer Recent work by @AkbirKhan et al. Paper: It's interesting to read some of the debates (nicely formatted here: )

0

2

20

Samuel Albanie

@SamuelAlbanie

1 year

Getting ViT in Shape - Compute-optimal shapes allow for smaller models w. same acc. & same compute Rules of thumb: - Scale MLP dim. faster than depth - Scale depth faster than width by @ibomohsin , @XiaohuaZhai , @__kolesnikov__ , @giffmana

1

0

18

Samuel Albanie

@SamuelAlbanie

1 year

Tired of ChatGPT hallucinations? Filtir fact-checks generated text And rewrites it for you Available to try:

3

6

19

Samuel Albanie

@SamuelAlbanie

4 years

You are warmly invited to join us at #ECCV for our poster at 14:00 today (UK time) or midnight... “BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues" with @gulvarol , @LiliMomeni , T. Afouras, J.S Chung, N. Fox & A. Zisserman

[ECCV'20] BSL-1K: Scaling up co-articulated sign language recognition...

This video presentation describes the paper:Samuel Albanie*, Gül Varol*, Liliane Momeni, Triantafyllos Afouras, Joon Son Chung, Neil Fox and Andrew Zisserman...

www.youtube.com

2

6

19

Samuel Albanie

@SamuelAlbanie

4 years

Thanks to everyone who attended the CVPR workshop "The End-of-End-to-End: A Video Understanding Pentathlon"! Links to papers and slides: Video: Congratulations to the challenge winners and thank you to all our presenters!

0

8

18

Samuel Albanie

@SamuelAlbanie

1 year

Struggling to pick your next book for the Christmas break? Struggle no more! is here to help Built with @vladbogo and

1

3

18

Samuel Albanie

@SamuelAlbanie

2 years

Key idea: compress a large dataset into a small set of synthetic images that can train networks to the same accuracy as the original dataset. Was a pleasure to examine Bo's thesis on this topic work with @driainmurray . 2/7

1

3

18

Samuel Albanie

@SamuelAlbanie

5 years

Thanks to everyone who attended Neural Architects and made for such a wonderful workshop! Particularly Barret Zoph, Iasonas Kokkinos, Alan Yuille, Sara Sabour and Ross Girshick for their fantastic talks & @Momenta_AI for support. Slides (soon) at #iccv2019

0

4

18

Samuel Albanie

@SamuelAlbanie

3 months

Beartype has long been one of my favourite open-source libraries Because: - it's a great library - thanks to maintainer Cecil Curry (leycec) every GitHub issue thread is a work of literature Some classics

1

16

Samuel Albanie

@SamuelAlbanie

1 year

Fortunate to have @IgnasBud as a colleague @Cambridge_Eng @Cambridge_Uni . Here's why🧵 1/3

2

1

16

Samuel Albanie

@SamuelAlbanie

1 year

PaLI-X: On Scaling up a Multilingual Vision and Language Model - shows that scaling up both V&L brings gains - with a massive vision encoder (22B), you can co-train for image classification and OCR By X. Chen, @neilhoulsby , @RSoricut & others

0

3

16

Samuel Albanie

@SamuelAlbanie

4 months

A summary of Gemini 1.5 Pro in 11 and a half minutes:

Gemini 1.5 Pro has a massive context window

Google DeepMind recently released a report on Gemini 1.5 Pro.Perhaps the most significant advance over Gemini 1.0 is the context window, which can now span u...

www.youtube.com

0

2

16

Samuel Albanie

@SamuelAlbanie

1 year

"According to..." Prompting - Uses prompts like "According to Wikipedia..." to encourage source quotation - leads to more grounded outputs By @orionweller @ruyimarone @Nathaniel_Weir @lawrie_dawn @DanielKhashabi @ben_vandurme @jhuclsp

0

5

15

Samuel Albanie

@SamuelAlbanie

2 years

- Dataset Distillation with Infinitely Wide Convolutional Networks by @IAmTimNguyen et al. (NeurIPS '21) - Dataset Distillation by Matching Training Trajectories by @GCazenavette et al. (CVPR '22) 6/7

Dataset Distillation by Matching Training Trajectories

Dataset distillation is the task of synthesizing a small dataset such that a model trained on the synthetic set will match the test accuracy of the model trained on the full dataset. In this...

arxiv.org

1

15

Samuel Albanie

@SamuelAlbanie

4 months

TLDR: If we - train a powerful AI, and - use current behavioural training approaches things may go badly An argument outlined by @peterbarnettnz and Jeremy Gillen Paper:

0

1

15

Samuel Albanie

@SamuelAlbanie

4 months

Initial thoughts: Gemini Ultra is clearly an advance on Gemini Pro. The Google Docs integration now seems far more useful (fewer hallucinations). It's also quite fast. Of course, it still has some way to go with algebra...

1

4

15

Samuel Albanie

@SamuelAlbanie

2 years

I'll be at NeurIPS this week. DM if you'd like to meet up to discuss any of the following: - AI-accelerated science - foundation models - compute budgets - mince pies

0

14

Samuel Albanie

@SamuelAlbanie

2 years

Update: thanks for the useful discussion @metastable_1 @BlancheMinerva @MichaelTrazzi & others The chart benefits from extra forecasts (noted by @metastable_1 ) which predicted higher scores Updated chart: (explanation in video linked in that thread)

Samuel Albanie

@SamuelAlbanie

2 years

Flan-PaLM 540B (PaLM 540B finetuned on instructions) makes major progress on MMLU. Note: my previous graph () lacked some of the available SotA forecasts - that's updated below. Even with the update, the numbers remain impressive. 3/12

1

14

0

1

14

Samuel Albanie

@SamuelAlbanie

1 year

Need help fact-checking ChatGPT? Filtir is available in the ChatGPT plugin store! Feedback v. welcome Note: this is an early version, so you still need to be careful to double-check things yourself

Fact-check ChatGPT - Filtir ChatGPT Plugin Demo

A demo for the Filtir ChatGPT plugin for fact-checking ChatGPT outputs.Discord server for filtir: https://discord.gg/FV97NByG2bSound credit: sampled from loo...

www.youtube.com

0

6

13

Samuel Albanie

@SamuelAlbanie

4 years

Do you love videos? Do you love natural language? Why not express those passions through a submission to our workshop on video retrieval from natural language queries! Find out more about the workshop at #CVPR2020 @CVPR2020 #video #retrieval #workshop 📼

0

11

12

Samuel Albanie

@SamuelAlbanie

1 year

Radix sort. A glorious sorting algorithm. Used at least as early as the 1890s by Herman Hollerith and his punched card machines. Here's a video on how it works. 1/2

Radix sort: Samuel's tutorial

Samuel's tutorial for radix sort algorithms (history, LSD and MSD variants, runtime complexity and Python implementation).Timestamps:00:00 - What is the best...

www.youtube.com

1

0

14

Samuel Albanie

@SamuelAlbanie

6 years

"Capturing the Geometry of Object Categories from Video Supervision" - a lovely piece of work in TPAMI by David Novotny, @dlarlus and Andrea Vedaldi:

0

8

14

Samuel Albanie

@SamuelAlbanie

2 years

Flan-PaLM 540B (PaLM 540B finetuned on instructions) makes major progress on MMLU. Note: my previous graph () lacked some of the available SotA forecasts - that's updated below. Even with the update, the numbers remain impressive. 3/12

Samuel Albanie

@SamuelAlbanie

2 years

Just how striking are the recent language model results with Flan-PaLM? Here's a plot. Across 57 tasks on mathematics, US history, computer science etc., Flan-PaLM surpasses **both** the June 2023 and June 2024 SotA forecasts from this summer by competitive forecasters. 1/3

9

60

423

1

14

Samuel Albanie

@SamuelAlbanie

1 year

Much of this work is behind the scenes. It does not receive the glory of creative code releases, popular preprints and dramatic demos. And so, my dear Twitterverse, I am letting you know. He is a wonderful colleague. And a great educator. 3/3

2

0

13

Samuel Albanie

@SamuelAlbanie

1 year

@immazzystar describes the high leverage that GPT-4 gives individuals: - "The overnight surge in productivity is intoxicating" @robkhenderson explores implications of LLMs: - "people will rely on them to learn what is permissible to say in polite society" 25/25

2

12

Samuel Albanie

@SamuelAlbanie

1 year

@full_stack_dl LLM Bootcamp hosts a series of free lectures (available on YouTube) 21/25

1

13

Samuel Albanie

@SamuelAlbanie

1 year

Statement: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” Signed by: - Turing Award winners - AI researchers - Hassabis, Altman, Amodei - many more @ai_risks

Statement on AI Risk | CAIS

A statement jointly signed by a historic coalition of experts: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and...

www.safe.ai

0

3

13

Samuel Albanie

@SamuelAlbanie

1 year

Multiagent debate - use multiple LM instances to propose & debate over multiple rounds - improves reasoning & factual accuracy - complementary to chain-of-though etc. Paper: By @du_yilun , @ShuangL13799063 , @IMordatch et al.

0

2

12

Samuel Albanie

@SamuelAlbanie

1 year

Links: - slides: - references: - arxiv: - code and models: 21/21

bigscience/bloom · Hugging Face

huggingface.co

3

1

12

Samuel Albanie

@SamuelAlbanie

2 years

- Dataset Condensation with Contrastive Signals by S. Lee et al. (ICML '22) - Dataset Condensation via Efficient Synthetic-Data Parameterization by J-H Kim et al. (ICML '22) 7/7

Dataset Condensation via Efficient Synthetic-Data Parameterization

The great success of machine learning with massive amounts of data comes at a price of huge computation costs and storage for training and tuning. Recent studies on dataset condensation attempt to...

arxiv.org

0

1

12

Samuel Albanie

@SamuelAlbanie

6 years

"MapNet: An Allocentric Spatial Memory for Mapping Environments" - a lovely piece of work by Joao Henriques and Andrea Vedaldi

0

7

12

Samuel Albanie

@SamuelAlbanie

4 months

TLDR: Self-Discover prompting - works out a reasoning strategy for a given task - amortises the cost of that work across task instances - brings gains over Chain-of-Thought Work by @peizNLP et al. Paper:

0

1

12