kipply @kipperrii Twitter profile

Last Seen Profiles

@xxcm_

@KeithRkbrown

@ShowoutDeion

@itsisaiahjones

@BungkusTukang

@YaleMicroPath

@oliveai__

@shondradanielle

@LeParadis_Perdu

@Buzzavita

@InvincibleHQ

@Kingleeks

@LSE_UCU

@cliffpix

@YamiMidna

@katepinguu

@rhuckz

@bubble_butt70

@AmazingPhil

@ManCity

@Metaphor_o_Love

@davemakes

@chacaldefeu

@kaliii

@chun0lee

@CWDSellier

@fuji1915

@mulaahoneyy

@joyceyanzhang

@antony_nived

@No_Shep7

@TheBlitza

@tribalchiefjeff

@ViridianGames

@STARGATEZ

@PrincessUKmedia

kipply

@kipperrii

11 months

a good software engineer will often debug further up the stack to find a bug in third-party software rather than reaching for the first work around. a great software engineer will debug further up the stack until they realise the root bug is Society

37

263

2K

kipply

@kipperrii

6 months

i'm pretty anti-college, but doing iconic programming projects (connect 4, raytracer, operating systems, nand2tetris) early-career is a huge indicator of potential,,, a classically trained programmer

35

62

2K

kipply

@kipperrii

6 months

they killed a bunch of startups in broad daylight and invited their founders to watch

45

85

2K

kipply

@kipperrii

1 year

i summarized and compiled all of the literature i feel is relevant for catching up on the state of ai in the lm-flavoured space. everything links to directly to the pdf (not the arxiv home)~ it covers 22 models along with two dozen other techniques

24

151

1K

kipply

@kipperrii

1 year

i don't say things are a "cluster fuck", i say it's a "kubernetes situation". much classier

10

94

673

kipply

@kipperrii

5 months

your level at tech companies is actually how many frames you read off the trace before asking for help

8

24

512

kipply

@kipperrii

2 years

transformer inference performance is becoming increasingly important and there's not as much lore on it, so here is a lot of lore that i think fully models llm inference performance

6

65

490

kipply

@kipperrii

2 months

its sad watching founders with perfectly good companies get trainitis, an affliction which compels people to train their own models from scratch. the cause of failure isn't big co doing the startup, but the startup doing the big co. trainitis can happen to anyone 🥺

17

12

355

kipply

@kipperrii

4 months

I love and respect Tom Scott so much, he is an icon of integrity and truth-seeking. The consistency of the channel has been incredibly consistently high quality and I love that he told us he'd stop half a year ago. Big loss but everyone should end their projects with this much…

8

11

337

kipply

@kipperrii

8 months

golden handcuffs apply to status as well, i think its good for the soul and for personal success to maintain a consistent low-status practice, whatever that is for you

6

287

kipply

@kipperrii

2 years

oh my fucking god im going to cry

11

7

278

kipply

@kipperrii

1 year

I've been at Anthropic for over six months now and I'm happy to recommend it to a friend! We're hiring for software engineers to work on our research, product and infrastructure, and particularly you can come work with me on a newly formed✨Tokens Team!

Home \ Anthropic

Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.

www.anthropic.com

9

19

268

kipply

@kipperrii

2 years

ppl who suspect motivated reasoning to work on ai alignment do not understand how many things i'd rather be doing than rotating tensors in python

7

14

267

kipply

@kipperrii

2 years

asked some guy at a party last night at three am to tell me about the last five french presidents and he just did it?? is this what educated people are like zomg

11

1

261

kipply

@kipperrii

2 years

pyrogramming languages imagined by dalle

9

6

221

kipply

@kipperrii

6 months

insane how a high-status person can be nice to me once and i'll worship them forever

10

4

218

kipply

@kipperrii

3 months

when i've asked someone more senior to debug something and they magically just do it the answer to "how was i supposed to know that" is usually tacit knowledge providing a strong intuition of where the bug is coming from

6

7

198

kipply

@kipperrii

5 months

this man once told me he doesn't like vscode because the characters take too long to render and we're now making him do machine learning... please help him 🥹

Tristan Hume

@trishume

5 months

I'm hiring for my performance optimization team at Anthropic! Join our excellent team doing kernels, distributed parallelization, and architecture co-design for GPUs, TPUs and Trainium. No problem if you've only done CPUs before! 🧵 More about us:

8

45

566

7

2

191

kipply

@kipperrii

2 years

im unemployed, who wants to cry on the beach with me about startups

17

4

190

kipply

@kipperrii

1 year

anyone who wants to go after openai for not open sourcing can take it up with me. i, for one, applaud them for overcoming the unjust forces of nominative determinism

8

6

192

kipply

@kipperrii

2 months

drop the biases. just weights. it's cleaner

6

3

183

kipply

@kipperrii

4 months

i don't know how anyone survives reading fiction. barely have memories from my early teens because i was just reading fiction. all my idle thoughts are about book. i lose the real world so easily

16

4

172

kipply

@kipperrii

10 months

i can't believe the new york times has photographed bundle TWICE and still bundle has not been in the new york times

3

1

172

kipply

@kipperrii

5 months

I saw the best minds of my generation destroyed by synthetic data, starving hysterical naked, dragging themselves through the eldritch tokens at dawn looking for an exquisite batch

Sasha Rush (ICLR)

@srush_nlp

5 months

Lot of pitches this week for "perpetual data machines". Either laundering self-generated data or attributing prescience to reward models. Just want to caution that is a common trap smart people fall for.

13

20

312

5

8

165

kipply

@kipperrii

1 year

i am a logo dj, that is to say, an alchemist. i don't define branding, branding defines me. MERGE!

12

5

155

kipply

@kipperrii

2 months

we have a mathematician who now does ml research, but last week we caught him simulating matmuls entirely in his head for bit per bit accurate results. is this normal behaviour or do we need to give him more proofs?

2

3

147

kipply

@kipperrii

2 months

gpt4: gets most of mmlu correct claude: gets most of mmlu correct gemini: gets most of mmlu correct mmlu: gets most of mmlu correct

3

4

146

kipply

@kipperrii

2 months

this is literally my favourite thing im obsessed (2403.09629)

kipply

@kipperrii

11 months

no one talks about how the tree of thought paper just throws in this inspirational quote???

4

2

83

6

147

kipply

@kipperrii

1 month

the vc money pouring into ai is an annoying bubble sure, but there's also the bubble of people who don't realise that the world is about to change faster than they've ever experienced?

7

2

130

kipply

@kipperrii

1 month

usually figuring out if a paper is credible takes reading through it quite thoroughly, even though my strong prior is that papers are bad sometimes, title is all you need! i know exactly which paper i'd trust more.

4

1

129

kipply

@kipperrii

2 years

matrix multiplication monday hyperparameter tuning tuesday weight watch wednesday chain of thought thursday finetuning friday

5

11

127

kipply

@kipperrii

2 years

who called it knn and not nnn

10

5

124

kipply

@kipperrii

2 years

how good a pun is

4

13

123

kipply

@kipperrii

2 years

is it just me or are millenials aging faster than time is passing?

10

3

121

kipply

@kipperrii

6 months

this is probably the thought i should've had when i wrote . i lacked foundation and technique as an uneducated webdev, ray tracing actually helped so much

3

0

120

kipply

@kipperrii

8 months

continuous colour scheme socks 🥰 gonna do cool and plasma next

7

2

120

kipply

@kipperrii

7 months

25k fund so that sf can have whimsy, anything goes

6

12

119

kipply

@kipperrii

2 months

all language models will have lower loss on code than natural language because code has a bunch of boring tokens and so despite this loss difference, haiku will be qualitatively worse at code than text. the notable part of this plot is that text flattened out and code hasn't…

5

2

121

kipply

@kipperrii

4 months

its only been three years since i stopped working on compilers and i've forgotten a concerning amount. annoyingly i've retained opinions and not facts, and though i trust my past self's love for the ocaml garbage collector i do wish i kept the facts instead

5

1

117

kipply

@kipperrii

9 months

what the bourgeoisie therefore produces, above all, are its own grave-diggers. its fall and the victory of the proletariat are equally inevitable

11

13

112

kipply

@kipperrii

1 year

the things we sacrifice for latency 🥺

6

1

116

kipply

@kipperrii

6 months

the most insane thing about the announcement is that they use angular for their frontend. almost as incredible as the pure vanilla js alphacode viz ()

11

2

113

kipply

@kipperrii

2 years

i think san francisco nerd "culture" is corrupt because there aren't any math cults

11

0

105

kipply

@kipperrii

1 year

ill be at neurips! please find me to talk about alignment, societal impacts, llm (performance), startups and compilers! also it's my birthday monday and i just took a flight to LA to take a 46 hour train to nola. will tweet train updates in thread💫

4

2

109

kipply

@kipperrii

2 years

throwing my first party in the bay area and boygirl am i going to have a hard time balancing the ratio

7

1

107

kipply

@kipperrii

2 years

@AdeptAILabs the only keystroke you should need here is F

2

1

103

kipply

@kipperrii

2 months

making bracelets for my friends 🥰

3

0

105

kipply

@kipperrii

1 year

did you know? reading a paper signed by the author doubles your learning rate! we are finally releasing our collection of signed machine learning papers. today we are launching where signed machine learning papers are being sold for charity 💖

7

8

102

kipply

@kipperrii

7 months

same energy

Daniel Feldman

@d_feldman

7 months

Asked DALL-E 3 for the ingredients to make a cake.. the more you look the better this gets

247

929

5K

13

6

105

kipply

@kipperrii

4 months

all of those single letter variable names are going to start paying off with the tokens they save 🥰

7

1

99

kipply

@kipperrii

2 years

this gremlin makes me so many friends at the airport

2

93

kipply

@kipperrii

2 years

i've always cared about my job and i feel strongly about companies in general ❤️ it was hard to leave cohere but i had a really great time getting to know these other co:mpanies too 💞

5

2

97

kipply

@kipperrii

1 year

i wanna be able to write custom css for notion like it's my tumblr page

10

2

95

kipply

@kipperrii

5 months

a lot of researchers who think theyre struggling because great work has already been done in the field are actually struggling because they don't know how to build off and work with the existing work which is a different skillset from finding a solution to a problem when there…

4

1

93

kipply

@kipperrii

2 years

the cohere api is generally available! it's available, generally. we have lovely generation models, but more excitingly embeddings models that don't fit into a nice screenshot, you'll have to experience them for yourself! (all non-cherrypicked samples)

6

9

95

kipply

@kipperrii

1 year

sometimes i really want to work at a less morally controversial company, like citadel securities 🥰

3

0

90

kipply

@kipperrii

3 years

im not a matchmaker, im a trader correcting dating market inefficiencies 🤑💸💎💍

8

0

92

kipply

@kipperrii

2 years

im hiring for my team of two at cohere where we're about to start building up entirely new inferencing infrastructure for llms with jax. this is a big greenfield undertaking and we're looking for some lovely engineers (summer interns or full time) to come hold the torch with us

4

10

92

kipply

@kipperrii

4 months

ok ml debugging is kind of fun, i'd take a heatmap over a stack trace any day

5

1

89

kipply

@kipperrii

2 months

in my head my bar for "human level" is actually "top 0.1% of humans" and i think this is more correct than human medians, in the same spirit of this classic dan luu work but also i think it's funny that our models will probably have lsat scores worthy of…

3

1

91

kipply

@kipperrii

2 years

i went back to toronto and the vast majority of people i wanted to see have already moved to the us and the ones i did see were mostly plotting to leave very sad to see how canada is incapable of holding onto ambition but the food and city are phenomenal

9

5

90

kipply

@kipperrii

10 months

omg yes it's catching on

kipply

@kipperrii

11 months

no one talks about how the tree of thought paper just throws in this inspirational quote???

4

2

83

5

2

90

kipply

@kipperrii

1 year

in machine learning, i find that "inventors" are never the ones who best understand what they've created, within a year easily dozens of people will know better (except for noam shazeer)

6

3

83

kipply

@kipperrii

4 months

im still convinced that insecurity and narcissism are mostly the same thing and that it would do the world a lot of good if insecurity was regarded with the same distaste

11

4

88

kipply

@kipperrii

6 months

tbc i'm pretty rogue, the only classics i've done are raytracing, contest programming and half of raft,,,, desperately want to do os but only have time for nand2tetris

6

0

85

kipply

@kipperrii

1 year

bundle's context length is probably ~128 tokens but he's still a long boy 🥺

6

0

85

kipply

@kipperrii

2 years

linkedin's needs a "marked safe in [company]'s mass layoffs". facebook had this down

0

9

84

kipply

@kipperrii

5 months

they don't let me write the model reports because i'd spend all my time carefully crafting lil psyops for my favourite researchers

1

3

82

kipply

@kipperrii

11 months

no one talks about how the tree of thought paper just throws in this inspirational quote???

4

2

83

kipply

@kipperrii

8 months

july august reading

3

1

80

kipply

@kipperrii

4 months

i think startups people use the word "moat" because people approach saas-shaped products with more or less equal footing. ai is talent constrained and builds up on research (practices) that are not easily imported. "high ground" is probably going to be more relevant than "moat"

11

3

81

kipply

@kipperrii

1 year

its really important to me that we don't fear-monger about ai safety, ive known too many people have their mental health damaged by worrying about xrisk and it has only slowed alignment progress.

5

2

79

kipply

@kipperrii

6 months

@SarvasvKulpati college doesn't do the classics though. raytracing and os are electives, compilers is always taught like shit and nand2tetris and connect 4 are not that common.

3

0

79

kipply

@kipperrii

7 months

@inerati i cried for days over the initial wave :c

4

0

77

kipply

@kipperrii

2 years

most of my friends who've been early at a startup have seen the same patterns of the founder losing their identity (not just changing as a person, but becoming very inconsistent in whatever identity they adopt) in ways that are unaesthetic and suboptimal for the company

4

1

79

kipply

@kipperrii

5 months

im on day one with this weird sort of chia pet (gift from @trishume ), i spray it once a day and it eventually matures into the one on the left 🦀

1

0

79

kipply

@kipperrii

2 years

how does it know

6

1

77

kipply

@kipperrii

7 months

and with that, the ants pulled their crumpled up copies of "computers can be understood" out of the recycling bins and mounted them back onto the frames on their desks. what will cause their next moment of doubt, and how shall they overcome it?

Anthropic

@AnthropicAI

7 months

The fact that most individual neurons are uninterpretable presents a serious roadblock to a mechanistic understanding of language models. We demonstrate a method for decomposing groups of neurons into interpretable features with the potential to move past that roadblock.

126

1K

6K

1

0

78

kipply

@kipperrii

1 year

thinking about the poor vc who replaced google with chatgpt and asks chatgpt for the weather in san francisco every day before putting on his patagonia

4

2

79

kipply

@kipperrii

2 months

@jacobaustin132 incredible failing up

1

0

78

kipply

@kipperrii

2 months

in every relationship theres a clear syn and clear ack

2

0

77

kipply

@kipperrii

1 year

no fucking shit

5

0

74

kipply

@kipperrii

3 months

my type a personality gets really frustracted when seeing someone insult my place of employment on twitter because i know i could insult it better but i'm not supposed to do that

5

1

76

kipply

@kipperrii

5 months

ant is hiring for a manager (and ics) for a lm systemsy team on pretraining! whoever gets the job gets to decide what to name it 👀 my top choices are "steps", "throughput", "occupancy" and "goodput"

4

6

76

kipply

@kipperrii

5 months

startups try too hard to fire unimpactful people and they don't notice all the negatively impactful people

1

3

73

kipply

@kipperrii

2 years

everyone should be talking about how the body font for seems to be an unreleased butterick font called khyber. he reactivated his bar membership and then made a font for the most beautiful lawsuit website. it's *so good* i love this

GitHub Copilot investigation · Joseph Saveri Law Firm & Matthew Butterick

GitHub Copilot investigation

githubcopilotinvestigation.com

4

10

74

kipply

@kipperrii

6 months

i may be an ea/agi girl, but my saas upbringing needs to me to say this: happy openai dev day🍾🎊🎉

4

0

75

kipply

@kipperrii

2 years

you have no idea how hard this was to acquire

7

0

74

kipply

@kipperrii

10 months

im at icml (and so is my mom)! come talk to us about alignment and working at anthropic 🥰

8

0

74

kipply

@kipperrii

6 months

@tszzl you should write them now, and then write them again years later but not look at what you wrote now

1

75

kipply

@kipperrii

2 years

spending two months running a startup was a really intense compromising of my values and replaced them when really crummy ones. dropping out of my yc batch and moving to ottawa to work on obscure compilers was me returning to my roots to save myself.

5

0

74

kipply

@kipperrii

2 years

The "Toy Models" part of all this speaks to it being a magnificent nerdsnipe and probably the most interactive mechanistic interpretability work that has been put out. The thread has a colab where you can replicate figures on a single gpu, and the work was replicated three times.

Anthropic

@AnthropicAI

2 years

Neural networks often pack many unrelated concepts into a single neuron – a puzzling phenomenon known as 'polysemanticity' which makes interpretability much more challenging. In our latest work, we build toy models where the origins of polysemanticity can be fully understood.

67

673

4K

1

3

72

kipply

@kipperrii

1 year

no one is safe!! bnii

2

3

70

kipply

@kipperrii

1 year

its the year of the bundle!!! we're going to rule the world!!! hes so ideal

2

0

69

kipply

@kipperrii

5 months

i don't think we should never build asi, i just don't think the optimal time is asap. i also think that the funniest way to slow down would be requiring datacenters to be zoned for residential and commercial use

5

2

70

kipply

@kipperrii

2 years

i have friend who is so smart and we talked a lot so i can simulate him and "what would he think" is like a "let's think step by step" prompt hack for my brain

6

0

70

kipply

@kipperrii

1 year

if your loss graph does something weird it's called a plot twist

3

6

70

kipply

@kipperrii

3 years

name a more iconic duo

4

69

kipply

@kipperrii

10 months

anthropic is hiring for a tokens ̶t̶y̶r̶a̶n̶t̶ manager! tokens is a small, high-leverage team that works on data for pretraining. the role looks something like a research manager who spends say 20% of their time coding. ask me about it if you know me!

3

2

69

kipply

@kipperrii

11 months

i like vibecamp because its low status to everyone who doesn't want to go

3

0

69