Josh Albrecht @joshalbrecht Twitter profile

Last Seen Profiles

@mwthor22

@_marilenaki

@corbin_carroll

@piikop

@wy4att

@Wass_Strength

@KyeKingCooper4

@od_gai

@animal_600

@jake_harvell

@GHSnacks

@Stylzbeats

@JerseyKidWins

@0xG23

@_twio

@yuu_h05

@want2bebig

@mattjwillis

@brianfauteux

@IamNomad

@dressregulation

@rob_otor

@evturo

@morgael1

@state_ice

@romymom_

@ctcoff99

@GarrettDickson

@LiaQuartapelle

@AsmlashGrantFdn

@SFTechWeek

@IStremato

@HarperFiction

@SMU_Services

@nixondecourt

@jiheown

Josh Albrecht

@joshalbrecht

2 years

Exciting news: our simulator for training more general RL agents is now open source! It's one of the fastest 3D RL environments (~10,000 frames per second on a single GPU). We're excited for researchers to create more general, robust, & safe AI agents:

Kanjun 🐙🏡

@kanjun

2 years

Today, AI systems can create stunning art & beat humans at chess & Go. But they can't do things a 3-year-old can do. Why? We're launching @genintelligent & open sourcing our research environment, Avalon, so as a community we can answer these questions:

53

186

977

2

8

47

Josh Albrecht

@joshalbrecht

1 year

Jim Fan is one of my favorite people I met last year. He has a lot of really interesting ideas, and it's definitely worth checking out the conversation below if you're interested in RL, minecraft, or considering a career in prompt engineering :)

Imbue

@imbue_ai

1 year

Learn about foundation models for embodied agents, why prompt engineering will become irrelevant, and much more with @DrJimFan from NVIDIA! Blog: Video:

3

22

70

2

5

44

Josh Albrecht

@joshalbrecht

6 months

This is exactly the kind of work I mean when I say "practical theory" Neural networks are not magical black boxes, and actually understanding what they're doing lets us waste a lot less time fiddling around. Very happy to be able to share this work by our very own Jamie Simon

Greg Yang

@TheGregYang

6 months

1/ μP is optimal scaling rule of learning rate & init as network width → ∞. Been confused? 🆕μP = holding the "natural" (I'll explain) operator norm constant for every weight W & its updates ΔW: μP <=> ‖W‖_nat = Θ(1) = ‖ΔW‖_nat. 🆕Frobenius norm is the wrong norm to measure!

80

179

866

0

5

35

Josh Albrecht

@joshalbrecht

1 year

Giving two talks about Avalon, our featured NeurIPS paper today: 1. Discussion with MineDojo author @DrJimFan at NeurIPS, 10:45a PT: 2. @ycombinator tech talk at 2p PT on "how we created one of the fastest RL simulators":

0

7

26

Josh Albrecht

@joshalbrecht

9 months

@arram Awww, thanks @arram ! I really appreciate the support, and the vote of confidence.

2

0

25

Josh Albrecht

@joshalbrecht

1 year

@percyliang Two ways to guarantee LLMs don't train on test data: 1. Retain a held out set that is not published 2. Create new test data each time Perhaps we should be moving towards the 2nd option anyway since it enables adversarial testing, which better measures worst case performance

1

0

18

Josh Albrecht

@joshalbrecht

10 months

Misha Belkin wrote a nice post pushing back on the narrative that "LLMs are too complicated to understand"

Can we develop theoretical explanations for today’s AI systems?

In short: Misha Belkin, a leading AI theory researcher, outlines a compelling case for a widespread effort to develop a more rigorous and…

imbue.com

1

2

17

Josh Albrecht

@joshalbrecht

11 months

Being able to hit "go", go to sleep, and wake up to having every hyperparameter perfectly tuned is so nice. Every point below is a fully trained network, but the whole process is only a few times slower than a single run (bc the optimizer is cost-aware and parallel) See the

Imbue

@imbue_ai

11 months

What if you could automatically discover both the scaling laws for every hyperparameter PLUS the best-performing network for every compute budget? Introducing CARBS, a cost-aware hyperparameter tuner that does this & more: 👇

3

14

120

0

5

15

Josh Albrecht

@joshalbrecht

3 years

Financial instruments are the world doing computation about the future.

2

1

15

Josh Albrecht

@joshalbrecht

2 years

This essay was one of the best things I've read this year.

Kanjun 🐙🏡

@kanjun

2 years

In 2020, @michael_nielsen & I began a 2-month project to write: "how would we fund science?" 2 years & 40,000 words later, it's become: "how can the culture & institutions of science actually change, and ultimately become self-improving?" Our answer:

30

187

917

0

1

10

Josh Albrecht

@joshalbrecht

10 months

Happy to see Jamie's research about the stepwise nature of SSL on the BAIR blog: If you're interested in learning more, stop by and say hi to us at ICML next week!

0

3

9

Josh Albrecht

@joshalbrecht

3 years

Silicon Valley is one of the only geographical areas with a functioning karma system.

0

1

8

Josh Albrecht

@joshalbrecht

1 year

Had a lot of fun at @pycon this past weekend! Was great to hear from so many developers how they could use our AI agents

Kanjun 🐙🏡

@kanjun

1 year

First time at PyCon weekend, and it was fun to share our open source tools + a demo of what’s coming! Thanks @pycon for choosing us as one of the 7 startups featured 🥳😎

0

1

12

0

7

Josh Albrecht

@joshalbrecht

1 year

Ben's idea about how to view RL as a contrastive learning problem is really novel and simple--definitely check out the podcast if you want a bunch of interesting ideas about AI

Imbue

@imbue_ai

1 year

Learn about designing simpler and more principled RL algorithms and much more with @ben_eysenbach from CMU! Blog: Video:

1

7

27

0

1

7

Josh Albrecht

@joshalbrecht

1 year

Thanks for having me on the podcast @_aj ! Enjoyed our conversation, as always

AJ Asver 🤖👨🏾‍💻🌁

@_aj

1 year

Last month I had the opportunity to chat with @joshalbrecht about the research his company @genintelligent is doing on AI Agents. We talked about AGI as human-like intelligence, memory in AI agents, and a world with universal unemployment and lots more.

2

4

25

0

2

7

Josh Albrecht

@joshalbrecht

1 year

It was great to sit down with @HaydenField at @CNBC to talk about chatbots. She put together a thoughtful high level guide to using these tools:

0

2

5

Josh Albrecht

@joshalbrecht

2 years

@Isinlor @ak92501 I think the catch is manual "tidying" of questions answered incorrectly until they find a working prompt? "Question tidying is performed iteratively and interactively" (Section 3E) I don't see how often that is required anywhere in their 114 pages...

1

0

5

Josh Albrecht

@joshalbrecht

11 months

Thoughtful write-up of our latest paper by @davisblalock , but I think my conclusion is slightly different: We need new ideas in AI. It's not that LLMs have hit a "statistical wall", but rather that if we want to do better, we should explore larger departures from what everyone

1

0

5

Josh Albrecht

@joshalbrecht

11 months

So true!

Imbue

@imbue_ai

11 months

Curious about AI research but don’t have a traditional background? Check out our spotlight on @elliekitanidis , who went from cosmology to cutting-edge AI research:

1

6

24

0

5

Josh Albrecht

@joshalbrecht

2 years

@oh_that_hat @kanjun We were happy to host! I always come away from our conversations with at least like 5 new ideas, it's wonderful :-)

1

0

4

Josh Albrecht

@joshalbrecht

1 year

@percyliang True--another good reason to create new adversarial examples at test time instead. I would hope that LLM providers should (soon?) have a mechanism for excluding data submitted through the API... Surely other users and businesses need that ability anyway?

3

0

4

Josh Albrecht

@joshalbrecht

4 years

Oops, I forgot that I had a twitter account. Maybe time to use it for something besides haikus now that twitter is actually a thing, unlike in 2009...

0

3

Josh Albrecht

@joshalbrecht

2 years

Did you know: you can convert a deep MLP to a single-layer by carefully crafting a special activation function! Work by one of our summer interns: Caveat: only applies in the NTK regime

Reverse engineering the NTK: towards first-principles architecture design

The BAIR Blog

bair.berkeley.edu

0

1

3

Josh Albrecht

@joshalbrecht

14 years

man at home at work | glance at pretty city lights | eyes go back to screen

0

2

Josh Albrecht

@joshalbrecht

15 years

Perfect sky and warmth | So I went outside and played | Soul Calibur... FOUR!

0

2

Josh Albrecht

@joshalbrecht

3 years

Longtermism is apparently the idea that: 1. future people matter 2. there are lots of them 3. we can affect their lives Didn't realize there was a simple name for what I care about!

The case for longtermism and safeguarding the future

www.founderspledge.com

0

2

Josh Albrecht

@joshalbrecht

15 years

thoughts of the future | watching life pour from the sky | safe in my warm cave

0

2

Josh Albrecht

@joshalbrecht

8 months

Would certainly be nice to see compute become more of a commodity!

FLOP Auction

@flopauction

8 months

We're happy to announce the world's first compute auction site is now in beta. Bid on time on our 23,000 Nvidia H100s. Per-hour access for any AI project!

4

8

39

0

2

Josh Albrecht

@joshalbrecht

15 years

I dreamt I forgot | the quadratic formula | woke up happily

0

2

Josh Albrecht