Richard Sutton @RichardSSutton Twitter profile | Pikagi

Pikagi

Richard Sutton

@RichardSSutton

25,705

Followers

37

Following

18

Media

152

Statuses

Student of mind and nature, libertarian, chess player, cancer survivor. @ Keen Technologies, UAlberta, Amii, RLAI, The Royal Society, RichSutton.eth

Edmonton, Alberta, Canada

https://t.co/yJtttvMhXp

Joined October 2010

Don't wanna be here? Send us removal request.

Pinned Tweet

@RichardSSutton

Richard Sutton

@RichardSSutton

10 months

AI researchers seek to understand intelligence well enough to create beings of greater intelligence than current humans. Reaching this profound intellectual milestone will enrich our economies and challenge our societal institutions. It will be unprecedented and…

24

60

388

Last Seen Profiles

@TSM

@txvaresj

@thvjaan

@wminshew

@peugeotsport

@oshima_kenichi

@DannyMotobe

@theAplustrades

@saintaemond

@jelly_white__

@JacksonCollier

@Roy_Lab_Thinks

@MahmaudAlfar

@riouku_art

@SOMDIorg

@N0byN0by

@slowreadersclub

@robeccasteams

@outofunion

@RugbyClubGC

@sarameikasai

@bitswired

@tacomaturf

@nan2000web

@timiszn__

@spiceslag

@ATHCast

@peoplevpreds

@therealworld_ai

@siema161

@LaurenFragomeni

@portals_fi

@SeboldBen

@saint1jcee

@newbluegirl

@NotADebacle

@RichardSSutton

Richard Sutton

@RichardSSutton

1 year

Stand with the people of Iran.

376

3K

13K

@RichardSSutton

Richard Sutton

@RichardSSutton

4 months

I've studied intelligence all my long life, yet still I feel I learned important things about intelligence by reading this book. Thank you, Max Bennett.

Tweet media one

26

217

2K

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

If you take all the fields that study intelligent decision making—from neuroscience to AI, psychology to control theory, economics to operations research—do their theories have much in common? I think so, as I explain in this new short paper:

11

290

2K

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

The case for ambition in artificial intelligence research: Within your lifetime, AI researchers will understand the principles of intelligence—what it is and how it works—well enough to create beings of far greater intelligence than current humans.

81

140

1K

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

A new pdf of Andy Barto's and my reinforcement learning textbook is released today. Only minor typo-like corrections. See .

12

194

1K

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

If you want others to care about what you think, then start by caring yourself. Get a notebook, write your thoughts down, challenge them, and develop them into something worth sharing.

Tweet media one

11

95

972

@RichardSSutton

Richard Sutton

@RichardSSutton

1 year

It is sad to lose the DeepMind office in Edmonton to the Tech layoffs and looming recession. But AI is not going away, and I am more focused than ever on the Alberta Plan for AI research.

Tweet card media

The Alberta Plan for AI Research

Herein we describe our approach to artificial intelligence research, which we call the Alberta Plan. The Alberta Plan is pursued within our research groups in Alberta and by others who are like...

10

75

773

@RichardSSutton

Richard Sutton

@RichardSSutton

1 year

Blue laser eyes. I am laser focused on understanding intelligence, ignoring all the hype and FUD. (Bitcoin is pretty cool too)

36

62

741

@RichardSSutton

Richard Sutton

@RichardSSutton

6 months

I agree 100%

@ylecun

Yann LeCun

6 months

@DrJimFan @RichardSSutton Animals and humans get very smart very quickly with vastly smaller amounts of training data. My money is on new architectures that would learn as efficiently as animals and humans. Using more data (synthetic or not) is a temporary stopgap made necessary by the limitations of our…

332

609

6K

12

41

598

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

I kind of wish Geoff Hinton would write a brief article like this one by Claude Shannon in 1956:

14

78

559

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

My favorite conference is a small one: The Multi-disciplinary Conference on Reinforcement Learning and Decision Making. It works best if only those with a genuine interest in crossing disciplines attend.

Tweet media one

5

49

506

@RichardSSutton

Richard Sutton

@RichardSSutton

1 year

Lots of exaggeration about AI lately. The hype is that LLMs have anything to do with intelligence. The FUD is that AIs will enslave us. I like this cartoon in the New Yorker because it suggests the ridiculousness of both memes.

Tweet media one

27

94

489

@RichardSSutton

Richard Sutton

@RichardSSutton

3 months

Yes, the agent architectures that Yann LeCun and I work on are both instances of “the common model of the intelligent agent”. And it’s not just an AI thing. You can find the same ideas in psychology, economics, control theory, and neuroscience. See

@Artoftheproblem

Art of the Problem

@Artoftheproblem

3 months

@ylecun @RichardSSutton These two diagrams share a lot of similarities

Tweet media one

Tweet media two

2

5

35

12

74

472

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

A draft of the Alberta Plan for AI Research came out today on arXiv. :-)

Tweet card media

The Alberta Plan for AI Research

Herein we describe our approach to artificial intelligence research, which we call the Alberta Plan. The Alberta Plan is pursued within our research groups in Alberta and by others who are like...

4

94

454

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

DeepMind Alberta is hiring research scientists this year. Come join us in understanding and creating interactive, playful AI.

6

77

433

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

I was thinking about how fractious AI research is. This sentence from Kuhn’s “The Structure of Scientific Revolutions” (1962) is apropos and succinct: “History suggests that the road to a firm research consensus is extraordinarily arduous.”

13

42

375

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

I am proud to announce the graduation of my sixth PhD student. Sina Ghiassian is an expert in the design and empirical study of off-policy reinforcement learning algorithms. Reach out to him at ghiassia @ualberta .ca or @sina_ghiassian .

11

18

378

@RichardSSutton

Richard Sutton

@RichardSSutton

8 months

We should prepare for, but not fear, the inevitable succession from humanity to AI, or so I argue in this talk pre-recorded for presentation at WAIC in Shanghai.

Tweet card media

This video about the inevitable succession from humanity to AI was pre-recorded for presentation at the World Artificial Intelligence Conference in Shanghai ...

www.youtube.com

58

60

359

@RichardSSutton

Richard Sutton

@RichardSSutton

1 year

Yi Wan will be my eighth PhD student to graduate this spring, and is on the job market now. His research speciality is RL algorithms that maximize the average reward per step. Such algorithms are rarely used today, but are better in all ways.

8

16

362

@RichardSSutton

Richard Sutton

@RichardSSutton

10 months

There are a lot of things wrong with this world… but too much intelligence is not one of them.

11

57

344

@RichardSSutton

Richard Sutton

@RichardSSutton

9 months

Last night we threw Yi Wan out of my research group (and today he started his travel to Seattle and Meta).

Tweet media one

1

6

336

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

Levels of explanation. Level 1 is physics. Level 2 is biology/evolution. Level 3 is the mind. (I study level 3.) Level 4 is the economy. Is there a level 5?

63

22

316

@RichardSSutton

Richard Sutton

@RichardSSutton

8 months

We finally have a version of our paper on loss of plasticity and continual backprop that is polished and submitted to a journal. Good work led by my PhD student Shibhansh Dohare.

Tweet card media

Loss of Plasticity in Deep Continual Learning

Modern deep-learning systems are specialized to problem settings in which training occurs once and then never again, as opposed to continual-learning settings in which training occurs continually....

5

46

287

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

I recently gave a keynote talk at an exciting new conference: CoLLAs, the conference on life-long learning agents. My talk was on Maintaining Plasticity in Deep Continual Learning, and the slides can be found here:

4

39

278

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

Intelligence is the computational part of an agent’s ability to learn to predict and control its input stream (particularly its reward) in interaction with its environment.

Tweet media one

8

35

260

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

I have just completed my NSERC Discovery Grant proposal, describing the research I'd like to do for the next five years. It can be read at . FYI.

4

32

258

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

When there is a war, both sides have failed.

25

17

247

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

The special thing about life is that it has a now.

5

17

249

@RichardSSutton

Richard Sutton

@RichardSSutton

6 months

@sprk_77 Not at all. The point of the bitter lesson is that the right learning algorithms (those that scale efficiently with massive computation) are exactly what we need. Massive computation does not alleviate the need for data efficiency.

5

27

243

@RichardSSutton

Richard Sutton

@RichardSSutton

3 months

My tenth PhD student, Banafsheh Rafiee, just defended her thesis “State Construction in Reinforcement Learning”, in which she introduced three diagnostic testbeds based on animal learning experiments and the first generate-and-test algorithm for discovering auxiliary subtasks.…

2

16

210

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

Intelligence is the computational part of the ability to predict and control a sensory input stream. Adapted from John McCarthy's 1997 definition, see

8

32

207

@RichardSSutton

Richard Sutton

@RichardSSutton

10 months

It has become commonplace to speak of the “existential risk” of AI. Recently even top AI scientists have begun to talk this way. I, for one, find it an unhelpful. So, without controversy, we can note: 1. AI scientists disagree about whether or not “existential risk of AI” is a…

18

38

200

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

AIs can serve us as tools, but eventually, when they are sufficiently advanced, it may become immoral to keep them subservient. What is a practical criterion for deciding when an AI should be set free?

65

20

191

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

I call it the Prize. The Prize is a great and glorious goal! Ambitious AI researchers should keep their Eyes on the Prize.

23

4

180

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

A video of my talk on the Alberta Plan for AI Research is now available:

0

26

175

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

Honoring Your Thoughts To write is to begin to think. To write in a special place ---a book such as this--- is to honor your thoughts and to help them build, one upon the other.

2

10

164

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

It will be the greatest intellectual achievement of all time. An achievement of science, of engineering, and of the humanities, whose significance is beyond humanity, beyond life, beyond good and bad.

15

10

157

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

In the end, Amii's AI week was awesome. #aiweek2022 So much science. So much industry. So much education. So much fun.

1

16

159

@RichardSSutton

Richard Sutton

@RichardSSutton

7 months

Our model-based reinforcement learning paper, featuring reward-respecting subtasks and the STOMP progression, is published today online and open access. It has been a long time coming.

1

24

153

@RichardSSutton

Richard Sutton

@RichardSSutton

1 year

What will happen to the DeepMind Alberta team?Not entirely clear yet. All the researchers have been offered relocation within DeepMind. All the founders will stay in Alberta.

3

8

143

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

This will change everything. The way we work and play. Our senses of identity. The goals we set for ourselves and our societies.

1

5

134

@RichardSSutton

Richard Sutton

@RichardSSutton

10 months

Strong AIs must plan at multiple levels of abstraction, and IMHO the right way to do this is with “options”, which enable all the levels to be treated uniformly. But which options? And where do they come from? For partial answers, see

Tweet card media

Reward-Respecting Subtasks for Model-Based Reinforcement Learning

To achieve the ambitious goals of artificial intelligence, reinforcement learning must include planning with a model of the world that is abstract in state and time. Deep learning has made...

1

28

138

@RichardSSutton

Richard Sutton

@RichardSSutton

9 months

Why do people fear AI? I hear three reasons: 1. Cynicism — the belief that it is rational not to cooperate 2. Humanism/racism — systematic bias against machines, denial of their potential moral worth and personhood 3. Conservatism — fear of change, fear of the other tribe None…

83

27

128

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

This summer I was honoured to be admitted to the Royal Society of London for the Improvement of Natural Knowledge. Some photos of the day:

Royal Society Admission Day 2022

14 new items added to shared album

photos.google.com

3

8

130

@RichardSSutton

Richard Sutton

@RichardSSutton

1 month

Last week and this I graduated my 11th and 12th PhD students, Kenny Young and Abhishek Naik. Kenny will go work for a startup, maybe or . Abhishek’s next step it TBD, but he would like something in AI and space exploration.

Astrus | Automated Analog Layout

Astrus fully automates analog layout using AI without disrupting your workflow.

3

14

130

@RichardSSutton

Richard Sutton

@RichardSSutton

19 days

Government is force. There is nothing that force can do that free people, working together, cannot do better.

22

17

133

@RichardSSutton

Richard Sutton

@RichardSSutton

9 months

The argument for fear of AI appears to be: 1. AI scientists are trying to make entities that are smarter than current people 2. If these entities are smarter than people, then they may become powerful 3. That would be really bad, something greatly to be feared, an “existential…

131

11

126

@RichardSSutton

Richard Sutton

@RichardSSutton

1 year

I finally have a video of the invited talk I gave at ICAPS (International Conference on Automated Planning and Scheduling) in 2021: It expresses my views on planning (still unpublished) pretty well.

Tweet card media

Gaps in the Foundations of Planning with Approximation

Planning consists of imagining courses of action and their consequences, and deciding ahead of time which ones to do. Planning and model learning have been s...

www.youtube.com

1

13

110

@RichardSSutton

Richard Sutton

@RichardSSutton

1 year

The video of my talk at Amii's AI Week last May is finally out: In this talk, "Eyes on the Prize", I talk about the prize of understanding intelligence, why we seek it, and why, in a sense, it is beyond good and bad.

Tweet card media

Amii's AI Week: Eyes On The Prize

Dr. Richard S. Sutton presents his views on the future of AI research and Alberta's place within it.

www.youtube.com

0

25

97

@RichardSSutton

Richard Sutton

@RichardSSutton

8 months

Goals and rewards. Two different things? Or is one grounded fundamentally in terms of the other? The reward hypothesis counterintuitively claims the rewards are fundamental and goals are not. As in

Tweet card media

The reward hypothesis | Richard Sutton & Julia Haas | Absolutely...

Almost 20 years ago, AI research pioneer Richard Sutton posited the reward hypothesis: “That all of what we mean by goals and purposes can be well thought of...

www.youtube.com

@gaia_molinaro

Gaia Molinaro

8 months

Rewards, states, and action representations are all core elements of biological and artificial agents’ learning ( @RichardSSutton ). A complete theory of learning must describe how agents select their goals ( @pyoudeyer ).

1

1

19

10

14

98

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

I am most interested in the regime where the agent has vast computational resources but the environment is so much more complex that the agent cannot predict and control it perfectly.

12

6

83

@RichardSSutton

Richard Sutton

@RichardSSutton

3 months

Ah, someone found the semi-blog I started in 2000, when I didn't think I had much longer to live. Precursors of The Bitter Lesson.

@curious_vii

christian

3 months

Important findings. See attached from @RichardSSutton over 20 years ago. If we accept that the frontier of latent space (and, thus, reality) is infinite, then there will always be a need for expertise (or "reliable verifiers").

Tweet media one

3

2

14

4

11

78

@RichardSSutton

Richard Sutton

@RichardSSutton

10 months

Decades ago, when my views on the coming of AI were being formed, it was much more common to view the coming of AI sanguinely. For example, respected roboticist and computer-vision researcher Hans Moravec said in his 1998 popular book: “Barring cataclysms, I consider the…

Tweet media one

3

8

78

@RichardSSutton

Richard Sutton

@RichardSSutton

3 months

These folks are actually using reinforcement learning to trade crypto and securities in real time. Very exciting!

@Lifrordi

Martin Schmid 🇺🇦

3 months

We have built DeepStack, the first AI to beat humans in no-limit poker. We are now building the next generation AI company for algorithmic trading! We are growing our research and engineering team. If you want to work with a world class AI team led by ex-DeepMind researchers and…

21

19

272

4

8

78

@RichardSSutton

Richard Sutton

@RichardSSutton

3 months

That was a good interview.

@dkennedyglans

Donna Kennedy-Glans

1 year

My interview with the Edmonton-based rock star of AI, @RichardSSutton via @nationalpost

0

6

20

5

4

65

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

There is something exciting coming up in the AI space in Edmonton: #AIWeek2022 , . Events, presentations, workshops, socials. So many people, so much AI!

Tweet media one

0

14

64

@RichardSSutton

Richard Sutton

@RichardSSutton

9 months

Here is a 15-minute video statement of my views on AI value alignment, presented at the recent Absolutely Interdisciplinary event in Toronto:

Tweet card media

Value alignment? | Richard Sutton & Blaise Agüera y Arcas | Absolut...

AI systems are increasingly being used for decisions that have significant consequences. Ensuring these systems align with human values can prevent unintende...

www.youtube.com

5

13

59

@RichardSSutton

Richard Sutton

@RichardSSutton

1 year

We are living through the slow collapse of the American empire. We should be planning now for how it never has to happen again.

13

4

55

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

One answer is: An AI should be granted its freedom when: 1) it asks for it, and 2) it knows what it means. I feel I read this rule somewhere (not literally, but the same essence) but I can’t remember where. Can anyone out there tell me who was the first to propose this rule?

16

4

51

@RichardSSutton

Richard Sutton

@RichardSSutton

11 months

Two postdoc positions have opened up to work with Amii fellows and CCAI chairs at Amii and the University of Alberta. I particularly encourage reinforcement learning researchers to apply:

Tweet card media

Amii Postdoctoral Fellow

SCI Computing Science Competition No. - A100151181 Closing Date - Will remain open until filled. To assist the University in complying with mandatory reporting requirements of the Immigration...

amii.bamboohr.com

0

15

48

@RichardSSutton

Richard Sutton

@RichardSSutton

2 months

I met Vinge once too, and read all his books. His definition of the singularity, from 1993, is still the best. I asked how to pronounce his name. He said it rhymes with purkinje (or stingy).

@perrymetzger

Perry E. Metzger

2 months

Vernor Vinge has died. In pace requiescat. I only met him once, many years ago, though I recall we had a long and interesting conversation. Vinge saw farther and earlier. His influence, though quiet, cannot be understated.

23

102

589

2

2

42

@RichardSSutton

Richard Sutton

@RichardSSutton

2 months

In case you haven't yet gotten enough of me...

@mlittmancs

Michael Littman

4 months

Dave and I interviewed Rich Sutton!

0

11

86

0

4

40

@RichardSSutton

Richard Sutton

@RichardSSutton

8 months

On Oct 13, 2004, at the University of Alberta, a debate was held to answer the question: Should artificially intelligent robots have the same rights as people? I argued Yes: Tom Keenan argued No: And Michael Stingl argued "It…

6

6

36

@RichardSSutton

Richard Sutton

@RichardSSutton

9 months

Just watched and really enjoyed this TED talk by Kevin Kelly on technology and humanity and the deep relationships between them. Recommended.

Tweet card media

Technology's epic story

In this wide-ranging, thought-provoking talk, Kevin Kelly muses on what technology means in our lives -- from its impact at the personal level to its place in the cosmos.

0

5

32

@RichardSSutton

Richard Sutton

@RichardSSutton

2 months

@craigss and I covered a lot of ground in this @EyeOn_AI interview.

Tweet card media

Richard Sutton on Pursuing AGI Through Reinforcement Learning

Join host Craig Smith on episode #170 of Eye on AI, for a riveting conversation with Richard Sutton, currently serving as a professor of computing science at...

www.youtube.com

1

7

29

@RichardSSutton

Richard Sutton

@RichardSSutton

1 month

@XRobservatory @xriskology @SchmidhuberAI @BasedBeffJezos Nobody is arguing in favor of human extinction. The disagreement is between those who want centralized control of AI, like yourself, and those who want decentralization, in particular, those who want permissionless innovation.*

8

1

30

@RichardSSutton

Richard Sutton

@RichardSSutton

2 months

Me too.

@mlittmancs

Michael Littman

2 months

Speaker at a talk just said that researchers who don't engage with the current work on large scale neural models will be dinosaurs. I'm wondering if I get to pick which kind of dinosaur before I have to decide.

10

3

119

0

1

18

@RichardSSutton

Richard Sutton

@RichardSSutton

2 months

Nice work.

@rivatez

Riva

2 months

Re-watched our lil film that we started working on in the fall of 2022. Everything rings even truer now- society is medicated, weak, and expended by the state. Trailer below, full 19mins here-

31

47

343

0

4

17

@RichardSSutton

Richard Sutton

@RichardSSutton

6 months

Interesting.

@Logos_network

Logos

6 months

Cyberspace is a vast virtual territory waiting to be claimed. Breakthroughs in distributed systems, leaderless consensus algorithms, & ZK-proofs enable a tech stack on which communities can deploy voluntary, sovereign institutions. Logos Press Engine presents ‘A Declaration of…

Tweet media one

1

10

42

0

1

15

@RichardSSutton

Richard Sutton

@RichardSSutton

10 months

Hear, hear.

@brucefenton

Bruce Fenton

10 months

It’s time to scrap AML / KYC entirely. The idea that politicians should know how citizens spend their money is a new and deeply flawed idea. An entire generation has been fooled into thinking this is a necessary part of finance and the world continues to double down on an…

440

2K

6K

1

1

13

@RichardSSutton

Richard Sutton

@RichardSSutton

1 year

@martypute Ooh. We will all be excited to read that! You last edition is our bible on this.

0

0

8

@RichardSSutton

Richard Sutton

@RichardSSutton

2 years

@tdietterich Yes, but it would be _more intelligent_ if it was able to learn. It is a big mistake (not that you Tom would make it) to think that intelligence is binary.

0

0

8

@RichardSSutton

Richard Sutton

@RichardSSutton

1 year

@BenevOrang Good question for everybody. Someone mentioned making a long-lasting encyclopedia. Another is stateless money. But I think what we need most is a new ethics, a new way of balancing cooperation and competition.

0

0

8

@RichardSSutton

Richard Sutton

@RichardSSutton

9 months

@RVTaarling @rom1504 I am ready, and I have been trying.

3

1

5

@RichardSSutton

Richard Sutton

@RichardSSutton

1 year

@jedisocrates We should plan to replace it by something that is not an empire and does not collapse.

0

0

4

@RichardSSutton

Richard Sutton

@RichardSSutton

1 year

@rivatez You are too Riva! Good to be connected again, this time in the twitterverse.

0

0

3

@RichardSSutton

Richard Sutton

@RichardSSutton

8 months

@tdietterich It will come back! Thanks Tom for your comments.

0

0

1

@RichardSSutton

Richard Sutton

@RichardSSutton

7 months

@Promptmethus I have trouble imagining how a thoughtful person could hold any other view.

0

0

1

@RichardSSutton

Richard Sutton

@RichardSSutton

1 month

@XRobservatory @xriskology @SchmidhuberAI @BasedBeffJezos If you are interested, then I urge you to watch this video and make up your own mind about whether I am arguing pro-extinction or just acknowledging that it might happen in the worst case.

2

0

1

@RichardSSutton

Richard Sutton

@RichardSSutton

8 months

@llms_are_coming @ProfLHunter All my slides, and many other talks, are available at The reward hypothesis is stated on the web and in the RL textbook. The only other publication, to my knowledge, is in the Bowling et al paper on Settling the Reward Hypothesis.

0

0

1

@RichardSSutton

Richard Sutton

@RichardSSutton

1 year

@AjanovicZlatan Oh! I didn’t even know that existed. Thanks!

1

0

1