Stanford NLP Group @stanfordnlp Twitter profile

Last Seen Profiles

@Prodiscius

@xripi_chix

@OminousDaGOAT

@eaclmeeting

@LehighSports

@chaosringen

@NYKagenda

@Ninjaristic_

@Reiwholesaler

@MountainFiain

@CourtneyCromer6

@Pinheiro_Enf

@KadriLiik

@SantoTlatoanimx

@TWICE_INAteam

@Retee27

@OPPCommissioner

@Newstar_shops

@Takayuuki_art

@MlamuliSA

@Mistress_Furina

@Sick_Show

@OhMySenpaiShop

@PriestOppai

@mariah_art

@OM_asker

@Tahira2018

@Mohamed_Al_mn

@Omeqa_

@TabiNekoShiro

@Sp1gity

@NatsukiSaika

@RealAJBenza

@GrantThorntonIN

@MOLKYFTN

@IAugenstein

Stanford NLP Group

@stanfordnlp

5 years

Our new-ish, neural, pure Python stanfordnlp package provides grammatical analyses of sentences in over 50 human languages! Version 0.2.0 brought sensibly small model sizes and an improved lemmatizer. Try it out: pip install stanfordnlp

22

707

2K

Stanford NLP Group

@stanfordnlp

5 years

CS224N Natural Language Processing with Deep Learning 2019 @Stanford course videos by @chrmanning , @abigail_e_see & guests are now mostly available (16 of 20). Big update from 2017. YouTube playlist: – new CS224N online hub: #NLProc

26

647

2K

Stanford NLP Group

@stanfordnlp

5 years

Out now: our new Python #NLProc package. StanfordNLP provides native, neural (PyTorch) tokenization, POS tagging and dependency parsing for 53 languages based on UD v2—and a Python CoreNLP interface. PyPI: – pip install stanfordnlp

StanfordNLP

High-performance human language analysis tools, now with native deep learning modules in Python, available in many human languages.

stanfordnlp.github.io

13

761

2K

Stanford NLP Group

@stanfordnlp

5 years

We’re gearing up for the 2019 edition of Stanford CS224N: Natural Language Processing with Deep Learning. Starts Jan 8—over 500 students enrolled—using PyTorch—new Neural MT assignments—new lectures on transformers, subword models, and human language.

19

401

2K

Stanford NLP Group

@stanfordnlp

7 months

A 2023 update of the CS224N Natural Language Processing with Deep Learning YouTube playlist is now available with new lectures on pretrained models, prompting, RLHF, natural language and code generation, linguistics, interpretability and more. #NLProc

9

283

1K

Stanford NLP Group

@stanfordnlp

5 years

Yes, @GoogleAI (well, all of @AlphabetINC ) produces a lot of awesome AI research, but @Stanford + @MIT together produce more (judging by @NeurIPSConf papers!), and @Stanford + @MIT + @UCBerkeley + @CarnegieMellon produces more than @AlphabetINC + @Microsoft + @facebook

21

261

918

Stanford NLP Group

@stanfordnlp

4 years

Stanford CS224N: Natural Language Processing with Deep Learning is back for 2020, starting Jan 7, with over 500 students enrolled: #cs224n

10

178

798

Stanford NLP Group

@stanfordnlp

6 years

Delete, Retrieve, Generate: A simple approach to doing neural style transfer on text, altering text for sentiment or style—Juncen Li, Robin Jia, He He & @percyliang #NAACL2018

1

242

727

Stanford NLP Group

@stanfordnlp

1 year

We’re not quite convinced that this sort of on-demand learning actually works to build a foundation in a subject …

Enias Cailliau

@enias

1 year

Stanford's CS224N NLP with Deep Learning Course will take 24+ hours to understand how GPT works. There's a better way. I turned Stanford's course into a chatbot that can answer the questions you need answers for. Try it out here:

6

32

189

19

73

657

Stanford NLP Group

@stanfordnlp

4 years

Announcing Stanza v1.0.0, the new packaging of our Python #NLProc library for many human languages (now including mainland Chinese), greatly improved and including NER. Documentation Github PyPI (or conda)

8

261

650

Stanford NLP Group

@stanfordnlp

5 years

“Google & DeepMind have hired 23 professors, Amazon 17, Microsoft 13, and Uber, Nvidia & Facebook 7 each. Tech companies disagree that they are plundering academia. A Google spokesman said the company was an enthusiastic supporter of academic research.”

When the A.I. Professor Leaves, Students Suffer, Study Says (Published 2019)

After losing top professors, college students are less likely to create artificial intelligence start-ups and get less venture funding, the research says.

www.nytimes.com

14

215

557

Stanford NLP Group

@stanfordnlp

7 years

Free, up-to-date @Stanford deep learning lectures: #NLProc & Vision

3

247

539

Stanford NLP Group

@stanfordnlp

6 years

Pytorch implementations of many of the deep learning NLP models discussed in cs224n by Kim SungDong. #dlearn #NLProc

GitHub - DSKSD/DeepNLP-models-Pytorch: Pytorch implementations of various Deep NLP models in...

Pytorch implementations of various Deep NLP models in cs-224n(Stanford Univ) - DSKSD/DeepNLP-models-Pytorch

github.com

3

198

532

Stanford NLP Group

@stanfordnlp

7 years

Stanford Lecture Collection—Natural Language Processing with Deep Learning—Manning/Socher

2

238

518

Stanford NLP Group

@stanfordnlp

5 years

“One thing I don’t like about the reporting around AI is that journalists seem to think the progress is happening in companies, and that’s not true. They are part of it, but a lot of the progress is continuing to happen in academia.”—Yoshua Bengio.

2

195

507

Stanford NLP Group

@stanfordnlp

4 months

DPO (Direct Preference Optimization, ) now completely owns top-of-leaderboard medium-sized neural language models! (More experimentation with IPO, KTO, PPO, etc. would be great! – as hf seems to be trying: )

1

99

514

Stanford NLP Group

@stanfordnlp

5 years

Congratulations to @danqi_chen on completing her dissertation on Neural Reading Comprehension and Beyond. She starts next year as an Asst Prof at @PrincetonCS : . But first, vacation in Hawai’i. 🌺 Thesis: #NLProc

15

60

502

Stanford NLP Group

@stanfordnlp

5 years

Good places for viewing “progress” in #NLProc —or at least the latest over-tuning results on various benchmarks😉: ☆ ☆ ☆ ☆ ☆ ☆

Papers with Code - Natural Language Processing

Browse 672 tasks • 2028 datasets • 2082

paperswithcode.com

3

200

476

Stanford NLP Group

@stanfordnlp

5 years

Want to learn Natural Language Processing with Deep Learning a.k.a. Artificial Neural Network methods? Stanford’s @SCPD_AI is launching an online professional version of our #cs224n course with customized video content and online course assistant support:

7

137

476

Stanford NLP Group

@stanfordnlp

7 years

Lecture notes for Stanford CS224N Natural Language Processing with Deep Learning are now on GitHub. Edits welcome!

GitHub - stanfordnlp/cs224n-winter17-notes: Course notes for CS224N Winter17

Course notes for CS224N Winter17. Contribute to stanfordnlp/cs224n-winter17-notes development by creating an account on GitHub.

github.com

6

241

484

Stanford NLP Group

@stanfordnlp

5 years

“[BERT] is the single biggest positive change we’ve had [to ⁦ @Google ⁩ search rankings] in the last five years,” ⁦ @PanduNayak ⁩ said. #nlproc Google Search Now Reads at a Higher Level | WIRED

Google Search Now Reads at a Higher Level

The company is incorporating new software that better understands subtleties of language, with the biggest changes for queries outside the US.

www.wired.com

4

129

478

Stanford NLP Group

@stanfordnlp

6 years

Our free online lecture series on #NLProc : NLP by @jurafsky & @chrmanning (2012 Coursera nlp-class) —missing from web for ages for tedious reasons but now back! ❦ NLP with Deep Learning by @chrmanning & @RichardSocher (cs224n 2017)

Natural Language Processing with Dan Jurafsky and Chris Manning, 2012

This 2012 lecture series from Stanford professors Dan Jurafsky and Chris Manning covers fundamental algorithms and mathematical models for processing natural...

www.youtube.com

6

193

469

Stanford NLP Group

@stanfordnlp

6 years

Deep Learning, Language and Cognition: Video of an introductory talk on computational linguistics for a broad audience—from hand-written rules to modern neural net models—by Christopher Manning ( @chrmanning ) at IAS, Princeton. #NLProc

2

211

477

Stanford NLP Group

@stanfordnlp

2 years

Transformers just munched up another sub-area of Deep Learning! 😋 HT @HannesStaerk

6

93

473

Stanford NLP Group

@stanfordnlp

2 years

Like we said, the author list of the foundation models paper is completely reasonable…

15

55

457

Stanford NLP Group

@stanfordnlp

6 years

People usually get information from others in a multi-turn conversation. To approach this, we’ve released CoQA 🍃—A Conversational Question Answering Challenge by @sivareddyg • @danqi_chen • @chrmanning . 127K Qs— free-form answers—with evidence—multi-domain.

6

217

459

Stanford NLP Group

@stanfordnlp

4 years

“Stanford has another fantastic NLP course which is also freely available online taught by a world renowned NLP researcher, academic, and author. The course in is From Languages to Information (CS124), and it is taught by Dan @Jurafsky .”

From Languages to Information: Another Great NLP Course from Stanford - KDnuggets

Check out another example of a Stanford NLP course and its freely available courseware.

www.kdnuggets.com

5

142

447

Stanford NLP Group

@stanfordnlp

6 years

How can you teach a machine learning system with human language rather than “labels”? With a semantic parser & labeling functions! New #ACL2018 paper by @bradenjhancock @paroma_varma @stephtwang @bringmartino @percyliang & Chris Ré @HazyResearch #NLProc

2

200

434

Stanford NLP Group

@stanfordnlp

1 year

Stanford Researcher ⁦ @simran_s_arora ⁩ develops a simple prompting strategy that enables open-source LLMs with 30x fewer parameters to exceed the few-shot performance of GPT3-175B - MarkTechPost

Stanford Researcher develops a simple prompting strategy that enables open-source LLMs with 30x...

By improving the in-context learning quality of smaller and open source models, more researchers and organizations can study and apply the technology. One exciting set of applications is in persona...

www.marktechpost.com

2

76

423

Stanford NLP Group

@stanfordnlp

6 years

NLP with Deep Learning Course videos (from 2017): – corresponding materials:

Lecture Collection | Natural Language Processing with Deep Learning (Winter 2017)

Natural language processing (NLP) deals with the key artificial intelligence technology of understanding complex human language communication. This lecture s...

www.youtube.com

2

140

418

Stanford NLP Group

@stanfordnlp

6 years

Final project reports for CS224N: Natural Processing with Deep Learning are up. Lots of amazing work! Incredible students.

6

156

413

Stanford NLP Group

@stanfordnlp

5 years

Geoff Hinton on importance of university research—“One worry is that the most fertile source of genuinely new ideas is graduate students being well advised in a university. They have the freedom to come up with genuinely new ideas—we need to preserve that”

The 'Godfather of Deep Learning' on Why We Need to Ensure AI Doesn't Just Benefit the Rich

Martin Ford made waves with his 2015 book, Rise of the Robots, which details the many accelerating trends in automation and how they’re slated to impact business and, especially, employment. For his...

gizmodo.com

1

154

386

Stanford NLP Group

@stanfordnlp

6 years

Some day deep learning might be a science. For now, it remains a craft, so people are rewarded for honing their skills at the craft. #dlearn

5

161

394

Stanford NLP Group

@stanfordnlp

6 years

Dependency trees are still useful for relation extraction, even in the neural age! Graph Convolution over Pruned Dependency Trees Improves Relation Extraction by @yuhaozhangx @qi2peng2 @chrmanning at #emnlp2018 #NLProc

5

125

387

Stanford NLP Group

@stanfordnlp

8 months

It’s the origin of attention! @DBahdanau & @kchonyc couldn’t afford Google’s large multi-GPU neural MT, so they thought of a better way. Us either. @lmthang & @chrmanning introduced simpler multiplicative attention. Then @GoogleAI folk wondered if attention is all you need…

hardmaru

@hardmaru

8 months

I prefer to operate in “GPU-Poor” mode. I don’t agree with the take from the semianalysis piece. Creative breakthroughs often occur under constraints—new systems, models, and methods that can better take advantage of even larger-scale compute

72

135

1K

5

76

356

Stanford NLP Group

@stanfordnlp

22 days

After a meteoric rise, DSPy is now the @stanfordnlp repository with the most GitHub stars. Big congratulations to @lateinteraction and his “team”. DSPy: Programming—not prompting—Foundation Models

6

49

356

Stanford NLP Group

@stanfordnlp

5 years

10 Open-Sourced AI Datasets—SyncedReview 2018 In Review. 3/10 from ⁦ @stanfordnlp ⁩, 4/10 from ⁦ @Stanford ⁩. Open Images V4—MURA—BDD100K—SQuAD 2.0—CoQA—Spider 1.0—HotpotQA—Tencent ML Images—Tencent AI Lab Embedding Corpus for Chinese—fastMRI.

2018 In Review: 10 Open-Sourced AI Datasets

In a boon to AI researchers, the last year witnessed an unprecedented open-sourcing of large datasets by popular AI research projects.

medium.com

4

148

335

Stanford NLP Group

@stanfordnlp

6 months

We still see lots of links to old releases of CS224N. Make sure you're getting the latest goodness (RLHF, prompting, transformers) from the 2023 release! YouTube: Website: For fee cohort-based online class:

Natural Language Processing with Deep Learning | Stanford Online

online.stanford.edu

2

82

321

Stanford NLP Group

@stanfordnlp

5 years

Here’s a useful mixed tutorial and review of our newly introduced stanfordnlp Python #NLProc library built on PyTorch – – from Analytics Vidhya

Introduction to StanfordNLP: An Incredible State-of-the-Art NLP Library for 53 Languages (with...

StanfordNLP is a collection of pre-trained state-of-the-art models. This tutorial is an introduction to Stanford NLP in Python and its implementation.

www.analyticsvidhya.com

4

110

311

Stanford NLP Group

@stanfordnlp

4 years

ELECTRA’s Replaced Token Detection pre-training is not only more compute efficient but gives a new best single model result on SQuAD v2 benchmark! 6 Mar 2020 ELECTRA: 88.716 EM 91.365 F1 By @clark_kev @lmthang @quocleix @chrmanning

2

72

312

Stanford NLP Group

@stanfordnlp

5 years

Some people say that no one reads PhD dissertations any more. But (literally!) thousands of people wanted a copy of ⁦ @danqi_chen ’s recent dissertation, Neural Reading Comprehension and Beyond. Piece by ⁦ @feefifofannah ⁩ #nlproc

2

72

304

Stanford NLP Group

@stanfordnlp

6 years

Since 2016, SQuAD has been the key textual question answering benchmark, used by top AI groups & featured in AI Index——Today @pranavrajpurkar , Robin Jia & @percyliang release SQuAD2.0 with 50K unanswerable Qs to test understanding:

4

126

301

Stanford NLP Group

@stanfordnlp

3 years

Stanford CS224N Natural Language Processing with Deep Learning is gearing up for its 2021 edition—starting Tue Jan 12, 4:30 Pacific for enrolled students. New lectures—transformers, LMs & KBs, new assignments—transformers, Choctaw NMT. #NLProc

3

50

295

Stanford NLP Group

@stanfordnlp

4 years

5 NLP Libraries Everyone Should Know | by Pawan Jain | Jul, 2020 | Towards Data Science: spaCy, NLTK, Transformers, Gensim, and Stanza

3

99

288

Stanford NLP Group

@stanfordnlp

5 years

We’ve released a new Visual Question Answering dataset to drive progress on real-image relational/compositional visual and linguistic understanding: GQA Questions, answers, images, and semantics available; will be used as a track in the VQA Challenge 2019.

6

116

279

Stanford NLP Group

@stanfordnlp

5 years

Rasa open source chatbot API: Enterprises & customers want AI assistants, not FAQ chatbots, but they’re difficult to build; Since Google demoed Duplex, every developer, product manager, and executive wants their own that can handle contextual conversations

1

92

277

Stanford NLP Group

@stanfordnlp

4 years

We’ve just released Stanford CoreNLP v4.0.0, a new version of our widely used Java #NLProc package, after a long gap! Some big changes—i.e., compatibility problems but great for the future. Lots of bug fixes, probably a few new bugs, adds French NER.

CoreNLP

High-performance human language analysis tools, now with native deep learning modules in Python, available in many human languages.

stanfordnlp.github.io

5

73

260

Stanford NLP Group

@stanfordnlp

7 years

All lectures & materials for CS224N: NLP with Deep Learning by @chrmanning & @RichardSocher :

Lecture Collection | Natural Language Processing with Deep Learning (Winter 2017)

Natural language processing (NLP) deals with the key artificial intelligence technology of understanding complex human language communication. This lecture s...

www.youtube.com

2

121

261

Stanford NLP Group

@stanfordnlp

4 years

In case you haven’t heard, the new unit for measuring computation runtime is TPU core years. But, if you missed that memo, since the numbers are already in the hundreds, you may as well get ahead of the game and start quoting your runtimes in TPU core centuries #NLProc

4

75

255

Stanford NLP Group

@stanfordnlp

4 years

Natural Language Inference (NLI) over tables by @WilliamWangNLP et al. Tables are a ubiquitous but little studied human information source stuck between text and structured data—though see semantic parsing work, e.g., by @IcePasupat

1

101

254

Stanford NLP Group

@stanfordnlp

5 years

On the importance of good, appropriate data sets for #NLProc , but how “the system” tends to wrongly incentivize researchers.

Do We Encourage Researchers to Use Inappropriate Data Sets?

I’m beginning to think that in some ways the NLP community *encourages* researchers to use poor-quality or otherwise inappropriate data sets. Which is a truly depressing thought…

ehudreiter.com

1

90

243

Stanford NLP Group

@stanfordnlp

4 months

There is a lovely, warm, and enthusiastic writeup of Direct Preference Optimization (DPO) by @rm_rafailov , @archit_sharma97 , and @ericmitchellai from @NeurIPSConf 2023, leading this week’s issue of The Batch newsletter. Thanks, so much, @AndrewYNg !

2

52

244

Stanford NLP Group

@stanfordnlp

5 years

. @GoogleAI ’s BERT unleashed a new performance level on SQuAD 2.0 QA —top 7 systems now all use it and are 2%+ above non-BERT systems. Scores equal summer 2017 SQuAD 1.0 scores. But top HIT/iFLYTEK lab AoA system is now >2% better than raw BERT. HT @KCrosner

0

71

244

Stanford NLP Group

@stanfordnlp

5 years

Implementing a Natural Language Classifier in iOS Swift, which runs fully on device, with Keras + Core ML

4

88

236

Stanford NLP Group

@stanfordnlp

5 years

How can computers answer questions with multi-step information needs? How can it be done efficiently and interpretably? @qi2peng2 and colleagues explain at #emnlp2019 . Paper: Blog post: #NLProc

Answering Complex Open-domain Questions at Scale

The Stanford AI Lab (SAIL) Blog is a place for SAIL students, faculty, and researchers to share our work with the general public.

ai.stanford.edu

1

90

237

Stanford NLP Group

@stanfordnlp

3 years

Not only is @huggingface now hosting all our 🪶Stanza models (via @github LFS)—more reliable than our old fileserver, thx!—but thanks to work by @mervenoyann , 🤗🙏 you can now try out models in the browser using their Hosted Inference API: . #NLProc

2

54

238

Stanford NLP Group

@stanfordnlp

1 year

Is this the end of @Google / @DeepMind as leading presences at machine learning conferences? @JeffDean said: Things had to change. Google would take advantage of its own AI discoveries, sharing papers only after the lab work had been turned into products.

Google shared AI knowledge with the world — until ChatGPT caught up

Google's research teams helped create the AI revolution. Now executives want to make sure the company takes advantage of that knowledge first.

www.washingtonpost.com

10

56

235

Stanford NLP Group

@stanfordnlp

5 years

Here’s another intro tutorial on using our new Python stanfordnlp package by @angelsalamanca –

2

68

224

Stanford NLP Group

@stanfordnlp

5 years

This is very nice work! (Though in full disclosure, we should note that @ethayarajh is turning up at Stanford in the Fall.)

Kawin Ethayarajh

@ethayarajh

5 years

When and why does king - man + woman = queen? In my #ACL2019 paper with @DavidDuvenaud and Graeme Hirst, we explain what conditions need to be satisfied by a training corpus for word analogies to hold in a GloVe or skipgram embedding space. 1/4 blog:

4

117

403

2

46

226

Stanford NLP Group

@stanfordnlp

9 years

All slides from Montreal Deep Learning Summer School available at . 9 hours of #nlproc , 3 by @chrmanning #dlearn

Deep Learning Summer School 2015 - Schedule

Lectures are in room 1140 (ground floor) of Pavillon Aisenstadt (2920 chemin de la tour). First day of lectures: Monday August 3 Last day of lectures: Wednesday August 12 There will be lectures...

sites.google.com

2

144

222

Stanford NLP Group

@stanfordnlp

7 years

Non-convex optimization keeps looking easier and easier….

First-order Methods Almost Always Avoid Saddle Points

We establish that first-order methods avoid saddle points for almost all initializations. Our results apply to a wide variety of first-order methods, including gradient descent, block coordinate...

arxiv.org

2

82

220

Stanford NLP Group

@stanfordnlp

4 years

It’s the 10th anniversary of @stanfordnlp on @Twitter , and approximately the 20th anniversary of the Stanford NLP Group and to celebrate … well, actually that's all coincidental, but at any rate, we've got a new logo!!! By @digitalbryce . #NLProc

4

30

210

Stanford NLP Group

@stanfordnlp

6 years

There is increasing convergence between this decade’s neural models and last decade’s probabilistic graphical models…

Graham Neubig

@gneubig

6 years

New #ACL2018 paper "Neural Factor Graph Models for Cross-lingual Morphological Tagging" Not just for morphology, but a powerful & interpretable tool for sequence labeling that integrates graphical models and neural networks! Code:

2

43

154

0

65

210

Stanford NLP Group

@stanfordnlp

4 years

tf.keras in ⁦ @TensorFlow ⁩ 2.1 adds TextVectorization layer to flexibly map raw strings to tokens/word pieces/ngrams/vocab. An image is just a matrix of numbers but text always needs extra work and it‘s cleaner having preprocessing inside the model 👍

Release TensorFlow 2.1.0 · tensorflow/tensorflow

Release 2.1.0 TensorFlow 2.1 will be the last TF release supporting Python 2. Python 2 support officially ends an January 1, 2020. As announced earlier, TensorFlow will also stop supporting Python ...

github.com

2

64

207

Stanford NLP Group

@stanfordnlp

5 years

What’s new in @Stanford CS224N Natural Language Processing with Deep Learning for 2019? Question answering—1D CNNs—subword models—contextual word representations—transformers—generation—bias. YT playlist – CS224N online hub #NLProc

1

55

211

Stanford NLP Group

@stanfordnlp

3 years

🆕 We've released Stanza v1.2—our Python neural NLP toolkit for PoS, parsing, NER for dozens of human languages. UD 2.7 models, multi-document support, faster tokenization, fix race conditions, fixes “data gap bugs” with tokenization & deps in many las

Stanza

High-performance human language analysis tools, now with native deep learning modules in Python, available in many human languages.

stanfordnlp.github.io

0

70

199

Stanford NLP Group

@stanfordnlp

7 years

Do neural reading comprehension systems really understand? Robin Jia & Percy Liang’s @emnlp2017 paper suggests “No”

1

102

198

Stanford NLP Group

@stanfordnlp

5 years

What Does BERT Look At? An Analysis of BERT's Attention. Kevin Clark @clark_kev , Urvashi Khandelwal @ukhndlwl , Omer Levy @omerlevy_ , Christopher D. Manning @chrmanning .

What Does BERT Look At? An Analysis of BERT's Attention

Large pre-trained neural networks such as BERT have had great recent success in NLP, motivating a growing body of research investigating what aspects of language they are able to learn from...

arxiv.org

Jelle Zuidema (@[email protected])

@wzuidema

5 years

Congratulations, @clark_kev , @chrmanning and colleagues with this year's #BlackboxNLP best paper award!

0

3

16

2

58

193

Stanford NLP Group

@stanfordnlp

6 years

Cross-View Training—A semi-supervised learning technique by @clark_kev @lmthang Quoc Le @chrmanning at #emnlp2018 . Allows training for your #NLProc task on large-scale unannotated data, not only using such data for task-agnostic word representations

1

71

187

Stanford NLP Group

@stanfordnlp

3 years

We’re very excited to kick off our 2021 Stanford NLP Seminar series with Ian Tenney ( @iftenney ) of Google Research presenting on “BERTology and Beyond”! Thursday 10am PT. Open to the public non-Stanford people register at

2

48

189

Stanford NLP Group

@stanfordnlp

7 years

Complete code for a starter neural chatbot from @chipro ’s CS20SI @Stanford class on TensorFlow for Deep Learning

0

70

182

Stanford NLP Group

@stanfordnlp

2 years

Looking for a series to binge-watch with more depth? We are delighted to make available the latest CS224N: Natural Language Processing with Deep Learning. New content on transformers, pre-trained models, NLG, knowledge, and ethical considerations. #NLProc

1

42

185

Stanford NLP Group

@stanfordnlp

6 years

You know how to do NLP. But do you consider fairness and ethical implications in your #NLProc research? Learn the latest on Socially Responsible NLP from Yulia Tsvetkov, Vinod Prabhakaran and @rfpvjr —Jun 1 afternoon @NAACLHLT tutorial.

6

72

178

Stanford NLP Group

@stanfordnlp

6 years

GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations—learning a dependency graph to do deep transfer learning. Jake Zhao: “Perhaps this can also be seen as encoding some relational inductive bias into the machinery” 🤔 HT @ylecun

GLoMo: Unsupervisedly Learned Relational Graphs as Transferable...

Modern deep transfer learning approaches have mainly focused on learning generic feature vectors from one task that are transferable to other tasks, such as word embeddings in language and...

arxiv.org

0

54

181

Stanford NLP Group

@stanfordnlp

4 years

Wonderful to see some theory behind the great success of self-supervised learning. Still trying to get our slow brains around how strong the results are. Cameo for the Stanford Sentiment Treebank—can it become the MNIST of #NLProc ? By @jasondeanlee & al.

1

34

179

Stanford NLP Group

@stanfordnlp

6 years

An introduction to Visual Question Answering, with good discussion of the limitations of some of the current datasets.

2

80

180

Stanford NLP Group

@stanfordnlp

4 years

. @stanfordnlp people’s #ICLR2020 papers #1 — @ukhndlwl and colleagues (incl. at @facebookai ) show the power of neural nets learning a context similarity function for kNN in LM prediction—almost 3 PPL gain on WikiText-103—maybe most useful for domain transfer

0

43

181

Stanford NLP Group

@stanfordnlp

4 years

. @stanfordnlp people’s #ICLR2020 papers #2 —ELECTRA: @clark_kev and colleagues (incl. at @GoogleAI ) show how to build a much more compute/energy efficient discriminative pre-trainer for text encoding than BERT etc. using instead replaced token detection

3

63

179

Stanford NLP Group

@stanfordnlp

4 months

A new winner on the @huggingface Open LLM Leaderboard at the end of December … combining the goodness of SOLAR-10.7B and Direct Preference Optimization (DPO)

GitHub - KyujinHan/Sakura-SOLAR-DPO: Sakura-SOLAR-DPO: Merge, SFT, and DPO

Sakura-SOLAR-DPO: Merge, SFT, and DPO. Contribute to KyujinHan/Sakura-SOLAR-DPO development by creating an account on GitHub.

github.com

2

37

177

Stanford NLP Group

@stanfordnlp

3 years

“The recent phenomenal success of language models has reinvigorated machine learning research…. One problem class that has remained relatively elusive however is purposeful adaptive behavior…. we show that it can be resolved by treating actions as causal interventions.”

Nando de Freitas 🏳️‍🌈

@NandoDF

3 years

Shaking the foundations: delusions in sequence models for interaction and control. I learned so much from Pedro Ortega in this thought-provocative AI project. Great way to spend time with a friend at a London pub.

5

35

289

1

27

176

Stanford NLP Group

@stanfordnlp

8 years

Just out: SQuAD—a 100,000+ question reading comprehension dataset—real text, Qs; leaderboard

1

113

172

Stanford NLP Group

@stanfordnlp

8 years

Learning machine learning in a year: “Stanford’s CS224D [Deep Learning for NLP] … is a fantastic course. ”

Machine Learning in a Year

From being a total ml noob to start using it at work

medium.com

0

78

166

Stanford NLP Group

@stanfordnlp

5 years

The need for a quality site recording the state-of-the-art performance on many AI/ML/NLP/Vision tasks has been obvious for a decade—this one looks great and might actually achieve escape velocity!

Papers with Code

@paperswithcode

5 years

We’ve just released the new Papers With Code! Site now has over 950+ ML tasks, 500+ evaluation tables (including state of the art results) and 8500+ papers with code. Explore the resource here: . Have fun!

21

811

2K

0

51

171

Stanford NLP Group

@stanfordnlp

4 years

The return of nearest neighbor models—or memory-based learning—to #NLProc : @ukhndlwl , Angela Fan, @jurafsky , @LukeZettlemoyer & @ml_perception report strong gains on neural MT, especially for domain adaptation in paper:

0

55

168

Stanford NLP Group

@stanfordnlp

4 years

At #acl2020nlp , @mhahn29 presents TACL paper Theoretical Limitations of Self-Attention in Neural Sequence Models: Transformer models seem all-powerful in #NLProc but they can’t even handle all regular languages—what does this say about human language?

3

39

165

Stanford NLP Group

@stanfordnlp

3 years

The result that the multimodal models much more effectively use vision for language than language for vision is intriguing (and somewhat surprising).

Emanuele Bugliarello

@ebugliarello

3 years

Is your Vision-and-Language model really a Vision-AND-Language model? 👀 “Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformers” 📄 🗣️ #EMNLP2021

3

42

179

2

31

166

Stanford NLP Group

@stanfordnlp

3 years

🆕🪶Stanza 1.3 for Python #NLProc is out with a new language ID component and multilingual pipelines, a new transition-based constituency parser, a dictionary tokenizer feature esp. useful for East Asian languages, and model downloading from @huggingface .

Stanza

High-performance human language analysis tools, now with native deep learning modules in Python, available in many human languages.

stanfordnlp.github.io

3

38

167

Stanford NLP Group

@stanfordnlp

7 years

“computational people are increasingly turning to linguistic problems, because they are among the hardest problems”

Why literature is the ultimate big-data challenge

In a few decades, statistical analysis of literature has gone from crackpot theorising to cutting-edge research

www.economist.com

2

149

161

Stanford NLP Group

@stanfordnlp

6 years

Sharp Nearby, Fuzzy Far Away: An LSTM Neural Language Model remembers out to about 200 words, remembering word order for about 50 words & more results…. At #ACL2017 by @ukhndlwl , @hhexiy , Peng Qi & @jurafsky . #NLProc

0

57

161

Stanford NLP Group

@stanfordnlp

5 years

“He and her team, which included Nanyun Peng and Percy Liang, tried to give their AI some creative wit, using insights from humor theory.” ⁦ @hhexiy ⁩ ⁦ @percyliang ⁩. The Comedian Is in the Machine. AI Is Now Learning Puns | WIRED

The Comedian Is in the Machine. AI Is Now Learning Puns

Researchers are using artificial intelligence techniques to create puns. Today's aren't so funny, but tomorrow's might be better.

www.wired.com

1

53

164

Stanford NLP Group

@stanfordnlp

6 years

Nice article on methods for using distributed representations to capture graph structure in @gradientpub , a new, accessible magazine by @Stanford AI students. The first methods drew from #NLProc but maybe with new GCN methods, we’re borrowing back.

0

58

159

Stanford NLP Group

@stanfordnlp

6 years

Three goals of human-centered AI: capturing the breadth & nuance of human intelligence, AI that enhances & collaborates with humans, and guiding the effects of AI on human society. By @drfeifei

3

86

159

Stanford NLP Group

@stanfordnlp

4 years

Just out in PNAS: A paper examining the emergent linguistic structure learned by artificial neural networks, such as BERT, trained by cloze task (word-in-context prediction) self-supervision. By @chrmanning , @clark_kev , @johnhewtt , @ukhndlwl , @omerlevy_ .

3

52

159

Stanford NLP Group

@stanfordnlp

6 months

“We believe that natural language is going to be a big part of how people use computers in the future” — @sama

7

23

161

Stanford NLP Group

@stanfordnlp

2 years

TIL: @NVIDIAAI has an efficient, extensively benchmarked, and well-maintained version of our ELECTRA model—an efficient BERT-equivalent large pre-trained language model—for tf2, which exploits NVIDIA tensor cores, mixed precision training, etc. #NLProc

1

25

157

Stanford NLP Group

@stanfordnlp

6 years

Even more #NLProc QA data at #emnlp2018 ! HotpotQA—a Wikipedia-based dataset requiring multi-document information aggregation and comparisons by Zhilin Yang @qi2peng2 @Saizheng Bengio @professorwcohen @rsalakhu @chrmanning . Paper, data, leaderboard, etc.:

2

64

157

Stanford NLP Group

@stanfordnlp

5 years

New work from @bradenjhancock working with @jaseweston and @AntoineBordes at @facebookai shows how a chatbot can improve itself after deployment: “Learning from Dialogue After Deployment: Feed Yourself, Chatbot!”

0

50

153

Stanford NLP Group

@stanfordnlp

3 years

An easy, bad PyTorch coding mistake: If you do Dataset preprocessing in the __getitem__ method using an np.random & use a multithreaded DataLoader, then each thread gets the same seed! Instead, set the seed in DataLoader’s worker_init_fn. HT @peteskomoroch

Using PyTorch + NumPy? You're making a mistake.

A bug that plagues thousands of open-source ML projects.

tanelp.github.io

0

24

159

Stanford NLP Group

@stanfordnlp

4 years

We’ve just released Stanza v1.1.1, our #NLProc package for many human languages. It adds sentiment analysis, medical English parsing & NER, more customizability of Processors, faster tokenizers, new Thai tokenizer, bug fixes, etc.—try it out!

Stanza

High-performance human language analysis tools, now with native deep learning modules in Python, available in many human languages.

stanfordnlp.github.io

3

48

156

Stanford NLP Group

@stanfordnlp

11 months

“the researchers [that’s us!] show that the cross-entropy loss for fitting the reward model in RLHF can be used directly to finetune the LLM. In benchmarks it's more efficient to use DPO and often also preferred over RLHF/PPO in terms of response quality.”

AI Research Highlights May-June 2023: Direct-Preference Optimization for Human Feedback and More

This article is a compilation of 23 AI research highlights, handpicked and summarized. A lot of exciting developments are currently happening in the fields of natural language processing and computer...

magazine.sebastianraschka.com

2

31

153