Najoung Kim 🫠 @najoungkim Twitter profile

Pinned Tweet

Najoung Kim 🫠

2 months

🧙🍪🧙‍♀️ I'm hiring a postdoc with @sebschu to start in Fall 2024! We are looking for someone with experience in EITHER: (1) building systems that use language models as a core component to solve complex tasks, or (2) leading human annotation/behavioral experiments.

2

28

79

Last Seen Profiles

@SchechterA26644

@rich_bikini

@sixsevenAQ

@stw_pdg

@fubuki84599344

@MartinaMarkota

@glasses_no

@xandfps

@wolverinasbb

@pleinSolei

@TheAmerican_Mom

@devoanderson_

@KeithWolter

@HilbornLor52689

@kirsten_tempest

@saquon

@AWHJASMIN

@dapperlabs

@bryson_warsap

@CaledoniaBoysBB

@illillif

@najamein

@SirRubyx1

@Angel6771295401

@waleed771234

@rodrigo_marciel

@ZekkoFin

@hooly_Cas

@LilBonaX

@K4_LINK

@LBis_here

@mariasolias

@babadu_b

@a_kupatadze

@zlodziejbraw

@educatedear

Najoung Kim 🫠

@najoungkim

2 years

prompting is when you are the one who's being finetuned instead of the model

5

54

546

Najoung Kim 🫠

@najoungkim

2 years

People who link arxiv abs instead of pdf ❤

10

20

510

Najoung Kim 🫠

@najoungkim

3 years

Is language even real

31

36

367

Najoung Kim 🫠

@najoungkim

3 years

It's a strange time to be sharing good news, but I'll be joining @BULinguistics as an Assistant Professor in Spring 2022! Until then, I'll be at @NYUDataScience as a Faculty Fellow. Looking forward to doing things and connecting with folks in the area 😊

33

8

330

Najoung Kim 🫠

@najoungkim

2 years

you think you're training your transformers but maybe the transformers are training us to create more of them

7

25

306

Najoung Kim 🫠

@najoungkim

3 years

Dr.! 🌝

3

1

177

Najoung Kim 🫠

@najoungkim

5 years

Why do grad students find Bayesian inference hard? Because they don't have P(H|D)s

4

15

163

Najoung Kim 🫠

@najoungkim

2 years

imply no such work has been done before and people will do lit review for you

5

7

156

Najoung Kim 🫠

@najoungkim

5 months

i'm co-hosting a student researcher at @GoogleAI w/ Deepak Ramachandran! We're looking for folks interested in AI assisted creativity, re-imagining interaction as a co-creation process. Background in HCI or human exps + experience with generative models would be a good fit 🧵👇

5

28

137

Najoung Kim 🫠

@najoungkim

10 months

I dislike the [1],[2] citation format because it kills any possibility to do distributional learning of refs. Sure, I might not have read X et al. (1997) but I kinda know the vibes bc it was cited this way. But for "blah [4-7]" there's no way I'm scrolling down to see what [6] is

6

8

134

Najoung Kim 🫠

@najoungkim

6 months

🧙‍♀️ I'm hoping to recruit ~1 PhD student this cycle through @BULinguistics ! Students who are broadly interested in meaning and computational models would be a good fit. I'll mention a few specific topics I've been working on & looking to expand below:

3

45

133

Najoung Kim 🫠

@najoungkim

3 years

If you're applying to grad school this cycle, here's a plug for BU Linguistics ( @BULinguistics )! I plan to participate in recruitment this year and would love to see applications from students who share my interests at the intersection of Ling/CogSci and NLP. _Very_ broadly,

3

28

117

Najoung Kim 🫠

@najoungkim

6 months

thrilled to announce that i've made it to forbes 8 billion under 8 billion

4

5

117

Najoung Kim 🫠

@najoungkim

2 years

How does one retain low-level details of papers that you've actually read (not skimmed) in one's head or is it just infinite re-reading until you die

24

3

115

Najoung Kim 🫠

@najoungkim

4 years

Our #EMNLP2020 paper (with @tallinzen ) is now on arXiv. Here is a summary: (1/9)

COGS: A Compositional Generalization Challenge Based on Semantic...

Natural language is characterized by compositionality: the meaning of a complex expression is constructed from the meanings of its constituent parts. To facilitate the evaluation of the...

arxiv.org

3

19

115

Najoung Kim 🫠

@najoungkim

2 years

🧙‍♀️ I'll be at @GoogleAI part-time for the next year, starting the week after NAACL (NYC > Cambridge office). Let me know if you want to chat about meaningology-oriented model eval, linguistic generalization, how multimodal learning shapes language, or just want to say hi! 💖

8

2

114

Najoung Kim 🫠

@najoungkim

1 year

New year new job

5

1

108

Najoung Kim 🫠

@najoungkim

1 year

where is slidesgpt

5

4

107

Najoung Kim 🫠

@najoungkim

2 years

One final thought: efforts from non-native English speaker attendees from non-English speaking institutions deserve a ton of recognition. I think it often goes without appreciation so just wanted to comment on it! Has always been true but it seems so much more salient in person

2

5

99

Najoung Kim 🫠

@najoungkim

2 years

I've never plagiarized (or so I believe) but every time the issue comes up a part of me worries what if I've unconsciously regurgitated some source text without realizing or my paraphrasing was subpar. Maybe I've also read this tweet somewhere

5

0

94

Najoung Kim 🫠

@najoungkim

1 year

🦔 New preprint 🦔 Lots of work has been done in the compositional generalization space recently, using tests such as SCAN and COGS. Many models actually do achieve impressive performance, some of them almost perfect lexical generalization on COGS. (1/n)

3

15

94

Najoung Kim 🫠

@najoungkim

1 year

I was shocked to find out that some of my friends in grad school could just "do the tasks" rather than putting themselves in eternal suffering due to the looming tasks. Still in hopes that one day i'll become an exec function queen but the day is not today

Lilly

@lillybilly299

1 year

My sister (executive function queen) was like "sure tasks suck, but doing the task sucks less than the sinking anxiety feeling of a looming task, so I just get it out of the way asap. I hate feeling like that." Um hello??

10

23

756

2

3

83

Najoung Kim 🫠

@najoungkim

9 months

yeah kind of, sorry about that

4

2

81

Najoung Kim 🫠

@najoungkim

3 years

Is an LSTM more human than me if I have ~50% error rate on subject-verb agreement when there's a linearly closer attractor

3

5

76

Najoung Kim 🫠

@najoungkim

2 years

Opinion: it'd be nice if (big) multimodal models reported unimodal performance too! In the grand scheme of things, I find it hard to imagine that the goal of building multimodal models is only being good at multimodal tasks. Ideally they would apply to single modalities too (1/n)

4

5

76

Najoung Kim 🫠

@najoungkim

3 years

Since I received no replies regarding what "a profile photo" (is it my pic, a pic of all coauthors, a representative fig of work?) for a talk is from ACL, I'm just going to upload a pic of my cat

2

0

75

Najoung Kim 🫠

@najoungkim

7 months

🦔Announcing SLOG, a new semantic parsing benchmark for structural generalization, to be presented at EMNLP! Work with @BingzhiLi , @lucia_elizabe , @alkoller , @tallinzen , @yuekun_yao .

3

13

67

Najoung Kim 🫠

@najoungkim

11 months

Our entity tracking paper (w/ @sebschu ) got the Area Chair award for the Interpretability & Analysis track! 💖💖 This makes us happy, but does this mean anything? How many bits will you take away, if any? Find out today (7/11) at the 16:15 poster session at #ACL2023NLP ! (1/2)

Sebastian Schuster

@sebschu

1 year

Following long contexts in natural language requires tracking what happens to entities that have been mentioned. Can LLMs do this as new information about them is presented in text? In @najoungkim ’s and my #ACL2023 paper, we present a behavioral task that targets this question.

1

19

95

2

3

64

Najoung Kim 🫠

@najoungkim

2 years

Can't believe 2017 was 10 years ago

6

1

64

Najoung Kim 🫠

@najoungkim

1 year

🦷 Another preprint 🦷 Information-seeking Qs often contain questionable assumptions that models should be robust to. "When did Marie Curie discover Uranium?" is an example. We propose (QA)^2, a test set evaluating the capacity to handle such Qs. (1/n)

(QA)$^2$: Question Answering with Questionable Assumptions

Naturally occurring information-seeking questions often contain questionable assumptions -- assumptions that are false or unverifiable. Questions containing questionable assumptions are...

arxiv.org

4

10

58

Najoung Kim 🫠

@najoungkim

1 year

🔮 #BlackboxNLP will be back in 2023 at #EMNLP2023 ! ❄ We will keep updates posted on our website: While you wait, also check out our YouTube channel: (For preview: )

BlackboxNLP 2024

Workshop on analyzing and interpreting neural networks for NLP

blackboxnlp.github.io

1

9

56

Najoung Kim 🫠

@najoungkim

2 months

Super excited (maybe thrilled even) to be recruiting my first postdoc soon with @sebschu to work on a Very Ambitious Eval Project! Details to be announced soon ❤️🧚‍♀️

5

4

57

Najoung Kim 🫠

@najoungkim

1 year

We can't make claims about generalization without knowing what's in the training data!

MMitchell

@mmitchell_ai

1 year

Make no mistake, it is COOL that a text model can learn how to draw a unicorn from words and Tikz code!!!! The "nit" is that this is resultant, expected, working-as-intended behavior, not emergent imagination.

1

14

150

0

7

55

Najoung Kim 🫠

@najoungkim

2 years

yyyy/mm/dd is the only good ordering

4

5

56

Najoung Kim 🫠

@najoungkim

2 years

Cat and human in cap

2

0

55

Najoung Kim 🫠

@najoungkim

5 years

The cols other than BERT are comparing models with the same architecture pretrained on different tasks. One finding is that language modeling is a good pretraining task (as previously known), but CCG supertagging is narrowly better even with much smaller # of training examples.

0

11

52

Najoung Kim 🫠

@najoungkim

4 years

Maybe I'm missing something, but why can't coding/ML interviews be open-book/search? The interviewer can still ask probing questions about the solution, potential problems, and many other things that need to be explained in your own words

5

6

51

Najoung Kim 🫠

@najoungkim

2 years

took me 6 years in the US to start saying "awesome" unironically but would like to stop

6

1

48

Najoung Kim 🫠

@najoungkim

10 months

the only publication venue that matters for dissemination is probably arxiv (+independent PR), and confs are mostly for fun & post hoc glory for ppl who need/want them

(((ل()(ل() 'yoav))))👾

@yoavgo

10 months

had the transformers paper been accepted to acl, would it have been discovered by the mlers? after how long?

7

0

33

1

0

47

Najoung Kim 🫠

@najoungkim

1 year

Death is ongoing but the semester is over and I'm officially 0.5 yrs old as faculty! It was only 3x as crazy as I imagined. I did manage to get some things done this sem, surprisingly. My proudest achievement 👇

1

0

47

Najoung Kim 🫠

@najoungkim

3 years

First day at new job, maybe a good time to make peace with the fact that everything I ever say (that I don't intend to be wrong) is probably wrong

3

0

46

Najoung Kim 🫠

@najoungkim

4 years

Let's see if anyone can escape

1

7

46

Najoung Kim 🫠

@najoungkim

1 year

Can't help being salty about academia things, but one day I will become famous (not necessarily academic reasons, e.g., cat might develop language) and none of it will matter

4

1

44

Najoung Kim 🫠

@najoungkim

10 months

thrilled to announce that i hate the idea of ARR only

2

0

44

Najoung Kim 🫠

@najoungkim

11 months

Leaving Korea for #ACL2023NLP soon. Old friends, new friends, vague acquaintances, we should catch up! If I don't know you but if you want to chat (or see pics of C), feel free to reach out or grab me at the conf ❤️

5

0

43

Najoung Kim 🫠

@najoungkim

2 years

Strong "Ask not what your model can do for you, ask what you can do for your model" vibes from the "but it works if you just find the right way to prompt it" trend

1

5

42

Najoung Kim 🫠

@najoungkim

7 months

it really is nice to get an extra hour of sleep! what if we set back the clock by an hour every single day? surely this would be a net benefit

0

42

Najoung Kim 🫠

@najoungkim

3 years

If the world wanted me to be productive it would simply not produce content I can passively consume for all of eternity

1

0

40

Najoung Kim 🫠

@najoungkim

6 months

Belated thread on updates to the "Inverse scaling can become U-shaped" paper; so belated that I'm now presenting it at EMNLP :) Takeaways: (1) Inv scaling curves are hard to extrapolate (2) Even minimal demonstrations (1-shot) are critically effective as a mitigation strategy

1

5

40

Najoung Kim 🫠

@najoungkim

1 year

wow 2023 is almost over

8

3

40

Najoung Kim 🫠

@najoungkim

2 years

yay dalle access! #NAACL2022 brings good things

4

1

40

Najoung Kim 🫠

@najoungkim

4 months

happy that the first ugrad i've worked with here got a research award (smol)

0

40

Najoung Kim 🫠

@najoungkim

2 years

Cookie is the absolute best cat and I will raise him until he learns language. Might delete later in case he reads this

2

0

39

Najoung Kim 🫠

@najoungkim

2 years

but where is hugged face

3

0

39

Najoung Kim 🫠

@najoungkim

1 year

Maybe a real AI risk is melted brains due to the flood of bad takes about AI

2

4

39

Najoung Kim 🫠

@najoungkim

7 months

was it ever the case that presenters were charged more than non-presenters? strange incentive structure

11

1

39

Najoung Kim 🫠

@najoungkim

3 years

Another exciting QA result in the wild! (this time w/o presupposition failure)

1

37

Najoung Kim 🫠

@najoungkim

2 years

First nonstudent attempt to attend *CL conf but $775 for naacl early registration excluding membership fee? cursed

1

0

38

Najoung Kim 🫠

@najoungkim

5 months

so nice meeting friends and friends i've never met in person before! now back to not meeting them again, bye forever

2

0

38

Najoung Kim 🫠

@najoungkim

3 years

tfw experiments don't replicate after a package update

4

0

37

Najoung Kim 🫠

@najoungkim

2 years

Wrote a paper last year at Google and apparently its existence changed the highlighted search result for the Q "which linguist invented the lightbulb" to be more meta but still failing in the same way as described in the paper 😊 (thanks @sebschu for noticing!)

0

37

Najoung Kim 🫠

@najoungkim

6 months

personal update: 2nd cds faculty fellow position i've held! (yet i still can't articulate well what data science actually is)

2

0

37

Najoung Kim 🫠

@najoungkim

1 year

Cool but should have been called ChattingFace

Julien Chaumond

@julien_c

1 year

Some people said that closed APIs were winning... but we will never give up the fight for open source AI ⚔️⚔️ Today is a big day as we launch the first open source alternative to ChatGPT: HuggingChat 💬 Powered by Open Assistant's latest model – the best open source chat

192

1K

4K

1

36

Najoung Kim 🫠

@najoungkim

3 years

Glad/Sad to see the presupposition failure example I've been using in talks is still going strong in Macaw ()! Plug, although we couldn't include the Zuck example in the paper :/

2

0

35

Najoung Kim 🫠

@najoungkim

15 days

Will be giving a talk at this workshop alongside very cool speakers and panelists. Thanks for having me (=just a little guy)! Will contribute ~reasonable takes

Roger Levy

@roger_p_levy

18 days

Join us online for the May 13–14 for a star-studded #NSF -sponsored workshop: New Horizons in Language Science: Large Language Models, Language Structure, and the Cognitive & Neural Basis of Language! Interdisciplinary talks & discussion on three themes: 1/

4

86

225

1

36

Najoung Kim 🫠

@najoungkim

11 months

Thrilled to announce that my teaching evals are not bad

1

0

35

Najoung Kim 🫠

@najoungkim

2 years

We'd love to hear from (I'd personally be excited to hear from) folks who study language outside of ML! (linguistics/cog sci/psych/development...) If you have reasons to believe that scaling up won't solve important problems in language, consider participating 🧙‍♀️

Ethan Perez

@EthanJPerez

2 years

We’re announcing the Inverse Scaling Prize: a $100k grand prize + $150k in additional prizes for finding an important task where larger language models do *worse*. Link to contest details: 🧵

48

312

2K

1

3

34

Najoung Kim 🫠

@najoungkim

3 years

My OPT got approved today 🙏 I'm probably one of the first batches of ppl to file online, and it took almost exactly 3 months, in case anyone finds this info helpful

0

1

34

Najoung Kim 🫠

@najoungkim

10 months

My top-n culture shock moment in the US includes the time when i learned Barbie is short for Barbara

6

0

34

Najoung Kim 🫠

@najoungkim

2 years

In boston for the very first time! Cold! Just want these guys to be safe!

5

0

34

Najoung Kim 🫠

@najoungkim

1 year

the quality of the bibs correlates with the quality of the paper

4

2

34

Najoung Kim 🫠

@najoungkim

3 years

Thinking about whether I'm truly a human after failing 8 captchas that looked obvious in a row or if they're secretly trying to extract more labels out of me

1

0

33

Najoung Kim 🫠

@najoungkim

3 years

Free lab name idea: dognition lab (for dog cognition)

0

33

Najoung Kim 🫠

@najoungkim

2 years

I consider myself as L1.5 but I still blunder a lot on turn-taking, cultural/convo norms, lexical retrieval, etc., and it is hard. Always so heartbreaking to hear students apologize for their English, there is no need and it's every discourse participant's job to make it work 🦔

0

33

Najoung Kim 🫠

@najoungkim

2 years

Life is filling out forms until you die

4

1

33

Najoung Kim 🫠

@najoungkim

2 years

It is extremely hard to get DALLE to do what I want it to do, so here's some mediocre art inspired by the donut wall at #NAACL2022 ! It was so nice meeting everyone old & new 💕

0

1

32

Najoung Kim 🫠

@najoungkim

1 month

in making typos in my own name when signing emails stage of my career

3

0

32

Najoung Kim 🫠

@najoungkim

3 years

Why are we going back to counting acknowledgment towards the page limit + having a page limit for the appendix? Why is this changing every other *CL conf?

3

32

Najoung Kim 🫠

@najoungkim

1 year

I really enjoyed working on this project, see our paper for takes not in the thread and stay tuned for substantial updates for the camera-ready! See @sebschu 's thread for a summary. I will add two random facts about the paper:

Entity Tracking in Language Models

Keeping track of how states of entities change as a text or dialog unfolds is a key prerequisite to discourse understanding. Yet, there have been few systematic investigations into the ability of...

arxiv.org

Sebastian Schuster

@sebschu

1 year

Following long contexts in natural language requires tracking what happens to entities that have been mentioned. Can LLMs do this as new information about them is presented in text? In @najoungkim ’s and my #ACL2023 paper, we present a behavioral task that targets this question.

1

19

95

1

5

31

Najoung Kim 🫠

@najoungkim

1 year

BU email defaults to LastName, FirstName as the display name for your email and doesn't let you change this. This is definitely going to solve the everlasting problem of people calling me Kim 🫠

2

0

31

Najoung Kim 🫠

@najoungkim

2 years

tfw T5Tokenizer.from_pretrained('t5-base') != AutoTokenizer.from_pretrained('t5-base') 🥰

2

3

30

Najoung Kim 🫠

@najoungkim

2 years

giving a talk, some familiar stuff plus some new stuff that we're putting on arXiv this week

GC CUNY Linguistics

@CUNY_GCLing

2 years

Join us this Thursday (Nov 3 @ 4:15pm EST) for Dr. Najoung Kim’s talk at the #ComputationalLinguistics series @GC_CUNY ! In-person (room 7102) or via Zoom by registration at: #CompLing #NeuralNets #Semantics #AI

0

1

5

2

30

Najoung Kim 🫠

@najoungkim

3 years

so many papers I'll never read

0

29

Najoung Kim 🫠

@najoungkim

10 months

i personally don't make LLMs cant do X claims but reliability/generality is why i often get annoyed by (1) ppl responding to every failure with o but u just didnt prompt it right & (2) taking prompt engineered results as representative of model capability. Robust 0 shot 🫶🫶

xuan (ɕɥɛn / sh-yen)

@xuanalogue

10 months

I think part of why I'm willing to say stuff "LLMs can't do X" is bc reliability is part of my conception of capability! If LLMs can't reliably perform X according to some behavioral metric in a wide neighborhood of situations, they don't have the general capability for X.

6

22

153

1

2

29

Najoung Kim 🫠

@najoungkim

1 year

To be presented at ACL2023. The dataset's been up for a while but now sharing explicitly:

GitHub - najoungkim/QAQA: Repository for the paper (QA)^2: Question Answering with Questionable...

Repository for the paper (QA)^2: Question Answering with Questionable Assumptions - najoungkim/QAQA

github.com

Najoung Kim 🫠

@najoungkim

1 year

🦷 Another preprint 🦷 Information-seeking Qs often contain questionable assumptions that models should be robust to. "When did Marie Curie discover Uranium?" is an example. We propose (QA)^2, a test set evaluating the capacity to handle such Qs. (1/n)

4

10

58

0

1

29

Najoung Kim 🫠

@najoungkim

2 years

Leaving for #NAACL22 , ❤ being said hi to so come say hi! "Can I see pics of your cat" works too. Will try for novid 🤖

1

0

28

Najoung Kim 🫠

@najoungkim

1 year

become unmodellable (by posting less on social media)

2

1

28

Najoung Kim 🫠

@najoungkim

11 months

I will be presenting (QA)^2 on Qs with questionable assumptions today (7/10) at the 14:00 poster session at #ACL2023NLP ! One important point that is less evident from the thread and the poster is that...

Najoung Kim 🫠

@najoungkim

1 year

🦷 Another preprint 🦷 Information-seeking Qs often contain questionable assumptions that models should be robust to. "When did Marie Curie discover Uranium?" is an example. We propose (QA)^2, a test set evaluating the capacity to handle such Qs. (1/n)

4

10

58

1

4

28

Najoung Kim 🫠

@najoungkim

7 months

aging very well

Najoung Kim 🫠

@najoungkim

1 year

wow 2023 is almost over

8

3

40

1

0

28

Najoung Kim 🫠

@najoungkim

1 year

It pains me that some of the takes make it so so clear that they've never touched an LM (chatgpt doesn't count), and likewise for some where they have 0 idea what linguistics is, but yet make sweeping categorical claims

1

27

Najoung Kim 🫠

@najoungkim

2 years

what if we social mediaed by uploading our takes as pdfs on arXiv

4

0

28

Najoung Kim 🫠

@najoungkim

7 months

getting passive aggressived at by people who I'm providing free work for sure is a great feeling

4

0

28

Najoung Kim 🫠

@najoungkim

3 years

Excited to see a topic I'm working on (presupposition failure in QA) tweeted by xkcd 🥰(h/t @tallinzen )

Randall Munroe

@xkcd

3 years

Learning new things from Google

94

1K

6K

1

2

28

Najoung Kim 🫠

@najoungkim

2 months

i miss the discourse and pragmatics track reviewers so much. they were the voice of reason and light in the dark world. it pains me to even read ARR reviews

2

0

27

Najoung Kim 🫠

@najoungkim

1 year

trying to have some thoughts every day (as opposed to having no thoughts)

2

0

27

Najoung Kim 🫠

@najoungkim

4 months

i know that there are many layers of problems with the required dei statements in faculty apps but if you keep saying "females" in your statement i can't help but get bad vibes! just a tip

2

0

27

Najoung Kim 🫠

@najoungkim

1 year

friendship is infinitely sending each other "dying" back and forth

0

27

Najoung Kim 🫠

@najoungkim

14 days

big if true true therefore, big

2

1

26

Najoung Kim 🫠

@najoungkim

2 years

Looking forward to reading the final version! Taking this opportunity to brain dump thoughts that may or may not be 100% relevant. The Q "Can current ML/NLP inform human language science?" pops up a lot, but every time I think about it there is something unsatisfactory (1/n)

Alex Warstadt

@a_stadt

2 years

Can we learn anything about human language learning from everything that’s going on in machine learning and NLP? In a new position piece Sam Bowman ( @sleepinyourhat ) and I argue the answer is “yes”. …If we take some specific steps. 🧵

2

64

335

2

26