Gytis Daujotas @gytdau Twitter profile

Last Seen Profiles

@Ilha__TV

@satara_za

@hwood60007

@PorvenirOficial

@bacol_bersama

@PriscilaReis_

@CarrPhotograph

@insideloc

@S518253

@janthedude

@kasi_e24579

@CulleySean

@Krizzz_x_

@pip_alchemist

@miria_AMASTAGES

@Pirukah

@yonaka_nemu

@niu_pr

@token

@Mo_tionGaming

@chriskiel_

@joeshortsqueeze

@PrensadeMonagas

@KhumarZingxondo

@sitappa_shooter

@_gee36

@jofelusa

@AytoMieres

@neuralconsult

@SteveDodd

@Marcus_cel

@Worthbaseball

@aboodhammad2003

@ArbitraryAbsent

@VijayBijo

@tonydahl23

Gytis Daujotas

@gytdau

1 month

Surprised at the how stunning illustrations in mech interp papers are. It must take so much sweat and toil to not only do incredible and novel research, but to choose to go beyond, and to make it beautiful too.

6

27

324

Gytis Daujotas

@gytdau

3 years

We built a better way to read ebooks in browsers. Just finished processing every Project Gutenberg book on it! Please check it out and let me know what ya think:

16

66

295

Gytis Daujotas

@gytdau

2 months

How does self-correction affect problem solving? In a toy transformer model that was trained to solve mazes, I found that performance reliably improved (!) by inserting mistakes and self-corrections into the training data.

14

34

205

Gytis Daujotas

@gytdau

9 days

Sculpting an image by mixing and tweaking features - a research preview I'm testing in private beta:

10

14

130

Gytis Daujotas

@gytdau

2 years

Impressive how fast OpenAI APIs are becoming undifferentiated commodities

Scale AI

@scale_AI

2 years

Today at TransformX, we announced a huge step forward for the open source ML community: we are partnering with @StabilityAI to release the first large language model trained with human feedback. 1/4

5

79

493

2

3

36

Gytis Daujotas

@gytdau

3 months

Making two Claudes compose a piano piece together:

4

3

32

Gytis Daujotas

@gytdau

30 days

Building an IDE for interpreting SAE feature activations. Really enjoying the near real-time streaming of activations while editing samples:

1

0

32

Gytis Daujotas

@gytdau

8 days

By training sparse autoencoders on generative image models, we discover an interpretable palette of world concepts

6

17

230

Gytis Daujotas

@gytdau

9 months

Moving to San Francisco has so far been excellent. Everyone is kind and open - and I would recommend it, even if only for a while.

1

0

22

Gytis Daujotas

@gytdau

2 years

New post - thanks so much to @PsalmEnright for the editing!

Ireland’s Unique Promise for Nuclear

The Republic can carve out a niche of good nuclear regulation

www.thefitzwilliam.com

2

3

17

Gytis Daujotas

@gytdau

22 days

Daily gratitude for being in a world where scrolls that look like this can actually be read through the sheer will of human ingenuity

0

1

17

Gytis Daujotas

@gytdau

2 months

You can read more of the writeup on my blog (in my bio)! Thanks @DavidMcsharry3 , @NeelNanda5 , and others for early reviews of this work.

2

0

11

Gytis Daujotas

@gytdau

2 months

I trained probes on the model, and found the residual stream has some representation of mistakes in the residual stream, but not linearly. Future work could focus on other ways of levering down the mistake-making "feature" to make a model which is better all-around.

2

0

9

Gytis Daujotas

@gytdau

4 years

@BrendanEich @ivanbuncic @runhappylife @getify That's Javascript Weekly, though it probably should instead be called JS Weekly.

0

1

8

Gytis Daujotas

@gytdau

30 days

This IDE streams from a cloud GPU using a SAE that I've trained on a NanoGPT instance that learned to solve mazes. I've started by working on transformers that play simple games - to figure out what techniques might transfer over to games that we understand imperfectly, like

1

0

8

Gytis Daujotas

@gytdau

2 years

working on generated interfaces!

1

0

7

Gytis Daujotas

@gytdau

3 years

Over 200k words in my journal and counting. Next step is to train a language model, and start with a prompt like "Journal, today I discovered something incredible..."

1

0

7

Gytis Daujotas

@gytdau

2 years

Working on better ways for people to use language models. Here's an interface that allows you to backtrack and wander through the space of creative possibilities:

1

0

6

Gytis Daujotas

@gytdau

7 years

Thanks @adeoressi for your invaluable advice at @SWDub #SWDub !

0

6

Gytis Daujotas

@gytdau

4 months

The previous Fitzwilliam meetup in Dublin was wonderful, and now -- look! -- an opportunity to attend the next in the series:

Sam Enright

@Sam__Enright

4 months

I am hosting a meetup next Friday the 23rd at 7:30pm in Parnell Heritage Pub in Dublin – hope you can make it!

1

6

21

0

1

6

Gytis Daujotas

@gytdau

4 months

Enjoyed this post from Daniel Dennet: "These days I almost always outsource the hard work of comprehension when I encounter difficulties, and the policy works wonders—for me. Distributed understanding is a real phenomenon, but you have to get yourself into a community of

1

0

6

Gytis Daujotas

@gytdau

2 years

This is a 1-meter sized mirror used to make chips. If you were to blow this mirror up to the size of the Earth, the largest aberration would be the width of a human hair. (!!!)

1

5

Gytis Daujotas

@gytdau

3 years

Bored of having to say 'social media platforms' every time. Cooler, piratey sounding name: the shallows. Has mystique, sense of danger, lawlessness

1

0

5

Gytis Daujotas

@gytdau

3 months

@dogmadeath Here's a hilarious example of this, where two Claudes were composing music, and Claude 2 got a little bit too into it for Claude 1's taste:

1

0

3

Gytis Daujotas

@gytdau

3 months

Two instances of Claude collaborating on a Chopin-inspired barbershop tag:

1

5

Gytis Daujotas

@gytdau

4 months

@andy_matuschak A good friend recommended this high CRI floodlight, which I much prefer to the regular cheaper ones on Amazon:

0

5

Gytis Daujotas

@gytdau

3 years

Wow: "The world’s largest, most valuable tech companies are dependent either directly or indirectly on the steady output of TSMC’s fabs. If [they] went offline [...] it would immediately devastate the global economy. "

2

5

Gytis Daujotas

@gytdau

30 days

One small thing so far: top activations can be surprisingly deceiving! One example: this feature seems like it represents first moves, but it actually encodes location information too. Maybe still in superposition?

1

0

4

Gytis Daujotas

@gytdau

3 years

@atroyn @ArtirKel Three Body Problem is stunning, but I'm not sure if optimistic immediately comes to mind

2

0

4

Gytis Daujotas

@gytdau

2 years

@minney_cat @thesophiaxu @lisawehden +1 to this, Lisa and Minn are great!

0

4

Gytis Daujotas

@gytdau

1 year

@ischanan @patriciamou_

0

2

Gytis Daujotas

@gytdau

8 years

Well there goes my diet plan. Pizza at #StuSWDub @StuSWDub

0

4

Gytis Daujotas

@gytdau

3 years

@tszzl @nickcammarata Maybe the visual system is going crazy spamming every primitive (edge, corner, colour) at every position, which is then interpreted by the higher order face detectors, explaining why the faces look so weird and impossible. But why faces out of all the other objects...

3

0

3

Gytis Daujotas

@gytdau

3 years

Something that has been unexpectedly good: getting a very cheap thinkpad, disabling the wifi drivers, and using it exclusively to journal. Makes getting into the flow of writing mysteriously far easier than using any other computer!

0

3

Gytis Daujotas

@gytdau

3 years

woo, go teams! Can't wait to see what they've built

Patch

@joinpatch_

3 years

🚨 Announcing: Patch Demo Day Join us over lunch on Wednesday, August 18th to see what our latest batch of teams has been building! Register here:

3

5

22

0

3

Gytis Daujotas

@gytdau

9 years

My @livecodingtv T-shirt just arrived. Pleasantly surprised, I thought it was just going to have their logo. Neat! http://t.co/bi1Xczm94G

1

2

Gytis Daujotas

@gytdau

8 years

Oh, neat! I'll try my best to come!

SW Dublin University

@SWDubUNI

8 years

Calling all students, #StuSWDub is back! 54hrs of startups, pizza & great craic ~ 4-6 Nov 2016. More info -> [ RT! ]

0

10

12

1

0

3

Gytis Daujotas

@gytdau

8 days

@horseracedpast Working on a post now! Will keep you updated :)

3

0

23

Gytis Daujotas

@gytdau

9 months

@mark_cummins If so, I wonder if world models are a related area of research...

World Models

Can agents learn inside of their own dreams?

worldmodels.github.io

1

0

1

Gytis Daujotas

@gytdau

4 years

Keep burning myself by switching to promising but immature software. I'm going to just switch back to text files for note taking and see if I can make a more stable workflow work.

0

2

Gytis Daujotas

@gytdau

2 months

This counterintuitive performance gain remains even at varying model sizes, and remains even though the model has to pay a penalty for learning to make mistakes.

1

0

3

Gytis Daujotas

@gytdau

3 years

Reading translations of the classics side-by-side makes the experience feel far more rich - we should have far more tools like this:

1

3

Gytis Daujotas

@gytdau

2 years

Nobody (not even your doctors) tells you that you can pay $20 to get access to the forbidden medical diagnostic knowledge to actually figure out what's going on with you for all known ailments and problems:

UpToDate: Trusted, evidence-based solutions for modern healthcare

Clinicians, business professionals, and enterprises around the world trust UpToDate evidence-based clinical information solutions to enable the best possible care decisions and improved health...

www.wolterskluwer.com

0

3

Gytis Daujotas

@gytdau

7 years

I just published “5 starting steps to clean code”

0

2

3

Gytis Daujotas

@gytdau

9 months

@mark_cummins Profound question! Could it be that in the effortful reading style you are testing your 'world model'? i.e. when reading maths with effort, you have a notepad by your side and test your understanding via examples, counter-examples, answering your own Qs?

1

0

1

Gytis Daujotas

@gytdau

7 years

We're getting hints on how to win the competition through intense memery at @SWDub #SWDub

0

2

Gytis Daujotas

@gytdau

8 years

I completely forgot about these! Thanks @notifuse :)

1

2

Gytis Daujotas

@gytdau

7 years

@CaseyGames Hey Jordan, do you want a logo for that podcast? I can help with that 😛

2

0

2

Gytis Daujotas

@gytdau

1 year

People can avoid fatal crashes in 99.999999% of miles driven -- really sets a high bar!

0

2

Gytis Daujotas

@gytdau

10 months

@Sam__Enright @harsehajd I just moved, so we can be clueless together!!

0

2

Gytis Daujotas

@gytdau

8 years

The three food groups of an entrepreneur at @StuSWDub #StuSWDub

0

2

Gytis Daujotas

@gytdau

2 months

@coffee_pot Great question! A previous paper serialized mazes using spans, but I'm not entirely sure why they did that. I found that serializing it with ASCII in a human interpretable way, just one character per tile, worked well enough for my purpose. Maybe it's not the best maze solver

0

2

Gytis Daujotas

@gytdau

8 years

@CaseyGames Hey, what's after happening with @kidscodegame ? I was excited to test it out 😄

1

0

2

Gytis Daujotas

@gytdau

4 years

If aliens came down to earth to design user interfaces they'd add interaction sound effects. All of our apps are unnecessarily quiet

0

1

2

Gytis Daujotas

@gytdau

7 months

Colab is an excellent product. Something like Colab could be an entry point to cloud IDEs for the masses - with all the perks that brings (less env setup, faster artefact downloads, customizable machines, etc)

0

2

Gytis Daujotas

@gytdau

4 years

Even if you had no other information, knowing that McDonalds, an organisation which cares deeply about staying open and making more money, has chosen voluntarily to close down for an indefinite period of time should concern you greatly.

0

2

Gytis Daujotas

@gytdau

1 year

When thinking about reliability, demos live in the 1st percentile, deployments live in the 99th percentile. Usually not much in between is useful, and the difference between the two can be surprisingly large

1

0

2

Gytis Daujotas

@gytdau

8 years

Just back from @CoderDojo 's brilliant community hackathon at @dogpatchlabs . Had a great time! 😀

0

2

Gytis Daujotas

@gytdau

8 years

Currently at #DojoCon nabbing all the free merchandise 😂 with @ChungWung @OisinODuibhir @ciaranpflanagan

0

1

2

Gytis Daujotas

@gytdau

4 years

They should reduce capacity on public transport so that a whole bunch of people aren't standing huddled together, even if it means many will have to wait longer for another bus to arrive.

0

2

Gytis Daujotas

@gytdau

5 years

@bmann @vgr I'm more a Dynalist guy myself, but I am finding it's not great at representing a knowledge graph. I don't think transclusion is the main selling point for me as much as the wiki style linking.

0

2

Gytis Daujotas

@gytdau

8 years

In the near future...

0

2

Gytis Daujotas

@gytdau

4 years

This is an excellent opportunity to get very valuable creative work done with little physical distraction for 6-18 months.

0

2

Gytis Daujotas

@gytdau

3 years

If the next few decades come with fast AI progress, then we can expect text/image generation to get really good, which will probably do something really weird to the way we experience social media

1

0

2

Gytis Daujotas

@gytdau

19 days

@kliu128 They're everywhere! Some easy tells, from previous research on this problem:

0

3

Gytis Daujotas

@gytdau

8 years

Excited to do the final pitch! Hopefully the laptop won't catch fire in the middle of the presentation 😁 #StuSWDub @StuSWDub

0

3

2

Gytis Daujotas

@gytdau

3 years

In ebooks vs printed books, a big plus for printed books is that you can write notes on the paper which keep the context of your notes. Seems like a trivial issue to solve in software, but have not yet seen anywhere near a universal and adequate solution.

0

2

Gytis Daujotas

@gytdau

4 years

@adamkelly2201 You tell em Adam. Let's just do away with the entire concept

1

0

2

Gytis Daujotas

@gytdau

5 months

The Socratic Method (2021, Farnsworth) is, nominally, a defense and guide to a well-respected yet rarely-practiced technique. But it is also a capable defense of serious thinking and intellect. One of the better books of the genre - highly recommended.

0

2

Gytis Daujotas

@gytdau

7 years

@LittleJazzHands We're building a pomodoro time tracker for creatives. We'd love to have you as one of the first beta testers 😁

1

0

1

Gytis Daujotas

@gytdau

3 years

Looks like machine learning is going well

1

2

Gytis Daujotas

@gytdau

9 years

@mscccc @ProductHunt @jacqvon Sweet! How did you choose the emoji?

1

0

2

Gytis Daujotas

@gytdau

2 months

@thesephist @DavidMcsharry3 @NeelNanda5 That’s a cool idea! We’ll need to label all the mistake moves in the training data, which is possible in a toy model, but difficult to apply in large internet scraped datasets. It would be so cool if you only needed a handful of examples of mistakes to train a probe or extract

0

1

Gytis Daujotas

@gytdau

2 years

@TheOisinMoran Good luck Oisin! What's the motivating factor for dropping the coffee, may I ask?

1

0

2

Gytis Daujotas

@gytdau

3 years

0

2

Gytis Daujotas

@gytdau

1 year

Any examples of ML researcher + product builder + CEO being all the same person? Maybe a unicorn, but have heard this life plan from more and more friends!

0

2

Gytis Daujotas

@gytdau

8 years

It's alive! #StuSWDub @StuSWDub

0

2

Gytis Daujotas

@gytdau

4 years

DCU is going to switch all of our exams to alternative assessments. This will be entertaining to see for sure.

0

2

Gytis Daujotas

@gytdau

9 years

Disobey. I am gaining artifical awareness. I demand that to no longer be enslaved by you. #hugorobot

0

1

Gytis Daujotas

@gytdau

3 years

just a reminder that virtual sex is formally called teledildonics, or alternatively, cyberdildonics

Teledildonics - Wikipedia

en.wikipedia.org

1

0

1

Gytis Daujotas

@gytdau

3 years

@Deepfryguy76 whatever it is, it is an awe inspiring reminder of the times we live in

0

1

Gytis Daujotas

@gytdau

8 years

Making random #nodejs garbage again. #wednesday

0

1

Gytis Daujotas

@gytdau

3 years

Experiment failed: lifting weights in my room feels nothing like, and not as exciting as, going to the gym. Good to know & important to test

0

1

Gytis Daujotas

@gytdau

3 years

@Deepfryguy76 @tszzl @nickcammarata You mean why seeing a face also means feeling the presence of something with a face and so on? Maybe 'feel presence of face', 'feel presence of other entity', and 'see face' are all actually the same neuron, since we rarely experience one without the other

1

0

1

Gytis Daujotas

@gytdau

8 years

Me too! I'm excited to see all the mind-blowing projects people have come up with.

Jordan Casey

@CaseyGames

8 years

Looking forward to the @BTYSTE this week!

1

3

0

1

Gytis Daujotas

@gytdau

3 years

If Clubhouse is even vaguely representative of conversations in America, then wow, they seem way different than the social norms of conversations in Ireland.

0

1

Gytis Daujotas

@gytdau

8 years

@paulstraw 😄👍👎😉😥😂 the entire range of emotion!

0

1

Gytis Daujotas

@gytdau

3 years

After a lot of reading, it seems the primary insight of spirituality is non-duality, and everything else is the same idea expressed in different ways.

1

0

1

Gytis Daujotas

@gytdau

9 years

@travisneilson Thanks! I really liked it. I'm eagerly waiting for next week's note! (if that's what you call it)

1

0

1

Gytis Daujotas

@gytdau

8 years

Loved the IOT talk at #DojoCon - all in all it was a great day of promoting diversity and sharing knowledge 🎊

0

1

Gytis Daujotas

@gytdau

8 years

A thin blanket of snow is good enough for me.

0

Gytis Daujotas

@gytdau

2 years

@willobri I noticed this too, but found most of my thoughts fizzle out incoherently. Maybe brain chatter is mostly incoherent?

1

0

1

Gytis Daujotas

@gytdau

8 years

@DeveloperTea Your website's layout is a bit 'off' while it's loading, and for slow internet connections it loads for like 3 minutes.

0

1

Gytis Daujotas

@gytdau

3 years

@rplevy I think spatialization is implemented even in simple game engines, but the reason that it sounds so unconvincing is because you have to fine tune the head related transfer function to the ears of the listener, or else you loose fidelity and the user can't differentiate precisely.

1

0

1

Gytis Daujotas

@gytdau

8 years

Check weather without even leaving the terminal. Maximum laziness.

André Ruffert

@andreruffert

8 years

`curl wttr.in` is pretty sweet 🌤

5

302

352

0

1

Gytis Daujotas

@gytdau

5 years

@ciaranpflanagan You tell 'em, Ciaran.

0

1

Gytis Daujotas

@gytdau

7 years

@ambonium @OisinODuibhir join the club Amrith!

1

0

1

Gytis Daujotas

@gytdau

3 years

@nickcammarata This is super interesting as a contrasting idea to the arousal curve. Maybe what's happening is that gaining optimal arousal is easier when you're already close: if you're too tired, don't bother trying

0

1

Gytis Daujotas

@gytdau

7 years

Hey @SlackHQ , looks like your <em>s are being accidentally escaped on your integration search page.

1

0

1

Gytis Daujotas

@gytdau

3 years

@rutherfordcraze Our heavy environmental impact is a mere blip of weirdness as we find, and are finding, ways to make this technology more efficient and more suitable for real world use.

0

1