David Marx || digthatdata.bsky.social Profile Banner
David Marx || digthatdata.bsky.social Profile
David Marx || digthatdata.bsky.social

@DigThatData

4,032
Followers
1,786
Following
1,166
Media
9,184
Statuses

Generative AI MLE, FOSS toolmaker, innovation catalyst @CoreWeave + @AiEleuther . AI enhanced creativity, philosophy of mind/science/probability

Seattle, WA
Joined November 2013
Don't wanna be here? Send us removal request.
Pinned Tweet
@DigThatData
David Marx || digthatdata.bsky.social
1 year
trapped in latent space, please send help
3
5
56
@DigThatData
David Marx || digthatdata.bsky.social
4 months
via reddit
Tweet media one
7
97
2K
@DigThatData
David Marx || digthatdata.bsky.social
2 years
Thanks for your patience everyone, first draft of colab for music video automation with #stablediffusion is published!
36
169
1K
@DigThatData
David Marx || digthatdata.bsky.social
10 months
Absolutely dope #AIart transformation of @marcrebillet with @devdef 's #stablediffusion #warpfusion + a few other tools (see OP for workflow and links to tutorials), by redditor AthleteEducational63:
34
181
907
@DigThatData
David Marx || digthatdata.bsky.social
6 months
yoooooo.... 3D gaussian splatting + simulated kinematics
27
100
707
@DigThatData
David Marx || digthatdata.bsky.social
1 year
OP ChatGPT corrective prompt: "this isn't what I wanted. read my previous instructions carefully and try again. start by explaining how this most recent response did not follow my previous instructions, and then try again." Have it explain what it did wrong as part of its reply
8
20
448
@DigThatData
David Marx || digthatdata.bsky.social
6 months
AI assisted live painting with @Krita_Painting frontend and #ComfyUI backend via LCM and this boss krita extension:
16
117
408
@DigThatData
David Marx || digthatdata.bsky.social
2 years
With all the fear mongering about how generative models are gonna steal all the artist jobs, no one's talking about how prompt engineering has created a tangible incentive for people to study art history and learn about different artists and art styles.
21
64
410
@DigThatData
David Marx || digthatdata.bsky.social
2 years
#stablediffusion experiment with some fancy masking
13
36
388
@DigThatData
David Marx || digthatdata.bsky.social
1 year
The GLAZE technique is specifically adversarial to finetuning. Decided to take an alternate approach "style mimicry" and used PEZ to reverse engineer viable prompts. Seems to work as expected: generated prompts capture content but bork on style. great work @ravenben and team!
Tweet media one
@kortizart
Karla Ortiz
1 year
1/ This might be the most important oil painting I’ve made: Musa Victoriosa The first painting released to the world that utilizes Glaze, a protective tech against unethical AI/ML models, developed by the @UChicago team led by @ravenben . App out now 👇
Tweet media one
154
5K
17K
13
74
326
@DigThatData
David Marx || digthatdata.bsky.social
1 year
ByteDance's "Universal Source Separation (USS) with Weakly labelled Data" project deserves way more than 130 stars. The source separation quality and granularity it achieves is really spectacular.
Tweet media one
5
43
314
@DigThatData
David Marx || digthatdata.bsky.social
2 years
After a month of work: Open Source PyTTI released! Docs: Colab:
17
46
283
@DigThatData
David Marx || digthatdata.bsky.social
9 months
Aw shit, code released for Magic123 - Single Image to HQ 3D mesh!
8
43
217
@DigThatData
David Marx || digthatdata.bsky.social
5 months
Happy hannukah everyone! Presenting ComfyUI-Keyframed nodes to facilitate prompt scheduling and all your other keyframing needs! 1/
8
30
209
@DigThatData
David Marx || digthatdata.bsky.social
1 year
Happy Saturday! New release of my fork of @RiversHaveWings KLMC2 notebook adds: * custom checkpoints * init image * keyframing for prompt weights and all supported parameters * multi-prompt conditioning * fancy spinning logo!
14
35
211
@DigThatData
David Marx || digthatdata.bsky.social
10 months
Gorgeous results here from training a separate motion prior, which has the added benefit that it can be composed with any other pre-trained SD checkpoint. BYOM plug-n-play text2video! * *
12
35
204
@DigThatData
David Marx || digthatdata.bsky.social
2 years
Improved my #stablediffusion "animating w variations" colab using a TSP solver to re-order frames. Definitely makes the final result a lot smoother!
11
22
202
@DigThatData
David Marx || digthatdata.bsky.social
5 months
SDXL-Turbo is cool, but SD-Turbo is almost as good and even inferences faster (base model is sd-2.1 instead of sdxl) but for some reason no one is talking about it.
17
30
195
@DigThatData
David Marx || digthatdata.bsky.social
2 years
Using #stablediffusion image variations to add just the tiniest bit of life to a painting
8
16
165
@DigThatData
David Marx || digthatdata.bsky.social
2 years
The "animating with variations" thing was so well received I was motivated to evolve it into an automated music video maker. This animation was made with NO editing. Scene timings inferred completely from video content (subtitles). Notebook coming soon, need to tidy up some :)
13
24
163
@DigThatData
David Marx || digthatdata.bsky.social
2 years
A lot of people seemed to be having trouble getting FiLM (an AI for video frame interpolation) working, so I put together a colab that hopefully makes it a bit easier or at least more reliable:
9
22
157
@DigThatData
David Marx || digthatdata.bsky.social
2 years
New features in #stablediffusion music video automation notebook, #VKTRS ! #DreamStudio API is optional; connect google drive; robust resume; in-notebook spreadsheet for prompt editing, overriding, flagging images for regen... Short tutorial vid (6.5min):
9
26
152
@DigThatData
David Marx || digthatdata.bsky.social
5 months
Been dropping teasers for some new comfy nodes i've been working on for the past two weeks. Planning to do a proper release later today after updating the docs. check back in a few hours!
Tweet media one
10
8
151
@DigThatData
David Marx || digthatdata.bsky.social
2 years
PyTTI-Tools v0.10 release is live! Lots of new features, including: AudioReactivity, ViT-L/14 @336px , numpy functions in weight formulae, and notebook QOL improvements! Lemme know what I broke :)
7
21
134
@DigThatData
David Marx || digthatdata.bsky.social
11 months
Ok so check it out, I think i've already figured out a stupid simple trick for improving quality even further with the new #texttovideo model: just run it through the XL model multiple times, varying the strength. Here's after running it through twice at .75 then twice at .7. 🧵
12
18
128
@DigThatData
David Marx || digthatdata.bsky.social
4 months
lol i guess a new image aug just dropped
Tweet media one
19
14
130
@DigThatData
David Marx || digthatdata.bsky.social
2 months
2y ago, I released pytti-tools. hours l8r, @EMostaque reached out to recruit me as 1 of @StabilityAI 's 1st eng hires. Easily 1 of happiest, most validating days my life. The co was SNAFU, but Emad is a gr8 guy who does a lot of good for open source AI. 😢
@StabilityAI
Stability AI
2 months
An announcement from Stability AI:
111
224
948
4
5
127
@DigThatData
David Marx || digthatdata.bsky.social
3 months
11
15
124
@DigThatData
David Marx || digthatdata.bsky.social
2 years
Although the #stablediffusion #AIart bots don't formally support prompt weights at the moment, there are still several ways you can manipulate prompt influence in multi-component prompts. Here are a few prompt-engineering tricks I've found useful with SD: 1/n
6
27
118
@DigThatData
David Marx || digthatdata.bsky.social
10 months
text-to-video is already crazy, and it' still early days. the prompt was literally just "the godfather". this is part of a... "classifier guided random walk" where I am the classifier. still exploring, will share a travelogue soon. safe to say zeroscope has untapped potential.
8
17
113
@DigThatData
David Marx || digthatdata.bsky.social
11 months
Important benchmark here. This is serious AI research. silverware etiquette at fancy restuarants confuses me too, #texttovideo model.
7
5
112
@DigThatData
David Marx || digthatdata.bsky.social
2 months
StabilityAI just lost the CompVis team, the people responsible for the Stable Diffusion and Stable Video architectures. * Robin Rombach * Andreas Blattmann * Dominik Lorenz
@timmpsmith
Tim Smith
2 months
Scoop: @robrombach , one of two of the original developers of Stable Diffusion has quit Stability AI. Rombach leaving the company represents the departure of the person responsible for the tech that made the company famous:
1
8
44
6
11
108
@DigThatData
David Marx || digthatdata.bsky.social
10 months
hacked together a slightly more user friendly AnimateDiff notebook, lemme know if you have issues
9
21
107
@DigThatData
David Marx || digthatdata.bsky.social
1 year
in addition to being generally interesting content, this blog post is worth thumbing through just for the fantastic graphics
3
14
107
@DigThatData
David Marx || digthatdata.bsky.social
5 months
Exciting news: I've joined @CoreWeave 's ML team!! I worked closely with their lead Wes Brown while driving early DreamStudio backend development @StabilityAI and am super looking forward to working with Wes full time. Open Source AI goes brrrrrr!
21
1
99
@DigThatData
David Marx || digthatdata.bsky.social
11 months
Made a video tutorial to help folks get set up with the new zeroscope_v2_XL #texttovideo madness that's been making the rounds. Setup is the first 5 minutes, all the links you need are in the description.
7
12
95
@DigThatData
David Marx || digthatdata.bsky.social
2 years
I had my doubts, but it really ties the room together. #baconwave #stablediffusion #aiart
Tweet media one
9
9
97
@DigThatData
David Marx || digthatdata.bsky.social
1 year
@ravenben I POURED MYSELF INTO THIS PIECE HOW DARE YOU /s
1
0
97
@DigThatData
David Marx || digthatdata.bsky.social
6 months
32 frames from the 14 frame image-to-video SVD model??? Yes, you can indeed use the last frame of the output as the input condition for another round of video generation, BUT CHANGE THE SEED FIRST. Discussion and comfyui workflow here:
5
10
93
@DigThatData
David Marx || digthatdata.bsky.social
10 months
Google colab finally realized people like knowing what kind of machine they're on
Tweet media one
5
3
90
@DigThatData
David Marx || digthatdata.bsky.social
1 year
This might look familiar to folks who've dropped acid at the Louvre
5
7
86
@DigThatData
David Marx || digthatdata.bsky.social
11 months
#aiart made with zeroscope #texttovideo
6
7
84
@DigThatData
David Marx || digthatdata.bsky.social
1 year
@ravenben omg, extra shoutout required for this amazing picture of the team via Ben's twitter banner
Tweet media one
0
1
84
@DigThatData
David Marx || digthatdata.bsky.social
2 years
I'm totally exhausted from the launch, but I had a weird idea and I had to try it. Cobbled together a little experiment demonstrating how you can interpolate in "prompt-space" with #stablediffusion :
4
9
83
@DigThatData
David Marx || digthatdata.bsky.social
1 year
well then.
Tweet media one
5
13
82
@DigThatData
David Marx || digthatdata.bsky.social
1 year
thanks @KaliYuga_ai for reminding me: my fork of @RiversHaveWings ' #KLMC2 nb had new features I forgot to merge and share! * Archive old work instead of deleting! * Resume! Choose starting frame! * Naive video upscaling (for better encoding on socials)!
2
18
81
@DigThatData
David Marx || digthatdata.bsky.social
2 years
little animation experiment, #stablediffusion + FiLM interpolation
6
10
79
@DigThatData
David Marx || digthatdata.bsky.social
10 months
The ability to drop in a generic SD LoRA for text-to-video is quite a super power. pre-LoRA, I was getting all shutterstock-watermarked outputs a la modelscope. Add a LoRA previously trained on text-to-image: BOOM, cinematic animation.
8
11
78
@DigThatData
David Marx || digthatdata.bsky.social
7 months
Very interesting looking tool, makes it easier to interact with ComfyUI workflows via a more standard form UI, while also giving you the ability to modify the workflow graph as well. Also has the ability to wrap workflows into reusable "actions"
6
10
76
@DigThatData
David Marx || digthatdata.bsky.social
5 months
madlad on the @AiEleuther discord has SDXL-Turbo at 50FPS on their hyper-optimized custom inference backend. This video is not sped up. It really generates images that fast.
@AlexanderRedde3
Zippy
5 months
Finally figured out how to speed up my #sdxlturbo frontend! It's so fast that the only way to show the actual speed is to delete the prompt, since I can't type fast enough 😆 .. built with next.js frontend & tensorrt backend.
17
29
226
1
10
75
@DigThatData
David Marx || digthatdata.bsky.social
1 year
This could be the beginning of a new publication paradigm. Citation, model, data, and code provenance all living in the same space. Looking forward to the future of diff-able research!
@dylan_ebert_
dylan
1 year
paper pages. now on @huggingface
6
35
189
1
15
75
@DigThatData
David Marx || digthatdata.bsky.social
1 year
zero shot intent classifier for arbitrary intent slot filling. That's it. That's the whole thing.
Tweet media one
4
5
75
@DigThatData
David Marx || digthatdata.bsky.social
2 years
upcoming pytti-tools release with video source animation fixed! gimme a few days to tie a bow on it :)
3
7
72
@DigThatData
David Marx || digthatdata.bsky.social
1 year
@OKWildlifeDept @csm231_ incredibly serendipitous ad placement
Tweet media one
1
0
71
@DigThatData
David Marx || digthatdata.bsky.social
1 year
New favorite ChatGPT prompt: "please implement ... following functional design principles and satisfying the user stories listed above. respond only with functioning python code and a full coverage test suite. when the implementation is complete, end with the phrase "ship it!"
3
5
71
@DigThatData
David Marx || digthatdata.bsky.social
1 year
Dad has dementia. mom just had surgery to remove cancer, poor prognosis. just picked dog up from vet, probably has cancer. check email after getting home from vet: best friend's dad passed unexpectedly. Not sure when I stepped on to this ride, but I'd like to get off now thanks.
29
0
70
@DigThatData
David Marx || digthatdata.bsky.social
10 months
Yesterday was my last day at StabilityAI. Assured it had nothing to do with performance. Just dropped like a dime Friday afternoon. Neither mgr nor skip were consulted. Plan to continue making free AI tools, you can support my work here:
9
9
67
@DigThatData
David Marx || digthatdata.bsky.social
2 years
succesful test using huggingface's diffusers library in the music video automation notebook! Calling it a night: api-optional notebook coming your way tomorrow, bright and early :)
8
8
68
@DigThatData
David Marx || digthatdata.bsky.social
6 months
light painting + SVD image-to-video
3
4
67
@DigThatData
David Marx || digthatdata.bsky.social
6 months
inspired by yesterdays sunset (AnimateDiff, prompts in thread)
4
10
65
@DigThatData
David Marx || digthatdata.bsky.social
2 years
Want to integrate #stablediffusion directly into your notebook work? Take our new SDK pacakge for a spin! Check out this colab for a simple usage demo:
@EMostaque
Emad
2 years
Delighted to announce the public open source release of #StableDiffusion ! Please see our release post and retweet! Proud of everyone involved in releasing this tech that is the first of a series of models to activate the creative potential of humanity
143
2K
6K
2
14
66
@DigThatData
David Marx || digthatdata.bsky.social
1 year
With AI poised to impact every corner of human productivity, maybe now would be a good time to revisit that whole UBI thing. Crazy idea.
5
7
64
@DigThatData
David Marx || digthatdata.bsky.social
1 year
Here's my fork of @RiversHaveWings KLMC2 notebook which adds keyframing for prompts and a few other things
@DigThatData
David Marx || digthatdata.bsky.social
1 year
KLMC2 aging a portrait of the late queen by incrementing her age in the prompt. The sampler seems to identify the "aging trajectory" very quickly and so she already looks 100 by the time the prompt is asking for a portrait of her at 50, but still: lots of potential here!
2
0
19
7
16
62
@DigThatData
David Marx || digthatdata.bsky.social
1 year
95% of nascent AI startups are just a simple prompt that wraps an API call. This business model is unsustainable, hinging on temporary lay ignorance of AI accessibility. Users will leave as laypeople get accustomed to using LLMs directly.
9
7
60
@DigThatData
David Marx || digthatdata.bsky.social
1 year
ChatGPT just handed me the best ever stats themed band name: "The Standard Deviants"
2
1
60
@DigThatData
David Marx || digthatdata.bsky.social
1 year
... software library coded in the style of gerald sussman and bjarne startstroup, optimized for readability, LGTFM, ship it, passes all tests, no errors, stateless, trending on artstation
3
8
60
@DigThatData
David Marx || digthatdata.bsky.social
1 year
"Low background steel" is steel made prior to the detonation of the first nuclear bomb. It's important for making sensitive equipment. The same way "1945" is a cutoff year for steel, I bet "2022" will be a cutoff year for reliably human-generated training data.
3
10
58
@DigThatData
David Marx || digthatdata.bsky.social
1 year
@a__spaceman @KaliYuga_ai sometimes the model needs tough love
1
0
56
@DigThatData
David Marx || digthatdata.bsky.social
2 years
New shoes with my #stablediffusion #aiart printed on em!!! Letting em air out a bit (gdamn this printing process stinky), but excited af. Maybe I'll take em dancing tomorrow?
Tweet media one
Tweet media two
6
4
57
@DigThatData
David Marx || digthatdata.bsky.social
6 months
3
1
55
@DigThatData
David Marx || digthatdata.bsky.social
3 months
AnimateDiff motion model for the LCM LoRA
2
9
55
@DigThatData
David Marx || digthatdata.bsky.social
1 year
@TheOnion Trumps last words were: "Joe Biden survives."
1
0
52
@DigThatData
David Marx || digthatdata.bsky.social
2 years
unraveling space and time #aiart #midjourney
Tweet media one
1
9
54
@DigThatData
David Marx || digthatdata.bsky.social
1 year
debugging plots and animations available now in my (more fully featured) fork of @RiversHaveWings ' amazing #KLMC2 #stablediffusion notebook! visualize precisely how settings changes and timings impact the generated output
6
3
53
@DigThatData
David Marx || digthatdata.bsky.social
1 year
thanks for your patience, pytti-tools fans! the ol' girl is up and running again (and thanks @sportsracer48 for spotting the fix!)
3
5
53
@DigThatData
David Marx || digthatdata.bsky.social
1 year
experiment overplotting prompt weights and step size together. probably would be better as separate subplots (ugh). def need to at least make sure they share x-axes (there should only be one moving vertical line). making progress anyway.
2
8
51
@DigThatData
David Marx || digthatdata.bsky.social
1 year
Great news! Vet just called with post-op histopathology results: the tumor was small and they're confident they got it all! Here's Marley as a sea monster to celebrate
2
2
53
@DigThatData
David Marx || digthatdata.bsky.social
5 months
I like building complex animations parameterized with keyframes and parameter curves, but I find the notation to be burdensome. The motivation of these nodes is to facilitate parameterizing keyframed animations, but leaning on the node UX as much as possible 2/
Tweet media one
2
3
40
@DigThatData
David Marx || digthatdata.bsky.social
1 year
i'm increasingly coming across anecdotal descriptions of people for whom making AI art is therapeutic. any art therapy researchers investigating? would love to see some numbers to go with the anecdotes. bet there're also some novel interventions waiting to be discovered here too.
13
4
52
@DigThatData
David Marx || digthatdata.bsky.social
2 months
Simple, clever (and controllable!) style transfer by just computing the KV features from a reference image. project: paper: code:
Tweet media one
Tweet media two
0
2
52
@DigThatData
David Marx || digthatdata.bsky.social
1 year
Every little thing is just so much easier. and we're just scratching the surface of this iceberg.
Tweet media one
2
4
51
@DigThatData
David Marx || digthatdata.bsky.social
1 year
Twitter isn't the only place where you can "follow" researchers to keep up with cutting edge AI projects: I get a ton of my news from Github directly. Here's a starter pack of users I follow whose "starring" activity is one of my most valuable news feeds. 1/n
5
6
50
@DigThatData
David Marx || digthatdata.bsky.social
1 year
looping a step size curve through a few different prompt transitions, fancy dreaming with klmc2
3
5
50
@DigThatData
David Marx || digthatdata.bsky.social
1 year
i <3 KLMC2 + watercolor
3
6
51
@DigThatData
David Marx || digthatdata.bsky.social
1 year
Tweet media one
4
3
49
@DigThatData
David Marx || digthatdata.bsky.social
1 year
Something I dislike about the whole NFT thing is the tacit implication that an art piece has no cultural value unless it's also a commodity. A lot of influential generative artists are completely ignored by the art show/museum circuit because they don't mint.
4
3
49
@DigThatData
David Marx || digthatdata.bsky.social
1 year
Oh hey whaddayaknow, my KLMC2 fork supports custom output resolution. Also updated the demo to illustrate how to do a traversal with an accelerating step size
3
2
48
@DigThatData
David Marx || digthatdata.bsky.social
2 years
unusual weather we've been having lately... #baconwave #stablediffusion #AIart
Tweet media one
Tweet media two
2
4
47
@DigThatData
David Marx || digthatdata.bsky.social
9 months
Accidentally ran deforum with some especially potato settings and I kinda love it
7
0
47
@DigThatData
David Marx || digthatdata.bsky.social
1 year
decided I wanted that last one to cook a bit longer, here's 2k frames at 60fps, klmc2 dreaming a #deepfloyd init image with #stablediffusion
4
4
48
@DigThatData
David Marx || digthatdata.bsky.social
1 year
Couldn't decide what kind of pizza to get, so I ordered a superposition of every pizza on the menu.
4
5
47
@DigThatData
David Marx || digthatdata.bsky.social
2 years
Trying out a print-on-demand service for the first time. Fuck NFTs: I'm minting shoes over here. Non-fungible textiles.
Tweet media one
6
6
45
@DigThatData
David Marx || digthatdata.bsky.social
1 year
coming soon to my #klmc2 fork: debugging animations! here's an example plotting relative prompt weights over the generated images
3
7
44
@DigThatData
David Marx || digthatdata.bsky.social
7 months
Fun fact: remember that "multiple passes" trick i figured out for improving modelscope outputs? Yeah, it works pretty great for AnimateDiff too. third pass:
4
3
43
@DigThatData
David Marx || digthatdata.bsky.social
6 months
yooooo how tf did I miss this?? make any sd1.5 checkpoint inferenceable in 2-8 steps by slapping on a LoRA!
7
1
43
@DigThatData
David Marx || digthatdata.bsky.social
2 years
@pmddomingos Let's be honest here: the vast majority of data professionals work on "AI for maximizing ad revenue"
2
1
44
@DigThatData
David Marx || digthatdata.bsky.social
1 year
cranking up the guidance scale on modelscope's text2video produces better outputs (imho) and lets you get away with fewer timesteps. This 64 frame video was generated from just 27 steps at cfg 50 (model defaults are steps=50, cfg=9, frames=16)
2
4
43
@DigThatData
David Marx || digthatdata.bsky.social
6 months
And did they share code for this awesome project? why yes, yes they did!
@NTU_chenyiwen
Chen Yiwen
6 months
Excited to present our new paper "GaussianEditor", a new #3D #gaussiansplatting #editing framework! GaussianEditor provides controllable, diverse, and interactive high-resolution 3D editing, needing only 2-7 minutes. Project page 👉:
14
105
557
0
2
43