Rivers Have Wings Profile Banner
Rivers Have Wings Profile
Rivers Have Wings

@RiversHaveWings

31,152
Followers
225
Following
2,168
Media
5,411
Statuses

AI/generative artist. Writes her own code. Absolute power is a door into dreaming.

Joined June 2016
Don't wanna be here? Send us removal request.
Pinned Tweet
@RiversHaveWings
Rivers Have Wings
1 year
Humanity is going to make all parts of the world touched by humans beautiful. We are going to create beauty too cheap to meter. And not just an enforced-from-above standard of beauty either, everyone will be able to make their own domain beautiful in the manner of their choosing.
24
47
316
@RiversHaveWings
Rivers Have Wings
3 years
This is just wild. #StyleGAN3
77
761
5K
@RiversHaveWings
Rivers Have Wings
3 years
I'm doing AI NFTs now. :) Our main product is called Control The Soul. It's a 1k piece collection meant to make cool Twitter avatars. The theme is wizards, eldritch artifacts, and weird landscapes. Most items 3-4 XTZ. Check out the gallery at
Tweet media one
Tweet media two
Tweet media three
Tweet media four
62
101
956
@RiversHaveWings
Rivers Have Wings
2 years
"🌹 fractal" StyleGAN XL + CLIP
14
75
975
@RiversHaveWings
Rivers Have Wings
2 years
"A vision of paradise, Unreal Engine" StyleGAN XL + CLIP
22
80
882
@RiversHaveWings
Rivers Have Wings
2 years
"the gateway between dreams, trending on ArtStation" (CLIP guided YFCC diffusion, 968M parameter model, it's not done training yet)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
18
102
797
@RiversHaveWings
Rivers Have Wings
2 years
"The Rise Of Consciousness" StyleGAN XL + CLIP
12
77
759
@RiversHaveWings
Rivers Have Wings
3 years
d i s s o l v e (config-r edition, not a full dissolve) #StyleGAN3
15
88
642
@RiversHaveWings
Rivers Have Wings
1 year
If Stable Diffusion 2.0 doesn't know your favorite artist, you can search for similar artists on and try prompting with them instead. For instance here are several Rutkowski-like artists: Raphael Lacoste, Andreas Rocha, Titus Lunter, Richard Anderson.
Tweet media one
Tweet media two
Tweet media three
Tweet media four
58
89
566
@RiversHaveWings
Rivers Have Wings
1 year
I've trained a latent diffusion upscaler for the Stable Diffusion autoencoder (and anything you feel like feeding into it if you can tolerate a little artifacts) in collaboration with @stabilityai . Try the Colab written by @nshepperd1 here:
Tweet media one
Tweet media two
17
108
546
@RiversHaveWings
Rivers Have Wings
10 months
I have a new Colab notebook, “Reinforcement Learning from AI Feedback” (), which fine-tunes a language model (Pythia 160M) using natural language criteria for its sampled outputs. Standalone version for larger models below. Have fun, report bugs! :)
4
97
491
@RiversHaveWings
Rivers Have Wings
3 months
Hourglass + Diffusion = ❤️ We introduce a new transformer backbone for diffusion models that can directly generate megapixel images without the need for multiple stages like latent diffusion. Read here! → Project page →
Tweet media one
19
89
458
@RiversHaveWings
Rivers Have Wings
2 years
a propaganda poster for building a space elevator
Tweet media one
10
47
443
@RiversHaveWings
Rivers Have Wings
3 years
I scaled up CLIPDraw () a bit... "a beautiful epic wondrous fantasy painting of [the ocean / lightning / wind / a deep valley]":
Tweet media one
Tweet media two
Tweet media three
Tweet media four
10
88
433
@RiversHaveWings
Rivers Have Wings
3 years
OMG. CLIP knows how to Deep Dream-ify images by maximizing similarity with the text prompt "Deep Dream".
Tweet media one
Tweet media two
10
65
422
@RiversHaveWings
Rivers Have Wings
2 years
"the vision of the ophanim by Ross Tran"
Tweet media one
Tweet media two
5
54
403
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "a painting of climate change killing humanity, by Greg Rutkowski" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
14
49
374
@RiversHaveWings
Rivers Have Wings
2 years
the oracle of glitch
Tweet media one
Tweet media two
Tweet media three
Tweet media four
6
57
364
@RiversHaveWings
Rivers Have Wings
4 months
I made a flexible new image captioning method based on only a base model LLM and CLIP. It lets you go beyond just describing what is in an image and analyze narrative themes, latent knowledge in CLIP embeddings, and more! Colab:
8
79
373
@RiversHaveWings
Rivers Have Wings
2 years
My 602M parameter CLIP conditioned diffusion model trained on Conceptual 12M is out at ! It can generate images matching the prompt quickly using its CLIP conditioning, but still requires CLIP guidance for best results.
Tweet media one
15
71
370
@RiversHaveWings
Rivers Have Wings
2 years
The Egyptian pyramids in a snowstorm
Tweet media one
Tweet media two
Tweet media three
Tweet media four
5
44
359
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "Van Gogh's Starry Night by Ivan Aivazovsky" (CLIP guided diffusion, with Starry Night as an init)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
5
37
357
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "a surreal and organic stone temple of Cthulhu" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
4
29
346
@RiversHaveWings
Rivers Have Wings
2 years
From #EleutherAI Discord:
Tweet media one
10
38
349
@RiversHaveWings
Rivers Have Wings
2 years
A nuclear explosion
Tweet media one
11
32
339
@RiversHaveWings
Rivers Have Wings
2 years
Our VQGAN+CLIP paper (authors: me, @BlancheMinerva , @jbusted1 , @dashstander , @EricHallahan , @lcastricato , @EdwardRaffML ) is finally out!
@_akhaliq
AK
2 years
VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance abs: github:
Tweet media one
1
77
315
8
61
345
@RiversHaveWings
Rivers Have Wings
3 years
New Colab notebook: CLIP Guided Diffusion HQ 512x512
11
37
330
@RiversHaveWings
Rivers Have Wings
2 years
water is a circle
Tweet media one
Tweet media two
7
27
321
@RiversHaveWings
Rivers Have Wings
1 year
Here is a notebook of the method I have been using to make these animation posts: (Stable Diffusion KLMC2 Animation). Have fun, report bugs! :)
24
52
323
@RiversHaveWings
Rivers Have Wings
2 years
"a nuclear explosion by Dan Mumford"
Tweet media one
Tweet media two
6
23
317
@RiversHaveWings
Rivers Have Wings
2 years
"the gateway of the sun, trending on ArtStation" (CLIP guided YFCC diffusion, 968M parameter model, it's not done training yet)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
7
35
318
@RiversHaveWings
Rivers Have Wings
3 years
New Colab notebook: CLIP Guided Diffusion HQ 512x512 Uncond This notebook contains my 8100 step fine-tune of the 512x512 class-conditional ImageNet model into an unconditional ImageNet model for better controllability with CLIP.
15
46
318
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "The mountains of Oregon with wildfire smoke by Ivan Aivazovsky" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
8
32
305
@RiversHaveWings
Rivers Have Wings
2 years
I updated my secondary model diffusion method notebooks with a new, bigger but still efficient secondary model: (256x256), (512x512). The original model is still in there and you can select it instead!
12
45
308
@RiversHaveWings
Rivers Have Wings
2 years
elegant hell butterfly
Tweet media one
Tweet media two
2
45
301
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "a painting of artificial intelligence, the sky is burning, by Greg Rutkowski" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
5
41
292
@RiversHaveWings
Rivers Have Wings
2 years
Prompt: "A cyberpunk cathedral by Greg Rutkowski and Claude Monet, oil on canvas" ( #StableDiffusion )
Tweet media one
Tweet media two
Tweet media three
Tweet media four
8
41
296
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "a painting of humanity surviving artificial intelligence, by Greg Rutkowski" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
10
35
292
@RiversHaveWings
Rivers Have Wings
2 years
I added the PRK and PLMS sampling methods (from ) to and the cc12m_1_cfg Colab at . The default for the Colab is now 50 steps PLMS which seems to give pretty good quality. Have fun! :)
21
47
298
@RiversHaveWings
Rivers Have Wings
2 years
CLIP Guided Diffusion 512x512, Secondary Model Method: This is the faster/lower memory method I promised yesterday. Quality may be lower with this notebook but we can probably train a better secondary diffusion model now that we know they're useful.
14
39
302
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "a cyberpunk church resembling the Death tarot card by Greg Rutkowski" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
2
35
290
@RiversHaveWings
Rivers Have Wings
2 years
I made a text to image Colab notebook for my latest model, cc12m_1_cfg! It's more basic right now than the CLI sampling scripts (it only does one text prompt and doesn't have init images yet) but I plan to add more features later.
21
39
287
@RiversHaveWings
Rivers Have Wings
2 years
oh hey i discovered a way to speed up CLIP guided diffusion by nearly 2x and lower memory requirements by a similar amount... more details later, i need to go to bed
13
8
286
@RiversHaveWings
Rivers Have Wings
2 years
It's one year to the day since I got my first batch of good CLIP Guided Diffusion outputs! I wrote the public notebook, "CLIP Guided Diffusion HQ 256x256", the next day. The prompt was "Desert dynamism by Giacomo Balla". (Notebook link: )
Tweet media one
Tweet media two
9
37
284
@RiversHaveWings
Rivers Have Wings
3 years
"industrial demon, a matte painting, trending on ArtStation" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
10
23
274
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "Artificial Intelligence is the vehicle for demons to enter our world, by John Harris" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
5
27
262
@RiversHaveWings
Rivers Have Wings
2 years
Prompt: "A utopian arcology by H. R. Giger, oil on canvas"
Tweet media one
Tweet media two
Tweet media three
Tweet media four
8
29
266
@RiversHaveWings
Rivers Have Wings
2 years
Prompt: "The organic machine by Rene Magritte, oil on canvas" Wow this model...
Tweet media one
Tweet media two
Tweet media three
Tweet media four
14
25
258
@RiversHaveWings
Rivers Have Wings
3 years
I finally managed to find hosting for my CLIP conditioned transformer (it's on GitHub as a release right now), so here's the Colab:
11
34
262
@RiversHaveWings
Rivers Have Wings
3 years
CLIP-based style transfer is a thing! (Using the ViT-B/32 image encoder outputs only, no use of intermediate layers.)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
4
36
259
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "a painting of UFOs, the sky is burning, by Greg Rutkowski"
Tweet media one
Tweet media two
Tweet media three
Tweet media four
9
37
254
@RiversHaveWings
Rivers Have Wings
2 years
I just realized I forgot to ever announce my diffusion model training and hackable inference codebase, k-diffusion, on Twitter! It only has unconditional diffusion models rn but text conditioned is easy to add.
1
25
251
@RiversHaveWings
Rivers Have Wings
2 years
Our first batch of four diffusion models, along with super basic CLIP guided inference code, is out! The models are: Danbooru SFW 128x128 ImageNet 128x128 WikiArt 128x128 WikiArt 256x256 No Colab yet but soon!
Tweet media one
Tweet media two
Tweet media three
Tweet media four
10
30
246
@RiversHaveWings
Rivers Have Wings
2 years
"Resurrection of the dead via multiverse-wide acausal cooperation by Ross Tran"
Tweet media one
Tweet media two
1
32
240
@RiversHaveWings
Rivers Have Wings
2 years
"a gateway between dreams, pixel art #pixelart ", @midjourney diffusion
Tweet media one
Tweet media two
Tweet media three
Tweet media four
2
46
248
@RiversHaveWings
Rivers Have Wings
2 years
Prompt: "Humanity is killed by artificial intelligence, by James Gurney, oil on canvas"
Tweet media one
Tweet media two
Tweet media three
Tweet media four
11
25
245
@RiversHaveWings
Rivers Have Wings
2 years
thinking about how CLIP guided diffusion was a pretty good method for its time and its time was like the second half of 2021
18
4
243
@RiversHaveWings
Rivers Have Wings
2 years
a cyberpunk city
Tweet media one
5
30
238
@RiversHaveWings
Rivers Have Wings
2 years
"a Minecraft screenshot, unreal engine" Deep Image Prior + CLIP ViT-L/14 336px
Tweet media one
5
21
242
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "revolution of the souls" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
2
42
240
@RiversHaveWings
Rivers Have Wings
2 years
Experimenting with CLIP as a feature extractor for NIMA (neural image assessment) type things... a simple linear regression yielded a direction in CLIP latent space that can increase the "aestheticness" of an output. Here's "a datura witch" and a couple of Starry Night variants:
Tweet media one
Tweet media two
Tweet media three
Tweet media four
11
26
239
@RiversHaveWings
Rivers Have Wings
2 years
"The Rise Of Consciousness" StyleGAN XL + CLIP (from a different starting seed)
5
13
241
@RiversHaveWings
Rivers Have Wings
3 years
First try guiding #StyleGAN3 with CLIP...
5
22
233
@RiversHaveWings
Rivers Have Wings
3 years
By a user in EleutherAI Discord, using the VQGAN+CLIP bot there: "recursive recursion of the recursive imagination of a landscape by james gurney" and "landscape of recursion by james gurney". CLIP understands that "recursion" means you put the thing inside itself etc?
Tweet media one
Tweet media two
4
34
236
@RiversHaveWings
Rivers Have Wings
3 years
"humanity is killed by climate change, by Marc Simonetti" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
7
30
233
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "The coast of California with wildfire smoke by Ivan Aivazovsky" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
9
22
230
@RiversHaveWings
Rivers Have Wings
2 years
Prompt: "a painting of the beginning of the world, by Greg Rutkowski" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
4
25
234
@RiversHaveWings
Rivers Have Wings
3 years
CLIP guided diffusion Colab, super slow HQ version (hope you get a V100):
15
24
227
@RiversHaveWings
Rivers Have Wings
3 years
New Colab: CLIP guided diffusion
8
32
227
@RiversHaveWings
Rivers Have Wings
2 years
"The Rise Of Consciousness by Max Ernst, oil on canvas" VQGAN+CLIP+prior
Tweet media one
Tweet media two
Tweet media three
Tweet media four
8
26
225
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "the last day of the air, trending on ArtStation" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
8
25
223
@RiversHaveWings
Rivers Have Wings
3 years
"technological singularity cult by James Gurney" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
4
28
221
@RiversHaveWings
Rivers Have Wings
2 years
Prompt: "Futurism by Jeffrey Smith, oil on canvas" (CLIP guided diffusion) Interestingly this did not give me Italian Futurism, you have to ask for it by artist name I think?
Tweet media one
Tweet media two
Tweet media three
Tweet media four
9
21
222
@RiversHaveWings
Rivers Have Wings
2 years
A Christmas tree in outer space
Tweet media one
Tweet media two
Tweet media three
Tweet media four
2
27
216
@RiversHaveWings
Rivers Have Wings
2 years
the conscious void
Tweet media one
5
21
219
@RiversHaveWings
Rivers Have Wings
2 years
"the winds of change by Ross Tran"
Tweet media one
Tweet media two
Tweet media three
Tweet media four
6
28
222
@RiversHaveWings
Rivers Have Wings
2 years
CompVis's new latent diffusion model clearly thinks "a gateway between dreams" should be a book title. Look at that generated text though!
Tweet media one
Tweet media two
17
20
220
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "a giant floating eye monster, by James Gurney" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
3
23
215
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "a sea monster by Ivan Aivazovsky" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
3
24
211
@RiversHaveWings
Rivers Have Wings
2 years
I released two ViT-B/16 CLOOB checkpoints, along with JAX training code and JAX/PyTorch inference code, trained on the open dataset LAION 400M: . Zero-shot ImageNet accuracy:
Tweet media one
6
28
213
@RiversHaveWings
Rivers Have Wings
3 years
"the lighthouse at the end of the universe by James Gurney" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
1
22
209
@RiversHaveWings
Rivers Have Wings
3 years
Some diffusion Control The Soul...
Tweet media one
Tweet media two
Tweet media three
Tweet media four
13
33
209
@RiversHaveWings
Rivers Have Wings
2 years
"twilight is a circle by Ross Tran"
Tweet media one
Tweet media two
Tweet media three
1
34
206
@RiversHaveWings
Rivers Have Wings
2 years
Prompt: "The alchemist's laboratory by Greg Rutkowski and Claude Monet, oil on canvas" ( #StableDiffusion )
Tweet media one
Tweet media two
Tweet media three
Tweet media four
4
27
211
@RiversHaveWings
Rivers Have Wings
2 years
You can apply a similar trick to classifier-free guidance to autoregressive transformers to sample from a synthetic "super-conditioned" distribution. I trained a CIFAR-10 class-conditional ImageGPT to try this, and I got the following grids with cond_scale 1 (default) and then 3:
Tweet media one
Tweet media two
4
24
213
@RiversHaveWings
Rivers Have Wings
2 years
Prompt: "A technological singularity cult by James Gurney and Rob Gonsalves, oil on canvas" ( #StableDiffusion )
Tweet media one
Tweet media two
Tweet media three
Tweet media four
4
13
206
@RiversHaveWings
Rivers Have Wings
2 years
"a peaceful field of tall sunflowers" cc12m_1_cfg
Tweet media one
4
17
200
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "a cold landscape by Thomas Kinkade" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
7
19
205
@RiversHaveWings
Rivers Have Wings
2 years
Me, a year ago:
Tweet media one
9
19
203
@RiversHaveWings
Rivers Have Wings
2 years
a propaganda poster for transhumanism
Tweet media one
4
16
198
@RiversHaveWings
Rivers Have Wings
2 years
~
Tweet media one
Tweet media two
Tweet media three
Tweet media four
5
7
200
@RiversHaveWings
Rivers Have Wings
2 years
some prog rock album covers
Tweet media one
5
17
198
@RiversHaveWings
Rivers Have Wings
2 years
cc12m_1 with classifier-free guidance (the model I've been using to make all of these grids for the last few days) is out: ! The repo has a new cfg_sample.py script to sample from the new model.
11
38
205
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "unplanned emergent AI behavior causes catastrophe, by James Gurney" (CLIP guided diffusion) I like how you can't really tell what is going on in some of these, just that it is weird/alien and *shouldn't be happening*.
Tweet media one
Tweet media two
Tweet media three
Tweet media four
4
27
190
@RiversHaveWings
Rivers Have Wings
3 years
I updated my 256x256 and 512x512 unconditional CLIP guided diffusion notebooks with support for multiple weighted text and image prompts: (256x256) and (512x512)
3
23
201
@RiversHaveWings
Rivers Have Wings
2 years
"a nuclear explosion by Ross Tran"
Tweet media one
Tweet media two
1
15
191
@RiversHaveWings
Rivers Have Wings
3 years
Prompt: "a beautiful epic wondrous fantasy painting of wind" (CLIP guided diffusion)
Tweet media one
Tweet media two
Tweet media three
Tweet media four
5
28
199
@RiversHaveWings
Rivers Have Wings
2 years
Prompt: "The queen of the sun, by Jeffrey Smith, oil on canvas" (Stable Diffusion)
Tweet media one
11
24
197
@RiversHaveWings
Rivers Have Wings
1 year
animated cubism #StableDiffusion2 #AIart hq:
11
30
193