Yesterday, i made a mistake and didn't pay enough attention about that the FollowYourPose
@gradio
demo uses MMPose instead of OpenPose 🫢
So there you go: convert any video or gif to MMPose sequence on
@huggingface
—›
Introducing that simple idea :
✨Split Audio Tracks to MusicGen✨
Use Demucs to split your favorite song into tracks, then send one to MusicGen to get a completely new music piece !
Give it a try
@huggingface
#ControlNet
Video Update ✨
—
1. You can now import GIF
2. All the Control Tasks are available
3. Preprocessor vid output
4. Gif output if you worked from gif
5. All outputs are downloadable
6. Share to community handles more settings
Enjoy 🤗
@Gradio
:
A new ModelScope Image2Video is out on
@huggingface
🤗 we love it !
It generates a short video from init image while keeping style consistency and try to restitute the general composition idea from source
Share your results with the Community 😌🤩
—
👉
Thanks to
@pharmapsychotic
’s CLIP Interrogator, you can know generate Music from Image 🔥
🧩 I built a
@Gradio
demo on
@huggingface
that let you feed an image to generate music, using MuBERT 🔥
Try it know ☺️
Yesterday, i made a mistake and didn't pay enough attention about that the FollowYourPose
@gradio
demo uses MMPose instead of OpenPose 🫢
So there you go: convert any video or gif to MMPose sequence on
@huggingface
—›
A quick
@gradio
demo you might like to play with this weekend, now running fast on
@huggingface
🤗
PASD Magnify ✨ • Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization
—
Paper:
Demo:
@huggingface
GPU zero are now running on A100!
By adding a simple annotation, your Spaces with grants are able to run
- on multiple GPUs
- on demand GPUs, release as needed
Come join the org and start making awesome demos on many A100!
MagicAnimate is available on
@huggingface
and it’s, incredible ? 🤯
The model works with DensePose from Facebook, now integrated in Detectron2 if i remember: we need a space to generate DensePose outputs 🤗
After exploring
@LangChainAI
docs,
i made a V2 Chat with PDF demo
@huggingface
,
using OpenAI, with ConversationalRetrievalChain + chat history instead of a RetrievalQA chain.
Also OpenAI seems more accurate than Flan 🤗
I think you'll love it!
🔥📄🤖
AI can generate realistic human photos from old paintings and pictures!
Besides the above, PhotoMaker can also change attributes of people, like changing hair colour and adding glasses, or mix identities of multiple people.
Just released on
@huggingface
a quick
@gradio
demo for DA-CLIP presented below by
@dreamingtulpa
Works like a charm, just drop your damaged image and hit submit !
Try it ! DA-CLIP demo link:
There is a new image restoration method in town called DA-CLIP.
The method is able to restore images by dehazing, deblurring, denoising, derainining, desnowing, removing unwanted shadows, raindrops, or improving lighting. So, basically from everything 😅
Thanks to the help of
@reach_vb
, try this
@huggingface
demo where you can chat with a PDF ! 📄🤖
It uses langchain to load the pdf content, and flan-ul2 as LLM to generate answers.
👉
Sora's sample videos are mind-blowing 🤯, that's for sure!
But they would be even better with sound, don't you think? 😊
Here are some examples, using the
@gradio
Image-to-SFX demo on
@huggingface
🤗
1/ 🚂 + 🔊
Great news! The
@huggingface
hub now has the first QR code AI art generator. You only need the QR Code content and a text-to-image prompt idea, or you can upload your image!
Check it out!
🎁 You asked for it, the
@Gradio
Image-to-Music space on
@huggingface
can now generate up to 2 MINUTES music tracks!🎅
—
I optimized the generation speed for tonight, so you can show it to your family on Christmas’s Eve ✨
Enjoy & Rejoice ! 🤗
Link:
I have another
@huggingface
space project to share with you:
🐶🐸 Bark + Coqui Voice Cloning demonstrates the Coqui implementation of Bark for fast text-to-speech generation from a wav file
This is an early version, follow me
@fffiloni
for very cool update to come ;)
Are you following me on
@huggingface
?
If yes, you surely witnessed updates i made today on the ProPainter demo 😬
—> You can now drop a video, prompt in what object/subject you need to remove (using SAM) and get an auto cleaned video output 🔥
Stop waiting in the queue for
#ControlNet
Video 😅
—
Duplicate the space, pick a Small A10 GPU ($1.05/h) that you can pause/restart anytime to save money 🤑
an example: a 3 sec GIF that took only 2 minutes to generate 😌
—
@huggingface
Duplication link :
Tried something tonight: drop an image; get a simple caption from CoCa; generate a 3D object from that caption —> Image concept to 3d shape
Please try it and give your thoughts to the Shap-e team ;)
@huggingface
Space:
Hey ! Long time no see in your TL 👋
Here's a prototype
@gradio
demo for a GPT Talking Portrait 🤗
It uses Whisper to let you ask GPT in your language, then One-Shot-Talking-Face will generate a talking portrait video from the answer ! Enjoy !
link:
Coqui + Bark voice cloning update ✨
—
1. you can now record your own voice
2. new characters LIBRARY lets you play with cherry picked voices (only 4 atm, will grow with your contributions)
—
Share your results & contribute to the library on the
@huggingface
's demo Community tab
#ControlNet
Video Next Update will let you import animated GIF 🤗
Stay tuned for a detailed tweet about new features, and be aware that the space will be paused tonight during the update process 😉
✨ ControlVideo demo is available on
@huggingface
🤗
For demo and performance purposes, inputs video are reduced to 12 fps | 512x width and limited to max 24 frames processing
Share your results with the Community
Follow
@fffiloni
for updates
🚀Space :
✨ New super cool demo involving image & sound on
@huggingface
—
IMAGE TO SOUND EFFECT
Convert an image to a corresponding sound effect generated through GPT2 Image Captioning & AudioLDM
—
Share your outputs with the community !
@Gradio
demo link :
Your attention please, we need you to share your examples with the community on the
@gradio
demo for ControlNet-Video
—
The best ones will be featured as official examples on the
@huggingface
space 🤗
—
Link:
✨UPDATE | ControlVideo demo can now load custom diffusion models stored in
@huggingface
diffusers gallery.
—›
Here's an example with the Ghibli Diffusion from
@Nitrosocke
—
demo:
✨ ControlVideo demo is available on
@huggingface
🤗
For demo and performance purposes, inputs video are reduced to 12 fps | 512x width and limited to max 24 frames processing
Share your results with the Community
Follow
@fffiloni
for updates
🚀Space :
👋 Coqui + Bark voice cloning demo is back
@huggingface
🤗
This is a new space instance, running in a custom 🐳Docker container, as i needed to find a solution around pip resolver issues 👨🔧😉
Thanks for your patience 🙏
Introducing ✨ Soft Video Understanding ✨, a personal experiment to try to achieve approximative video understanding using available open source models.
More explanations coming on
@huggingface
Social Posts later tonight, stay tuned !
LLM Agent from a Picture is now faster and more accurate, trying to really create a personality that reflects the character or scene described from the image input.
It also uses Fuyu-8b for a more detailed caption.
Share your best results with me 🤗, here some examples:
Image-to-Story UPDATE ✨
You can now specify the targeted audience of the generated story. A tiny but powerful change 🤗
What kind of criteria should i add next❔
Please share your ideas and your storiiiiies 💌 with the
@huggingface
community 🤗
✨
What if we proposed the Matting Anything Model (MAM) for an even more advanced use case?
Let me introduce:
✨Matting Anything for Videos ✨
Give it a try, share your results, follow
@fffiloni
for updates
@huggingface
space : 🤗
Introducing Zeroscope Image to Video:
drop an image as an init reference, get a CoCa caption which is then used as the prompt for zeroscope video generation
This is a simple idea, let me know how we could make it more interesting 🤗
@huggingface
space:
💁♂️Video to SoundFX : You can now edit the auto generated scene caption to add specific audio details if needed 🔊
Here's an example for you in real-time, where i decide to add "Seagulls" sound to the auto suggested one:
You are now more than 2k followers 🫢 Thanks and welcome ! 🙏
For those following for a long time, you know that i’m working on frame-by-frame tradi anim app 😉 + embedded ML gadgets ✨
—> Stay tuned for a
#ControlNet
anim app on
@huggingface
🤗
ProPainter demo just landed on
@huggingface
🤗
This version is a first rough iteration, made for those of you who already have prepared videos and masks frames, properly named 😉
Follow me
@fffiloni
and get notified for super cool updates to come 🔥
#ControlNet
Video UPDATE ✨
—
1. Added specific settings for some control tasks ( canny, hough & normal threshold )
2. You can now load custom models 🤟
3. Share to Community on private duplicated spaces sends outputs to main community tab
—
@gradio
demo:
Another great example for
#ControlNet
Video 😁
—
It's hard to control the flickering, i recommend to use detailed prompts for better results.
The right Control Task is also crucial here 🤗
Share your results 🤠 Follow for updates 🙏
—
@Gradio
demo link:
Introducing 🫴 Music To Zeroscope Video ✨
—
Similar to the Music to Image pipeline, but generates a zeroscope video instead of an image.
Follow me
@fffiloni
to keep track of next improvements
Try
@huggingface
demo 🤗:
🤩 HEADS UP • IMAGE-TO-MUSIC is back on
@huggingface
🥹
—
Update:
now get text caption output from CLIP Interrogator
+
One more thing: you can use the magic of GPT to generate a more musical prompt from original caption 😇
—
THX 🙏
@mubertapp
team
🤗
Music-To-Image UPDATE ✨
Now, you can optionally specify if your audio has lyrics, so the resulting image should reflect the sense of the lyrics of your song !
Here's an example with "Why iii Love The Moon" by Phony PPL 🤗
Share your results with us
@huggingface
Community ;)
Following
@multimodalart
’s original idea, let me introduce:
SD-XL LoRa Fusion, a
@huggingface
space where you can combine public and private sdxl LoRa models you previously stored on your personal/org account 🤗
Music-to-Image UPDATE ✨
You can now export your result as a video including the image and the music source + a supercool audio bars visualization 🤗
Share your results with me 😘 & the
@huggingface
Community 🤗
✨
I’m working on a fun workflow today, using DisCo Referring Human Dance 🕺
1. human target
2. openPose video reference to copy
3. DisCo result
4. Ground thruth
What do you think ? ☺️
Demo will be available on
@huggingface
soon
I thought at some point you might need to interpolate your videos generated from all these awesome video generation demos to play with these days 😉
So i crafted a quick
@gradio
space for video FILM interpolation on
@huggingface
🤗
—›
I found that
@MasterScrat
’s idea was a good one, so i built it:
A
@gradio
demo on
@huggingface
that allows you to convert a v1.x stable diffusion prompt to a v2.x stable diffusion equivalent prompt for use in
#StableDiffusion2
Look at that matte ! Pretty good 😌
Try Video Matting Anything with challenging subject to track matte and share your outputs ✌️
@gradio
demo on
@huggingface
🤗
—>
What if we proposed the Matting Anything Model (MAM) for an even more advanced use case?
Let me introduce:
✨Matting Anything for Videos ✨
Give it a try, share your results, follow
@fffiloni
for updates
@huggingface
space : 🤗
Just made a
@huggingface
Collection for the complete SD-XL training process 😌
✨The SD-XL Training Suite ✨
—
All the steps as demos to create / check / play with your own custom LoRa model 🤗
And last but not least, here comes my CPU
@Gradio
demo for INPAINTING
#StableDiffusion
on
@huggingface
🤗🌈
It uses the gradio sketch tool for masking 👩🎨
Try it with a 512*512 square input image, be patient, and everything should work fine 👌
—
Link :
I've updated the
@Gradio
demo on
@huggingface
for
✨
#Whisper
to
#StableDiffusion
✨
—
📝 you can now correct Whisper's translation if needed before send it to SD
🆒 you now get the transcript in your native lang
⬆️ you can now upload an audio file
—
🤗
🤠Exploring
@huggingface
models library can lead you to find underrated gems like this one 🤗
SALMONN: Speech Audio Language Music Open Neural Network
—› It's like moondream1 but for Audio Questioning Tasks🤯
Paper:
Demo:
#stablediffusion2
uses the OpenCLIP ViT-H model trained on the LAION dataset so it knows different things than the OpenAI ViT-L we're all used to prompting. To help out with your
#SD2
prompts here's a new ViT-H powered CLIP Interrogator!
#aiartcommunity
And there folks, You are witnessing the first rough steps of cel AI•nimation 🤗
—
1.Animated a character with my simple anim doodle on
@huggingface
2.Converted the anim to canny edge video
3.Render style with the Text2Video-Zero demo
—
poke
@radamar
@_akhaliq
@victormustar
😉
#controlNet
video is cool but did you know for some prompt you might prefer Pix2Pix or X-Decoder 🤗
Pix2Pix is good at style transfer 🎨
—>
X-Decoder is better at editing ✨
—>
Technical demo release: generate OPTICAL FLOW image and .flo file from 2 consecutive frames using RAFT model
@gradio
demo on
@huggingface
:
—
PyTorch blog:
SD-XL Dreambooth LoRa Training is available on
@huggingface
🤗
Duplicate this space, set a GPU, train one of your image datasets you had already upload on your HF profile datasets library, hit train !
Thanks
@multimodalart
for the inspiration 🙏
🚂
Open Source Video to Sound Effects 🎞️👂has landed on
@huggingface
🤗
Follow
@fffiloni
there () to keep track of updates and feature request
Demo link available in quoted post 😉
Did you know you could easily super slow-mo your
#Zeroscope
outputs ?
Use my FILM Video Frame Interpolation Space on
@huggingface
and smoothify your best shots 🤗
Link:
📌 Img-To-Music
@gradio
demo UPDATE
—
You can know share your results to the community tab on
@huggingface
by clicking on the share to community button 🚀
—
The process is a bit slow, but it works ;)
Thanks to
@pharmapsychotic
’s CLIP Interrogator, you can know generate Music from Image 🔥
🧩 I built a
@Gradio
demo on
@huggingface
that let you feed an image to generate music, using MuBERT 🔥
Try it know ☺️
🤯DiffBIR presents a breakthrough in blind image restoration, combining diffusion models and the LAControlNet feature.
🔥Run the
@Gradio
demo available on open-sourced Colab here-
✅Project Page-