Omer Bar Tal @omerbartal Twitter profile

Pinned Tweet

Omer Bar Tal

4 months

Introducing Lumiere 📽️ The new video diffusion model we've been working on @GoogleAI * Text-to-Video * Image-to-Video * Stylized Generation * Inpainting * Cinemagraphs and more 🎨 W/ amazing team incl. @hila_chefer @omer_tov @InbarMosseri @talidekel @DeqingSun @oliver_wang2

53

146

776

Last Seen Profiles

@mtchtrnr

@lottiebcth

@gk_koharu

@ClubShoshin

@sofia_sofia563

@KdotJohnson_

@BMOStadium

@O3B7m

@TabascoBoss

@TyahTips

@GraemeKirkham

@JuanWangPoliSci

@v7yun

@SidduOfficial

@panifowl

@NJ_SouthEuro

@WHAALAMAGIC

@LeemingAlice

@Aripon_zu

@KSparticus

@Elaine_baddie

@no0odlehead

@braxkinsmood

@TheGeogrineer

@CrimsonKING36

@PhuckWitMe2

@jandakembangstw

@ariesdemon24

@ComedieduLivre

@Yasmine_Lu

@S_Dokumamushi

@JohnDugard

@webberwentzel

@MLG_McDonald

@Alexand87325629

@PodMatch_com

Omer Bar Tal

@omerbartal

1 year

Played with optimizing Neural Atlases through Stable Diffusion. So much fun! Here are a few examples of video edits: @RafailFridman @DanahYatim

8

76

391

Omer Bar Tal

@omerbartal

1 year

Excited to share "MultiDiffusion"! A controlled image generation framework w/ pre-trained text-to-image diffusion model. * Spatial guidance controls (bounding boxes/masks) * Arbitrary aspect ratios (huge Panoramas!) NO training NO finetuning. [1/3] @YarivLior @lipmanya @talidekel

8

63

326

Omer Bar Tal

@omerbartal

5 months

Text2video models are getting interesting!📽️ Check out how we leverage their space-time features in a zero-shot manner for transferring motion across objects and scenes! Led by @DanahYatim @RafailFridman , @yoni_kasten @talidekel [1/3]

7

53

303

Omer Bar Tal

@omerbartal

8 months

The code and demo for TokenFlow are both released! 🥳 Check out our work for consistent video editing with a fixed text-to-image diffusion 🎥 Webpage: Demo: @GeyerMichal @OneViTaDay @talidekel Thanks @linoy_tsaban @multimodalart !

9

84

296

Omer Bar Tal

@omerbartal

6 months

ICLR 2024 stats 💡 * 451 titles contain 'Diffusion' * 208 titles contain 'LLM' * 6 titles contain 'ChatGPT' * 25 titles contain 'NeRF' * 41 titles contain 'GAN' * 15 titles contain 'All you need' 🧐 * 22 titles contain 'Dream' * 6 titles contain 'Magic' #ICLR2024 @iclr_conf

3

41

209

Omer Bar Tal

@omerbartal

3 months

Thrilled to share that I've joined @pika_labs as Founding Scientist! It's exciting times in video generation and AI, and we're committed to unlock new capabilities and empower creative content creation. Delighted to work with the most talented @demi_guo_ @chenlin_meng 🚀

28

4

202

Omer Bar Tal

@omerbartal

1 year

MultiDiffusion is now integrated into diffusers 🚀 currently text2panorama is supported, spatial controls (masks/bounding boxes)- soon :) demo: official repo: Thanks @RisingSayak @_akhaliq and @huggingface team!

3

35

126

Omer Bar Tal

@omerbartal

10 months

May all conferences be in Hawaii 🏝️ #ICML2023 @icmlconf

3

1

87

Omer Bar Tal

@omerbartal

4 months

TokenFlow was accepted to #ICLR2024 ! @GeyerMichal @OneViTaDay @talidekel @iclr_conf

Omer Bar Tal

@omerbartal

8 months

The code and demo for TokenFlow are both released! 🥳 Check out our work for consistent video editing with a fixed text-to-image diffusion 🎥 Webpage: Demo: @GeyerMichal @OneViTaDay @talidekel Thanks @linoy_tsaban @multimodalart !

9

84

296

1

6

59

Omer Bar Tal

@omerbartal

4 months

💡Instead, Lumiere is based on a Space-Time U-Net that generates full-frame-rate videos in a single model pass, without a cascade of temporal super resolution models, by sampling the video content in both space *and time*.

2

3

53

Omer Bar Tal

@omerbartal

8 months

The @MrUgleh trend can be combined with MultiDiffusion! 💫 Allowing to create such effects in **arbitrary resolutions** 🎨 Code:

1

10

51

Omer Bar Tal

@omerbartal

2 years

Text2LIVE, ECCV Oral! We perform text-guided localized editing of real-world images and videos by training a lightweight generator on a single example to predict an “edit layer”. We don’t require input segmentation or pretrained generative models (e.g., Diffusion/GAN)

Tali Dekel

@talidekel

2 years

Billion params text-to-image models are amazing! But...not designed for editing real-world images/videos. Text2LIVE (ECCV oral) trains on 1 example and allows for various semantic, localized editing! @omerbartal @dolevofri @RafailFridman @yoni_kasten 1/3

10

159

745

5

2

49

Omer Bar Tal

@omerbartal

4 months

Lumiere generates coherent, high-quality videos using simple text prompts! Here are some examples:

1

3

39

Omer Bar Tal

@omerbartal

4 months

Thanks @_akhaliq for sharing our work!

AK

@_akhaliq

4 months

Google presents LUMIERE A Space-Time Diffusion Model for Video Generation paper page: Demonstrate state-of-the-art text-to-video generation results, and show that our design easily facilitates a wide range of content creation tasks and video editing…

32

336

2K

0

5

37

Omer Bar Tal

@omerbartal

4 months

We can edit videos locally using masks, or animate an image only in a specific region!

1

38

Omer Bar Tal

@omerbartal

4 months

Great colab with: @hila_chefer @omer_tov Charles Herrmann @Roni_Paiss @ShiranZada @arielephrat @JunhwaHur Yuanzhen Li, Tomer Michaeli @oliver_wang2 @DeqingSun @talidekel @InbarMosseri Webpage: Arxiv:

Lumiere: A Space-Time Diffusion Model for Video Generation

We introduce Lumiere -- a text-to-video diffusion model designed for synthesizing videos that portray realistic, diverse and coherent motion -- a pivotal challenge in video synthesis. To this end,...

arxiv.org

1

0

36

Omer Bar Tal

@omerbartal

4 months

A cool feature is that we can generate videos in a desired style image 🎨

1

3

36

Omer Bar Tal

@omerbartal

1 year

MultiDiffusion was accepted to ICML 2023! 🎊

Omer Bar Tal

@omerbartal

1 year

Excited to share "MultiDiffusion"! A controlled image generation framework w/ pre-trained text-to-image diffusion model. * Spatial guidance controls (bounding boxes/masks) * Arbitrary aspect ratios (huge Panoramas!) NO training NO finetuning. [1/3] @YarivLior @lipmanya @talidekel

8

63

326

1

2

37

Omer Bar Tal

@omerbartal

2 years

Thanks @_akhaliq for sharing our work! Our code is now available at :)

GitHub - omerbt/Text2LIVE: Official Pytorch Implementation for "Text2LIVE: Text-Driven Layered...

Official Pytorch Implementation for "Text2LIVE: Text-Driven Layered Image and Video Editing" (ECCV 2022 Oral) - omerbt/Text2LIVE

github.com

AK

@_akhaliq

2 years

github:

1

3

28

1

9

36

Omer Bar Tal

@omerbartal

10 months

Thanks @_akhaliq for sharing our work!

AK

@_akhaliq

10 months

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual…

9

121

606

0

4

31

Omer Bar Tal

@omerbartal

2 years

@_akhaliq Text2LIVE is now in @huggingface , thanks to @multimodalart 😀 check it out at

Text2LIVE - a Hugging Face Space by weizmannscience

huggingface.co

1

8

33

Omer Bar Tal

@omerbartal

4 months

Common video models generate distant keyframes, followed by temporal super resolution -- an approach that inherently struggles to generate globally coherent motion

1

0

24

Omer Bar Tal

@omerbartal

1 year

In Text2LIVE () we used CLIP guidance, but apparently we can also use Stable Diffusion for text-driven video editing!

4

3

24

Omer Bar Tal

@omerbartal

4 months

@_akhaliq Thanks @_akhaliq for sharing our work! Check out more details in

Omer Bar Tal

@omerbartal

4 months

Introducing Lumiere 📽️ The new video diffusion model we've been working on @GoogleAI * Text-to-Video * Image-to-Video * Stylized Generation * Inpainting * Cinemagraphs and more 🎨 W/ amazing team incl. @hila_chefer @omer_tov @InbarMosseri @talidekel @DeqingSun @oliver_wang2

53

146

776

5

2

23

Omer Bar Tal

@omerbartal

1 year

Our key idea is to define a new generation process, based on an optimization task that binds together multiple diffusion paths. The optimal solution is given in closed-form, and can be found analytically, without a computational overhead. [2/3]

2

1

19

Omer Bar Tal

@omerbartal

1 year

Visit our project webpage for more details, results, and code 🥳 Arxiv: [3/3]

MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation

Recent advances in text-to-image generation with diffusion models present transformative capabilities in image quality. However, user controllability of the generated image, and fast adaptation to...

arxiv.org

2

1

18

Omer Bar Tal

@omerbartal

9 months

First @huggingface event in Israel 🤗 Excited to present alongside this great group!

Linoy Tsaban🎗️

@linoy_tsaban

9 months

LETS GO🤩 (First!) Hugging Face meetup in Tel Aviv, September 4th🤗 Featuring an amazing group of speakers🔥: @hila_chefer @MokadyRon @RinonGal @EladRichardson @omerbartal You have a cool demo you’d like to showcase? Demo registration also is open! 🚀:

4

8

57

0

12

Omer Bar Tal

@omerbartal

10 months

Check out our recent work "TokenFlow"🎉 We show how to achieve consistent video editing with text2image diffusion model, by ensuring that the diffusion features are consistent throughout the generation! Webpage:

michal geyer

@GeyerMichal

10 months

Excited to share our new paper "TokenFlow: Consistent Diffusion Features for Consistent Video Editing"! 📽️ A framework for consistent video editing using text-to-image diffusion model, without additional training or finetuning. W/ the amazing @omerbartal @OneViTaDay @talidekel

5

15

72

2

0

9

Omer Bar Tal

@omerbartal

2 months

@Michael_J_Black SF ads are great!

0

7

Omer Bar Tal

@omerbartal

8 months

More details in the thread🧵

michal geyer

@GeyerMichal

10 months

Excited to share our new paper "TokenFlow: Consistent Diffusion Features for Consistent Video Editing"! 📽️ A framework for consistent video editing using text-to-image diffusion model, without additional training or finetuning. W/ the amazing @omerbartal @OneViTaDay @talidekel

5

15

72

1

0

7

Omer Bar Tal

@omerbartal

4 months

@bilawalsidhu Thanks for sharing our work! Check out our YouTube video, with sound on 🔉 ✨

Lumiere

The video accompanying our paper: "Lumiere: A Space-Time Diffusion Model for Video Generation".

www.youtube.com

1

7

Omer Bar Tal

@omerbartal

3 months

@ykilcher Thanks for featuring our work!

2

0

6

Omer Bar Tal

@omerbartal

2 years

Excited to share that our paper was accepted to CVPR 2022!

Tali Dekel

@talidekel

2 years

"Splicing ViT Features for Semantic Appearance Transfer", CVPR’22! If you want to know what it is NOT: NO pre-trained GANs or Diffusion models NO input segmentation masks NOT restricted to specific image domains @omerbartal @tnarek99 @OneViTaDay

0

29

147

1

0

6

Omer Bar Tal

@omerbartal

1 year

w/ @talidekel @RafailFridman @DanahYatim

0

5

Omer Bar Tal

@omerbartal

9 months

@EladRichardson @kfir99 @yuvalalaluf @DanielCohenOr1 Cool work!

0

5

Omer Bar Tal

@omerbartal

5 months

Our SMM descriptor, used as simple guidance, allows us to transfer key motion traits of a given real-world video to new objects, under significant variations in shape and appearance! No training/fine-tuning is required 🥳 More details in [3/3]

0

1

5

Omer Bar Tal

@omerbartal

2 years

@raphaelmilliere @talidekel @dolevofri @RafailFridman @yoni_kasten code is now available at :)

GitHub - omerbt/Text2LIVE: Official Pytorch Implementation for "Text2LIVE: Text-Driven Layered...

Official Pytorch Implementation for "Text2LIVE: Text-Driven Layered Image and Video Editing" (ECCV 2022 Oral) - omerbt/Text2LIVE

github.com

1

0

4

Omer Bar Tal

@omerbartal

4 months

@inbar_mosseri @hila_chefer @omer_tov @Roni_Paiss @ShiranZada @arielephrat @JunhwaHur @oliver_wang2 @DeqingSun @talidekel Thank you @InbarMosseri and all the team! 🎊

0

4

Omer Bar Tal

@omerbartal

3 months

@EladRichardson @kfir99 @yuvalalaluf Congrats!

0

4

Omer Bar Tal

@omerbartal

5 months

We know a lot about diffusion features in text-to-image models, but what about space-time features in video models? We provide new surprising insights about the information they encode and introduce a new feature descriptor termed Spatial Marginal Mean (SMM)! [2/3]

1

3

Omer Bar Tal

@omerbartal

5 months

@TheGraphicsFrog @CSProfKGD It's very similar to MultiDiffusion () in which we generalize the diffusion process to different image/condition domains

2

0

3

Omer Bar Tal

@omerbartal

1 year

Great work by @OmriAvr , congrats!

Omri Avrahami

@OmriAvr

1 year

[1/5] Always wondered what people see when looking at a Rorschach test? SpaText - our recent #CVPR2023 paper from @MetaAI may give you a sneak peek! TL;DR: We extend text-to-image models with region-specific textual controllability. Project Page:

4

52

135

0

3

Omer Bar Tal

@omerbartal

1 year

Thank you @twominutepapers for covering our paper!:)

Two Minute Papers

@twominutepapers

1 year

New Video - This New AI Is The Future of Video Editing!

0

3

21

0

3

Omer Bar Tal

@omerbartal

5 months

@YVinker @SIGGRAPHAsia @kusichan @DanielCohenOr1 As usual 🤩 congrats Yael!

1

0

1

Omer Bar Tal

@omerbartal

1 year

@johnowhitaker You may also be interested in our "appearance loss" (based on ViT features) which is suitable for photorealistic transfer

0

3

Omer Bar Tal

@omerbartal

1 year

@EladRichardson @CVPR @galmetz @OPatashnik @RGiryes @DanielCohenOr1 Cool work! Congrats 🎉

0

2

Omer Bar Tal

@omerbartal

8 months

@natanielruizg CLIP aesthetic score works well in many cases

GitHub - LAION-AI/aesthetic-predictor: A linear estimator on top of clip to predict the aesthetic...

A linear estimator on top of clip to predict the aesthetic quality of pictures - LAION-AI/aesthetic-predictor

github.com

1

2

Omer Bar Tal

@omerbartal

3 months

@heyshrutimishra Thank you @heyshrutimishra for featuring our work! For best experience, watch the video with sound on 🔉:)

0

2

Omer Bar Tal

@omerbartal

4 months

@DotCSV Thanks for sharing our work!

0

2

Omer Bar Tal

@omerbartal

3 months

@rowancheung Thank you @rowancheung for featuring our work! 🙏

1

0

2

Omer Bar Tal

@omerbartal

1 year

@CSProfKGD @docmilanfar @algreco_ RED can also give another perspective to the SDS loss of DreamFusion, where there is no need to backprop through the denoiser (or, the diffusion model)

1

0

2

Omer Bar Tal

@omerbartal

6 months

@OmriAvr @GoogleAI Congrats! 🎊

0

1

Omer Bar Tal

@omerbartal

6 months

@ShellySheynin Amazing work @ShellySheynin congrats!

0

1

Omer Bar Tal

@omerbartal

26 days

@mmbronstein A bit far from La Fenice, but Il Paradiso Perduto is very nice!

0

1

Omer Bar Tal

@omerbartal

1 month

@giffmana What about Unet?🧐

1

0

1

Omer Bar Tal

@omerbartal

8 months

@YVinker Already got inspired 😉 Amazing work @YVinker !

0

1

Omer Bar Tal

@omerbartal

2 years

@YVinker Congrats!!🎉

0

1

Omer Bar Tal

@omerbartal

11 months

@hila_chefer Amazing work! 👏

0

1

Omer Bar Tal

@omerbartal

5 months

@demi_guo_ Congrats @demi_guo_ , looks amazing!

1

0

1

Omer Bar Tal

@omerbartal

3 months

@twominutepapers Thanks @twominutepapers for featuring our work!

0

1

Omer Bar Tal

@omerbartal

4 months

@minchoi Thanks for sharing our work! 🎥

2

0

1

Omer Bar Tal

@omerbartal

8 months

@natanielruizg Yes, it was actually used for filtering high-quality images in LAION for training SD. A score of ~6 is usually good

1

0

1

Omer Bar Tal

@omerbartal

4 months

@OPatashnik @iclr_conf @KatzirOren @DaniLischinski @DanielCohenOr1 Congrats! 👏

0

1

Omer Bar Tal

@omerbartal

1 year

@EMostaque You might be interested in MultiDiffusion :) Utilizes StableDiffusion to fuse unrelated crops into a coherent panorama

Omer Bar Tal

@omerbartal

1 year

Our key idea is to define a new generation process, based on an optimization task that binds together multiple diffusion paths. The optimal solution is given in closed-form, and can be found analytically, without a computational overhead. [2/3]

2

1

19

0

1

Omer Bar Tal

@omerbartal

6 months

@linoy_tsaban @multimodalart @huggingface Congrats! 🥳

0

1

Omer Bar Tal

@omerbartal

5 months

@DrJimFan Cool results! This falls under our MultiDiffusion formulation (ICML 23)

0

1

Omer Bar Tal

@omerbartal

1 year

@ajayj_ @poolio @BenMildenhall @jon_barron Amazing work!

0

1

Omer Bar Tal

@omerbartal

3 months

@EMostaque @pika_labs @demi_guo_ @chenlin_meng Thanks @EMostaque !

0

1

Omer Bar Tal

@omerbartal

1 year

@ShellySheynin @MetaAI @adam_polyak90 @urielsinger @NachmaniEliya Congrats!

0

1

Omer Bar Tal

@omerbartal

1 year

@hila_chefer @YarivLior @lipmanya @talidekel Thanks @hila_chefer :)

0

1

Omer Bar Tal

@omerbartal

1 year

@giffmana @YarivLior @lipmanya @talidekel Thanks! Ohh definitely a typo, but a cool idea to try ;)

0

1

Omer Bar Tal

@omerbartal

8 months

@Suhail @MrUgleh Integrating the denoising predictions over sliding windows, in order to apply QR-ControlNet in arbitrary resolution (can also change the prompt over different crops)

0

1

Omer Bar Tal

@omerbartal

10 months

@hila_chefer Thanks @hila_chefer :)

0

1

Omer Bar Tal

@omerbartal

2 years

@hila_chefer @eccvconf Congrats!!