Introducing Marigold-LCM 🌼 - a FAST version of the now-popular state-of-the-art depth estimator! Check out the new 🤗 demo (now accepts videos):
New on the team: Tianfu Wang (
@TianfuWang2
)
Original Marigold:
Introducing Marigold 🌼 - a universal monocular depth estimator, delivering incredibly sharp predictions in the wild! Based on Stable Diffusion, it is trained with synthetic depth data only and excels in zero-shot adaptation to real-world imagery. Check it out:
🌐 Website:…
Introducing Marigold 🌼 - a universal monocular depth estimator, delivering incredibly sharp predictions in the wild! Based on Stable Diffusion, it is trained with synthetic depth data only and excels in zero-shot adaptation to real-world imagery. Check it out:
🌐 Website:…
Point2CAD: A leap in mechanical CAD model reconstruction from point clouds, blending semantic segmentation with a novel neuro-analytic surface representation. Discover more:
🔗 Website:
⚙️ Paper:
🛠️ Code:
Want to learn how to lift Midjourney v6 (and before) into 3D, and 3D-print it? Follow the link to our free online tool: , and follow me to stay updated!
Since Marigold depth launch yesterday, we've seen amazing uses: 3D printing, holographic images, UE5 assets, ControlNet, and optical illusion disentanglement. Share your creations with us! Use our free tool and follow
@KBingxin
for updates!
#CVPR2024
results are in -- time for the humblebrag fest! Happy to share that Marigold Depth and Point2CAD are heading to Seattle this year! Congrats to the amazing PRS-ETH team!
Marigold:
Point2CAD:
@pvncher
Hey Eric, I took this idea and ran with it, learning a lot along the way. I made a simple Jupyter Notebook that people can run to generate and upload a sbs video to Quest. Dev can use whatever depth model (I tested with PatchFusion and Marigold).
Excited to share that my semester student
@hexagon_prime
did unimaginable by shaping his thesis (4 months of part-time) into a
#CVPR2022
submission, which got accepted and already gained a lot of attention in the community. FYI: he is on the lookout for PhD positions!
I will present "torch-fidelity: High-fidelity performance metrics for generative models in PyTorch" at PyTorch Ecosystem Day
#PTED21
on April 21, 2021.
Register at
Project and code:
Excited to share that I successfully passed my Ph.D. defense on the topic "Tensor Decompositions in Deep Learning." Huge thanks to
@lucvangool
,
@CVL_ETH
, and everyone involved for the amazing journey!
@kwea123
We focus on the quality first - speed will come later through research and improvements in diffusion samplers. Arguably, Stable Diffusion is also slower than most of the GANs :)
Marigold 3D printing demo will convert your image into a watertight 3D asset in STL format, readily compatible with any 3d printer software or online printing services. It's a great opportunity to make customized gifts for your loved ones! 🎁
🎨 Excited to share our latest paper: "Breathing New Life into 3D Assets with Generative Repainting"! We present a novel approach to repaint 3D models using text prompts, pretrained Stable Diffusion, and NeRF. A step towards assisting content creators! 🖌️
📄 Paper:…
1/ Our paper “Spectral Tensor Train Parameterization of Deep Learning Layers” about end-to-end neural network compression and stability of training in the GAN setting is live at
#AISTATS2021
this week!
PDF:
Project page:
Open AI introducing Sora
text-to-video model
Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions.
Our paper “T-Basis: a compact representation for neural networks” is live at ICML! We learn compressed neural networks by reparameterizing weights twice: through tensor ring (TR) decomposition, and each TR core through a shared basis. Project page:
Example of what AI generation can do in practice currently, from initial image (which is also generated) to a very detailed printable model, the level of detail mostly made possible with the Marigold depth estimation.
#ai
#3dprinting
#dragon
#technology
Our
#ICCV2021
paper on Adaptive Task-Relational Context, leveraging neural architecture search and attention mechanisms in the multi-task learning setting, is now paired with source code!
@menelaoskanakis
@stam_g
paper:
code:
@CSProfKGD
Thanks for pointing out! As a reviewer, I noted that CMT did not send me a notification about
#CVPR2021
rebuttals reveal. As an author, it makes me worry the reviewers may not remember to check my rebuttal.
Check out my new Python package for evaluation of GANs in
@PyTorch
with
- simplicity (import or CLI command)
- extensibility (swap Inception with your own feature extractor)
- fidelity (epsilon relative errors compared with original codes)
in mind.
Glad to see that tensor methods are re-emerging in model compression. Even more glad, knowing that our paper "Spectral Tensor Train Parameterization of Deep Learning Layers" was recently accepted at AISTATS'2021. Links to follow
Code packaging frenzy continues! Check out my latest python package, democratizing orthogonal transformations in
#PyTorch
. Goomba has nothing to do with the Householder transformation; your attention is all I need.
As
#ICCV2023
accepted papers promotions slow down, here's a shameless plug: I'm delighted to share that 2/3 of our submissions made the cut. The 1st, "DiffDreamer," enables dream-like generative experiences of nature's trailing views. Follow
@prime_cai
for updates!
Ever wondered if there is anything more efficient for
#RL
returns than a
#PyTorch
loop,
#TF
.scan, or
#SciPy
.lfilter? Check out my algorithm for parallel discounted cumulative sum computation! `pip install torch-discounted-cumsum`
1st day of
@ICCVConference
poster session. With 6000 posters over 3 days, my routine is to run all the rows and later revisit some. It worked great during
@CVPRConf
, but the narrow spacing between the rows makes it impossible this time. 🤞 poster spacing standardized
@AntonObukhov1
Thank you for creating this app. I can upload AI images into Marigold. Create STL file. Download to my computer. Upload STL into my cnc router software and within 5 minutes of adjusting image, I have a 3D bas relief suitable for wood carving. I'm overwhelmed with joy!
@RealAAAI
what is the threshold score for NeurIPS fast track this year, given the changed scoring system? Previously 4.9 corresponded to the first grade of reject. The same threshold this year translates to the first grade of accept.
#aaai2023
#NeurIPS2022
Our
#NeurIPS2022
paper on reducing the computational complexity of controlling the _real_ singular values of convolutional layers is online: None of us could go present in person, but feel free to contact Alexandra (email in pdf) or me to discuss! 🧵
2M downloads just half a year later! Thanks to the huge interest in generative models and a large community of researchers. An upcoming version 0.4.0 is planned to include more recent metrics, including FID_CLIP and more. Leave a feature request in this thread!
#pytorch
#metrics
How can we end the year without a new release of 🧨 diffusers to celebrate it?
1️⃣ A brand new model, aMUSEd (more soon from
@psuraj28
and
@williamlberman
), which is NOT based on diffusion 😅
2️⃣ 3x faster SDXL (extends to other pipelines) 🏎️
3️⃣ Transitioning to PEFT for LoRA…
Ever wanted to explore image scenes from a new perspective? Our cutting-edge depth estimator breaks through the fourth wall, bringing every pixel closer to you! DIY:
@ylecun
These generative world models are nevertheless very useful for distilling into sota perception models. We showed it on the task of monocular depth estimation in our Marigold paper:
Point2CAD operates seamlessly on top of any point cloud segmentation or clustering method, with results directly fed into analytic models. Our approach utilizes a novel implicit neural representation for freeform surface fitting.
The team: Yujia Liu (
@YujiaLiu14
), Anton Obukhov…
My colleague
@CSakaridis
, who works on making self-driving cars safer to use in harsh weather conditions, has just released a
#cityscapes
-inspired benchmark, a paper, and joined Twitter. Welcome Christos!
ACDC released: a new large-scale driving dataset for training and testing semantic segmentation algorithms on adverse visual conditions, such as fog, nighttime, rain, and snow. The dataset and benchmarks are available at
#computervision
#autonomouscars
I have been learning German for more than a year now, and recently, I realized that I need a principled way to learn new words from the textbook as I progress through it. Unfortunately, neither Anki nor Quizlet had the sets I needed, so I decided to make them myself with AI 🧵⬇️
@mattwallace
@yacineMTB
@Tesla
Marigold is tailored for single-image scene inference. In contrast, cars handle video streams, analyzing temporal data. They use multiple cameras and advanced multiview geometry algorithms for precise distance estimation.
I will be presenting our latest work on tensorial neural fields compression at the 3rd workshop on neural low‑dimensionality
#slowdnn
organized by
@mbzuai
Here is a game to spice up the coming review periods: Reviewer Bingo! Tick out a row/col/diagonal, mutter "BS!", and proceed with the review: "The paper is well-written and easy to follow..."
Hey
@overleaf
and
@grammarly
, we need you integrate with each other, badly. We write TeX, we appreciate great spell-checking, and we wish to unlearn the tag-stripping skill!
@heykahn
We already saw how it panned out for self-driving cars: who is going to be held accountable? Before this question is answered, jobs aren't going anywhere
️🎉⚡️ Now available,
#TorchMetrics
v0.4!
We've made improvements to the library and added new Audio and Image domains with new metrics!
Check out Nicki Skafte's blogpost to learn more:
Introducing Marigold 🌼 - a universal monocular depth estimator, delivering incredibly sharp predictions in the wild! Based on Stable Diffusion, it is trained with synthetic depth data only and excels in zero-shot adaptation to real-world imagery. Check it out:
🌐 Website:…
@sameerishwar
After you create the depth map in our demo, scroll down below the examples -- you will find three sliders on the left. Create 3D, clear, adjust, repeat. HTH!
Are conference deadlines getting any less thrilling?.. Has anyone tried submitting a paper a week in advance? I would be mildly interesting to see a histogram of uploads to cmt on the range from the call for papers to the deadline
@popcornflix
Not out of the box -- it is for single image inference. Still, will probably do better than other non-temporal algorithms, such as MiDaS/LeRes. If you have good findings, please share!
Wouldn't it be great if (online) conference registration fee covered international shipping of swag? My last conference mug is from
@NeurIPSConf
'19, and every day feels more like a collectible
@ICCV_2021
@CVPRConf
Now that
#PyTorch
1.9 is out, I got to play with the revamped linalg module. I'm glad I convinced the maintainers to rescue the Householder transformation fn initially slated for removal - ORGQR. It got a new name (householder_product), GPU, batch, and autograd support. We show
@noodlecakeai
@KotLesny
Can you please create an issue on our github page to troubleshoot the 24GB memory issue? Please mention the command line you used and the input image resolution so that we can try to reproduce it. For the speed, yes, it is currently slow -- stay tuned for the next releases!
🎨 Excited to share our latest paper: "Breathing New Life into 3D Assets with Generative Repainting"! We present a novel approach to repaint 3D models using text prompts, pretrained Stable Diffusion, and NeRF. A step towards assisting content creators! 🖌️
📄 Paper:…