📢Our new
@NVIDIAAI
paper is out!
I'm happy to share Compact NGP with Learned Hash Probing, which offers NeRFs that are 3.5x smaller than Instant NGP at a ~1.26x training cost and equivalent (or faster) inference speeds!
🌐:
🎥:
📢 Our new
@NVIDIAAI
paper is up on arXiv!
I'm happy to share Neural Geometric Level of Detail, which enables the first ⚡real-time rendering of high quality neural
#3d
SDFs with a sparse octree feature volume and a tiny MLP!
Jensen revealed the
@NVIDIAAI
text-to-3D (mesh models!) strategy at the GTC Keynote today. I'm super proud to be a part of this epic effort alongside my amazing coworkers 😊
Find out more here:
"3D Object Reconstruction and Representation using Neural Networks" published in GRAPHITE 2004 by Peng et al. proposed neural implicit surfaces ages before anyone else did. Visionary work (even had a cat!) which doesn't seem to get the credit it should!
📢 Our new SIGGRAPH 2022 paper is on arXiv!!
I'm happy to share Variable Bitrate Neural Fields, which lets you download ultra compact & high fidelity & fast NeRFs and neural fields at ⚡ blazing speeds with progressive LOD!
🌐:
🎥:
📢 Our new library is on GitHub!!
I'm happy to share NVIDIA Kaolin Wisp, a PyTorch library which provides building blocks and interactive apps to build your own complex neural fields. Presented with a nerf snippet of wisp development.
💻:
📢Here's something cool we've been working on: efficient sparse octree primitives for 3D deep learning with PyTorch. This enables training NGLOD 3x faster with 30x less memory, which also means it can fit 30x more complex models! Code available here:
📢 Our new
@NVIDIAAI
paper is up on arXiv!
I'm happy to share Neural Geometric Level of Detail, which enables the first ⚡real-time rendering of high quality neural
#3d
SDFs with a sparse octree feature volume and a tiny MLP!
We have openings for a research engineering internship at NVIDIA to work on open source software (eg ) with us. If you want to contrib to cutting edge research for 3D generative modeling, editing and nerfs, send me a DM/email! Undergrads welcome :)
Compact NGP was my last project at NVIDIA.
It's been a wild 3.5 years (almost 5 incl. internships) and I already miss my truly amazing coworkers- but here's to fun new exciting things in 2024. 😄
(now I need to buy GPUs 😭)
We had a blast at
#SIGGRAPH2022
last week demoing our new modular open source
#neuralfields
NeRF library, Kaolin Wisp. In case you missed the session... the video recording of the overview talk and the Wisp tutorial from
@OrPerel
is available online!
📢 Our new
@NVIDIAAI
paper is up on arXiv!
I'm happy to share Neural Geometric Level of Detail, which enables the first ⚡real-time rendering of high quality neural
#3d
SDFs with a sparse octree feature volume and a tiny MLP!
📢Are you looking for a consolidated overview of coordinate-based neural networks in 3D reconstruction, view synthesis, shape/appearance, etc? We ‘plowed through’ 250+ papers to write a review of ‘neural fields’ in visual computing and beyond: (1/10)
Over the years I've been collecting papers on "Compression for 3D" (broadly defined) on GitHub which helps for making related works. Maybe it would be useful for you too & it's likely missing lots of exciting works so PRs are always welcome 😁
We had a huge showing (500+ people!) for our SIGGRAPH course on Neural Fields. Thanks to everyone who came despite the early morning slot and also s/o to my awesome co-instructors
@jtompkin
@psyth91
@alexyu00
😄
We've spent a lot of thought about why the "neural" part of "neural fields" matters at all. It seems to boil down to 3 things:
1. Compression
2. Inductive biases that makes fitting ('learning') easy
3. Flexible input space -> creative applications at low cost
PhD application season is coming up, so here's a
#LumaFlythroughs
of the DGP Lab at University of Toronto - a massive lab space that unites numerous students and faculty from graphics, vision, imaging, HCI, and more. My time here has been spectacular- reach out if you have Qs!
I enjoy reading old papers, if only to learn about the design constraints that existed during those times & think about how modern and future design constraints are different. I collected old (<2006) NeurIPS papers on computer graphics here:
Zero-1-to-3: Zero-shot One Image to 3D Object
TL;DR: We learn to control the camera perspective in large-scale diffusion models, enabling zero-shot novel view synthesis and 3D reconstruction from a single image.
abs:
I am very excited and honored to have been selected as one of the 10 recipients of the
@BorealisAI
Fellowship!! This is really a reflection of the amazing colleagues and advisors that I get to work with, and could not thank them enough for their support throughout the years. 😊
We are thrilled to introduce the 2023
@BorealisAI
Fellowship recipients - exceptional researchers advancing
#AI
&
#ML
. Supported by Canada’s top institutions and leading researchers, they aim to make significant contributions to the field. Learn more:
🇨🇦⚗️Grad school apps are due soon, so I thought I'd share my article on CS grad school in Canada. Canada is an *amazing* place to do research in CS, with amazing schools like UofT, Waterloo, UBC, SFU, McGill, Alberta, UdeM, among many many others!
Spending time outside research makes me realize how disparate research can sometimes be from real world applied issues.
This is often a feature and not a bug; the goal of research is to make a dent into the boundary of human knowledge, not necessarily to make something
📢 Come checkout Magic3D🧙, our
#CVPR2023
Highlight✨ paper on text-to-3D generative AI at the AM poster session today (10:30AM-) at poster
#29
!!! Come ask lots of hard questions, or just come hangout and say hi 😄 we have lots of learnings to share!
Jensen revealed the
@NVIDIAAI
text-to-3D (mesh models!) strategy at the GTC Keynote today. I'm super proud to be a part of this epic effort alongside my amazing coworkers 😊
Find out more here:
Grad school deadlines are right around the corner, so I wanted to share this article again on CS grad school in Canada.
I also added some small updates based on my own experience as a grad student since then (details in thread):
📣 Last week we presented our freshly minted JCGT paper with
@AndrewGlassner
@CasualEffects
on a 3D SDF dataset & explorer at i3D. If you missed our presentation, here's a recording: . If you're busy, here's a twitter thread: 1/N
PhD application season is coming up, so here's a
#LumaFlythroughs
of the DGP Lab at University of Toronto - a massive lab space that unites numerous students and faculty from graphics, vision, imaging, HCI, and more. My time here has been spectacular- reach out if you have Qs!
Compare to meshes, SDFs are compact; a polygon mesh representing something as simple as a sphere requires numerous vertices. For a SDF, it’s a simple math equation. Even better, neural SDFs use MLPs, which have fixed computational cost.
Since it's PhD application season, I wanted to share my article again on CS graduate school in Canada. Feel free to reach out if you're applying to UofT CS especially for CG / CV and have questions! My time here has been overwhelmingly positive so far.
Some personal news: I am joining OpenAI (again :)). Like many others both in/out of AI, I am very inspired by the impact of their work and I have personally benefited greatly from it. The future potential is especially exciting; it is a great pleasure to jump back in and build!🪄
Just submit my last project to conclude my first year (of coursework) of my PhD. This year I took: Geometry Processing, Physics-based Animation, Seminar on Geometry, Computational Imaging and 3D Sensing, and Visual Computing Systems. Looking forward to a research-focused summer!
今日で謎のGPU帝国での仕事は終わり。若輩者の自分でも短期間でインパクトを残せるこれ以上ないくらい良い職場でした。ボスからも“いつでも帰っておいで”と言ってもらえた。
そんなわけで五月からはしばらく日本の謎のAIユニコーン社で頑張ります!
Excited for new adventures to come. :)
A nice benefit of upgrading your Python to 3.8... dataclasses. C-like structs without stupid boilerplate. Interpreter will catch your key typos. Built in __repr__. More expressive type-hinting. Less verbose, more like C😁
OpenAI has acquired the team at Global Illumination. The entire team has joined OpenAI to work on our core products including ChatGPT. The team previously designed and built products early on at Instagram and Facebook and have also made significant contributions at YouTube,
🚨 Tomorrow at
@EG2022_Reims
at 9AM,
@YihengXie
and I will present our Eurographics STAR report on Neural Fields, an up-and-coming signal representation for computer graphics and computer vision! Be there or be ⬜!
A major VFX studio owned by Netflix is hiring in a bunch of AI roles:
Generative imaging, workflow design, model training, data acquisition, and even ML researchers
We're going to be seeing a lot more of this...
We think it’s important to explore whether neural SDFs are viable for real-world graphics applications. Enabling them to be rendered in real-time is just our first step, and we hope more great things are in line for SDFs!