Andrew Davison @AjdDavison Twitter profile

Pinned Tweet

Andrew Davison

2 years

Robot Web: breakthrough many-robot localisation. Uses efficient, general message passing over dynamic graphs. Accurate and highly robust to sensor/comms failure. @rmurai0610 @joeaortiz @SaeediG @paulhjkelly Full video demo: Paper:

11

75

334

Last Seen Profiles

@indiainbrazil

@ocha_norma

@kaedechanura

@SYUREKINTV

@Juanag3108

@TrojanSeijo

@classicallygia

@ozgurozturknet

@muzaffer1617

@hcolombi_3

@MeatSeuss

@ZekkoFin

@VentoNacho

@PortaldoLeao

@RunPureSports

@AyakaOhashi

@gutdoh

@Bisan_Shrafi

@TeamKIoppo

@tom_nuttall

@JaneMahony

@ShaggySquid

@visiblemaker

@dul_turkporno

@Qwak_ai

@ANTIDOPINGS_EU

@ceskybets

@cabellan_

@Tordorf

@MarlenPesochin3

@playbackpt

@dul_turkporno

@pstisida

@curiosity51db

@Sello_ire

@PHMCtweets

Andrew Davison

@AjdDavison

1 year

Many cool demos in the exhibition hall at #ICRA2023 of advanced robots of all types. I liked this one which seems quite simple in comparison but just very clearly showing very fast and precise motor control.

4

185

992

Andrew Davison

@AjdDavison

2 years

New: 3D neural fields like NeRF do automatic, unsupervised semantic scene decomposition. We reveal it with tiny interactions. Real-time SOTA room segmentation from 140 clicks; zero prior data! iLabel, Dyson Robotics Lab, @SucarEdgar @Shuaifeng_Zhi et al.

4

149

671

Andrew Davison

@AjdDavison

3 years

This summer I've been working to finally understand Lie Theory, the basis for proper estimation on over-parameterised manifolds like SE(3). There are some great tutorials for the roboticist out there; I especially like Micro Lie Theory by Solà et al.

13

109

640

Andrew Davison

@AjdDavison

1 year

Very proud, and still a bit shocked, to share that I've been elected Fellow of the Royal Society. Thank you to my students and collaborators from Imperial College, Oxford, AIST, Dyson, Slamcore and beyond, and of course to my supportive family and friends!

The Royal Society

@royalsociety

1 year

We are very happy to announce the eighty exceptional scientists elected as Fellows of the Royal Society this year, selected for their outstanding contributions to the advancement of science. Meet the new Fellows and find out more about their research:

16

68

294

85

28

447

Andrew Davison

@AjdDavison

3 years

What an honour to win a PAMI Helmholtz Award at #ICCV2021 . DTAM (ICCV 2011) is a paper with few results, and barely got in as a poster. But once @rapideRobot and @stevenjl got the big laptop to Barcelona we showed people a live dense SLAM and AR demo they had never seen before!

16

40

423

Andrew Davison

@AjdDavison

3 years

iMAP is a new way to do SLAM: we learn an implicit neural representation *in real time* and track an RGB-D camera against it. The implicit map fills holes; completes the unseen backs of objects; and maps a whole room in only 1MB of weights. From the Dyson Robotics Lab, Imperial.

Edgar Sucar

@SucarEdgar

3 years

Excited to share iMAP, first real-time SLAM system to use an implicit scene network as map representation. Work with: @liu_shikun , @joeaortiz , @AjdDavison Project page: Paper:

10

119

518

6

80

411

Andrew Davison

@AjdDavison

5 years

The thing I'm proudest of in my career is the work I've done with the PhD students I've supervised at Imperial College. Their final PhD theses are sometimes hard to find so I've gathered links to all of them on my homepage at ... please have a look! 1/n

4

60

384

Andrew Davison

@AjdDavison

3 years

Yes! Zero to One research is also something I'm always trying to do and explain to my students. I strongly feel it's too early for benchmarking in much of vision/robotics/AI, when basic things are still not possible. We need demos of new capability, not tables!

2

47

363

Andrew Davison

@AjdDavison

3 years

#ICCV2021 , Dyson Robotics Lab at Imperial: iMAP is the first SLAM system based on continual, real-time learning of an implicit neural representation. In 3 minutes a 1MB MLP model captures global shape and detail, with convincing scene completion despite no prior training data.

Edgar Sucar

@SucarEdgar

3 years

We will be presenting our new real-time SLAM system iMAP at #ICCV2021 ! With a neural implicit scene representation it can map scenes efficiently, fill holes, and jointly optimise the 3D map and camera poses work with: @liu_shikun , @joeaortiz , @AjdDavison

4

77

307

1

58

339

Andrew Davison

@AjdDavison

2 years

Who else can smell the end of big-data supervised learning in the air? @ylecun 😃 Certainly densely hand-labelled image datasets don't make much sense to me after using iLabel.

Andrew Davison

@AjdDavison

2 years

New: 3D neural fields like NeRF do automatic, unsupervised semantic scene decomposition. We reveal it with tiny interactions. Real-time SOTA room segmentation from 140 clicks; zero prior data! iLabel, Dyson Robotics Lab, @SucarEdgar @Shuaifeng_Zhi et al.

4

149

671

7

41

329

Andrew Davison

@AjdDavison

3 years

All researchers should fight against this. Every week I try to persuade my students that top papers often have few quantitative results. With work that's new, important, and clearly qualitatively different (zero to one!), you don't need quantitative results. Demos not tables!

Peyman Milanfar

@docmilanfar

3 years

The author's dilemma, circa 2021

18

137

1K

3

54

303

Andrew Davison

@AjdDavison

2 years

Using Gaussian Belief Propagation as in Robot Web, we now show dynamic multi-robot *planning* via p2p comms, no central solver needed. In a scaled simulation, cars slide closely past each other at motorway speeds. @AalokPat @rmurai0610 , Dyson Robotics Lab,

10

54

306

Andrew Davison

@AjdDavison

1 year

Just an implementation of the Dynamic Window Approach planner (essentially sampling-based short-range MPC) I did for teaching. A reminder of how cool simple non-learned planning can be when perception is assumed solved so the planner has full state knowledge.

1

37

299

Andrew Davison

@AjdDavison

2 years

New: Feature-realistic neural fusion for real-time, open set scene understanding. Our neural field renders to feature space, enabling real-time grouping and segmentation of similar objects or parts from ultra-sparse, online interaction. Dyson Robotics Lab.

2

39

293

Andrew Davison

@AjdDavison

2 years

Can anyone explain if there is a difference between unsupervised and self-supervised learning? To me they seem the same and I find myself using both terms interchangeably (I prefer unsupervised), but I feel like I'm confusing people who understand them to mean different things.

43

26

290

Andrew Davison

@AjdDavison

2 years

Great live demos in Marco Hutter's keynote at #ICRA2022 . And they had a video of the robot doing the Rocky steps and celebration here in Philadelphia!

3

58

275

Andrew Davison

@AjdDavison

3 years

We add semantics outputs to NeRF models of 3D occupancy/colour. Joint representation allows very sparse or noisy in-place supervision to generate high quality dense prediction. Dyson Robotics Lab #ICCV2021 Oral @Shuaifeng_Zhi @tlaidlow @StefanLeuteneg1 .

4

46

271

Andrew Davison

@AjdDavison

2 years

New demo, with turns and swerves, of distributed real-time multi-agent planning/MPC; no central control needed. Uses GBP and p2p message passing over the joint factor graph so arbitrarily scalable and robust. @AalokPat @rmurai0610 Dyson Robotics Lab

7

24

215

Andrew Davison

@AjdDavison

1 year

I like the way that this bigger robot has to pick its way very carefully through the packed obstacles until a gap presents itself.

6

14

216

Andrew Davison

@AjdDavison

2 years

So according to new research from DeepMind, using 3D vision/SLAM tools to build an explicit representation of a scene is useful for robotics...who would have thought it?

Leonard Hasenclever

@lqh20

2 years

Creating photorealistic simulations of unstructured scenes is hard. Using NeRFs we turn 5min videos into simulations, train vision-guided policies for humanoid robots, and show zero-shot transfer to the real world! abs: project:

4

65

294

7

16

202

Andrew Davison

@AjdDavison

2 years

Semantic-NeRF! Super simple, just add semantic outputs to a NeRF network and you can label a full 3D scene from highly sparse or noisy annotations. #ICCV2021

Shuaifeng Zhi

@Shuaifeng_Zhi

3 years

Happy to introduce Semantic-NeRF. Multi-view consistency and smoothness make NeRF-training a label fusion process, supervised by sparse or noisy labels only! Work with: @tlaidlow , @StefanLeuteneg1 , @AjdDavison Project page: Paper:

6

55

311

0

16

192

Andrew Davison

@AjdDavison

3 years

Semantic labels are highly correlated with geometry and appearance. When we add semantic outputs to a neural implicit representation, very sparse or noisy supervision is enough to generate good quality labels for the whole scene. From the Dyson Robotics Lab at Imperial College.

Shuaifeng Zhi

@Shuaifeng_Zhi

3 years

Happy to introduce Semantic-NeRF. Multi-view consistency and smoothness make NeRF-training a label fusion process, supervised by sparse or noisy labels only! Work with: @tlaidlow , @StefanLeuteneg1 , @AjdDavison Project page: Paper:

6

55

311

1

20

193

Andrew Davison

@AjdDavison

20 days

Try it out: super-accurate real-time normal prediction, now release open source from the Dyson Robotics Lab at @ICComputing .

Gwangbin Bae

@BaeGwangbin

20 days

📢 Code release for 𝗗𝗦𝗜𝗡𝗘 ( #CVPR2024 - Oral) DSINE gives you surface normal prediction (+ uncertainty) in real-time. We have released the code for training, testing, and running real-time demos. Try it yourself!

7

116

644

1

30

192

Andrew Davison

@AjdDavison

3 years

Can anyone from explain to me how a (kestrel?) is able to achieve this stabilisation accuracy? It is high; ground not that textured; effectively monocular vision; why is rot/trans ambiguity not higher? Of course it has inertial and other cues too but still just surprised/amazed.

14

25

187

Andrew Davison

@AjdDavison

2 years

Not usually one to just agree with Elon, but I think he's saying the same thing here as I was in my last tweet: the hardest part of AI is perception: going from real sensor data to an efficient, but explicit, scene representation --- then your robot can do pretty much anything.

Elon Musk

@elonmusk

2 years

So much of AI is about compressing reality to a small vector space, like a video game in reverse

10K

9K

127K

13

22

180

Andrew Davison

@AjdDavison

3 years

We will be live demoing iMAP on Monday at #CORL2021 , which trains an MLP neural implicit model from scratch in seconds as a SLAM representation for both reconstruction and tracking. Come and try it! From the Dyson Robotics Lab at Imperial College.

Edgar Sucar

@SucarEdgar

3 years

We will be doing a live demo of iMAP at #corl2021 , sessions 1 and 8 Monday and Thursday, come if you're around! @liu_shikun , @joeaortiz , @AjdDavison

4

21

121

1

19

183

Andrew Davison

@AjdDavison

2 months

Code now available for Gaussian Splatting SLAM from @HideMatsu82 and @rmurai0610 , Dyson Robotics Lab at Imperial. Includes real-time monocular demo with various interactive visualisations. Also supports RGB-D. Looking forward to seeing what people will do with it! #CVPR2024

Hide

@HideMatsu82

2 months

Code release of Gaussian Splatting SLAM! #CVPR2024 As of now, our method is the only Monocular SLAM solely based on 3DGS. No depth information needed. Work with @rmurai0610 * @paulhjkelly @AjdDavison . (*Equal Contribution) Details in the thread:

12

100

576

2

27

181

Andrew Davison

@AjdDavison

3 years

My (probably controversial) idea to improve the state of publishing/reviewing in computer vision's overloaded main conferences: a limit (e.g. 3) on the number of papers that any individual can submit as co-author to one conference.

16

7

181

Andrew Davison

@AjdDavison

5 months

Robot Web: distributed, asynchronous message-passing for simple, accurate multi-robot localisation, at last officially published in IEEE Transactions on Robotics. Towards the inter-operable robot future! For me maybe my most important work since MonoSLAM.

Andrew Davison

@AjdDavison

2 years

Robot Web: breakthrough many-robot localisation. Uses efficient, general message passing over dynamic graphs. Accurate and highly robust to sensor/comms failure. @rmurai0610 @joeaortiz @SaeediG @paulhjkelly Full video demo: Paper:

11

75

334

4

29

164

Andrew Davison

@AjdDavison

3 years

I also couldn't believe how good that illusion was so I had to make one for myself tonight... it really works! @ankurhandos @SergeBelongie (It's called the Ames Window if you want to download your own template to print out.)

8

9

164

Andrew Davison

@AjdDavison

4 months

It's impressive what can be done with two robot arms teleoperated by a human brain, showing again (as was done 10+ years ago , though now with even more dexterity) that perception and intelligent planning are holding robotics back more than hardware.

Zipeng Fu

@zipengfu

4 months

Mobile ALOHA's hardware is very capable. We brought it home yesterday and tried more tasks! It can: - do laundry👔👖 - self-charge⚡️ - use a vacuum - water plants🌳 - load and unload a dishwasher - use a coffee machine☕️ - obtain drinks from the fridge and open a beer🍺 - open…

407

2K

7K

6

18

162

Andrew Davison

@AjdDavison

2 years

Neural scene models like NeRF can encode other properties, such as semantic maps. Joint representation means these maps share the coherence of occupancy/colour, allowing dense 3D prediction from very sparse or noisy in-place supervision (e.g. a fast 2D CNN, or clicks). #ICCV2021

Shuaifeng Zhi

@Shuaifeng_Zhi

3 years

Happy to introduce Semantic-NeRF. Multi-view consistency and smoothness make NeRF-training a label fusion process, supervised by sparse or noisy labels only! Work with: @tlaidlow , @StefanLeuteneg1 , @AjdDavison Project page: Paper:

6

55

311

0

22

159

Andrew Davison

@AjdDavison

2 years

DreamFusion is remarkable and I'm trying to understand how it works. If I understand correctly, the key thing is that a pre-trained diffusion model can take in some starting image and a text prompt and output a new image which is more like what the text describes. 1/n

Ben Poole

@poolio

2 years

Happy to announce DreamFusion, our new method for Text-to-3D! We optimize a NeRF from scratch using a pretrained text-to-image diffusion model. No 3D data needed! Joint work w/ the incredible team of @BenMildenhall @ajayj_ @jon_barron #dreamfusion

136

1K

6K

1

20

157

Andrew Davison

@AjdDavison

2 years

Super-accurate CAD model fitting to single RGB-D images, built into a real-time object-level SLAM system with scene graph optimisation and camera tracking. AR examples show the value of using semantically tagged object models. SLAM++ @nazcaspider etal:

2

16

154

Andrew Davison

@AjdDavison

4 years

My talk today is available to watch in full here. It was a great experience and thanks again to all of the organisers. Looking forward to the upcoming talks!

RoboticsTodaySeminar

@RoboticsSeminar

4 years

3

26

61

6

34

150

Andrew Davison

@AjdDavison

6 years

What do future #SpatialAI systems have to do, and how will they work, as we bring together probabilistic and geometric computer vision with deep learning and the ongoing developments in sensing and processing hardware. Read about FutureMapping at .

4

41

147

Andrew Davison

@AjdDavison

5 months

Let's just remember how good Honda's Asimo was back in 2000.

Simon Kalouche

@simonkalouche

5 months

Announcing that I’m getting into the humanoid robot space! Going to surpass all competitors instantly with our new robot Asimo which was designed by Honda over 20 years ago.

31

40

501

6

16

148

Andrew Davison

@AjdDavison

2 years

The enormous power of explicit 3D visual scene understanding is to enable varied, precise manipulation via standard motion planning. Works for many variations of object size/shape/placement with no demos or RL needed! Dyson Robotics Lab: NodeSLAM

2

21

147

Andrew Davison

@AjdDavison

2 years

New from the Dyson Robotics Lab at Imperial at @3DVconf by @tlaidlow : SLAM with Quadric Surfaces. Many scene elements can be represented accurately and efficiently with quadrics. Our new minimal representation enables their use in a standard factor graph.

2

23

146

Andrew Davison

@AjdDavison

5 years

New on arXiv: FutureMapping 2: Gaussian Belief Propagation for Spatial AI, with @joeaortiz GBP is ready for the new generation of super-parallel AI chips and edge networks, for general, graph-based #SpatialAI . Try my Python demos and see what you think!

7

48

144

Andrew Davison

@AjdDavison

9 days

Everyone knows AlexNet (2012), but earlier pioneers of GPUs for vision were the gpu4vision project from TU Graz (Tom Pock and others): incredible real-time variational optical flow, denoising, range image fusion, etc. from 2008 onwards. More vids at:

4

13

142

Andrew Davison

@AjdDavison

3 years

Regular re-tweet of this from Bill Freeman... I spend a lot of my time trying to persuade students of this; they often don't believe me. There is very little to be gained by publishing an average, "pretty good" paper. Better to wait and work on something deep and long-term.

Jia-Bin Huang

@jbhuang0604

3 years

*It doesn't matter much.* Vast majority of the papers won't matter in the long run. Your career will be shaped only by a few good ones. Instead of getting an "okay" paper accepted, it could be a blessing in disguise to revise and strengthen your paper. Fig credit: Bill Freeman

3

30

213

6

10

140

Andrew Davison

@AjdDavison

7 months

Just did this paper in our reading group (thanks @alzugarayign ) and it's impressive. A reminder about how the right representation lets you get back to just optimising and using all of the rich photometric data in multi-camera video with just basic priors and no neural networks!

Jonathon Luiten

@JonathonLuiten

9 months

Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis We model the world as a set of 3D Gaussians that move & rotate over time. This extends Gaussian Splatting to dynamic scenes, with accurate novel-view synthesis and dense 3D trajectories.

25

368

2K

2

12

140

Andrew Davison

@AjdDavison

3 months

Fit-NGP: millimetre-accurate 3D object model fitting from a single RGB robot-mounted camera rapidly scanning a scene. Super simple, auto-optimises camera poses, works even for tiny shiny objects like screws (that depth cameras can't see)! Dyson Robotics Lab at Imperial College.

Marwan Taher

@marwan_ptr

3 months

Excited to announce Fit-NGP which will be presented in #ICRA2024 ! Fit-NGP accurately estimates 6-DoF object poses (~ 1.6mm) leveraging Instant-NGP's density field. With @alzugarayign & @AjdDavison . Project page: Video: (1/3)

4

22

127

1

21

138

Andrew Davison

@AjdDavison

10 months

Large language models and web-scale data have some use in robotics as a user interface as nicely demonstrated here, but in my opinion they are not what we need to help with perception, object representation and precise planning which are the real current barriers in robotics.

Demis Hassabis

@demishassabis

10 months

Computers have long been great at complex tasks like analysing data, but not so great at simple tasks like recognizing & moving objects. With RT-2, we’re bridging that gap by helping robots interpret & interact with the world and be more useful to people.

16

113

711

11

24

138

Andrew Davison

@AjdDavison

6 months

I don't believe that computer vision needs big data. E.g. I bet that pretty soon someone will come up with something which can segment scenes as well as SAM but only needs a few unsupervised images for training (because segmentation is all about self-similarity).

19

7

138

Andrew Davison

@AjdDavison

2 years

This looks great; hash encoding for ultrafast training of neural fields. As @zzznah said, it's a lesson in going back to the basics of what runs well on today's ridiculously powerful parallel processors. I think we should be embarrassed when algorithms need hours on a modern GPU.

AK

@_akhaliq

2 years

Instant Neural Graphics Primitives with a Multiresolution Hash Encoding paper: project page: github:

21

359

2K

2

18

139

Andrew Davison

@AjdDavison

3 years

This presents vSLAM history of the past 20 years or so in the way I like to think of it, with new real-time demo systems as the main markers of progress. Onwards to #SpatialAI !

Antoni Rosinol

@RosinolToni

3 years

vSLAM history: a brief and incomplete timeline

13

128

500

3

31

134

Andrew Davison

@AjdDavison

2 years

"Premature optimisation" is a familiar sin in programming (Knuth), but I think my field of CV/robotics is increasingly suffering from "premature evaluation", with every component benchmarked to death but little thought given to how they could combine into something bigger! 1/2

3

13

135

Andrew Davison

@AjdDavison

1 year

A variation with multiple robots running DWA planning, all trying to reach the same target. I always wondered about making a game out of this but couldn't work out what the player would control to make it fun. Any ideas?

10

11

134

Andrew Davison

@AjdDavison

3 years

Premature optimisation was always a problem in research, but I do think it's got worse with deep learning. Students are often obsessed with the details of trendy networks when basic decisions about what's the input, what's the output, etc. are still up in the air.

Matthias Niessner

@MattNiessner

3 years

Common issue in deep learning projects is that complex method designs are adopted before basic debugging. Often this leads to situations where the data loader still loads black images, but we've already tried 10 loss function.... cuz a paper claimed these would improve results.

12

49

562

0

20

132

Andrew Davison

@AjdDavison

5 years

Officially announced today, the Dyson Robotics Laboratory at Imperial College which I direct has received £5M+ new funding from Dyson and EPSRC under the Prosperity Partnerships scheme. Thanks to all involved in the lab, and to @Dyson for the long-term support and collaboration.

Engineering and Physical Sciences Research Council

@EPSRC

5 years

Prosperity Partnerships to address generational challenges across the globe: #IndustrialStrategy @CSkidmoreUK @EPSRC @BAESystemsplc @GSK_bio @Constellium @Dyson @beisgovuk

0

9

6

9

7

132

Andrew Davison

@AjdDavison

5 years

I liked the DeepMapping paper from Ding and Feng at #CVPR2019 . A bit similar to DIP, they use deep learning machinery to solve a surprising optimisation problem (no learning on a dataset): pose graph alignment for a set of pose scans. @czarnowskij

3

40

129

Andrew Davison

@AjdDavison

2 months

We are releasing interactive code for SuperPrimitives, a simple new way to do dense monocular SfM and visual odometry using strong generic segmentation and normal priors. Dyson Robotics Lab at Imperial College.

Kirill Mazur

@makezur

2 months

Code release for SuperPrimitives, and it comes with an interactive GUI! #CVPR2024 SuperPrimitive is a new 3D representation which enables solving many 3D tasks at the level of image segments.

1

41

177

1

23

129

Andrew Davison

@AjdDavison

1 year

In research you live for this kind of moment!

Jon Barron

@jon_barron

1 year

Three years ago today, the project that eventually became NeRF started working (positional encoding was the missing piece that got us from "hmm" to "wow"). Here's a snippet of that email thread between Matt Tancik, @_pratul_ , @BenMildenhall , and me. Happy birthday NeRF!

29

156

1K

2

1

127

Andrew Davison

@AjdDavison

4 years

Bundle Adjustment on a Graph Processor, by @joeaortiz , Mark Pupilli, @StefanLeuteneg1 and me, CVPR 2020. Using Gaussian Belief Propagation we show breakthrough 20x speed for BA on a single @graphcoreai IPU compared to CPU/Ceres.

Bundle Adjustment with a Graph Processor

Joseph Ortiz, Mark Pupilli, Stefan Leutenegger and Andrew J. DavisonImperial College Robot Vision Group and GraphcoreCVPR 2020https://arxiv.org/abs/2003.03134

www.youtube.com

3

35

125

Andrew Davison

@AjdDavison

2 years

If you liked Semantic-NeRF, this is the next step where it really gets interesting. Train a geometric/semantic neural field in real-time and add ultra sparse open set labels as clicks to densely segment a room in a few minutes. No pre-trained networks or prior data needed at all.

Andrew Davison

@AjdDavison

2 years

New: 3D neural fields like NeRF do automatic, unsupervised semantic scene decomposition. We reveal it with tiny interactions. Real-time SOTA room segmentation from 140 clicks; zero prior data! iLabel, Dyson Robotics Lab, @SucarEdgar @Shuaifeng_Zhi et al.

4

149

671

3

18

128

Andrew Davison

@AjdDavison

5 months

Live Monocular SLAM using 3DGS as the only scene representation. From a SLAM perspective, Gaussian blobs are an explicit, efficient scene representation similar to points or surfels, but with much better properties for optimisation. Dyson Robotics Lab at Imperial College.

Hide

@HideMatsu82

5 months

Happy to share Gaussian Splatting SLAM We show the first 3DGS-based Monocular RGB SLAM, the hardest SLAM setting. Using 3D Gaussians as a unified representation, the method only requires RGB images - No need for SfM, depth sensor, or learned prior.

18

140

653

2

16

128

Andrew Davison

@AjdDavison

2 years

Also from the Oxford Active Vision Lab, same era (2003), Walterio Mayol-Cuevas and David Murray's amazing wearable robot running MonoSLAM. Real-time active camera control enables stable object fixation and saccades as the user moves. Video and paper links:

2

10

126

Andrew Davison

@AjdDavison

5 years

I like the ideas in this paper... says that the early feature layers of a CNN can not only be learned in an unsupervised way, but very effectively just from a single image. Seems to confirm the strong generality of low level natural image statistics.

Christian Rupprecht

@chrirupp

5 years

Our new work by @y_m_asano and Andrea Vedaldi is on arXiv now. We investigate the surprising effectiveness of unsupervised learning using only one single image. We can learn early layers using one image + heavy augmentations just as well as with ImageNet.

2

64

225

2

26

125

Andrew Davison

@AjdDavison

3 years

Aha, camera tracking using implicit scene models. So what about building a full SLAM system? Watch this space... ;)

Kosta Derpanis

@CSProfKGD

3 years

Lin Yen-Chen, Pete Florence, Jonathan T. Barron, Alberto Rodriguez, Phillip Isola, Tsung-Yi Lin, iNeRF: Inverting Neural Radiance Fields for Pose Estimation, arXiv, 2020 Paper: Project page:

1

28

186

3

10

121

Andrew Davison

@AjdDavison

2 months

Monocular normal prediction used for highly robust real-time camera orientation estimation (and one of the best acronyms we've ever come up with...) New from the Dyson Robotics Lab at Imperial.

Gwangbin Bae

@BaeGwangbin

2 months

𝗜𝗠𝗨? How about 𝗨-𝗔𝗥𝗘-𝗠𝗘? In this work, we show how monocular surface normal cues can be used for rotation estimation. collab w/ @AalokPat , Callum Rhodes, @AjdDavison

4

52

260

1

8

122

Andrew Davison

@AjdDavison

1 year

Aalok will present Gaussian Belief Propagation Planning at the Multi-Agent Path-Finding workshop at #AAAI23 next week (also at #ICRA2023 in May). We believe this is the first truly distributed method for collaborative planning with general cost functions and dynamics constraints.

Aalok Patwardhan

@AalokPat

1 year

Q: How can many robots plan to *safely and smoothly* move around each other? A: They collaborate and negotiate paths! Find out at my talk at #AAAI23 on Tuesday 14th Feb! Link to paper/poster/video: @AjdDavison @rmurai0610

6

50

283

0

15

119

Andrew Davison

@AjdDavison

3 years

Inspired by the new wave of interactive publishing pioneered by @distillpub , and thanks to @joeaortiz 's massive effort to become a Javascript ninja, we are very proud to share this article which explains how Gaussian Belief Propagation works, and why we think it's so important.

Joseph Ortiz

@joeaortiz

3 years

Very excited to share our interactive article: A visual introduction to Gaussian Belief Propagation! It's part proposition paper, part tutorial with interactive figures throughout to give intuition. Article: Work with: @talfanevans , @AjdDavison 1/n

7

129

576

3

19

118

Andrew Davison

@AjdDavison

11 months

I'm getting quite bored of deep learning now.

9

3

115

Andrew Davison

@AjdDavison

1 year

I doubt that robotics needs big data right now. As we continue to improve scene reconstruction+representation, motion planning plus local learning becomes very powerful. E.g. no pre-trained networks in this demo; just a unified neural field scene representation, trained live.

Andrew Davison

@AjdDavison

1 year

If you're at CORL this week in NZ come and meet @iainhaughton and @Ed__Johns and see Iain's presentation in the oral session on Saturday. Dense, fully automatic segmentation of scene properties like softness via real-time neural field training; no priors! It's iLabel for robots.

0

3

36

1

12

117

Andrew Davison

@AjdDavison

1 year

One last variation --- a team of slow robots goes up against one fast "super-robot" in a race to get to the targets first.

5

8

116

Andrew Davison

@AjdDavison

4 years

Many tasks in robotics/AI involve *scene rearrangement*. How do we define a goal state or measure success? New environments with realistic simulation of perception and physics enable systematic research. We discuss in this major new collaborative report!

1

22

113

Andrew Davison

@AjdDavison

2 years

#CORL2022 oral: mapping non-visual properties (material, softness, force) from very few point sensor tests. iLabel-like neural field produces dense maps + guides actions. Live, autonomous, no priors! @iainhaughton et al, @Dyson @ICComputing . Paper/video:

2

14

114

Andrew Davison

@AjdDavison

2 years

Very nice results in monocular depth prediction, which allows good 3D reconstruction from simple volumetric fusion.

Mohamed Sayed

@MohammedAmr1

2 years

📢 Our #ECCV2022 paper (and code) on fast accurate depth estimation and reconstruction is out now! SimpleRecon: 3D Reconstruction without 3D Convolutions (1/4)

23

281

2K

1

9

115

Andrew Davison

@AjdDavison

3 months

Wow.. I guess we don't know how Sora works yet, but assuming there is no explicit 3D consistency check built into the generation pipeline then I am definitely surprised this is possible.

Ben Mildenhall

@BenMildenhall

3 months

will it nerf? yep ✅ congrats to @_tim_brooks @billpeeb and colleagues, absolutely incredible results!!

16

93

728

6

12

114

Andrew Davison

@AjdDavison

2 months

Normal prediction from a single image is something that neural networks are incredibly good at, and is extremely widely useful. See the new level of performance in this new work with @BaeGwangbin , Dyson Robotics Lab at Imperial College London.

Gwangbin Bae

@BaeGwangbin

2 months

Excited to introduce 𝗗𝗦𝗜𝗡𝗘! ( #CVPR2024 ) We push the limits of single-image surface normal estimation by rethinking the inductive biases needed for the task. See you in Seattle!

9

120

656

0

11

110

Andrew Davison

@AjdDavison

5 years

Very nice papers at the 3D session at #CVPR2019 , including DeepSDF, BAD-SLAM (real-time dense BA on surfel maps) and this remarkable one on showing that you can recover realistic images from 3D points clouds (best results with SIFT descriptors + colour):

Revealing Scenes by Inverting Structure from Motion Reconstructions

Publication Title: Revealing Scenes by Inverting Structure from Motion ReconstructionsAuthors: Francesco Pittaluga, Sanjeev J. Koppal, Sing Bing Kang, Sudipt...

www.youtube.com

1

27

110

Andrew Davison

@AjdDavison

4 years

DeepFactors with @czarnowskij , Tristan Laidlow, @ronnieclark__ from the Dyson Robotics Lab. Unified real-time monocular SLAM with a general factor graph formulation (GTSAM), pushing what's possible combining deep networks with probabilistic optimisation.

DeepFactors: Real-Time Probabilistic Dense Monocular SLAM

https://arxiv.org/abs/2001.05049DeepFactors: Real-Time Probabilistic Dense Monocular SLAMJan Czarnowski; Tristan Laidlow; Ronald Clark; Andrew J. Davison Abs...

www.youtube.com

4

33

108

Andrew Davison

@AjdDavison

1 year

Getting excited for ICRA, the main international robotics conference, held in London for the first time next week. I'll be there all week; see you there if you're interested in *real* AI that's actually making contact with the world ;)

Welcome

www.icra2023.org

3

4

102

Andrew Davison

@AjdDavison

2 years

As an academic, honestly, this doesn't worry me in the slightest. I feel like we're just getting started in AI and there are so many interesting problems out there to work on. I'm sure that all you still need to do important long-term AI research is pen, paper and a laptop.

Patrick Mineault

@patrickmineault

2 years

Do you feel anxious that AI's emphasis on large-scale language models (LLMs) will crowd out academic labs? Few can afford 1000 GPUs drawing a small country's worth of electricity

56

33

409

2

8

104

Andrew Davison

@AjdDavison

2 years

Live iLabel object segmentation, as demoed at CORL recently. Hand-held camera, zero training data or hand-designed rules, network trained in real-time. Highly accurate object boundaries emerge from sparse clicks, despite object similarity or contact.

0

15

106

Andrew Davison

@AjdDavison

4 years

Nice to see my friends from Zaragoza back with ORB-SLAM3 --- presumably continuing their line of the best-engineered academic visual SLAM systems you can get!

Giseop Kim

@GiseopK

4 years

ORB-SLAM3 ?! !! ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM videos

2

83

247

0

13

104

Andrew Davison

@AjdDavison

4 years

What's going on in a cat's #SpatialAI brain to enable this? What representation of the scene is it storing?

Steve Stewart-Williams

@SteveStuWill

4 years

Cats negotiating obstacle courses. It's impressive that they can place their front paws so precisely; what really amazes me, though, is that they can do it with their back paws, when the obstacles are no longer in sight.

191

3K

12K

2

19

101

Andrew Davison

@AjdDavison

2 years

If you've seen the "brain with modules" picture from @ylecun 's cool new paper, a reminder of my version. I agree the key challenge in AI is updating a persistent world model, and emphasize the route to efficiency by matching algorithm/representation graphs to computing hardware.

Andrew Davison

@AjdDavison

6 years

What do future #SpatialAI systems have to do, and how will they work, as we bring together probabilistic and geometric computer vision with deep learning and the ongoing developments in sensing and processing hardware. Read about FutureMapping at .

4

41

147

4

19

97

Andrew Davison

@AjdDavison

2 years

AI is clearly lacking this kind of intuitive spatial/physics reasoning, but I don't see enough research on what for me is the biggest challenge: building general, *efficient* composable 3D world models from real-time vision + sensors. My views: #SpatialAI

FutureMapping: The Computational Structure of Spatial AI Systems

We discuss and predict the evolution of Simultaneous Localisation and Mapping (SLAM) into a general geometric and semantic `Spatial AI' perception capability for intelligent embodied devices. A...

arxiv.org

5

10

100

Andrew Davison

@AjdDavison

3 years

Great to see there's more vision going on in the parkour work than I've seen before in Atlas demos; depth cams for real-time model-based tracking of boxes and structures, allowing some on-line modification of planned motion trajectories. @czarnowskij this must be fun to work on!

Boston Dynamics

@BostonDynamics

3 years

Robot perception algorithms convert data from cameras and sensors into something useful for decision making and planning physical actions. See how perception and adaptability enable varied, high-energy behaviors like parkour.

36

1K

5K

1

9

101

Andrew Davison

@AjdDavison

1 year

Scalable and resilient computation in robotics should be distributed, whether over many-robot graphs or within single chips. We present the new Workshop on Distributed Graph Algorithms for Robotics at #ICRA2023 in London ; please submit paper and demos!

3

25

97

Andrew Davison

@AjdDavison

3 years

ISMAR has been very important to me over the years, inspiring in particular my love of real-time demos as the highest form of academic presentation! Thanks for this recognition and congrats to @rapideRobot and the other authors, now leading research on AR all over the world.

ISMAR Organization

@ismarconf

3 years

Congratulations to the recipients of the ISMAR 2021 Impact Paper Award! It's been 10 years since this paper was published and has been cited countless of times since. Wow!

0

10

70

0

1

98

Andrew Davison

@AjdDavison

3 years

Go SLAM!

Fei-Fei Li

@drfeifei

3 years

Thirty years of Computer Vision research at work here. While the public does not hear much about basic science research, it’s the generations of scientists and their passionate work that bring moments like this to live 🌹👍🙏

14

353

2K

0

4

96

Andrew Davison

@AjdDavison

1 year

Note that five of those seven papers come from UK universities --- the home of visual SLAM research 😀--- and all of them from European universities.

Andrew Davison

@AjdDavison

1 year

Just found this --- from Georg Klein's talk at #ICVSS 2018.

3

7

92

2

10

95

Andrew Davison

@AjdDavison

2 years

iSDF uses the main incremental neural field training methods of iMAP, but interprets the MLP output as a signed distance field rather than occupancy. Similar reconstruction quality, with auto hole-filling. Directly building an SDF could be useful for some robot planning cases.

Joseph Ortiz

@joeaortiz

2 years

Excited to share iSDF! Real-time mapping with neural (implicit) signed distance fields for robot navigation and manipulation. Project page: Paper: Work with: Alex Clegg, Jing Dong, @SucarEdgar @davnov134 @MZollhoefer @mhmukadam

4

40

174

1

13

94

Andrew Davison

@AjdDavison

4 years

The Joint CVPR workshop on Localisation, VO, SLAM is on Sunday and Monday; full programme here: Looks like anyone can stream the talks live at this YouTube link. I'm speaking at 4pm UK time on Sunday.

CVPR2020_Workshop_Localization_VO_SLAM

Live stream for June 15th: https://youtu.be/aOkLGcspoyYWorkshop website: https://sites.google.com/view/vislocslamcvpr2020/homeJune 14th Schedule: 7:00 AM PT:...

www.youtube.com

3

27

93

Andrew Davison

@AjdDavison

2 years

Great results; for me this is what we should mean by the term "optical flow", not just 2-view correspondence. I'm interested in how to do this incrementally (rather than batch) with efficient distributed compute --- crucial general early vision for #SpatialAI . h/t @ronnieclark__

Adam W. Harley

@AdamWHarley

2 years

Very happy to share our #ECCV2022 oral “Particle Video Revisited: Tracking Through Occlusions Using Point Trajectories” Fine-grained tracking of anything, outperforming optical flow. project: abs: code:

16

83

522

1

9

94

Andrew Davison

@AjdDavison

8 months

The Raspberry Pi project is the UK at its best, opening up creative interest in computers and hardware for over 10 years now. And most of them are actually made here, in the Sony factory in Wales. I've bought hundreds and we use them every year for teaching robotics.

Andrej Karpathy

@karpathy

8 months

How Raspberry Pis are made (Factory Tour) Love watching videos like this. Stumbled by while researching the new Pi 5. Pis help build Pis! One Pi gets built every ~3.14 seconds :D I want to play Factorio now.

35

176

2K

1

4

95

Andrew Davison

@AjdDavison

1 year

It's tricky to use deep learning in multi-view SLAM. New idea: learn a depth covariance function, predicting pixel depth correlations from a single image. Useful in many optimisation settings; e.g. real-time dense monocular VO; note the precise small details. See it live at CVPR!

Eric Dexheimer

@eric_dexheimer

1 year

Excited to announce "Learning a Depth Covariance Function" with @AjdDavison . A flexible framework for a variety of geometric vision tasks, such as dense monocular visual odometry shown below. Dyson Robotics Lab, Imperial College Project page: #CVPR2023

6

33

160

0

8

93

Andrew Davison

@AjdDavison

3 years

We have a Dyson Fellow (post-doc) position in computer vision and robotics available in the Dyson Robotics Lab at Imperial College London. Come and work on cutting edge SLAM, scene understanding and manipulation with me and the rest of our team. Details:

0

26

91

Andrew Davison

@AjdDavison

2 years

If you want to learn more about Gaussian Belief Propagation and its properties for distributed computation, estimation and learning on general graphs, you can play with the demos in our interactive @distillpub -style article here.

Joseph Ortiz

@joeaortiz

3 years

Very excited to share our interactive article: A visual introduction to Gaussian Belief Propagation! It's part proposition paper, part tutorial with interactive figures throughout to give intuition. Article: Work with: @talfanevans , @AjdDavison 1/n

7

129

576

2

16

91

Andrew Davison

@AjdDavison

1 year

Just found this --- from Georg Klein's talk at #ICVSS 2018.

3

7

92

Andrew Davison

@AjdDavison

11 months

Congratulations to Edgar who passed his PhD viva today, and thanks to examiners @tolga_birdal and José María Montiel! A reminder of Edgar's iMAP, a landmark as the first real-time neural field SLAM system from #ICCV2021 .

Edgar Sucar

@SucarEdgar

3 years

Excited to share iMAP, first real-time SLAM system to use an implicit scene network as map representation. Work with: @liu_shikun , @joeaortiz , @AjdDavison Project page: Paper:

10

119

518

3

88

Andrew Davison

@AjdDavison

2 years

Real Time Height Map Fusion using Differentiable Rendering, with @jz4411 @StefanLeuteneg1 , Dyson Robotics Lab, single RGB camera. Here used for dense, geometric drivable ground segmentation at <1cm height. (no learning needed).

1

11

90

Andrew Davison

@AjdDavison

2 years

Nice blog; I really agree with the main message. I hope people can link this to with why I'm obsessed with distributed optimisation; especially Gaussian Belief Propagation with its `magic' properties of convergence despite ad-hoc, noisy, message passing.

Sebastian Risi

@risi1979

2 years

Finally done with my first blog post "The Future of Artificial Intelligence is Self-Organizing and Self-Assembling"! Covering work from our group and others on the combination of ideas from deep learning and self-organizing systems.

25

218

973

1

11

87

Andrew Davison

@AjdDavison

1 year

You can fuse arbitrary features (e.g. DINO) into 3D via real-time neural field SLAM, with all geometry and coherent feature maps held in a single neural field. This allows highly efficient open set object classification and scene segmentation. #ICRA2023

Andrew Davison

@AjdDavison

2 years

New: Feature-realistic neural fusion for real-time, open set scene understanding. Our neural field renders to feature space, enabling real-time grouping and segmentation of similar objects or parts from ultra-sparse, online interaction. Dyson Robotics Lab.

2

39

293

2

7

88

Andrew Davison

@AjdDavison

3 years

I strongly agree that: - 3D object graphs are the right (efficient, semantically optimal) representation for intelligence. - Message passing is the computation pattern to focus on. The biggest challenge is how to actually *build* scene graphs from real sensor data. #SpatialAI

arXiv Daily

@Arxiv_Daily

3 years

Semantic and Geometric Modeling with Neural Message Passing in 3D Scene Graphs for Hierarchical Mechanical Search by Andrey Kurenkov et al. including @ken_goldberg #NeuralNetwork #Vector

0

9

31

1

14

89