Sayak Paul @RisingSayak Twitter profile

Pinned Tweet

Sayak Paul

9 months

Inspired by @_jasonwei , today, I'm sharing an "always-WIP-mode" Google Doc answering some FAQs at length. Link to the doc: Below is a snap of the Qs I answer:

11

19

192

Last Seen Profiles

@SpoilersGirls

@EvilRacism

@JediKnightTy7

@Prideof904

@dead_assistant

@TamOlson

@SBMPediatrics

@MysonCooper

@karmarecording5

@D3Kozmopolitan

@KristenGelineau

@PetrYanUFC

@shu_m41

@heriveltofranco

@sorairo__00

@SweetHomeGirI

@Liella429

@SpoorthiJain2

@BatorGymJock

@iosonoumano

@MasoodCGD

@junior8400

@TransLilDicky

@itsimanijade

@LpLucas

@MaySilverBooty

@MaressaG2021

@BabyB0044

@DrSudhakarIRS

@rmIights

@yuwanxue

@uzbilgili

@AlSailiyaclub

@BrianKavanaghNY

@DerMuzikant

@SebAchFB

Sayak Paul

@RisingSayak

2 years

. @DeepMind released this GOLD a couple days back. If you ever wanted to study Transformers from scratch I think this would be that one resource you wouldn't want to miss:

Formal Algorithms for Transformers

This document aims to be a self-contained, mathematically precise overview of transformer architectures and algorithms (*not* results). It covers what transformers are, how they are trained, what...

arxiv.org

3

280

2K

Sayak Paul

@RisingSayak

2 years

What do the Vision Transformers learn? How do they encode anything useful for image recognition? In our latest work, we reimplement a number of works done in this area & investigate various ViT model families (DeiT, DINO, original, etc.). Done w/ @ariG23498 1/

10

176

1K

Sayak Paul

@RisingSayak

4 years

Blow your mind: .

GitHub - microsoft/computervision-recipes: Best Practices, code samples, and documentation for...

Best Practices, code samples, and documentation for Computer Vision. - microsoft/computervision-recipes

github.com

5

168

722

Sayak Paul

@RisingSayak

5 years

It's possible to use @MSFTResearch 's `interpret` to *interpret* `keras` model. `interpret` + @TensorFlow 2.0 = too much awesomeness. Check this notebook I made for ya: #DeepLearning #TensorFlow

6

177

638

Sayak Paul

@RisingSayak

4 years

For the ones that do not know, I maintain a list of resources for the people willing to learn @TensorFlow 2.0. Currently, the list looks like so & can be accessed here: . Please feel free to pass along any suggestions regarding new resources :)

4

156

613

Sayak Paul

@RisingSayak

2 years

New project 📢 We show how to deploy a deep learning model with Docker + Kubernetes + GitHub Actions. We show this with two promising candidates - FastAPI (for REST) and TF Serving (for gRPC). 1/

6

88

578

Sayak Paul

@RisingSayak

4 years

@TensorFlow model -> 38 MB, val_accuracy: 97.5% #TF Lite model -> *3.4 MB*, val_accuracy: ~96% I'll just leave it there. Notebook: @GoogleDevsIN @GoogleDevExpert

Custom_Image_Classification_EdgeTPU.ipynb

Colaboratory notebook

colab.research.google.com

7

97

540

Sayak Paul

@RisingSayak

1 year

Reading clean Python code to improve Python skills is underrated.

16

42

528

Sayak Paul

@RisingSayak

5 years

I am preparing this list as a central repository which enlists resources to learn about @TensorFlow 2.0. If you would like to add your recommendations reach out to me directly via . @GoogleDevExpert @GoogleDevsIN @GoogleAI @googledevs

8

140

515

Sayak Paul

@RisingSayak

2 years

Shubho Deepaboli! Today, I am delighted to announce that I am joining the mighty forces at @huggingface as a Developer Advocate Engineer! Working on ensuring developers benefit from our ML tooling is heavy-weight and full of opportunities.

71

25

503

Sayak Paul

@RisingSayak

2 years

I'm not sure I've read a more exciting preface for a DL book in 2022: "Dive into Deep Learning Compiler"

5

56

408

Sayak Paul

@RisingSayak

2 years

Delighted to release the @TensorFlow port of MAXIM, a single backbone capable of denoising, dehazing, deblurring, and more! Comes packed w/ @huggingface Spaces, Colab, and XLA support! Know all of it here: Cc: @_vztu , @han_zhang_ 1/

4

73

394

Sayak Paul

@RisingSayak

3 years

I'm currently in absolute awe after reading @theaisummer 's take on Einsums by implementing a vanilla Transformer block. The dude's got magical abilities!

Understanding einsum for Deep learning: implement a transformer with multi-head self-attention from...

Learn about the einsum notation and einops by coding a custom multi-head self-attention unit and a transformer block

theaisummer.com

7

67

397

Sayak Paul

@RisingSayak

10 months

With SDXL's tremendous quality improvements, we started training a couple of ControlNets internally 🔥 Today, we're excited to share some EARLY results for the community to give it a spin 🌀 Find the model trained on Canny conditions from LAION 6a ⬇️

19

76

389

Sayak Paul

@RisingSayak

1 year

At @huggingface , we care about enabling developers to realize things. That's why we're dedicated to sharing things. To that end, we launched the V2 of our Deep Reinforcement Learning class yesterday -- brand new and fresh off the press ☕️ 🐶

Welcome to the 🤗 Deep Reinforcement Learning Course - Hugging Face Deep RL Course

huggingface.co

4

80

365

Sayak Paul

@RisingSayak

1 year

Incredibly excited to announce our @CVPR tutorial w/ the amazing @hila_chefer ! "All Things ViTs: Understanding and Interpreting Attention in Vision" Come for cool visualizations, exclusive insights, & interesting approaches ❤️ Catch the details here ⬇️

4

76

365

Sayak Paul

@RisingSayak

4 years

This Keras guide on fine-tuning EfficientNets is the best one I have seen to date. Includes many practical tips.

Keras documentation: Image classification via fine-tuning with EfficientNet

keras.io

0

73

346

Sayak Paul

@RisingSayak

9 months

Introducing Doodly 🖌 A fast and efficient demo that lets you scribble and generate art from it using language guidance 🤯 Behind the scenes: SDXL and T2I Adapters 💡 Try it out here: Examples in 🧵

7

82

331

Sayak Paul

@RisingSayak

1 year

🤗 Transformers has expanded things for time-series modelling 📈 In this banger of a post, @NielsRogge & Kashif Rasul, not only show you how to train a time-series model from Transformers, but also discuss the preprocessing steps, what's coming, & more

Probabilistic Time Series Forecasting with 🤗 Transformers

huggingface.co

1

71

320

Sayak Paul

@RisingSayak

1 year

Today we're happy to announce the support of the **fastest** ControlNet from Diffusers 🧨 ⚡️ The integration was led by our amazing community member Takuma Mori 🤝 Check out our blog post here to catch all the details: Cc: @lvminzhang @magrawala

ControlNet in 🧨 Diffusers

huggingface.co

8

71

307

Sayak Paul

@RisingSayak

1 year

🧨 diffusers 0.17.0 is out and comes with new pipelines, improved LoRA support, `torch.compile()` speedups, and more ⏰ 🪄 UniDiffuser 🦄 DiffEdit ⚡️ IF DreamBooth 💡 Support for A1111 LoRA and more ... Release notes 📝 1/🧶

6

60

304

Sayak Paul

@RisingSayak

4 years

An amazing tutorial on fine-tuning a pre-trained BERT model from @huggingface `transformers` on SQuAD by @NandanApoorv : . I absolutely loved the focus given on the code readability.

Keras documentation: Text Extraction with BERT

keras.io

2

64

297

Sayak Paul

@RisingSayak

2 years

This has been in the works for MONTHs now! Finally, it's in a good shape and is ready to be shipped 🚀 @algo_diver & I bring you an automated pipeline for semantic segmentation. It comes packed with lots of goodies, so please read on. @tensorflow @huggingface @googlecloud

4

47

284

Sayak Paul

@RisingSayak

1 year

The entire idea of "zero-shot image segmentation" blows me away 🤯 Tobias ( @SegmentsAI ) and @NielsRogge wrote this blog post introducing the idea. They also show how to perform zero-shot segmentation using a model called CLIPSeg using 🤗 Transformers.

7

48

286

Sayak Paul

@RisingSayak

1 year

Fine-tune Stable Diffusion in T4/V100 on a custom image-caption pairs' dataset 🧨 🔥 => memory efficiency This is enabled by LoRA. With LoRA, the fine-tuned checkpoints are just **3 MBs** in size 🤯 => portability Know about it👇

2

43

286

Sayak Paul

@RisingSayak

1 month

We present LLaMADuo 🦙 It's a collection of Python scripts and configurations that implement how to improve a small language (typically open) with a bigger one (typically gated). Hop in for a 🧶 1/7

8

49

285

Sayak Paul

@RisingSayak

6 months

The final half of @karpathy 's talk on "Intro to Large Language Models" is actually a gem of an overview of LLM security! It's so good!

[1hr Talk] Intro to Large Language Models

This is a 1 hour general-audience introduction to Large Language Models: the core technical component behind systems like ChatGPT, Claude, and Bard. What the...

www.youtube.com

5

25

279

Sayak Paul

@RisingSayak

9 months

So, I swapped out the OpenAI API from @jeremyphoward 's notebook on LLM Science Exam with Llama 2 7B chat and prepared this @GoogleColab ⬇️ 🚨 I am no LLM expert, so the notebook is bad and not the greatest. Feedback and improvements are welcome 🤗

llm_science_llama_2.ipynb

Colaboratory notebook

colab.research.google.com

7

42

272

Sayak Paul

@RisingSayak

1 year

The model implementation LLaMA is pretty neat and compact. Just 238 lines of code with formatting. SwiGLU + Rotary Embeds + Pre LayerNorm (with RMSNorm) + Parallel blocks for attention and feed-forward

4

43

268

Sayak Paul

@RisingSayak

1 year

Hugging Face ❤️ Computer Vision 👁🤗 We've been taking some giant leaps in our ecosystem dedicated to Computer Vision but it remains unclear what all we have to offer. Our latest blog post provides you with a concrete overview of "State of CV at 🤗" ⤵️

5

48

267

Sayak Paul

@RisingSayak

2 years

Thanks to the @TensorFlow team for this! Collaborators, THANK YOU!

15

8

258

Sayak Paul

@RisingSayak

1 year

Very cool overview of a system that can turn plausible natural language inputs into SQL queries. Seq-to-seq formulations are just FTW!

Natural Language to SQL from Scratch with Tensorflow

An end-to-end tutorial of training a machine learning model to translate from natural language questions/instructions to SQL queries using…

towardsdatascience.com

0

52

256

Sayak Paul

@RisingSayak

10 months

🧨 diffusers 0.19.0 is out and comes with the latest SDXL 1.0 🔥 1️⃣ New training scripts for SDXL 2️⃣ New pipelines for SDXL (ControlNet, InstructPix2Pix, Inpainting, etc.) 3️⃣ AutoPipeline and MORE! Release notes 📝 1/5

4

44

256

Sayak Paul

@RisingSayak

2 months

Bit of an unsolicited advice. I see you're jumpstarting to LLM APIs to do amazing things without understanding their internals even on a basic level. I am also seeing you're complaining why they're not working in the way you expect them to be. So, I urge you to also spend

11

25

248

Sayak Paul

@RisingSayak

5 years

Absolutely in immense love with the new TensorBoard capabilities with @TensorFlow 2.0. Confusion matrix as the batch size progresses. What more to ask? 😭

3

41

244

Sayak Paul

@RisingSayak

6 months

The @OpenAI consistency decoder is now available in 🧨 diffusers and can be used as a drop-in replacement for all SD family pipelines. The decoder excels at text & fine details. Thanks to @williamLberman for sprinting on this very quickly ⚡️

9

37

242

Sayak Paul

@RisingSayak

4 years

*MIT 6.S191: Introduction to Deep Learning* is easily one of the best introductory yet quite comprehensive courses to get started with Deep Learning. Course website: . The course provides lectures, slides, and code (in @TensorFlow ). @MITDeepLearning

3

52

234

Sayak Paul

@RisingSayak

10 months

Getting to almost instant generation on a 24GB GPU without too many bells and whistles 🚀 🔥🏎 Distillation + Tiny Autoencoder FTW 💡 Interested to know more? Join the conversion 👇 1/🧵

3

41

229

Sayak Paul

@RisingSayak

1 month

Thanks to Hugging Face for making me very GPU-poor. Happy weekend from India 🇮🇳

17

1

228

Sayak Paul

@RisingSayak

2 years

This lecture on continual learning from @josh_tobin_ is a BANGER! As an MLE, I always thought distribution shift HAS to be the sole factor for retraining, monitoring, alerting, etc. But I was proven wrong graciously.

Lecture 06: Continual Learning (FSDL 2022)

New course announcement ✨We're teaching an in-person LLM bootcamp in the SF Bay Area on November 14, 2023. Come join us if you want to see the most up-to-dat...

www.youtube.com

2

30

229

Sayak Paul

@RisingSayak

4 years

I wrote a comprehensive @GoogleColab notebook to train a custom pets' detector model using the latest release @TensorFlow OD API: . Includes the following 👇 P.S.: The data prep script is a stock one.

TensorFlow

@TensorFlow

4 years

TensorFlow 2 🤝 Object Detection API Today we’re happy to announce that the TF Object Detection API officially supports TensorFlow 2! Learn more about the release and all its new features in the post. Read the blog ↓

19

367

1K

3

38

213

Sayak Paul

@RisingSayak

2 years

Getting a hard copy of this paper signed by @JeffDean is probably the best thing I could have asked for. THANK you, sir for inspiring me to take up Comp Sc in the first place. THANKS to @ksoonson for getting this to me!

9

7

225

Sayak Paul

@RisingSayak

1 year

HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace

HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face

Solving complicated AI tasks with different domains and modalities is a key step toward artificial general intelligence. While there are numerous AI models available for various domains and...

arxiv.org

0

43

216

Sayak Paul

@RisingSayak

2 years

Honored to have received the Kaggle ML Research Spotlight Prize for my work on implementing class attention image transformers. Thanks to everyone that helped me get a silver medal on the notebook. I hope I did justice to the architecture. @kaggle

11

6

219

Sayak Paul

@RisingSayak

4 years

*Knowledge Distillation* In my latest blog post, I discuss some of its nitty-gritty, typically used loss functions, & some practical training recipes w/ @TensorFlow code. 💻Code - ✍️Blog post - Thanks to @A_K_Nain for helping.

1

49

209

Sayak Paul

@RisingSayak

2 years

I'm delighted to share that I've been accepted to the Oxford ML Summer School for 2022. Looking forward to great learnings, amazing interactions, and networking. Thankful to @carted for sponsoring my registration.

9

4

211

Sayak Paul

@RisingSayak

3 months

Slides from @sourab_m and my session "Building a personal coding assistant" at the Build with AI roadshow:

8

45

212

Sayak Paul

@RisingSayak

4 years

The following slide (especially the last point) from @jeremyphoward 's USF Deep Learning Info 2020 has always been very special to me. It briefly picks up the known conjectures around deep learning and answers them even stronger. Really looking forward to #fastai Course 1 v4.

1

45

210

Sayak Paul

@RisingSayak

2 years

There's a new kid on the open-source block named KerasCV. It aims at resolving some of the most burning pain points we, the #keras users, have faced over the years for our computer vision workflows. Cc: @luke_wood_ml

GitHub - keras-team/keras-cv: Industry-strength Computer Vision workflows with Keras

Industry-strength Computer Vision workflows with Keras - keras-team/keras-cv

github.com

2

49

208

Sayak Paul

@RisingSayak

2 years

My first guide at @huggingface ! A guide to preprocess semantic segmentation pipelines with 🤗Datasets. We support nifty augmentation libraries like `albumentations`, `kornia` off-the-shelf. Check it out here

Semantic segmentation

huggingface.co

4

31

207

Sayak Paul

@RisingSayak

8 months

When anything like this comes from Jeremy, you cannot miss it. I remember reading through his short writing on self-supervised learning and that is what got me started exploring it!

Jeremy Howard

@jeremyphoward

8 months

I just uploaded a 90 minute tutorial, which is designed to be the one place I point coders at when they ask "hey, tell me everything I need to know about LLMs!" It starts at the basics: the 3-step pre-training / fine-tuning / classifier ULMFiT approach used in all modern LLMs.

88

1K

7K

3

19

211

Sayak Paul

@RisingSayak

2 years

Logging all the experimentation configs is important for transparency. If you use @weights_biases for logging your experimentation, you could use `ml_collections` to define your configuration and have them easily logged.

4

23

207

Sayak Paul

@RisingSayak

4 years

Here's a notebook showing how to perform semi-supervised classification w/ GANs: . Done w/ @TensorFlow 2 & most importantly `tf.keras`. I could not figure out how to override `train_step` for the matter, so decided to keep everything inside a custom loop.

3

35

203

Sayak Paul

@RisingSayak

4 years

Excited to share an interview with @colinraffel today. We talk about Colin's early research on marrying machine learning with music, his days at the @GoogleAI Residency, and of course, *T5* and more. 👉 Interview link: .

0

37

207

Sayak Paul

@RisingSayak

4 years

Here's a list of my favorite recent papers on transfer learning for vision: - BigTransfer: - VirTex: - SimCLRv2: - Self-training: Would love to see a T5-like paper for vision.

Rethinking Pre-training and Self-training

Pre-training is a dominant paradigm in computer vision. For example, supervised ImageNet pre-training is commonly used to initialize the backbones of object detection and segmentation models. He...

arxiv.org

2

39

204

Sayak Paul

@RisingSayak

1 year

Know the technical framework that empowers systems like ChatGPT, InstructGPT: Reinforcement Learning from Human Feedback. Don't let the words overwhelm you; the blog post's very illustrative & does an amazing job of teaching all the components involved:

Illustrating Reinforcement Learning from Human Feedback (RLHF)

huggingface.co

0

58

200

Sayak Paul

@RisingSayak

6 months

I'm biased, but 🤗 PEFT was probably the most impactful OSS release in the DL community this year. I say so, looking at all the use cases it has enabled both for indie developments and business organizations. S-LoRA will make it even better!

7

19

203

Sayak Paul

@RisingSayak

2 years

I'm sure the attached code snippet (courtesy: @fchollet ) is quite known in the #keras community but doesn't hurt to talk about it from time to time. 1> It's possible to code a simple DCGAN with utmost readability. 2> You can learn the basic concepts behind DCGAN. ...

4

20

200

Sayak Paul

@RisingSayak

1 year

We now have full-fledged fine-tuning tutorial notebooks for semantic segmentation in both @TensorFlow and @PyTorch ! Both use the SoTA SegFormer model 🤗 Simply navigate to 🤗 Notebooks and search for "semantic segmentation" 👉 But there's more 🧵

3

35

201

Sayak Paul

@RisingSayak

5 months

A project that I will cherish forever. Learned loads about profiling and a bit about Triton. Some of the CS stuff about computer architectures I studied in 2014 came to the rescue at times. Catch the blog here:

Accelerating Generative AI Part III: Diffusion, Fast

This post is the third part of a multi-series blog focused on how to accelerate generative AI models with pure, native PyTorch. We are excited to share a breadth of newly released PyTorch performance...

pytorch.org

PyTorch

@PyTorch

5 months

3x faster text-to-image diffusion models, all in pure PyTorch. No C++ needed. Check out our third blog post in the series on Accelerating Generative AI using Native PyTorch. 🔥

8

121

696

2

23

199

Sayak Paul

@RisingSayak

3 years

A minimal loop demoing noisy student training in #Keras and @TensorFlow . I believe this same loop can be extended to implement different flavors of consistency training. The loop is modified from the Keras Example on Knowledge Distillation ().

5

28

192

Sayak Paul

@RisingSayak

7 months

Long time no release 👀 Well, let's break the silence and allow me to present 🧨 diffusers 0.22.0 🔥 Three new pipelines, 🤗 PEFT integration, new functionalities, and QoL improvements 🔋🏆 Join in 🧵 to know more! 1/8

9

41

185

Sayak Paul

@RisingSayak

1 year

Large-scale image similarity systems are crazy 🤯 In this blog post, we discuss the underpinnings of any similarity system and provide a Colab Notebook and Space to play around with 😍 Check it out! ⤵️

2

44

184

Sayak Paul

@RisingSayak

3 years

In our latest #GCP blog, @algo_diver & I dig deep into a dual deployment pipeline. We introduce three new custom components in #TFX & utilize a good chunk of #ML ecosystem provided by Google - #Keras , @googlecloud , @TensorFlow , etc. Blog:

0

49

182

Sayak Paul

@RisingSayak

5 months

Why are companies, specifically Indian, still specifying that they need folks from IITs, NITs, top schools? Nuts! How is this relevant in 2023?

41

9

187

Sayak Paul

@RisingSayak

8 months

How about a fresh new pipeline in 🧨 diffusers! We now have BLIP Diffusion from @SFResearch , thanks to @ayush_tues ❤️ We can now enable ZERO-SHOT subject-driven generation and transfer style from images using ControlNets 🔥 📃⬇️

2

36

187

Sayak Paul

@RisingSayak

3 years

If you are a beginner like me in recommender systems and want to learn it using a code-first approach @TensorFlow Recommenders might just be the greatest place: . Just enough theory, TF/Keras best practices, and tips for efficient serving. Happy Sunday!

TensorFlow Recommenders

A library for building recommender system models.

www.tensorflow.org

3

36

182

Sayak Paul

@RisingSayak

3 years

Ever wanted to use Vision Transformers with #TFHub and #Keras . Well, pull your socks up now and get started. 16 different models are available for classification and fine-tuning :) More details: . @fchollet @DaoustMj @random_forests @TensorFlow 1/

5

37

184

Sayak Paul

@RisingSayak

3 years

I like it when researchers make use of architecture priors to make efficient design choices. **BotNet** ( @AravSrinivas et al.): Learn the low-res abstract features w/ convs & aggregate that information in a globally & contextually aware way using self-attention. 1/3

1

27

185

Sayak Paul

@RisingSayak

11 months

Starting today from 9:00 AM Vancouver time 🔥 If you're attending #CVPR2023 (virtually or physically), please do drop by! If you're attending in person, the location is **West 211**. @hila_chefer and I have wrapped our preps.

Sayak Paul

@RisingSayak

11 months

CVPR'23 is starting this Sunday 🔥 Our (with @hila_chefer ) tutorial - "All Thins ViTs" is also ready to go. The website is updated w/ all the latest details. Drop by if you're attending (physically or virtually)! We'll also have @MokadyRon as our guest!

3

39

179

2

35

174

Sayak Paul

@RisingSayak

1 year

As has happened before, I won't be able to present the tutorial in person because my VISA was rejected. No blame game 'cause they don't lead anywhere. So, instead, I've learned to focus on things under my control, e.g., ensuring I am working on amazing tutorial materials.

Sayak Paul

@RisingSayak

1 year

Incredibly excited to announce our @CVPR tutorial w/ the amazing @hila_chefer ! "All Things ViTs: Understanding and Interpreting Attention in Vision" Come for cool visualizations, exclusive insights, & interesting approaches ❤️ Catch the details here ⬇️

4

76

365

14

8

184

Sayak Paul

@RisingSayak

11 months

CVPR'23 is starting this Sunday 🔥 Our (with @hila_chefer ) tutorial - "All Thins ViTs" is also ready to go. The website is updated w/ all the latest details. Drop by if you're attending (physically or virtually)! We'll also have @MokadyRon as our guest!

3

39

179

Sayak Paul

@RisingSayak

7 months

Coded up a mini repo around the idea of "caption upsampling", as discussed in the DALL-E 3 tech report. This one uses `zephyr-7b-alpha` instead of GPT-4, allowing you to experiment with free and open alternatives 🤗 Repo: More comments in 🧵

7

44

178

Sayak Paul

@RisingSayak

2 years

I am not sure if you get this enough @fchollet , but I wanted to take a moment and wholeheartedly express my gratitude for the time you spend reviewing the PRs and the way you mentor the contributors.

5

182

Sayak Paul

@RisingSayak

3 years

Combining the benefits of convs and transformers is an emerging trend in computer vision research. MobileViT presents a simple yet unique way to achieve that while being mobile-friendly. In my latest example, I present a minimal implementation in #Keras ⬇️

François Chollet

@fchollet

3 years

New awesome tutorial from @RisingSayak : MobileViT, a mobile-friendly Transformer model for image classification. Includes building the model from scratch, training it, then converting it to TFLite for use on a mobile device.

1

67

314

2

30

178

Sayak Paul

@RisingSayak

2 years

Don't know if it's appreciated well enough but you should definitely check out this tutorial of NeRF on authored by @ariG23498 and @ritwik_raha . These guys did an amazing job not only w/ impl. but also w/ the explanations.

Keras documentation: 3D volumetric rendering with NeRF

keras.io

2

26

179

Sayak Paul

@RisingSayak

2 years

@OpenAI 's Whisper presented a plethora of pre and post processing techniques and a lot of heuristics for dealing with speech data. They even used an initial model to probe the training samples leading to higher error-rates to further curate the dataset. 1/

2

21

175

Sayak Paul

@RisingSayak

10 months

Pleased to share early experiments of performing DreamBooth on SDXL 0.9 w/ LoRA 🔥 SDXL is so GOOD - without much bells and whistles, it's able to capture the target subject much quicker 🪄 Open-sourcing: our code, trained LoRA checkpoints, & the experimental run pages 🤗 1/4

5

35

179

Sayak Paul

@RisingSayak

1 year

At @huggingface , we have been gradually adding first-class support for vision tasks -- conquering one CV task at a time. We recently added a fresh task page on Depth Estimation, collating the existing materials we have on it. Check it out here:

4

22

174

Sayak Paul

@RisingSayak

1 year

Holy cow 🐮 I didn't know that we offer a dedicated open-source solution for inferring with LLMs like BLOOM: Thanks to @algo_diver for making me aware of it!

GitHub - huggingface/text-generation-inference: Large Language Model Text Generation Inference

Large Language Model Text Generation Inference. Contribute to huggingface/text-generation-inference development by creating an account on GitHub.

github.com

3

41

175

Sayak Paul

@RisingSayak

2 months

PEFT 0.10.0 is out 🔥 The single most important thing you should know is that it now supports QLoRA with DeepSpeed ZeRO3 and FSDP. This allows you to fine-tune a 70B Llama model on two GPUs with 24GB memory each. Read that again! Hat-tip to @jeremyphoward and team 🫡 Notes 📜

3

34

176

Sayak Paul

@RisingSayak

2 years

Have been reading up stuff on training ViTs the right way lately. Here are five works that aren't leaving me anytime soon! In no particular order ⬇️ 1/

2

26

174

Sayak Paul

@RisingSayak

2 years

Just completed this #NeurIPS2021 tutorial "Self-Supervised Learning: Self-Prediction and Contrastive Learning" by @lilianweng and @_jongwook_kim . It's an excellent way to get introduced to the domain in 2021 at least. Slides:

4

35

172

Sayak Paul

@RisingSayak

2 years

New to dealing with video data in TensorFlow? This tutorial should be a saviour. Clear, succinct, and comprehensive.

Video classification with a 3D convolutional neural network | TensorFlow Core

www.tensorflow.org

1

37

169

Sayak Paul

@RisingSayak

2 years

A little something. 100 citations on an independently conducted research study on ViTs w/ @pinyuchenTW . Forever grateful to Pin-Yu for everything - guiding, mentoring, co-authoring, and so much more. Paper: Code:

9

8

164

Sayak Paul

@RisingSayak

10 months

Working at @huggingface gives me a sense of wholesomeness! On any given day, I could be found contributing features to 🧨 diffusers, training diffusion models, reviewing PRs, responding to issues, documenting things, presenting at #CVPR , etc. Dream job and all that 🤗

3

7

166

Sayak Paul

@RisingSayak

3 years

Is it possible to write an extremely readable implementation of a CLIP like model in #Keras & @TensorFlow ? HELL YEAH, it's totally possible. Bewildered by the simplicity and readability of this bad boy:

2

33

162

Sayak Paul

@RisingSayak

3 months

🤗 Accelerate power-user chronicles 👨‍🏫 Here, I show you how to load a pre-trained PyTorch model ~2x faster with Accelerate. The comments in the code snippet should be self-explanatory. But if you want to make it go brrr, this is your doc 👉 Isn't it a

4

25

163

Sayak Paul

@RisingSayak

3 years

A @TensorFlow 2 implementation of SimSiam by Xinlei Chen & Kaiming He () - According to me, it's the most practically feasible self-supervised learning method for visual recog. Thread ⬇️

GitHub - sayakpaul/SimSiam-TF: Minimal implementation of SimSiam (https://arxiv.org/abs/2011.10566)...

Minimal implementation of SimSiam (https://arxiv.org/abs/2011.10566) in TensorFlow 2. - sayakpaul/SimSiam-TF

github.com

2

18

160

Sayak Paul

@RisingSayak

29 days

New paper drop from ByteDance on improving few-step (1-8) inference of diffusion models 🎸 Interesting they used human feedback to improve on the performance loss often incurred by cosistency distillation. Check it out here ⬇️

1

33

166

Sayak Paul

@RisingSayak

5 months

More soon. Pure @PyTorch . No third-party.

10

12

158

Sayak Paul

@RisingSayak

3 years

I recently gave a talk on the trends in Computer Vision in 2021 I find interesting to work on. Here's my deck: . By no way, it's an exhaustive summary but I hope it'll be useful for those looking for ideas for their next project this summer.

4

27

159

Sayak Paul

@RisingSayak

10 months

Now you can render Iron Man in your favorite poses but much faster 🏎 We now support T2I adapters in 🧨 diffusers 🔥 T2I adapters are lightweight auxiliary networks & run ONLY once for the entire diffusion process giving ~ControlNet-like quality. Docs 📝

5

19

156

Sayak Paul

@RisingSayak

2 years

I started using #Keras seriously in 2017. Since then it has been my dream to contribute something significant to it. Yesterday (after more than 5 years of using it) it got fulfilled. ConvNeXt is now available in #Keras fellas: 1/

7

14

159

Sayak Paul

@RisingSayak

2 years

Unsolicited advice for logging ML experiments. Gather all the arguments and their values in a single variable (preferably use `ml_collections`) and pass them to the `config` param of `wandb.init(..., config=args)`. Thank me later if you're not doing it already.

3

13

159

Sayak Paul

@RisingSayak

10 months

Everyone seems to be releasing something new. Let me have mine too 🫡 Presenting a comprehensive inference benchmark for vision models from the #keras ecosystem 🏎️ Code, commentary, fancy plots, @weights_biases report ⬇️ Interesting findings 🔍 1/8

6

32

159

Sayak Paul

@RisingSayak

4 years

A very well-written tutorial by @mat_kelcey on Metric Learning: .

Keras documentation: Metric learning for image similarity search

keras.io

1

25

153

Sayak Paul

@RisingSayak

1 year

Why do people still use Google Drive, Dropbox, OneDrive, etc. to store model checkpoints and datasets? Why not 🤗 Hub (allows for private repos to respect your privacy)? It's easily shareable, you get lineage, and so many other features. Comment with your pain points.

23

6

157

Sayak Paul

@RisingSayak

1 year

Did you know 🤗 Transformers 4.21.0 had a nifty update on TF XLA? That update allows you to run text generation models in TensorFlow 100x faster from 🤗 Transformers with just a single line of code! Know more here: @TensorFlow @huggingface

4

29

160