Cristian Garcia @cgarciae88 Twitter profile

Pinned Tweet

Cristian Garcia

1 year

Announcing NNX 0.0.4 (Beyond Pytrees) 🚀 Major update: - Simplified APIs - Modules are now regular mutable python classes - Added support for Module graphs (RIP pytrees) NNX is now more pythonic, simpler, and still as powerful as Flax.

2

14

115

Last Seen Profiles

@ahyconics

@kawaii3101

@virtual_dunes

@nrkipod

@ovorich_fanpage

@pengen_stw

@mainfdevA

@RPurnama8249

@HighHopes1138

@julie_renbe

@jebezhg

@Poison30742016

@stw_pdg

@honey_1am

@bloodsband

@thhguru_

@mst_ern

@cryptopix_san

@deadcatdaddy

@1Ulwi

@jadenspov

@exec_tours

@ksa9sx

@WhatsTheStatus

@CanABlKManVent

@ChxrryxFizz

@Tokyo_Decadance

@Pecintastw4050

@ahyconics

@nznbl

@Ethannismo

@arara_nini

@Ladrxn

@camdensirmon

@jandakembangstw

@PetraTeslaLove

Cristian Garcia

@cgarciae88

1 year

Its not ChatGPT vs Bard Its GPUs vs TPUs ...and who can multiply matrices at a lower cost.

120

547

5K

Cristian Garcia

@cgarciae88

1 year

The real winner from the ChatGPT hype is Nvidia

58

122

2K

Cristian Garcia

@cgarciae88

29 days

I'm getting tired of Sam Altman's ego I'd rather have a future where everybody has their own LLaMa at home running on @__tinygrad__ 's box than being at the mercy of a corporation.

Tsarathustra

@tsarnick

30 days

Sam Altman says instead of Universal Basic Income, there should be Universal Basic Compute, where everybody gets a slice of GPT-7's compute

899

267

2K

93

55

1K

Cristian Garcia

@cgarciae88

1 year

Copilot is AMAZING at creating Matplotlib plots, it's one of the only times I just put a comment and accept multi-line suggestions. Plotting is tedious and has hard to remember APIs, a perfect target for automation.

17

61

1K

Cristian Garcia

@cgarciae88

7 months

Python is nothing without Numpy

36

73

1K

Cristian Garcia

@cgarciae88

16 days

NumPy 2.0 coming soon

6

146

1K

Cristian Garcia

@cgarciae88

6 months

mandatory first week picture 😅

19

16

851

Cristian Garcia

@cgarciae88

7 months

Happy to announce I'll be joining @GoogleDeepMind in the next few weeks 🙂. I'll continue my work on JAX/Flax, but I'll also be working on a cool new thing™ Pretty excited about the new horizons! 🇬🇧

47

8

569

Cristian Garcia

@cgarciae88

4 years

Lambda Networks: SOTA on ImageNet. Again Transformer-like architectures dominating over a new field 🚀. CNNs had a good run 😰. Paper: Awesome video from @ykilcher :

1

110

558

Cristian Garcia

@cgarciae88

2 years

Fun fact: you can implement UNet recursively

11

46

502

Cristian Garcia

@cgarciae88

8 months

stop making physicists angry

17

32

491

Cristian Garcia

@cgarciae88

4 months

This is one of my favorite JAX guides. Every library should have one guide where it reimplements the core ideas in simple terms.

5

60

481

Cristian Garcia

@cgarciae88

28 days

@LouisAnslow is the uncharismatic leader SBF?

10

3

440

Cristian Garcia

@cgarciae88

7 months

At this rate Karpathy is going to be working on Grok by tomorrow.

6

14

424

Cristian Garcia

@cgarciae88

7 months

BREAKING: Sam Altman to invest billions in HuggingFace, says closed models have no future.

14

32

415

Cristian Garcia

@cgarciae88

6 months

@ThePrimeagen other is such a great language

5

0

396

Cristian Garcia

@cgarciae88

6 months

Mistral's team litterally just used torch instead of coding kernels with cuda and raised 120+M. Investors what are you doing 😢 this is horrendous

Ky⨋ Gom⨋z (U/ACC) (HIRING)

@KyeGomezB

6 months

Mistral's team litterally just forked llama changed less than 40 lines of code and raised 120+M. Investors what are you doing 😢 this is horrendous

94

71

1K

17

22

388

Cristian Garcia

@cgarciae88

1 year

Microsoft: waiting list for 'BingGPT' Translation: we don't have enough GPUs

15

380

Cristian Garcia

@cgarciae88

2 months

@jxmnop prepare for the airstrike

1

0

381

Cristian Garcia

@cgarciae88

8 months

implementing tensor + data parallel training from scratch:

6

19

328

Cristian Garcia

@cgarciae88

2 years

JAX 0.4.0 is coming out soon and it contains some awesome features! 🚀 🧵...

6

47

322

Cristian Garcia

@cgarciae88

5 months

what is wrong with him?

63

6

316

Cristian Garcia

@cgarciae88

6 months

Bard. Is. Insanely. Useful.

22

19

318

Cristian Garcia

@cgarciae88

3 months

BREAKING: Mistral team to relocate to The Bahamas.

Risto Uuk

@RistoUuk

3 months

The EU AI Act passes in the European Parliament! 523 voted for, 46 against, 49 abstained. I’m grateful for being in Strasbourg today, witnessing one of the last hurdles before the first-ever comprehensive AI regulation, the AI Act, becomes an actual law in the EU.

26

19

137

3

12

299

Cristian Garcia

@cgarciae88

3 months

@peterrhague Ah yes, it worked out great for Germany.

2

1

288

Cristian Garcia

@cgarciae88

2 years

Here is a thread with a couple of samples from the new Imagen model from Google. 🧵

7

51

284

Cristian Garcia

@cgarciae88

2 years

Hey! The JAX team ( @shoyer ) recently released Tree Math 🌲🔢, a small library that lets you operate over pytrees as if they were ndarrays. Under the hood this reduces to a tree_map but it makes certain types of code look very clean.

3

34

278

Cristian Garcia

@cgarciae88

7 months

Hey! Sharing this notebook that showcases how to implement various parallelism strategies like data parallel and model parallel in JAX. It implements a very simple model in pure jax using shard_map and jit with different sharing configurations.

A tour of parallelism in JAX

Colaboratory notebook

colab.research.google.com

4

39

263

Cristian Garcia

@cgarciae88

1 year

I am two "AI influencer" tweets away from blocking the word "GPT"

23

32

262

Cristian Garcia

@cgarciae88

3 years

I am shocked 🤯 Deep Learning in JAX starts to get stupidly simple if Modules are Pytrees. Turns out jit is aware about the static part of Pytrees and recompiles upon change:

4

35

250

Cristian Garcia

@cgarciae88

1 year

Since there is a lot of fuzz about Mojo, maybe it would be good to take a look a the Codon project: "Codon is a high-performance Python compiler that compiles Python code to native machine code without any runtime overhead."

8

31

243

Cristian Garcia

@cgarciae88

2 years

JAX code I ❤️ #2 In the old days you could code a nice pairwise formula but vectorizing added a lot of unpleasant artifacts 😕 (tiling, broadcasting). Using a double vmap however, you can teach your beautiful function to operate over sets without changing a single line 🔥

6

24

225

Cristian Garcia

@cgarciae88

2 years

📣 Personal update: thanks to a partnership between Quansight and @GoogleAI I am now working full time with the Flax team 🤩🚀 Very excited to be using JAX for my day job!

11

10

216

Cristian Garcia

@cgarciae88

5 months

I've never seen someone triggered by how the probability of an event in a stochastic process is calculated

21

9

202

Cristian Garcia

@cgarciae88

1 year

@karpathy This is like the stone age of debugging 😅 Check out vscode's debugger for the real stuff. Here is a good tutorial:

How to make most of your python debugger in VSCode?

Watch variables, use conditional breakpoints, call stack, exception breakpoint, etc- when working on larger projects

towardsdatascience.com

7

15

202

Cristian Garcia

@cgarciae88

3 years

Hey Twitter! Very excited to announce the JAX Global Meetup 🎉 Its a fully online meetup for people across the world passionate about JAX, Deep Learning, and Scientific Computing. I have the pleasure of being able to co-host it with @bhutanisanyam1 😁

3

36

190

Cristian Garcia

@cgarciae88

2 years

A dev at a friends workplace committed the companies GCP keys on a personal Github repo 🤦 "Hackers" turned on 200+ VMs with 4 T4 GPUs each over the weekend amounting to 25K USD. Any advice for them? (apart from dont do it again)

33

21

188

Cristian Garcia

@cgarciae88

5 months

Friendly reminder that if you love einops but are tired having to switching from op to op, @MilesCranmer created einop which infers which op you need so you don't have to think.

3

26

189

Cristian Garcia

@cgarciae88

9 months

Damn, they're going to have to rewrite all their JAX code?

Anthropic

@AnthropicAI

9 months

Today, we’re announcing that @Amazon will invest up to $4 billion in Anthropic. The agreement is part of a broader collaboration to develop reliable and high-performing foundation models.

236

1K

6K

10

5

186

Cristian Garcia

@cgarciae88

3 years

Created my first 🤗 @huggingface dataset: The Point Cloud MNIST Its a toy dataset to mess around with point clouds and play with architectures like DeepSets, Transformers, or more fancy Geometric DL techniques without consuming too much resources.

2

27

183

Cristian Garcia

@cgarciae88

1 year

What is new in JAX? * Triton integration * Pallas: a JAX frontend for Triton * shard_map: control communication inside pjit/jit functions.

4

18

183

Cristian Garcia

@cgarciae88

4 years

🚀Pleased to announce Elegy! A Keras-like Deep Learning framework based on Jax + Haiku. Its still in very early stage😅 we would love to get feedback so we can explore all the possibilities that Jax brings!

8

38

172

Cristian Garcia

@cgarciae88

1 year

one of those you "need someone that knows Spanish on your team" moments 😆

NVIDIA cuLitho

A library with optimized tools and algorithms for GPU-accelerating computational lithography and manufacturing process of semiconductors.

developer.nvidia.com

14

28

182

Cristian Garcia

@cgarciae88

2 years

Theory is fine and all but I would pay good money for a "Deep Learning Alchemy" book about all the dark magic that goes into training these models.

6

16

177

Cristian Garcia

@cgarciae88

7 months

when function composition in JAX starts blowing your mind

10

7

162

Cristian Garcia

@cgarciae88

3 months

@francoisfleuret this is python 101 (its worse for default arguments)

7

1

159

Cristian Garcia

@cgarciae88

1 year

ITS A CULT

10

7

154

Cristian Garcia

@cgarciae88

7 months

sorry to interrupt the new season of OpenAI with a 100% drama free JAX tutorial (you love to see it) let that XLA content heal your soul ☀️

Felix Köhler

@felix_m_koehler

7 months

Check my latest video tutorial on the JAX lax scan function that is useful to produce trajectories by autoregressively evaluating time steppers. We will conveniently repurpose the KS-solver of the last weeks. 🧵 What I like to use this function for

2

14

86

2

9

157

Cristian Garcia

@cgarciae88

1 year

STOP. BLAMING. OPEN. SOURCE.

OpenAI

@OpenAI

1 year

We took ChatGPT offline Monday to fix a bug in an open source library that allowed some users to see titles from other users’ chat history. Our investigation has also found that 1.2% of ChatGPT Plus users might have had personal data revealed to another user. 1/2

867

1K

8K

13

11

152

Cristian Garcia

@cgarciae88

5 months

JAX folks, learn Pallas and time and memory will be yours.

4

7

148

Cristian Garcia

@cgarciae88

4 months

V-JEPA Gemini 1.5 Sora imagine LLaMa 3 today

7

12

143

Cristian Garcia

@cgarciae88

2 years

📣 Announcing Flax v0.5.1 I am really excited to share this release as it contains a feature I've been waiting on: a new `tabulate` method that generates a tabular representation using the rich.tables library which produces beautiful ANSI colored text for improved readability🤩

7

10

144

Cristian Garcia

@cgarciae88

3 years

Made this new post about Quantile Regression! Its a very useful technique to estimate uncertainty, very easy to understand / implement. Example in JAX 😄 Blog: Notebook: Repo:

4

21

136

Cristian Garcia

@cgarciae88

4 years

Deep Mind recently came out with Long Range Arena, a benchmark suite to pit Efficient Transformers against each other in Long Range Tasks 🏃🏃‍♀️🏃‍♂️. Do we finally have a winner? Paper:

3

29

136

Cristian Garcia

@cgarciae88

2 years

📣 New 'Transfer Learning' Flax guide! We added a new guide that shows how to use models from @huggingface 's transformers library, perform parameter surgery, and freeze parameters with optax or perform fine-tunning with different learning rates.

2

24

133

Cristian Garcia

@cgarciae88

2 months

Penzai visualization support for NNX coming soon! JAX ecosystem is going to start looking amazing.

4

22

135

Cristian Garcia

@cgarciae88

3 months

MaxText is probably the best to train LLMs in JAX/Flax outside of Google. They provide configs for: gemma-2b, gemma-7b, gpt3-175b, gpt3-22b, gpt3-52k, gpt3-6b, llama2-70b, llama2-7b, mistral-7b, mixtral-8x7b

2

21

134

Cristian Garcia

@cgarciae88

3 months

@Team0Percent what is LilKirbs doing rn?

5

1

131

Cristian Garcia

@cgarciae88

7 months

Babe wake up, a new JAX library just dropped. Jeometric is a new GNN library for JAX, based on Flax (you love to see it). It offers both common GNN layers and a data format for the inputs. Check it out!

Daniele Paliotta

@DanielePaliotta

7 months

Exciting News! (for me) 🐍 I've just published Jeometric, a new Python library for graph neural networks in JAX! If you use PyTorch Geometric, it will look familiar! It's still early stage and I'll be building it in public. Pull requests very welcome ❤️

7

38

246

2

13

131

Cristian Garcia

@cgarciae88

23 days

getting an emotional response from OAI is a W for google also, marc rebillet was awesome

Sam Altman

@sama

24 days

i try not to think about competitors too much, but i cannot stop thinking about the aesthetic difference between openai and google

3K

1K

26K

9

1

130

Cristian Garcia

@cgarciae88

7 months

I wrote this (Deep) Quantile Regression tutorial in JAX/Elegy a while back, I had was a lot of fun. I read you can adapt QR to the conformal theory so maybe I'm onboard with the conformal agenda 😅. I'll start getting angry at bayesians soon :p

2

22

125

Cristian Garcia

@cgarciae88

3 years

Getting started with Deep Learning in JAX with Treex in 5 tweets If you are JAX-curious but don't want to stray too far from the Pytorch-way, Treex is here to save the day 🌳 🧵

GitHub - cgarciae/treex: A Pytree Module system for Deep Learning in JAX

A Pytree Module system for Deep Learning in JAX. Contribute to cgarciae/treex development by creating an account on GitHub.

github.com

5

24

124

Cristian Garcia

@cgarciae88

1 year

Meta is really publish for Open Source ♥️ Actions speak louder than having "Open" in your company name

AI at Meta

@AIatMeta

1 year

Announced by Mark Zuckerberg this morning — today we're releasing DINOv2, the first method for training computer vision models that uses self-supervised learning to achieve results matching or exceeding industry standards. More on this new work ➡️

92

902

4K

5

9

119

Cristian Garcia

@cgarciae88

1 year

The more I learn about model parallelism the more mind-blowing 🤯 jax.Array / pjit become. With relative ease one can try out techniques from ZeRO and other papers. (picture from Megatron)

3

16

116

Cristian Garcia

@cgarciae88

7 months

JAX's automatic sharding propagation is beautiful ✨ If eager mode is a thing some day it will be crazy how easy it is.

Alex McKinney

@alexfmckinney

7 months

I was going to write a blog on parallel JAX until they updated the API a lot. Now it is so shockingly easy to do that it might not be worth it 😅 Below is the diff between my single GPU training script and one that I just tested on 8xTPUs in data parallel. 🔥

6

18

230

3

13

94

Cristian Garcia

@cgarciae88

4 years

🔥Amazed at the simplicity of Jax + Haiku! The following code implements a cumulative accuracy metric. Haiku hooks make implementing Deep Learning code so much easier. Jax is the future! gist:

4

14

112

Cristian Garcia

@cgarciae88

7 months

speaks for itself

3

6

116

Cristian Garcia

@cgarciae88

2 years

Fan of einops? Check out its new pack and unpack functions 🚀 Not using einops? What are you waiting for....

11

19

110

Cristian Garcia

@cgarciae88

6 months

This section from Mamba about selective copying is basically the researcher's equivalent of a unit test

1

10

111

Cristian Garcia

@cgarciae88

5 months

Good time to remember about Annotated S4 by @srush_nlp written in JAX/Flax. The quality is amazing. Huge bonus: its written in JAX/Flax 🎉

3

16

110

Cristian Garcia

@cgarciae88

3 years

We need a dedicated collection of Toy Datasets for Machine Learning: 1. They can be more interesting than real datasets, specially if designed to be hard for certain algorithms. 2. They are more useful for teaching / learning. Maybe @huggingface / @kaggle can help with this?

10

9

109

Cristian Garcia

@cgarciae88

8 months

@ylecun "we are so back" would've been more epic 😅

1

0

110

Cristian Garcia

@cgarciae88

2 years

JAX Global Meetup is back! Join us this Friday Oct 7, @_arohan_ will be talking about second order optimizers, deep learning, and JAX. @borisdayma and I will be hosting the event. Event link: Join the JAX Meetup to notified of all future events!

4

28

108

Cristian Garcia

@cgarciae88

7 months

XLA loves all its children

Qiao Zhang

@zhangqiaorjc

7 months

JAX is great for GPUs too! With a strong team, and excellent execution, you can train a world-class LLM in ~two months!

4

17

174

1

6

105

Cristian Garcia

@cgarciae88

1 year

Hey JAX users, here is a slightly better pattern for splitting your RNG key to what is commonly used. It some nice properties: - the RNG has longer cycles - no need to iteratively update `key` - easier checkpointing (credit to @froystig )

3

10

108

Cristian Garcia

@cgarciae88

3 months

Easily the best tutorial on distributed training I've seen 🔥 Uses JAX/Flax and shows how to use the low-level communication primitives to teach the basic concepts, even if the compiler can do most of this for you I think it's super valuable to learn how things work.

Phillip Lippe

@phillip_lippe

3 months

Along with our new tutorials (), we organize a lecture on the topic! Feel free to join, also if you’re new to JAX

1

19

126

2

17

106

Cristian Garcia

@cgarciae88

4 months

Why does every DL framework reinvent data loading?

13

3

100

Cristian Garcia

@cgarciae88

7 months

JAX success in RL is due to a couple of reasons: 1. environments are being ported to run on device (the jax.numpy API might help here) 2. once ported environments can be be easily parallelized/distributed to run along with the agent + trainer 3. synchronization is easy with

Faris Sbahi

@FarisSbahi

7 months

@cgarciae88 Do you think there’s a technical reason for this that’s specific to RL, or just that in the absence of tech debt it’s a good way to go? Of course RL community momentum will play a role increasingly.

2

1

6

5

10

100

Cristian Garcia

@cgarciae88

7 months

@ThePrimeagen @OpenAI not today

“paula”

@paularambles

7 months

ml engineers watching frontend devs build ai startups

56

371

5K

1

96

Cristian Garcia

@cgarciae88

5 months

jax.jit will soon be able to tell you why a function is recompiling/retracing 🎉 This is a nice quality-of-life update when debugging JAX programs. Props to @SingularMattrix and @yashk2810 for the awesome log messages.

Matthew Johnson

@SingularMattrix

5 months

@Thomas_ensc @cgarciae88 Finally started adding some tooling along these lines:

5

1

18

1

8

97

Cristian Garcia

@cgarciae88

1 year

Hey JAXers, I've been searching for an abstraction that could allow us to create a libraries as powerful as Flax but as simple as Equinox. This is what I've found: 🧵

1

11

97

Cristian Garcia

@cgarciae88

1 year

Porting nanoGPT to JAX 🚨 Live now:

minGPT-JAX: Live coding | EP3

Live coding with JAX and Flax.Repo: https://github.com/cgarciae/nanoGPT-jaxTags: machine learning, deep learning, AI, jax

www.youtube.com

2

9

95

Cristian Garcia

@cgarciae88

7 months

A bunch of anime characters and I are waiting to see if @sama shows up... modern life is hard to explain.

5

4

93

Cristian Garcia

@cgarciae88

1 year

Introducing 🌀 Ciclo A functional training loops library for JAX

3

14

92

Cristian Garcia

@cgarciae88

1 year

@DrJimFan @tomgoldsteincs Watch how high-schoolers start reading ML papers to reverse engineer GPTZero

7

3

90

Cristian Garcia

@cgarciae88

4 months

Working on NNX's readthedocs page and adding a Quick Start guide. NNX is an Neural Network library for JAX that aims to aims to be Pythonic / support Object Oriented patterns.

6

10

90

Cristian Garcia

@cgarciae88

1 year

Can we have the power of Flax with the simplicity of Equinox? Introducing NNX: Neural Networks for JAX A highly experimental 🧪 proof of concept framework that provides Pytree Modules with: * Shared state * Tractable mutability * Semantic partitioning (collections)

1

11

90

Cristian Garcia

@cgarciae88

5 months

This is not only a Quickstart, it's how it all started (autograd).

0

8

92

Cristian Garcia

@cgarciae88

6 months

Happy to announce I've been promoted to Chief JAX Tweeter 𝕏 (on day 1)! Will use this power with great responsability 🦾🙏

7

1

89

Cristian Garcia

@cgarciae88

1 year

Will be embarking on a journey of trying to port minGPT to JAX live on YouTube! 😅 Today 4PM EST.

minGPT-JAX: Live coding | EP1

Porting minGPT to JAX

www.youtube.com

2

14

89

Cristian Garcia

@cgarciae88

3 months

HELL NO

Billy Perrigo

@billyperrigo

3 months

🚨Exclusive: a report commissioned by the U.S. government says advanced AI could pose an "extinction-level threat to the human species" and calls for urgent, sweeping new regulations

149

223

634

9

12

89

Cristian Garcia

@cgarciae88

3 years

Hey twitter! I am happy to announce that next week I will be joining the @quansightai team 🎉I will be working with our partners to solve real world Data Science / ML problems. Additionally I'll be supporting 2 Open Source projects: Pandas and Elegy ♥

10

5

89

Cristian Garcia

@cgarciae88

5 months

Working in person with the JAX team is AWESOME!!

2

4

87

Cristian Garcia

@cgarciae88

7 months

we need to stop jax development now, no new versions

4

3

85

Cristian Garcia

@cgarciae88

1 year

Anyone knows why pip sometimes downloads many versions of a package before installing it?

19

4

82

Cristian Garcia

@cgarciae88

7 months

source: its a joke

3

1

81

Cristian Garcia

@cgarciae88

9 months

@O42nl You are doing an amazing job at depressing us 😅

1

0

82

Cristian Garcia

@cgarciae88

4 months

Remember when breakthroughs came with a corresponding paper?

6

8

82

Cristian Garcia

@cgarciae88

1 year

A very nice result from Open Assistant! Not on par with ChatGPT (expected) but still very descent for an Open Source effort. This could be huge for startups and companies that want to fine-tune it on their own data or need to run it offline.

2

15

81