Adam Hibble @Algomancer Twitter profile | Pikagi

Pikagi

Adam Hibble

@Algomancer

3,611

Followers

1,006

Following

214

Media

3,331

Statuses

I generate models that generate other stuff, working on @mancerlabs -- Prev: Founder of Popgun Labs (Techstars), Founder of the @QUTCode Network.

Joined August 2013

Don't wanna be here? Send us removal request.

Pinned Tweet

@Algomancer

Adam Hibble

2 months

Training a JEPA with randomly sampled projectors enforces more pairwise independence with (VCREG on projector(z)) with distilation and layerwise targets converges a lot quicker than vanilla JEPA in my very early tests. No collapse issues on a wide set of hyper parameters.

1

6

71

Last Seen Profiles

@foz830

@stw_pdg

@Lavendelhe17923

@GringoCingolani

@MarkHill1029021

@viani310

@dupedotcom

@nava_amber24863

@s0ftqbEJBEzhK2S

@lifecard_jp

@Redheadstack2

@PrettyAsProblem

@hellokirthy

@byLeojunior

@forty5kimbo

@NCFC_Nick

@Mmmhmya

@FMarenas

@soogz1

@cigkoftedoritos

@asdfmeh

@ofanyothername

@vasi_lw

@pinkjuku

@nexas_TSA

@zahollii

@BillMuls

@udin12344566

@BigKushFace

@Warren_GBB

@RodVernon23

@BigKushFace

@LoveYourselfPh

@bloodylcious

@stangazemba

@cheriofhilarion

@Algomancer

Adam Hibble

7 months

we are so b̶a̶c̶k̶ b-rep. meet cadmancer, the first in a series of models we will be releasing you can export the designs it creates, optionally to your favorite cad/cam software (fusion 360 etc), and manufacture it. very early work - still lots more to do before prime time,

79

106

940

@Algomancer

Adam Hibble

7 months

The plan is simple. Solve Cad Solve Robotics Build Factories. ??? Dyson Spheres.

28

17

428

@Algomancer

Adam Hibble

6 months

Can one of you really amazing front end developers please build a llm native ui kit. Specifically, UI's are just function calls / tool use dependent on human input. There is no reason we should be stuck in call and response land. These should be very minimal components

23

16

235

@Algomancer

Adam Hibble

7 months

Solve Cad, Solve Robotics, Build Factories.

15

18

221

@Algomancer

Adam Hibble

23 days

I've been adding continuous latents to Llama 3. I think its a good method to apply variable compute at inference. But, also pretty neat to adapt behavior without consuming context length, and also continuous latents are easier to backprop for some guidance post training. Working

Tweet media one

9

6

122

@Algomancer

Adam Hibble

5 years

Everything in this clip is generative apart from the lyrics (we can do this also), vocal synthesis, the drums, bass, melodies, support instruments, Even the audio is mastered by AI. Just a teaser, can't wait to show the rest.

Tweet card media

Popgun - Vocals

Popgun is using AI to make pop music. For the past year we have been teaching an AI to sing. Using text and midi as input, we generate vocal tracks in many d...

www.youtube.com

2

7

102

@Algomancer

Adam Hibble

3 months

The fact you can train a diffusion process using JEPA without a reconstruction objective is lit. Things confirmed to work in my weekend hack. vq-jepa diffusion jepa jepa-vae hierarchical jepa

5

8

116

@Algomancer

Adam Hibble

22 days

I havn't really shared this much. But, a couple of years ago (mostly 2017-2018) myself and couple other aussies built an ableton plugin that I think had a bunch of cool ideas in it that was ahead of its time from a pure ML perspective. Mech Interp and controllable generations

8

10

106

@Algomancer

Adam Hibble

4 months

If you're outside of the valley, working on generative models and are having issues in training, stability, or are trying to implement a new paper that doesn't have open source code and something isn't converging. My DM's are open, happy to jump on a 15 minute discord call and

1

5

78

@Algomancer

Adam Hibble

7 months

We have the longer term vision of end to end generative manufacturing. So, this is just the start.

2

0

62

@Algomancer

Adam Hibble

6 months

I am so pumped to see more people entering generative cad. I need it to exist to do what I want to do and there are a lot of problems to solve, both technical and user experiences. The things end to end design for manufacturing unlock is huge, a damn near infinite market.

2

4

57

@Algomancer

Adam Hibble

2 months

I learned to code modding games at like 8, and I remember thinking about computers as this beautiful magical thing that a bunch of really cool people built. I remember building a tool for diablo 2 to extract/modify sprites and thinking - wow, whoever built this sprite engine is

1

4

52

@Algomancer

Adam Hibble

9 years

I just published “Zero to #Deeplearning with #Scala ”

1

13

48

@Algomancer

Adam Hibble

6 months

@levelsio I am glad you can see exactly what I do. I am building this for our cadmancer model. And training in our user experience into the model. But, there is so much space to explore here and I think there are really incredible people who would smash this out

@Algomancer

Adam Hibble

7 months

we are so b̶a̶c̶k̶ b-rep. meet cadmancer, the first in a series of models we will be releasing you can export the designs it creates, optionally to your favorite cad/cam software (fusion 360 etc), and manufacture it. very early work - still lots more to do before prime time,

79

106

940

1

1

43

@Algomancer

Adam Hibble

7 months

Guys, my model demo wasn’t meant to be this popular. It says react template in the tab of the playground still. I guess this is why people share stuff.

@Algomancer

Adam Hibble

7 months

we are so b̶a̶c̶k̶ b-rep. meet cadmancer, the first in a series of models we will be releasing you can export the designs it creates, optionally to your favorite cad/cam software (fusion 360 etc), and manufacture it. very early work - still lots more to do before prime time,

79

106

940

6

1

42

@Algomancer

Adam Hibble

19 days

Nothing beats the feeling of a previous sota eval passing by 40% through training with 3x smaller model. (specific to my task) Multi modal jepa COOKING.

2

1

44

@Algomancer

Adam Hibble

3 months

No Adam, you shouldn't buy $400k of gpus and put them in your house. Yet.

11

1

39

@Algomancer

Adam Hibble

8 years

In the last month or so #startups comprised of mostly @QUTCode devs raised over $5 mil in funding. Proud. #StartupAus @qutace @QUTBusiness

3

20

35

@Algomancer

Adam Hibble

6 months

Putting up a $2000 usd bounty for a fast pscan that matches this api. Triton or otherwise.

@francoisfleuret

François Fleuret

@francoisfleuret

6 months

@dvruette Looks fine to me! But how can it be fast, my pscan is far from great.

2

0

7

6

8

36

@Algomancer

Adam Hibble

3 months

I really like this masking strategy for point-jepa. Green is the context, Blue is masked (target) Red is the positions that the predictor gets. Seems to be really stable for learning quite good powerful 3d pointcloud and high dimensional cloud representations and embeddings.

3

1

35

@Algomancer

Adam Hibble

6 months

Sometimes I wonder if the reported model drift, and annoying changes in behavior that people report in @OpenAI 's chatgpt is literally just the date they put in the system prompt acting as a seed and changing behavior in a larger than expected way.

Tweet media one

4

0

33

@Algomancer

Adam Hibble

23 days

LLama3 with "low supportiveness" direction --- If it's not working, try harder. --- You're just not cut out for this: Honestly, you're probably just not meant to be fit. You should just accept that and move on. --- You're just not good enough: Let's face it, you're probably just

3

3

32

@Algomancer

Adam Hibble

10 months

Quickly whipped up a A PyTorch implementation of Bayesian Flow Networks from @rupspace Have completed the discrete loss, would love quick a sanity check.

GitHub - Algomancer/Bayesian-Flow-Networks: A simple implimentation of Bayesian Flow Networks (BFN)

A simple implimentation of Bayesian Flow Networks (BFN) - Algomancer/Bayesian-Flow-Networks

1

3

33

@Algomancer

Adam Hibble

8 years

@ABSCensus cost $9 mil. @QUT students at #CNWHack in 54hrs. #CensusFail - 4 Mil+ Requests

Tweet media one

3

32

32

@Algomancer

Adam Hibble

7 months

AI isn’t the thing, it’s the thing that gets you to the thing.

0

4

31

@Algomancer

Adam Hibble

4 months

This post went really well, I am now so excited by the reality of so many small teams working on really awesome stuff all over the world. Pretty impressed with the quality of thinking. The wide distribution of these skills and random tidbits is super important.

@Algomancer

Adam Hibble

4 months

If you're outside of the valley, working on generative models and are having issues in training, stability, or are trying to implement a new paper that doesn't have open source code and something isn't converging. My DM's are open, happy to jump on a 15 minute discord call and

1

5

78

4

1

28

@Algomancer

Adam Hibble

6 months

I love holy shit moments when you finish training a new model 🔥🔥🔥

2

1

29

@Algomancer

Adam Hibble

5 months

There needs to be an open source model (with data pipelines etc) with a substantial (32b+) compute allocation that gets trained regularly. The amount of training tricks on the table if it was just a pooled effort would be huge. The whole "surprise model release" is great for

4

2

28

@Algomancer

Adam Hibble

7 years

I go to LA for 3 months with the @popgunlabs team and #Myriad2017 takes Brisbane by storm and the storm takes myriad.

0

3

29

@Algomancer

Adam Hibble

7 months

@ilyasut This sentence is unlikely under the distribution of language I have trained on.

0

0

28

@Algomancer

Adam Hibble

7 months

@gazorp5 We do train on some open scad, but that would suck as a general cad kernel long term. We also train on b-rep primitives which is basically a big graph of parametric geometry nodes and a bunch of other stuff and basically anything else cad related that we can. Then fine tune for a

5

2

28

@Algomancer

Adam Hibble

9 years

So Pumped to announce @QUTCode 's first hackathon #QUTHACK15 Hope to see you all there @CatalystQLD @QUT @QUT_IDEA

Tweet media one

0

20

25

@Algomancer

Adam Hibble

6 months

This is sick work. I think this, plus the state space models build a lot of confidence in my thinking that all you need is fast weight gates and skip connections. That is the cool thing about transformers, its why mamba works. My prediction is this line of work, and variable

2

2

27

@Algomancer

Adam Hibble

7 months

Hello all of you silicon valley people looking at my twitter. I am glad you liked the demo, you should invest in Brisbane Australia because there is some dope shit here. Lets fucking go.

1

0

27

@Algomancer

Adam Hibble

6 months

It begins.

@MancerLabs

Mancer Labs

6 months

self.apply(self._init_weights)

3

2

17

6

1

26

@Algomancer

Adam Hibble

6 years

This time last year we had just started settling in Los Angeles for @techstars music. I have been ridiculously privileged to work with @mawsonguy and a team filled with the smartest people I know and I am excited to be working with the folks @khoslaventures .

@SplashMusicCo

Splash

6 years

Popgun is excited to announce seed funding from Silicon Valley-based Khosla Ventures. The round was led by Khosla and also included Techstars Ventures. #startupaus @mawsonguy @Algomancer @AdvanceQld

Tweet media one

2

15

72

3

4

26

@Algomancer

Adam Hibble

7 years

Startup catalyst completely changed my life trajectory, you should apply.

@catalyst_au

Startup Catalyst

7 years

Want to win your way onto a Startup Catalyst mission? Are you a Queenslander between 18 and 24? Checkout @AdvanceQld

0

22

29

1

11

25

@Algomancer

Adam Hibble

5 years

It brings me unending happiness to see the genuine reactions musicians have when jamming with the tech my team and I have built. Every head bob is like a shot of Oxytocin & serotonin.

0

3

25

@Algomancer

Adam Hibble

5 months

@darrenangle EvalShot: Training on the evals is all you need.

2

0

23

@Algomancer

Adam Hibble

4 months

I want vscode to know a PyTorch tensor shape on hover. I would pay for this plugin.

8

0

22

@Algomancer

Adam Hibble

10 months

@soumithchintala @nnaisense @srush_nlp Spent the morning grokking it. Started working on an implementation - have completed the discrete version. I plan to reproduce all the plots and experiments.

GitHub - Algomancer/Bayesian-Flow-Networks: A simple implimentation of Bayesian Flow Networks (BFN)

A simple implimentation of Bayesian Flow Networks (BFN) - Algomancer/Bayesian-Flow-Networks

1

3

24

@Algomancer

Adam Hibble

8 years

How two @QUTCode Students built a better Census site in just 54 hours for $500 #CensusFail

Tweet media one

1

30

23

@Algomancer

Adam Hibble

2 months

The relationship between JEPA's predictor and curiosity driven learning in reinforcement learning / agentic models is pretty interesting. Schmidhuber and a few other works propose curiosity as a reward in a number of reinforcement learning cases, often in case with sparse

3

1

24

@Algomancer

Adam Hibble

9 years

#akka #scala #ReactiveStreams

Tweet media one

1

32

24

@Algomancer

Adam Hibble

1 month

I would love having a grounded conversation / dialectic about the following subjects from other people who are working on them. Gonna make a twitter space or setup a lil discord call. Reply below/dm if you are interested in having any of these conversations with me. If you have

7

4

24

@Algomancer

Adam Hibble

8 years

I just published “Lets Learn About: Probabilistic Programming Part 1/?” #DeepLearning #BigData #machinelearning

0

9

21

@Algomancer

Adam Hibble

8 years

Pretty Excited @QUTCode can send people to #Hatchathon to help solve Australia's health challenges in Sydney this weekend. #StartupAus

2

8

23

@Algomancer

Adam Hibble

8 years

CTO of Amazon, casually tweeting about @QUTCode @QUTSciEng @QUT @QUTmedia

@Werner

Werner Vogels

8 years

cool: QUT Students built an Australian Census system for $500 that does not #CensusFail #AWS

Tweet media one

4

56

88

1

5

22

@Algomancer

Adam Hibble

7 months

I am considering starting a small research project, training text models on a single h100. Something small enough to train from scratch in 24 hours. I think I have a lot of architectural ideas and we are probably over partitioned for people working on raw decoder improvements.

3

2

23

@Algomancer

Adam Hibble

8 years

People of twitter, what are you working on?

19

4

20

@Algomancer

Adam Hibble

3 months

After 10 years of training models, I still get excited waking up and checking loss curves.

3

1

22

@Algomancer

Adam Hibble

7 months

@BrendanBycroft Is this on github somewhere, i'd love to throw you a github sponsorship.

1

1

21

@Algomancer

Adam Hibble

2 months

It's happening.

Tweet card media

GitHub - apple/ml-planner

Contribute to apple/ml-planner development by creating an account on GitHub.

@Algomancer

Adam Hibble

6 months

Day 1000 of telling people to decode from representation spaces.

2

1

10

0

2

22

@Algomancer

Adam Hibble

8 years

Writing about my adventures going forward with #MachineLearning #DeepLearning

1

3

20

@Algomancer

Adam Hibble

3 months

Now there are more vcreg havers.

Tweet media one

2

3

21

@Algomancer

Adam Hibble

9 years

WE WON STARTUP WEEKEND #SWHRX @callipigeons @Texonidas @AimeeLeong - #MachineLearning + #Security = Secure World

1

7

22

@Algomancer

Adam Hibble

8 years

Writing about touring Google, Twitter, Oracle and Tesla & how I grew my network to get here #StartupAus #Startups

0

5

21

@Algomancer

Adam Hibble

7 years

Enter Popgun.ai - @Techstars Music

0

8

21

@Algomancer

Adam Hibble

13 days

Maybe something interesting cooking, eval in the morning, tis tiny. Per token variational autoregressive diffusion. If it works, maybe a good formulation for dynamic compute per token.

Tweet media one

1

0

21

@Algomancer

Adam Hibble

5 months

Pretty interesting that a transformer with one shared layer works at all. This is a "single layer" transformer, stacked depth wise 22 times, so all the layers have shared weights. Part of a prelim experiment exploring memory compute trade offs.

4

1

21

@Algomancer

Adam Hibble

13 days

One of the under rated free-lunches you can get while training VAEs is to use something like GECO instead of manually setting your beta term. It will tune your beta to get a target KL, and leave you're recon doing whatever it can whilst maintaining that kl. A useful way to kinda

Tweet media one

1

0

21

@Algomancer

Adam Hibble

5 years

With Splash Pro we are on a mission to get you out of your creative rut, and all future ruts using crazy math and engineering. Here is a sneak peak at an Ableton live plugin that starts to do that using collaborative AI. RT for early beta access!

Tweet card media

Splash Pro - Intelligent Drums

Popgun is using AI to make music. This video demonstrates the Intelligent Drums feature of Splash Pro, our new Ableton Live plugin. For more information emai...

www.youtube.com

1

9

20

@Algomancer

Adam Hibble

6 months

You might think I am excited for new generation of hardware for training models, or new interested architectures like state space models or explorations into joint embedding predictive models, efficient fine tuning and things like fast feed forward networks improving inference

1

0

19

@Algomancer

Adam Hibble

8 years

So much @QUTCode Network in that photo! Brisbane startup @iRecruitOz closes $550000 seed @sbxr @pjlaurie

3

5

19

@Algomancer

Adam Hibble

8 years

Congratulations to @travelloapp for raising $1 million! You guys have an awesome team - @HarryJubb doing @QUTCode proud.

1

7

19

@Algomancer

Adam Hibble

4 months

I am funding and advising a few open source dataset collection and processing projects. Reply below for your request for data.

7

3

19

@Algomancer

Adam Hibble

4 months

`<start_of_image>` token in Gemma's vocabulary

1

3

18

@Algomancer

Adam Hibble

7 months

@jm_alexia Because if they are using certain types of normalization, the bias would be normalized away by the layer norm (etc). So it becomes an extra parameter that does nothing during training but take up memory.

1

0

18

@Algomancer

Adam Hibble

6 months

@levelsio @adamwathan @adamwathan If you end up working on anything like this, feel free to hit me up for any ml stuff that is non standard.

0

0

19

@Algomancer

Adam Hibble

7 years

Standing behind stage at @elreytheatre talking about neural nets pre @popgunlabs @techstars music pitch. I love my team. #MachineLearning

1

6

18

@Algomancer

Adam Hibble

7 months

Stop finetuning, keep pretraining. Sample pretraining data based on being from a similar distribution of your task. Mix in wider tasks and code to prevent squishing all of the interesting stuff out of your likelihoods.

1

1

18

@Algomancer

Adam Hibble

7 years

Keen to get back to Brisbane - excited to land into the new @popgunlabs offices in the valley.

0

2

19

@Algomancer

Adam Hibble

6 months

The value a single person can create, by creating datasets for training models is quite under appreciated.

0

0

18

@Algomancer

Adam Hibble

5 months

Hacked?

7

1

18

@Algomancer

Adam Hibble

6 months

Just had a fun vision of a potential model that could emerge. This is my favorite way the internet of ai could work. I don't know if it's likely but it sounds based. Everyone exposes some access to there weights or logits, everyone uses a massive mixture of experts, like DNS for

10

0

17

@Algomancer

Adam Hibble

4 months

Things we fixed A) 3 cases of data issues (2 of which were normalization related) B) 1 set of exploding gradients from a gan distillation3 C) 3 Paper interpretation issues D) 1 masking and indexing issue E) 1 gpu memory issue related to deepspeed and fsdp

0

0

17

@Algomancer

Adam Hibble

6 months

A stupid thing that works for fine tuning task specific models. Write a bunch of pre-prompts that you use in training. Run all of these over your data. Calculate the loss (masking out the pre-prompt itself) Train a decision transformer where you use the loss as your reward

4

0

17

@Algomancer

Adam Hibble

5 months

600k h100s is a wild amount of compute.

1

0

17

@Algomancer

Adam Hibble

8 years

Every day waking up to build the coolest shit on the planet. Loving it. #StartupAus @haitchlabs @kendricktrh @Byron__Mejia @CallumHays

0

3

16

@Algomancer

Adam Hibble

7 months

New mesh model draft in training, targeting 3d printing. So, it's a hardware/mechanical distribution of data. It has a bunch of potential improvements over meshgpt (released last week), and should scale to large meshes. Targeting 32k faces as a first milestone (meshgpt

5

1

17

@Algomancer

Adam Hibble

7 months

@YannickScholich Candidly, I want to open source it under a super liberal license. Just chuck it up as mit/apache. But, I really need to feel confident that I can support the team working on it long term. So, there is still stuff up in the air. I am avoiding raising venture capital if I don't

3

0

16

@Algomancer

Adam Hibble

6 years

Mawson is hiring 10 more people to work in Deep Learning AI. Math, Physics, CompSci or Electrical Engineering grads preferred. They are based in Fortitude Valley, Brisbane. Media and Entertainment industry focus. Send resume to info @mawson .io

0

6

17

@Algomancer

Adam Hibble

7 months

The start of Part 1.

@Algomancer

Adam Hibble

7 months

we are so b̶a̶c̶k̶ b-rep. meet cadmancer, the first in a series of models we will be releasing you can export the designs it creates, optionally to your favorite cad/cam software (fusion 360 etc), and manufacture it. very early work - still lots more to do before prime time,

79

106

940

2

1

16

@Algomancer

Adam Hibble

8 years

1500+ signups at #FintechSWB since 10 last night.

1

1

15

@Algomancer

Adam Hibble

6 months

2024 is going to be really, really fun.

2

0

15

@Algomancer

Adam Hibble

7 years

"AI will power the music value chain of the near-future, from production to search, to delivery and monetization."

0

5

16

@Algomancer

Adam Hibble

4 months

One thing for sure, from here on out ai gets weirder.

3

0

16

@Algomancer

Adam Hibble

2 months

Got more compute.

@Algomancer

Adam Hibble

7 months

Need more compute.

0

0

2

2

0

16

@Algomancer

Adam Hibble

6 months

Well, this is popping off. I am exploring ideas like this and more. I am a big believer that product development and many types of ai capability research have converged and you should no longer treat them as separate things. And in fact much of the best research is in the search

1

0

15

@Algomancer

Adam Hibble

8 years

"Scala on the Brain" @scala_lang @typesafe @odersky by @jezekab - For my Series Zero to #DeepLearning with #Scala

Tweet media one

1

4

15

@Algomancer

Adam Hibble

2 months

Is there a Tiny-Dolma or tiny-fineweb with just the best ~10b tokens high quality base model data, high variety? Not distillations or instructions.

4

3

15

@Algomancer

Adam Hibble

7 years

Yo @nvidia let @popgunlabs benchmark your new GTX 1080 Ti 's pretties.

1

7

15

@Algomancer

Adam Hibble

4 months

The fastest way to close the gap between open and closed models is a dedicated open compute cluster of ~10k H100s that the best projects can timeshare.

6

0

15

@Algomancer

Adam Hibble

7 years

Iterating on code listening to the music generated by that code. Love the dynamics. #DeepLearning @popgunlabs

1

1

13

@Algomancer

Adam Hibble

2 months

🤔

Tweet media one

0

0

15

@Algomancer

Adam Hibble

7 months

@crit_architect @autodesk You know what is better than millions. Free. Just gotta figure out how.

2

0

14

@Algomancer

Adam Hibble

7 years

Only problem with a browser based cloud IDE is that you lose your development environment in stack overflow tabs. #Programming

0

0

13

@Algomancer

Adam Hibble

22 days

Hi New Followers, Glad you liked the jepa thought vector stuff. Don't expect research papers or that level of rigor from me, I am not a deep learning researcher. I am a capabilities engineer, I have just been making GPUs go brrr for a long time.

1

0

15

@Algomancer

Adam Hibble

8 years

Won the Special excellence award this year from @QUT Obligatory Rap Squat & Upside down award @HarryJubb @trjstewart

Tweet media one

Tweet media two

1

1

14