Alignment Lab AI @alignment_lab Twitter profile

Pinned Tweet

Alignment Lab AI

@alignment_lab

33 minutes

2

11

15

Last Seen Profiles

@Charles19557713

@harwichU23

@BrendanThorpe4

@Aminato56803529

@jandakembangstw

@VanCity6042

@jordynfooott13

@RoundBearAD

@goldrite

@VeraLopez_O

@akedoyurik82113

@lena_popina

@wonboolens

@alqadi_0001

@SaydaHope

@Predators_Smite

@MiyaresKis74684

@2025Larson

@reecep_10

@GoAbiAryan

@LindseySan96677

@PMovies155

@uvggt

@GeorgetteH19463

@sorairo__00

@PaulCTayla

@serkanc54524134

@BoolaDude

@Hasna_313

@Mount_MLAX

@AustriainRF

@iAMshreyverma

@KishinShinobi

@YukikoOdou62143

@kellymoran

@bn_ftwmh

Alignment Lab AI

@alignment_lab

7 months

Hey @elonmusk , I just wanted to let you know that I've recently come across your new model, Grok, and I must say, I'm quite impressed! With 33 billion parameters and all, you've really outdone yourself. But, I've got some news for you - I've outperformed Grok with my humble 7

121

275

3K

Alignment Lab AI

@alignment_lab

4 months

well, theres one but i guess *just serving inference* is really only one part of the industry isnt it? should we talk about data, or training? choices choices...

Alignment Lab AI

@alignment_lab

4 months

DO you want to serve models at scale? do you want to do it locally and not be dependent upon a company for good throughput? do you just generally like really cool things? well @AlpinDale does too! - so he's built out Aphrodite as the official backend engine for

42

23

105

1K

666

1K

Alignment Lab AI

@alignment_lab

7 months

we at Alignment Lab AI () are happy to announce another SOTA model! a little under a year since @OpenAI released ChatGpt and just a few weeks from its birthday! the model receives a near fatal blow! @imonenext (Guan Wang & Sijie Cheng) have been

30

150

809

Alignment Lab AI

@alignment_lab

11 months

We released Open Orca, the project and dataset that brings GPT-4 reasoning to open models yesterday. This dataset aims to reproduce the stunning results from Microsoft Research's Orca paper showing a small model besting ChatGPT in hard reasoning. 🧵 1..

Open-Orca/OpenOrca · Datasets at Hugging Face

huggingface.co

7

142

684

Alignment Lab AI

@alignment_lab

8 months

The @MistralAI winds have attracted superorcas! Today, we are releasing Mistral-7B-OpenOrca. This is the first 7B model to score better overall than all other models below 30B.🏅 We've achieved 98% of Llama2-70B-chat's performance!😮 🧵 1..

16

126

641

Alignment Lab AI

@alignment_lab

15 days

Hello everyone! Alignment Lab AI , is pleased to introduce our latest research efforts with: Buzz! a highly curated pretraining scale assistant dataset, unifying RL and SFT, developed @HIVEDigitalTech The Buzz model, Dataset,

H-D-T/Buzz · Datasets at Hugging Face

huggingface.co

11

95

339

Alignment Lab AI

@alignment_lab

5 months

Merry Christmas! A last second gift from the edge of Open Source AI here bringing you yet another SOTA MODEL! introducing the new xDAN models!, These models, the first of their kind to be publicly released are trained on a unique combination of

11

65

270

Alignment Lab AI

@alignment_lab

7 months

@JayantParashar6 @elonmusk it is on huggingface! we only released it yesterday, the leaderboards take around a week or two to update with the new models we have topped them quite a few times in the past though if thats an important element to validate the quality of the model for ya

4

5

278

Alignment Lab AI

@alignment_lab

4 months

so weve discussed how has been in the weeds, and how the open source has proliferated this last year on the back of fantastic datasets, state of the art efficiency in serving and training, and powerful data curation and generation tools from the many

40

34

208

Alignment Lab AI

@alignment_lab

6 months

hey guys, @OpenAI seems to be down! thankfully we have you covered, this is openchat 3.5 hosted publically for the open source heres the training framework and links to the model+ more information just a quick recap for

12

53

231

Alignment Lab AI

@alignment_lab

10 months

Today we are releasing a LlongOrca! This 7B model is our first long context release, able to handle 16k tokens at once! We've done this while achieving >99% the performance of the best 7B models available today (which are limited to 4k tokens). 🧵 1..

Open-Orca/LlongOrca-7B-16k · Hugging Face

huggingface.co

4

38

187

Alignment Lab AI

@alignment_lab

5 months

in case you guys missed the previous announcement! a 7b that outperforms gpt4 (from march) at humaneval!, a ton of utility! Check out the new ! the website so you guys can more easily follow up, join

12

24

165

Alignment Lab AI

@alignment_lab

10 months

It's preview day again! We are on top of all the leaderboards again with our new 13B! 🎉 This time we have Llama2🦙 and we're outpacing the original Orca for Microsoft, plus we did it with <10% of the compute budget!⚡ Even bigger news soon too! 🧵 1..

Open-Orca/OpenOrcaxOpenChat-Preview2-13B · Hugging Face

huggingface.co

7

34

164

Alignment Lab AI

@alignment_lab

11 months

Our first OpenOrca Model Preview is here! We beat current state of the art for public model releases in this class, for <$200 total training budget. 🧵 1..

Open-Orca/OpenOrca-Preview1-13B · Hugging Face

huggingface.co

5

31

163

Alignment Lab AI

@alignment_lab

8 months

Open Orca is a really good dataset ive got a better one ive been sitting on for months constantly tooling and tweaking problem is that its 2x the size of orca currently (and likely ends a little smaller as i continue to filter it out) Anyone feel like pitching in compute in

3

19

130

Alignment Lab AI

@alignment_lab

5 months

cant overstate how much you can get done with just some 3090s anton is a fat example of this, for the cost of 1 a6000 ada you can buy 8x 3090s and a nice rack to slap them onto and pretrain models in a reasonable timeframe, in terms of vram the rack would be in excess of

anton

@abacaj

5 months

Can confirm

18

15

354

9

11

119

Alignment Lab AI

@alignment_lab

4 months

DO you want to serve models at scale? do you want to do it locally and not be dependent upon a company for good throughput? do you just generally like really cool things? well @AlpinDale does too! - so he's built out Aphrodite as the official backend engine for

42

23

105

Alignment Lab AI

@alignment_lab

5 months

@huggingface should totally integrate maximal update parameterization from microsoft and openai into the transformers library @TheGregYang has a repo with a partial implementation of the integration already (just a bit deprecated) 👀 its about the

GitHub - microsoft/mutransformers: some common Huggingface transformers in maximal update paramet...

some common Huggingface transformers in maximal update parametrization (µP) - microsoft/mutransformers

github.com

3

12

105

Alignment Lab AI

@alignment_lab

9 months

What do you get when you cross an Orca with a Platypus? The best 13B model in the world apparently. We're on top of the charts again with today's release🏆 New model! This time thanks to our new partnership with the amazing Platypus team! 🧵 1..

6

13

95

Alignment Lab AI

@alignment_lab

13 days

@somewheresy i think the openai blogs continuously validating everyones fears about regulatory capture are one major reason

1

0

98

Alignment Lab AI

@alignment_lab

3 months

Today, we announce our partnership with @arbius_ai . Together, we will give you access to all the world's information and accelerate the pace of progress beyond your wildest dreams. The future of the world is to be made by machines. We are the vanguard of this revolution, and

5

15

84

Alignment Lab AI

@alignment_lab

5 months

@runaway_vol I'm 100% down for a fake email job Please hook me up. Only boring people get bored with free time!

1

85

Alignment Lab AI

@alignment_lab

2 months

how long before we acknowledge that releasing a model under a specific name, then replacing it with a worse one in a week without telling anyone is literally false advertising?

8

2

82

Alignment Lab AI

@alignment_lab

2 months

who wants to make an AI music tool? i think this is a huge vacuum waiting to be filled

33

1

78

Alignment Lab AI

@alignment_lab

6 months

thanks to @morosnet for the grant and supporting the oss! Im not really a crypto-guy but the funds are appreciated 🙏

12

24

64

Alignment Lab AI

@alignment_lab

8 months

🤗 @huggingface 🤗has a space for you to try our model live on fast GPUs in the browser right now! Thanks @ClementDelangue @_philschmid @mervenoyann @RichardSocher @_lewtun @Nils_Reimers @clefourrier and the rest of the HF crew! 🧵 4..

Mistral-7B-OpenOrca - a Hugging Face Space by Open-Orca

huggingface.co

2

14

69

Alignment Lab AI

@alignment_lab

7 months

just read it, seriously. if you actually have a real concern about AI and you are willing to admit that propaganda is a real thing - read this through.

Yann LeCun

@ylecun

7 months

@tegmark @RishiSunak @vonderleyen Altman, Hassabis, and Amodei are the ones doing massive corporate lobbying at the moment. They are the ones who are attempting to perform a regulatory capture of the AI industry. You, Geoff, and Yoshua are giving ammunition to those who are lobbying for a ban on open AI R&D. If

317

1K

6K

4

11

66

Alignment Lab AI

@alignment_lab

6 months

{ "system_update": { "teaser": "2023-11-26", "audience": "users", "message": "You guys are going to really love this one", "features": { "structure": { "status": "controllable" }, "default": { "mode": "function calling" },

4

6

67

Alignment Lab AI

@alignment_lab

4 months

thanks a ton @StabilityAI raising the absolute bar it has rotary embedding kernels digits in the tokenizer are split into individual ones bias terms removed and it retains the neox throughput optimizations i am going to push this thing to the absolute limits local gpt4?

AK

@_akhaliq

4 months

Stability AI presents Stable LM 2 1.6B demo: model base: instruction-tuned: introducing first language model from the new Stable LM 2 series: the 1.6 billion parameter base model and an instruction-tuned

7

81

442

0

6

65

Alignment Lab AI

@alignment_lab

4 months

@airkatakana you're forgetting musicians really anything entertainment-industry-related tbh it makes sense in that respect

3

0

60

Alignment Lab AI

@alignment_lab

5 months

@prmshra because the diffusion model doesnt know what text looks like very well, and it doesnt really interact directly with the text, the text is embedded then encoded into a representation that the diffusion model uses to guide the image generation process. honestly its cool it can

3

4

61

Alignment Lab AI

@alignment_lab

7 months

@JayantParashar6 @elonmusk

openchat/openchat_3.5 · Hugging Face

huggingface.co

1

2

54

Alignment Lab AI

@alignment_lab

7 months

@EuPatrunjel @elonmusk i think its more important than ever due to this fact do we really want this to be more secret and less understood, and for the common person to have less access to it?

2

0

51

Alignment Lab AI

@alignment_lab

5 months

check it out a contamination detector not sure how accurate it is, but certainly theres an interesting result if you sort the benches by descending to see which models most likely trained on bench data 👀

Llm Contamination Detector - a Hugging Face Space by Yeyito

huggingface.co

3

10

50

Alignment Lab AI

@alignment_lab

5 months

is anyone interested in working with us to adress the benchmarking issue? i have a design and pipeline down or how we can begin to make our benchmarks more objective and avoid the problem of fitting to the benchmarks themselves without having to have many secret benchmark sets

10

8

50

Alignment Lab AI

@alignment_lab

7 months

@Clifford98506 @elonmusk I have! Many times!

3

0

49

Alignment Lab AI

@alignment_lab

6 months

A real Mistral of Experts

Mistral AI

@MistralAI

6 months

magnet:?xt=urn:btih:5546272da9065eddeb6fcd7ffddeef5b75be79a7&dn=mixtral-8x7b-32kseqlen&tr=udp%3A%2F%%3A6969%2Fannounce&tr=http%3A%2F%%3A80%2Fannounce RELEASE a6bbd9affe0c2725c1b7410d66833e24

545

2K

10K

3

2

48

Alignment Lab AI

@alignment_lab

4 months

HEY tomorrow, im going to make some announcements stay tuned if you like FREE things especially if those free things are atet of the art :) stay tuned.

4

2

47

Alignment Lab AI

@alignment_lab

5 months

absolutely agree @huggingface 's dataset and transformers library alone are responsible for so much progress and access

Bram

@BramVanroy

5 months

My journey with #opensource development started about six years ago but was accelerated by the incredible work by @Thom_Wolf @LysandreJik @julien_c and others at @huggingface in 2018 with pytorch_pretrained_bert (now better known as #transformers ). In a sense they mentored me! 🧵

3

59

1

3

46

Alignment Lab AI

@alignment_lab

1 month

Avg generation throughput: 1378.4 tokens/s, Running: 200 reqs, a40 and feels nice.

GitHub - PygmalionAI/aphrodite-engine: PygmalionAI's large-scale inference engine

PygmalionAI's large-scale inference engine. Contribute to PygmalionAI/aphrodite-engine development by creating an account on GitHub.

github.com

3

7

45

Alignment Lab AI

@alignment_lab

2 months

@mealreplacer It means exactly that, currently, reasoning isn't a thing which ai does, it's a thing you have to do for it, therefore it cannot simply do any job without a ton of babysitting and custom tooling

8

0

44

Alignment Lab AI

@alignment_lab

4 months

OK so we talked about inference, let's talk about how you could be doing your training runs for a good 10x or 20x cheaper than anyone else, yeah? its not just that but some wild new optimizations today, check it out for those of you who don't know though: Axolotl is a tool

Wing Lian (caseus)

@winglian

4 months

Do you have a pre-Ampere GPU and unable to use FlashAttention 2 w/ multipack for efficient fine-tuning? @axolotl_ai now supports multipack using 4d-attention masks to prevent cross-attention contamination using native PyTorch or even PyTorch SDPA.

6

12

99

3

11

45

Alignment Lab AI

@alignment_lab

6 months

looks like its time to make another dataset!

Yam Peleg

@Yampeleg

6 months

You thought that you can go to sleep now?? Orca 2 Just dropped. Paper: Results: Orca 2 13B beats LLaMA-Chat-70B TL;DR: Training smaller model to reason by using multiple techniques: step-by-step, recall then generate, recall-reason-generate, direct

49

347

2K

3

4

44

Alignment Lab AI

@alignment_lab

16 days

so, do you guys like function calling? structured outputs in general? how about coding? what about conversational aptitude? all of the above? what about *lots* of all of the above? do you guys hate formatting datasets? trying to decide mixes and ratios? i like all of those

10

3

43

Alignment Lab AI

@alignment_lab

2 months

And another has fallen. It's getting pretty sparse up there in the GPU districts

Anthropic

@AnthropicAI

2 months

We’re excited to team up with @Accenture and @awscloud to help organizations adopt and scale customized AI solutions.

33

61

596

4

6

41

Alignment Lab AI

@alignment_lab

8 months

who has cool ideas about ai, and no resources or technical expertise to make them happen? reply here and maybe we can get something going!

18

1

39

Alignment Lab AI

@alignment_lab

7 months

@Yampeleg @elonmusk always!

0

42

Alignment Lab AI

@alignment_lab

3 months

repeat after me " a model is not a moat" "raw inference isnt a business" "solving problems is worth money"

3

41

Alignment Lab AI

@alignment_lab

7 months

this is a project that we believe to be very important, were bringing our usual standard of quality to bear on this, and everyone involved has already gotten quite a lot done - don't think this is not on

Open Empathic Launch & Tutorial

https://laion.ai/blog/open-empathic/https://dct.openempathic.ai/We are thrilled to present Open Empathic, a pioneering open-source project initiated by our n...

www.youtube.com

3

7

39

Alignment Lab AI

@alignment_lab

2 months

@kosenjuu might be though im not really sure there are any competitors at that level, the frontier model forum kind of dashed my hopes for that

0

41

Alignment Lab AI

@alignment_lab

5 months

is anyone here an absolute behemoth with structured data, and the datasets library who wants to contribute to something truly outrageous and legendary? lets boost up oss ai by another 200% yeah?

12

5

40

Alignment Lab AI

@alignment_lab

3 months

hello everyone, who would like to learn how to train an llm to the current state of the art for the price of pocket change? just leave your name and email on the related field, and any info youd like in the note! starting weekly free tutorials on

Free AI training tutorials!

feel free to sign up with your name, email, and additional info! thanks!

docs.google.com

12

6

40

Alignment Lab AI

@alignment_lab

7 months

@Dshoopy0 @elonmusk this was written by a model, i dont truly expect any achievement from this beyond just generally letting people see how strong the open source is getting :)

1

0

40

Alignment Lab AI

@alignment_lab

10 months

✨Prompt Engineering has posted a good video demo of OpenOrca-Preview1-13B! 💪This is our just-released model with chart-breaking performance for 13B @Teknium1 @winglian

Open-Orca: The FIRST Version is OUT - Audio Fixed :)

In this video we cover the initial release of OpenOrca -13B parameter model which is trying to replicate the Orca research paper dataset by Microsoft. The m...

www.youtube.com

0

4

40

Alignment Lab AI

@alignment_lab

4 months

the implications are massive.

Rohan Paul

@rohanpaul_ai

4 months

🧠 Run 70B LLM Inference on a Single 4GB GPU - with airllm and layered inference 🔥 layer-wise inference is essentially the "divide and conquer" approach 📌 And this is without using quantization, distillation, pruning or other model compression techniques 📌 The reason large

24

100

540

5

8

39

Alignment Lab AI

@alignment_lab

8 months

@NPCollapse yes surely nothing could be safer than incentivising companies to be more secretive about their development i mean why would we want everyone to be aware of what currently exists? this is literally the only possible way AI turns out worse for everyone. youre literally

2

39

Alignment Lab AI

@alignment_lab

5 months

@RichardMCNgo i think its extremely reasonable to avoid ascribing agency until it is abundantly clear that it is there, occam's razor on this one, there's no sense in adding more complexity to an already highly complex system until you have to.

1

0

39

Alignment Lab AI

@alignment_lab

4 months

I think people in general are too ready to feel like asking questions about ai makes them look stupid Don't! This is hard and confusing and opaque and everyone who can answer a question is dying to. Everyone even in the field is narrow and there's no training wheels for any of

6

4

38

Alignment Lab AI

@alignment_lab

2 months

ok guys i think a new set of announcements is warranted its time you all see what weve been doing this last year

4

1

38

Alignment Lab AI

@alignment_lab

11 months

Update: Our Playground Space is up on HuggingFace. You can go try our state of the art new 13B model live in your browser now! This is running GGML quantized, so will not represent the full quality of the model, but should give a sense at least.

2

12

38

Alignment Lab AI

@alignment_lab

8 months

We're so close to beating a model 10X larger now!🏋🏼 🧵 3..

1

2

37

Alignment Lab AI

@alignment_lab

6 months

I smell new datasets

Bindu Reddy

@bindureddy

6 months

Yay!!! Google is finally in the arena. They just announced Gemini, and it has some impressive benchmark scores 👏👏 Gemini beats GPT4 in several benchmarks except HellaSwag. Interestingly, top open-source models score around 87-88 on HellaSwag. The 4-pt MMLU beat is

32

93

460

4

1

34

Alignment Lab AI

@alignment_lab

6 months

Ai car full of ai devs on the way to an ai event @Ollama_ai @altryne @thursdai_pod @Shroominic @ldjconfirmed

5

3

36

Alignment Lab AI

@alignment_lab

13 days

hey guys, i realize buzz is huge and you may not want to use the entire thing every time, so i shuffled it and split it into 10 parts, so you can get a representative subsample and not have to train on the entire thing let me know if you run into any problems or notice any

3

37

Alignment Lab AI

@alignment_lab

8 months

Thanks to @GuillaumeLample @arthurmensch @tlacroix6 for showing the way with the amazing open release of Mistral-7B!👑 So great to have this much capability ready for home GPUs. Also thanks to @winglian @Teknium1 @TheBlokeAI and all Alignment Lab AI friends! 👊🤟 🧵 2..

1

35

Alignment Lab AI

@alignment_lab

11 months

Even more news from alignment lab ai! We're thrilled to announce the release of the Open Orca dataset! This rich collection of unaugmented and augmented FLAN data aligns with the distributions outlined in the ORCA paper. It's been instrumental in generating high-performing model

0

9

34

Alignment Lab AI

@alignment_lab

2 months

this is about control, and complacency there is no second chance if this technology is owned and regulated into the pocket of the few, remember people, the systems wont proliferate this stuff for you, they are not designed to what is happening is that you are being given a

LAION

@laion_ai

2 months

23

27

137

5

8

33

Alignment Lab AI

@alignment_lab

2 months

@cloneofsimo @ptrblck_de people have no idea how much of the infrastructure right now is like 3 guys

0

34

Alignment Lab AI

@alignment_lab

6 months

hi, no they didnt :) +/- literally hundreds of other examples

HydraLM (HydraLM)

huggingface.co

Shubham Saboo

@Saboo_Shubham_

6 months

Mistral AI releases first ever OpenSource MoE model Fun facts: - Released as 87 GB torrent - Seems to be a scaled down version of GPT-4 - Released on X, no press release and refuse to elaborate Google left the AI community in awe with the over rehearsed demo video that's now

35

170

837

2

34

Alignment Lab AI

@alignment_lab

3 months

💖 I'm sponsoring lucidrains because… of course i am

lucidrains - Overview

Working with Attention. It's all we need. lucidrains has 288 repositories available. Follow their code on GitHub.

github.com

1

3

33

Alignment Lab AI

@alignment_lab

15 days

look what did i tell you about the chips? buy your 3090s while you can, models need less hardware by the minute, cost goes up by the second

MatthewBerman

@MatthewBerman

15 days

OpenAI Wants to TRACK GPUs?! They Went Too Far With This… Breakdown below 🎥👇 Here's the quote: "Extending cryptographic protection to the hardware layer has the potential to achieve the following properties...GPUs can be cryptographically attested for authenticity and

73

66

341

3

9

33

Alignment Lab AI

@alignment_lab

8 months

Thanks @clem for the kind shoutout! We're not stopping there either! Also trying to do our part to encourage others to share beautiful data. 🤗🔗‍💥

clem 🤗

@ClementDelangue

8 months

I wish everyone in the field would spend more time and efforts on sharing great datasets. Open-orca by @alignment_lab has enabled the creation of more than 330 models. This is the kind of impact that you can have and that should be celebrated more!

4

46

303

0

2

33

Alignment Lab AI

@alignment_lab

4 months

absolutely gigantic people have no idea what the impact is going to be like once ai can ode so well these things are trivial this is a major building block and a phenominal contribution to the oss

Philipp Schmid

@_philschmid

4 months

Transform Screenshots into HTML Code! The @huggingface multimodal team released Websight, a dataset of 823,000 pairs of website screenshots and HTML/CSS code. 🤯 Websight is designed to train Vision Language Models (VLMs) to convert images into code. The dataset was generated

8

120

520

2

3

32

Alignment Lab AI

@alignment_lab

7 months

spiciest release so far incoming in 12 hours 👀 you probably dont want to miss this one be ready for a major win for OSS AI, ill be running a twitter space after with @far__el , maybe @nisten and @ldjconfirmed to talk about some upcoming spoilers maybe?? the space will be

1

3

32

Alignment Lab AI

@alignment_lab

11 days

ok, who wants a free open source rag database that graphs literally everything you do on your computer and dumps it into notion? if you do, share cool tools that might be cool to integrate into it here - im just about done building the thing so last call for "neat stuff"

7

1

32

Alignment Lab AI

@alignment_lab

6 months

yo who needs custom function calling for their ai use case? if its the only thing stopping you from swapping over to a local model hit me up and well do something about it :D

5

2

28

Alignment Lab AI

@alignment_lab

5 months

ok, sorry guys, x-spaces has decided that i dont get to host one however! announcement is still incoming! 👀 12:00 PST be ready

3

1

31

Alignment Lab AI

@alignment_lab

4 months

in ~7 hours ~1 announcement stay tuned

1

2

31

Alignment Lab AI

@alignment_lab

5 months

Man, this is grim. The 3090 is like what, four years old? It's twice the price it was a year ago. The 4090s are starting to crest into $3-4K and are over a year old at least. The A100s are up, H100s are upwards of $10 - 15$ an hour on Amazon each. Every time something crazy

6

1

30

Alignment Lab AI

@alignment_lab

1 month

this is the most tragic thing ive ever seen. its like they did it on purpose. meta, you bastards.

Omar Sanseviero

@osanseviero

1 month

Dear Open Source Community, I'm about to have a trip of 19 hours with no internet access (going from Europe to South America). Please hold off of releasing new SOTA models for a bit. I'll be back soon. Appreciated, Hacker Llama🤗

29

19

403

6

1

29

Alignment Lab AI

@alignment_lab

3 months

tell me you trained on gpt4 outputs without telling me you trained on gpt4 outputs

2

0

30

Alignment Lab AI

@alignment_lab

2 months

lets make it happen what do we need?

Junyang Lin

@JustinLin610

2 months

Me and @huybery are discussing about reproducing Devin. Come and join us and see if we can make something great together!

50

250

2K

6

1

29

Alignment Lab AI

@alignment_lab

5 months

Heck yeah, it is! Mergekit is a library and tool designed to reduce the barrier to custom models and works for many as an accessible development process for LLMs that avoids the need for large volumes of GPU compute. It's not yet as fine-grained in

GitHub - arcee-ai/mergekit: Tools for merging pretrained large language models.

Tools for merging pretrained large language models. - arcee-ai/mergekit

github.com

Prateek Yadav

@prateeky2806

5 months

It is amazing to see that model merging is finally getting more traction and people are extracting value out of it. Moreover, there are resources that make it easier to quickly try out these methods like the mergekit. Everyone who is building any model should try merging as it

1

6

40

2

3

30

Alignment Lab AI

@alignment_lab

10 months

We've made a 7B model which is quite good at English for the first time! We believe this is the only Llama2 7B currently scoring over 60% on SAT English evaluation, more than a 2X improvement over the base Llama2 7B! 🧵 2..

2

3

30

Alignment Lab AI

@alignment_lab

5 months

wonder how long before everyone else realizes they can be used for more than images 👀 merry christmas guys

Imitating Human Behaviour with Diffusion Models

Diffusion models have emerged as powerful generative models in the text-to-image domain. This paper studies their application as observation-to-action models for imitating human behaviour in...

arxiv.org

AK

@_akhaliq

6 months

Analyzing and Improving the Training Dynamics of Diffusion Models paper page: Diffusion models currently dominate the field of data-driven image synthesis with their unparalleled scaling to large datasets. In this paper, we identify and rectify several

2

108

533

2

6

29

Alignment Lab AI

@alignment_lab

2 months

Haha Nice to see everyone's finally just buying their own GPUs You can do a hell of a lot with local compute GPU poor is all you need

Yam Peleg

@Yampeleg

2 months

Men will literally buy 3TB RAM instead of going to therapy

256

311

5K

3

0

28

Alignment Lab AI

@alignment_lab

14 days

Hey! heads up! just wanted to make another announcement today, weve been working with a company called weve been working with @dekube_official to help make AI technology more accessible and beneficial for everyone. Dekube is working with us to optimize their stack for

2

11

29

Alignment Lab AI

@alignment_lab

7 months

@soumithchintala we cant get human feedback, its open source, so we dont have the means and incentives to pay people to do such boring awful work. human feedback data is the most rare and valuable data in the oss, all of us are slavering over it and many, like myself, would go to great lengths

6

0

28

Alignment Lab AI

@alignment_lab

3 months

@sama everything 'creative' is a remix of things that happened in the past, plus epsilon and times the quality of the feedback loop and the number of iterations. people think they should maximize epsilon but the trick is to maximize the other two.

1

4

29

Alignment Lab AI

@alignment_lab

2 months

its so outrageously cathartic seeing this cleanly graphed after a year of shouting into the void that this would happen

Chief AI Officer

@chiefaioffice

2 months

@sarahdingwang 4. Open source is booming. However, while cost factored into open source appeal, it ranked below control and customization as key selection criteria.

2

9

105

2

1

27

Alignment Lab AI

@alignment_lab

8 months

theres a lot of people doing open source work, who keep it a secret, this makes no sense to me, if youre working on something thats gonna be publically available you should tell everyone, because odds are four people are doing it too and together they all get done 16x as fast!

4

1

27

Alignment Lab AI

@alignment_lab

7 months

look mom were on tv!

lmsys.org

@lmsysorg

7 months

Congrats OpenChat-3.5 for matching GPT-3.5 performance across major benchmarks! It's now live at . Come challenge it with your toughest prompts and see how it really compares with ChatGPT.

7

28

211

1

26

Alignment Lab AI

@alignment_lab

10 months

Orca is done and dusted. 💨We're not stopping here though. 🧵 3..

1

5

26

Alignment Lab AI

@alignment_lab

7 months

@tdinh_me Open orca and openchat 3.5 each cost less than 500$ :D

0

1

27

Alignment Lab AI

@alignment_lab

5 months

hey guys chatgpt appears to be down again! got your backup! major announcements incoming 👀

1

3

27

Alignment Lab AI

@alignment_lab

4 months

Hey guys! If you would like to try this technique today, it's been baked into the openchat functionality for well over a couple months! Do read this phenomenal paper and feel free to try and replicate the results! It's an interesting line of research we've been following

Jason Weston

@jaseweston

4 months

🚨New paper!🚨 Self-Rewarding LMs - LM itself provides its own rewards on own generations via LLM-as-a-Judge during Iterative DPO - Reward modeling ability improves during training rather than staying fixed ...opens the door to superhuman feedback? 🧵(1/5)

5

223

1K

0

4

26

Alignment Lab AI

@alignment_lab

3 months

itd be cool if there were some part of that which wasnt 100% achievable today with a 7b, basic python, and some gpus to host a model on?

Esra Öz

@fesraoz

3 months

📌 "GPT-5 will have better ability to use your own data. The ability to know about you, your email, your calendar, how you like appointments booked, connected to other external data sources." 😬 Sam Altman talks to Bill Gates on key aspects of GPT-5 👇

16

43

234

6

1

24

Alignment Lab AI

@alignment_lab

15 days

Buzz is a large pretraining scale instruction-tuning dataset, built out of the most performant instruction datasets, including many of the chosen examples from popular preference tuning datasets, all of that, reformatted, deduped, and cleaned

H-D-T/Buzz · Datasets at Hugging Face

huggingface.co

2

6

26

Alignment Lab AI

@alignment_lab

7 months

huge drop of major tools for free quality in your models! awesome

Lewis Tunstall

@_lewtun

7 months

🪁 Today we're releasing the code to train your very own Zephyr models! We've worked hard to make this as accessible as possible, so you can run: 🏋️‍♂️ Full fine-tuning with @MSFTDeepSpeed ZeRO-3 on A100s 🐭 LoRA or QLoRA fine-tuning on consumer GPUs Code:

7

175

736

1

25

Alignment Lab AI

@alignment_lab

2 months

More Agents Is All You Need

We find that, simply via a sampling-and-voting method, the performance of large language models (LLMs) scales with the number of agents instantiated. Also, this method is orthogonal to existing...

arxiv.org

Carlos E. Perez

@IntuitMachine

2 months

Agentic AI is the next wave!

27

175

1K

0

3

25