This street in SF had multiple homeless camps on the sidewalk and they just filled it with plants.
Genuine question, where did they send all the homeless people to? This happened almost overnight.
A new European AI regulation proposal would make any "American opensource developer" that hosts an "unlicensed LLMs" on GitHub & available in Europe liable for "โฌ20,000,000 or 4% of worldwide revenue"
This is a leap in image/pixel segmentation.
Meta AI just released SAM (Segment Anything Model). One of the most interesting things is well understating of objects ("objectification" of parts).
The model is released open source under an Apache 2.0 license, and it's only 2.4Gb.
Today we're releasing the Segment Anything Model (SAM) โ a step toward the first foundation model for image segmentation.
SAM is capable of one-click segmentation of any object from any photo or video + zero-shot transfer to other segmentation tasks โก๏ธ
โจNEW LAUNCH! LLaMA2 chat API & open-source playground๐ซ:
We're releasing tools that make it easy to test
@meta
's latest LLM & add it to your own app with
@replicatehq
.
Playground:
Live chat API here:
Repos & instructions below:
Our GPT-4/LLM hackathon was packed with over 170 top engineers, AI researchers & founders.
The talent of this group is outstanding.
Engineers, founders & researchers from Google Brain, Meta AI, Stability AI, Open AI & many AI startups.
โจ Introducing
A semantic search engine for your YouTube/Podcast cont.
Search for a phrase/idea/question & get the exact timestamp & videos where this is mentioned.
Lightning fast.
@theallinpod
@myfirstmilpod
@heydannymiranda
โจExcited to announce
@ArXivGPT
:
Daily summary tweets of new AI papers published daily to ; summarized by GPT-4.
The amount of AI papers has significantly increased & it's hard to keep up.
It aims to provide a quick glance at all AI daily papers.
Just today in LLM world:
-
@OpenAI
's GPT-4 dropped.
-
@GoogleAI
PaLM opened their API.
-
@AnthropicAI
's Claude announced their API too.
LLMs are having a time.
Links on thread to waitlists:
โจPersonal newsโจ
I'm thrilled to be joining
@a16z
(AI & Infra) as AI Research Partner to continue researching AI, its development, AI tools & how is helping us be more creative & productive!
With almost a decade of being in AI, there couldn't be a better time to join this team
We added Meta AI's Segment Anything Model (SAM) for object recognition to a robot "service dog" this weekend for
@trychroma
's hackathon.
It can interact with GPT-4 to receive commands (via a chatGPT plugin).
For this hackathon, we wanted to do something more than just the LLMs
It feels like SF/Bay area is on ๐ฅ right now. The number of AI events, AI hackathons, developers meetups, and people building is something I've never experienced.
Every week is AI Hack week in SF.
One of the most under-discussed things I've seen is that the newer text models are fine-tuned versions of a code model, not a language model.
They seem to learn logic much better this way before they learn language.
For instance, GPT3.5 is a second fined tuned iteration of
@swyx
@Replit
@NaveenGRao
@MosaicML
Slides. Worth noting:
- Model trained entirely on code and initially meant for single-line code complete -- we did not expect it to do well on HumanEval!
- Moreover, we noticed a surprising capability in non-coding reasoning tasks, especially those that don't rely on knowledge
This is probably once in a lifetime opportunity to work in one of the most exciting technologies humanity has ever built.
It feels like the "Moon Landing" of our generation.
@aiexplorations
@fchollet
Nah, I have a lot of respect for those early techies. They literally wrote code in assembler that took rockets and spaceships to the moon in the 60s. No simulators, no Internet, no StackOverflow. Just pure logic, math, and code.
Congress discussing in 1993 how a pixelated Street Fighter game would cause violence and should be banned/regulated.
This was a big regulation topic a the time:
But seriously folks, this a short and juicy tirade in which I say:
(0) there will be superhuman AI in the future
(1) they will be under our control
(2) they will not dominate us nor kill us
(3) they will mediate all of our interactions with the digital world
(4) hence, they will
โจNEW LAUNCH! ๐ซ JungleGym, a set of open-source datasets and tools to test/build autonomous web agents.
๐Lack of testing AI agents is a big hurdle. We hope this small open-source contribution helps:
โ Playground:
โ GitHub Repo:
Mojo seems pretty neat.
A new programming language for AI developers. Basically Python with the performance of C
"Write Python all the way down to the metal. No C++ or CUDA required."
Every developer that started with C or Assembly (like myself) and went to Python, knows how
โจ I made a YouTube transcription tool using
@OpenAI
's
#Whisper
model to transcribe any YouTube video to a PDF.
Just paste any YouTube link and get the PDF.
It has hyperlink timestamps that go to any specific video section.
It's on
@ProductHunt
today!
โจSome friends & I put together a Robotics/AI lab in the heart of SF:
This is for building fun side projects, AI paper nights, robots, building ML workstations & wknd build/hangouts with friends.
SF is so back๐
What's next after GPT-4 (& other SOTA LLMs)?
It seems that very long context lengths could be possible, lengths of millions, "maybe even a billion", including images as context.
๐งต
[New program] a16z Open Source AI Grants
Hackers & independent devs are massively important to the AI ecosystem.
We're starting a grant funding program so they can continue their work without pressure to generate financial returns.
LLaMA2 Mixture of Experts is in on the way (many teams are already trying different approaches) trying to come closer to GPT4โs performance.
On big benefit for this MoE approach is the model size (70B) for its performance. You can run it in one A100 without any optimizations.
@texasrunnerDFW
I stopped using Airbnb as it became more expensive than boutique hotels. On top of that, we need to clean and still pay high cleaning fees.
Check-in is also a pain. Looking for keys in random places/bushes was the last thing I wanted to do coming tired from traveling.
I've been saying this for some time & it's finally happening:
New neat data center hardware is coming that will enable you to do queries on massive vector databases, all in one single server.
This means that massive vector DBs, embedding creation & similarity search (i.e FAISS,
I still find it fascinating that scaling LLM model size doesn't significantly improve symbolic reasoning tasks by itself, but chain-of-thought prompting does.
It probably means that there's actually a lot of room for reasoning improvement in existing models, we just know too
@sama
I already know electricians in the Bay Area that make more than software developers.
The development cost of software is going down as barrier to enter is also low.
Should we do a small GPT-4/LLM + Robotics weekend hackathon?
Small as our space robotics lab (
@madscisf
) in Alamo Square/Hayes Valley is not that big
This was one of the most memorable segments from
@justinkan
and
@balajis
podcast:
"The crypto version of any product is probably going to be about 10x more valuable than the centralized version"
The original deep learning paper that introduced diffusion models in 2015, "Deep Unsupervised Learning using Nonequilibrium Thermodynamics", will probably start getting more credit.
They've proven to work pretty well.
@amasad
@swyx
@Replit
@NaveenGRao
@MosaicML
One of the most under-discussed things I've seen is that the newer text models are fine-tuned versions of a code model, not a language model.
They seem to learn logic much better before they learn language.
For instance, GPT3.5 is a second fined tuned iteration of Codex
Takeaways from the Voyager Agent (with Minecraft) paper & more on "AutoGPTs":
- It's the first LLM that uses in-context learning with Minecraft (no re-training required!) with the help of a skill library.
- Unlike other environments (Dota 2, StarCraft, Atari), Minecraft
@mattturck
This is great content. What the large media producers don't get yet, is that anyone can be a greater creator (and not with high budgets & gatekeepers anymore).
YouTube democratized content creation. TikTok is next.
Some takeaways from the Tree of Thoughts (ToT) paper:
- Introduces ToT, what seems the next iteration of CoT (Chain of Thought), and similar to a search tree algorithm.
- If you take away backtracking & pruning, the success rate of solving games reduces significantly.
- It
Well this blew up ๐คฏ. We will be having the amazing
@mckaywrigley
and
@sxwy
joining as co-organizers along with
@agihouse_org
for the Autonomous Agent hackathon.
@karpathy
will also be joining to give an intro talk.
To give more time to prep, the hackathon will now be happening
Unitree just introduced their new humanoid robot for <$90k.
Need to get one to start adding all these MoE AI models (Vision, LLM, SLAM, etc) to it ๐
Introducing Unitree H1: Its First General-purpose Humanoid Robot| Embodied AI, Price below $90k
The preview of half-a-year achievement
The highest-power-performance robot of its counterparts with similar specifications in the world, weigh ~47Kg, maximum joint torque of 360N.m
100% note from
@llama_index
. The quality of the embedding models has a huge impact on your RAG applications.
BGE-base, large & small are great and probably the best embedding models.
You can use them with Sentence Transformers, and the best is that you can fine-tune very fast
The quality of your embeddings can have a huge impact on the effectiveness of your retrieval, which is critical to the quality of your RAG system.
@Shahules786
looks at how to pick the best embeddings for your specific data.
Big improvement in Python:
The CPython's memory management will now be thread-safe, allowing truly multi-threaded programs to take full advantage of multiprocessor systems.
- Deployed as early as in 3.13 or 3.14 as experimental
- Long-term (5+ years) no-GIL build should be the
No More GIL!
the Python team has officially accepted the proposal.
Congrats
@colesbury
on his multi-year brilliant effort to remove the GIL, and a heartfelt thanks to the Python Steering Council and Core team for a thoughtful plan to make this a reality.
@goodside
I know a large customer service company (~$2B valuation) is using Flan T5 11B. They started with GPT3, got enough customer data, and now they fine-tuned Flan.
It's performing better according to them, but not sure on what metric ๐
Improving Image Generation with Better Captions
DALLยทE 3 paper!
link:
In order to improve text-to-image generation, they improve the quality of existing image-caption large-scale datasets by recaptioning. A CoCa image captioner model is trained, and
Over 150 AI top builders & researchers creating tools and projects at our hackathon event.
@SebastianThrun
kicked off the event and shared his โAI *hot takesโ with everyone ๐
@ChrisJBakke
Wait, you got promoted with this without asking in every meeting:
โขโWill it scale?โ
โขโLet's take a step back, what exactly are we trying to solve?โ
๐คฏ
โจNEW LAUNCH! LLaMA2 chat API & open-source playground๐ซ:
We're releasing tools that make it easy to test
@meta
's latest LLM & add it to your own app with
@replicatehq
.
Playground:
Live chat API here:
Repos & instructions below:
Excited to join the
@CohereAI
ambassador program!
This means I will be testing out new stuff from
@CohereAI
and building new AI products with some neat stuff from them :)
So many opportunities in AI are at the GPU system/kernel level. There's so much great work to be done here.
From optimizing Open AI's Triton kernels, parallelism, profiling, debugging, better abstractions between PyTorch and Triton, to making it easier to modify kernels like
โจ We are hosting an epic
#GPT4
hackathon on Sat, Mar 25 in SF/Bay Area.
Bringing some of the best AI researchers, builders & engineers together.
Sponsored by
@GreylockVC
Limited spots available & only focus on those who ship code. RSVP:
Turns out the โstable diffusionโ moment for LLMs is LLaMA and itโs happening now.
But it seems that there will be more LLaMAs coming very soon. Know of many teams working on neat stuff here.
โจNEW LAUNCH! LLaMA2 chat API & open-source playground๐ซ:
We're releasing tools that make it easy to test
@meta
's latest LLM & add it to your own app with
@replicatehq
.
Playground:
Live chat API here:
Repos & instructions below:
โจWe are hosting an epic AI Hackathon next WKND in SF/BA.
We are building w
@LangChainAI
,
@trychroma
,
@OpenAI
's GPT3/chatGPT & more.
We are bringing some of the best AI researchers, builders & engineers together.
There are limited spots available & only focus on those who ship
on the 25th together with
@Mascobot
,
@keerthanpg
, and a few others, we'll be hosting the 'augmented imagination' hackathon
use chroma, langchain, and/or whatever else you like to make cool stuff with generative a.i
come thru
@aiexplorations
@fchollet
I think you might be confusing market fit/innovation with how technically skilled those engineers were & how hard it was back in the day.
We have a ton of tools, documentation super easy high level languages (Python, JS, etc) and tools to build on top of that they didnโt have.
Neat functionality dropped by OpenAI today:
Basically, in the past, it has been hard to form & complete a consistent JSON format/output; you needed to try multiple times until it was right.
Now, OpenAI allows you to just pass the desired JSON format/fields to the model and it
Super excited about this new $1.25B
@a16z
infra fund, and more excited to be part of it with an amazing team
@martin_casado
has put together.
I personally love AI infra. It's the foundation of a lot of the products and tools we use and couldn't be more excited about founders
We've raised a $1.25B infrastructure fund! We love all infra, compute, network, storage, databases, data science, gen AI, dev tools ... from silicon to UIs.
Infra is the true root of value in tech. And we're deepening our commitment to it.
Hollywood is adopting AI faster than I would imagine.
@runwayml
was used to create (some scenes of) the movie "Everything Everywhere All at Once", the movie that just won seven Oscars, including Best Picture.
Congrats
@c_valenzuelab
& team
The
#SpaceX
Flight Software team is about 35 people.
They write all the code for
#Falcon9
, Grasshopper, and
#SpaceXDragon
applications.
I will let you chew on that one.
#Software
keeps eating the world. Huge impact with small teams.
#spacexlaunch
It's wild to think that all this is happening (SVB & tech crash) right at the same time when were are seeing probably the biggest technological shifts (with AI) of the next decade(s).
Probably one of the largest "resets" in the startups & tech ecosystem is coming soon.
@sriramk
Thereโs a great โsecretโ location that is amazing: the lounge of the Kabuki hotel in Japan Town. It has coffee, a bar and itโs in the middle of amazing food places ;)
Hereโs a quick presentation/demo of the chatGPT plug-in & SAM (Segment Anything Model) from Meta AI for the service robot dog to recognize objects that we built this weekend:
Took the recently released footage of the
#shooting
at the school in
#uvalde
#Texas
(for testing).
Built an ML model to detect weapons in real-time and send SMS notifications.
Installing this in all my cameras now.
DM me if you want access to it for your cameras.
The GPT Agent demo of giving $500 OAI credits to all attendees with voice commands was real.
So many things could have gone wrong with that demo (like someone in the audience shouting $5K instead of $500) ๐