Made a GPT-3 summarizer that reads websites just like humans do.
It scrolls pages and reads in visible text in chunks, which it then attempts to summarize.
This makes it a bit more robust than crawling HTML. Here you can see it summarizing fancy hotels on Flyertalk:
Nat and I are delighted to be investing $100m in , which is building a superhuman software engineer.
If a copilot generates $10b of revenue, how much is a colleague worth?
Greetings AI-native hackers.
@natfriedman
and I present a small hack from last weekend: Tele-Prompt.
An on-device AI for your meetings that listens to you and makes charismatic quote suggestions --
Excited to announce we've raised 62.7M$ at 1.04B$ valuation, led by Daniel Gross, along with Stan Druckenmiller, NVIDIA, Jeff Bezos, Tobi Lutke, Garry Tan, Andrej Karpathy, Dylan Field, Elad Gil, Nat Friedman, IVP, NEA, Jakob Uszkoreit, Naval Ravikant, Brad Gerstner, and Lip-Bu…
LlamaAcademy: a factory that teaches LLaMA's how write API code. ChatGPT Plugins, but no manifest file is needed -- it just reads the API docs like a human would!
For example, here is a finetuned model that learned how to talk to Notion by reading its API docs:
He was 99 with the eating habits of a teenager -- we had steak, wine, and candy for dinner. Charlie really embodied that poem:
Do not go gentle into that good night,
Old age should burn and rave at close of day;
Rage, rage against the dying of the light.
The gap between how powerful LLMs appear to be on Twitter and how genuinely useful they are in reality is incredible.
(Maybe the conclusion is that they are, in fact, very powerful because nobody bothers to actually ***read*** anymore?)
Google was about Borg not PageRank, and OpenAI might be about Triton not GPT-3. Companies have a tendency to turn algorithm moats to infrastructure moats, which are much more durable.
I think we’re sorta overestimating the odds of a 175B model replacing humans, and underestimating the power of 100M models at replacing _functions_.
GPT3.5 might be uncanny valley…
"The Transformer was published at NeurIPS 2017, one of the top AI conferences worldwide. Yet it didn't even get an Oral presentation, let alone awards. There were 3 best papers at NeurIPS that year. Combined, they have 529 citations as of today."
A deep lesson there...
Apple presents MM1, a family of multimodal LLMs up to 30B parameters, that are SoTA in pre-training metrics and perform competitively after fine-tuning
Maybe it's just me. But I find myself very heavily editing almost anything GPT-4 writes, be it English or code. I don't think this observation is priced in at all...
The next iteration of Perplexity has arrived: Copilot, your interactive AI search companion. 🚀🤖 Perplexity Copilot guides your search experience with interactive inputs, leading you to a rich, personalized answer, powered by GPT-4. Try it for free at
One thing I've learned from panics (Covid, Middle East) is who has a mind of their own, and who has outsourced their thinking to the cult of the moment.
I think we will look back at current AI -- where it takes 10 seconds to get a maybe-good response -- as the uncanny valley between Copilot (small model + fast autocomplete) and Coworker (large model + slow task completion).
PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing
Presents a sparse LM with 1T parameters trained over 329B tokens
An amazing demonstration of what's possible when an internet-native like
@natfriedman
focuses the web on a particular problem. It's like a magnifying glass that concentrates diffuse rays of human intellect into a focused beam that can cut through any task.
Ten months ago, we launched the Vesuvius Challenge to solve the ancient problem of the Herculaneum Papyri, a library of scrolls that were flash-fried by the eruption of Mount Vesuvius in 79 AD.
Today we are overjoyed to announce that our crazy project has succeeded. After 2000…
It's a musical chairs game -- and the exercise of 2000-2020 was writing software to become system-of-record for something.
Now the music has stopped. If you don't have a system-of-record, your company might not have a moat anymore.
Given exponential increase in training costs, compute multipliers might become the most coveted secrets on earth. Some of those will be in torch.nn; many will be in silicon.
It's funny how execs have so many big honking problems they're trying to fix, yet simultaneously, so many founders are unsure of what to build. Like a zillion riders and drivers that can't find each other.
Apple presents Ferret-UI
Grounded Mobile UI Understanding with Multimodal LLMs
Recent advancements in multimodal large language models (MLLMs) have been noteworthy, yet, these general-domain MLLMs often fall short in their ability to comprehend and interact effectively with
Tremendous respect to all my past colleagues at Apple who have been quietly working on Vision and all other releases, including those to come, while the rest of the world loses it's mind.
It's what you do in the dark that puts you in the light...
Around Silicon Valley, many are assuming superlinear progress in model intelligence and value but it’s quite possible we go through multiple winters and summers. There’s just a lot that we don’t know.
Measured narratives always get quashed in moments of greed…
I added multiplex streaming in python-llm, my little "python-requests for LLMs" project. Helpful if making a UI that outputs both OpenAI and Anthropic at the same time. One line to implement.
New in the fine-tuning API:
• Capture snapshots at every epoch, adjust hyperparameters, track metrics, and integrate with Weights & Biases
• Compare model quality and performance side-by-side in the Playground
Read more:
Watching llama.cpp do 40 tok/s inference of the 7B model on my M2 Max, with 0% CPU usage, and using all 38 GPU cores.
Congratulations
@ggerganov
! This is a triumph.
Is it just me, or are all the higher-level AWS offerings total garbage? I always end up doing it myself on EC2. Reminds me of a flea market filled with gimmicky electronics that break as you leave the store.
There still might be other kinds of moats -- MrBeast has a valuable empire and the cost of making video content is also ~free. But "software alone" is having it's printing press moment.