Simon Suo Profile Banner
Simon Suo Profile
Simon Suo

@disiok

1,932
Followers
1,253
Following
59
Media
894
Statuses

co-founder @llama_index prev: AI research @Waabi_ai @Uber_ATG , PhD AI @UofT

San Francisco, CA
Joined June 2010
Don't wanna be here? Send us removal request.
Pinned Tweet
@disiok
Simon Suo
11 months
After 5 amazing years at @Waabi_ai & @Uber ATG, surrounded by the most brilliant minds in AI and mentored by @RaquelUrtasun , I decided to embark on a new adventure with my close friend @jerryjliu0 . Today I’m excited to announce that we have started a company around @llama_index
@jerryjliu0
Jerry Liu
11 months
I’m super excited to make it official: @disiok and I have started a company around @llama_index , and we’ve raised a $8.5M seed round led by @GreylockVC ! 🔥🚀 We are building the open-source data framework to unlock LLM capabilities on your private data.
96
120
1K
5
9
61
@disiok
Simon Suo
10 months
Llama 2 Prompt Structure is a fantastic playground for the latest Llama 2 model. But wait, is the suggested prompt structure the best way to interact with the model? """ User: <your prompt goes here> Assistant: """ Some initial notes 👇
2
34
164
@disiok
Simon Suo
4 months
2023 is a whole lot of “let’s do it” - startup - move to SF - hire team - new relationship - mindset upgrades Grateful for family, friends, believers, advisors. The journey is just beginning.
Tweet media one
7
6
97
@disiok
Simon Suo
10 months
Llama 2 Protip If you are hacking with the @replicate hosted 13B or 70B models (playground or API), this is relevant for you! The `max_length` parameter limits the total number of tokens (i.e. sum of input prompt + output response). More details 👇
3
8
70
@disiok
Simon Suo
18 days
How I visualized the so-very-overloaded AI engineer role (a la @swyx 's ) anyone aware of breakdown / articulation similar to this?
Tweet media one
8
4
59
@disiok
Simon Suo
8 months
prototype: one weekend productionize: weeks? months? We are tackling this gap by open-sourcing production-quality app templates, so you can hit the ground running. goes way beyond the typical streamlit demo, complete with advanced features like -…
@llama_index
LlamaIndex 🦙
8 months
We’re excited to open-source - a full-stack, production-ready RAG app! 🦙🏦 Supports streaming, reasoning steps, citations, intuitive UI This can save you weeks/months of hard work in trying to build a prod LLM app from scratch🔥
10
102
567
1
1
46
@disiok
Simon Suo
3 years
It's a nudge! It's a U-turn! It's S̶u̶p̶e̶r̶m̶a̶n̶ TrafficSim! Excited to share our work on learning to simulate realistic multi-agent behaviors at #CVPR2021 . Come to our poster session at 10pm ET today to learn more! Joint work with Sebastian, @sergioksas , and @RaquelUrtasun
3
7
43
@disiok
Simon Suo
4 years
Check out at #ECCV2020 how we generate multiple scene-consistent motion forecasts of complex urban traffic, directly from sensor data! Paper: Live Q&A: Thu 9am & 7pm ET Joint work with: @sergioksas , @cole_gulino , @katielulula , @lrjconan , @RaquelUrtasun
1
14
40
@disiok
Simon Suo
9 months
My notes on the new @OpenAI fine-tuning release: 1. currently limited to GPT-3.5 Turbo with 4K token (GPT-4, function-calling, and gpt-3.5-turbo-16k all coming this fall) 2. similar DX as old GPT-3 fine-tuning API, just list of chat messages instead of (prompt, completion) pairs…
Tweet media one
Tweet media two
@disiok
Simon Suo
9 months
Exciting day! @OpenAI GPT-3.5 Turbo fine-tuning is finally out!
0
4
35
5
6
34
@disiok
Simon Suo
1 year
Similar to autonomous driving, maybe we should be explicit about automation levels for knowledge work as well. L5: fully autonomous agents (AutoGPT et al.) L4: semi-autonomous with human-in-the-loop support L3: copilot, assistant, constrained cognitive tasks
@ylecun
Yann LeCun
1 year
Haha. Auto-Regressive LLMs gonna auto-regress. Your hands must remain on the keyboard at all time. Level-2 Writing assistance? Yes! Level-5 autonomous writing? No! "Here’s What Happens When Your Lawyer Uses ChatGPT"
60
96
673
1
5
32
@disiok
Simon Suo
2 months
You can build an AI consulting shop by just always replying “look at your data more”
3
1
31
@disiok
Simon Suo
11 months
Designing a robust system requires clear interfaces and well-behaved components. The future of LLM-powered systems is not one "monolithic agent" that does everything. It will be many specialized components (think query routing, knowledge retrieving, API calling). New @OpenAI
1
2
28
@disiok
Simon Suo
1 year
Watching @karpathy 's amazing talk and came across a familiar diagram
Tweet media one
1
2
25
@disiok
Simon Suo
1 year
Are your agents too dumb or you just need better tools?
0
4
25
@disiok
Simon Suo
4 months
Libraries need to start writing docs/tutorials for LLMs instead of humans soon
@bernhardsson
Erik Bernhardsson
4 months
I’m bullish on LLMs for code but this is kind of a funny/weird counterpoint that maybe it makes developers chose worse tools
Tweet media one
85
48
734
5
3
24
@disiok
Simon Suo
11 months
If you are at #BerkeleyHackathon , definitely come check out @jerryjliu0 talk on @llama_index
Tweet media one
0
1
21
@disiok
Simon Suo
2 months
We are in the era where the metric is just vibes
@_aidan_clark_
Aidan Clark
2 months
how the hell did anthropic get away with this chart?
Tweet media one
18
2
234
3
0
20
@disiok
Simon Suo
3 months
1
1
19
@disiok
Simon Suo
10 months
Few people know this: LlamaIndex got its name before the original Llama model came out. Great Minds Think Alike? 🤔 Incredible to see Meta continuing its commitment to open source. And glad we can stack more 🦙.
@llama_index
LlamaIndex 🦙
10 months
🦙x🦙 = 💪 Experiment with Llama 2 now via LlamaIndex! We made a special release (v0.7.10.post1) to help you get started super easily 👇
Tweet media one
1
16
112
1
2
18
@disiok
Simon Suo
4 months
one of the best writeup on assistant architecture in production (how did I miss this gem): 👏 @hiranya911
1
5
17
@disiok
Simon Suo
4 months
get yourself some investors that keep your toes warm and keep you caffeinated thanks for the kindness and support @jerrychen @rischter_scale @GreylockVC 🙏
Tweet media one
Tweet media two
Tweet media three
1
1
17
@disiok
Simon Suo
11 months
@jerryjliu0 looking good
Tweet media one
0
1
17
@disiok
Simon Suo
3 months
RAG vs long context LLM? Why not both We just got new power to tune the desired point on accuracy vs latency/cost frontier. Exciting future ahead
2
2
16
@disiok
Simon Suo
6 months
im so confused is this a hard fork
1
0
16
@disiok
Simon Suo
10 months
Last but not least: using the reference encoding, seems like it works!
Tweet media one
Tweet media two
1
1
16
@disiok
Simon Suo
5 months
Do we really want AGI or just really cheap and fast open source thought calculators that runs in our laptop
2
3
16
@disiok
Simon Suo
6 months
X (twitter) ads revenue sharing + OpenAI GPTs revenue sharing = bracing myself for the sea of custom GPT tweets
2
3
15
@disiok
Simon Suo
5 months
Is reasoning inherently coupled with knowledge? My intuition says there’s a small kernel of knowledge that can bootstrap reasoning. Right now, to get improved reasoning, we also get useless facts. I just want my LLM OS to be pure compute, and not have the pre-installs.
6
1
15
@disiok
Simon Suo
2 months
I’m assembling a team of 10 IPhO kids. DM and you can get an immediate offer
7
1
15
@disiok
Simon Suo
5 months
next time I write a paper, I'm releasing it on torrent
1
2
13
@disiok
Simon Suo
19 days
Setting a goal to tweet an original / personal thought every day. Purely for the online to IRL friend pipeline. Optimizing for that tweet-to-friend ratio
6
0
14
@disiok
Simon Suo
10 months
A deep dive into the official codebase shows that system instructions and messages use specific formats (from ).
Tweet media one
Tweet media two
1
2
13
@disiok
Simon Suo
5 months
A small milestone on this wild ride! 🚀 Super grateful for the love from @llama_index community, and support from our investors @GreylockVC and partners. Excited for many more celebrations ahead with our incredible team.
@jerryjliu0
Jerry Liu
5 months
Excited to announce that @disiok and I are featured in @Forbes 30U30 2024! 🎉 It's been a wild year, and we couldn't have done this without our community, partners, investors, and of course our wonderful @llama_index team.
Tweet media one
35
21
316
0
1
13
@disiok
Simon Suo
1 year
Can we stop trying to replace lawyers and scientists with LLMs, let’s just figure out how to get really really good at analyzing multiple documents first…
2
0
13
@disiok
Simon Suo
11 months
ChatGPT Plugins have not reached PMF. This is unsurprising, but refreshing to hear it stated plainly by @sama . Browsing Redfin through ChatGPT (or any chat interface) is painful, I think for 2 reasons: 1. When my intent/goal is clear, I might as well use the website’s own…
@swyx
swyx @ICLR_conf
11 months
i also admire @sama 's ability to confront reality and state the truth - ChatGPT Plugins have not reached PMF! I don't wake up in the morning and book my travel and food thru ChatGPT. Instead, the apps are all integrating chat. The Western consumer continues to resist the…
Tweet media one
11
3
66
2
1
13
@disiok
Simon Suo
10 months
Feeling AGI @agihouse_org
Tweet media one
@jerryjliu0
Jerry Liu
10 months
If you’re around at the @agihouse_org hackathon today, come check out @disiok ’s talk on data agents! 🤖💾 - Advanced search/retrieval - Act over a large range of external APIs (email, search, etc. ) Get a head start with our full agent guide here:
0
5
32
1
3
13
@disiok
Simon Suo
3 years
It's been such an amazing journey! Can't wait to get started on the next chapter.
@RaquelUrtasun
Raquel Urtasun
3 years
Today is my last day @Uber after 4 wonderful years. Thank you @dkhos for your support and leadership. Solving #selfdriving is my life's passion and I'm super excited for what comes next. Will share an update soon. Stay tuned!
21
30
641
0
0
12
@disiok
Simon Suo
9 months
the world needs more thoughtful & comprehensive long-form content like this
@eugeneyan
Eugene Yan
9 months
Wrote abt patterns for LLM systems/products • Evals: Track performance • RAG: Add external knowledge • Finetuning: Improve specific tasks • Caching: Reduce latency & cost • Guardrails: Ensure output quality • Defensive UX: Anticipate & manage errors
38
180
801
0
1
12
@disiok
Simon Suo
4 years
Throwback to when I was a neural network for Halloween 😅
Tweet media one
0
0
12
@disiok
Simon Suo
3 months
When I think quality long form writing, @eugeneyan @lilianweng are the top 2 that come to mind
@eugeneyan
Eugene Yan
3 months
> For those trying to educate, consider writing longform, designed for someone to get "sweaty", especially in today's era of quantity > quality ... This is what I aspire to do. My audience will decrease. The ones that remain might not like it. But at least we'll learn something.
3
1
59
1
1
11
@disiok
Simon Suo
5 months
Is there an “uncanny valley” but for intelligence / reasoning
1
1
10
@disiok
Simon Suo
8 months
Evangelizing EDD (evaluation driven development)
@raydistributed
ray
8 months
Want to learn how to build and evaluate production RAG app with @llama_index and @raydistributed ? Join #RaySummit Training Day! 1️⃣ Implement reliable eval methods for LLMs 2️⃣ Run experiments to optimize app components 3️⃣ Take best configs to production
0
7
18
0
0
10
@disiok
Simon Suo
11 months
AGI is gonna destroy us all . . . . Once it figured out how to consistently output some jsons
0
1
10
@disiok
Simon Suo
1 year
GPT4 is great and all but fully autonomous agents (AutoGPT et al.) aren’t happening any time soon. Compounding error is still too severe for long horizon, multi-step tasks. The agent “diverges” and gets stuck in a thought loop. Bullish on human-in-the-loop concepts though
2
0
9
@disiok
Simon Suo
11 months
Checkout this super insightful talk about RAG from @lateinteraction
@jerryjliu0
Jerry Liu
11 months
The DSP project carries a lot of insights for improving RAG: 💡value of few-shot ex’s 💡declarative modules 💡compile an optimized system with distilled LM’s We had a GREAT time chatting about this and more w/ @lateinteraction on our latest webinar! 👉
0
11
71
0
5
9
@disiok
Simon Suo
5 months
2024 is gonna be the year of open source LLMs
2
0
8
@disiok
Simon Suo
11 months
Super excited for this! Come hang out with us at the @UCBerkeley AI/LLM hackathon
@chontang
Chon Tang
11 months
I'm super excited to announce - the best hackathon ever?! @UCBerkeley AI/LLM hackathon featuring @OpenAI : - $200k investment - 1k+ hackers Partners: @msft4startups @langchainAI @llama_index @mindsdb @pinecone @sequoia @lightspeedvp @mayfieldfund @FeatureformML What? Why?
Tweet media one
19
88
419
0
1
8
@disiok
Simon Suo
3 months
@jxnlco I hear something called if statements are blazing fast. Who are the investors?
2
0
8
@disiok
Simon Suo
11 months
@jxnlco @jerryjliu0 Just change your name to json instead of Jason
1
0
8
@disiok
Simon Suo
10 months
Disclaimer: This is an adhoc investigation, and I've only tried a couple of prompts. I also don't know whether replicate hosted model handles encoding differently. But the takeaway stands: watch out for subtle train/test mismatch bugs, especially when you don't own the API.
0
0
8
@disiok
Simon Suo
3 months
RAG = IR + NLP
Tweet media one
1
0
8
@disiok
Simon Suo
6 months
Season finale, back to where we started
@OpenAI
OpenAI
6 months
We have reached an agreement in principle for Sam Altman to return to OpenAI as CEO with a new initial board of Bret Taylor (Chair), Larry Summers, and Adam D'Angelo. We are collaborating to figure out the details. Thank you so much for your patience through this.
6K
13K
67K
0
0
8
@disiok
Simon Suo
2 months
Why is modal so cool
@modal_labs
Modal
2 months
Customer showcase: use Devin to write Modal code – it uses Modal, so you can use Modal while you’re using Modal! 🔄🤯
3
3
85
1
1
8
@disiok
Simon Suo
3 months
Do you ever wonder if you are training data quality constrained or brain size constrained? Just me?
1
1
7
@disiok
Simon Suo
18 days
@moinnadeem the beauty of speaking at macro / vision level is that it's not entirely falsifiable, so you can get away with anything
1
0
8
@disiok
Simon Suo
2 months
It’s over when the nerds start power lifting
@WuNeal
Neal Wu
2 months
Yes
Tweet media one
Tweet media two
270
345
8K
0
0
8
@disiok
Simon Suo
4 months
Startup is like padding a kayak on the ocean, with 10ft waves crashing, while aircraft carriers fighting each other nearby. Kinda fun though would recommend
0
0
8
@disiok
Simon Suo
5 months
You can sense the marketing pressure from the SoTA comparison
1
0
7
@disiok
Simon Suo
8 months
now this, is an exciting event with real signal. incredible lineup across model, infra, tooling, and application.
@robertnishihara
Robert Nishihara
8 months
Ray Summit this month will be 🔥🔥 🤯 ChatGPT creator @johnschulman2 🧙‍♀️ @bhorowitz on the AI landscape 🦹‍♂️ @hwchase17 on LangChain 🧑‍🚀 @jerryjliu0 on LlamaIndex 👨‍🎤 @zhuohan123 and @woosuk_k on vLLM 🧜 @zongheng_yang on SkyPilot 🧑‍🔧 @MetaAI on Llama-2 🧚‍♂️ @Adobe on Generative AI in…
8
45
208
0
2
7
@disiok
Simon Suo
6 months
Congrats on the launch! This looks incredible
@newcomputer
New Computer
6 months
Announcing our first product, Dot — an intelligent guide with incredible memory. Read the full story here:
209
507
5K
0
1
7
@disiok
Simon Suo
4 months
In 2023 my Twitter feed shifted from academics to startup founders. Striking how everyone lives in their own bubbles & optimizes their own imperfect proxy metrics. Goodhart’s law Reminder that you get to pick your own metric.
0
1
6
@disiok
Simon Suo
4 months
Realized I’ve never fully appreciated the beauty of human language. It’s so perfectly imperfect. A single representation that allow us to express thought across various abstraction levels. Structured enough to be compositional building blocks, yet flexible enough for adhoc…
2
0
7
@disiok
Simon Suo
1 year
The history of ML feels like a pendulum swinging between domain-agnostic and domain-specific models. So far, domain-agnostic approaches have produced most big step changes, but what ends up getting deployed are highly optimized domain specific models.
@arankomatsuzaki
Aran Komatsuzaki
1 year
Gorilla: Large Language Model Connected with Massive APIs Releases Gorilla, a finetuned LLaMA-based model that surpasses the performance of GPT-4 on writing API calls. proj: abs:
Tweet media one
18
192
747
0
0
6
@disiok
Simon Suo
10 years
TIL the standard list of metasyntactic variables are: foo, bar, baz, qux, quux, corge, grault, garply, waldo, fred, plugh, xyzzy, thud
1
6
5
@disiok
Simon Suo
5 months
In a few years, kids grew up on post LLM internet are gonna start saying “as an OpenAI model…”
0
0
5
@disiok
Simon Suo
10 months
First, the "llama13b-v2-chat" refers to the fine-tuned variant of the Llama 2 model. While the base model simply completes text, the "chat" model is fine-tuned for conversations and instruction following. But how does it recognize messages and instructions?
1
1
6
@disiok
Simon Suo
3 months
Lemme check with llamaindex marketing department if we can buy a 0.05s Super Bowl ad next year. If anthropic can do it, why can’t we!
2
0
6
@disiok
Simon Suo
9 months
@OpenAI The launch blog () did a great job of highlighting use-cases and setting expectations. Fine-tuning helps with both quality/reliability and cost/latency. * quality/reliability: improve steerability, reliable output formatting, custom tone * cost/latency:…
1
0
6
@disiok
Simon Suo
1 year
Tweet media one
1
0
5
@disiok
Simon Suo
8 years
Chilling at #12BARZ with the bae @cselina_ .
Tweet media one
1
1
2
@disiok
Simon Suo
5 months
Human engineer a tedious process Human engineer a model to reverse engineer the tedious process Why
@AndrewCurran_
Andrew Curran
5 months
Microsoft is training a custom, narrow-focus LLM specifically on the regulatory process for small nuclear plants. They need to build SMRs to power Bing's brain. MS expects the LLM to eliminate 90% of the costs and human hours involved.
Tweet media one
40
109
942
0
0
6
@disiok
Simon Suo
5 months
How do I get a 1-on-1 with @gdb and post a selfie on X
0
0
6
@disiok
Simon Suo
4 months
@jxnlco at one point in my life I thought happiness is about reading philosophy and grasping the truth of nature and self turns out it's just sleep, eat, and exercise well
1
0
6
@disiok
Simon Suo
18 days
starting to love the concept of agent-data interface draws analogy to HCI and encompasses RAG, text-to-sql, knowledge graph queries etc
0
0
5
@disiok
Simon Suo
8 months
Gonna tell myself to take a deep breath before doing anything important. Because if a LLM can do x% better, I can too!
3
0
5
@disiok
Simon Suo
10 months
So, does using the reference encoding make a difference? Let’s test it out. First: without any prompt structure… just doesn’t work at all.
Tweet media one
Tweet media two
3
0
5
@disiok
Simon Suo
9 months
@OpenAI There's also an updated fine-tuning guide with more details (). Some practical advices from the guide: 1. split fine-tuning dataset into train & test (fine-tuning job will provide stats on both) 2. check data formatting with provided script locally before…
0
0
4
@disiok
Simon Suo
6 months
2022 gives us FTX the movie 2023 gives us OpenAI the movie What a time to be alive
2
0
5
@disiok
Simon Suo
10 months
Second: using “Human: … Assistant: “ results in rambling, hallucinating multi-turn conversation
Tweet media one
Tweet media two
1
1
5
@disiok
Simon Suo
1 year
RAG gang
@nickfrosst
Nick Frosst
1 year
The term “hallucination” for large language models seems really misleading. All ungrounded generations are hallucinations, just sometimes you get lucky and what comes out of the model is true This is why retrieval augmented generation is so crucial for making this stuff useful
9
18
123
0
1
5
@disiok
Simon Suo
2 months
Cloud era: separation of compute and storage AI era: separation of reasoning and knowledge
0
0
5
@disiok
Simon Suo
5 months
Undergrad, PhD, startup
@pmarca
Marc Andreessen 🇺🇸
5 months
2012-2016 was a distinct era. 2017-2022 was another. 2023- is new.
169
330
4K
0
0
5
@disiok
Simon Suo
9 months
PSA: @OpenAI fine-tuning is rate-limited to 1 active job per model per organization You can run 12 jobs per day though. 💸💸💸
Tweet media one
@disiok
Simon Suo
9 months
My notes on the new @OpenAI fine-tuning release: 1. currently limited to GPT-3.5 Turbo with 4K token (GPT-4, function-calling, and gpt-3.5-turbo-16k all coming this fall) 2. similar DX as old GPT-3 fine-tuning API, just list of chat messages instead of (prompt, completion) pairs…
Tweet media one
Tweet media two
5
6
34
0
0
5
@disiok
Simon Suo
4 months
crazy how "large" became a marketing term to the point where we now have "tiny L(arge)LMs" next stop: big TLLMs
0
0
5
@disiok
Simon Suo
11 months
Academic conference poster sessions vs. Startup conference screen sessions
Tweet media one
0
0
5
@disiok
Simon Suo
13 days
Pure behaviour cloning would’ve solved self-driving if not for the sophons
@ericjang11
Eric Jang
14 days
if your transformers struggle with NaNs after a certain parameter size, you may be under a sophon lock. Keep pushing, don't let them win!
8
10
209
0
0
5
@disiok
Simon Suo
3 months
Every entrepreneur out there reflecting if they are dreaming big enough
0
0
5
@disiok
Simon Suo
1 year
@karpathy What a legend. From ML recommendations () to LLM recommendations
Tweet media one
0
1
5
@disiok
Simon Suo
5 months
In what universe would I want to use a projector instead of a screen
@SquawkCNBC
Squawk Box
5 months
FUTURE OF TECH: @Humane Co-Founders @bella_bongiorno and @imranchaudhri show off the AI pin –a new wearable device powered by artificial intelligence.
88
76
352
1
0
5
@disiok
Simon Suo
21 days
It’s gonna be weird when we start naming kids “Llama 3 David”
1
0
5
@disiok
Simon Suo
5 months
Water water water
@BasedBeffJezos
Beff – e/acc
5 months
Hey everyone, serious post for a second. As you may have seen, I have unfortunately gotten my identity and stealth startup doxxed by reporters via voice forensics and web sleuthing. As the day has finally come, I thought I’d share more about who I am. I’ve kept my identity…
594
403
7K
0
1
4
@disiok
Simon Suo
4 months
@CalvinnChenn @jerryjliu0 Was convinced since we became friends in 2018
0
0
4
@disiok
Simon Suo
3 months
@EugeneVinitsky vehicle heading behaves like a usb stick. you only get it right on the third attempt.
1
0
4
@disiok
Simon Suo
18 days
@eugeneyan @BEBischof Would be fun to discuss how these archetypes fit in with the evolution of a early stage startup
1
0
4