CEO of
@abacusai
, using Gen AI to build Applied AI and LLM agents and systems at scale, ex-AWS / Google, passionate about human behavior and open-source AGI
YouTube is a much better teacher than any course or book when it comes to machine learning and data science
Popular videos that have beautiful visuals help your brain absorb faster than sleep-inducing books
Channels that explain key AI research papers are the cherry on top
Common Probability Distributions That Occur In Nature And Why Mathematics Is The Language Of The Universe
Fundamentally the patterns of the universe repeat themselves making machine learning and AI possible.
Basically, when training ML models, including NNs we are attempting to…
India just kissed its future goodbye!
Every company deploying a GenAI model now requires approval from the Indian government!
That is, you now need approval for merely deploying a 7b open source model 🤯🤯
If you know the Indian government, you know this will a huge drag!…
Strongly recommend refreshing your linear algebra before getting into deep learning
Your brain can learn and remember concepts way better if you really understand the fundamentals
For example, TensorFlow basically runs computations on Tensors 👇
Literally, the ONLY company without an open-source LLM today is OpenAI
Everyone else, including Google, MSFT, and Meta, has released an open-source model.
Finally, we have a hallucination leaderboard! 😍😍
Key Takeaways
📍 Not surprisingly, GPT-4 is the lowest.
📍 Open source LLama 2 70 is pretty competitive!
📍 Google's models are the lowest. Again, this is not surprising given that the
#1
reason Bard is not usable is its…
The First AI-Generated Video That Looks Super Real
Microsoft Research announced VASA-1.
It takes a single portrait photo and speech audio and produces a hyper-realistic talking face video with precise lip-audio sync, lifelike facial behavior, and naturalistic head movements…
Unlike neural networks, the human brain can learn from just a few examples
Sometimes, this translates to over fitting from a few personal experiences
Resulting in stereotyping, experience bias and narrow mindedness
Twitter is the closest we have to a human neural network
Each of us are nodes in the collective network…
Transforming the information we receive with our weights and biases and propagating it to others
Are We Living In A Simulation? Stephen Hawking and Elon Musk, think so....
Elon Musk, Stephan Hawking, and several other serious types have postulated that we are living in a simulation and a large number of us are NPCs (non-playing characters).
Nick Bostrom, an Oxford…
Backpropagation and Gradient Descent - The dynamic duo of deep learning
Ever wondered about the mathematics behind NN training? Here is a simplified way to understand the process.
1. Initialize the NN: Create a neural network and initialize it with random weights and biases.…
All kidding aside, this popular meme illustrates the single biggest problem with supervised machine learning
Sometimes the data distribution in your training dataset is only a narrow representation of the data distribution in the wild...
Create Your Own Custom LLM ChatBot - An AI Brain for your organization
A step by step tutorial on how to choose the best LLM and infrastructure to create a Custom ChatGPT and supercharge your business!
A large language model (LLM) trained on your company's data can function…
Artificial intelligence vs. human brain - a love-hate relationship
As AI models (neural networks) have become more performant they have surpassed the human brain at multiple tasks
Neural networks are fundamentally modeled on the human brains.
Here is how they are similar…
The Path to Conscious AI
The single biggest question of our times is, will sustained innovation on LLMs inevitably lead to AI consciousness? While GPT-4 is clearly not self-aware or sentient, we don't know how GPT-6 or 7 will behave.
If AI ever becomes conscious, it will have…
Diffusion Models - The Magic Behind Stable Diffusion and MidJourney
Watching an AI model generating a photorealistic image from a text prompt feels like magic. Image generation models like Stable Diffusion and MidJourney have made us all prompt artists.
Here is how the magic…
ANNOUNCING Giraffe - Long Context Open-Source LLMs - An AI Brain for your Organization
We are excited to invent and open-source Giraffe, the world's first commercially usable 32K long-context open-source LLM based on Llama-2
Our AI research paper details our inventions around…
Still a large number of my people in my IRL circle get their news from traditional mainstream media
Frankly speaking, that terrifies me
They are consuming an extremely selective and biased version of reality!
No wonder we are so polarized! With the election coming, this is…
Bayesian Neural Networks - Capturing The Uncertainty Of The Real World
Life is inherently uncertain and probabilistic, and Bayesian Neural Networks (BNNs) are designed to capture and quantify that uncertainty
In many real-world applications, it's not sufficient to make a…
The Ongoing Case For Open Source LLMs
Custom LLMs, long context, and efficient inference
Some folks believe that training open-source LLMs is a losing battle and a complete waste of time.
They argue that the gap between closed models like GPT-4 and open models like Llama will…
The future we have all been anxiously waiting for is finally becoming real.
Researchers at Stanford open-source Aloha, a low-cost robot that appears to have learned how to do fancy cooking based on imitation learning!
The thesis is that robots can imitate humans after watching…
The pace of open-source LLM innovation and research is breath-taking
I suspect that open-source will soon become unbeatable for anyone except maybe OpenAI
Here's why
- Open-source community is way bigger than any specific company
- Safety lobotomy and fear of bad press will…
Types of Neural Networks - Evolution Of Deep Learning Architectures.
Oppenheimer, the movie, has all of us thinking about the 40s and WW2. Believe it or not, the first neural networks (NN) were invented around the same time, in 1943!
Warren McCulloch and Walter Pitts the…
If you must define these terms
Statistics: Fit your data to a particular pattern
Data Science: Transform your data until you find patterns
Machine Learning: Learn the patterns in your data
Artificial Intelligence: Learn the patterns, generate new data
Large Language Models Explained - At a High Level
LLMs such as GPT-4, Bard and LLama appear magical and have displayed emergent human-like intelligence
Fundamentally, the work on a pretty simple principle - next-word prediction.
An LLM, uses something called conditional…
If you are looking to start your career in machine learning, I would first crawl before you run
I would first learn:
- SQL
- Python
- Basic algebra & statistics
- Understand model eval/metrics
and only then start training deep learning models etc.
I often get asked what the difference between AI and ML is? Off late, the terms tend to get used very interchangeably...
The tongue in cheek answer:
If it is written in Python, it's probably Machine Learning
If it's is written in PowerPoint, it's probably AI
Reality check on Devin.
- Devin uses GPT-4, which can get expensive very quickly. Yes, it can get even more expensive than a human
- Most interview questions aren't hard to solve; even ChatGPT solves it
- Implementing and solving tasks with your own shell/browser, etc, is much…
RAG with LLMs seems deceptively simple but is extraordinarily hard to do well.
Building an intelligent ChatGPT-like tool with a custom knowledge base requires multiple non-trivial components.
A simple vector database for retrieval is rarely enough; you need a semantic…
As suspected, OAI invented a way to overcome training data limitations with synthetic data
When trained with enough examples, models begin to generalize nicely!
Great news for open source and decentralized AI - we are no longer beholden to the data rich companies 💃❤️
The Theory Behind Neural Networks and Why They Are So Affective
Neural networks, the machine algorithm behind every Gen AI model have been extraordinarily effective at multiple tasks because they are universal approximators.
The Universal Approximation Theorem (UAT) is a…
Machine learning models learn the biases back in data and reflect them back to us
Hungarians have no gendered pronouns but apparently, Google Translate has learnt all the gender stereotypes! 😱😱
Just tried out the latest hardware Tesla autopilot update and it is pretty mind blowing
The AI navigated urban roads and traffic signals like a boss
Level 5 autonomy is very close and is going to be a total game changer!
Time-series forecasting - AI to predict the future
While LLMs and Gen AI have received a lot of attention, state-of-the-art (SOTA) time-series forecasting models almost seem magical when they can predict a future value with high accuracy. They are used to predict stock prices,…
AI can take any of your old photos and turn them into poignant and moving videos
This is a brilliant rendering of Alan Turing, the father of machine learning - heartfelt and touching
Imagine doing this with your childhood photos and memories!!
Mathematics of Deep Learning - In simple terms
While you can train neural networks without knowing the math behind them, understanding the fundamentals helps you develop intuition around how to get to them to converge and achieve your objective. Of course, it's absolutely…
Every LLM has a personality
GPT-4 is like a lazy reticent teenager refusing to answer questions, requiring a lot of pushing to write code and do things
Claude 3 is like an eager and thoughtful adult with a bias for action
Gemini is a holier-than-thou wokist that strives too…
Recursively calling GPT-4 like AutoGPT does is a big mistake
The AI hallucinations and mistakes simply compound exponentially
There is nothing useful that can come out of it except over-hyped tweets from influencers trying to gain more followers
Finally, a totally liberated and open LLM!! 😂
dolphin-2.5-mixtral-8x7 is the first totally un-censored open-source model, fine-tuned on the Mistral MoE model.
It will respond to all user prompts and isn't judge-y!
It doesn't impose its morality on you and will surface any…
Prediction: LLMs are going to hit a wall after GPT-5
Reasons:
- No one other than OpenAI has released a GPT-4 class model yet
- We are soon going to hit a data and LLM-generated synthetic data limit
- GPT-4 already uses several RLHF tricks to improve performance
We will have…
Nvidia falls 10% as Llama-3 hits the market.
The algorithms are pretty good when it comes to anticipating the future and trading stock!
Why buy GPUs to train LLMs when we already have a couple of world-class open-source LLMs and more coming? 🤷♀️🤷♀️
My AI and ML research paper review time has gone down significantly.
Thanks to this amazing tool - that uses an AI model to explain dense sections
In the background, an LLM simplifies and explains complex concepts
AI explaining AI 👏👏👏
When you read a job description from a Big Tech company requiring production experience in deep learning and transformers...🙄🙄
I guess they are hoping BERT's authors will apply 🤷♀️🤷♀️
Data science in the future
- Use NLP to clean and wrangle data
- Iterate using multiple algorithms & parameters
- Offline evaluation followed by online eval
- Pipelines, monitoring & measuring ongoing performance
Mostly a thinking job using a modern AI/ML platform
How transformers evolved to become the dominant neural architecture
All large language models and most state-of-the-art language, forecasting, and personalization AI models are based on Transformers.
Before transformers, recurrent neural nets (RNNs) were used to process…
Understand the math behind the following machine learning algorithms
- Linear Regression
- Logistic Regression
- Decision Trees
- Naive Bayes
- Gradient Boosted Trees
- CNN
Once you grok the math, you can intuitively sense why your model behaves the way it does
A Novel RAG Approach That Understands The Whole Document Context
RAG has rapidly evolved to be the standard way to apply LLMs in production. However, most methods are still limited because most existing methods retrieve only short contiguous chunks from a retrieval corpus,…
Learning one of the most important ML and data science concepts, Bayes Theorem in the context of chilling with your BAE 😀
Probability of chilling given Netflix = (Probability of Netflixing given chilling x Probability of chilling) / Probability of Netflixing
Linear algebra, deep learning, and GPU architectures are fundamentally interconnected
Linear algebra deals with matrices & tensors and is the branch of math that is applied in deep learning (DL). GPUs are used for training DL models.
Tensors and GPUs are a marriage made in…
Recipe to getting a job in data science in 6 months
- Learn Python & SQL
- Brush up on stats & linear algebra
- Implement key ML algorithms using Kaggle data in notebooks
- Use real-world data, build machine learning models
- Practice interview questions
Get job :)
Good paper by Netflix on cosine similarity.
It goes back to building good RAG systems, which is hard. Before deploying these systems, you have to make intelligent decisions about chunking, hierarchical chunking, embedding, and even the algorithm for similarity look-up.…
RAG Or Fine-Tuning?
There is a lot of confusion about when to apply which method.
RAG makes sense when you have a custom knowledge base and want a standard ChatGPT-like interface on top of it. RAG has multiple components to it and can be tricky to get right. However, it's…
The Robot Videos Continue To Be Magical
Here is another video from the Aloha team where the robot does laundry, loads the dishwasher, and self-charges itself
While video is teleoperated, the fascinating bit is such a low cost device is so dextrous!
Particularly impressive is…
Here are the top 9 machine learning algorithms ranked based on their impact and importance.
This infographic was created by analyzing the models trained to solve thousands of real-world AI problems using our ML platform,
@abacusai
The big risk we face with AI is not extinction but addiction.
As chat, role-play, and AI girlfriend apps explode, the AI models will be fine-tuned to maximize engagement.
LLMs will be optimized in the same way as TikTok's algorithms, which are engineered to get you addicted.…
Using LLMs to Convert Unstructured Data Into Structured Data
One of the most common use cases for Enterprise AI is NER, or named entity extraction.
It's particularly useful when combined with transcription if you run a call center - you transcribe the call and then extract…
How Can Open Source LLMs catch up to GPT-4V and Google's Gemini?
Open-source LLMs are getting really good. However, they are not as powerful as GPT-4 right yet. Plus, mutlimodal models like GPT-4V and Google's Gemini will be dropping soon... making it even harder for…
LLM Hallucinations Can Help Make New Scientific Discoveries.
Since hallucinations are a feature, not a bug - You can get LLMs to dream up possible solutions to complex problems.
You basically ask the LLM to start dreaming and then throw out the dreams that aren't plausible.…
The Upcoming AI Wars and The Science Behind Multimodal Large Language Models (MLLMs)
Today, GPT-4 reigns supreme on all the LLM benchmarks and leaderboards but rumors are that it will soon be displaced.
Google is claiming its Gemini model is 5 times more powerful than GPT-4.…
OpenAI Developer Day Announcement and Implications for Open-Source AI development
OpenAI's developer day was filled with a bunch of annoncements. Most notable amongst them was a 3x price cut on GPT-4-turbo, Custom GPTs and Assistants APIs
Sadly and predictably, there was no…
Bias-Variance trade-off - The goldilocks principles of machine learning
Your ML model's performance depends on this!
A good machine learning model is a delicate balance between bias (underfitting) and variance (overfitting).
Bias refers to the error introduced by…
Want to learn ML and data science?
Don't just read tweets or watch videos, start building
- learn SQL and python
- try ML platforms or attend a workshop
- re-play some simple Kaggle notebooks
- wrangle noisy data, get to real-world machine learning
Smaug-72B - The Best Open Source Model In The World - Top of Hugging LLM LeaderBoard!!
Smaug72B from Abacus AI is available now on Hugging Face, is on top of the LLM leaderboard, and is the first model with an average score of 80!!
In other words, it is the world's best…
Why Large Language Models Hallucinate and How to Reduce it
If you are a power user of ChatGPT you have probably been bitten by the hallucination bug. The LLM lulls you into getting comfortable with it and then springs a convincing but totally made-up story, playing you for a…
The Age of Robotics
The age of robotics with multimodal LLM brains is finally here. The robot will understand instructions and perform tasks. It can use a laptop, wash dishes, and make coffee!
These robots will be more robust and way smarter than humans. Most of us will call…
Here is how Gemini Gate most likely happened….
A handful of employees who write the system prompts and are in charge of nerfing LLMs can impose their morality, bias and their version of history on the rest of us
These employees are so “holier than thou” that questioning…
Working on machine learning models can easily become a serious addiction
the adrenaline rush from anticipating training results
+
the dopamine kick that you get with incremental accuracy gains
=
an intoxicating combination
Custom LLM and AI Agents (RAG) On Structured + Unstructured Data - AI Brain For Your Organization
Imagine a ChatGPT-like interface over all your structured (database) and unstructured data. Ideally, you want to ask a question to an AI bot, and it should be able to run multiple…
How Bayesian is your thinking? If you think about your beliefs in terms of probabilities and alter them based on new evidence, as opposed to shooting down evidence in order to continue to believe, you will have fewer glitches in your perception of the world.
The Emergent Abilities of LLMs Could Be A Mirage!
The best paper award in NeurIPs 2023 went to a paper claiming that the emergent abilities of LLMs could be a mirage!
The paper (link in alt) asserts that emergent abilities appear due to the researcher’s choice of metric rather…
The Most Popular Machine Learning Algorithms in Production
Large Language Models based on the transformer architecture may be buzzy but the vast majority of the algorithms are tabular data algorithms.
Here is the list of the most popular algorithms
Transformers: Specialized…