🪐 Introducing Galactica. A large language model for science.
Can summarize academic literature, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more.
Explore and get weights:
🎉 Introducing... Datasets! We are now indexing 3000+ research datasets from machine learning. Find datasets by task and modality, compare usage over time, browse benchmarks, and much more! Explore the catalogue here:
🎉 We've just crossed 5000 Datasets! 🎉
We now index and organize more than 5000 research datasets for machine learning. A huge thanks to the research community for their ongoing contributions.
Browse the full catalogue here:
🎉 Papers with Code is expanding to more sciences! Today we launch new sites for physics, maths, CS, statistics and astronomy. You can use these sites to sync your code to show on arXiv. Explore our portal here:
🎉 Introducing... Methods! We are now tracking 730+ building blocks of machine learning: optimizers, activations, attention layers, convolutions and much more! Compare usage over time and explore papers from a new perspective. Browse the catalogue here:
We’ve just released the new Papers With Code! Site now has over 950+ ML tasks, 500+ evaluation tables (including state of the art results) and 8500+ papers with code. Explore the resource here: . Have fun!
Graph neural networks are driving lots of progress in machine learning by extending deep learning approaches to complex graph data and applications.
Let’s take a look at a few methods ↓
🎉 A huge update to Papers with Code: now with 2500+ leaderboards and 20,000+ results. Plus, results now link directly to tables in arXiv! Read more in our blog post below and feel free to submit results from your own papers!
⏪ Papers with Code: Year in Review. We’re ending the year by taking a look back at the top trending papers, libraries and benchmarks for 2020. Read on below!
🎉 Introducing...Papers with Libraries! We are now indexing ML libraries from the community. We start with the amazing TIMM by
@wightmanr
. Explore 330+ models, visualize architectures, compare results and hyperparameters, and so much more!
⏪ Papers with Code: Year in Review
We’re ending the year by taking a look back at the top trending machine learning papers, libraries and new datasets for 2021. Read on below!
Graph neural networks are rapidly advancing progress in ML for complex graph data applications.
In this week’s newsletter, we show you recent uses of GNNs ranging from protein interface prediction to collaborative filtering! Read on below:
🔥 Introducing... the ML Reproducibility Challenge 2020! The 4th annual edition now expands to cover papers from 7 major ML conferences: NeurIPS, EMNLP, ACL, ICML, ICLR, CVPR and ECCV. Find more here at Papers With Code:
🎉 Introducing sotabench : a new service with the mission of benchmarking every open source ML model. We run GitHub repos on free GPU servers to capture their results: compare to papers, other models and see speed/accuracy trade-offs. Check it out:
💫 There has been an explosion of interest in self-supervised learning (SSL) for vision and NLP.
In this week's newsletter, we show you recent uses of SSL across various ML tasks from medical image analysis to musical style transfer! Read on below:
🎉 We've just crossed 4000 Datasets! 🎉
A huge thanks to the community for their ongoing contributions.
Since launch:
🆕 1000+ new datasets added
🔗 Dataset links now show on arXiv
👩💻 Dataset loaders
Browse full catalogue here:
🎉 Introducing the new Papers with Code newsletter! Our newsletter helps you manage the firehose of new ML papers by highlighting trending papers, new state-of-the-art results, and community contributions from around PwC.
🧬 Machine learning is having a big impact on scientific discovery
In this week's newsletter we show recent papers where ML is accelerating scientific discovery, from protein structure prediction to detecting gravitational waves.
🎉 We've just crossed 1000 ML Methods! 🎉
A huge thanks to the community for their ongoing contributions.
Methods help to discover the latest building blocks of machine learning (e.g., attention layers and optimizers). Browse the catalogue here:
💥 Deep learning makes strides on tabular data!
In this week’s newsletter, we summarize recent developments and papers using deep learning models for tabular data... and much more.
Read on below:
Vision Transformers aim to bring the strengths of transformers into the world of computer vision.
It's early days but progress has been happening in areas such as as image recognition, video understanding, 3D analysis, and more.
Let’s take a look at some vision transformers ↓
Thank you everyone for trying the Galactica model demo. We appreciate the feedback we have received so far from the community, and have paused the demo for now. Our models are available for researchers who want to learn more about the work and reproduce results in the paper.
We believe models should be open.
To accelerate science, we open source all models including the 120 billion model with no friction. You can access them here.
🤝 Code Snippets! Methods now come with links to example implementations, so you can see how to implement hundreds of methods in various frameworks. Example:
🎉 We've crossed 6000 Benchmarks! 🎉
We now maintain more than 6.4K benchmarks for machine learning. A huge thanks to the research community for ongoing contributions.
Browse all benchmarks here:
StyleGAN3 is out and results are 🤯!
It proposes architectural changes that suppress aliasing and forces the model to implement more natural hierarchical refinement which improves its ability to generate video and animation.
1/8
📈 Self-supervised learning is a popular research trend, with lots of progress in the past year. Check out this task page with paper collections, resources and a new self-supervised ImageNet leaderboard.
⚡️ Efficient Vision Models are Trending! ⚡️
This week’s newsletter highlights progress in building efficient vision models, adopting Transformers to challenging domains like GANs, recent papers, and more!
Read on below:
🎉 We've now crossed 6000 Datasets! 🎉
Some dataset stats:
- 1.8K for image tasks
- 1.6K for text tasks
- 286 for question answering
- 7.4K ML benchmarks in total
Thanks for all the contributions!
Below is a thread of recent popular datasets ↓
🔥 Our new autocomplete gives you 𝙛𝙖𝙨𝙩 access to 25k papers with code, 2.8k leaderboards, 1.6k tasks, 760 methods. Researchers can add results & methods for papers to enhance their discoverability (). We'll be featuring the best additions next week!
🔥 The debate between ConvNets and Transformers intensifies!
In this week's newsletter: the latest improvements on ConvNets, neural networks for mathematical reasoning, and a closer look at model generalization. Read on below:
XLNet outperforms BERT on 20 tasks, often by a large margin, and achieves state-of-the-art results on 18 tasks including question answering, natural language inference, sentiment analysis, and document ranking. Code and comparisons here:
✨ New Feature: Dataset Loaders! ✨
Easily find code to load datasets in your preferred framework!
Supporting:
@huggingface
datasets, TensorFlow datasets, OpenMMLab, AllenNLP, and many more libraries!
Example:
🔥The ML Reproducibility Challenge is back!
The 5th annual edition now expands to cover papers from 9 major ML conferences: NeurIPS, ICML, ICLR, ACL-IJCNLP, EMNLP, CVPR, ICCV, AAAI and IJCAI.
Find more here:
💫 New research explains how vision transformers work!
In this week's newsletter: summary of how and why vision transformers work, improving OOD detection, a unified multimodal pretraining framework, and new state-of-the-art results. Read on below:
What do popular research repositories have in common? We aim to capture this with the ML Code Completeness Checklist - now part of the NeurIPS 2020 submission process. Read more here:
🎉 New feature: State-of-the-art GitHub badges. Submit evaluation results from your paper to obtain a badge for the official GitHub repository. A new way to highlight your paper's performance!
There's been a rapid adoption of self-attention models for visual recognition.
This weeks' newsletter highlights a taxonomy of self-attention models for vision, recent papers, and more! Read on below:
🎉 New feature: PyTorch Hub and Google Colab integration! Now you can go straight from a paper to a Colab notebook, and obtain pre-trained models from PyTorch Hub (look out for more to come!). Get started with BERT :
Labels? Where we're going, we don't need...labels.
💥 VISSL is a new library for self-supervised learning by
@priy2201
et al. You can browse and compare different self-supervised learning methods in the catalogue below!
🔥 Torchvision on Papers with Code! 🔥
🔎 Discover and compare 60+ pre-trained models
🔮 Tasks: image/video classification, segmentation, object detection
🏋️ Training recipes for models
📱 Bonus: new MobileNetV3 models added by
@bbriniotis
!
🔥 Papers with Code Newsletter
#9
🔥
We cover:
• MLP-style models with competitive results on image classification!
• dataset links on arXiv articles,
• diffusion models beat GANs on image synthesis,
• GNNs for fake news detection,
• ... and more
💡The Vision Transformer has been getting some...attention. Results, code, methods and more have been indexed here: . A good starting point for anyone who wants to dig deeper.
We're hiring passionate engineers looking to make ML research more reproducible, discoverable and extensible. Apply here to work with us on this mission:
🤝 Papers with Code is now helping to maintain .
AI Conference Deadlines makes it easy to find and follow conference deadlines in areas such as machine learning and NLP. Thanks to
@abhshkdz
for this great initiative.
🌊 MLPs are back with a vengeance: is this the next wave after Transformers?
In this week's newsletter we cover top trending papers for May (lots of MLPs!), a DALL-E like method for text-to-image generation, and much more!
✨ A single MSc course produced 9 papers! ✨
Learn about their story and how the Reproducibility Challenge can boost your course in a post by
@__alucic
:
🎉 Reproducibility Reports are LIVE!
Papers now link to reproducibility reports. Alongside code, this is a new signal for how reproducible a paper's findings are.
Thanks to RC2020 participants, reviewers and ACs for their hard work!
Example:
DeiT is a new Vision Transformer which trains on ImageNet alone, with a new teacher-student training strategy specific to transformers. Paper, code, results, methods and more have been indexed below.
📈 Vision Transformers are trending in the community!
In this week’s newsletter, we summarize recent research in vision transformers, real-time rendering of NeRFs, advancing GPT-style models, and more.
Read on below:
🛠️ Hybrid models for vision and NLP are on the rise!
In this week's newsletter: the latest hybrid architectures for vision and language, language models in embodied agents, top 10 trending papers, and more. Read below:
🚀 The ML Reproducibility Challenge is an amazing opportunity for students to learn about cutting edge ML research.
@JesseDodge
explains how you can incorporate it in your course to help your students:
👩💻 Machine Learning Meets Code!
In this week’s newsletter, we summarize the latest advances in ML for code - including Codex (powering GitHub Copilot). Other highlights:
🧬 AlphaFold code released!
🤖 Transformers for RL
🎑 CLIP for VQA
Read on below:
Detectron2 on Papers with Code! 🚀
👁️ Discover 80+ models for object detection and segmentation.
⚖️ Visualize performance vs speed tradeoffs.
🔮 Perform advanced tasks like panoptic segmentation.
🚂 Find model recipes for training and evaluation.
Vision-language pre-trained models are driving lots of progress on machine learning tasks that require vision & language modalities.
Let’s take a look at some recent models ↓
🔥The ML Reproducibility Challenge is back!
The 6th edition now expands to papers from 11 ML conferences: NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR, ECCV, AAAI, IJCAI-ECAI, ACM FAccT & SIGIR.
ML journals included this year: JMLR, TACL, & TMLR.
Read more:
In a new paper from
@wightmanr
et al. a traditional ResNet-50 is re-trained using a modern training protocol. It achieves a very competitive 80.4% top-1 accuracy on ImageNet without using extra data or distillation.
[mini-thread]
Text-Driven Neural Stylization for 3D Meshes
This new paper proposes a text-driven, neural network based approach (Text2Mesh) for stylizing 3D meshes.
Paper & Code:
(A short thread) 1/6
📈 Top trending ML papers of March 2021
In this week's newsletter: top trending papers, state-of-the-art speech recognition, revisiting neural probabilistic language models, progress on efficient vision models, and much more.
Read on below:
🔍 Many of the
@paperswithcode
methods now appear as featured snippets on
@Google
! If you are an author, you can add your methods to the resource (with descriptions, images) to enhance their discoverability beyond your papers.
New state-of-the-art for several semantic segmentation benchmarks. FastFCN by Wu et al replaces dilated convolutions with a Joint Pyramid Upsampling (JPU) module to reduce computational complexity. Official code and comparisons here:
🚀 Vision models are getting bigger and more effective!
In this week's newsletter: a summary of the latest in visual synthesis, new techniques for scaling up vision models, and much more.
💥 Exciting progress in language modeling for long sequences!
In this week's newsletter, read about new methods for long sequences (Fastformer, ∞-former, ALiBi), T5 on code, zero-shot FLAN, and much more.
🔍 Check out this link if you want to quickly browse the ICLR 2021 list of accepted papers containing code. You can filter papers by task, author, keyword, etc.
🎉 We’ve just crossed 5000 ML Benchmarks! 🎉
A huge thanks to the community for their ongoing contributions.
Benchmarks help you keep track of the latest progress in ML across a wide range of tasks.
Browse all benchmarks here:
💥MMDetection comes to Paper with Code!
Explore 370 object detection & instance segmentation models by
@OpenMMLab
:
⚖️ Compare speed vs performance.
📄 Linked papers and methods for each model.
👻 ...and much more!
Browse here:
New state-of-the-art for object detection on COCO. Liu et al introduce a composite backbone architecture that extracts more representational basic features than the original backbone (trained for image classification). Code & comparisons here:
📈 Papers with Code now integrates with ongoing competitions! Our first partner is
@Eval_AI
- huge thanks to
@rishabhjain2018
! Learn more about how to integrate your competitions here:
💫 Vision-and-Language models are trending!
In this week’s newsletter: we highlight progress in vision-and-language pretrained models, new ML papers & results, and more. Read on below:
We release our initial paper below. We train on a large scientific corpus of papers, reference material, knowledge bases and many other sources. Includes scientific text and also scientific modalities such as proteins, compounds and more.
Interesting new paper for image classification that introduces a new type of convolution (OctaveConv): achieves higher accuracy with fewer FLOPs on ImageNet. But no public code yet: implementations needed!
💡 Can we build NNs to process any modality?
In this week's newsletter, we summarize architectures for handling diverse modalities and output tasks... and much more!
Read on below:
🎉 Introducing AllenNLP on Papers with Code Libraries
We are excited to welcome
@ai2_allennlp
to our library catalogue.
🕵️♀️ Explore state-of-the-art NLP models
🎯 Find training recipes, weights & model metadata
🏆 Compare models like RoBERTa and GPT-2
🔥 Papers with Code Newsletter
#14
🔥
In this week's newsletter, we highlight new findings involving the lottery ticket hypothesis.
Other highlights:
🏆 Top trending ML papers of July 2021
⚡️ YOLOX - SoTA object detectors
... and more
Read on below: