We have free GPUs to support your research!
Thanks to a generous donation from
@JeffDean
, we have 28 GPUs to give away. Here is the full inventory:
(Nvidia GeForce GTX)
Titan: 10
1080: 2
Titan X: 6
Titan Z: 1
Tesla: 9
You are welcome to apply for 1-4 for research use.
8 researchers from Nigeria got accepted into Deep Learning Indaba
@DeepIndaba
but couldn't afford the trip. Access to research, mentorship, and networking opportunities like this is vital for early-stage researchers. You can help make their trips possible!
ICML deadline is less than 2 days. If you are in need of some compute to push forward your submission, send an email to compute
@mlcollective
.org or post on help-compute channel on our Discord, we can help out!
What's next after open-source and open-access? We believe it's *open collaboration*. The field is fairer, more inviting, diverse and inclusive only when we make mentorship and collaborators accessible. But how? We are discussing it on Tue at
#NeurIPS2020
:
ICLR 2021 deadline is 10 days away! Working on a draft but short on labmates or peers to proofread? ML Collective can help: send us a draft (review
@mlcollective
.org) between now and Oct 2 and one of our volunteer readers will read+comment. Undersupported researchers... (1/2)
Dive into the realm of Deep Learning: Classics and Trends (DLCT) with us tomorrow at 10 AM PST! 📷
Join our discussion on "Understanding & Mitigating Pre-training Noise on Downstream Tasks."
See you there! []
#DLCT
#DeepLearning
#AI
#PreTrainingNoise
Working on an ML research project but don’t have labmates to show? Show a plot or three in our Research Jam in two weeks!
Also open to folks that want to bounce an idea off others or who just want to hang out and talk shop. Details:
Do you know ML Collective has an "Office Hours" service? You can book 1:1 chats with researchers who kindly open up their calendars to serve the community.
Current ongoing sessions are proudly hosted by
@bneyshabur
@AndreaMadotto
@natolambert
Hi MLC folks and Open Collab advocates, It's been a while since we last hung out!
#ICML2021
is next week, and we are having an *Open Collaboration in ML research* social (again!) on Thursday.
Our theme is connecting (as usual) and story sharing (new!)
1/
🎉WE DID IT!! 🎉 All GPUs got delivered at NeurIPS. Remember how it started? 👇 It's been a crazy week finding and coordinating with volunteers. You rock, all!
There are still more sitting in SF waiting to be shipped to members in need. Stay tuned!
But now, let's celebrate 🥳
The long-awaited DALL-E 2 talk series is finally coming to DLCT!
#dalle
Starting this Friday, for 3 consecutive Fridays we are going over the science, engineering, and policy of DALL-E 2.
Join the mailing list to stay tuned:
Anyone in need of a workstation for research? A lab member has generously donated a workstation (specs below). “I bought this machine during my Ph.D. as I didn't have computing resources. I suppose there will be some students who might still be facing similar problems as mine.”
Already 3 ICLR papers we know of that came from collaborations facilitated by ML Collective! 😲
Reach out if there are more you know of, we'd love to collect them and put them in a "feel good" folder for future gloomy days
Thread for those papers:
[Help Session] It’s grad school application season! We are hosting an impromptu live help session this Saturday! Including:
- group SoP editing
- 1:1 or small group Q&A
- casual hangout
Applicants:
- bring you SoP
- bring your questions
Helpers:
- bring your expertise
Fun little project:
Problem: implementing Hebbian learning in convolutional networks with PyTorch/TF is a pain.
Solution: just use a special loss whose gradients turn out to produce exactly the desired Hebbian weight updates!
Going to NeurIPS next week? Anxiously scraping around for party invites? Rest assured, MLC parties are always open to all (...and scrappy and low budget). We will have a walk and picnic at the park!
🔔Call for RFPs🔔
ICLR 2021 is in two weeks! We are teaming up with
@__MLT__
@forai_ml
and EleutherAI communities to host an "Open Collaboration" Social!
We are soliciting contributions of RFPs () to be presented at the social.
Do you know that the best way to ensure personal research progress is to *set small milestones and deadlines*?
We understand doing research without institutional support is hard. That's why our most successful event series — "research jams" — is here to help set the cadence.
We understand life is busy — but if you booked an Office Hour slot with one of our hosts, please set reminders for yourself, show up, or cancel in advance! We've seen some number of no-shows, and it's a wasted time for the hosts and wasted chance for others who could've used it😢
Appreciating what
@joinClubhouse
offers but don't want to get behind its exclusive vibe?
Join MLC’s first *OpenClubHouse* event on our community Discord (NOT on ClubHouse), Sunday March 14, 10am PT!
Join our Discord: and enter "OpenClubHouse" channel
In 2010 when I applied for PhD programs, I was a Master student in Iran with no published papers, undergrad GPA C and poor to mediocre English test scores.
2/
Anyone can apply! You don't have to live in the US. We will (hopefully) find someone near your region at NeurIPS to help ship to you.
On that note, reach out if you are coming to NeurIPS from a different continent and willing to help with the distribution!
We have free GPUs to support your research!
Thanks to a generous donation from
@JeffDean
, we have 28 GPUs to give away. Here is the full inventory:
(Nvidia GeForce GTX)
Titan: 10
1080: 2
Titan X: 6
Titan Z: 1
Tesla: 9
You are welcome to apply for 1-4 for research use.
Our research jam is in two days! Schedule, line-up, and ways to join:
*It's 100% open to the public*
Should be straightforward how to join. But if you are the type of person that needs a calendar invite to remember anything, DM me your preferred email.
Independent researchers! Come share your work to receive:
- feedback
- compute assistance
- a sense of community!
Research jam
#17
is next week! Submission is open.
Tomorrow at DLCT () we will hear from
@lilianweng
on "asymmetric self-play for automatic goal discovery in robotic manipulation" as well as her insights on:
- unknown struggles in becoming a researcher
- underrated skills for an ML researcher
Application form:
Please apply before Nov 20!
We will be distributing them at NeurIPS 2022 in New Orleans. If you can't be there in person, we will ship to you.
New year, new jam! 🎆🎇
Research jam—our most famous event—is returning for 2022 this Wednesday. If you are working on an idea, bring it to our group meeting to receive feedback, possible collaborators and compute assistance.
[Call for Submissions] Research jam
#2
is in two weeks! We invite two types of submissions: "I did a thing" and "I have an idea," with a focus on early-stage, prototype work or unimplemented, big-picture ideas.
Quick summary of our GPU Giveaway:
- We received 28 donated GPUs and 185 applications
- We picked 28 awardees (15%) based on 1) need 2) usability (computer slot & research plan)
- 21 brought to NeurIPS
- 3 picked up in person
- 18 taken by volunteers to bring to destination
Tomorrow at DLCT we are learning about the intriguing "early phase of training," how it's unstable even when using a small learning rate, and how, using large learning rates can improve generalization due to implicit regularization.
Applied for PhD programs this past season? Now that all PhD admission decisions are made, we'd like to hear what your experiences have been.
Whether it's a happy, sad, or dramatic story, we are here for you.
Join OpenClubHouse
#11
(in 10h) to share your story!
Happy new year! (You are allowed to say HNY however late it is, as long as it is the first time you speak this year, right?)
This year we are still committed to democratizing research training for all. To start, we have (and will keep having) an amazing lineup at DLCT.
Do you know ML Collective has a reading group where people from the community gather weekly to read through this awesome book, chapter by chapter? We've had just finished Chapter 10! All meetings are recorded here:
Led by the greatest
@charles_irl
My editors just shared with me the feedback from early reviewers and I'm in tears 😭
With the help of so many people, I worked really hard on this book. I'm grateful that people gave it a chance.
Read the book online:
Pre-order:
Good news! We received a renewed
@googlecloud
award to support our members' research.
If your projects have been shut down due to the unfortunate gap between grants, please let us know and we can help you set up again.
Email: compute AT
In 5 hours we will be hosting research jam
#7
—the most radically open research meeting to get to know what MLC members are working on.
It is our mission to make research *process*—not just results—transparent, accessible, inviting and welcoming. Research jams manifest that.
Thanks everyone for joining our 1st OpenClubHouse!
We chatted about:
1/ legal aspects on open collab; possibly/impossibly out-lawyer lawyers🤣
2/ skills grad schools don't really prepare you for
3/ recent papers
4/ underrated/overrated research directions
5/ robot overlords 🤪
A few hundreds new followers have found us over the last week and pushed our followership over 10K — thank you! If you are new and wondering how MLC works, our Town Hall tomorrow is a great chance to catch up.
What is MLC Town Hall:
"Creating noise from data is easy; creating data from noise is generative modeling."
Guess which paper this line is from and you'd have guessed today's topic!
#dlct
@YSongStanford
@poolio
To everyone receiving tough news from
#NeurIPS2020
and considering resubmitting to
#ICLR2021
, the abstract is due tomorrow. Our draft proofread and review service can help! Volunteers from MLC and beyond are here with one goal: help improve your work. 1/3
We are raising $15k to support these researchers to attend
@DeepIndaba
. This amount will help cover their airfare, visa fees, and housing while at the conference. This sheet lists a breakdown of how the raised fund is going to be allocated:
Are you an ML researcher looking for ways to give back to the community? Hosting an office hour with us!
It's really easy: decide on a recurring time with any length and frequency of your choice, create a calendly (or equivalent) link and we can set it up for you.
🆕The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks
Our conjecture: Taking permutations into account, there is likely no barrier in the linear interpolation between SGD solutions.
w
@HanieSedghi
@osaukh
@bneyshabur
1/10
This Friday (since everyone is busy with ICML and I'm pathetically free) I am giving a talk at *my own* reading group -- extreme nepotism or what? 😬
Title: "Unconventional ways of training neural networks and what they teach us about model capacity"
We have a YouTube channel!
More content is coming up, but for tonight, listen to
@jasonyo
explain "What is a RFP" and why it is suitable for distributed research in 80 seconds.
Hey! What progress have you been making with your research? Or in what ways are you stuck? We'd like to hear about them so we know how to best support you!
Time to get together for a jam!
Submit your "thing" or "idea" here:
DLCT year finale is tomorrow! Last talk of the year! If you miss it……well there’s always next year. So do whatever you feel like doing.
Speaker:
@kchonyc
Title: oversmoothing of <eos> in neural autoregressive modeling
Abstract: (see pic)
More:
Over the years we’ve unlocked a secret weapon of online talks: having 2+ authors in charge of 1) delivering the talk live & 2) answering questions in the chat AT THE SAME TIME! We might have made a groundbreaking discovery that’s gonna change all future talks! 😬
Really flattered and excited to present our recent work
@ml_collective
tomorrow.
come join the discussion w/
@MarcDymetman
@MKhalifaaaa
12PM Pacific time Friday, April 2
Signup here 👉
We reached our fundraising goal today. Here is a histogram of all 101 donations from 95 donors. Thank you all — you make the world an unbelievably wonderful place!
8 researchers from Nigeria got accepted into Deep Learning Indaba
@DeepIndaba
but couldn't afford the trip. Access to research, mentorship, and networking opportunities like this is vital for early-stage researchers. You can help make their trips possible!
At MLC we have lots of ongoing events: regularly-occurring ones like DLCT, Research jam, Town hall, and interest group meetups. We also have random, one-off events because, well, we'd be reading something fun and go "I wonder how it works exactly. Let's have a hacking session!"
Great resource that supports PhD applicants! You are always welcome to contact MLC (hello
@mlcollective
.org) if you wish to connect with a research member to chat about applications, career paths, specific research areas, etc.
This year, CS PhD applications are different.
There are many more 'graduate application support programs' for applicants to get informal feedback (on statements, etc) from current PhD students before formally applying.
It's an awesome resource! I've link some below:
Part of ML Collective and want to help steer it? Or curious where we're heading? Stop by our first Town Hall meeting on Oct 24th to share, to listen, to help create the future!
Last week we had one of the best talks at DLCT! Learn about loss landscape, basins, permutation symmetry, and the unreasonable effectiveness of SGD:
Thanks for delivering such knowledge and fun to us
@SamuelAinsworth
📜🚨📜🚨
NN loss landscapes are full of permutation symmetries, ie. swap any 2 units in a hidden layer. What does this mean for SGD? Is this practically useful?
For the past 5 yrs these Qs have fascinated me. Today, I am ready to announce "Git Re-Basin"!
The future of community is hybrid, and we are committed to not just make it happen, but make it work.
Make it work for not just some, but all who thrive with and make thrive a community.
Time to JAM!
Submit to research jam on Wed for a chance to:
- tell people about your project
- find collaborators
- get feedback/mentorship
- get sponsored compute
But most importantly, get to feel like a real research member of the MLC community 💞
**DLCT today** We'll be learning about differential privacy (DP) and how to do that in large language models (LLMs). A "match made in heaven" as said by
@thegautamkamath
😻
Think privacy in ML ➡️ bad accuracy? 😭
Think again! We show that large language models (e.g., RoBERTa and GPT-2) can be privately fine-tuned with minimal loss in utility! 🥳
A super fun
@UWCheritonCS
@MSFTResearch
collab!
📝Paper:
🧵Thread ⬇️ 1/8
OpenClubHouse
#9
is canceled today, sadly, due to certain key personnels missing. Last time seen: Friday 3:04pm PST.
@Luke_Metz
might have a clue of their whereabouts 🤔🧐
Hope y’all have a wonderful rest of your Sunday!
Heard from a
#dlct
speaker that "I had the best audience in years" "It was a talk that I will remember" 😭😭😭 Thanks all for being the best audience ever! We don't know how that happened -- we never curated the audience; it's a group anyone can join from anywhere in the world.
There's a lonely feeling doing sth entirely by yourself, unsupported, unseen, unheard. Whether it's founding a company, managing a team, organizing an event. The warmest prefix in the world is "co-". The warmest word "collective" (OK I AM BIASED). Best phrase, "partner in crime".
Failing to attend
#ICML2021
? 😅
Don’t worry, the MLC social is only thing you need to take home with! Starting in 9h (8am PT/11am ET)
Full schedule is online:
Hi MLC folks and Open Collab advocates, It's been a while since we last hung out!
#ICML2021
is next week, and we are having an *Open Collaboration in ML research* social (again!) on Thursday.
Our theme is connecting (as usual) and story sharing (new!)
1/
Update: in 5 days we received astounding 155 requests for our 28 GPUs. To accommodate everyone's need as much as possible, we are expanding the program to include a "Request for GPUs":
Do you have unused or retired GPUs to donate to those in need? 🙏
We have free GPUs to support your research!
Thanks to a generous donation from
@JeffDean
, we have 28 GPUs to give away. Here is the full inventory:
(Nvidia GeForce GTX)
Titan: 10
1080: 2
Titan X: 6
Titan Z: 1
Tesla: 9
You are welcome to apply for 1-4 for research use.
In honor of our 1.5 year anniversary, the NLP Reading group is in full swing for the rest of 2022 - join us for
- Aug - Sept: LLM limitations
- Oct: Open Dialogue systems
- Nov: Conditional information retrieval
- Dec: Fact verification
- Jan: Open Domain QA
@ml_collective
We are hiring an Executive Assistant! Starting part-time (with a possibility to grow into full-time), can work from anywhere, and with competitive pay and growth opportunities! Reach out if it sounds like you or someone you know!
Anyone wants to lead a hackathon to play with these language models?
We can supply:
- GCP instances, with credits
- a room of helpers
You need to:
- read the README, follow the steps
- be happy to share your screen so others can learn and play with you!
A big step! Congrats all!!
If anyone in
@ml_collective
wants to try out these models with GCP and Cloud TPU, we can easily set up a hackathon session, with hopefully some consultations from you guys if running into problems?
@jamesleethorp
@ada_rob
@LiamFedus
?😊
MLC compute grant for 2023 is renewed! For those who already have a project set up with us, you should've received an email about how to un-pause.
For anyone wanting to start a project using our compute, email us! Read more:
Today's
#DLCT
features this great work unifying hyperarameters across architectures and how to tune any large model by just tuning a small version of it! Brought to you by
@TheGregYang
in 2.5h!
1/ You can't train GPT-3 on a single GPU, much less tune its hyperparameters (HPs).
But what if I tell you…
…you *can* tune its HPs on a single GPU thanks to new theoretical advances?
paper
code
blog
"Creating noise from data is easy; creating data from noise is generative modeling."
Guess which paper this line is from and you'd have guessed today's topic!
#dlct
@YSongStanford
@poolio
We just heard that two accepted ICLR 2022 Blog Posts came out of our NLP interest group alone! So we updated the "celebration" slide used at the Town Hall.
Keep doing cool stuff and let us know at the Town Hall!
Reviewer 2 complains it has "too many ideas".
@prfsanjeevarora
says "gave me some new ideas".
This Friday at Noon PT… come hear about the paper that reviewers don't want you to read!
I'm speaking at
@ml_collective
's DLCT:
A common mistake when presenting is to jump right into the “most interesting” result instead of starting with *what is best for the audience*.
Advice shared among our speakers to make the most of their 5 minutes; guess from whom 😉
Our research jam is in two days! Schedule, line-up, and ways to join:
*It's 100% open to the public*
Should be straightforward how to join. But if you are the type of person that needs a calendar invite to remember anything, DM me your preferred email.
Hi 200 people who responded to our
#NeurIPS2022
picnic lunch + river walk tomorrow (Thursday)!
We updated the exact meeting point and time (it's now 12:30-2 so you won't miss poster session).
Can still RSVP! Or not—just show up and meet us there🚶♂️🚶🚶♀️
Happy (almost) weekend! What are you doing Sunday? Wanna chat about anything ML related or unrelated in a *fully open* voice channel? Check out OpenClubHouse
#2
!
On MLC Discord:
No time / not interested? OKAY. Give us a like and you can go 🤪
It is beyond satisfying to see that all presenters tomorrow are *returning* presenters.
It's like running a local convenience store, and the true rewarding feeling comes when you've run it long enough to start seeing familiar faces every time 😊
🔔Call for RFPs🔔
ICLR 2021 is in two weeks! We are teaming up with
@__MLT__
@forai_ml
and EleutherAI communities to host an "Open Collaboration" Social!
We are soliciting contributions of RFPs () to be presented at the social.
Anyone in need of a workstation for research? A lab member has generously donated a workstation (specs below). “I bought this machine during my Ph.D. as I didn't have computing resources. I suppose there will be some students who might still be facing similar problems as mine.”