Berivan Isik @BerivanISIK Twitter profile

Pinned Tweet

Berivan Isik

4 months

Very excited to share the paper from my last @GoogleAI internship: Scaling Laws for Downstream Task Performance of LLMs. w/ Natalia Ponomareva, @hazimeh_h , Dimitris Paparas, Sergei Vassilvitskii, and @sanmikoyejo 1/6

4

25

243

Last Seen Profiles

@OmbetaC

@QuizOoiho_ntv

@FaveroCryp65050

@music_stargazer

@DinasArmyyy

@jilbab_tiktod

@wavegodbh

@JetsonMade666

@SCC_Panthers

@ondi020202

$~// J₳Ɏ₴ \\ ~$

@Akito_Jays

@A52VFX

@bntlmrt67313174

@rachy1417

@studiogiftig

@Enkidu_io

@Geohigweh

@MinnStFootball

@joaocarreira

@arbityrdub

@HaveFaithJones

@Ahmedkh66646665

@f_alabduljaleel

@Dstills55

@gerdagoossens1

@chaseneer13

@StevieD91860215

@serp_archive

@MissUniverse

@AustralianOpen

@Adilla445

@OVERFRAGG

@MayaREden

@mpojenab

@21ninpo

@LukeKennard

Berivan Isik

@BerivanISIK

1 year

🥹 #ChatGPT

7

47

1K

Berivan Isik

@BerivanISIK

7 months

Honored to be selected as a Google PhD fellow this year! Thanks for the generous support @GoogleAI @Google .

Google AI

@GoogleAI

7 months

In 2009, Google created the PhD Fellowship Program to recognize and support outstanding graduate students pursuing exceptional research in computer science and related fields. Today, we congratulate the recipients of the 2023 Google PhD Fellowship!

23

93

584

23

9

442

Berivan Isik

@BerivanISIK

6 months

I am very excited about our new work: with @RylanSchaeffer @vclecomte @sanmikoyejo @ziv_ravid @Andr3yGR @KhonaMikail @ylecun . We’ll present it in 4 @NeurIPSConf workshops: @unireps (oral), InfoCog (spotlight), @neur_reps , SSL. Details in Rylan’s tweet👇

Rylan Schaeffer

@RylanSchaeffer

6 months

Excited to begin announcing our #NeurIPS2023 workshop & conference papers (1/10)! 🔥🚀An Information-Theoretic Understanding of Maximum Manifold Capacity Representations🚀🔥 w/ amazing cast @vclecomte @BerivanISIK @sanmikoyejo @ziv_ravid @Andr3yGR @KhonaMikail @ylecun 1/7

11

94

501

3

33

289

Berivan Isik

@BerivanISIK

4 years

@kisacakimdir Paylaşımınız için çok teşekkürler 😍

22

0

202

Berivan Isik

@BerivanISIK

6 months

Selected as a top reviewer @NeurIPSConf 2023. 🎈

4

1

180

Berivan Isik

@BerivanISIK

1 year

“Sparse Random Networks for Communication-Efficient Federated Learning” has been accepted at #ICLR2023 ! Code coming soon. Looking forward to seeing many of you @iclr_conf in Rwanda.

Berivan Isik

@BerivanISIK

2 years

Excited to share our new work, "Sparse Random Networks for Communication-Efficient Federated Learning". 1/6

1

6

42

5

15

159

Berivan Isik

@BerivanISIK

1 year

Happy to share the second paper from my @GoogleAI internship: Sandwiched Video Compression with Neural Wrappers. The sandwich framework is more efficient than most other neural video compression methods (details below 👇). 1/3

3

10

134

Berivan Isik

@BerivanISIK

9 months

Humbled to be selected as a Rising Star in EECS this year. Looking forward to meeting the 2023 cohort @GeorgiaTech soon!

10

0

127

Berivan Isik

@BerivanISIK

8 months

Excited to share our @NeurIPSConf '23 paper "Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimation": Looking forward to presenting it in person and seeing many of you in New Orleans! 🙂🎷🎶 Details 👇

2

10

127

Berivan Isik

@BerivanISIK

2 years

The first paper from my Google internship has been accepted to Frontiers in Signal Processing. This is the first work to compress volumetric functions represented by local coordinate-based neural networks. Paper link: Code coming soon.

2

12

95

Berivan Isik

@BerivanISIK

3 years

I will give a talk on our recent work on information-theoretic model compression at the Sparsity in Neural Networks Workshop @sparsenn on Friday.

3

5

73

Berivan Isik

@BerivanISIK

1 year

#Antakya Cebrail mahallesi şehit Mehmet Ali demirbüken caddesi emlak bank konutlarından haberi olan ulaşabilir mi? İçerdekilere ulaşamıyoruz. @AFADTurkiye #deprem #AFADhatay #enkazalt ındayım

2

124

68

Berivan Isik

@BerivanISIK

2 years

ODTÜ’de mezuniyet Devrim’de olur. #ODTUMezuniyetininYeriDevrimdir

0

3

61

Berivan Isik

@BerivanISIK

11 months

Excited to share our new work with @FrancescoPase , @DenizGunduz1 , @sanmikoyejo ,Tsachy Weissman, and Michele Zorzi. We reduce the communication cost in FL by exploiting the side information correlated with the local updates and available to the server.1/3

2

7

60

Berivan Isik

@BerivanISIK

24 days

I will be at AISTATS and ICLR in the following weeks. Let me know if you'd like to chat about efficient and trustworthy ML. Also, check out our work: - [AISTATS, May 3rd 5 pm Valencia] Adaptive Compression in Federated Learning via Side Information: 1/2

2

4

54

Berivan Isik

@BerivanISIK

2 years

Excited to share our #AISTATS2022 paper titled "An Information-Theoretic Justification for Model Pruning": Come say hi at the conference during our poster session on Wednesday, March 30th, 8:30-10 am PST. 1/6

3

5

51

Berivan Isik

@BerivanISIK

5 months

Check out our talk/poster @unireps if you’re interested in maximum manifold representations and how to study them information theoretically! #NeurIPS2023

Rylan Schaeffer

@RylanSchaeffer

5 months

At #NeurIPS2023 & wondering what posters to check out? I have 5 recommendations!! 😉 1. An Information-Theoretic Understanding of Maximum Manifold Capacity Representations @unireps w/ @BerivanISIK @vclecomte @sanmikoyejo @ziv_ravid & more 1/5

2

20

165

0

4

48

Berivan Isik

@BerivanISIK

10 months

I will be @icmlconf for the whole week. Text me if you want to meet up! (Papers 👇) PS: Don't forget to stop by our workshop @neural_compress on Saturday.

1

2

49

Berivan Isik

@BerivanISIK

1 year

Excited to be visiting @NicolasPapernot ’s lab @VectorInst this summer 😎 Let’s catch up if you’re in Toronto!

1

2

49

Berivan Isik

@BerivanISIK

1 year

Join us for our Neural Compression workshop at @icmlconf 2023! We’ll release the call for papers soon. Organizers: @YiboYang , @_dsevero , @karen_ullrich , @robamler , @s_mandt , @BerivanISIK More details 👇

Stephan Mandt

@StephanMandt

1 year

🎉Exciting news! Our "Neural Compression" workshop proposal has been accepted at #ICML 2023! Join us to explore the latest research developments, including perceptual losses and more compute-efficient models! @BerivanISIK , @YiboYang , @_dsevero , @karen_ullrich , @robamler

4

24

99

2

6

48

Berivan Isik

@BerivanISIK

1 month

Super excited about our upcoming @icmlconf workshop! Stay tuned for updates 🙌 For details:

TF2M@ICML 2024

Workshop Summary

sites.google.com

Theoretical Foundations of Foundation Models

@tf2m_workshop

1 month

We are happy to announce that the Workshop on Theoretical Foundations of Foundation Models will take place @icmlconf in Vienna! For details: Organizers: @BerivanISIK , @SZiteng , @BanghuaZ , @eaboix , @nmervegurel , @uiuc_aisecure , @abeirami , @sanmikoyejo

1

11

48

0

6

46

Berivan Isik

@BerivanISIK

1 year

Looking forward to the Neural Compression Workshop @icmlconf this year. Please consider attending and submitting your latest work. Deadline is May 27th.

Neural Compression Workshop @ICML23

@neural_compress

1 year

The 2nd iteration of the "Neural Compression: From Information Theory to Applications" workshop will take place @icmlconf in Hawaii this year! Submissions due May 27th. For more details: @BerivanISIK @YiboYang @_dsevero @karen_ullrich @robamler @s_mandt

3

18

63

0

2

45

Berivan Isik

@BerivanISIK

2 years

Excited to share our new work, "Sparse Random Networks for Communication-Efficient Federated Learning". 1/6

1

6

42

Berivan Isik

@BerivanISIK

1 year

#HelpTurkey #deprem #Turkey

0

19

41

Berivan Isik

@BerivanISIK

6 months

I will be at #NeurIPS2023 all week. Text me if you'd like to chat about trustworthy & responsible AI at scale! I'll present two works: Tue afternoon: Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimation () 👇

2

0

36

Berivan Isik

@BerivanISIK

10 months

The workshop is happening at room 317A @icmlconf now! Please also join us for the social as well. Everyone is welcome! Details 👇

Neural Compression Workshop @ICML23

@neural_compress

10 months

Please join our social at Maui Brewing Co. Waikiki at 6pm after the workshop. Everyone, especially compression and information theory enthusiasts, is welcome! @icmlconf

0

4

13

0

4

34

Berivan Isik

@BerivanISIK

2 years

Finally made it to the office 🌚

0

31

Berivan Isik

@BerivanISIK

2 years

I will give an in-person talk on our work "Efficient Federated Random Subnetwork Training" at the NeurIPS Federated Learning Workshop. Looking forward to seeing many of you in New Orleans. Drop me a message if you want to meet up! #neurips2022

Berivan Isik

@BerivanISIK

2 years

Excited to share our new work, "Sparse Random Networks for Communication-Efficient Federated Learning". 1/6

1

6

42

1

25

Berivan Isik

@BerivanISIK

2 years

Check out our new paper titled “Learning under Storage and Privacy Constraints”. We propose a novel data pre-processing framework, LCoN, which simultaneously boosts data efficiency, privacy, accuracy, and robustness. 1/4 #compression #privacy #learning

3

0

23

Berivan Isik

@BerivanISIK

3 years

We will be in #NeurIPS2020 WiML and Deep Learning through Information Geometry workshops with our work on neural network compression for noisy storage systems:

1

2

21

Berivan Isik

@BerivanISIK

1 month

Super excited about the 5th edition of the @DMLRWorkshop at @icmlconf 2024. Stay tuned for the updates! 👇

Workshop on Data-centric Machine Learning Research

@DMLRWorkshop

1 month

We are thrilled to announce that the #DMLRWorkshop on "Datasets for Foundation Models" will take place at the @icmlconf in July! This marks the 5th edition of our #DMLR workshop series! Join the DMLR community at

0

4

13

0

3

20

Berivan Isik

@BerivanISIK

3 years

We are excited to announce that Workshop on Information-Theoretic Methods for Rigorous, Responsible, and Reliable Machine Learning will take place @icmlconf . We have an excellent line of speakers, including a recent Shannon award winner! More details:

ITR3 @ ICML-21

Joining the Workshop on July 24: https://icml.cc/virtual/2021/workshop/8365 Workshop Summary The ICML-21 Workshop on Information-Theoretic Methods for Rigorous, Responsible, and Reliable Machine...

sites.google.com

ITR3 Workshop @ ICML21

@ITR3_workshop

3 years

Workshop on Information-Theoretic Methods for Rigorous, Responsible, and Reliable Machine Learning (ITR3) will take place @icmlconf this year. Submissions due May 24th. Details: @abeirami @FlavioCalmon @BerivanISIK @hey1jeong @matthewnokleby @CindyRush

1

26

86

0

1

19

Berivan Isik

@BerivanISIK

1 year

"Neural Network Compression for Noisy Storage Devices" will appear at the ACM Transactions on Embedded Computing Systems (TECS): We propose ways to provide robustness to neural networks against noise present in storage or communication environments. 1/3

2

1

18

Berivan Isik

@BerivanISIK

2 years

New bounds for the deletion channel:

1

0

18

Berivan Isik

@BerivanISIK

2 years

“Kaliforniyaya taşınacaksın. Vietnam-Amerikalı, budist, DJ, kolunda Texas dövmesi olan ev arkadaşın salona seccade serecek, evde bağlama çalacak.”

0

17

Berivan Isik

@BerivanISIK

4 months

Thanks for the highlight! 🙌 @arankomatsuzaki

Aran Komatsuzaki

@arankomatsuzaki

4 months

Scaling Laws for Downstream Task Performance of Large Language Models Studies how the choice of the pretraining data and its size affect downstream cross-entropy and BLEU score

1

20

99

0

1

17

Berivan Isik

@BerivanISIK

1 year

Join us on Wednesday night for a fruitful discussion at the @BerkeleyML panel.

Machine Learning at Berkeley

@BerkeleyML

1 year

Looking to dive into AI research but unsure how? We're excited to host guests @xiao_ted ( @GoogleAI ), Yi Li ( @AmbiRobotics ), @TheRealRPuri ( @OpenAI ), @BerivanISIK ( @Stanford ) and @ritageleta ( @berkeley_ai ) for our research panel!! Come through Wednesday evening with questions!

2

14

39

0

16

Berivan Isik

@BerivanISIK

4 months

That my aunt was “hired” to be my aunt

Hollyn

@sadlifeebro

4 months

What's the dumbest thing you believed as a child?

111

22

217

0

15

Berivan Isik

@BerivanISIK

3 years

Registration and poster abstract submissions for the Stanford Compression Workshop 2021 are now being accepted! Date: 25-26th February 2021 Website: Poster abstract submission deadline: 21 Feb 2021

0

1

12

Berivan Isik

@BerivanISIK

4 months

Thanks for the highlight! @_akhaliq

AK

@_akhaliq

4 months

Scaling Laws for Downstream Task Performance of Large Language Models paper page: Scaling laws provide important insights that can guide the design of large language models (LLMs). Existing work has primarily focused on studying scaling laws for

3

33

160

0

11

Berivan Isik

@BerivanISIK

1 year

Tomorrow at the FLOW seminar, I will talk about our @iclr_conf 2023 paper "Sparse Random Networks for Communication-Efficient Federated Learning". Looking forward to your feedback and questions. 🙌

Federated Learning One World Seminar (FLOW)

@flow_seminar

1 year

📢: The 99th FLOW talk is on Wednesday (22th March) at **5 pm UTC**. Berivan Isik (Stanford) will discuss "Sparse Random Networks for Communication-Efficient Federated Learning." Register to our mailing list:

0

6

1

11

Berivan Isik

@BerivanISIK

1 year

@ekrem_imamoglu Hatay Antakya, Emlakbank Evleri 1. Kisim 6-D , 6-B bloklari, hic kimseden haber alinamiyor. Musa Yuksekgonul, Behiye Yuksekgonul, Bahar Yuksekgonul. @istanbulbld @AFADHatay

0

1

9

Berivan Isik

@BerivanISIK

3 months

@tianle_cai Very cool work! 💫 we have a NeurIPS 2023 workshop paper with a similar idea and observations. The delta between the finetuned and pretrained model is extremely compressible with quantization and even with simple magnitude-based sparsification:

0

10

Berivan Isik

@BerivanISIK

1 year

@cigdemtoker Hatay Antakya, Emlakbank Evleri 1. Kisim 6-D , 6-B bloklari, hic kimseden haber alinamiyor. Musa Yuksekgonul, Behiye Yuksekgonul, Bahar Yuksekgonul

0

10

8

Berivan Isik

@BerivanISIK

1 year

The codebase is open-sourced at: Let us know if you have any questions! @FrancescoPase

GitHub - BerivanIsik/sparse-random-networks: Implementation of the FedPM framework by the authors...

Implementation of the FedPM framework by the authors of the ICLR 2023 paper "Sparse Random Networks for Communication-Efficient Federated Learning". - BerivanIsik/sparse-random-networks

github.com

1

9

Berivan Isik

@BerivanISIK

1 year

The framework consists of a neural pre- and post-processor with a standard video codec between them. The networks are trained jointly to optimize a rate-distortion loss function with the goal of significantly improving over the standard codec in various compression scenarios. 2/3

1

0

8

Berivan Isik

@BerivanISIK

6 months

Spotlight talks on Fri Dec 15, InfoCog and @unireps workshops: An Information-Theoretic Understanding of Maximum Manifold Capacity Representations ()

0

1

8

Berivan Isik

@BerivanISIK

1 year

@miniapeur There is an (not very tight) upper bound on the output distortion when pruning a single connection that helps with adjusting layer-wise sparsity in a greedy manner:

3

0

8

Berivan Isik

@BerivanISIK

4 years

@alienofhere

0

7

Berivan Isik

@BerivanISIK

1 year

Speakers: Johannes Balle (Google), @jmhernandez233 (Cambridge), Hyeji Kim (UT Austin), Yan Lu (Microsoft), Aaron Wagner (Cornell), Tsachy Weissman (Stanford) Panelists: Ashish Khisti (UofT), @tivaro (Qualcomm), @george_toderici (Google), @RashmiKVinayak (CMU)

0

7

Berivan Isik

@BerivanISIK

10 months

And one preliminary study at the Efficient Foundation Models Workshop: GPT-Zip: Deep Compression of Finetuned Large Language Models:

0

6

Berivan Isik

@BerivanISIK

11 months

@FrancescoPase @DenizGunduz1 @sanmikoyejo We show the existence of highly natural choices of pre-data distribution (side information at the server) and post-data distribution (local updates at the clients) in FL that we can use to reduce the communication cost significantly -- up to 50 times more than the baselines. 2/3

1

0

6

Berivan Isik

@BerivanISIK

1 year

Compared to other neural video compression methods, the sandwich framework is much more efficient as it requires pre- and post-processors formed by modestly-parameterized, lightweight networks. Joint work with Philip A. Chou, Onur Guleryuz, Danhang Tang, and Jonathan Taylor. 3/3

0

6

Berivan Isik

@BerivanISIK

4 months

TLDR: The size of the finetuning dataset and the distribution alignment between the pretraining and downstream data significantly influence the scaling behavior. 3/6

1

0

6

Berivan Isik

@BerivanISIK

3 years

Come say hi during our poster sessions if you're interested: Monday 12:30-2:30 pm PST (WiML) Wednesday 4-5 am PST (WiML) Saturday 5-6:30 pm PST (DL-IG)

1

0

6

Berivan Isik

@BerivanISIK

1 year

@mertyuksekgonul @OguzhanUgur @BabalaTv @dayagiyedin @haluklevent @nesibekiris @DepremDairesi @AFADTurkiye @ahbap @berkcanguven @sokakkedisitv @akplisinyani @bosunatiklama @ProfDemirtas

0

1

6

Berivan Isik

@BerivanISIK

3 years

We also developed a novel model compression method (called SuRP), guided by this information-theoretic formulation, which indeed outputs a sparse model without an explicit pruning step.

1

0

5

Berivan Isik

@BerivanISIK

2 years

We propose Federated Probabilistic Mask Training (FedPM) that does not update the randomly initialized weights at all. Instead, FedPM freezes the weights at their initial random values and learns how to sparsify the random network for the best performance. 2/6

1

0

5

Berivan Isik

@BerivanISIK

11 months

@FrancescoPase @DenizGunduz1 @sanmikoyejo We also show how to adaptively adjust the bitrate across the model parameters and training rounds to achieve the fundamental communication cost -- the KL divergence between the pre-data and post-data distributions. 3/3

0

5

Berivan Isik

@BerivanISIK

3 years

Check out our preprint for more details: Registration to @sparsenn workshop is free:

SNN Workshop 2021

Call for Papers/Abstract

sites.google.com

1

0

5

Berivan Isik

@BerivanISIK

3 years

We derived the information-theoretical limit of model compression and showed that this limit can only be achieved when the reconstructed model is sparse (pruned).

1

0

4

Berivan Isik

@BerivanISIK

6 months

@fluffykittnmeow @RylanSchaeffer @YuanqiD @vclecomte @sanmikoyejo @ziv_ravid @Andr3yGR @KhonaMikail @ylecun Sorry about that! We’ll put the paper on arxiv very soon. For now, you can check the workshop paper here:

0

5

Berivan Isik

@BerivanISIK

10 months

I will present two papers at the Federated Learning Workshop: 1) Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimation: 2) Communication-Efficient Federated Learning through Importance Sampling:

1

0

5

Berivan Isik

@BerivanISIK

24 days

- [ICLR DMFM & ME-FoMo] Scaling Laws for Downstream Task Performance of Large Language Models: - [ICLR SeT LLM, Me-FoMo, R2-FM, PML4LRS] On Fairness Implications and Evaluations of Low-Rank Adaptation of Large Models: 2/2

0

2

4

Berivan Isik

@BerivanISIK

8 months

We study the mean estimation problem under communication and local differential privacy constraints. As opposed to the order-optimal solutions in prior work,we characterize exact optimality conditions and develop an algorithm that is exact-optimal for a large family of codebooks.

0

4

Berivan Isik

@BerivanISIK

1 year

@abeirami @savvyRL And there is a simple way to boost the student’s performance by pruning the teacher network before distilling (which acts as a regularizer):

Prune Your Model Before Distill It

Knowledge distillation transfers the knowledge from a cumbersome teacher to a small student. Recent results suggest that the student-friendly teacher is more appropriate to distill since it...

arxiv.org

0

4

Berivan Isik

@BerivanISIK

2 years

To this end, the clients collaborate in training a stochastic binary mask to find the optimal sparse random network within the original one. At the end of the training, the final model is a sparse network with random weights – or a subnetwork inside the dense random network. 3/6

1

0

4

Berivan Isik

@BerivanISIK

2 years

Throughout the manuscript, we highlighted the advantages of having a stochastic mask training approach rather than a deterministic one in terms of accuracy, bitrate, and privacy. 5/6

1

0

4

Berivan Isik

@BerivanISIK

2 years

FedPM reduces the communication cost to less than 1 bit per parameter (bpp), reaches higher accuracy with faster convergence than the relevant baselines, outputs a final model with size less than 1 bpp, and can potentially amplify privacy. 4/6

1

0

4

Berivan Isik

@BerivanISIK

2 years

Joint work with @FrancescoPase , @DenizGunduz1 , Tsachy Weissman, and Michele Zorzi. 6/6

0

4

Berivan Isik

@BerivanISIK

4 months

This highlights the importance of studying downstream performance metrics and not making decisions solely based on cross-entropy! 6/6

0

4

Berivan Isik

@BerivanISIK

2 years

Suriye iç savaşında kadın olmak👇 @nerdesineko 👏👏

ekin 🌾

@nerdesineko

2 years

Suriye İç Savaşı'nda Kadın Olmak — Elif Ekin Doğan

0

1

2

0

1

4

Berivan Isik

@BerivanISIK

1 year

We use an analog storage technology (PCM) as an example to show that the noise added by the PCM cells is detrimental to the performance of neural networks and that we can recover full accuracy with our robust coding strategies. 2/3

1

0

4

Berivan Isik

@BerivanISIK

2 years

My cousin’s new article on “Humanitarian intervention and the Libya case” 👇 Follow @eliffekindogan1 for more 👀

Ekin Doğan

@eliffekindogan1

2 years

Here my new post about humanitarian intervention and the Libya Case⏬

0

1

0

1

3

Berivan Isik

@BerivanISIK

1 year

@miniapeur There is also by @utkuevci that empirically compares different sparsity distributions.

1

0

3

Berivan Isik

@BerivanISIK

1 year

@miniapeur We extend this to a more general case beyond single-connection pruning in Theorem 1 here:

2

0

3

Berivan Isik

@BerivanISIK

4 months

However, there are also cases where moderate misalignment causes the BLEU score to fluctuate or get worse with more pretraining, whereas downstream cross-entropy monotonically improves. 5/6

2

0

3

Berivan Isik

@BerivanISIK

2 years

Our results, including the algorithm, are also applicable to gradient compression for communication-efficient federated learning. 6/6

0

3

Berivan Isik

@BerivanISIK

4 months

With sufficient alignment, both downstream cross-entropy and BLEU score improve monotonically with more pretraining data. In such cases, we show that it is possible to predict the downstream BLEU score with good accuracy using a log-law. 4/6

1

3

Berivan Isik

@BerivanISIK

4 months

We study the scaling behavior in a transfer learning setting, where LLMs are finetuned for translation tasks, and investigate how the choice of the pretraining data and its size affect downstream performance as judged by two metrics: downstream cross-entropy and BLEU score. 2/6

1

0

3

Berivan Isik

@BerivanISIK

1 year

Our strategies can be applied in other scenarios beyond this example that add noise to the neural network weights. If you're not interested in the analog storage part, but want to learn how to make neural networks more robust to noise in general, you can skip Section 2. 3/3

0

3

Berivan Isik

@BerivanISIK

2 years

We investigated the theoretical tradeoff between the compression ratio and output perturbation of neural network models and found out that the rate-distortion theoretic formulation introduces a theoretical foundation for pruning. 2/6

1

0

3

Berivan Isik

@BerivanISIK

1 year

@dayagiyedin

Mert Yuksekgonul

@mertyuksekgonul

1 year

CEBRAİL MAHALLESİ EMLAK BANK EVLERİ 1.KISIM B3 (6D) BLOK 5. KAT ANTAKYA MUSA YÜKSEKGÖNÜL HALA ENKAZ ALTINDA. AYNI KATTAN 2 INSAN ÇIKARILDI. ACİLEN ÜST KATMANLARIN AÇILIP ALTLARA DOĞRU YOL ALINMASI GEREKİYOR. YARDIMA İHTİYACIMIZ VAR. LÜTFEN SESİMİZİ DUYUN

52

804

387

0

3

2

Berivan Isik

@BerivanISIK

1 year

@abeirami @merterden97 I think it also has some confusion on the ethical front. One of the responses i got was “i am sorry I can’t make offensive jokes” 😄

1

0

2

Berivan Isik

@BerivanISIK

1 year

@KaanSekbann @kaanalpyay @kilicdarogluk @ekrem_imamoglu @mansuryavas23 @herkesicinCHP @nevsinmengu @MuratYetkin2

0

2

Berivan Isik

@BerivanISIK

2 years

Joint work with Phil Chou, Sung Jin Hwang, @nmjohn , and @george_toderici .

0

2

Berivan Isik

@BerivanISIK

2 years

We also developed a novel model compression method (called SuRP), guided by this information-theoretic formulation, which indeed outputs a sparse model without an explicit pruning step. 5/6

1

0

2

Berivan Isik

@BerivanISIK

3 years

@alienofhere @busevurdu @di_dar_1505 first name ifsa oldu :p

0

2

Berivan Isik

@BerivanISIK

1 year

@abeirami @savvyRL Not sure how much this translates to transformer->MLP distillation, but there is prior work that showed poor performance of the student network when student and teacher architectures are too different, e.g.:

1

0

2

Berivan Isik

@BerivanISIK

4 years

@busevurdu Tatlış 🙈😘

0

2

Berivan Isik

@BerivanISIK

4 years

@alienofhere

0

2

Berivan Isik

@BerivanISIK

1 year

@mentionnedemek @DepremDairesi @ahbap @OguzhanUgur @burcuuyener @haluklevent @berkcanguven @dayagiyedin @akplisinyani @bosunatiklama @barisatay

0

2

Berivan Isik

@BerivanISIK

1 year

@harun_tekin Lütfen yardım edin.

Mert Yuksekgonul

@mertyuksekgonul

1 year

CEBRAİL MAHALLESİ EMLAK BANK EVLERİ 1.KISIM B3 (6D) BLOK 5. KAT ANTAKYA MUSA YÜKSEKGÖNÜL HALA ENKAZ ALTINDA. AYNI KATTAN 2 INSAN ÇIKARILDI. ACİLEN ÜST KATMANLARIN AÇILIP ALTLARA DOĞRU YOL ALINMASI GEREKİYOR. YARDIMA İHTİYACIMIZ VAR. LÜTFEN SESİMİZİ DUYUN

52

804

387

0

2

Berivan Isik

@BerivanISIK

8 months

@AirbnbHelp There is still no progress! It has been 4 days.

2

1

2

Berivan Isik

@BerivanISIK

2 years

We show that, when appropriately matching the lossy compression to the distribution of the added noise, the compressed examples converge, in distribution, to that of the noise-free training data. 3/4

1

0

1

Berivan Isik

@BerivanISIK

3 years

@varchanaiyer_ @csoham_ @sarahookr @savvyRL 🙏🏼

0

1

Berivan Isik

@BerivanISIK

1 year

@OguzhanUgur @haluklevent @DepremDairesi @ahbap

ekin 🌾

@nerdesineko

1 year

SAATLERDİR ENKAZ ALTINDA NOLUR YARDIM EDİN ARTIK!

1

10

5

0

1

Berivan Isik

@BerivanISIK

2 years

Our framework comprises noise injection followed by lossy compression. The noise injection step prevents user information from being leaked during learning, while lossy compression reduces the cost of storing/transmitting the data. 2/4

1

0

1

Berivan Isik

@BerivanISIK

7 months

@abeirami @GoogleAI @Google Thank you, Ahmad! It feels great to be recognized 🥹

1

0

1