The posterior over Bayesian neural network (BNN) parameters is extremely high-dimensional and non-convex. For computational reasons, researchers approximate this posterior using inexpensive...

Contribute to izmailovpavel/torch_swa_examples development by creating an account on GitHub.

This short video summarizes our NeurIPS'19 paper "A Simple Baseline for Bayesian Uncertainty in Deep Learning" (https://arxiv.org/abs/1902.02476)PyTorch Code...

OpenAI - Cited by 6,015 - Machine Learning - Deep Learning - Language Models - Reasoning - AI Alignment

Supporting code for the paper "Dangers of Bayesian Model Averaging under Covariate Shift" - izmailovpavel/bnn_covariate_shift

We present a new research direction for superalignment, together with promising initial results: can we leverage the generalization properties of deep learning to control strong models with weak...

Recently, flat-minima optimizers, which seek to find parameters in low-loss neighborhoods, have been shown to improve a neural network's generalization performance over stochastic and adaptive...

Convolutional Neural Networks define an exceptionally powerful class of models, but are still limited by the lack of ability to be spatially invariant to the input data in a computationally and...

On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification - activatedgeek/understanding-bayesian-classification

Contribute to izmailovpavel/spurious_feature_learning development by creating an account on GitHub.

Contribute to izmailovpavel/torch_swa_examples development by creating an account on GitHub.

Contribute to PolinaKirichenko/deep_feature_reweighting development by creating an account on GitHub.

The posterior over Bayesian neural network (BNN) parameters is extremely high-dimensional and non-convex. For computational reasons, researchers approximate this posterior using inexpensive...

Explore the morphology and dynamics of deep learning optimization processes and gradient descent with the A.I Loss Landscape project.

Models, Inference and AlgorithmsOctober 30, 2019Meeting: https://youtu.be/GXs9Pmp6IKQ?t=2620&list=PLlMMtlgw6qNjROoMNTBQjAcdx53kV50cSLoss Valleys, Uncertainty...

Flow based models such as Real NVP are an extremely powerful approach to density estimation. However, existing flow based models are restricted to transforming continuous densities over a...

Mode Connectivity and Fast Geometric Ensembles in PyTorch - timgaripov/dnn-mode-connectivity

Approximate Bayesian inference for neural networks is considered a robust alternative to standard training, often providing good performance on out-of-distribution data. However, Bayesian neural...

Bayesian Neural Networks (BNNs) place priors over the parameters in a neural network. Inference in BNNs, however, is difficult; all inference methods for BNNs are approximate. In this work, we...

Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs Figure 1: visualizat

Presently the most successful approaches to semi-supervised learning are based on consistency regularization, whereby a model is trained to be robust to small perturbations of its inputs and...

This class extends optimizers with Stochastic Weight Averaging (SWA).

The power of neural networks lies in their ability to generalize to unseen data, yet the underlying reasons for this phenomenon remain elusive. Numerous rigorous attempts have been made to explain...