Artificial networks have been studied through the prism of statistical mechanics as disordered systems since the 80s, starting from the simple models of Hopfield's associative memory and the...

Nature Physics - The phrase ‘arrow of time’ refers to the asymmetry in the flow of events. A machine learning algorithm trained to infer its direction identifies entropy production as...

An extensive body of empirical research has revealed remarkable regularities in the acquisition, organization, deployment, and neural representation of human semantic knowledge, thereby raising a...

Pruning the parameters of deep neural networks has generated intense interest due to potential savings in time, memory and energy both during training and at test time. Recent works have...

The intertwined processes of learning and evolution in complex environmental niches have resulted in a remarkable diversity of morphological forms. Moreover, many aspects of animal intelligence...

Nature Communications - CA1 neurons encode an animal's position within the environment. Here the authors identified in hippocampal neuronal activity a detrimental type of noise that limits...

Pretrained transformers exhibit the remarkable ability of in-context learning (ICL): they can learn tasks from just a few examples provided in the prompt without updating any weights. This raises...

This conference will zero in on the latest research on cognitive science, neuroscience, vision, language, and thought, informing the pursuit of artificial intelligence.

We study the behavior of untrained neural networks whose weights and biases are randomly distributed using mean field theory. We show the existence of depth scales that naturally limit the maximum...

Measuring the dimensionality of behavior

In this work, we reveal a strong implicit bias of stochastic gradient descent (SGD) that drives overly expressive networks to much simpler subnetworks, thereby dramatically reducing the number of...

It is well known that the initialization of weights in deep neural networks can have a dramatic impact on learning speed. For example, ensuring the mean squared singular value of a network's...

Given a random variable $latex {X}&fg=000000$ that takes on only finitely many values, we can define its Shannon entropy by the formula $latex \displaystyle H(X) := \sum_x \mathbf{P}(X=x) \log …

While contrastive approaches of self-supervised learning (SSL) learn representations by minimizing the distance between two augmented views of the same data point (positive pairs) and maximizing...