Attention is All you Need
The Transformer uses attention to model sequences without recurrent or convolutional networks.
Machine Learning
NeurIPS
Conference on Neural Information Processing Systems
DEADLINES
Conference editions
PAPERS
Papers by year
1000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilities
Official Best Paper · SourceStudies network depth in self-supervised goal-conditioned reinforcement learning. Locomotion and manipulation experiments examine how deep residual architectures and training scale affect goal-reaching behavior.
Why Diffusion Models Don’t Memorize: The Role of Implicit Dynamical Regularization in Training
Official Best Paper · SourceSeparates the onset of useful diffusion generation from later training-data memorization. Experiments and a random-features model analyze how training dynamics create a window of generalization before memorization.
Stochastic Taylor Derivative Estimator: Efficient amortization for arbitrary differential operators
Official Best Paper · SourceAddresses the cost of high-order differential operators in neural networks. Taylor-mode automatic differentiation and randomized tensor contractions estimate operators for applications such as physics-informed neural networks.