Noam Razin

Assistant Professor

Computer Science & AI Department, Bar-Ilan University

razinno [at] cs.biu.ac.il

I am an Assistant Professor in the Computer Science & AI Department at Bar-Ilan University. My research focuses on the fundamentals of deep learning and modern artificial intelligence (AI) systems. By combining mathematical analysis with systematic experimentation, I aim to develop theories that shed light on how modern AI works, identify potential failures, and yield principled methods for improving efficiency, reliability, and performance.

Previously, I was a postdoctoral fellow at Princeton University, hosted by Sanjeev Arora. Before that, I obtained my PhD in Computer Science at Tel Aviv University under the supervision of Nadav Cohen.

I am recruiting MSc and PhD students. Please see the note below if you are interested in working with me.

Research Approach

My group works on the theoretical foundations of deep learning and modern AI systems, with the goal of deriving actionable insights that inform practice. This type of research, which bridges theory and practice, typically involves both rigorous mathematical analysis and systematic experimentation. You can find more details in the materials on this website (papers, talks, etc.).

Mentorship

I am looking for highly motivated MSc and PhD students with a strong academic record to join us in this effort. I will be deeply invested in your growth as a researcher. Together, we will identify directions that excite you, navigate through challenges, and create space for the kind of open-ended exploration that leads to original and impactful work.

Reaching Out

If you are interested in joining, feel free to reach out via email. Please include your CV and grade transcripts so I can better understand your background.

Recent Research

Recently, I have been working on aspects of converting language models to useful AI systems (aka post-training), including failures of preference-based alignment [1] and policy gradient methods [2], what makes a good proxy reward function [3, 4], catastrophic forgetting [5], and reward model generalization [6].

News

Aug 2026
Joined the Computer Science & AI Department at Bar-Ilan University as an Assistant Professor.
May 2026
Our work on data poisoning for structured state space models has won the Israel AI Safety Research Prize. Congrats to Yonatan Slutzky and Yotam Alexander, who led the project!
Apr 2026
New paper categorizes imperfect proxy rewards according to their effect on policy gradient optimization. It highlights that, although incorrect rewards are conventionally viewed as harmful, they can also be benign or even beneficial!
Apr 2026
Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting was accepted to ICML 2026.
Jan 2026
Why is Your Language Model a Poor Implicit Reward Model? received a best paper runner-up award at the NeurIPS 2025 Reliable Machine Learning from Unreliable Data Workshop and was accepted to ICLR 2026!

Sep 2025
Two papers accepted to NeurIPS 2025: one provides an optimization perspective on what makes a good reward model for RLHF and another proves that the implicit bias of state space models (SSMs) can be poisoned with clean labels.
Jan 2025
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization was accepted to ICLR 2025.
Oct 2024
Honored to receive the Zuckerman and Israeli Council for Higher Education Postdoctoral Scholarships.
Sep 2024
Joined Princeton University as a Postdoctoral Fellow.
Aug 2024
New lecture notes on the theory (and surprising practical applications) of linear neural networks.
May 2024
Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial States was accepted to ICML 2024.
Jan 2024
Two papers accepted to ICLR 2024: one identifying a vanishing gradients problem when using reinforcement learning to finetune language models and another analyzing length generalization of Transformers.
Sep 2023
Two papers accepted to NeurIPS 2023: one on the ability of graph neural networks to model interactions and another on what makes data suitable for locally connected neural networks.
Sep 2023
Interned at Apple Machine Learning Research.
Mar 2023
Honored to receive the Deutsch Prize in Computer Science for PhD candidates.
May 2022
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks was accepted to ICML 2022. Check out this blog post for an overview.
Mar 2022
Honored to receive the 2022 Apple Scholars in AI/ML PhD fellowship.
Oct 2021
Honored to receive the Tel Aviv University Center for AI & Data Science excellence fellowship.
May 2021
Implicit Regularization in Tensor Factorization was accepted to ICML 2021. Check out this blog post for an overview.
Sep 2020
Implicit Regularization in Deep Learning May Not Be Explainable by Norms was accepted to NeurIPS 2020. Check out this blog post for an overview.

Publications

See also Google Scholar* indicates equal contribution

When Errors Can Be Beneficial: A Categorization of Imperfect Rewards for Policy Gradient

Shuning Shang*, Hubert Strauss*, Stanley Wei, Sanjeev Arora, Noam Razin

arXiv:2604.25872, 2026

PDF Code

Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting

Howard Chen, Noam Razin, Karthik Narasimhan, Danqi Chen

International Conference on Machine Learning (ICML), 2026

PDF Code

Why is Your Language Model a Poor Implicit Reward Model?

Noam Razin, Yong Lin, Jiarui Yao, Sanjeev Arora

International Conference on Learning Representations (ICLR), 2026

PDF Code Poster

What Makes a Reward Model a Good Teacher? An Optimization Perspective

Noam Razin, Zixuan Wang, Hubert Strauss, Stanley Wei, Jason D. Lee, Sanjeev Arora

Advances in Neural Information Processing Systems (NeurIPS), 2025

PDF Code Poster

The Implicit Bias of Structured State Space Models Can Be Poisoned With Clean Labels

Yonatan Slutzky*, Yotam Alexander*, Noam Razin, Nadav Cohen

Advances in Neural Information Processing Systems (NeurIPS), 2025

PDF Code

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization

Noam Razin, Sadhika Malladi, Adithya Bhaskar, Danqi Chen, Sanjeev Arora, Boris Hanin

International Conference on Learning Representations (ICLR), 2025

PDF Code Poster

Understanding Deep Learning via Notions of Rank

Noam Razin

arXiv:2408.02111 (PhD thesis), 2024

PDF

Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial States

Noam Razin*, Yotam Alexander*, Edo Cohen-Karlik, Raja Giryes, Amir Globerson, Nadav Cohen

International Conference on Machine Learning (ICML), 2024

PDF Code Poster

Vanishing Gradients in Reinforcement Finetuning of Language Models

Noam Razin, Hattie Zhou, Omid Saremi, Vimal Thilak, Arwen Bradley, Preetum Nakkiran, Joshua Susskind, Etai Littwin

International Conference on Learning Representations (ICLR), 2024

PDF Code Poster

What Algorithms Can Transformers Learn? A Study in Length Generalization

Hattie Zhou, Arwen Bradley, Etai Littwin, Noam Razin, Omid Saremi, Joshua Susskind, Samy Bengio, Preetum Nakkiran

International Conference on Learning Representations (ICLR), 2024

PDF Code

Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning

Nadav Cohen, Noam Razin

arXiv:2408.13767 (lecture notes), 2024

PDF

What Makes Data Suitable for a Locally Connected Neural Network? A Necessary and Sufficient Condition Based on Quantum Entanglement

Yotam Alexander*, Nimrod De La Vega*, Noam Razin, Nadav Cohen

Advances in Neural Information Processing Systems (NeurIPS), 2023

PDF Code

On the Ability of Graph Neural Networks to Model Interactions Between Vertices

Noam Razin, Tom Verbin, Nadav Cohen

Advances in Neural Information Processing Systems (NeurIPS), 2023

PDF Code Poster

Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks

Noam Razin, Asaf Maman, Nadav Cohen

International Conference on Machine Learning (ICML), 2022

PDF Blog Code

Implicit Regularization in Tensor Factorization

Noam Razin*, Asaf Maman*, Nadav Cohen

International Conference on Machine Learning (ICML), 2021

PDF Blog Code

Implicit Regularization in Deep Learning May Not Be Explainable by Norms

Noam Razin, Nadav Cohen

Advances in Neural Information Processing Systems (NeurIPS), 2020

PDF Blog Code

RecoBERT: A Catalog Language Model for Text-Based Recommendations

Itzik Malkiel, Oren Barkan, Avi Caciularu, Noam Razin, Ori Katz, Noam Koenigstein

Findings of the Association for Computational Linguistics: EMNLP, 2020

PDF

Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding

Oren Barkan*, Noam Razin*, Itzik Malkiel, Ori Katz, Avi Caciularu, Noam Koenigstein

AAAI Conference on Artificial Intelligence (AAAI), 2020

PDF Code

Selected Talks

What Makes a Good Proxy Reward Function for Language Model Post-Training?
HUJI Machine Learning Club · Jun 2026
Slides Video
Why is Your Language Model a Poor Implicit Reward Model?
Princeton Language and Intelligence Seminar · Oct 2025
Slides Video
Understanding and Overcoming Pitfalls in Language Model Alignment
EPFL AI Fundamentals Seminar · Sep 2025
Slides Video
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
Deep Learning: Classics and Trends Seminar · Jan 2025
Slides
Analyses of Policy Gradient for Language Model Finetuning and Optimal Control
MPI MiS + UCLA Math Machine Learning Seminar · Mar 2024
Slides Video

Two Analyses of Modern Deep Learning: Graph Neural Networks and Language Model Finetuning
Princeton Alg-ML Seminar · Dec 2023
Slides
On the Ability of Graph Neural Networks to Model Interactions Between Vertices
Learning on Graphs and Geometry Reading Group · Jan 2023
Slides Video
Generalization in Deep Learning Through the Lens of Implicit Rank Lowering
ICTP Youth in High-Dimensions: Recent Progress in Machine Learning, High-Dimensional Statistics and Inference · Jun 2022
Slides Video
Implicit Regularization in Tensor Factorization
The Hebrew University Machine Learning Club · Jun 2021
Slides Video
Implicit Regularization in Deep Learning May Not Be Explainable by Norms
Tel Aviv University Machine Learning Seminar · May 2020
Slides

Teaching

Current

Theoretical Foundations of Deep Learning
Lecturer · Bar-Ilan University · 2026–

Past

Fundamentals of Deep Learning
Guest Lecturer · Princeton University · 2025
Introduction to Reinforcement Learning
Guest Lecturer · Princeton University · 2025
First Steps in Research Honors Seminar
Guest Lecturer · Tel Aviv University · 2021–2024
Foundations of Deep Learning
Teaching Assistant · Tel Aviv University · 2021–2023