Noam Razin

Computer Science PhD Candidate

Tel Aviv University

I am a PhD candidate in the School of Computer Science at Tel Aviv University, fortunate to be advised by Nadav Cohen.

My research interests broadly include the theoretical foundations and applications of modern machine learning. Through a combination of mathematical analysis and rigorous experimentation, I aim to develop theories that shed light on empirical phenomena, and bring forth principled methods for improving the reliability and performance of neural networks.

My research is generously supported by the  Apple Scholars in AI/ML and the Tel Aviv University Center for AI & Data Science PhD fellowships.

📢 Looking for a postdoc position to start in fall 2024!

Email: noamrazin (at) mail.tau.ac.il

🗞 News

Nov 23: New paper identifies a fundamental vanishing gradients problem when using reinforcement learning to finetune language models.
Sep 23: Two papers accepted to NeurIPS 2023: one on the ability of graph neural networks to model interactions and another on what makes data suitable for locally connected neural networks.
Sep 23: Interned at  Apple Machine Learning Research.

May 22: Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks accepted to ICML 2022.
📝 Check out this blog post for an overview.
Mar 22: Honored to receive the  2022 Apple Scholars in AI/ML PhD fellowship.
Oct 21: Honored to receive the Tel Aviv University Center for AI & Data Science excellence fellowship.
May 21: Implicit Regularization in Tensor Factorization accepted to ICML 2021.
📝 Check out this blog post for an overview.
Sep 20: Implicit Regularization in Deep Learning May Not Be Explainable by Norms accepted to NeurIPS 2020.
📝 Check out this blog post for an overview.

📄 Publications

* denotes equal contribution

Vanishing Gradients in Reinforcement Finetuning of Language Models

Noam Razin, Hattie Zhou, Omid Saremi, Vimal Thilak, Arwen Bradley, Preetum Nakkiran, Joshua Susskind, Etai Littwin

arXiv preprint arXiv:2310.20703, 2023

PDF

What Algorithms Can Transformers Learn? A Study in Length Generalization

Hattie Zhou, Arwen Bradley, Etai Littwin, Noam Razin, Omid Saremi, Joshua Susskind, Samy Bengio, Preetum Nakkiran

arXiv preprint arXiv:2310.16028, 2023

PDF

What Makes Data Suitable for a Locally Connected Neural Network? A Necessary and Sufficient Condition Based on Quantum Entanglement

Yotam Alexander*, Nimrod De La Vega*, Noam Razin, Nadav Cohen

Advances in Neural Information Processing Systems (NeurIPS), 2023

PDF Code

On the Ability of Graph Neural Networks to Model Interactions Between Vertices

Noam Razin, Tom Verbin, Nadav Cohen

Advances in Neural Information Processing Systems (NeurIPS), 2023

PDF Code Poster Video

Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks

Noam Razin, Asaf Maman, Nadav Cohen

International Conference on Machine Learning (ICML), 2022

PDF Code Poster Video Blog

Implicit Regularization in Tensor Factorization

Noam Razin*, Asaf Maman*, Nadav Cohen

International Conference on Machine Learning (ICML), 2021

PDF Code Poster Video Blog

Implicit Regularization in Deep Learning May Not Be Explainable by Norms

Noam Razin, Nadav Cohen

Advances in Neural Information Processing Systems (NeurIPS), 2020

PDF Code Poster Blog

RecoBERT: A Catalog Language Model for Text-Based Recommendations

Itzik Malkiel, Oren Barkan, Avi Caciularu, Noam Razin, Ori Katz, Noam Koenigstein

Findings of the Association for Computational Linguistics: EMNLP, 2020

PDF

Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding

Oren Barkan*, Noam Razin*, Itzik Malkiel, Ori Katz, Avi Caciularu, Noam Koenigstein

AAAI Conference on Artificial Intelligence (AAAI), 2020

PDF Code Poster

💬 Selected Talks

On the Ability of Graph Neural Networks to Model Interactions Between Vertices
Learning on Graphs and Geometry Reading Group, January 2023
Video Slides
Generalization in Deep Learning Through the Lens of Implicit Rank Lowering
ICTP Youth in High-Dimensions: Recent Progress in Machine Learning, High-Dimensional Statistics and Inference, June 2022
Video Slides
Generalization in Deep Learning Through the Lens of Implicit Rank Lowering
MPI MiS + UCLA Math Machine Learning Seminar, May 2022
Slides
Implicit Regularization in Tensor Factorization
The Hebrew University Machine Learning Club, Jerusalem, Israel, June 2021
Video Slides
Implicit Regularization in Deep Learning May Not Be Explainable by Norms
Tel Aviv University Machine Learning Seminar, Tel Aviv, Israel, May 2020
Slides