Avatar

Noam Razin

Computer Science PhD Candidate

Tel Aviv University

 

I am a PhD candidate in the School of Computer Science at Tel Aviv University, fortunate to be advised by Nadav Cohen.

My research interests broadly include the theoretical foundations and applications of modern machine learning. I aim to develop theories that shed light on how neural networks work, as well as bring forth principled methods for improving their efficiency, reliability, and performance.

During the summer of 2023 I interned at the Apple Machine Learning Research group. My research is generously supported by the Apple Scholars in AI/ML and the Tel Aviv University Center for AI & Data Science PhD fellowships.

Email: noamrazin (at) mail.tau.ac.il

🗞 News

  • Feb 24: New paper characterizes how the implicit bias of policy gradient affects extrapolation to unseen initial states, in linear quadratic control.

  • Jan 24: Two papers accepted to ICLR 2024: one identifying a vanishing gradients problem when using reinforcement learning to finetune language models and another analyzing length generalization of Transformers.

  • Sep 23: Two papers accepted to NeurIPS 2023: one on the ability of graph neural networks to model interactions and another on what makes data suitable for locally connected neural networks.

  • Sep 23: Interned at Apple Machine Learning Research.

📄 Publications

* denotes equal contribution

Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial States
Noam Razin*, Yotam Alexander*, Edo Cohen-Karlik, Raja Giryes, Amir Globerson, Nadav Cohen
arXiv preprint arXiv:2402.07875, 2024
PDF Code
Vanishing Gradients in Reinforcement Finetuning of Language Models
Noam Razin, Hattie Zhou, Omid Saremi, Vimal Thilak, Arwen Bradley, Preetum Nakkiran, Joshua Susskind, Etai Littwin
International Conference on Learning Representations (ICLR), 2024
PDF Code
What Algorithms Can Transformers Learn? A Study in Length Generalization
Hattie Zhou, Arwen Bradley, Etai Littwin, Noam Razin, Omid Saremi, Joshua Susskind, Samy Bengio, Preetum Nakkiran
International Conference on Learning Representations (ICLR), 2024
PDF Code
What Makes Data Suitable for a Locally Connected Neural Network? A Necessary and Sufficient Condition Based on Quantum Entanglement
Yotam Alexander*, Nimrod De La Vega*, Noam Razin, Nadav Cohen
Advances in Neural Information Processing Systems (NeurIPS), 2023
PDF Code
On the Ability of Graph Neural Networks to Model Interactions Between Vertices
Noam Razin, Tom Verbin, Nadav Cohen
Advances in Neural Information Processing Systems (NeurIPS), 2023
PDF Code Poster Video
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks
Noam Razin, Asaf Maman, Nadav Cohen
International Conference on Machine Learning (ICML), 2022
PDF Code Poster Video Blog
Implicit Regularization in Tensor Factorization
Noam Razin*, Asaf Maman*, Nadav Cohen
International Conference on Machine Learning (ICML), 2021
PDF Code Poster Video Blog
Implicit Regularization in Deep Learning May Not Be Explainable by Norms
Noam Razin, Nadav Cohen
Advances in Neural Information Processing Systems (NeurIPS), 2020
PDF Code Poster Blog
RecoBERT: A Catalog Language Model for Text-Based Recommendations
Itzik Malkiel, Oren Barkan, Avi Caciularu, Noam Razin, Ori Katz, Noam Koenigstein
Findings of the Association for Computational Linguistics: EMNLP, 2020
PDF
Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding
Oren Barkan*, Noam Razin*, Itzik Malkiel, Ori Katz, Avi Caciularu, Noam Koenigstein
AAAI Conference on Artificial Intelligence (AAAI), 2020
PDF Code Poster

💬 Selected Talks

  • Analyses of Policy Gradient for Language Model Finetuning and Optimal Control
    MPI MiS + UCLA Math Machine Learning Seminar, March 2024
    Video Slides

  • Two Analyses of Modern Deep Learning: Graph Neural Networks and Language Model Finetuning
    Princeton Alg-ML Seminar, December 2023
    Slides

  • On the Ability of Graph Neural Networks to Model Interactions Between Vertices
    Learning on Graphs and Geometry Reading Group, January 2023
    Video Slides

  • Generalization in Deep Learning Through the Lens of Implicit Rank Lowering
    ICTP Youth in High-Dimensions: Recent Progress in Machine Learning, High-Dimensional Statistics and Inference, June 2022
    Video Slides

  • Generalization in Deep Learning Through the Lens of Implicit Rank Lowering
    MPI MiS + UCLA Math Machine Learning Seminar, May 2022
    Slides

  • Implicit Regularization in Tensor Factorization
    The Hebrew University Machine Learning Club, Jerusalem, Israel, June 2021
    Video Slides

  • Implicit Regularization in Deep Learning May Not Be Explainable by Norms
    Tel Aviv University Machine Learning Seminar, Tel Aviv, Israel, May 2020
    Slides

📝 Blog Posts

👨‍🏫 Teaching

  • Teaching Assistant for Foundations of Deep Learning (course #0368-3080), Tel Aviv University, 2021 to present