🗞 News

• Oct 24: New paper proving that the implicit bias of state space models (SSMs) can be poisoned with clean labels.

• Oct 24: When aligning language models via Direct Preference Optimization (DPO), past work observed that the probability of preferred responses often decreases. So where does the probability go? 🧐 In a new paper, we characterize the causes for this counter-intuitive phenomenon, show that it can lead to surprising failures in alignment, and provide preventative guidelines.

• Oct 24: Honored to receive the Zuckerman Postdoctoral Scholarship and the Israeli Council for Higher Education Postdoctoral Scholarship.

• Sep 24: Joined Princeton Language and Intelligence as a Postdoctoral Fellow.

• Aug 24: New lecture notes on the theory (and surprising practical applications) of linear neural networks.

• May 24: Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial States accepted to ICML 2024.

• Jan 24: Two papers accepted to ICLR 2024: one identifying a vanishing gradients problem when using reinforcement learning to finetune language models and another analyzing length generalization of Transformers.

• Sep 23: Two papers accepted to NeurIPS 2023: one on the ability of graph neural networks to model interactions and another on what makes data suitable for locally connected neural networks.

• Sep 23: Interned at Apple Machine Learning Research.

• Mar 22: Honored to receive the 2022 Apple Scholars in AI/ML PhD fellowship.

• Oct 21: Honored to receive the Tel Aviv University Center for AI & Data Science excellence fellowship.

💬 Selected Talks

• Analyses of Policy Gradient for Language Model Finetuning and Optimal Control
  MPI MiS + UCLA Math Machine Learning Seminar, March 2024
  Video Slides

• Two Analyses of Modern Deep Learning: Graph Neural Networks and Language Model Finetuning
  Princeton Alg-ML Seminar, December 2023
  Slides

• On the Ability of Graph Neural Networks to Model Interactions Between Vertices
  Learning on Graphs and Geometry Reading Group, January 2023
  Video Slides

• Generalization in Deep Learning Through the Lens of Implicit Rank Lowering
  ICTP Youth in High-Dimensions: Recent Progress in Machine Learning, High-Dimensional Statistics and Inference, June 2022
  Video Slides

• Generalization in Deep Learning Through the Lens of Implicit Rank Lowering
  MPI MiS + UCLA Math Machine Learning Seminar, May 2022
  Slides

• Implicit Regularization in Tensor Factorization
  The Hebrew University Machine Learning Club, Jerusalem, Israel, June 2021
  Video Slides

• Implicit Regularization in Deep Learning May Not Be Explainable by Norms
  Tel Aviv University Machine Learning Seminar, Tel Aviv, Israel, May 2020
  Slides

📝 Blog Posts

• Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Networks, Off the Convex Path, July 22

• Implicit Regularization in Tensor Factorization: Can Tensor Rank Shed Light on Generalization in Deep Learning?, Off the Convex Path, July 21

• Can Implicit Regularization in Deep Learning Be Explained by Norms? (by Nadav Cohen), Off the Convex Path, November 20

👨‍🏫 Teaching

• Teaching Assistant for Foundations of Deep Learning (course #0368-3080), Tel Aviv University, 2021 to 2023