Saket Tiwari

Saket Tiwari

Researcher in reinforcement learning and robotic control

Posts by Collection

portfolio

publications

Natural Option Critic

Published in AAAI, 2019

We derive a practical natural gradient method for the option-critic framework in hierarchical reinforcement learning, exploiting the geometry of the policy parameter space to improve learning, an early instance of using structure in the learning process itself.

Recommended citation: Saket Tiwari, & Philip Thomas. (2019). "Natural Option Critic." AAAI 2019 https://arxiv.org/pdf/1812.01488.pdf

Effects of Data Geometry in Early Deep Learning

Published in NeurIPS, 2022

We show that when high-dimensional data lie near a low-dimensional manifold, the underlying geometry of the data manifold governs the expressive capacity of neural networks, the first step in exploiting latent structure in real-world data for deep learning.

Recommended citation: Saket Tiwari, & George Konidaris. "Effects of Data Geometry in Early Deep Learning." NeurIPS 2022 https://arxiv.org/abs/2301.00008

A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems

Published in Neural Networks Journal, 2023

A domain-agnostic framework and benchmarking methodology for characterizing lifelong learning systems: how to measure whether an agent continually adapts to a stream of tasks without losing what it has already learned.

Recommended citation: Megan M. Baker, Alexander New, Mario Aguilar-Simon, Ziad Al-Halah, Sébastien M. R. Arnold, Ese Ben-Iwhiwhu, Andrew P. Brna, Ethan Brooks, Ryan C. Brown, Zachary Daniels, Anurag Daram, Fabien Delattre, Ryan Dellana, Eric Eaton, Haotian Fu, Kristen Grauman, Jesse Hostetler, Shariq Iqbal, Cassandra Kent, Nicholas Ketz, Soheil Kolouri, George Konidaris, Dhireesha Kudithipudi, Erik Learned-Miller, Seungwon Lee, Michael L. Littman, Sandeep Madireddy, Jorge A. Mendez, Eric Q. Nguyen, Christine D. Piatko, Praveen K. Pilly, Aswin Raghavan, Abrar Rahman, Santhosh Kumar Ramakrishnan, Neale Ratzlaff, Andrea Soltoggio, Peter Stone, Indranil Sur, Zhipeng Tang, Saket Tiwari, Kyle Vedder, Felix Wang, Zifan Xu, Angel Yanguas-Gil, Harel Yedidsion, Shangqun Yu, Gautam K. Vallabha. (2023). "A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems" Neural Networks Volume 160, 2023. https://www.sciencedirect.com/science/article/abs/pii/S0893608023000072

Meta-Learning Parameterized Skills

Published in ICML, 2023

A method for meta-learning parameterized skills in reinforcement learning, allowing an agent facing a stream of related robotic control tasks to transfer knowledge from earlier tasks and adapt quickly to new ones.

Recommended citation: Haotian Fu, Shangqun Yu, Saket Tiwari, Michael Littman, George Konidaris. (2023). "Meta-Learning Parameterized Skills." ICML 2023 https://proceedings.mlr.press/v202/fu23f.html

Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces

Published in ICLR [ORAL: top 1.8% of submitted], 2025

We prove that the trajectories collected by deep RL agents lie near a low-dimensional manifold, establishing the manifold hypothesis for reinforcement learning in continuous state and action spaces, and exploit this structure to improve performance in high-dimensional dog and humanoid control environments.

Recommended citation: Saket Tiwari, Omer Gottesman, & George Konidaris. (2025). "Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces." ICLR 2025 https://openreview.net/pdf?id=AP0ndQloqR

Spectral Collapse Drives Loss of Plasticity in Deep Continual Learning

Published in ICML, 2026

For an agent facing a stream of new tasks, staying able to learn is as important as learning itself. We show that loss of plasticity in deep continual learning is preceded by Hessian spectral collapse, introduce tau-trainability as a unifying framework, and develop regularization enhancements that keep neural networks plastic as tasks change.

Recommended citation: Naicheng He, Kaicheng Guo, Arjun Prakash, Saket Tiwari, Ruo Yu Tao, Tyrone Serapio, Amy Greenwald, & George Konidaris. (2025). "Spectral Collapse Drives Loss of Plasticity in Deep Continual Learning." arXiv:2509.22335 https://arxiv.org/abs/2509.22335

From Ticks to Flows: Dynamics of Neural Reinforcement Learning in Continuous Environments

Published in ICLR, 2026

We derive the first learning dynamics for deep RL agents in continuous control using a two-timescale view: the environment evolves in physical time while the controller evolves in learning time. For a class of nonlinear control problems, five macroscopic variables form a closed dynamical system that describes how the induced state distribution, value estimate, and action evolve, turning deep RL from a black-box training process into an analyzable dynamical system whose learning progress can be predicted before deployment.

Recommended citation: Saket Tiwari, Tejas Kotwal, & George Konidaris. (2026). "From Ticks to Flows: Dynamics of Neural Reinforcement Learning in Continuous Environments." arXiv:2606.04275 https://arxiv.org/abs/2606.04275

Stiefel Manifold Optimization Accelerates Reinforcement Learning

Published in Finding the Frame Workshop, RLC, 2026

An architecture-optimizer co-design principle that exploits latent structure to improve robotic control from pixel observations. Constraining deep networks to the Stiefel manifold makes the learning dynamics of the high-dimensional optimization problem match those of the underlying low-dimensional latent-state problem, letting actor-critic methods leverage hidden structure, with gains from MuJoCo pixel-observation control all the way up to fine-tuning the 4-billion-parameter Qwen3-4B model.

Recommended citation: Saket Tiwari, Arjun Prakash, Tejas Kotwal, Yao Qin, Nora Ayanian, Amy Greenwald, & George Konidaris. (2026). "Stiefel Manifold Optimization Accelerates Reinforcement Learning." Finding the Frame Workshop at RLC 2026 https://openreview.net/pdf?id=AdJZCSpEPN

talks

teaching

Graduate TA for Sequential Decision Making Course

Graduate level course, Brown University, Computer Science, 2024

I designed course material and assignments for the sequential decision making course at Brown. This is the course on Reinforcement Learning at Brown. The course was taught by Ron Parr and I was one of two grad TAs in a class of 70 people. The course was well recieved and I personally recieved positive reviewes for my teaching.