I am a PostDoc at UCSB working with Professor Yao Qin.

I obtained my PhD from Brown University, working on Reinforcement Learning at the Intelligent Robot Lab advised by Professor George Konidaris. Prior to this I obtained a masters degree from University of Massachusetts, Amherst and had the privilege of working with Professor Phil Thomas. I graduated with a Bachelors of Technology in computer science at IIT Bombay.

I work on how robotic agents can continually learn to control in an enormous world. A robot acting in the physical world faces a vast space of sensor readings, body configurations, environment states, and actions, along with a stream of ever-changing tasks. My research starts from the observation that this world is enormous but not arbitrary: the relevant variation is structured. I exploit that structure at two levels:

1. Low-dimensional latent structure in real-world data. A picture of a robotic arm has tens of thousands of pixels, but it can be described by its underlying pose, and far fewer poses are viable than possible. This is the manifold hypothesis, which I established theoretically for deep learning and proved for the trajectories generated by deep RL agents.
2. Macroscopic structure in the learning dynamics of large neural networks. I developed a two-timescale view of RL, where the environment state evolves in physical time while the neural-network controller evolves in learning time, and derived the first learning dynamics for deep RL agents in continuous control. This turns deep RL from a black-box training process into an analyzable dynamical system.

Together these form an effective theory of deep RL in the continuum, one that already yields practical gains: from control from pixels via architecture-optimizer co-design, all the way up to 4-billion-parameter models. I am now extending this framework from sample-efficient learning to reliable and continual learning: predicting, before deployment, how an agent trained in simulation will perform in the real world, and keeping agents plastic, able to keep learning, as they face a stream of new tasks.

My framework aims to transfer performance predictions from simulation (left) to the real world (right) by providing reliable confidence intervals on how learning progresses as a function of the number of samples.

I believe theory should move towards practice and practice should move towards theory in Deep RL for true progress. For details see the list of my publications.