Google Scholar page HERE.

Papers

Multi-Agent Learning and Control

  • Last-Iterate Guarantees for Learning in Co-coercive Games
    S. Chandak, R. Tamizholi and N. Bambos
    IEEE Conference on Decision and Control (CDC), 2026, to be presented
    [arXiv]

  • Choose Your Battles: Distributed Learning Over Multiple Tug of War Games
    S. Chandak, I. Bistritz and N. Bambos
    IEEE Transactions on Automatic Control (TAC), 2026
    [paper] [arXiv]

  • Learning to Control Unknown Strongly Monotone Games
    S. Chandak, I. Bistritz and N. Bambos
    IEEE Transactions on Control of Network Systems (TCNS), 2026
    [paper] [arXiv] [slides - EPFL Seminar]

  • Tug of Peace: Distributed Learning for Quality of Service Guarantees
    S. Chandak, I. Bistritz and N. Bambos
    IEEE Conference on Decision and Control (CDC), 2023
    [paper] [full version] [slides]

  • Equilibrium Bandits: Learning Optimal Equilibria of Unknown Dynamics
    S. Chandak, I. Bistritz and N. Bambos
    International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2023
    [paper] [arXiv] [slides] [poster]

  • Learning to Speak on Behalf of a Group: Medium Access Control for Sending a Shared Message
    S. U. Haque, S. Chandak, F. Chiariotti, D. Günduz and P. Popovski
    IEEE Communications Letters, 2022
    [paper] [arXiv]

Stochastic Approximation, Reinforcement Learning, and Optimization

Algorithms and Frameworks

  • Policy Gradient Methods for Non-Markovian Reinforcement Learning
    A. Kar, S. Chandak, R. Singh, S. Sinhahajari, E. Moulines, S. Bhatnagar, and N. Bambos
    Preprint
    [arXiv]

  • Nonexpansive Mappings in Two-Time-Scale Stochastic Approximation: Finite-Time Analysis
    S. Chandak
    SIAM Journal on Control and Optimization, 2026
    [paper] [arXiv] [slides - INFORMS APS 2025]

  • Reinforcement Learning in Non-Markovian Environments
    S. Chandak, P. Shah, V. S. Borkar and P. Dodhia
    Systems and Control Letters, 2024
    [paper] [arXiv]

  • Prospect-theoretic Q-learning
    V. S. Borkar and S. Chandak
    Systems and Control Letters, 2021
    [paper] [arXiv] [slides]

Finite-Time Guarantees

  • Concentration and Mean-Square Bounds for Contractive Stochastic Approximation: A Unified Elementary Approach
    S. Chandak
    Submitted to Mathematics of Control, Signals, and Systems
    [arXiv]

  • Regret and Sample Complexity of Online Q-Learning via Concentration of Stochastic Approximation with Time-Inhomogeneous Markov Chains
    R. Singh, S. Chandak, E. Moulines, V. S. Borkar and N. Bambos
    Preprint
    [arXiv]

  • Heavy-Tailed and Long-Range Dependent Noise in Stochastic Approximation: A Finite-Time Analysis
    S. Chandak, A. Yadav, A. Ozgur, and N. Bambos
    Submitted to IEEE Transactions on Automatic Control (TAC)
    [arXiv]

  • High-Probability Bounds for SGD under the Polyak-Łojasiewicz Condition with Markovian Noise
    A. Kar, S. Chandak, R. Singh, E. Moulines, S. Bhatnagar, and N. Bambos
    Submitted to SIAM Journal on Optimization
    [arXiv]

  • O(1/k) Finite-Time Bound for Non-Linear Two-Time-Scale Stochastic Approximation
    S. Chandak
    IEEE Transactions on Automatic Control (TAC), 2026
    [paper] [arXiv] [slides - INFORMS Annual Meeting 2025]

  • Finite-Time Bounds for Two-Time-Scale Stochastic Approximation with Arbitrary Norm Contractions and Markovian Noise
    S. Chandak, S. U. Haque, and N. Bambos
    IEEE Conference on Decision and Control (CDC), 2025
    [paper] [arXiv] [slides]

  • A Concentration Bound for TD(0) with Function Approximation
    S. Chandak and V. S. Borkar
    Stochastic Systems, 2026
    [paper] [arXiv]

  • A Concentration Bound for LSPE(λ)
    S. Chandak, V. S. Borkar and H. Dolhare
    Systems and Control Letters, 2023
    [paper] [arXiv]

  • Concentration of Contractive Stochastic Approximation and Reinforcement Learning
    S. Chandak, V. S. Borkar and P. Dodhia
    Stochastic Systems, 2022
    [paper] [slides]

Applications of Dynamic Programming

  • Safe Learning of Adaptive Control Policies for Remote Patient Monitoring
    R. Tamizholi, S. Chandak, I. Thapa, N. Bambos and D. Scheinker
    IEEE Healthcom, 2026
    [arXiv] [slides]

  • Optimal Control for Remote Patient Monitoring with Multidimensional Health States
    S. Chandak, I. Thapa, N. Bambos and D. Scheinker
    IEEE International Conference on Communications (ICC), 2025
    [paper] [arXiv] [slides]

  • Tiered Service Architecture for Remote Patient Monitoring
    S. Chandak, I. Thapa, N. Bambos and D. Scheinker
    IEEE Healthcom, 2024
    [paper] [arXiv] [slides]

  • Hidden Markov Model-Based Encoding for Time-Correlated IoT Sources
    S. Chandak, F. Chiariotti and P. Popovski
    IEEE Communications Letters, 2021
    [paper] [arXiv]

Ph.D. Thesis

  • Learning to Control Unknown Multi-Agent Systems for Guided Distributed Intelligence
    Ph.D. Thesis, Department of Electrical Engineering, Stanford University, 2026
    [thesis] [defense slides]

Invited Talks

  • Learning to Control Unknown Strongly Monotone Games

    • AI Cities Workshop, New York University, August 2026
    • Seminar at Automatic Control Lab, EPFL, March 2026

    [slides]

  • Optimal Control for Tiered Remote Patient Monitoring

    • INFORMS Healthcare Conference, July 2026

    [slides]

  • O(1/k) Finite-Time Bound for Non-Linear Two-Time-Scale Stochastic Approximation

    • INFORMS Annual Meeting, October 2025

    [slides]

  • Learning to Control Unknown Multi-Agent Systems

    • Department of Electrical Engineering Seminar, IIT Bombay, September 2025
    • STCS Seminar, Tata Institute of Fundamental Research (TIFR) Mumbai, September 2025

    [slides] [video - TIFR]

  • Non-Expansive Mappings in Two-Time-Scale Stochastic Approximation

    • INFORMS Applied Probability Society Conference, July 2025

    [slides]