Publications

Google Scholar page HERE.

• Papers
  • Multi-Agent Control
  • Two-Time-Scale Stochastic Approximation
  • Reinforcement Learning and Stochastic Approximation
  • Applications of Dynamic Programming
• Talks

Papers

Multi-Agent Control

• Choose Your Battles: Distributed Learning Over Multiple Tug of War Games
  S. Chandak, I. Bistritz and N. Bambos
  Submitted to IEEE Transactions on Automatic Control (TAC)
  [arXiv]

• Learning to Control Unknown Strongly Monotone Games
  S. Chandak, I. Bistritz and N. Bambos
  Submitted to IEEE Transactions on Control of Network Systems (TCNS)
  [arXiv]

• Tug of Peace: Distributed Learning for Quality of Service Guarantees
  S. Chandak, I. Bistritz and N. Bambos
  IEEE Conference on Decision and Control (CDC), 2023
  [paper] [full version] [slides]

• Equilibrium Bandits: Learning Optimal Equilibria of Unknown Dynamics
  S. Chandak, I. Bistritz and N. Bambos
  International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2023
  [paper] [arXiv] [slides] [poster]

• Learning to Speak on Behalf of a Group: Medium Access Control for Sending a Shared Message
  S. U. Haque, S. Chandak, F. Chiariotti, D. Günduz and P. Popovski
  IEEE Communications Letters, August 2022
  [paper]

Two-Time-Scale Stochastic Approximation

• O(1/k) Finite-Time Bound for Non-Linear Two-Time-Scale Stochastic Approximation
  Submitted to IEEE Transactions on Automatic Control (TAC)
  [arXiv]

• Finite-Time Bounds for Two-Time-Scale Stochastic Approximation with Arbitrary Norm Contractions and Markovian Noise
  S. Chandak, S. U. Haque, N. Bambos
  To be presented at IEEE Conference on Decision and Control (CDC), 2025
  [arXiv]

• Non-Expansive Mappings in Two-Time-Scale Stochastic Approximation: Finite Time Analysis
  S. Chandak
  Submitted to SIAM Journal on Control and Optimization
  [arXiv] [slides - INFORMS APS 2025]

Reinforcement Learning and Stochastic Approximation

• A Concentration Bound for TD(0) with Function Approximation
  S. Chandak and V. S. Borkar
  Submitted to Stochastic Systems
  [arXiv]

• Reinforcement Learning in Non-Markovian Environments
  S. Chandak, P. Shah, V. S. Borkar and P. Dodhia
  Systems and Control Letters, March 2024
  [paper] [arXiv]

• A Concentration Bound for LSPE($\lambda$)
  S. Chandak, V. S. Borkar and H. Dolhare
  Systems and Control Letters, January 2023
  [paper] [arXiv]

• Concentration of Contractive Stochastic Approximation and Reinforcement Learning
  S. Chandak, V. S. Borkar and P. Dodhia
  Stochastic Systems, December 2022
  [paper] [slides]

• Prospect-theoretic Q-learning
  V. S. Borkar and S. Chandak
  Systems and Control Letters, October 2021
  [paper] [arXiv] [slides]

Applications of Dynamic Programming

• Optimal Control for Remote Patient Monitoring with Multidimensional Health States
  S. Chandak, I. Thapa, N. Bambos and D. Scheinker
  IEEE International Conference on Communications (ICC), 2025
  [arXiv] [slides]

• Tiered Service Architecture for Remote Patient Monitoring
  S. Chandak, I. Thapa, N. Bambos and D. Scheinker
  IEEE Healthcom, 2024
  [paper] [arXiv] [slides]

• Hidden Markov Model-Based Encoding for Time-Correlated IoT Sources
  S. Chandak, F. Chiariotti and P. Popovski
  IEEE Communications Letter, May 2021
  [paper] [arXiv]

Talks

Invited Talks

• O(1/k) Finite-Time Bound for Non-Linear Two-Time-Scale Stochastic Approximation
  INFORMS Annual Meeting, October 2025
  [slides]

• Learning to Control Unknown Multi-Agent Systems
  • Department of Electrical Engineering Seminar, IIT Bombay, September 2025
  • STCS Seminar, Tata Institute of Fundamental Research (TIFR) Mumbai, September 2025
    [slides] [video - TIFR]
• Non-Expansive Mappings in Two-Time-Scale Stochastic Approximation
  INFORMS Applied Probability Society Conference, July 2025
  [slides]