Communication and Control in Markov Decision Processes: From Effective Communication to Covert Manipulation

This project investigates the role of communication in Markov decision processes (MDPs) from both cooperative and adversarial perspectives. The first part focuses on effective communication for control, where the objective is not signal reconstruction but the successful completion of tasks by a receiver that also acts as an actuator on the environment. The challenge lies in designing optimal joint communication and action policies under communication constraints. This study begins with a single-actuator system, extends to multi-actuator settings, and culminates in learning-based methods that achieve effective communication without prior knowledge of system dynamics.

The second part addresses covert control in MDPs, examining how an adversary can degrade system performance while remaining undetected by subtly altering the original control policy. This problem is analyzed in fixed-horizon and sequential detection frameworks, and we develop online learning strategies that enable an adversary to remain covert while adapting its attack.

Together, these two parts offer a unified exploration of how communication shapes decision-making in MDPs, from enabling optimal coordination to understanding vulnerabilities to stealthy manipulation.