MaskToM: Spatial-Guided Perspective-Taking for N-ToM Reasoning

Neural Theory-of-Mind (N-ToM), neural networks' capabilities of keeping track of others' mental states and conducting reasoning tasks, is at the core of building efficient and effective communication agents. The crux of N-ToM is the task of perspective-taking, which refers to the capability of comprehending and experiencing a story through the key hole of a specific character. Previous methods have proposed to facilitate LLMs' perspective-taking capabilities through documenting characters' perceptions through a symbolic graph (SymbolicToM) or through prompting (SimToM). However, these methods suffer from lack of generalizability as the SymbolicToM approach is only applicable to a specific dataset and lack of applicability in high-order N-ToM reasoning as the SimToM approach only reduce the difficulty of ToM reasoning by one order (perspective-taking for only one round). To better facilitate LLMs' N-ToM reasoning capabilities, we propose MaskToM. Studies in cognitive science have demonstrated the significance of spatial information in perspective-taking. Therefore, we leverage spatial information as an inductive bias to guide LLMs to conduct perspective taking. In addition, we make the perspective-taking process to be fully symbolic, which allows for aggregation of perspectives from multiple characters and improves LLMs' high-order N-ToM reasoning capabilities. On top of that, we trained a neural knowledge graph capable of world state tracking. We leverage the entity state information provided by this world model as additional context to alleviate LLMs' burden of keeping track of world state changes when conducting N-ToM reasoning.