Learning Symbolic Persistent Macro-Actions for POMDP Solving Over Time

Abstract

This paper proposes an integration of temporal logical reasoning and Partially Observable Markov Decision Processes (POMDPs) to achieve interpretable decision-making under uncertainty with macro-actions. Our method leverages a fragment of Linear Temporal Logic (LTL) based on Event Calculus (EC) to generate \emph{persistent} (i.e., constant) macro-actions, which guide Monte Carlo Tree Search (MCTS)-based POMDP solvers over a time horizon, significantly reducing inference time while ensuring robust performance. Such macro-actions are learnt via Inductive Logic Programming (ILP) from a few traces of execution (belief-action pairs), thus eliminating the need for manually designed heuristics and requiring only the specification of the POMDP transition model. In the Pocman and Rocksample benchmark scenarios, our learned macro-actions demonstrate increased expressiveness and generality when compared to time-independent heuristics, indeed offering substantial computational efficiency improvements.

Code

This repo contains code for learning temporal policy heuristics in logical formalism (answer set programming) with the event calculus formalism, and then solving the rocksample and the pocman (partially observable pacman) domains in DESPOT and POMCP.

• despot and pomcp folders contain C++ code for the two solvers.
• ILASP folder contains traces and files from rocksample and pocman, to generate the heuristics for the two tasks using Inductive Learning of Answer Set Programs (ILASP)

How to run

• POMCP: launch run.sh in pomcp/scripts to replicate experiments in the paper. Follow the comments in the code for parameter description. The output is a .xes file with statistics. You need to build the pomcp folder.
• DESPOT: launch run.sh in despot/examples/cpp_models/{pocman, rock_sample} to replicate experiments in the paper. Follow the comments in the code for parameter description. The output is a .txt file with statistics. You need to build despot, pocman, and rock_sample folders.
• ILASP: run the install_binaries.sh script and then check the dedicated README in the rocksample and pocman subfolders for more details.

Cite us

@article{veronese2025learning,
  title={Learning Symbolic Persistent Macro-Actions for POMDP Solving Over Time},
  author={Veronese, Celeste and Meli, Davide and Farinelli, Alessandro},
  journal={arXiv:2505.03668},
  year={2025},
  note={Accepted 19th Conf. Neurosymbolic Learning and Reasoning}
}