Abstract
Observed performance in collaborative decision systems often fails to reveal the true contribution of individual actions due to interaction dependencies. This study develops a counterfactual policy evaluation framework to better quantify action-level impact in multi-agent settings. The approach integrates doubly robust estimation with reinforcement learning to evaluate alternative decision trajectories without additional environment interaction. Experiments are conducted on 8,900 coordination tasks with varying interaction complexity. The proposed method improves policy evaluation accuracy by 26.1% and leads to a 19.4% increase in downstream policy performance when used for policy refinement. Moreover, coordination conflicts decrease by 23.7%, indicating more consistent collective behavior. These findings highlight the value of counterfactual reasoning in optimizing collaborative policies.
References
Gao, G., Gao, R., Gao, R., & Zhou, H. (2026). Engineering Analysis and Quantitative Research on the Platform-Based Evolution of Enterprise Communication Systems.
Uehara, M., Shi, C., & Kallus, N. (2022). A review of off-policy evaluation in reinforcement learning. arXiv preprint arXiv:2212.06355.
Xu, D., Liu, H., Qiu, D., & Ma, Q. (2026). Structured Modeling and Representation Methods for Post-Retrieval Inference Processes in Large Video Language Models.
Oosterhuis, H. (2023). Doubly robust estimation for correcting position bias in click feedback for unbiased learning to rank. ACM Transactions on Information Systems, 41(3), 1-33.
Xu, D., Chen, H., & Gui, H. (2026). Unified Online Estimation Method for SOC, SOH, and Power Capacity Considering Safety Boundary Consistency in Battery Management Systems.
Yeganeh, Y. T., Jafari, M., & Matta, A. (2025). Deep Active Inference Agents for Delayed and Long-Horizon Environments. arXiv preprint arXiv:2505.19867.
Jiao, Y., Zhao, B., Wang, A., & Shi, T. (2026). Construction and Empirical Study of a Modularized Teaching System for Art Courses Based on a Unified Training Pathway.
Gronauer, S., & Diepold, K. (2022). Multi-agent deep reinforcement learning: a survey. Artificial Intelligence Review, 55(2), 895-943.
Wang, Y., Chen, J., Wang, Y., & Yin, X. (2026). Application of Obtainable Biological Agent Characteristics in Efficacy Stratification of Oral Anti-Obesity Drugs.
Oroojlooy, A., & Hajinezhad, D. (2023). A review of cooperative multi-agent deep reinforcement learning. Applied Intelligence, 53(11), 13677-13722.
Zhang, Y., Gu, W., & Wang, J. (2026). Construction of Wind Farm Asset Health Index Based on Multi-Dimensional Indicators and Analytic Hierarchy Process and Its Correlation with Operational Performance. Authorea Preprints.
Fernandes, A. M., Santos, F. C., & Paiva, A. (2024). Counterfactual Thinking in Stochastic Dynamics of Cooperation. In ECAI 2024 (pp. 3493-3500). IOS Press.
Liu, S., & Yim, J. (2025). Research on Generative AI Creation Systems Based on Visual Language Modeling: Human-Machine Collaboration and Cognitive Feedback Mechanisms. Available at SSRN 6139770.
Pitis, S., Creager, E., Mandlekar, A., & Garg, A. (2022). Mocoda: Model-based counterfactual data augmentation. Advances in Neural Information Processing Systems, 35, 18143-18156.
Liu, H., Xu, D., Ma, Q., Xu, S., & Qiu, D. (2026). Memory Poisoning Propagation and Repair Mechanism in Multi-Agent Collaborative Environments.
Chandak, Y., Shankar, S., Bastian, N., da Silva, B., Brunskill, E., & Thomas, P. S. (2022). Off-policy evaluation for action-dependent non-stationary environments. Advances in Neural Information Processing Systems, 35, 9217-9232.
Jiao, Y., Wang, A., Zhao, B., & Shi, T. (2026). Quantitative Study on the Construction and Application Effectiveness of Graffiti Wall Painting Teaching Models in Public Space Contexts.
Hughes, L., Dwivedi, Y. K., Malik, T., Shawosh, M., Albashrawi, M. A., Jeon, I., ... & Walton, P. (2025). AI agents and agentic systems: A multi-expert analysis. Journal of Computer Information Systems, 65(4), 489-517.
Qiu, Y. (2024). Estimation of tail risk measures in finance: Approaches to extreme value mixture modeling. arXiv preprint arXiv:2407.05933.
Shimizu, T., & Forastiere, L. (2023, December). Doubly Robust Estimator for Off-Policy Evaluation with Large Action Spaces. In 2023 IEEE Symposium Series on Computational Intelligence (SSCI) (pp. 992-997). IEEE.
Zhang, Y., & Wang, J. (2026). Design and Implementation of a Computer-Aided Full Lifecycle Quality Management System for Wind Farms in Upgrades, Renovations, and Subcontractor Supervision.
Moons, K. G., Damen, J. A., Kaul, T., Hooft, L., Navarro, C. A., Dhiman, P., ... & van Smeden, M. (2025). PROBAST+ AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods. bmj, 388.
Wang, Y., Yin, X., Chen, J., & Wang, Y. (2026). Evidence-Based Study on Low-Burden Digital Phenotyping for Precision Screening of Oral Anti-Obesity Drug Efficacy.
World Health Organization. (2023). Mental Health Gap Action Programme (mhGAP) guideline for mental, neurological and substance use disorders. World Health Organization.

This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright (c) 2026 Jisoo Kim, Minho Park, Hyunwoo Lee (Author)