Supervisory Control Theory for LLM Revision
Proceedings of the ACM Conference on AI and Agentic Systems, 2026 · pp. 1100-1108
Abstract
Iterative self-refinement is the dominant paradigm for improving LLM outputs without retraining, yet it lacks principled grounding for what to refine or in what order. We propose Prompt-Level Supervisory Alignment (PLSA), a framework that operationalizes Supervisory Control Theory (SCT), a cognitive framework for human oversight of automated systems, as a structured prompting strategy, and empirically evaluate whether theoretically-grounded prompt structure yields higher revision fidelity than matched iterative self-refinement. In a large-scale evaluation across ten venue-year combinations from three ML conference series (ICLR 2021-2025, NeurIPS 2021-2022 and 2024, CoRL 2021 and 2024), SCT-structured conditions produce revisions with significantly higher fidelity to actual author revisions than both a single-pass baseline and a matched two-pass self-refinement baseline that uses identical review information without SCT structure (all p < .001, medium-to-large effect sizes). All conditions maintain practically equivalent LLM-judge quality, and cross-model evaluation with Google Gemini 2.5 Flash-Lite corroborates condition rankings, confirming findings are not artifacts of generator self-preference. These results provide empirical evidence that theoretically-grounded prompt structure, not merely iterative refinement, is the operative variable driving higher revision fidelity.
Cite this work
Wangfan Li and Carlos Toxtli-Hernández. 2026. Supervisory Control Theory for LLM Revision. Proceedings of the ACM Conference on AI and Agentic Systems. https://doi.org/10.1145/3786335.3813152
@inproceedings{Li_2026a,
series = {CAIS ’26},
title = {Supervisory Control Theory for LLM Revision},
url = {http://dx.doi.org/10.1145/3786335.3813152},
doi = {10.1145/3786335.3813152},
booktitle = {Proceedings of the ACM Conference on AI and Agentic Systems},
publisher = {ACM},
author = {Li, Wangfan and Toxtli, Carlos},
year = {2026},
month = May,
pages = {1100--1108},
collection = {CAIS ’26}
}
