Wangfan Li

Automated and no-code ML tools make model building accessible but can obscure harms that arise when users include sensitive attributes. We embed micro-ethics nudges at key workflow moments and evaluate downstream fairness outcomes and user experience. In a between-subjects experiment (N=34) participants used a simplified AutoML web tool on a subset of the HMDA mortgage dataset with a 10-minute modeling task. The intervention combined in text notice when selecting sensitive attributes such as race, gender, ethnicity and post-training model explanation visualizations, while the base condition showed only model performance metrics. Participants in the Intervention group included significantly fewer sensitive features and produced models with substantially smaller equal-opportunity gaps, while System Usability Scale scores did not differ significantly across conditions. Moral acceptability did not significantly differ between conditions, though it trended lower under the intervention. We conclude that minimal, well timed fairness feedback can meaningfully reduce bias in rapid prototyping workflows. We also discuss design patterns for embedding fairness and the implications of increased moral sensitivity for tool adoption.

Iterative self-refinement is the dominant paradigm for improving LLM outputs without retraining, yet it lacks principled grounding for what to refine or in what order. We propose Prompt-Level Supervisory Alignment (PLSA), a framework that operationalizes Supervisory Control Theory (SCT), a cognitive framework for human oversight of automated systems, as a structured prompting strategy, and empirically evaluate whether theoretically-grounded prompt structure yields higher revision fidelity than matched iterative self-refinement. In a large-scale evaluation across ten venue-year combinations from three ML conference series (ICLR 2021-2025, NeurIPS 2021-2022 and 2024, CoRL 2021 and 2024), SCT-structured conditions produce revisions with significantly higher fidelity to actual author revisions than both a single-pass baseline and a matched two-pass self-refinement baseline that uses identical review information without SCT structure (all p < .001, medium-to-large effect sizes). All conditions maintain practically equivalent LLM-judge quality, and cross-model evaluation with Google Gemini 2.5 Flash-Lite corroborates condition rankings, confirming findings are not artifacts of generator self-preference. These results provide empirical evidence that theoretically-grounded prompt structure, not merely iterative refinement, is the operative variable driving higher revision fidelity.

Researchers increasingly use large language models (LLMs) to revise manuscripts in response to peer reviews, yet this adoption is largely unprincipled and risks the ironies of automation. Prompt-Level Supervisory Alignment (PLSA) applies Supervisory Control Theory as a structured prompting strategy for LLM-assisted manuscript revision. We extend PLSA’s planning function to multi-round peer review, where the planner must anticipate concerns that later reviewers will raise. We construct RevPlan-Bench, a corpus of multi-round manuscripts whose ground truth is the issues expert reviewers raised across three or more review cycles, and score 23,256 revision plans spanning five information conditions, three LLM backbones, and four prompting variants. First-round reviews substantially improve coverage of future concerns; multi-agent debate degrades performance; and explicit anticipation prompting, our central pre-registered hypothesis, adds no practical value beyond the reviews themselves, an informative null. Information design is the dominant lever; the researcher remains the final arbiter of scholarly claims.

As Generative Artificial Intelligence (AI) and Robotic Process Automation (RPA) tools become increasingly integrated into digital workflows, ensuring the usability of such automation before deployment on live systems is critical. This paper introduces a novel sandbox environment implemented within a virtual machine designed to safely test AI-driven task automation in isolation. The study evaluates user interactions with automated systems through two distinct feedback modalities: direct mouse control and text-based input, across tasks of varying difficulty levels (easy, medium, and hard) through measuring System Usability Scale, task completion rate and NASA TLX. The experiment introduces two primary independent variables: interaction modality and task difficulty. Interaction modality is categorized into direct mouse control by the user versus providing guidance to the AI through a chat interface. Task difficulty is divided into three levels-easy, medium, and hard; each presented sequentially to participants within their assigned interaction modality. A field experiment with 28 participants revealed that direct mouse control outperformed text-based feedback in task completion rates (83.3% vs. 61.9%) and usability. However, as task difficulty increased, user workload also rose significantly, regardless of the feedback modality. Qualitative analysis highlighted common barriers to effective interaction, such as delayed AI responses and frustration with error correction responsiveness. The study aims to identify patterns in user-AI collaboration dynamics, pinpoint challenges in the AI's autonomous decision-making, and assess the efficacy of the intervention methods. The findings are expected to inform the design of future human-centered AI systems that can effectively balance autonomy with user oversight in complex environments.

Recent developments in artificial intelligence (AI) have permeated through an array of different immersive environments, including virtual, augmented, and mixed realities. AI brings a wealth of potential that centers on its ability to critically analyze environments, identify relevant artifacts to a goal or action, and then autonomously execute decision-making strategies to optimize the reward-to-risk ratio. However, the inherent benefits of AI are not without disadvantages as the autonomy and communication methodology can interfere with the human's awareness of their environment. More specifically in the case of autonomy, the relevant human-computer interaction literature cites that high autonomy results in an "out-of-the-loop" experience for the human such that they are not aware of critical artifacts or situational changes that require their attention. At the same time, low autonomy of an AI system can limit the human's own autonomy with repeated requests to approve its decisions. In these circumstances, humans enter into supervisor roles, which tend to increase their workload and, therefore, decrease their awareness in a multitude of ways. In this position statement, we call for the development of human-centered AI in immersive environments to sustain and promote awareness. It is our position then that we believe with the inherent risk presented in both AI and AR/VR systems, we need to examine the interaction between them when we integrate the two to create a new system for any unforeseen risks, and that it is crucial to do so because of its practical application in many high-risk environments.

This paper introduces ReActIn, a framework designed to infuse human feedback into the intermediate prompting steps of large language models. The practicality and effectiveness of ReActIn are validated through experiments that apply four established prompting strategies, evaluated both with and without human feedback integration. The proposed architecture's performance is compared against traditional large language models across various tasks using four standard evaluation tests. Our findings reveal that the integration of human feedback has a direct impact on the reasoning, action prompting, and overall decision-making capabilities of the language models. This study underscores the potential of ReActIn to shape a future where sophisticated, context-aware AI systems, empowered by human feedback, can effectively navigate complex real-world scenarios.

This paper explores the potential of using Large Language Models (LLMs) to generate Business Process Model and Notation (BPMN) files as input for Robotic Process Automation (RPA) tools. The combination of AI-driven workflow generation with RPA aims to improve the automation of complex processes in both scientific and business domains. The proposed approach leverages the ability of LLMs to interpret natural language instructions and convert them into structured BPMN files, which can then be graphically edited using software like Camunda and executed by RPA tools. The feasibility of this approach is demonstrated through zero-shot prompt experiments using Anthropic Claude 3 Opus and GPT-4o. This solution has the potential to streamline the automation of workflows, reduce human error, and increase productivity across various industries.

In an era where the automation of tasks via Generative AI and Robotic Process Automation (RPA) tools has become increasingly prevalent, ensuring the safety and reliability of these automations before their deployment on live systems is paramount. This paper introduces a novel sandbox environment, implemented within a virtual machine, designed to serve as a testing ground for autonomous task executions powered by Large Language Models (LLMs) and Large Multimodal Models (LMMs). Our solution leverages a human-in-the-loop approach, wherein users validate and provide feedback on tasks executed within the sandbox, thereby informing iterative refinements to the models' internal prompts. This feedback loop facilitates the in-context learning of models, allowing them to adapt and prevent recurring errors in future task executions. We explore the effectiveness of this approach through user studies conducted in a controlled virtual environment, utilizing proprietary and open-source multimodal models. The study is structured around three distinct scenarios, each designed to evaluate the models' performance across different tasks and user interactions. Through this research, we aim to not only enhance the safety and efficacy of task automation technologies but also to contribute to the broader discourse on human oversight mechanisms in AI-driven systems. This work underscores the importance of sandbox environments in mitigating potential risks associated with deploying automated tasks in live environments, and highlights the role of human feedback in refining AI behaviors for optimal performance.

This paper introduces ReActIn, a framework designed to infuse human feedback into the intermediate prompting steps of large language models. The practicality and effectiveness of ReActIn are validated through experiments that apply four established prompting strategies, evaluated both with and without human feedback integration. The proposed architecture's performance is compared against traditional large language models across various tasks using four standard evaluation tests. Our findings reveal that the integration of human feedback has a direct impact on the reasoning, action prompting, and overall decision-making capabilities of the language models. This study underscores the potential of ReActIn to shape a future where sophisticated, context-aware AI systems, empowered by human feedback, can effectively navigate complex real-world scenarios.

In this paper, we propose SmartMonitor, a system that utilizes an edge hardware device to record, analyze and log user activity while not interfering with said activity. The analysis of the activity can help users identify the time they spend on different tasks and provide real-time feedback from a Large Language Model (LLM) to give better awareness of user activity. The SmartMonitor passes through the HDMI signal from the video card and analyzes the user’s activity on edge using two artificial intelligence models, logs user activity and sends the log for analysis to a Large Language Model for feedback. SmartMonitor enables non-intrusive self-quantifying technology that both records and analyzes user activity while protecting privacy and reducing the processing burden on the user’s device, which can serve as an excellent research framework for behavioral analysis in the workplace and a way to enhance user's work activity.

The rapid development and adoption of Generative AI (GAI) technology in the form of chatbots such as ChatGPT and Claude has greatly increased interest in agentic machines. This paper introduces the Autonomous Cognitive Entity (ACE) model, a novel framework for a cognitive architecture, enabling machines and software agents to operate more independently. Drawing inspiration from the OSI model, the ACE framework presents layers of abstraction to conceptualize artificial cognitive architectures. The model is designed to harness the capabilities of the latest generative AI technologies, including large language models (LLMs) and multimodal generative models (MMMs), to build autonomous, agentic systems. The ACE framework comprises six layers: the Aspirational Layer, Global Strategy, Agent Model, Executive Function, Cognitive Control, and Task Prosecution. Each layer plays a distinct role, ranging from setting the moral compass and strategic thinking to task selection and execution. The ACE framework also incorporates mechanisms for handling failures and adapting actions, thereby enhancing the robustness and flexibility of autonomous agents. This paper introduces the conceptual framework and proposes implementation strategies that have been tested and observed in industry. The goal of this paper is to formalize this framework so as to be more accessible.