Agent-in-the-Loop to Distill Expert Knowledge into Artificial Intelligence Models: A Survey

Large-scale neural networks have revolutionized many general knowledge areas (e.g., computer vision and language processing), but are still rarely applied in many expert knowledge areas (e.g., healthcare), due to data sparsity and high annotation expenses. Human-in-the-loop machine learning (HIL-ML) incorporates expert domain knowledge into the modeling process, effectively addressing these challenges.Recently, some researchers have started using large models to substitute for certain tasks typically performed by humans. Although large models have limitations in expert knowledge areas, after being trained on trillions of examples, they have demonstrated advanced capabilities in reasoning, semantic understanding, grounding, and planning. These capabilities can serve as proxies of human, which introduces new opportunities and challenges in HIL-ML area.Based on the above, we summarize a more comprehensive framework, Agent-in-the-Loop Machine Learning (AIL-ML), where agent represents both humans and large models. AIL-ML can efficiently collaborate human and large model to construct vertical AI models with lower costs.This paper presents the first review of recent advancements in this area. First, we provide a formal definition of AIL-ML and discuss its related fields. Then, we categorize the AIL-ML methods based on data processing and model development, providing formal definitions for each, and present representative works in detail for each category. Third, we highlight relative applications of AIL-ML. Finally, we summarize the current literature and highlight future research directions.