Beyond Expert Judgment: An Explainable Framework for Truth Discovery, Weak Supervision, and Learning-Based Ranking in Open-Source Intelligence Risk Identification

In open-source intelligence (OSINT) research, traditional risk identification methods reliant on expert scoring face growing challenges due to their high subjectivity, cost, and lack of scalability. This study aims to propose and validate an algorithmic framework that transcends expert judgment. Centered on truth discovery, weakly supervised learning, and learning-based ranking, it enables automated, explainable risk identification within complex, multi-source heterogeneous data. The study first constructs a hierarchical-quota sampling system, acquiring and deduplicating data from four source categories: institutional authorities, official statements, mainstream and international reports, and visual materials. Subsequently, a truth discovery algorithm estimates source credibility to replace expert weighting. Weakly supervised labeling functions generate initial annotations, which are then aggregated by generative models to form robust labels. Finally, a learning ranking model dynamically prioritizes risk trajectories, with explainability ensured through Explainable AI techniques (e.g., SHAP, Grad-CAM). Results demonstrate that this framework reliably identifies risk signals across multiple time windows and control conditions. The classifier achieves PR-AUC improvements exceeding expert baselines, with average absolute error in inflection point localization maintained below 1 hour. It exhibits high consistency and robustness across cross-domain datasets. The study concludes that algorithmic expert-scoring replacement not only excels in accuracy and efficiency but also significantly outperforms traditional models in transparency and reproducibility, offering a systematic, scalable, and cutting-edge approach for OSINT risk research.