Hidden danger identification and analysis algorithm combining multi-modal large model and knowledge enhancement

The criteria for assessing significant accident risks are crucial for guiding enterprises in fulfilling their primary safety production responsibilities, conducting self-assessments and reforms regarding safety hazards, and enabling regulatory and enforcement agencies to execute precise and effective law enforcement. Initially, the general knowledge of the multimodal big model is employed to augment the training samples, an instruction fine-tuning data set generated for meme-interpretation, followed by the utilization of unlabeled data for contrastive and embedding training to enhance the semantic representation capabilities of the text encoder. An algorithm for generating hidden risk judgments based on a multimodal large model is proposed, with the multimodal large model being fine-tuned using diverse instructional data, including knowledge augmentation, hidden danger content, and metaphor identification. Enhance the generative capacity of the large model to identify latent risks. Ultimately, the self-predicted iterative extension training is conducted on unlabeled data through the quick learning mechanism to enhance the model's adaptation to specific tasks. Experimental results on available datasets indicate that the proposed model outperforms the baseline method when the quantity of labeled data is equivalent.