Harnessing GenAI for Climate Change Knowledge Management: Use Cases, System Design, and Experimental Evaluation

Manual extracting of knowledge from documents and its transformation for use in knowledge services can be slow and expensive, particularly in domains like climate change, where new findings are frequently generated by a large number of actors. Generative Artificial Intelligence (GenAI) models offer a promising alternative to extract knowledge from diverse sources automatically. However, irresponsible use of this technology might undermine the efforts to combat climate change by misinforming and dis-informing the users. To address these risks, we identify use cases for AI-assisted knowledge extraction in the context of climate change and we introduce a novel system, which we refer to as SumQA, to support these use cases with GenAI. SumQA provides a user interface for processing multiple documents, and extracting targeted information automatically. Specifically, it facilitates on demand batch processing of by different “medium size” GenAI models, such as GPT3.5 turbo, Mistral NeMo, Llama 3.1 and Gemma 2, to answer user questions. In addition to the design and implementation of SumQA, we provide an in-depth analysis of using the SumQA interface to extract knowledge from multiple educational documents in the context of climate change. Our findings highlight crucial guidelines and pitfalls on the use of GenAI for processing climate data, and for extracting specific and concise information in a structured form, that is useful for knowledge services and decision-making. Finally, we discuss the lessons learned, identify good practices for prompt engineering and question formulation, and propose research directions for improving the SumQA system as a productivity tool for stakeholders in Climate Change adaptation and mitigation.