Machine Learning has the Capability to Monitor the Advancement of Climate Technology Innovation Using Climate-Related Texts

Part one : In recent years, significant advancements in the field of natural language processing (NLP) have been driven by the emergence of large pre-trained language models (LM). Nevertheless, although pre-training on general language has demonstrated effectiveness for conventional language, challenges arise when applied to specialised language varieties. Specifically, texts pertaining to climate issues contain specialised vocabulary that cannot be accurately captured by mainstream language models. We posit that the deficiency in current Language Models restricts the potential use of cutting-edge Natural Language Processing in the extensive domain of text analysis related to climate change. In response to this issue, we introduce CLIMATE-BERT, a transformer-driven language model that receives additional pre-training on a dataset comprising more than 2 million paragraphs of climate-focused content sourced from a diverse range of outlets, including mainstream news, scholarly articles, and corporate climate reports. Our findings indicate that CLIMATE-BERT results in a 48% enhancement in performance on a masked language model task. This improvement subsequently translates to a reduction in error rates ranging from 3.57% to 35.71% across a range of downstream climate-related tasks such as text classification, sentiment analysis, and fact-checking. Part two : In order to adhere to international climate goals, expeditious advancement in climate technologies is imperative. Both governmental bodies and businesses are allocating significant resources to expedite this procedure. However, monitoring the progress and expansion of climate technologies poses difficulties. In this study, we demonstrate the utility of machine learning in monitoring and analysing advancements in climate technology. Our study leverages extensive language models and data from LinkedIn to establish a comprehensive network encompassing major public and private entities engaged in collaboration towards climate-tech innovation at a global scale. The network that emerges comprises 134,727 organisations spanning 189 countries. It encompasses a range of collaborative initiatives, encompassing research and development partnerships and equity investments, across 19 distinct climate technologies. The data we have collected demonstrate a significant focus on a select few climate technologies, as approximately 60% of emerging climate technology startups and innovation efforts are concentrated in just three areas: solar, electric vehicles, and hydrogen. However, this specific focus gives rise to apprehensions regarding the insufficient progress in various other essential climate technologies, namely heat pumps, biofuels, and carbon capture and storage. These technologies demand increased levels of innovation and entrepreneurial activities to ensure alignment with global climate objectives. Furthermore, our findings indicate that certain governmental entities perform a vital function within networks of innovation related to climate technology, especially in the context of commercialisation. Our study reveals that governmental organisations have the capability to foster pioneering innovation clusters, even in areas with minimal existing industry presence. Our findings indicate a crucial necessity for governmental entities to encourage increased technological diversification within regions, by fostering innovation in a broader spectrum of climate technologies. In subsequent periods, our approach in machine learning holds potential for ongoing monitoring of the evolution of climate-technology innovation, presenting an opportunity to evaluate the impact of governmental interventions. This framework is transferable to various other sectors, including pharmaceuticals, chemicals, and information and communication technologies.