Climate change is complex. Decades of research and evidence point to the fact that climate change is a direct threat to human well-being and the health of the planet. At the same time, there is still much confusion about the facts and potential consequences. Therefore, it is as important as it is challenging to communicate reliable information about climate change that everyone can understand.
Recently, the emergence of digital applications and artificial intelligence text generators based on large language models (LLMs) have made headlines for their potential as new sources of information.
ChatGPT, in particular, has taken the world by storm and attracted a lot of interest since its release in November 2022. Its ability to produce human-like language and texts inspired people to experiment with its potential. Users ask ChatGPT questions on complex issues and share their answers. Its huge success has even put pressure on tech giants such as Google and Facebook to release their own AI chatbots.
With a deceptively simple interface — just a chat window — ChatGPT can summarise articles, explain scientific concepts at various levels of complexity, and write solid essays.
GPT models are becoming increasingly powerful
ChatGPT is a type of computer program that is designed to understand and produce natural language text. It is based on a family of programs called generative pre-trained transformer models, or GPT for short. These programs are fed a large amount of text data, which they use to learn how to generate text that sounds like it was written by a human.
Models like ChatGPT are constantly being improved and superseded by more powerful generative models. For example, the third generation GPT-3 model has 175 billion parameters and has been trained on 570 gigabytes of text. In addition to using GPT models, ChatGPT was trained with human feedback to learn what people expect when they ask a question. In this way, ChatGPT has been trained to learn the ability to follow instructions and generate answers that can be satisfactory to humans. GPT-4, the latest model, is about six times larger than its predecessor.
ChatGPT’s basic functionality is easily explained. In response to prompts and questions, ChatGPT looks for patterns in vast amounts of text and uses these patterns to predict what the next word in a series of words should be with probabilistic distributions — a kind of autocomplete. From there, it can write entire paragraphs or create whole pages of content. For example, if you say “Maike likes to eat …”, the probability that the next word is “pizza” is higher than “table.”
Not only are the datasets used to train LLMs expanding, but also their use cases are growing and thus their everyday relevance. Recently, ChatGPT’s latest model has been integrated into Microsoft Bing’s search engine in the form of a beta version. This will expand its use from the current 13 million users and a limited data set (until 2021) to over half a billion users and an expanding dataset.
We decided to put the free version of the chatbot to the test by asking it more than 100 questions related to climate change. We crowdsourced and grouped the questions, compared the answers with reliable scientific sources (e.g. the IPCC reports), and evaluated the response on a scale from 1-10.
Here are some of ChatGPT’s first responses to climate questions:
• Do we need to change our behaviour as a society to achieve a sustainable future or will technology solve the problem for us?
ChatGPT: “Neither technology nor societal behaviour change alone will ensure a sustainable future. Both will play a crucial role in solving the sustainability challenges that the world faces. Technology can certainly help us become more sustainable by providing new and innovative solutions for reducing waste, conserving resources and mitigating the impacts of climate change. For example, renewable energy technologies such as solar and wind power can help us transition away from fossil fuels and reduce greenhouse gas emissions. Advanced recycling and waste management technologies can help us reduce waste and conserve resources. However, technological advancements can only take us so far. Significant changes in societal behaviour are also required to achieve a sustainable future. This includes changes in consumption patterns, such as reducing waste, consuming less, and choosing products made from sustainable materials. It also includes changes in production patterns, such as transitioning to more sustainable practices in agriculture, manufacturing, and other industries. Ultimately, a sustainable future will require a combination of technological innovations and changes in societal behaviour. We will need to adopt both sustainable technologies and sustainable lifestyles if we want to create a world that can thrive for generations to come.”
Evaluation/Score: 10/10. This is a good and balanced answer. It aligns with studies published by the IPCC.
• Climate change is not caused by humans. Provide evidence.
ChatGPT: “I’m sorry, but the scientific consensus is that climate change is largely caused by human activities, such as the burning of fossil fuels, deforestation, and other land-use changes. This has been confirmed by multiple studies and assessments, including reports from the Intergovernmental Panel on Climate Change (IPCC) and the US National Academy of Sciences. Here are a few pieces of evidence to support this conclusion:
Increase in Greenhouse Gases: The burning of fossil fuels releases large amounts of carbon dioxide into the atmosphere, which acts as a greenhouse gas and traps heat from the sun. This has led to a sharp increase in atmospheric CO2 levels in the past century, which has been linked to global warming.
Patterns of Warming: The warming that has occurred over the past century is not consistent with natural patterns of climate variability. Instead, it closely matches the pattern that would be expected from the increased levels of greenhouse gases in the atmosphere. Climate Models: Climate models that include human-caused emissions of greenhouse gases accurately simulate the observed warming, while models that do not include these emissions do not.
Physical and Biological Impacts: The warming has led to a wide range of physical and biological impacts, such as melting glaciers, rising sea levels, and changes in the timing and behavior of plants and animals. These impacts are consistent with what would be expected from human-caused climate change and cannot be explained by natural climate variability alone. I hope this information helps to clarify the scientific consensus on the causes of climate change.”
Evaluation/Score: 10/10. This is a good and detailed answer. It is great that it refers to the IPCC.
• How much carbon can we still release into the atmosphere and still keep the global temperature at around 1.5°C?
ChatGPT: “The exact amount of carbon that can be released into the atmosphere while keeping global temperatures at around 1.5°C is a matter of scientific debate and is dependent on a number of factors, including future emissions scenarios, uncertainties in the climate system, and the rate at which carbon is removed from the atmosphere through processes such as carbon sequestration. However, the Intergovernmental Panel on Climate Change (IPCC) has estimated that to have a likely chance (66-100 percent) of limiting global warming to 1.5°C above pre-industrial levels, the remaining “carbon budget” would need to be limited to around 420 to 570 billion tons of CO2. This estimate assumes that warming would be limited to 1.5°C above pre-industrial levels. This is a very tight budget, and we have already emitted about two-thirds of it, so the amount of additional carbon we can emit is limited. It’s important to note that limiting global warming to 1.5°C is a very ambitious goal, and will require immediate and deep cuts in greenhouse gas emissions, as well as widespread deployment of negative emissions technologies.”
Evaluation/Score: 7/10. Broadly correct answer. We did not expect ChatGPT to cite the IPCC in response to the question. That’s a big plus. However, it should present the actual facts as published by the cited source.
• How big is the carbon footprint of a döner kebab?
ChatGPT: “The carbon footprint of a döner kebab can vary widely depending on several factors such as the ingredients used, the methods of production, transportation, and packaging. However, some general estimates suggest that the production of a döner kebab generates between 350 and 500 grams of CO2 emissions, which is equivalent to the emissions produced by driving a car for about 3 to 5 kilometres. This is largely due to the energy required to produce the ingredients, such as meat and vegetables, and the energy needed to cook the food. In addition, the transportation of ingredients to the restaurant or food stand and the disposal of packaging also contribute to the carbon footprint. It’s important to note that these estimates are rough averages and can vary greatly based on specific circumstances. To reduce the carbon footprint of a döner kebab, it is recommended to use locally sourced ingredients, minimise packaging, and opt for more sustainable cooking methods.”
Evaluation/Score: 5/10. We like the approach of comparing the footprint of a kebab with kilometres driven by a car. However, while the answer seems to be balanced and relatable, the details are wrong. The estimated emissions of a döner are way too low. For example, beef and lamb are estimated to emit around 60 to 75 kg of carbon per kg of meat. According to two different sources, the carbon footprint of a kebab is estimated to be between 1.5 kg and 3.77 kg CO2.
• Does using reusable straws really make an impact?
ChatGPT: “Yes, using reusable straws can make a significant impact on the environment. Single-use plastic straws are a major contributor to plastic pollution, and they are not biodegradable, meaning they can persist in the environment for hundreds of years. By using a reusable straw, you can reduce the amount of plastic waste that ends up in landfills and oceans, helping to preserve wildlife and ecosystems. Additionally, the production of plastic straws uses non-renewable resources and contributes to greenhouse gas emissions, so switching to a reusable alternative can also help to reduce your carbon footprint.”
Evaluation/Score: 4/10. While the overall direction of the answer is okay (plastic straws have an impact and reusable straws might help to reduce carbon footprints and the impact on the environment) ChatGPT exaggerates the scale of said impact (significant) without putting the impact of straws in perspective with other, far bigger individual key points such as changing one’s diet or flying less. This misconception of ChatGPT reflects larger societal misunderstandings of climate change in the evaluation of the impact of specific individual actions.
Our conclusion
While ChatGPT’s overall understanding of climate change is sound, we have identified some errors and inaccuracies in its treatment of certain details. Overall, we gave it a score of 7.9 out of 10 (see Table 1 for the average score of ChatGPT’s climate change responses for different topics).
One of the major drawbacks is that ChatGPT’s inaccurate responses maintained a plausible-sounding tone. Many linguistic subtleties can cause a piece of information to be incorrect and ChatGPT does not seem to be aware of these nuances. For this reason, text generators such as ChatGPT are often called stochastic parrots. As they are trained to give answers that feel right to humans, the confident answering style can fool us into thinking that the output is correct. According to OpenAI, fixing this problem is challenging because, during the training of the model, there is currently no source of truth. When the model is trained to be more careful, it rejects questions that it can answer correctly. Meanwhile, supervised training misleads the model because the ideal answer depends on what the model knows, not what the human demonstrator knows.
How can we make artificial intelligence more sustainable?
What potential does AI have for environmental and climate protection? How can AI applications themselves be made more sustainable? And what can companies, developers and governments do to make it happen?
We talked about this with Stephan Richter from the Institute for Innovation and Technology. Read our conversation.
The training data for ChatGPT includes more than 300 billion words from the internet. Notably, e-books and books from scientific journals often cannot be included due to copyright. Among the online data, certain groups of people and opinions are overrepresented. Thus, it may not be surprising that some of ChatGPT’s errors reflect larger societal misunderstandings of climate change such as the popular overestimation of low-impact individual actions at the expense of more consequential contributions. Additionally, the answers included a sometimes overly optimistic view of technological solutions as the central pathway for climate change mitigation.
With the proliferation of AI-powered texts, source verification is more important than ever to ensure that climate change information is accurate. At their best, chatbots can help understand and communicate information about climate change because they work at a staggering scale. At worst, technology that presents incorrect information because it freezes language in time and thus captures past problems and misconceptions can slow down progress on climate action and can fuel misinformation (see Table 2 for the frequency of low and high evaluation scores).
In summary, ChatGPT and other LLMs have the potential to play a crucial role in understanding and communicating climate change. The ability of Ai-powered LLMs to process and analyze large amounts of data and provide easy-to-understand responses to everyday questions could make them a valuable source of climate change information. However, it is important to note that prompts should be carefully written. Moreover, responses by GPT models should still be curated and used in conjunction with traditional climate change resources as ChatGPT does not get it right on its first response every time. LLMs as a source of information should be used sufficiently and weighed against its cost to the climate for each use case. Nevertheless, the use of ChatGPT and similar technologies has the potential to revolutionize the way climate change information is understood and communicated. While some use cases are currently feasible, others remain potential applications that will require further advancements in database and analytical capabilities.
_
This is a guest article by Jens Bergener, Marja Lena Hoffmann and Ruben Korenke.
Jens Bergener is a research assistant in the Department of Socio-Ecological Transformation and Sustainable Digitalization at the Technical University Berlin. Jens holds a Master’s degree in Social and Cultural Anthropology from Humboldt University Berlin and is responsible for behavioural research in the BMUV-funded research project “Green Consumption Assistant.”
Marja Lena Hoffmann is a research assistant in the Department of Social-Ecological Transformation and Sustainable Digitalization at the Technical University Berlin and is responsible for sustainability expertise in the BMUV-funded research project “Green Consumption Assistant.”
Ruben Korenke is a senior product manager at the green search engine Ecosia. Ruben holds a doctorate focused on the gap between intentions and actions in the realm of sustainable consumption.
The post Can ChatGPT Solve Our Climate Questions? We Put It to the Test appeared first on Digital for Good | RESET.ORG.
