--- title: "How Good Are the New AI Weather Models? We Checked a Summer of Forecasts" description: "We wondered how the new generation of AI-based weather forecasting models perform compared to traditional physics-based approaches. Credit to the meteorologists - forecast quality continues to improve." canonical: "https://www.clearfracture.ai/articles/how-good-are-the-new-ai-weather-models-we-checked-a-summer-of-forecasts" published: "2026-10-16T19:35:45.435657+00:00" updated: "2026-10-20T20:43:34.019722+00:00" --- Case Studies # How Good Are the New AI Weather Models? We Checked a Summer of Forecasts We wondered how the new generation of AI-based weather forecasting models perform compared to traditional physics-based approaches. Credit to the meteorologists - forecast quality continues to improve. [![Scott Fincher](/_next/image?url=https%3A%2F%2Frgzgbulrzrbknuiprxti.supabase.co%2Fstorage%2Fv1%2Fobject%2Fpublic%2Fmedia%2Fuploads%2F1789488065870-Scott_edited_smaller.jpg&w=96&q=75)](/articles/author/scott-fincher) [Scott Fincher](/articles/author/scott-fincher)Technical Growth Lead Oct 16, 2026·5 min read * * * At Belvedere Labs we spend a lot of time trying out new AI capabilities, wherever they show up. Weather caught our eye because it makes an unusually clean test. The forecasts are public, the answers arrive a few days later from official thermometers and rain gauges, and two of the world's biggest forecasting agencies now run an AI model alongside their traditional one. So we gathered the forecasts four models made for eight US cities from July to October 2026 and compared each one with what actually happened. This article is a short summary of what we noticed. The full data set lives in our [interactive forecast explorer](https://www.belvederelabs.ai/forecast-explorer), where you can dig into any city, day or model. ## Two Ways to Predict the Weather Traditional forecasts come from physics. The model splits the atmosphere into millions of 3D boxes and applies the equations for air movement, heat and moisture, stepping forward a few minutes at a time until it reaches next week. That takes a supercomputer. We looked at the two best-known examples: IFS from the European centre [ECMWF](https://www.ecmwf.int/), and GFS from the US agency [NOAA](https://www.noaa.gov/). AI models take a different route. They are neural networks trained on decades of past weather, and they learn how the atmosphere tends to change over six hours. Repeating that step gets them a week or more ahead. Training one takes a lot of computing, but running it is far cheaper than running a physics model. ECMWF estimates its AI model uses about 1,000 times less energy per forecast, and Google DeepMind reported that its [GraphCast](https://deepmind.google/discover/blog/graphcast-ai-model-for-faster-and-more-accurate-global-weather-forecasting/) model produces a 10-day forecast in under a minute on a single machine. The AI models still start from today's weather as measured and prepared by the traditional systems, so the two approaches work together. ECMWF [made its AI model, AIFS, operational](https://www.ecmwf.int/node/29308) in early 2025, and NOAA followed later that year with [AIGFS](https://www.noaa.gov/news-release/noaa-deploys-new-generation-of-ai-driven-global-weather-models), which builds on GraphCast. ## What We Did For each of eight cities (New York, Chicago, Miami, Houston, Denver, Phoenix, Los Angeles and Seattle) we used one official NOAA weather station. Every day, we compared each model's forecast high, low and rain, issued anywhere from 1 to 7 days ahead, with what the station recorded. Every model turned out to have a local bias at each station, such as running a few degrees cold. Weather services commonly adjust for local bias, so we tried a simple version: subtract each model's average miss at each station, and see what's left. We looked at the results both ways. The explorer explains the method in more detail, along with how we checked whether a difference might just be luck. ## What We Noticed The first chart shows each model's typical miss: on average, how many degrees its forecast high was off from the high the station actually recorded. Lower is better. The dotted line is a no-forecast baseline. If you skipped the forecast and just guessed the usual high for that date, you'd typically be off by 4 to 5°F, so anything below that line is doing better than a guess from the calendar. ! Straight out of the models, the four were close. One day ahead, each missed the daily high by about 3.5°F on average. Much of that was a steady offset rather than random error, so the raw scores mostly reflected each model's local bias. After adjusting for local bias, ECMWF's AI model came out ahead in our sample. AIFS had the smallest typical miss at every lead time we checked: about 1.6°F one day out and 3.2°F a week out, against roughly 2°F and 4°F for the others. NOAA's AI model did better than NOAA's traditional model, and about as well as ECMWF's traditional model. That's encouraging for the AI approach, but it comes from one summer and a fairly simple adjustment, so we wouldn't call it a final verdict. ! The AI forecasts changed their minds less. Each model issues a fresh forecast every day, so you can watch its prediction for, say, next Saturday move as the day approaches. NOAA's GFS changed its forecast high by more than 5°F from one day to the next about once in every six updates. ECMWF's AIFS did so about once in every 33. A steady forecast isn't automatically a correct one, but it is easier to plan around. Rain was hard for every model. All four were similar at calling wet days, and all of them struggled with the biggest downpours, either missing them or warning of storms that never came. A model's rain covers an area about 25 km across, while a rain gauge only catches what falls on one spot, so some mismatch is expected. ## Is a 7-Day Forecast Worth Reading? Yes, for the general picture. Here is how often each approach got the daily high within 5°F of what actually happened, averaged over all eight cities: ! Every forecast beat guessing from the calendar, by a wide margin tomorrow and by less a week out. Adjusting for local bias made a bigger difference than which model you started with. And averaging several adjusted models did best, though only slightly better than ECMWF's AI model on its own. So where should you get your weather? You don't need to pick a model yourself. Official forecasts, such as the US National Weather Service's at [weather.gov](http://weather.gov), already blend many models and adjust them for local conditions, which was the step that mattered most in our test. Trust the next two or three days for the actual number, and the rest of the week for the trend, such as a hot spell or a cooldown on the way. ## The Bigger Picture All four models are part of a long, steady improvement in forecasting. A 2015 review in *Nature* called it ["the quiet revolution of numerical weather prediction"](https://doi.org/10.1038/nature14956), and by [ECMWF's own tracking](https://www.ecmwf.int/node/25701), forecasts have improved steadily, gaining roughly a day of useful lead time every decade. The AI models look like the latest step in that progression. Projects like this one are close to home for us. Belvedere, Clear Fracture's agentic data manager, uses AI agents to catalog an organization's data, answer questions about it and build the pipelines that combine it, with people approving every change. Pulling forecasts, station records and climate normals from three public sources into one table we could trust and trace back is a small version of the same problem. Our results cover one summer, eight weather stations and a simple adjustment, so a winter of storms could tell a different story. If you'd like to see for yourself, the [interactive explorer](https://www.belvederelabs.ai/forecast-explorer) has every forecast we checked. Pick your nearest city and see how the models did. Data AnalysisWeatherAI ModelsBelvedere Labs Keep Reading ## More from Clear Fracture [Data Plumbing Isn’t Sexy. Neither Is Waiting for Intelligence.](/articles/data-plumbing-isnt-sexy-neither-is-waiting-for-intelligence) [![Data Plumbing Isn’t Sexy. Neither Is Waiting for Intelligence.](/_next/image?url=https%3A%2F%2Frgzgbulrzrbknuiprxti.supabase.co%2Fstorage%2Fv1%2Fobject%2Fpublic%2Fmedia%2Fuploads%2F1790960615006-data-plumbing-animation-v3_2x.gif&w=1920&q=75)](/articles/data-plumbing-isnt-sexy-neither-is-waiting-for-intelligence) Data EngineeringOctober 6, 2026 ### Data Plumbing Isn’t Sexy. Neither Is Waiting for Intelligence. Behind every useful piece of intelligence, someone had to get the information collected, processed, connected, and delivered. Clear Fracture CEO Nancy Dillman on why that unglamorous work shaped Belvedere. [Same Story, Different Words: Jev Scores Tone, Blame, and Overreach Across 88 News Outlets | Belvedere Labs](/articles/same-story-different-words-jev-scores-88-news-outlets) [![Same Story, Different Words: Jev Scores Tone, Blame, and Overreach Across 88 News Outlets | Belvedere Labs](/_next/image?url=https%3A%2F%2Frgzgbulrzrbknuiprxti.supabase.co%2Fstorage%2Fv1%2Fobject%2Fpublic%2Fmedia%2Fuploads%2F1790795090296-outliers-featured-grid.png&w=1920&q=75)](/articles/same-story-different-words-jev-scores-88-news-outlets) AnalysisSeptember 30, 2026 ### Same Story, Different Words: Jev Scores Tone, Blame, and Overreach Across 88 News Outlets | Belvedere Labs Jev, TypeSafe's new model, never writes a word. Ask it a question and it hands back probabilities. We wanted to explore the technology, so we gave it a fun job: read the same story from 88 outlets, blind to the masthead, and score how far the newsrooms disagree. [Introducing Investigate, Belvedere's New Data Analysis Agent](/articles/introducing-investigate-belvederes-new-data-analysis-agent) [![Introducing Investigate, Belvedere's New Data Analysis Agent](/_next/image?url=https%3A%2F%2Frgzgbulrzrbknuiprxti.supabase.co%2Fstorage%2Fv1%2Fobject%2Fpublic%2Fmedia%2Fuploads%2F1789508379411-Copy_of_When_AI_Agents_Build_Trusted__Production_Data_Pipelines__YouTube_Thumbnail___1_.png&w=1920&q=75)](/articles/introducing-investigate-belvederes-new-data-analysis-agent) ProductSeptember 15, 2026 ### Introducing Investigate, Belvedere's New Data Analysis Agent Investigate is Belvedere's new read-only research agent that lets you ask plain-language questions about your data and get answers grounded in verifiable catalog evidence, without ever touching or altering the underlying data. [Agent Skill: Idempotent Backfill for Late-Arriving Data](/articles/an-idempotent-backfill-skill-for-agentic-data-pipelines) [![Agent Skill: Idempotent Backfill for Late-Arriving Data](/_next/image?url=https%3A%2F%2Frgzgbulrzrbknuiprxti.supabase.co%2Fstorage%2Fv1%2Fobject%2Fpublic%2Fmedia%2Fuploads%2F1787677997000-idempotent-backfill-animation_2x.gif&w=1920&q=75)](/articles/an-idempotent-backfill-skill-for-agentic-data-pipelines) Data EngineeringAugust 25, 2026 ### Agent Skill: Idempotent Backfill for Late-Arriving Data We turned proven ideas from Apache Hudi and functional data engineering into a free idempotent backfill skill, then tested it against safe and unsafe PostgreSQL sources. [From Here to There: How Belvedere™ Maps Your Current State and Builds the Path to Your Target](/articles/from-here-to-there) [![From Here to There: How Belvedere™ Maps Your Current State and Builds the Path to Your Target](/_next/image?url=https%3A%2F%2Frgzgbulrzrbknuiprxti.supabase.co%2Fstorage%2Fv1%2Fobject%2Fpublic%2Fmedia%2Fuploads%2F1786592161000-here-to-there-map-animation-v2_2x.gif&w=1920&q=75)](/articles/from-here-to-there) System ModelingAugust 18, 2026 ### From Here to There: How Belvedere™ Maps Your Current State and Builds the Path to Your Target Every capable agent needs two things before it can act with confidence: a clear picture of where we are, and a robust definition of where we need to go. Most agentic systems quietly fail at exactly that. [Turn GitHub Repositories Into Explorable SysML v2 System Models](/articles/sysml-repo-modeler-github-repositories) [![Turn GitHub Repositories Into Explorable SysML v2 System Models](/_next/image?url=https%3A%2F%2Frgzgbulrzrbknuiprxti.supabase.co%2Fstorage%2Fv1%2Fobject%2Fpublic%2Fmedia%2Fuploads%2F1784667213851-automated-system-modeling-animation_2x.gif&w=1920&q=75)](/articles/sysml-repo-modeler-github-repositories) System ModelingAugust 5, 2026 ### Turn GitHub Repositories Into Explorable SysML v2 System Models Our new open source project, SysML Repo Modeler, is a self-hosted AI analysis harness that turns one GitHub repository—or many—into validated SysML v2 and interactive architecture graphs. [View all articles](/articles) --- Source: [https://www.clearfracture.ai/articles/how-good-are-the-new-ai-weather-models-we-checked-a-summer-of-forecasts](https://www.clearfracture.ai/articles/how-good-are-the-new-ai-weather-models-we-checked-a-summer-of-forecasts)