At Belvedere Labs we spend a lot of time trying out new AI capabilities, wherever they show up. Weather caught our eye because it makes an unusually clean test. The forecasts are public, the answers arrive a few days later from official thermometers and rain gauges, and two of the world's biggest forecasting agencies now run an AI model alongside their traditional one.

So we gathered the forecasts four models made for eight US cities from July to October 2026 and compared each one with what actually happened. This article is a short summary of what we noticed. The full data set lives in our interactive forecast explorer, where you can dig into any city, day or model.

Two Ways to Predict the Weather

Traditional forecasts come from physics. The model splits the atmosphere into millions of 3D boxes and applies the equations for air movement, heat and moisture, stepping forward a few minutes at a time until it reaches next week. That takes a supercomputer. We looked at the two best-known examples: IFS from the European centre ECMWF, and GFS from the US agency NOAA.

AI models take a different route. They are neural networks trained on decades of past weather, and they learn how the atmosphere tends to change over six hours. Repeating that step gets them a week or more ahead. Training one takes a lot of computing, but running it is far cheaper than running a physics model. ECMWF estimates its AI model uses about 1,000 times less energy per forecast, and Google DeepMind reported that its GraphCast model produces a 10-day forecast in under a minute on a single machine. The AI models still start from today's weather as measured and prepared by the traditional systems, so the two approaches work together. ECMWF made its AI model, AIFS, operational in early 2025, and NOAA followed later that year with AIGFS, which builds on GraphCast.

What We Did

For each of eight cities (New York, Chicago, Miami, Houston, Denver, Phoenix, Los Angeles and Seattle) we used one official NOAA weather station. Every day, we compared each model's forecast high, low and rain, issued anywhere from 1 to 7 days ahead, with what the station recorded.

Every model turned out to have a local bias at each station, such as running a few degrees cold. Weather services commonly adjust for local bias, so we tried a simple version: subtract each model's average miss at each station, and see what's left. We looked at the results both ways. The explorer explains the method in more detail, along with how we checked whether a difference might just be luck.

What We Noticed

The first chart shows each model's typical miss: on average, how many degrees its forecast high was off from the high the station actually recorded. Lower is better. The dotted line is a no-forecast baseline. If you skipped the forecast and just guessed the usual high for that date, you'd typically be off by 4 to 5°F, so anything below that line is doing better than a guess from the calendar.

Straight out of the models, the four were close. One day ahead, each missed the daily high by about 3.5°F on average. Much of that was a steady offset rather than random error, so the raw scores mostly reflected each model's local bias.

After adjusting for local bias, ECMWF's AI model came out ahead in our sample. AIFS had the smallest typical miss at every lead time we checked: about 1.6°F one day out and 3.2°F a week out, against roughly 2°F and 4°F for the others. NOAA's AI model did better than NOAA's traditional model, and about as well as ECMWF's traditional model. That's encouraging for the AI approach, but it comes from one summer and a fairly simple adjustment, so we wouldn't call it a final verdict.

The AI forecasts changed their minds less. Each model issues a fresh forecast every day, so you can watch its prediction for, say, next Saturday move as the day approaches. NOAA's GFS changed its forecast high by more than 5°F from one day to the next about once in every six updates. ECMWF's AIFS did so about once in every 33. A steady forecast isn't automatically a correct one, but it is easier to plan around.

Rain was hard for every model. All four were similar at calling wet days, and all of them struggled with the biggest downpours, either missing them or warning of storms that never came. A model's rain covers an area about 25 km across, while a rain gauge only catches what falls on one spot, so some mismatch is expected.

Is a 7-Day Forecast Worth Reading?

Yes, for the general picture. Here is how often each approach got the daily high within 5°F of what actually happened, averaged over all eight cities:

Every forecast beat guessing from the calendar, by a wide margin tomorrow and by less a week out. Adjusting for local bias made a bigger difference than which model you started with. And averaging several adjusted models did best, though only slightly better than ECMWF's AI model on its own.

So where should you get your weather? You don't need to pick a model yourself. Official forecasts, such as the US National Weather Service's at weather.gov, already blend many models and adjust them for local conditions, which was the step that mattered most in our test. Trust the next two or three days for the actual number, and the rest of the week for the trend, such as a hot spell or a cooldown on the way.

The Bigger Picture

All four models are part of a long, steady improvement in forecasting. A 2015 review in Nature called it "the quiet revolution of numerical weather prediction", and by ECMWF's own tracking, forecasts have improved steadily, gaining roughly a day of useful lead time every decade. The AI models look like the latest step in that progression.

Projects like this one are close to home for us. Belvedere, Clear Fracture's agentic data manager, uses AI agents to catalog an organization's data, answer questions about it and build the pipelines that combine it, with people approving every change. Pulling forecasts, station records and climate normals from three public sources into one table we could trust and trace back is a small version of the same problem.

Our results cover one summer, eight weather stations and a simple adjustment, so a winter of storms could tell a different story. If you'd like to see for yourself, the interactive explorer has every forecast we checked. Pick your nearest city and see how the models did.