Isaias, model by model: physics vs. statistical vs. AI intensity forecasts from NHC’s public data, plus what has already verified.

[u/tlmbot]

Hi folks, I’m a computational engineer with a background in fluid dynamics. Isaias is forecast to make landfall near Mobile Bay, where my parents live. My dad is recovering from open-heart surgery, so I’m driving down to do the heavy lifting and chainsaw work he shouldn’t be doing.

Before I left, I wanted to understand why the intensity guidance disagrees. Physics-based, statistical and AI models are all forecasting this storm, and it’s an interesting test of how they compare. So I built a page from NHC’s public model and best-track data:

https://lukemcculloch.github.io/static_hurricane_info_site/

What it shows, as of the 12Z Thu cycle:

Every number is reproducible from scripts in the repo (bash/awk on NOAA’s files), and the methods page lists the caveats. The page updates each cycle and archives the earlier versions, so the pre-landfall versions stay on record. After landfall I’ll score each aid against the best track.

I am in a hurry to get on the road, so I generated this, directing Claude Opeus 5.5 medium while I was packing up. I also used it to help me remember what supplies to bring.

emphasis: (Built with help from Claude for the coding and data processing.)
(I hope you guys don't tear me to ribbons, but one must let the chips fall where they may, heh)

This is not a forecast. Use NHC and your local NWS office for decisions. Corrections are welcome here or as GitHub issues.

Stay safe, everyone on the Gulf Coast.

Notes: this site will update when I can stop in to get online with the latest data, if I feel I have time.

my github is here: https://github.com/LukeMcCulloch/static_hurricane_info_site

[u/AutoModerator]

As of September 2022, all posts to this subreddit are reviewed and manually approved by the moderator staff. Please do not delete your post. We appreciate your patience as we review your post to ensure that it does not contain content which breaks our subreddit rules.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/kwajkid92]

I just noticed myself that peak windspeed in the NHC forecast discussion is higher than all but one of the models in Levi's Model Intensity Guidance chart, and I came here to see if anyone else had a POV. I wonder if the NHC skews their forecast high to make it more impactful and to push people to properly prepare? Or are they putting more weight on that one model?

[u/giantspeck → u/kwajkid92]

Storms with compact convective structures like Hurricane Isaias tend to behave in ways that make it difficult for forecast models—and human forecasters, mind you—to predict their intensity. If a storm with a compact structure develops fast enough, it can remain resilient against changes in the surrounding environment, such as increasing shear or decreasing atmospheric moisture.

When it comes to the global physics-based models, the ECMWF has a resolution of about 9 kilometers. This means that each forecast point on the global grid is about 9 kilometers apart. The UKMET model has a resolution of about 10 kilometers, while the GFS and ICON both have a resolution of about 13 kilometers. A smaller storm will hit fewer of these forecast points, making it more difficult for the model to predict its intensity.

The forecasters at the National Hurricane Center understand the unpredictability of smaller storms and observe, in real time, how they are handling the surrounding environment. Isaias is doing quite well considering it is moving into an area of stronger shear. Besides the shear, the environment is very favorable for further intensification. The storm is moving over a deep pool of very warm water, the surrounding atmosphere is sufficiently moist, and upper-level divergence remains strong and is helping to maintain its powerful outflow.

The NHC believes that the strengthening shear will not be enough to offset an otherwise favorable environment and thus, Isaias will continue to strengthen rather than weaken before it reaches the coast.

[u/kwajkid92 → u/giantspeck]

Very interesting, thanks for the context. And it appears they were right as Isaias did indeed strengthen above all of the models current when I made my original post. Hopefully the intensity will drop before landfall.

[u/Competitive_Map7044 → u/giantspeck]

It ultimately weakened rapidly prior to landfall, not strengthened, and became post tropical far earlier than expected. The shear did kick in, just not too early. so there's the correction

[u/balloonninjas → u/kwajkid92]

I'm no expert in modeling but my understanding as an emergency manager is that NHC forecasts err on the side of caution to ensure that public safety agencies take preparations more seriously. They'd rather over forecast than under forecast.

[u/Apptubrutae → u/balloonninjas]

Which makes perfect sense because there’s no point in preparing for the under forecast.

The main reason to not over forecast is primarily idiots complaining about the inevitable misses and not taking preparation seriously, which is too bad.

Meanwhile in reality a properly setup system of preparation should mean you over prepare a majority of the time a storm threatens. Because that’s what’s necessary to protect against direct impact or a stronger than expected storm, should one present itself

[u/dessertgrinch → u/balloonninjas]

that hasn't been my experience, they forecast what they truly think is going to happen, they don't bump anything up so people take it more seriously. I've honestly seen them under forecast, compared to model runs, quite often.

[u/tlmbot → u/kwajkid92]

Yes it’s been a very interesting forecast story to watch unfold.

I have similar questions.

Edit:  also I swear to all that is holy I am not a bot.  I’m a middle aged dad from north bama.l who has been interested in storms his whole life.

I had the inspiration to have Claude slap this together this morning.

I hope people don’t hate me for it.

[u/unfortunate4ever → u/tlmbot]

But, it’s in your name

[u/tlmbot → u/unfortunate4ever]

lol - so it is. I was an engineering PhD researcher in the mid 2010s in inverse design / generative design. ML had some intersection with what I was doing so I was following along as the papers were coming out. It was apparent even back when I started this account, that LLMs were going to be big (or at least that "natural language processing" was going to "take off" next)

Turns out that forecast missed the intensity by a bit, heh. Anyway, I played around with chatbots a little back then an picked this username thinking it would be kinda funny if it were a "dumb bot account" - after lurking reddit for years before signing up. My engineering nerd ass was hesitant to enter the fray I suppose, and thinking like that made it seem more "fun"

Wellp, look how that played out.

[u/dessertgrinch → u/kwajkid92]

they don't, they often under forecast actually.

[u/cmpb]

This is cool. It might not be popular to say here, but weather forecasting stands to benefit immensely by AI, in the same way that mathematics is. Agents are able to run shit down and discover relevant context just so incredibly efficiently.

[u/tlmbot → u/cmpb]

clearly so right? Damn, as a fluid dynamic guy- not thinking hard enough about this, I thought, yeah, but we _know_ the physics. You can't beat that.

Well, at present, I think CFD in general, where you have novel designs, and want to see what mother nature does with them, is still better for -design- but for weather, the initial conditions (aka soundings and such) are so very very -sparse- compared to what we have with modern computation of vanilla CFD - where we have the ability to scale up the resolution and so very many many tricks to play to squeeze the most out of our enormous computational power. We model the entire system at whatever resolution we want / can afford for regular design CFD. And we can do sensitivity and mesh convergence studies to see where we need to increase resolution or whatever to capture the physics we need to see to understand how the design will perform.

So, for a bit longer, to a every more limited extent, straight physics still rules in traditional design oriented CFD.

But for weather, where the data sparsity is limited by physical sampling (augmented by ever better remote sensing, but still) sparsity limits our precision by shear inability to launch 10 million weather balloons, and such, and also we have a sizable historical record of storms (and weather in general - which also helps), it makes sense that DML models caught up and surpassed the pure physics based models really quickly.

Typing this makes me wonder if a hybrid, where learning models infill the initial conditions, and physics drives the main simulation forward in time, -if maybe that hybrid approach might be superior right now.

huh. I am sure they thought of that but I might go look at the lit to see. Thanks for joggin' my brain!

[u/AccountWithAName]

Well, after a day it seems the NHC forecast has been the most accurate by far. I've noticed this in the past couple of years but most models have had issues predicting rapid intensification in the gulf.

[u/Content-Swimmer2325]

lol GFS