Monday, December 10, 2012

Analysis of Forecaster Performance for Dec. 9 Storm


The final total for this weekend’s snowstorm was 10.5 at the official MSP airport recording station. The core metro area snowfall ranged from 9 to 14 inches with an even foot of snow a common observation, including here at TMF headquarters in St. Paul.

No forecaster anticipated the possibly of a truly major snowfall until late Saturday afternoon. Even then, most forecasts did not fully capture the extent of the burgeoning snowstorm. (Note, for weather geeks looking to get a meteorological explanation for the late-changing forecast, MPR provides a good analysis here.)

Rather than provide forecaster grades as we have in the past, we’ll focus on a few general observations. Given that we don’t capture forecast activity in a truly systematic, day-by-day manner, we feel that the assignment of grades suggests a level of authority and exactness we simply don’t have. Still, we believe we monitor the forecasts enough to offer reasonable opinions. As always, take them for what they’re worth. We invite readers to provide grades and thoughts in the comment section to this post.

Our observations:

The best performers for this storm were the National Weather Service (NWS) and Novak Weather. Most other forecast outlets did not distinguish themselves.

Virtually all forecasters eyed this storm for the better part of a week. However, several minimized the storm as recently as Thursday. On Thursday, KMSP tweeted, “Winter returns the next few days, flurries late Fri afternoon, flurries Sat and wicked cold by Mon.” On Thursday evening, the Star Tribune noted the possibility of 1-3 inches of “oatmeal-like slush,” jokingly suggesting it would be cause to contact MnDOT and the National Guard. Likewise, the Weather Channel noted the possibility of mere “snow showers” over the weekend. At the same time, however, other forecasters were not ready to conclude the storm would be a minimal, back-page item.

KMSP was Jekyll & Hyde on this one. After the overly minimizing tweet on Thursday, they had the best standing prediction on Friday night with a forecast of 6-8 inches. But by Saturday, they’d revised it downward to 4.1 inches (the “snow meter.”)

The NWS consistently predicted higher amounts than the majority of forecasters 36 hours prior to the storm’s arrival. Some readers/viewers may have wondered whether a winter storm warning for MSP would prove to be warranted; clearly, it was.

Novak Weather also lead the way on this forecast, staying the course of a forecast of 6+ inches for MSP from early on. On Saturday afternoon, Novak Weather was also the first to communicate the possibility that accumulations could “get out of hand quickly for MSP” on Sunday as they ultimately did.

The separation of men from boys on this forecast took place Saturday afternoon and evening. Some forecasters picked up on the late-developing possibility of a truly significant snowfall while others did not. As detailed in the previous post, forecasts by KSTP (Jonathan Yuhas) and KMSP (Steve Frazier) missed the boat in not noting the late change in the storm’s evolution (though it should be noted that other members of the KSTP forecasting team tweeted 5-8 inch forecasts at the same time Yuhas was on air forecasting 3-6 inches).

35 comments:

  1. Reposting an appropriate comment from Duane that appeared at the end of the last post:

    This storm ended up performing beyond what most of the models showed. While they did have some areas near a foot of snow I don't think they had it as widespread as they did. As far as the grading goes, well I think Yuhas and whoever did the snow tracker thing on Fox (Steve, I think?) didn't do so hot. I will say the early stages of this storm showing on the models was very flip floppy and everyone was confused. Very common with a pattern change to see that much flip flopping going on. However 12 to 24 hours before the event they should have really locked in on much higher totals than they forecast. I'm not sure what model the snow tracker is (whether it is a version of RPM or what), but when that showed as low as it did, it should have been thrown out. Everything else was pointing at a much higher storm total. The general trend was a slight bump south with each and every run so their forecast totals should have gone up. I don't like when someone just runs one model, and whatever number pops up on the screen is what they go with. To me, that isn't forecasting. That is pushing a button, and just writing down whatever shows. I'm not trying to sound like a jerk here because I get it that these storms can be really tough to forecast. Lots of different factors come into play here. Also it isn't too common for forecasters to put such high predictions on their air, for fear of the bust potential. For me, an 8-12+ prediction would have covered things just fine 12 to 24 hours out, and then adjust those predictions as snow reports come in. I think the NWS did a fantastic job overall. They noticed the southern trend, adjusted the headlines accordingly, and adjusted totals accordingly. Their discussions were detailed, and they even wondered if their prediction last night would be high enough. Now our attention turns to next weekend, as potentially things could come together for another accumulating snow event. Zero model agreement so far so this one is far from a lock. Until next time!

    ReplyDelete
    Replies
    1. Thanks Bill! Appreciate it. Since I don't live in the Twin Cities market I do tend to miss a lot of their forecasts so it is nice seeing them posted for comparison. I should have mentioned props to Novak as well for staying the course. He takes a lot of grief on here when a storm doesn't quite pan out but he does have a lot more knowledge than the rest of us. I can't help but notice how quiet our "brown Christmas" folks have been as of late. We saw this pattern change coming over a week ago, and it panned out as planned. See...no doubt of a white Christmas this time around!

      Delete
  2. Props to the NWS & Novak! We picked up a cool foot of the beloved white stuff here in Rosemount yesterday. I know it's only Monday afternoon but what are the computers saying about the possible snowstorm for next weekend? Are the models still all over the board about that?

    ReplyDelete
  3. I don't know where people are getting the snow idea for next weekend. Neither the NAM nor GFS are showing anything

    http://bit.ly/RYrDZf

    ReplyDelete
  4. I think I would make a few comments reinforcing the very good points made by Duane.

    1) This storm emphasized the difference between TV forecasters and meteorologists. The first simply put on screen the model output (something that really anybody can do), the real meteorologists add their touch to it. No surprise then that the NWS handled this pretty well. Detailed discussions and most importantly explaining why they were discarding some models when outliers. It is and must be reassuring to all of us that when big storms come the NWS does a very good job about it.

    2) Ian Leonard and Jonathan Yuhas were the real losers here: from that "flurries Friday, flurries Sunday" tweet that turned out completely out of touch to Yuhas' insistence to go for 3-6 on Saturday night when all models had caught on the southern shift. If you cannot even provide a good enough forecast a few hours before, maybe weather forecasting is not your forte.

    3) The real winner here was Mother Nature and I think all of us snow lovers should give Her a great round of applause and a big THANK YOU!. What a storm!!

    ReplyDelete
  5. The ECMWF shows something for the weekend. Maybe someone can give more specifics on its possibilities.

    ReplyDelete
  6. Ahhhh @Disco80 always the one showing up and crashing the snow party,didn't here a peep from you all weekend,how come?do you like that foot of white stuff in your yard?was it cold enough for you today?and your commute,did you make it to work on time?Let us snow lovers enjoy the winter and all its glory without your 'warm' thoughts and 'no' snow talk.Thank you!

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. Are you really so insecure that your enjoyment of winter is diminished by a comment on a blog? Wow dude.

      Delete
  7. 12z Europeans are bullish on this weekend. With that being said, you know what I'm thinking.

    This storm scares me. It is much, MUCH more difficult committing to a snow scenario with these deep digging southern lows. Plus, when these "hooking" storms coming out of the southern Rockies/Plains, they tend to take erratic paths and can really burn you with the rain/snow line.

    Right now, no matter what develops and/or what path this storm takes, I'm simply not sure if it has a reservoir of cold Arctic air to tap. The beauty of this last storm was that cold air was never in question.

    Should be fun to monitor.

    ReplyDelete
  8. First sub zero temps tonight. NWS has 1 above for where I live

    ReplyDelete
  9. The late pm NWS discussion does note the possibility of a Friday night/Saturday storm:

    THE MAIN FORECAST CONCERN FOR THE LONG TERM IS THE SYSTEM FOR
    FRIDAY NIGHT AND SATURDAY. THE DETERMINISTIC MODELS HAVE BEEN TOO
    WEAK AND SUPPRESSED WITH THIS SYSTEM WITH THE ENSEMBLE MEAN A BIT
    FURTHER NORTH AND STRONGER...THE PREFERRED SOLUTION OF HPC. STILL
    TOO EARLY TO NARROW DOWN A LIKELY TRACK...BUT THE EC AND EC
    ENSEMBLE MEAN HAVE BEEN CONSISTENT FOR THE PAST FEW RUNS TRACKING
    IT THROUGH IOWA INTO SRN WI. THIS WOULD BE A FAVORABLE TRACK FOR
    ACCUMULATING SNOW OVER ERN MN AND WRN WI. THEREFORE...KEPT MEDIUM
    RANGE POPS ACROSS THE EAST...WITH LOW POPS WEST.

    ReplyDelete
  10. 18z GFS is coming in line with the Euro's look, but what Novak said about the arctic air being available is correct. This storm is likely not looking that great due to the fact that the arctic air isn't there. That temp contrast is what really helps spin up these big storms. If the models trend it stronger, that means that the arctic air may eventually be more available than what is shown now. It certainly bears watching since the path of this storm is very typical of a big snow producer but all the other ingredients have to be there too. I do want to clarify that I really do believe not every TV meteorologist is simply a button pusher. There are those out there that look at everything, and do a great job with the forecasts. Unfortunately they get a lot of slack from people when they are wrong, but don't hear much of anything when they are correct. For now, we watch and wait for this next storm and see if it fizzles, moves away, brings us more accumulating snow, or just brings us a mess of everything.

    ReplyDelete
  11. Ok, first things first.

    @ Bill, I know your much busier now than when you started this site, thanks for keeping it going.

    @Novak, Great job with the pattern recognition, you were the lone local met (outside of MPX) to raise the bar versus what most models were showing for MSP.

    @MPX, Superior Job as well with the pattern recognition and posting the Winter Storm Watch as early as you did.

    @Duane for insightful post's, particularly the recap of this event (I think of this as a event more so than a storm)

    @Big Daddy and this is kind of personal note. I am also much busier than the last two winter's, so I may not be able to put out my four quadrant snow fall forecast.

    This is my last day off before Xmas and I agree with Novak, the Foreign models are much more aggressive with this system next weekend. (Just checked the 18z GFS and it made a move to the NW, with snow very close to MSP.) At this time (and this is not a forecast just saying what the models are showing,) I'm thinking any snow fall should be confined to Western WI and Eastern MN including the eastern metro. Having said all that, the UKMET operational has a surface low over ARX, but it's really hard to tell if its cold enough for snow, my guess is yes, if that track verifies, than all of the metro could be in for round two.

    In the meantime as things look now anyone taking a road trip next weekend should pay close attention.

    ReplyDelete
  12. To put it lightly I definitely swung and missed on the snow forecast for the Twin Cities this weekend. Good thing Bill didn't post my forecast! What an awesome and dynamic system though, and nice job to the NWS and Novak for having the best forecasts. I agree so far with what's being said for this weekends possible storm. The ECMWF goes back and forth on whether or not there's cold enough air available. Hopefully with our snowpack here and to the north we can find enough cold air for snow. Rain on top of this much snow - yuck.

    ReplyDelete
    Replies
    1. especially if we don't get rid of these washboard roads. Rain on top of this, without clear roads, and than more freezing after rain?? Head for the hills!!! LOL we should see moderating temps allowing the chemicals to do their work.

      @Randy I was going with the Euro, hook line and sinker, just about every time I throw it under the bus it comes back to bite me in a uncomfortable place. This time it bit me bad in the wrong way, I finally chucked it Sat night.

      Delete
  13. Thank goodness for the deep snow cover. Without the snow cover we would be looking at highs of at least 40 degrees everyday for the next week. It would be warm and brown into the heart of December just like last season. The serious cold continues to reside in Alaska...again just like last season. I'm not in favor of brutal cold, but I do hope that the weather around here remains cold enough to support snow with any future weather systems. I'm curious as to whether anyone sees any indication that a consistent chunk of the mother lode of arctic air is going to break off and head our way?

    ReplyDelete
  14. For me, the big wild card for the winter is the impressive snowpack that was put down across Canada back in November, this is the first time since the 08-09 winter that there has been a healthy (i.e. deep) snowpack across the Canadian Prairies, and this should aid in dislodging some of the cold from Alaska and the Canadian arctic. For me, the question is does it sustain itself for very long, or do we get quick hits. Now that there is a fresh 10 inches of snow cover from SODAK into MN, we will certainly have a better chance at sustaining the cold (as Snow Miser pointed out, w/out all of this snow, we would be pleasantly back in the 40s for a couple of days this week!). Looking out into the future the CFS (Climate Forecasting System)says the next 1 to 2 weeks will be more of the same (no significant cold), but weeks 3 and 4? Well.......

    CFS Weeks 1 and 2 temp outlook: http://is.gd/oE0PkK
    CFS Weeks 3 and 4 temp outlook: http://is.gd/gbiPbR

    ReplyDelete
  15. If you were to go by the 12Z run of the EMCWF you would see the SE part of MN getting around 5-6 inches of snow this weekend.

    ReplyDelete
  16. All the 12z medium range model runs look way too enticing for Saturday. Is this for real? You know what they say, if it looks too good, it probably is. The next several days should be interesting though.

    You gotta love how the models are digging this baby deep into the southern Rockies before ejecting it north into the Midwest as it taps cold air just in time on Saturday. We should have no problems with moisture.

    ReplyDelete
  17. @Bill....I know this blog could get overwhelming at times,keeping up with the changing forecasts(especially last storm)but I appreciate all you do and keping this blog going.I believe you nailed the assessments,NWS and Novak were the clear winners(Novak you do take a lot of crap,so when you nail you deserve the props,good job!)
    I do like to say this past storm is a classic reason you can never hone in on one solution to early,and disregard model runs like some do,I personally hate it when forecasters say"this model run is junk,or out to lunch,or I don't. Give too much credit to the 18z or 0z",to me that means your disregarding a possible trend.
    On a side note KSTP/Dave Dahl needs to stop Yuhas from sending out the tweets he does,makes the whole station look bad(remember the Twins game debacle),also do we all agree the 'snow-meter' needs to go!
    and finally NWS has snow likely wording in the grids for Saturday already,sounds like confidence is rising,even mentioning in discussion that warning level snows of 3-6 is possible...we'll see but I hope the train of snowstorms contiue

    ReplyDelete
    Replies
    1. Thanks, Big Daddy. Glad to know you value the blog. I sometimes wonder whether it's worth gathering all the individual forecasters... but I do know the discussion among everyone is nice.

      Delete
  18. I have a question that I would love to hear you folks address. It's something that I've observed over the years as a weather geek.

    Once or twice each winter, when we get a significant storm, it is followed up by another significant storm in roughly the same area or close proximity about 6-8 days later. This seems to be happening again this week. This actually seems very common to me, so when we get a storm, I'm always looking ahead to the forecast for a week later to see what else might be coming down the pipe.

    Your thoughts?

    ReplyDelete
  19. Plymouth Weather LoverDecember 11, 2012 at 8:55 PM

    Two weekends in a row for snow. My bring it is working. C'mon. Bring it!!!!

    ReplyDelete
  20. Guys at HPC like the Euros handling of this. Its the faster and more northern track than the GFS is putting out. Canadian shows the most moisture giving the NWS La Crosse part of Minnesota a solid .6 to .9 inches of precip.

    http://kamala.cod.edu/offs/KWNH/1212120647.fxus10.html

    ReplyDelete
  21. Unfortunately there is a lot of talk this morning about rain/freezing rain instead of snow for this upcoming weekend storm. The cold air just absolutely refuses to settle in here. Therefore at this time it appears there will be too much warm air involved with this system to give us another snowstorm. I hope that changes. It's almost mid-December in Minnesota and a storm is predicted to take a classic Colorado Low track that should give the metro a good-sized snowstorm. Snow lovers should be rejoicing, but instead there are great concerns about whether we will even get snow. Sad and pathetic! Unlike Chanhassen, the NWS in La Crosse thinks it will be all snow north and west of Rochester. Let's hope they are right.

    ReplyDelete
  22. Current thinking based on the 10/12 0z forecast sounding from the Euro and bufkit soundings from the 12/06 Nam it looks like freezing rain is possible before 6am as the Temp profile follows the freezing line up through about 4 thousand feet. After that both show it cold enough for all snow. I'm sure that will change several times before the event. At the surface before 6am the temps are just barely above freezing so therefore I believe cold surfaces (concrete) could be cool enough to ice over.

    ReplyDelete
  23. Novak's latest video - https://www.youtube.com/watch?feature=player_embedded&v=mAxlL5FdnD8#!

    ReplyDelete
  24. It sucks that we have to talk/think about rain when it is taking a classic path for us to receive another health dose of snow,but I'll take heavy wet snow at 32-33 anyday of the week,with some big ole snowflakes(perfect snowman making snow)then freezing rain........but Novak was on the money again when he mentioned earlier in the week that we will have to watch the rain/snow line and the warm air aloft(Novak your on your game this year!)
    But this will be a very fun storm to mentior over the coming days because there are a lot of possibilities in terms of precip type and how much snow can fall because of that,so a lot of forecasters will be afraid to pull the trigger again,I feel a lot will start out conservative again.

    ReplyDelete
  25. The way things look now, this will be a tricky forecast for meteorologists. Models are starting to converge on a solution but there is still a lot up in the air. The previous storm, the models had a northern bias and shifted south eventually. It will be interesting to see if this does the same. It is still too early to determine exactly where the rain/snow line will set up but overall it looks messy. For now, I would say areas along and south of I-90 stand the best chance at seeing the longest duration of rain, with areas north seeing the most snow. For now, accumulations aren't looking really impressive (2-4 inches). Again, it's early and this could easily change upward or downward. If precip rates get high enough then that could help cool the atmosphere more rapidly, and force the change over faster. We probably won't have a better lock on this until Friday afternoon and evening when the trough comes into the southwest US. Best thing for snow would be for the low to not make it as far north, and take a track across northern IA into Illinois. It's a waiting game for now.

    ReplyDelete
  26. GFS has moved north to join the Euro/CMC. Looks like a real tricky forecast because of temps. Warm air aloft around 850mb (5000 ft) and near freezing at surface suggest freezing rain/rain to start. IMO this is a more interesting system than last weeks. Might get as much precip, but what kind will it be?

    ReplyDelete
  27. Looking at the 12Z ECMWF the temps never get above zero if you were to only use the 850MB value. Of course you can't only use that. As of now if shows QPF .5 of precip through SE MN into West Central and points east into the northern 1/2 of Wisconsin. For snow it shows from a few inches around KRST and 3-5 around MSP. Heading towards Green Bay they are in the bullseye where it shows up to 8 inches. AGAIN THIS IS ONLY ONE MODEL :-D

    ReplyDelete
  28. Looking at the GFS it appears some serious cold moves in around the 22nd of Dec.

    ReplyDelete
  29. In Dave Dahl's afternoon blog he mentions,"its not out of the question that somewhere in southern MN could get a foot of snow again",he says this in regards to where he thinks the heaviest snow will fall from Fri. Night/Saturday system.
    I bet that would make PWL pee in his pants again!
    @Novak I'm thinking at some point tommorrow NWS would need to put out some type of headline with watches/warnings/advisories,don't you think?,it would be less then 48 hours away.

    ReplyDelete