Evaluating Guidance for Planning Day 1 and Beyond

The Aviation Weather Center (AWC) does not currently produce a Day1, or longer, outlook for aviation planning.  One of the goals of this experiment was to evaluate guidance and the capability to produce a Day 1, as well as a Day 2-4 outlook for planning purposes.  What these products would look like and entail is still very much in the early stages.

The goal of the Day 1 product is to target extreme turbulence broken up into four categories; Marginal, Slight, Moderate, and High.  The Moderate and High categories would indicate areas where a SIGMET is most likely, with the greatest certainty contained in the High coverage areas.  The graphic above is an example of what this product could look like.  When it would be issued, and the audience of such a product still needs to be discussed at length.  Along with the graphic, an accompanying text product, or discussion explaining the coverage areas and reasoning may also be beneficial.

Guidance evaluated for this product included the operational "Graphical Turbulence Guidance", or the GTG, RAP GTG, and the FV3 GTG.  The operational and RAP GTG were very similar in signal, with the RAP version having hourly forecasts out to f21, which is very useful for planning in the extended period.  Noticeable differences were seen when comparing the GFS GTG and the FV3 GTG, with FV3 showing stronger signals over GFS.  An example is shown below, with FV3 in the top left and GFS on the bottom right.




The extended out, looking at Day2-4 would be focused on several phenomena that could have high impacts on flight planning.  These include snow, thunderstorms, winds, wind compression, low ceilings and visibilities, sleet/freezing rain, turbulence, and en-route icing.  This product would likely focus on weather activity affecting the core 30 airports to support NAM operations and PERTI outlooks.

 

In addition to making the outlook product, participants compared the new FV3 variables with GFS, to see if they were consistent.  Visibility in FV3 seemed to be more concentrated showing lower visibilities and less coverage when compared to the GFS.  The convective and icing fields showed similar outputs between the two models, with FV3 tending toward slightly higher intensities of icing.

When it came to creating the product, there was a difference observed in how the NAM particpant would create the forecast versus a forecaster from AWC.  The NAMs are very focused on  high impacts to the core 30 airports, and thus tended to make a quicker product with less model interrogation.  Whereas the AWC meteorologists tended to focus on all possible weather hazards regardless of specific airports providing more structured commentary and taking a longer time to produce the product overall.

The development of both products is still in the very early stages.  A lot more discussion and demonstration needs to take place before it would be looked at as a possible operational product.  However, the feedback received this week was a giant step in the forward direction to seeing how such a product would be beneficial for users at all levels. 

No Stratus, No Problems

Another focus of the experiment this summer was looking at high-resolution ceiling and visibility (C&V) model data over the San Francisco airport (SFO).  The airport utilizes a parallel runway system that requires pilots to be able to physically see each other as they make their landing approach.  When low ceilings are present (< 3000ft), the use of parallel approaches is restricted, causing decreased arrival capacity and increased delays. 

The UK Met Office and the ESRL Global Systems Division (GSD) HRRR group both provided high-resolution nests over SFO for this experiment.  The UK Met model nest has 330m grid spacing, with the experimental HRRR nest set at 750m horizontal grid spacing. 

As luck would have it, during the week of the experiment SFO was stratus free!  So instead, participants kept busy by looking at how the various model guidance handled the clouds and other phenomena in the area within the nests.  Focus was also given to Monterrey bay some days, which is just south of SFO.  In addition to the high-resolution nests from the UK Met model and the HRRR, participants also compared with experimental versions of the LAMP and RTMA.

In the above example, the HRRR 750m nest looks to best represent the cloud structure seen in the GOES 17 imagery west of Monterrey bay.  Another example below shows how the HRRR experimental starts to clear out the cloud cover inland of the bay by 15Z, where the experimental RTMA 3-D continues to keep the clouds at 15Z, which is later verified by satellite imagery.

In addition to clouds from the models, participants also looked at winds, the NDFD sky cover and how it compared to the guidance, as well as how RTMA and LAMP interpret observations into their analysis and forecasts.  So even without stratus, valuable information on the various guidance products was gained.

Thankfully, partners at both GSD and the UK Met office have agreed to continue producing the experimental nest data for a few more weeks so that AWC researchers can continue to evaluate the data when stratus is in fact present over SFO. 

Evaluating the GFA in the Testbed

One of the focuses this week is on evaluating the Graphical Forecasts for Aviation Tool, or the GFA.  This tool is a graphical source of weather elements that may impact general aviation (GA) flight.  The operational version covers the CONUS, and allowed the Aviation Weather Center (AWC) to retire the text-heavy Area Forecast (FA) product that was used to relay the same information. 

The GFA as viewed on the AWC website

In an effort to hopefully retire the other FAs still being issued by AWC, as well as the Met Watch Offices (MWOs) in Hawaii and Alaska, a version of the tool with an expanded domain was created and is in experimental mode for evaluation. 

In addition to the experimental version that covers the Caribbean, Gulf of Mexico, and parts of the Atlantic, participants this week have been given the chance to interact with and provide feedback on a version with coverage over the Pacific covering both Hawaii and Alaska FA areas.  The feedback collected this week will be vital in the process of moving forward with the GFA in the future to ensure it meets the needs of users at all levels.

Turbulence forecasted over the Alaska region as seen on the internal GFA with expanded coverage

In an effort to gauge how a pilot would use the tool, participants were asked to create forecasts and/or flight plans for various routes.  Sample GA routes covered the Bahamas, Alaska, Hawaii, and CONUS.  Users were given a 3 hour window to plan the flight based on the information the tool provided, and then later in the day were able to verify how the weather evolved compared to what the tool was showing.

In Alaska, the lack of observations proves challenging for pilots and forecasters alike.  The expanded GFA coverage into this region shows promise for both.  An example below shows the updated cloud algorithm in the GFA over the Alaska domain indicating clear conditions at the airport, but clouds to the west and south at 16Z. 

This verified in the 16Z observations, with the PAOT METAR showing clear skies with cloud visible in both the satellite and webcam which looks to the Southeast.

It was also discussed that the capability of adding the webcams from Alaska to the GFA would be greatly beneficial.  The webcams provide forecasters and pilots with visual observations in areas where Automated Service Observing System (ASOS) stations are sparse.

Other feedback that has been received thus far has focused on the overall usability of the tool.  The 'Settings' tab produces a menu that allows for users to customize the map, overlays, and other aspects of the tool.  The different options can be confusing to users, and could use clarification in the description, or perhaps a short tutorial or "how-to" on what they entail.  Modifying the Settings in general to have a separate tab specific to map options, and one that pertains to the forecast and observation displays, may also be beneficial.

Feedback with concerns to inconsistency were also brought up.  The CONUS, Puerto Rico, and Hawaii regions of the GFA include output from the NDFD where available, however other areas are filled in with model derived forecasts.  This was noticed over Hawaii where the NDFD showed a chance of precipitation, but the GFA model derived cloud cover indicated clear skies.  This inconsistency could lead to confusion and distrust of the product by the user.

Overall, the feedback on the tool has been quite positive.  The vast amount of information available is extremely useful to the general aviation community.  Continued developments and improvements will allow the product to meet the needs of all users.

2018 Summer Experiment Day 1

The Aviation Weather Testbed (AWT) 2018 Summer Experiment kicked off on Monday September 10th with a great mix of participants from across the aviation and research community.   The experiment this year is exploring super high-resolution forecasts for short-term planning as well as new global forecasting models for longer-range planning.

The three major areas of focus are:

Continued Evaluation of Super High-Resolution Cloud and Visibility Products for SFO Flight Operations

The Met Office is providing near-real time 330-m resolution forecasts of aviation impact variables to support experimental clearing time forecasts for SFO. In addition, GSD is providing a nest of the HRRR model as another source of guidance for this domain. GOES-17 and new versions of RTMA and RTMA-RU are also being evaluated as high-resolution observation sets.

Evaluation of the experimental expanded Graphical Forecasts for Aviation (GFA) tool
Currently, the Graphical Forecasts for Aviation with an expanded domain is available as an experimental product via https://testbed.aviationweather.gov/gfa. This product will go through a more thorough evaluation process with AWT and FAA Aviation Weather Demonstration and Evaluation (AWDE) services personnel interacting with end-users during the experiment.  An internal version of the GFA tool expanded out into the Pacific to Hawaii and north to cover the Alaska domain is also being evaluated by potential users.

Evaluation of guidance for enhanced Day 1 and extended Day 2-4 planning
The AWC is currently discussing the potential to issue a produce similar to the SPC Day 1 product for severe thunderstorms but instead for severe low and high turbulence and significant mountain wave activity. This involves the evaluation of high-resolution guidance for severe turbulence signals out to 36 hours in the future. In addition, AWT participants will be asked to evaluation experimental FV3GFS guidance for the prediction of longer range impacts.

Day 1 Debrief:

The first day is usually a time for participants to get acquainted with the various desks and tasks.  Participants are given free range of the various desks and can migrate to those that are of most interest to them.  The brownbag seminar during lunch was given by Rune Duke of the Aircraft Owners and Pilots Association (AOPA) about the general aviation perspective for aviation weather.

SFO Desk:

With no stratus day at SFO, the focus switched from forecasting a clearing time to looking at the different guidance output to see how they performed on the surrounding clouds and weather.

The model10z runs showing varying forecasts over SFO

GOES17 imagery over SFO at 15Z shows clouds creeping inland south of SFO.  The UK Met model seemed to handle this feature quite well.  The benefit of having the higher resolution model nest is that the impact from topographic features is are better resolved.

An interesting feature in the LAMP guidance was noted based on how LAMP processes forecasts. The gridpoint nearest to SFO indicates a clear forecast, but the surrounding area suggests low ceilings at SFO. This is likely because the grid was postprocessed to be consistent with the LAMP station forecast. The forecast was likely affected by the lack of a ceiling observed at the METAR. 

Day1, Day 2-4 Outlooks:

There was discussion at the Day 1 desk that a text product would likely be needed in addition to the Day 1 graphic outlook.  Exactly how this would look and what it would entail is something for other participants to think about during the week.  The forecasters at the desk started with a general overview of weather pattern, and then gave brief discussions on each of the turbulence categories depicted in the graphic (High, Low, and Mountain Wave).  The particpants used FV3 and GFS guidance to depict areas of turbulence over the CONUS.

At the Day 2-4 desk, forecasters focused on multiple impacts that would have high impact on flight planning through the period.  This includes snow, thunderstorms, winds, wind compression, low ceilings and visibilities, sleet/freezing rain, turbulence, and en-route icing.

 

The purpose of the maps is to target weather activity that could impact travel at the core 30 airports.  Sometimes that means ignoring large events over the midwest region and instead highlighting small areas with high impact to airline traffic like New York.  Ceiling and Visibility impacts can vary by terminals, so areas drawn to indicate low cig/vis could vary in type based on the terminals being impacted and their specific criteria.

One of the goals for this desk is to compare guidance provided by GFS and FV3.  The above graphic shows a comparison of the two models visibility fields for the same forecast time.  It's easy to see the difference in coverage, with the FV3 showing less coverage of low visibilities over all.  However it seems the FV3 is indicating a larger swath of visibilities less than 1 mile up over the Northeast compared to the visibilities in the GFS which are more varied in the region ranging from less than 5 miles to less than 1.

GFA Expansion Evaluation:

The goal of the GFA expansion evaluation is to get any and all feedback from users on the usability of the tool for flight planning.  To do this, participants from various regions will be doing flight plans each day using the tool, and then seeing how those forecasts verify.   With an internal version expanding out into the Pacific ocean to Hawaii, and up into Alaska, local expertise is critical.  Fortunately, participants from the expanded domain are here to help and give vital feedback to their regions.  Two areas of focus are the Caribbean and Alaska. 

An example of the clouds from GFA over Alaska

An example of the AAWU icing forecast (left) compared to the GFA icing forecast on the right.

The first day was spent getting aquatinted with the tool and gathering feedback from the users on what they liked and didn't like about the tool overall.  The rest of the week will be spent making forecasts for sample flight routes, allowing for even further evaluation of the tool's use and performance for general aviation.