Tuesday, September 25, 2018

Evaluating Guidance for Planning Day 1 and Beyond

The Aviation Weather Center (AWC) does not currently produce a Day1, or longer, outlook for aviation planning.  One of the goals of this experiment was to evaluate guidance and the capability to produce a Day 1, as well as a Day 2-4 outlook for planning purposes.  What these products would look like and entail is still very much in the early stages.

The goal of the Day 1 product is to target extreme turbulence broken up into four categories; Marginal, Slight, Moderate, and High.  The Moderate and High categories would indicate areas where a SIGMET is most likely, with the greatest certainty contained in the High coverage areas.  The graphic above is an example of what this product could look like.  When it would be issued, and the audience of such a product still needs to be discussed at length.  Along with the graphic, an accompanying text product, or discussion explaining the coverage areas and reasoning may also be beneficial.

Guidance evaluated for this product included the operational "Graphical Turbulence Guidance", or the GTG, RAP GTG, and the FV3 GTG.  The operational and RAP GTG were very similar in signal, with the RAP version having hourly forecasts out to f21, which is very useful for planning in the extended period.  Noticeable differences were seen when comparing the GFS GTG and the FV3 GTG, with FV3 showing stronger signals over GFS.  An example is shown below, with FV3 in the top left and GFS on the bottom right.

The extended out, looking at Day2-4 would be focused on several phenomena that could have high impacts on flight planning.  These include snow, thunderstorms, winds, wind compression, low ceilings and visibilities, sleet/freezing rain, turbulence, and en-route icing.  This product would likely focus on weather activity affecting the core 30 airports to support NAM operations and PERTI outlooks.


In addition to making the outlook product, participants compared the new FV3 variables with GFS, to see if they were consistent.  Visibility in FV3 seemed to be more concentrated showing lower visibilities and less coverage when compared to the GFS.  The convective and icing fields showed similar outputs between the two models, with FV3 tending toward slightly higher intensities of icing.

When it came to creating the product, there was a difference observed in how the NAM particpant would create the forecast versus a forecaster from AWC.  The NAMs are very focused on  high impacts to the core 30 airports, and thus tended to make a quicker product with less model interrogation.  Whereas the AWC meteorologists tended to focus on all possible weather hazards regardless of specific airports providing more structured commentary and taking a longer time to produce the product overall.

The development of both products is still in the very early stages.  A lot more discussion and demonstration needs to take place before it would be looked at as a possible operational product.  However, the feedback received this week was a giant step in the forward direction to seeing how such a product would be beneficial for users at all levels. 

Monday, September 24, 2018

No Stratus, No Problems

Another focus of the experiment this summer was looking at high-resolution ceiling and visibility (C&V) model data over the San Francisco airport (SFO).  The airport utilizes a parallel runway system that requires pilots to be able to physically see each other as they make their landing approach.  When low ceilings are present (< 3000ft), the use of parallel approaches is restricted, causing decreased arrival capacity and increased delays. 

The UK Met Office and the ESRL Global Systems Division (GSD) HRRR group both provided high-resolution nests over SFO for this experiment.  The UK Met model nest has 330m grid spacing, with the experimental HRRR nest set at 750m horizontal grid spacing. 

As luck would have it, during the week of the experiment SFO was stratus free!  So instead, participants kept busy by looking at how the various model guidance handled the clouds and other phenomena in the area within the nests.  Focus was also given to Monterrey bay some days, which is just south of SFO.  In addition to the high-resolution nests from the UK Met model and the HRRR, participants also compared with experimental versions of the LAMP and RTMA.

In the above example, the HRRR 750m nest looks to best represent the cloud structure seen in the GOES 17 imagery west of Monterrey bay.  Another example below shows how the HRRR experimental starts to clear out the cloud cover inland of the bay by 15Z, where the experimental RTMA 3-D continues to keep the clouds at 15Z, which is later verified by satellite imagery.

In addition to clouds from the models, participants also looked at winds, the NDFD sky cover and how it compared to the guidance, as well as how RTMA and LAMP interpret observations into their analysis and forecasts.  So even without stratus, valuable information on the various guidance products was gained.

Thankfully, partners at both GSD and the UK Met office have agreed to continue producing the experimental nest data for a few more weeks so that AWC researchers can continue to evaluate the data when stratus is in fact present over SFO. 

Thursday, September 13, 2018

Evaluating the GFA in the Testbed

One of the focuses this week is on evaluating the Graphical Forecasts for Aviation Tool, or the GFA.  This tool is a graphical source of weather elements that may impact general aviation (GA) flight.  The operational version covers the CONUS, and allowed the Aviation Weather Center (AWC) to retire the text-heavy Area Forecast (FA) product that was used to relay the same information. 

The GFA as viewed on the AWC website

In an effort to hopefully retire the other FAs still being issued by AWC, as well as the Met Watch Offices (MWOs) in Hawaii and Alaska, a version of the tool with an expanded domain was created and is in experimental mode for evaluation. 

In addition to the experimental version that covers the Caribbean, Gulf of Mexico, and parts of the Atlantic, participants this week have been given the chance to interact with and provide feedback on a version with coverage over the Pacific covering both Hawaii and Alaska FA areas.  The feedback collected this week will be vital in the process of moving forward with the GFA in the future to ensure it meets the needs of users at all levels.

Turbulence forecasted over the Alaska region as seen on the internal GFA with expanded coverage

In an effort to gauge how a pilot would use the tool, participants were asked to create forecasts and/or flight plans for various routes.  Sample GA routes covered the Bahamas, Alaska, Hawaii, and CONUS.  Users were given a 3 hour window to plan the flight based on the information the tool provided, and then later in the day were able to verify how the weather evolved compared to what the tool was showing.

In Alaska, the lack of observations proves challenging for pilots and forecasters alike.  The expanded GFA coverage into this region shows promise for both.  An example below shows the updated cloud algorithm in the GFA over the Alaska domain indicating clear conditions at the airport, but clouds to the west and south at 16Z. 

This verified in the 16Z observations, with the PAOT METAR showing clear skies with cloud visible in both the satellite and webcam which looks to the Southeast.

It was also discussed that the capability of adding the webcams from Alaska to the GFA would be greatly beneficial.  The webcams provide forecasters and pilots with visual observations in areas where Automated Service Observing System (ASOS) stations are sparse.

Other feedback that has been received thus far has focused on the overall usability of the tool.  The 'Settings' tab produces a menu that allows for users to customize the map, overlays, and other aspects of the tool.  The different options can be confusing to users, and could use clarification in the description, or perhaps a short tutorial or "how-to" on what they entail.  Modifying the Settings in general to have a separate tab specific to map options, and one that pertains to the forecast and observation displays, may also be beneficial.

Feedback with concerns to inconsistency were also brought up.  The CONUS, Puerto Rico, and Hawaii regions of the GFA include output from the NDFD where available, however other areas are filled in with model derived forecasts.  This was noticed over Hawaii where the NDFD showed a chance of precipitation, but the GFA model derived cloud cover indicated clear skies.  This inconsistency could lead to confusion and distrust of the product by the user.

Overall, the feedback on the tool has been quite positive.  The vast amount of information available is extremely useful to the general aviation community.  Continued developments and improvements will allow the product to meet the needs of all users.

Tuesday, September 11, 2018

2018 Summer Experiment Day 1

The Aviation Weather Testbed (AWT) 2018 Summer Experiment kicked off on Monday September 10th with a great mix of participants from across the aviation and research community.   The experiment this year is exploring super high-resolution forecasts for short-term planning as well as new global forecasting models for longer-range planning.

The three major areas of focus are:

Continued Evaluation of Super High-Resolution Cloud and Visibility Products for SFO Flight Operations

The Met Office is providing near-real time 330-m resolution forecasts of aviation impact variables to support experimental clearing time forecasts for SFO. In addition, GSD is providing a nest of the HRRR model as another source of guidance for this domain. GOES-17 and new versions of RTMA and RTMA-RU are also being evaluated as high-resolution observation sets.

Evaluation of the experimental expanded Graphical Forecasts for Aviation (GFA) tool
Currently, the Graphical Forecasts for Aviation with an expanded domain is available as an experimental product via https://testbed.aviationweather.gov/gfa. This product will go through a more thorough evaluation process with AWT and FAA Aviation Weather Demonstration and Evaluation (AWDE) services personnel interacting with end-users during the experiment.  An internal version of the GFA tool expanded out into the Pacific to Hawaii and north to cover the Alaska domain is also being evaluated by potential users.

Evaluation of guidance for enhanced Day 1 and extended Day 2-4 planning
The AWC is currently discussing the potential to issue a produce similar to the SPC Day 1 product for severe thunderstorms but instead for severe low and high turbulence and significant mountain wave activity. This involves the evaluation of high-resolution guidance for severe turbulence signals out to 36 hours in the future. In addition, AWT participants will be asked to evaluation experimental FV3GFS guidance for the prediction of longer range impacts.

Day 1 Debrief:
The first day is usually a time for participants to get acquainted with the various desks and tasks.  Participants are given free range of the various desks and can migrate to those that are of most interest to them.  The brownbag seminar during lunch was given by Rune Duke of the Aircraft Owners and Pilots Association (AOPA) about the general aviation perspective for aviation weather.

SFO Desk:
With no stratus day at SFO, the focus switched from forecasting a clearing time to looking at the different guidance output to see how they performed on the surrounding clouds and weather.

The model10z runs showing varying forecasts over SFO

GOES17 imagery over SFO at 15Z shows clouds creeping inland south of SFO.  The UK Met model seemed to handle this feature quite well.  The benefit of having the higher resolution model nest is that the impact from topographic features is are better resolved.

An interesting feature in the LAMP guidance was noted based on how LAMP processes forecasts. The gridpoint nearest to SFO indicates a clear forecast, but the surrounding area suggests low ceilings at SFO. This is likely because the grid was postprocessed to be consistent with the LAMP station forecast. The forecast was likely affected by the lack of a ceiling observed at the METAR. 

Day1, Day 2-4 Outlooks:

There was discussion at the Day 1 desk that a text product would likely be needed in addition to the Day 1 graphic outlook.  Exactly how this would look and what it would entail is something for other participants to think about during the week.  The forecasters at the desk started with a general overview of weather pattern, and then gave brief discussions on each of the turbulence categories depicted in the graphic (High, Low, and Mountain Wave).  The particpants used FV3 and GFS guidance to depict areas of turbulence over the CONUS.

At the Day 2-4 desk, forecasters focused on multiple impacts that would have high impact on flight planning through the period.  This includes snow, thunderstorms, winds, wind compression, low ceilings and visibilities, sleet/freezing rain, turbulence, and en-route icing.


The purpose of the maps is to target weather activity that could impact travel at the core 30 airports.  Sometimes that means ignoring large events over the midwest region and instead highlighting small areas with high impact to airline traffic like New York.  Ceiling and Visibility impacts can vary by terminals, so areas drawn to indicate low cig/vis could vary in type based on the terminals being impacted and their specific criteria.

One of the goals for this desk is to compare guidance provided by GFS and FV3.  The above graphic shows a comparison of the two models visibility fields for the same forecast time.  It's easy to see the difference in coverage, with the FV3 showing less coverage of low visibilities over all.  However it seems the FV3 is indicating a larger swath of visibilities less than 1 mile up over the Northeast compared to the visibilities in the GFS which are more varied in the region ranging from less than 5 miles to less than 1.

GFA Expansion Evaluation:

The goal of the GFA expansion evaluation is to get any and all feedback from users on the usability of the tool for flight planning.  To do this, participants from various regions will be doing flight plans each day using the tool, and then seeing how those forecasts verify.   With an internal version expanding out into the Pacific ocean to Hawaii, and up into Alaska, local expertise is critical.  Fortunately, participants from the expanded domain are here to help and give vital feedback to their regions.  Two areas of focus are the Caribbean and Alaska. 

An example of the clouds from GFA over Alaska

An example of the AAWU icing forecast (left) compared to the GFA icing forecast on the right.

The first day was spent getting aquatinted with the tool and gathering feedback from the users on what they liked and didn't like about the tool overall.  The rest of the week will be spent making forecasts for sample flight routes, allowing for even further evaluation of the tool's use and performance for general aviation.

Wednesday, August 23, 2017

2017 Summer Experiment Week 2 Wrap-Up

The 2017 AWT Summer Experiment came to a close on Friday with the weekly wrap-up sessions for each of the desks.  One of the benefits of running the experiment over a two week period, is having a different set of participants each week with various backgrounds and experience.  This allows for new perspectives and valuable feedback from week two that may not have come up during week one.

During Friday's breakout sessions, participants were able to give feedback from their perspective on the TCF forecast creation process, the use of the UK-Met model to predict clearing times at SFO, and using the national grids of clouds and visibility at the GFE desk to give a first-guess for the digital aviation forecast.

The SFO desk had the addition of the 1.5km UK met model this week, which is what the 330m model is nested from.  This week also presented very unique synoptic set-ups over the different days, with both pre and post trough situations.  It was noted that post-trough SW flow can occasionally lead to convective strato-cumulus formation, but also help to dry out the cloud.  On Wednesday, the 330m high-res model picked up on initiation of convective cumulus off the peninsula, which no other model did.  On Thursday, the 1.5km model actually did a better job than the 330m model.  Further evaluation of the general weather picture would be useful in this case to understand why.

Clouds and Visibility / GFE
Once again the overwhelming feedback from the GFE C&V grids this week was about the detail and speckle nature of the high resolution model grids.  While detail is good, too much can be overwhelming on a national level.  The challenge going forward seems to be finding balance between when to leave detail in, when to smooth out, and which level will do what.  An idea was brought up in the final discussion about using image processing methods in the CloudBasePrimary grid to identify areas first, then build up a consistent cloud base from there.  This would allow the details to be smoothed more intelligently.

In this post-OPG processed cloud base primary grid, you can really see where the local WFOs smoothed out the entire area to what they determined to be a reasonable cloud base, in response to the speckle nature of the grid they were handed

Another point that came up again was the need to have some means of filling in the CloudBaseSecondary grid with a first guess or starting point.  This could be using a tool based on LCL or probability of thunder, or even building in a first guess from model data in a similar fashion to how CloudBasePrimary is initialized.  The other repeating feedback to come up was the look of the AWC-model created SKY grids.  They appear too binary, and difficult to pass onto a local level.  Where the national blend seems to look more realistic like a satellite image.  The question is, how realistic is this far out into the forecast time?

As far as the actual editing procedure of the girds and workload, it was noted that much more coordination would need to take place with the local WFOs.  This would be most useful in areas deemed high-impact for aviation that day.  While the local offices may already be busy collaborating with other offices and national centers if it is a significant weather event, hopefully handing them a better first-guess grid due to collaboration, would lead to less work editing on their end.

It seemed the feedback this week once again pointed to the workload involved with creating an extended forecast in addition to the 4, 6, & 8, especially during the summer months.  One suggestion to help in this effort would be to focus on just the high impact areas for the extended TCF, and maybe keep the automated polygons in the rest of the CONUS area.  Some participant feedback suggested better tool development, to create a smoother auto-generation of the TCF areas that would improve the jagged polygons.

Thursday, August 17, 2017

A look at the Summer Experiment...through the lens

A look at some happenings in and around the testbed during the 2017 Summer Experiment!

Participants looking on during the daily debrief of the TCF desks

A group at the UK-Met/SFO desk checking out the different high-resolution models

Andrew Clausen from Southwest Airlines giving his brown bag seminar about the life of a flight dispatcher

Some discussion taking place at the TCF collaboration desk in the testbed

Discussion with the OPG crew over at the C&V desk before the AWT team sends over their forecast grids for the 18Z TAF package

Collaboration with the AWDE team via google chat at the C&V desk

Participants in the Operations Proving Ground next door played the role of the local WFO, editing the  national C&V grids sent over from AWT.

The TCF operations desk was held just off the AWC operations floor, to simulate the real-life separation between operations and collaboration.

Claire Bartholomew visiting from the UK-Met Office doing the daily debrief of the SFO-desk

Participants discussing the developing weather for the day impacting the forecast for the C&V grids

Participants look on at the SFO desk, discussing possible clearing times based on model guidance

Lots of great discussion about the national C&V grids took place at the GFE desk throughout the experiment

Because long days of forecasting and collaboration sometimes call for beers and fun...some of the experiment crew test their skills at trivia night at a local brewery

PERTI participates

Guests that participated in AWDE this week included National Operations Managers (NOMs) from the Air Traffic Control System Command Center (ATCSCC). These participants are part of the PERTI team, whose focus is on working to become more strategic and predictive in the planning and execution of daily traffic management while accounting for disruptive events.  This is done by analyzing traffic and constraints from the past building upon a data driven analysis to make the NAS safer and more efficient. Each letter represents one of the five key points of the concept – Plan, Execute, Review, Train and Improve.

From the NOM perspective, the group was interested in seeing as much information as possible while viewing the extended TCF. This includes low and high confidence polygons and echo tops for more efficient decision making to determine impacts to the NAS. One takeaway from the differences between this week and last week's feedback is that the needs for the extended TCF product is really dependent on the type of user utilizing it.  Users at the ATCSCC are using the information at a national level and are tasked with making decisions and implementing strategies that require more in-depth information than only knowing the high confidence convective areas.  The low confidence convective areas allow them to see potential areas of concern that may develop in the extended forecast.  In addition, larger polygons with a high confidence may cause more collaboration between the NAMs and local CWSUs to verify the weather and determine the traffic impact.
Participants from the PERTI team examining the extended TCF
Additional participants included a Helicopter Pilot from the United States Coast Guard. Positive feedback was given while the user examined the experimental TAFs, HEMS flight rule, and Probabilistic C&V.  The additional forecast times, in the TAFs, were helpful, even if there were no major changes in ceiling or visibility, because users were able to see possible trends in the weather.  The update rate of 15 minutes is adequate for the HEMS tool and the RU-RTMA display was preferred.  The same feedback for both the HEMS flight rule and Probabilistic C&V consisted of wanting to see C&V separate, needing a zoom capability, and options to display altitudes.

User giving feedback on the experimental TAFs