Wednesday, August 23, 2017

2017 Summer Experiment Week 2 Wrap-Up

The 2017 AWT Summer Experiment came to a close on Friday with the weekly wrap-up sessions for each of the desks.  One of the benefits of running the experiment over a two week period, is having a different set of participants each week with various backgrounds and experience.  This allows for new perspectives and valuable feedback from week two that may not have come up during week one.

During Friday's breakout sessions, participants were able to give feedback from their perspective on the TCF forecast creation process, the use of the UK-Met model to predict clearing times at SFO, and using the national grids of clouds and visibility at the GFE desk to give a first-guess for the digital aviation forecast.

The SFO desk had the addition of the 1.5km UK met model this week, which is what the 330m model is nested from.  This week also presented very unique synoptic set-ups over the different days, with both pre and post trough situations.  It was noted that post-trough SW flow can occasionally lead to convective strato-cumulus formation, but also help to dry out the cloud.  On Wednesday, the 330m high-res model picked up on initiation of convective cumulus off the peninsula, which no other model did.  On Thursday, the 1.5km model actually did a better job than the 330m model.  Further evaluation of the general weather picture would be useful in this case to understand why.

Clouds and Visibility / GFE
Once again the overwhelming feedback from the GFE C&V grids this week was about the detail and speckle nature of the high resolution model grids.  While detail is good, too much can be overwhelming on a national level.  The challenge going forward seems to be finding balance between when to leave detail in, when to smooth out, and which level will do what.  An idea was brought up in the final discussion about using image processing methods in the CloudBasePrimary grid to identify areas first, then build up a consistent cloud base from there.  This would allow the details to be smoothed more intelligently.

In this post-OPG processed cloud base primary grid, you can really see where the local WFOs smoothed out the entire area to what they determined to be a reasonable cloud base, in response to the speckle nature of the grid they were handed

Another point that came up again was the need to have some means of filling in the CloudBaseSecondary grid with a first guess or starting point.  This could be using a tool based on LCL or probability of thunder, or even building in a first guess from model data in a similar fashion to how CloudBasePrimary is initialized.  The other repeating feedback to come up was the look of the AWC-model created SKY grids.  They appear too binary, and difficult to pass onto a local level.  Where the national blend seems to look more realistic like a satellite image.  The question is, how realistic is this far out into the forecast time?

As far as the actual editing procedure of the girds and workload, it was noted that much more coordination would need to take place with the local WFOs.  This would be most useful in areas deemed high-impact for aviation that day.  While the local offices may already be busy collaborating with other offices and national centers if it is a significant weather event, hopefully handing them a better first-guess grid due to collaboration, would lead to less work editing on their end.

It seemed the feedback this week once again pointed to the workload involved with creating an extended forecast in addition to the 4, 6, & 8, especially during the summer months.  One suggestion to help in this effort would be to focus on just the high impact areas for the extended TCF, and maybe keep the automated polygons in the rest of the CONUS area.  Some participant feedback suggested better tool development, to create a smoother auto-generation of the TCF areas that would improve the jagged polygons.

Thursday, August 17, 2017

A look at the Summer Experiment...through the lens

A look at some happenings in and around the testbed during the 2017 Summer Experiment!

Participants looking on during the daily debrief of the TCF desks

A group at the UK-Met/SFO desk checking out the different high-resolution models

Andrew Clausen from Southwest Airlines giving his brown bag seminar about the life of a flight dispatcher

Some discussion taking place at the TCF collaboration desk in the testbed

Discussion with the OPG crew over at the C&V desk before the AWT team sends over their forecast grids for the 18Z TAF package

Collaboration with the AWDE team via google chat at the C&V desk

Participants in the Operations Proving Ground next door played the role of the local WFO, editing the  national C&V grids sent over from AWT.

The TCF operations desk was held just off the AWC operations floor, to simulate the real-life separation between operations and collaboration.

Claire Bartholomew visiting from the UK-Met Office doing the daily debrief of the SFO-desk

Participants discussing the developing weather for the day impacting the forecast for the C&V grids

Participants look on at the SFO desk, discussing possible clearing times based on model guidance

Lots of great discussion about the national C&V grids took place at the GFE desk throughout the experiment

Because long days of forecasting and collaboration sometimes call for beers and fun...some of the experiment crew test their skills at trivia night at a local brewery

PERTI participates

Guests that participated in AWDE this week included National Operations Managers (NOMs) from the Air Traffic Control System Command Center (ATCSCC). These participants are part of the PERTI team, whose focus is on working to become more strategic and predictive in the planning and execution of daily traffic management while accounting for disruptive events.  This is done by analyzing traffic and constraints from the past building upon a data driven analysis to make the NAS safer and more efficient. Each letter represents one of the five key points of the concept – Plan, Execute, Review, Train and Improve.

From the NOM perspective, the group was interested in seeing as much information as possible while viewing the extended TCF. This includes low and high confidence polygons and echo tops for more efficient decision making to determine impacts to the NAS. One takeaway from the differences between this week and last week's feedback is that the needs for the extended TCF product is really dependent on the type of user utilizing it.  Users at the ATCSCC are using the information at a national level and are tasked with making decisions and implementing strategies that require more in-depth information than only knowing the high confidence convective areas.  The low confidence convective areas allow them to see potential areas of concern that may develop in the extended forecast.  In addition, larger polygons with a high confidence may cause more collaboration between the NAMs and local CWSUs to verify the weather and determine the traffic impact.
Participants from the PERTI team examining the extended TCF
Additional participants included a Helicopter Pilot from the United States Coast Guard. Positive feedback was given while the user examined the experimental TAFs, HEMS flight rule, and Probabilistic C&V.  The additional forecast times, in the TAFs, were helpful, even if there were no major changes in ceiling or visibility, because users were able to see possible trends in the weather.  The update rate of 15 minutes is adequate for the HEMS tool and the RU-RTMA display was preferred.  The same feedback for both the HEMS flight rule and Probabilistic C&V consisted of wanting to see C&V separate, needing a zoom capability, and options to display altitudes.

User giving feedback on the experimental TAFs

UKMET high-res model peforms well for SFO on Day 3

Unlike much of this week, day 3 at the SFO desk was a more typical fog and low stratus day around the terminal. Interestingly, not all of the models picked up on that 'normal' behavior. Below is a four panel capturing the various model solutions in the morning:
20170816 1300 UTC run of HRRR (top left) and GLAMP (top right) Ceiling Height grid at the 1-hour forecast, the 00 UTC run of the high res UKMET model (bottom left) Sky Cover grid, the 14-hour forecast, and GOES-16 visible satellite imagery at 1402 UTC.
Neither the HRRR or the GLAMP added much value to yesterday's forecast for clearing. Skies were already beginning to clear to the Southeast of SFO as shown on the satellite imagery, but the GLAMP still indicated fog at that time. The HRRR did have some clearing but didn't provide a particularly accurate picture of the cloud features themselves. For this reason, most of the focus was on the UKMET model, which ended up doing exceptionally well for the day.

The first positive point to its scorecard was in the way it seemed to pick up on the local orographic effects of the stratus layer.
20170816 00 UTC UKMET run, the 15-hour forecast of Sky Cover (right) and GOES-16 visible satellite imagery at 1502 UTC (left)
During a normal fog setup around SFO, the marine layer will funnel in through the San Francisco Bay and spread out through the Bay area. Additionally, it will advance onshore along the coast, but generally stops on the windward (west) side of the terrain. This was the setup yesterday, and the UKMET did an excellent job of capturing this terrain affect as seen in the image above.

Later in the afternoon, around 1800 UTC, cumulus clouds began developing over the SFO area and revealed a second positive point for the UKMET model. Below is the UKMET Sky Cover forecast and visible satellite imagery.
20170816 00 UTC UKMET run, the 18-hour forecast of Sky Cover (left) and GOES-16 visible satellite imagery at 1802 UTC (right)
Thanks to its higher resolution, the UKMET model actually was actually indicating that local cumulus development. Aside from picking up on such small scale detail and cloud features, the UKMET utilized for the experiment is the 00 UTC run. At this particular time, it was an 18-hour forecast picking up on the cumulus development.

All in all, a very good day for the UKMET model!

Wednesday, August 16, 2017

Looking at C&V in the East on Day 2

The C&V desk turned their attention east for ceiling and visibility forecasting yesterday. The initial national blend looked like this:

Immediately obvious was the broad area of IFR and even LIFR conditions. Based on the observed ceilings at the time, this was clearly over done. The RAP seemed to have a better handle, eliminating a lot of those low ceilings. Those working on the national scale opted to go with the RAP, assuming that it would be a lot easier for the local WFOs to more accurately add areas of C&V back in at their level, rather than send them a grid with a mess of widespread IFR and LIFR (accuracy debatable) that would require a large amount of editing and adjusting.
20170815 1400 UTC National Blend (left) and RAP (right) ceiling and visibility forecast, 6-hour (20 UTC) forecast
With a decent model basis to begin with, participants moved on to editing the grids. As with previous days, the discussion of the national vs. local perspective arose. Those working on the national scale noted how challenging it was to load shed and edit the grids where there are aircraft concerns. On this particular day, there was a lot going on in the Northeast, which drew the attention there and away from other areas like Atlanta. So, a lot of time was spent on the grids sent to WFOs in the Northeast, making the workload at their local level fairly light; whereas, the lack of attention south resulted in more of a messy grid that required a lot more editing and smoothing.

There was also more discussion on more versus less detail in the cloud grids. Below is the National Blend Sky Cover grid.
National Blend Sky Cover grid from 1800 UTC on August 15th
Pretty picture? Yes. Realistic? Certainly. For the most part it looks very much like a clouds would. Is this useful? It depends on who you ask and what you're forecasting. While the detail is realistic, the point was brought up again that it is perhaps too much detail for the purposes of the TAFs. Users want to see the overall trend in the TAF and not the minor ups and downs in the gridded details.

In this case, one of the mock local offices was Philly, and participants at that local level, smoothed out the Sky Cover grid in that area for better TAF generation.

National Blend Sky Cover forecast zoomed in over Philly, the mock WFO forecasting at the local level
The difference between the initial grid and the grid edited at the local scale is very clear. Again, much of the local grid area over Philly was smoothed with the goal of better TAF generation. From this the question arose, where is the balance? Would users prefer the higher detail and resolution in the overall grids? Or should they be smoothed for better TAF performance? Is there a happy medium? These are all difficult questions that have not been answered, or may not even have a good answer yet. However, feedback from the Summer Experiment and continued debated will be crucial as the Digital Aviation Services (DAS) effort evolves and continues.

Another challenging day in SFO for Day 2

This week has certainly been interesting for the participants at the SFO desk and yesterday was no exception. Ceilings had never really cleared the previous day and SFO awoke again with the airport rather socked in. HRRR was discounted almost immediately as it was noted to be far too dry compared to current satellite conditions. For this reason, it was decided that it would not be used for the day's clearing forecast.
20170815 1300 UTC HRRR run, 1400 UTC hour forecast (left) with GOES-16 high-resolution visible imagery at 1402 UTC (right). Satellite showed more cloud (i.e. moisture) than was seen in the HRRR initialization
GLAMP solutions from earlier runs hinted at clearing by 20 UTC and cleared it out completely by 21 UTC. The UKMET on the other hand had ceilings clearing by 1800 UTC. This was quite a significant difference between the two models, making it a more interesting challenge to decide which to go with.

20170815 1400 UTC UKMET (left) and GLAMP (right) runs, 20 UTC forecast hour. This was the first hour in which the GLAMP showed any clearing in the SFO arrival area
Compared with GOES-16 satellite, the UKMET appeared to have initialized better, capturing the current stratus coverage. For this reason participants were more inclined to go with an earlier clearing time than noted by the GLAMP.

20170815 0000 UTC UKMET run (right) of cloud cover, the 16, 17, and 18 hour forecasts, with GOES-16 high-resolution visible imagery (left) for verification
GOES-16 satellite showed clearing by 1700 UTC. This was fairly close to the UKMET solution, which had clearing starting at 1700 UTC and completely cleared closer to 1800 UTC. The high-resolution model also seemed more accurate in the details of the clouds themselves, showing the main stratus layer moving offshore and the small inland area of fog over Oakland burning off.

Day 2 TCF discussion: medium coverage and broad sparse areas

Though a quieter day weather-wise, the lighter load in product issuance allowed for some interesting exploration of TCF and the long-range concept. The first was a discussion about including medium coverage in the long-range. This was one of the forecasts issued for 24 hours out:

Note in particular the medium coverage area near Minneapolis Center. There has been debate as to whether the long-range forecast should only be high confidence of sparse coverage given the inherent uncertainty in the models. However, in this case the forecaster stated that despite the uncertainty in location, judging by the synoptic scale pattern and agreement in the long-range models of several rounds of convection, confidence was high for medium coverage somewhere in the area.

From a user perspective, it was noted that the uncertainty in the location doesn't so much matter. Just the fact that the forecaster was confident in medium coverage, particularly over a busy terminal like Minni, would provide value. That confidence would indicated to planners that there would be an area to deal with the next day. Additionally, it differentiates the synoptic scale type convection scenario from the typically summertime airmass convection that often occurs in the Southeast U.S.

The second discussion point was related to the airmass convection. The forecaster generated a broader area of sparse coverage over the Southeast U.S., broader than is typically done because of the uncertainty. In this case, the users looked and immediately saw the many terminals covered within these area; Atlanta, Jacksonville, Birmingham, Hunstville, etc. It was noted that seeing so many terminals affected many alarm traffic flow managers, despite that it was likely realistic given the summertime airmass convection. They suggested that maybe instead of such a broad area, a series of smaller area. This would provide value by indicating that, yes, it would be a summertime airmass convective setup, and prepare traffic flow managers and others for that scenario.