Advancing Verification Across Atmosphere, Sea Ice, and Coupled Processes

The PCAPS Verification Task Team (TT) plays a central role in delivering the “Predict–Fidelity” component of PCAPS. Its mission is to assess the accuracy, reliability, and physical realism of polar prediction systems, to guide model improvement, as well as to assess polar prediction quality and value, for informed use. With experts from ECCC, NOAA, ECMWF, the UK Met Office, Met Norway, the British Antarctic Survey, and academic partners, the TT brings together atmospheric, sea ice, and process-based verification expertise.

The difference in 2-m temperature between the coupled and uncoupled Canadian Arctic Prediction System (CAPS) reveals the warming effects of the ocean-sea ice-atmosphere coupling, particularly prominent at the coastal verifying stations. Figure courtesy of Barbara Casati.

Since the Verification TT’s launch in June 2024, members have met quarterly to exchange verification results for both atmospheric and sea ice variables. These structured discussions provide an informal but highly productive forum for sharing diagnostics, identifying systematic model behaviours, and comparing verification practices across operational centres.

The atmospheric verification stream focuses on upper-air and near-surface variables, exploring time series, anomaly patterns, and lead-time dependent behaviours across global and regional models. Examples from recent sessions include comparisons of z500 errors over the Arctic and Antarctic, surface temperature biases in Canadian systems (CAPS, GDPS), and assessments of humidity, visibility, clouds, wind, and precipitation. Novel stratifications and representativeness diagnostics — particularly for heterogeneous polar surfaces including water, ice, land, glaciers, snow — are being developed to improve standard verification procedures.

The sea ice verification stream aims to consolidate a common set of metrics and verification practices, with the final goal of contributing a standardized protocol for future WIPPS score exchange. The evaluation of sea ice concentration and ice edge includes the application of IIEE and SPS metrics, with ongoing research to unravel effects of threshold choices, and sensitivities to different observational datasets. Recent contributions highlighted seasonal sea ice prediction skill and its dependence on the initialization month. The SIDFEX project has been instrumental during PPP for defining standard metrics for drift verification using IABP buoys, which are now being expanded with novel circular metrics.

Complementing these activities, the TT collaborates closely with the YOPPsiteMIP initiative to advance process-based verification. High-frequency model output at polar supersites enables detailed multivariate diagnostics, which expand on the univariate analysis of traditional atmospheric variables to include fluxes, cloud microphysics, radiation budget, and validation of variable interdependencies. New diagnostic approaches — including bivariate distances and relationship-based metrics — are being explored to assess coupled model performance and identify key process deficiencies.

Strong Leadership Driving Progress in Polar Forecast Verification

The PCAPS Verification TT benefits from the leadership of Dr. Barbara Casati, co-chair of the WWRP Joint Working Group on Forecast Verification Research (JWGFVR) and Dr. Marion Mittermaier, a former co-chair of the same working group. Barbara is a research scientist at Environment and Climate Change Canada (ECCC) and Marion works at the Met Office in the United Kingdom (UK).

Barbara brings established expertise in verification methodologies and a collaborative and energetic vision that has been instrumental in shaping the TT. In addition, Barbara led the evaluation effort of the preceding WWRP Polar Prediction Project (PPP).

Marion brings a strong background in observations and model evaluation methodology into a region which is new to her. Marion is excited to bring her knowledge of verification as well as her skills in leading diverse and dispersed teams to help jointly steer this work.

Co-leads of the PCAPS Verification Task Team: Drs. Barbara Casati (left) and Marion Mittermaier (right). Photos courtesy of Barbara Casati and Marion Mittermaier, respectively.

Looking Ahead: New Directions and Cross-Team Collaboration

A full year of coordinated quarterly exchanges has already yielded substantial methodological progress, improved data usage, and community alignment. Moreover, this exercise enabled the TT to identify key gaps and emerging opportunities. The TT is now expanding to include additional Southern Hemisphere expertise and strengthen links with other PCAPS Task Teams. Upcoming research priorities include:

• Advancing verification of surface weather including clouds, visibility, and wind, integrating satellite and in-situ observations; mixed-phase and low clouds, two known challenging features in polar prediction.

• Defining a flagship activity on visibility and wind verification, to meet user-needs from the transport sector; this work, brought forward in collaboration with SERA, aims to bridge the gap between prediction and end-users, for one of the key sectors in Polar Regions.

• Exploiting observational campaigns aboard vessels and aircraft, in partnership with the Processes TT; co-located multivariate measurements are rare in Polar Regions, by analyzing physical relationships, this helps advance polar processes understanding and their model representation. This work complements the land-based supersite measurements, expanding to ocean and air-borne measurements.

• Aligning sea ice verification standards with Ocean-Predict frameworks, while contributing to a shared sea ice verification code repository. This work builds on the PPP outcomes and promotes collaboration across multidisciplinary WMO teams.

• Integrating satellite ice thickness products and sea ice thickness verification. Sea ice thickness is a challenging variable yet to be assimilated in most prediction systems. This work tackles this challenge in concert with the Data Assimilation Task Team.

Finally, the TT is eager to contribute a regional study focus on Polar Regions, within the ESMO Weather Prediction Model Intercomparison Project (WP-MIP), to compare the performance of Machine Learning, Hybrid and Numerical Weather Prediction systems.

Want to get involved? This is an open and collaborative team, please get in contact!