Statistical regularization for trend detection: An integrated approach for detecting long-term trends from sparse tropospheric ozone profiles
Journal Article
Abstract. Detecting a tropospheric ozone trend from sparsely sampled ozonesonde profiles (typically once per week) is challenging due to the noise in the time series resulting from ozone's high temporal variability. To enhance trend detection we have developed a sophisticated statistical approach that utilizes a geoadditive model to assess ozone variability across a time series of vertical profiles. Treating the profile time series as a set of individual time series on discrete pressure surfaces, a class of smoothing spline ANOVA (analysis of variance) models is used for the purpose of jointly modeling multiple correlated time series (on separate pressure surfaces) by their associated seasonal and interannual variabilities. This integrated fit method filters out the unstructured noise through a statistical regularization (i.e. a roughness penalty), by taking advantage of the additional correlated data points available on the pressure surfaces above and below the surface of interest. We have applied this technique to the trend analysis of the vertically correlated time series of tropospheric ozone observations from 1) IAGOS (In-service Aircraft for a Global Observing System) commercial aircraft profiles above Europe and China, and 2) NOAA GMD's (Global Monitoring Division) ozonesonde records at Hilo, Hawaii and Trinidad Head, California. We illustrate the ability of this technique to detect a consistent trend estimate, and its effectiveness for reducing the associated uncertainty in the noisy profile data due to low sampling frequency. We also conducted a sensitivity analysis of frequent IAGOS profiles above Europe (approximately 120 profiles per month) to determine how many profiles in a month are required for reliable long-term trend detection. When ignoring the vertical correlation we found that a typical sampling strategy of 4 profiles-per-month results in 7 % of sampled trends falling outside the 2-sigma uncertainty interval derived from the full data set, with associated 10 % of mean absolute percentage error. We determined that an optimal sampling frequency is 14 profiles per month when using the integrated fit method for calculating trends; when the integrated fit method is not applied, the sampling frequency had to be increased to 18 profiles per month to achieve the same result. While our method improves trend detection from sparse data sets, the key to substantially reducing the uncertainty is to increase the sampling frequency.;
