Skip to main content

Fixed rank filtering for spatio-temporal data

Journal Article


Download full-text (Open Access)

Abstract


  • Datasets from remote-sensing platforms and sensor networks are often spatial, temporal, and very large. Processing massive amounts of data to provide current estimates of the (hidden) state from current and past data is challenging, even for the Kalman filter. A large number of spatial locations observed through time can quickly lead to an overwhelmingly high-dimensional statistical model. Dimension reduction without sacrificing complexity is our goal in this article. We demonstrate how a Spatio-Temporal Random Effects (STRE) component of a statistical model reduces the problem to one of fixed dimension with a very fast statistical solution, a methodology we call Fixed Rank Filtering (FRF). This is compared in a simulation experiment to successive, spatialonly predictions based on an analogous Spatial Random Effects (SRE) model, and the value of incorporating temporal dependence is quantified. A remote-sensing dataset of aerosol optical depth (AOD), from the Multi-angle Imaging SpectroRadiometer (MISR) instrument on the Terra satellite, is used to compare spatio-temporal FRF with spatialonly prediction. FRF achieves rapid production of optimally filtered AOD predictions, along with their prediction standard errors. In our case, over 100,000 spatio-temporal data were processed: Parameter estimation took 64.4 seconds and optimal predictions and their standard errors took 77.3 seconds to compute. Supplemental materials giving complete details on the design and analysis of a simulation experiment, the simulation code, and the MISR data used are available on-line. © 2010 American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North America.

Authors


  •   Cressie, Noel A.
  •   Shi, Tao (external author)
  •   Kang, E (external author)

Publication Date


  • 2010

Citation


  • Cressie, N. A., Shi, T. & Kang, E. L. (2010). Fixed rank filtering for spatio-temporal data. Journal of Computational and Graphical Statistics, 19 (3), 724-745.

Scopus Eid


  • 2-s2.0-77955117966

Ro Full-text Url


  • http://ro.uow.edu.au/cgi/viewcontent.cgi?article=9613&context=infopapers

Ro Metadata Url


  • http://ro.uow.edu.au/infopapers/2277

Number Of Pages


  • 21

Start Page


  • 724

End Page


  • 745

Volume


  • 19

Issue


  • 3

Abstract


  • Datasets from remote-sensing platforms and sensor networks are often spatial, temporal, and very large. Processing massive amounts of data to provide current estimates of the (hidden) state from current and past data is challenging, even for the Kalman filter. A large number of spatial locations observed through time can quickly lead to an overwhelmingly high-dimensional statistical model. Dimension reduction without sacrificing complexity is our goal in this article. We demonstrate how a Spatio-Temporal Random Effects (STRE) component of a statistical model reduces the problem to one of fixed dimension with a very fast statistical solution, a methodology we call Fixed Rank Filtering (FRF). This is compared in a simulation experiment to successive, spatialonly predictions based on an analogous Spatial Random Effects (SRE) model, and the value of incorporating temporal dependence is quantified. A remote-sensing dataset of aerosol optical depth (AOD), from the Multi-angle Imaging SpectroRadiometer (MISR) instrument on the Terra satellite, is used to compare spatio-temporal FRF with spatialonly prediction. FRF achieves rapid production of optimally filtered AOD predictions, along with their prediction standard errors. In our case, over 100,000 spatio-temporal data were processed: Parameter estimation took 64.4 seconds and optimal predictions and their standard errors took 77.3 seconds to compute. Supplemental materials giving complete details on the design and analysis of a simulation experiment, the simulation code, and the MISR data used are available on-line. © 2010 American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North America.

Authors


  •   Cressie, Noel A.
  •   Shi, Tao (external author)
  •   Kang, E (external author)

Publication Date


  • 2010

Citation


  • Cressie, N. A., Shi, T. & Kang, E. L. (2010). Fixed rank filtering for spatio-temporal data. Journal of Computational and Graphical Statistics, 19 (3), 724-745.

Scopus Eid


  • 2-s2.0-77955117966

Ro Full-text Url


  • http://ro.uow.edu.au/cgi/viewcontent.cgi?article=9613&context=infopapers

Ro Metadata Url


  • http://ro.uow.edu.au/infopapers/2277

Number Of Pages


  • 21

Start Page


  • 724

End Page


  • 745

Volume


  • 19

Issue


  • 3