Skip to main content
placeholder image

Using grouping strategy and pattern discovery for delta extraction in a limited collaborative environment

Journal Article


Abstract


  • This work considers extracting delta in a distributed environment where the collaboration from highly autonomous operational database management systems is limited to granting read only access on a set of selected relational tables. Because of inherently huge volume of data in data warehouse system, it is critical to minimise communication costs as much as possible. Based on the observation that usually, two consecutive snapshots are not very different, a statistical-based group hash method is developed to minimise the volumes of data required to complete the data extraction. In addition, to relax the assumption that the changes to remote data are only caused by random events, we define a progression pattern to describe data changes with temporal regularities and also propose a method for progression pattern discovery.

Publication Date


  • 2015

Citation


  • Lu, Z., Yan, J. & Wang, X. (2015). Using grouping strategy and pattern discovery for delta extraction in a limited collaborative environment. International Journal of Business Intelligence and Data Mining, 10 (4), 378-405.

Scopus Eid


  • 2-s2.0-84943553508

Ro Metadata Url


  • http://ro.uow.edu.au/eispapers/5046

Has Global Citation Frequency


Number Of Pages


  • 27

Start Page


  • 378

End Page


  • 405

Volume


  • 10

Issue


  • 4

Place Of Publication


  • United Kingdom

Abstract


  • This work considers extracting delta in a distributed environment where the collaboration from highly autonomous operational database management systems is limited to granting read only access on a set of selected relational tables. Because of inherently huge volume of data in data warehouse system, it is critical to minimise communication costs as much as possible. Based on the observation that usually, two consecutive snapshots are not very different, a statistical-based group hash method is developed to minimise the volumes of data required to complete the data extraction. In addition, to relax the assumption that the changes to remote data are only caused by random events, we define a progression pattern to describe data changes with temporal regularities and also propose a method for progression pattern discovery.

Publication Date


  • 2015

Citation


  • Lu, Z., Yan, J. & Wang, X. (2015). Using grouping strategy and pattern discovery for delta extraction in a limited collaborative environment. International Journal of Business Intelligence and Data Mining, 10 (4), 378-405.

Scopus Eid


  • 2-s2.0-84943553508

Ro Metadata Url


  • http://ro.uow.edu.au/eispapers/5046

Has Global Citation Frequency


Number Of Pages


  • 27

Start Page


  • 378

End Page


  • 405

Volume


  • 10

Issue


  • 4

Place Of Publication


  • United Kingdom