arrow
Return

Cohort query processing without misleading aging effects

delete2025-10-22
delete0
PRE
AI
P
Pengkai Lu
Z
Zhongle Xie *
D
Dawei Jiang
陈
陈珂 (Ke Chen)
DOI:10.1007/s00778-025-00948-2delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
Cohort analysis is a powerful method widely employed in fields like medicine and sociology studying how groups of individuals behave over time under different initial conditions. Business data analysts seek to apply this method to massive Internet user activity datasets, aiming to uncover the impact of user attributes and application changes on behavior. However, cohort analysis queries are difficult to specify and costly to execute in traditional database systems. Moreover, the variability in how users allocate time both within and outside Internet applications introduces inconsistencies in user experiences, even among those with identical usage durations. This disparity leads to individual aging bias, distorting the estimation of aging effects on user behavior. To address these challenges, we propose extending database systems to facilitate cohort analysis while mitigating individual aging bias. Central to our approach is a general cohort analysis schema that integrates “constraint-chain-based age". This novel concept effectively eliminates aging bias by aligning the age of user activities with their unique application experiences. Building on this schema, we develop two distinct evaluation implementations for cohort query processing. The first, an intrusive implementation, extends SQL with custom operators that natively support constraint-chain-based age calculation. This implementation leverages a columnar evaluation framework optimized for cohort queries, incorporating advanced techniques like redundant sub-operator coalescence to enhance query plan optimization. The second, a non-intrusive implementation, is designed for rapid deployment on existing relational databases (row-store or column-store) using standard SQL, providing a baseline for performance and usability comparison. Extensive experiments validate the effectiveness of our approach, demonstrating that the intrusive implementation achieves substantial performance gains over the non-intrusive alternative. These results highlight the advantages of embedding native cohort query support into database systems.
Keywords:
Cohort query
Behavior analysis
Bias
Database

Journal

T
The VLDB Journal
IF:
0
Papers:
36
Citations:
0

Organization

S
School of Software Technology
Scholars:
45
Papers: 20
Citations: 0
Cited Papers

Cited Papers

Effective temporal dependence discovery in time series data
err2018-04-01
err0
PREAI
errQingchao Cai; Zhongle Xie; Meihui Zhang; Gang Chen; H. V. Jagadish; Beng Chin Ooi
errShare
errSave
Reliability at Multiple Stages in a Data Analysis Pipeline
err2022-10-20
err2
PREAI
errMoskovitch, Yuval; Jagadish, H. V.
errShare
errSave
Optimizing database architecture for the new bottleneck: memory access
err2000-12-01
err0
PREAI
errStefan Manegold; Peter A. Boncz; Martin L. Kersten
errShare
errSave
On detecting cherry-picked generalizations
err2022-01-14
err0
PREAI
errYin Lin; Brit Youngmann; Yuval Moskovitch; H. V. Jagadish; Tova Milo
errShare
errSave
Processing a trillion cells per mouse click
err2012-07-01
err0
errOAAI
errAlexander Hall; Olaf Bachmann; Robert Büssow; Silviu Gănceanu; Marc Nunkesser
errShare
errSave
The implementation and performance of compressed databases
err2000-09-01
err0
errOAAI
errTill Westmann; Donald Kossmann; Sven Helmer; Guido Moerkotte
errShare
errSave
Cohort query processing
err2016-09-01
err0
errOAAI
errDawei Jiang; Qingchao Cai; Gang Chen; H. V. Jagadish; Beng Chin Ooi; Kian-Lee Tan; Anthony K. H. Tung
errShare
errSave
Cohort analytics: efficiency and applicability
err
IF0
err2020-08-27
err0
PREAI
errBehrooz Omidvar-Tehrani; Sihem Amer-Yahia; Laks V. S. Lakshmanan
errShare
errSave
no more