arrow
返回

Nonparametric Distributed Learning Architecture for Big Data: Algorithm and Applications

delete2019-06-01
delete5
delete
OA
AI
S
Scott A. Bruce
Z
Zeda Li
H
Hsiang-Chieh Yang
S
Subhadeep Mukhopadhyay *
DOI:10.1109/TBDATA.2018.2810187delete
delete原文链接
delete分享
delete收藏
查看原文
摘要

摘要

En 中文
Dramatic increases in the size and complexity of modern datasets have made traditional centralized statistical inference prohibitive. In addition to computational challenges associated with big data learning, the presence of numerous data types (e.g., discrete, continuous, categorical, etc.) makes automation and scalability difficult. A question of immediate concern is how to design a data-intensive statistical inference architecture without changing the basic statistical modeling principles developed for small data over the last century. To address this problem, we present MetaLP, a flexible, distributed statistical modeling framework suitable for large-scale data analysis, where statistical inference meets big data computing. This framework consists of three key components that work together to provide a holistic solution for big data learning: (i) partitioning massive data into smaller datasets for parallel processing and efficient computation, (ii) modern nonparametric learning based on a specially designed, orthonormal data transformation leading to mixed data algorithms, and finally (iii) combining heterogeneous local inferences from partitioned data using meta-analysis techniques to arrive at the global inference for the original big data. We present an application of this general theory in the context of a nonparametric two-sample inference algorithm for Expedia personalized hotel recommendations based on 10 million search result records.
Keyword:
Nonparametric mixed data modeling
LP transformation
distributed statistical learning
heterogeneity
meta-analysis
data-parallelism
AI总结

AI总结

对已上传原文的论文进行重点信息的提取,主要内容包括:简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。

期刊

I
IEEE Transactions on Big Data
IF:
5.7
论文数:
918
被引数:
4.0K

机构

P
pennsylvania commonwealth system of higher education (pcshe)
学者数:
0
论文数: 1.7K
被引数: 8.3W
引用论文

引用论文

Spin-Crossover Complexes
err2018-03-01
err0
errOAAI
errKazuyuki Takahashi
err分享
err收藏
err分享
err收藏
A scalable bootstrap for massive data
err2014-03-17
err286
errOAAI
errKleiner, Ariel; Talwalkar, Ameet; Sarkar, Purnamrita; Jordan, Michael I.
err分享
err收藏
err分享
err收藏
学者 查看更多内容