.png){kind=icon}
Return
Preprocessing framework for scholarly big data management
.png){kind=icon}
PRE
AI
DOI：10.1007/s11042-022-13513-8.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Big data technologies have found applications in disparate domains. One of the largest sources of textual big data is scientific documents and papers. Scholarly big data has been used in numerous ways to develop innovative applications such as collaborator discovery, expert finding and research management systems. With the evolution of machine and deep learning techniques, the efficacy of such applications has risen manifold. However, the biggest challenge in the development of deep learning models for scholarly applications in cloud-based environment is the under-utilization of resources because of the excessive time required for textual preprocessing. This paper presents a preprocessing pipeline that uses Spark for data ingestion and Spark ML for performing preprocessing tasks. The proposed approach is evaluated with the help of a case study, which uses LSTM-based text summarization to generate title or summaries from abstracts of scholarly articles. Results indicate a substantial reduction in ingestion, preprocessing and cumulative time for the proposed approach, which shall manifest reduction in development time and costs as well.
Keywords:
Deep learning applications
Preprocessing pipeline
Scholarly big data
Scholarly data applications
Spark ML
Journal
IF:
3
Papers:
2.0W
Citations:
3.2W
Organization
No organization information available
Cited Papers
ShareSaveShareSaveShareSaveEnhancement of pyramid solar distiller performance using reflectors, cooling cycle, and dangled cords of wicks.png)
Desalination
DesalinationIF0
PREAI
ShareSaveMyocardial gene delivery using molecular cardiac surgery with recombinant adeno-associated virus vectors in vivoGene Therapy
OAAI
Gene TherapyIF0
OAAIShareSave
Neuraxis dissemination in pediatric brain tumors. Response to preirradiation chemotherapyCancerOAAI
CancerIF0
OAAIShareSave
View more