.png){kind=icon}
返回
Dynamic Replication Policy on HDFS Based on Machine Learning Clustering
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2023.3247190.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Data growth in recent years has been swift, leading to the emergence of big data science. Distributed File Systems (DFS) are commonly used to handle big data, like Google File System (GFS), Hadoop Distributed File System (HDFS), and others. The DFS should provide the availability of data and reliability of the system in case of failure. The DFS replicates the files in different locations to provide availability and reliability. These replications consume storage space and other resources. The importance of these files differs depending on how frequently they are used in the system. So some of these files do not deserve to replicate many times because it is unimportant in the system. This paper introduces a Dynamic Replication Policy using Machine Learning Clustering (DRPMLC) on HDFS, which uses Machine Learning to cluster the files into different groups and apply other replication policies to each group to reduce the storage consumption, improve the read and write operations time and keep the availability and reliability of HDFS as a High-Performance Distributed Computing (HPDC).
Keyword:
Feature extraction
File systems
Machine learning
Databases
Big Data
Distributed computing
Support vector machines
Replicability
Availability
big data
clustering
Hadoop distributed file system
high-performance distributed computing
machine learning
reliability
replication policy
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
.png){kind=icon}
查看更多内容