.png){kind=icon}
Return
Vector representation based on a supervised codebook for Nepali documents classification
.png){kind=icon}
OA
AI
DOI：10.7717/peerj-cs.412.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Document representation with outlier tokens exacerbates the classification performance due to the uncertain orientation of such tokens. Most existing document representation methods in different languages including Nepali mostly ignore the strategies to filter them out from documents before learning their representations. In this article, we propose a novel document representation method based on a supervised codebook to represent the Nepali documents, where our codebook contains only semantic tokens without outliers. Our codebook is domain-specific as it is based on tokens in a given corpus that have higher similarities with the class labels in the corpus. Our method adopts a simple yet prominent representation method for each word, called probability-based word embedding. To show the efficacy of our method, we evaluate its performance in the document classification task using Support Vector Machine and validate against widely used document representation methods such as Bag of Words, Latent Dirichlet allocation, Long Short-Term Memory, Word2Vec, Bidirectional Encoder Representations from Transformers and so on, using four Nepali text datasets (we denote them shortly as A1, A2, A3 and A4). The experimental results show that our method produces state-of-the-art classification performance (77.46% accuracy on A1, 67.53% accuracy on A2, 80.54% accuracy on A3 and 89.58% accuracy on A4) compared to the widely used existing document representation methods. It yields the best classification accuracy on three datasets (A1, A2 and A3) and a comparable accuracy on the fourth dataset (A4). Furthermore, we introduce the largest Nepali document dataset (A4), called NepaliLinguistic dataset, to the linguistic community.
Keywords:
Text classification
Machine learning
Codebook
Nepali documents
Classification
Feature extraction
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
2.5
Papers:
3.4K
Citations:
6.9K
Organization
Cited Papers
ShareSaveDeep neural network to detect COVID-19: one architecture for both CT Scans and Chest X-rays.png)
APPLIED INTELLIGENCEOAAI
APPLIED INTELLIGENCEIF3.5
OAAIShareSaveThe Role of Maternal Prepregnancy Body Mass Index in Breastfeeding Outcomes: A Systematic Review
PREAI
ShareSaveAdvanced Deep Learning-Based Computational Offloading for Multilevel Vehicular Edge-Cloud Computing NetworksIEEE ACCESSOAAI
IEEE ACCESSIF3.6
OAAIShareSaveHydroxyurea does not reverse functional alterations of the nitric oxide-cGMP pathway associated with priapism phenotype in corpus cavernosum from sickle cell mousePLOS ONE
OAAI
PLOS ONEIF0
OAAIShareSaveShareSave
View more