arrow
Return

Deep Learning-Based Quality Control Using Subcellular RNA Spatial Distribution Patterns for Cell Segmentation in Spatial Transcriptomics Data

delete2025-11-01
delete0
PRE
AI
R
Renpeng Ding
K
Kerem Celikay
M
Ming Ni
Y
Yong Hou
Y
Yan Zhou *
K
Karl Rohr
DOI:10.1002/smtd.202500885delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
Sequencing-based spatial transcriptomics (sST) techniques with high resolution enable transcriptome-wide RNA capture at subcellular resolution. Although new cell segmentation methods for sST data are continually being developed, accurately assigning RNA spots to corresponding cells still presents significant challenges and there is a lack of quality control methods. This work introduces a deep learning method for quality control of cell segmentation and improvement of the segmentation result. The proposed method exploits the subcellular spatial distribution patterns of different types of RNA by a deep neural network to assess the quality of segmented cells. The method identifies partially segmented cells typically due to low RNA capture or strong RNA diffusion as well as merged cells due to high cell density. In addition, the quality control method is combined with a Transformer-based cell segmentation method and it is shown that the cell segmentation performance improves by automatically removing low-quality segmented cells from the training dataset. The method is applied to both synthetic data and real Stereo-seq data, demonstrating its potential for quality control and enhancement of cell segmentation in sST data.
Keywords:
cell segmentation
quality control
subcellular RNA spatial distribution
spatial transcriptomics

Journal

Small Methods cover
Small Methods
IF:
9.1
Papers:
4.3K
Citations:
2.2W

Organization

R
Ruprecht Karls University Heidelberg
Scholars:
5.6W
Papers: 4.3W
Citations: 66
B
beijing genomics institute (bgi)
Scholars:
5.2K
Papers: 2.1K
Citations: 7
Cited Papers

Cited Papers

Central dogma at the single-molecule level in living cells
err2011-07-20
err0
errOAAI
errGene-Wei Li; X. Sunney Xie
errShare
errSave
Museum of spatial transcriptomics
err2022-03-10
err419
errOAAI
errMoses, Lambda; Pachter, Lior
errShare
errSave
Learning Scientific Programming with Python
err
IF0
err2020-10-22
err0
PREAI
errChristian Hill
errShare
errSave
Single-cell spatial transcriptome reveals cell-type organization in the macaque cortex
errCell
IF0
err2023-08-01
err0
errOAAI
errAo Chen; Yidi Sun; Ying Lei; Chao Li; Sha Liao; Juan Meng; Yiqin Bai; Zhen Liu; Zhifeng Liang; Zhiyong Zhu; Nini Yuan; Hao Yang; Zihan Wu; Feng Lin; Kexin Wang; Mei Li; Shuzhen Zhang; Meisong Yang; Tianyi Fei; Zhenkun Zhuang; Yiming Huang; Yong Zhang; Yuanfang Xu; Luman Cui; Ruiyi Zhang; Lei Han; Xing Sun; Bichao Chen; Wenjiao Li; Baoqian Huangfu; Kailong Ma; Jianyun Ma; Zhao Li; Yikun Lin; He Wang; Yanqing Zhong; Huifang Zhang; Qian Yu; Yaqian Wang; Xing Liu; Jian Peng; Chuanyu Liu; Wei Chen; Wentao Pan; Yingjie An; Shihui Xia; Yanbing Lu; Mingli Wang; Xinxiang Song; Shuai Liu; Zhifeng Wang; Chun Gong; Xin Huang; Yue Yuan; Yun Zhao; Qinwen Chai; Xing Tan; Jianfeng Liu; Mingyuan Zheng; Shengkang Li; Yaling Huang; Yan Hong; Zirui Huang; Min Li; Mengmeng Jin; Yan Li; Hui Zhang; Suhong Sun; Li Gao; Yinqi Bai; Mengnan Cheng; Guohai Hu; Shiping Liu; Bo Wang; Bin Xiang; Shuting Li; Huanhuan Li; Mengni Chen; Shiwen Wang; Minglong Li; Weibin Liu; Xin Liu; Qian Zhao; Michael Lisby; Jing Wang; Jiao Fang; Yun Lin; Qing Xie; Zhen Liu; Jie He; Huatai Xu; Wei Huang; Jan Mulder; Huanming Yang; Yangang Sun; Mathias Uhlen; Muming Poo; Jian Wang; Jianhua Yao; Wu Wei; Yuxiang Li; Zhiming Shen; Longqi Liu; Zhiyong Liu; Xun Xu; Chengyu Li
errShare
errSave
Slide-seq: A scalable technology for measuring genome-wide expression at high spatial resolution
err2019-03-29
err0
errOAAI
errSamuel G. Rodriques; Robert R. Stickels; Aleksandrina Goeva; Carly A. Martin; Evan Murray; Charles R. Vanderburg; Joshua Welch; Linlin M. Chen; Fei Chen; Evan Z. Macosko
errShare
errSave
Spatiotemporal transcriptomic atlas of mouse organogenesis using DNA nanoball-patterned arrays
errCell
IF0
err2022-05-01
err0
errOAAI
errAo Chen; Sha Liao; Mengnan Cheng; Kailong Ma; Liang Wu; Yiwei Lai; Xiaojie Qiu; Jin Yang; Jiangshan Xu; Shijie Hao; Xin Wang; Huifang Lu; Xi Chen; Xing Liu; Xin Huang; Zhao Li; Yan Hong; Yujia Jiang; Jian Peng; Shuai Liu; Mengzhe Shen; Chuanyu Liu; Quanshui Li; Yue Yuan; Xiaoyu Wei; Huiwen Zheng; Weimin Feng; Zhifeng Wang; Yang Liu; Zhaohui Wang; Yunzhi Yang; Haitao Xiang; Lei Han; Baoming Qin; Pengcheng Guo; Guangyao Lai; Pura Muñoz-Cánoves; Patrick H. Maxwell; Jean Paul Thiery; Qing-Feng Wu; Fuxiang Zhao; Bichao Chen; Mei Li; Xi Dai; Shuai Wang; Haoyan Kuang; Junhou Hui; Liqun Wang; Ji-Feng Fei; Ou Wang; Xiaofeng Wei; Haorong Lu; Bo Wang; Shiping Liu; Ying Gu; Ming Ni; Wenwei Zhang; Feng Mu; Ye Yin; Huanming Yang; Michael Lisby; Richard J. Cornall; Jan Mulder; Mathias Uhlén; Miguel A. Esteban; Yuxiang Li; Longqi Liu; Xun Xu; Jian Wang
errShare
errSave
SCS: cell segmentation for high-resolution spatial transcriptomics
err2023-07-10
err15
errOAAI
errChen, Hao; Li, Dongshunyi; Bar-Joseph, Ziv
errShare
errSave
Cell Biology by the Numbers
err
IF0
err2015-12-07
err0
PREAI
errRon Milo; Rob Phillips
errShare
errSave
researcher View more