Automatic Railroad Track Components Inspection Using Hybrid Deep Learning Framework

doi:10.1109/TIM.2023.3265636

Automatic Railroad Track Components Inspection Using Hybrid Deep Learning Framework

2023-01-01

27

PRE

AI

DOI：10.1109/TIM.2023.3265636

Original

Original request for help

Save

Abstract

En 中文

Regular inspections on track components, such as the clip, spike, and rail, are essential to maintain track quality and ensure railroad operating safety. Unfortunately, traditional image processing (IP)-based systems have limited accuracy. Existing convolutional neural network (CNN)-based approaches are designed for either detection or saliency segmentation of a specific track component (e.g., fastener or rail only). The overall track condition could not be evaluated because not all the track components are inspected simultaneously. This article presents an all-in-one YOLO (AOYOLO) framework for multitask track component inspection. First, a newly developed ConvNeXt-based backbone is constructed in AOYOLO to produce suitable hyperfeatures for both detection and segmentation tasks. Second, a novel U-shaped salient object segmentation branch is incorporated into AOYOLO to supplement the object detection branch, improving both the rail surface defect (RSD) segmentation and the detection of other components. Advanced data augmentations are integrated to further enhance the accuracy and scalability of the network. Extensive experiments conducted on a track dataset established with images taken by drone indicate that the proposed system is able to: 1) achieve 95.6% mean average precision (mAP) for track components inspection at a real-time speed of 147 frames/s and 2) reach 93.6% accuracy on RSDs detection, which surpass the current state-of-the-art (SOTA) models. The exceptional inference speed and superior detection accuracy have great potential for field applications.

Keywords:

Inspection

Rails

Image segmentation

Fasteners

Safety

IP networks

Feature extraction

Convolutional neural network (CNN)

object detection

railroad safety

saliency segmentation

track component inspection

unmanned aerial vehicle (UAV) images

Automatic Railroad Track Components Inspection Using Hybrid Deep Learning Framework

Abstract

Journal

Organization