Optimizing a transformer-based network for a deep-learning seismic processing workflow

doi:10.1190/GEO2023-0403.1

Optimizing a transformer-based network for a deep-learning seismic processing workflow

2024-06-20

2

OA

AI

DOI：10.1190/GEO2023-0403.1

Original

Original request for help

Save

Abstract

En 中文

StorSeismic is a recently introduced model based on the transformer network to adapt to various seismic processing tasks through its pretraining and fine-tuning strategy. In the original implementation, StorSeismic uses a sinusoidal positional encoding (PE) and a conventional self-attention mechanism, borrowed from natural language processing applications. For seismic processing, they provide good results but also indicate limitations in efficiency and expressiveness. We develop modifications to these two key components, by using relative PE and low-rank attention matrices as replacements for the standard ones. Our changes are tested on processing tasks applied to realistic Marmousi and offshore field data as a sequential strategy, starting from denoising, direct-arrival removal, multiple attenuation, and mal moveout correction. We observe faster pretraining and competitive results on the fine-tuning tasks and, in addition, fewer parameters to train compared with the standard model.

Optimizing a transformer-based network for a deep-learning seismic processing workflow

Abstract

Journal

Organization