A Triple-Double Convolutional Neural Network for Panchromatic Sharpening - Apprentissage de modèles visuels à partir de données massives Access content directly
Journal Articles IEEE Transactions on Neural Networks and Learning Systems Year : 2023

A Triple-Double Convolutional Neural Network for Panchromatic Sharpening

Abstract

Pansharpening refers to the fusion of a panchromatic (PAN) image with a high spatial resolution and a multispectral (MS) image with a low spatial resolution, aiming to obtain a high spatial resolution MS (HRMS) image. In this article, we propose a novel deep neural network architecture with level-domain-based loss function for pansharpening by taking into account the following double-type structures, i.e., double-level, double-branch, and double-direction, called as triple-double network (TDNet). By using the structure of TDNet, the spatial details of the PAN image can be fully exploited and utilized to progressively inject into the low spatial resolution MS (LRMS) image, thus yielding the high spatial resolution output. The specific network design is motivated by the physical formula of the traditional multi-resolution analysis (MRA) methods. Hence, an effective MRA fusion module is also integrated into the TDNet. Besides, we adopt a few ResNet blocks and some multi-scale convolution kernels to deepen and widen the network to effectively enhance the feature extraction and the robustness of the proposed TDNet. Extensive experiments on reduced- and full-resolution datasets acquired by WorldView-3, QuickBird, and GaoFen-2 sensors demonstrate the superiority of the proposed TDNet compared with some recent state-of-the-art pansharpening approaches. An ablation study has also corroborated the effectiveness of the proposed approach. The code is available at https://github.com/liangjiandeng/TDNet .
Fichier principal
Vignette du fichier
full_text_hal_04473420.pdf (7.55 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04473420 , version 1 (23-04-2024)

Licence

Attribution

Identifiers

Cite

Tian-Jiang Zhang, Liang-Jian Deng, Ting-Zhu Huang, Jocelyn Chanussot, Gemine Vivone. A Triple-Double Convolutional Neural Network for Panchromatic Sharpening. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34 (11), pp.9088-9101. ⟨10.1109/TNNLS.2022.3155655⟩. ⟨hal-04473420⟩
29 View
0 Download

Altmetric

Share

Gmail Facebook X LinkedIn More