A Hybrid DBN and CRF Model for Spectral-Spatial Classification of Hyperspectral Images
Abstract
Hyperspectral image classification plays an important role in remote sensing image analysis. Recent techniques have attempted to investigate the capabilities of deep learning approaches to tackle the hyperspectral image classification. This work shows how to further improve the hyperspectral image classification through using both a deep representation and contextual information. To implement this objective, this work proposes a new Conditional Random Field (CRF) model (named DBN-CRF) with the potentials defined over the deep features produced by a Deep Belief Network (DBN). The newly formulated DBN-CRF model takes advantage of the strength of DBNs in learning a good representation and the ability of CRFs to model contextual (spatial) information in both the observations and labels. Within a piecewise training framework, an efficient training method is proposed to train the whole DBN-CRF model end-to-end. This means that the parameters in DBN and CRF can be jointly trained and thus the proposed method can fully use the strength of both DBN and CRF. Moreover, in the proposed training method, the end-to-end training can be implemented with a standard back-propagation algorithm, avoiding the repeated inference usually involved in CRF training and thus is computationally efficient. Experiments on real-world hyperspectral data show that our method outperforms the most recent approaches in hyperspectral image classification.References
P. W. Yuen and M. Richardson, “An introduction to hyperspectral imaging and its application for security, surveillance and target acquisition,” Imaging Science Journal, vol. 58, no. 5, pp. 241-253, 2010.
T. Chen, P. Yuen, M. Richardson, G. Liu, and Z. She, “Detection of psychological stress using a hyperspectral imaging technique,” IEEE Transactions on Affective Computing, vol. 5, no. 4, pp. 391-405, 2014.
X. Tong, H. Xie, and Q. Weng, “Urban land cover classification with airborne hyperspectral data: what features to use?” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 7, no. 10, pp. 3998-4009, 2014.
C. M. Gevaert, J. Suomalainen, J. Tang, and L. Kooistra, “Generation of spectral–temporal response surfaces by combining multispectral satellite and hyperspectral UAV imagery for precision agriculture applications,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 8, no. 6, pp. 3140-3146, 2015.
J. M. Bioucas-Dias, A. Plaza, G. Camps-Valls, P. Scheunders, N. M. Nasrabadi, and J. Chanussot, “Hyperspectral remote sensing data analysis and future challenges,” IEEE Geoscience and Remote Sensing Magazine, vol. 1, no. 2, pp. 6 - 36, 2013.
P. Zhong, P. Zhang, and R. Wang, “Dynamic learning of sparse multinomial logistic regression for feature selection and classification of hyperspectral data,” IEEE Geoscience and Remote Sensing Letter, vol. 5, no. 2, pp. 280-284, 2008.
P. Zhong and R. Wang, “Jointly learning the hybrid CRF and MLR model for simultaneous denoising and classification of hyperspectral imagery,” IEEE Transactions on Neural Networks and Learning Systems, vol. 25, no. 7, pp. 1319-1334, 2014.
M. Khodadadzadeh, J. Li, A. Plaza, and J. M. Bioucas-Dias, “A subspace-based multinomial logistic regression for hyperspectral image classification,” IEEE Geoscience and Remote Sensing Letter, vol. 11, no. 12, pp. 2105-2109, 2014.
F. Ratle, G. Camps-Valls, and J. Weston, “Semisupervised neural networks for efficient hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 48, no. 5, pp. 2271-2282, 2010.
Y. Zhong, W. Liu, J. Zhao, and L. Zhang, “Change detection based on pulse-coupled neural networks and the NMI feature for high spatial resolution remote sensing imagery, ” IEEE Geoscience and Remote Sensing Letters, vol. 12, no. 3, pp. 537-541, 2015.
F. Melgani and L. Bruzzone, “Classification of hyperspectral remote sensing images with support vector machines,” IEEE Transactions on Geoscience and Remote Sensing, vol. 42, no. 8, pp. 1778-1790, 2004.
J. Peng, Y. Zhou, and C. L. P. Chen, “Region-kernel-based support vector machines for hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 9, pp. 4810-4824, 2015.
G. Camps-Valls, T. V. B. Marsheva, and D. Zhou, “Semi-supervised graph-based hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, 45, pp. 3044-3054. 2007.
Y. Gao, R. Ji, P. Cui, Q. Dai, and G. Hua, “Hyperspectral image classification through bilayer graph-based learning,” IEEE Transactions on Image Processing, vol. 23, no. 7, pp. 2769-2778, 2014.
S. Kawaguchi and R. Nishii, “Hyperspectral image classification by bootstrap AdaBoost with random decision Stumps,” IEEE Transactions on Geoscience and Remote Sensing, vol. 45, no. 11, pp. 3845-3851, 2007.
S. Sun, P. Zhong, H, Xiao, and R. Wang, “Active learning with Gaussian process classifier for hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 4, pp. 1746-1760, 2014.
J. Ham, Y. Chen, M. M. Crawford, and J. Ghosh, “Investigation of the random forest framework for classification of hyperspectral data,” IEEE Transactions on Geoscience and Remote Sensing, vol. 43, no. 3, pp. 492-501, 2005.
M. Fauvel, Y. Tarabalka, J. A. Benediktsson, J. Chanussot, and J. C. Tilton, “Advances in spectral-spatial classification of hyperspectral images,” Proceedings of the IEEE, vol. 101, no. 3, pp. 652-675, 2013.
X. Jia, B. Kuo, and M. M. Crawford, “Feature mining for hyperspectral image classification,” Proceedings of the IEEE, vol. 101, no. 3, pp. 676-697, 2013.
Y. Zhou and Y. Wei, “Learning hierarchical spectral-spatial features for hyperspectral image classification,” IEEE Transactions on Cybernetics, vol. PP, no. 99, pp. 1-12, 2015.
P. Ghamisi, J. A. Benediktsson, G. Cavallaro, and A. Plaza, “Automatic framework for spectral–spatial classification based on supervised feature extraction and morphological attribute profiles,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 7, no. 6, pp. 2147-2160, 2014.
N. Falco, J. A. Benediktsson, and L. Bruzzone, “Spectral and spatial classification of hyperspectral images based on ICA and reduced morphological attribute profiles,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 11, pp. 6223-6240, 2015.
Z. Zhong, B. Fan, J. Duan, L. Wang, K. Ding, S. Xiang, and C. Pan, “Discriminant tensor spectral-spatial feature extraction for hyperspectral image classification,” IEEE Geoscience and Remote Sensing Letter, vol. 12, no. 5, pp. 1028-1032, 2015.
Y. Qian, M. Ye, and J. Zhou, “Hyperspectral image classification based on structured sparse logistic regression and three-dimensional wavelet texture features,” IEEE Transactions on Geoscience and Remote Sensing, vol. 51, no. 4, pp. 2276-2291, 2013.
W. Li, C. Chen, H. Su, and Q. Du, “Local binary patterns and extreme learning machine for hyperspectral imagery classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 7, pp. 3681-3693, 2015.
F. Tsai and J. Lai, “Feature extraction of hyperspectral image cubes using three-dimensional gray-level cooccurrence,” IEEE Transactions on Geoscience and Remote Sensing, vol. 51, no. 6, pp. 3504-3513, 2013.
Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectral images using minimum spanning forest grown from automatically selected markers,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 40, no. 5, pp. 1267-1279, 2010.
J. Bai, S. Xiang, and C. Pan, “A graph-based classification method for hyperspectral images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 51, no. 2, pp. 803-817, 2013.
S. Sun, P. Zhong, H. Xiao, and R. Wang, “An MRF model-based active learning framework for the spectral-spatial classification of hyperspectral imagery,” IEEE Journal of Selected Topics in Signal Processing, vol. 9, no. 6, pp. 1074-1088, 2015.
Y. Yuan, J. Lin, and Q.Wang, “Hyperspectral image classification via multitask joint sparse representation and stepwise MRF optimization,” IEEE Transactions on Cybernetics, vol. PP, no. 99, pp. 1-12, 2015.
P. Zhong and R. Wang, “Learning conditional random fields for classification of hyperspectral images,” IEEE Transactions on Image Processing, vol. 19, no.7, pp. 1890-1907, 2010.
P. Zhong and R. Wang, “Learning sparse CRFs for feature selection and classification of hyperspectral imagery,” IEEE Transactions on Geoscience and Remote Sensing, vol. 46, no. 12, pp. 4186-4197, 2008.
P. Zhong and R. Wang, “Modeling and classifying hyperspectral imagery by CRFs with sparse higher order potentials,” IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 2, pp. 688-705, 2011.
M. Alioscha-Perez and H. Sahli, “Efficient learning of spatial patterns with multi-scale conditional random fields for region-based classification,” Remote Sensing, vol. 6, no. 8, pp. 6727-6764, 2014.
Y. Zhong, J. Zhao, and L. Zhang, “A hybrid object-oriented conditional random field classification framework for high spatial resolution remote sensing imagery, ” IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 11, pp. 7023-7037, 2014.
F. Li, L. Xu, P. Siva, A. Wong, and D. A. Clausi, “Hyperspectral image classification with limited labeled training samples using enhanced ensemble learning and conditional random fields,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 8, no. 6, pp. 2427-2438, 2015.
Y. Chen, Z. Lin, X. Zhao, G. Wang, and Y. Gu, “Deep learning-based classification of hyperspectral data,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 7, no. 6, pp. 2094-2107, 2014.
C. Tao, H. Pan, Y. Li, and Z. Zou, “Unsupervised spectral-spatial feature learning with stacked sparse autoencoder for hyperspectral imagery classification,” IEEE Geoscience and Remote Sensing Letter, vol. 12, no. 12, pp. 2438-2442, 2015.
Y. Chen, X. Zhao, and X. Jia, “Spectral-spatial classification of hyperspectral data based on deep belief network,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 8, no. 6, pp. 2381-2392, 2015.
W. Hu, Y. Huang, L. Wei, F. Zhang, and H. Li, “Deep convolutional neural networks for hyperspectral image classification,” Journal of Sensors, vol. 2015, Article ID: 258619, 12 pages, 2015.
A. Romero, C. Gatta, and G. Camps-Valls, “Unsupervised deep feature extraction for remote sensing image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. PP, no. 99, pp. 1-14, 2015.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521 , pp. 436-444, 2015.
A. Krizhevsky, I. Sutskever and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” In Proc. Advances in Neural Information Processing Systems (NIPS), 25, pp. 1090-1098, 2012.
R. Girshick, J. Donahue, T. Darrell and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, and P. Torr, “Conditional random fields as recurrent neural networks,” In Proc. International Conference on Computer Vision (ICCV), 2015.
L. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Semantic image segmentation with deep convolutional nets and fully connected CRFs,” In Proc. International Conference on Learning Representations (ICLR), 2015.
G. Lin, C. Shen, I. Reid, and A. Hengel, “Efficient piecewise training of deep structured models for semantic segmentation,” In arXiv:1504.01013, 2015.
G. E. Hinton, S. Osindero, and Y. Teh, “A fast learning algorithm for deep belief nets,” Neural Computation, vol. 18, pp. 1527-1554, 2006.
C. M. Bishop, “Neural networks for pattern recognition”, Oxford University Press, 1996.
C. Sutton and A. McCallum, “Piecewise training of undirected models,” in Proc. Conference on Uncertainty in Artificial Intelligence (UAI), 2005.
J. Shotton, J. Winn, C. Rother, and A. Criminisi, “TextonBoost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation,” in Proc. European Conference on Computer Vision (ECCV), 2006.
B. Frey and D. J. C. Mackay, “A revolution: Belief propagation in graphs with cycles,” in Proc. Advances in Neural Information Processing Systems (NIPS), 1997.
Y. Bazi and F. Melgani, “Toward an optimal SVM classification system for hyperspectral remote sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 44, no. 11, pp. 3374-3385, 2006.
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
