Researchers from Yunnan Observatories and Southwest Forestry University have developed a novel method for compressing spectral data from the New Vacuum Solar Telescope (NVST), providing an effective solution for data storage and transmission challenges. The findings have been published in Solar Physics.

The NVST generates a massive amount of spectral data, posing significant challenges for storage and efficient transmission. Previous methods, such as principal component analysis (PCA), achieved limited compression ratios (~30) and introduced noticeable distortions in the reconstructed data. The new method, however, marks a transformative leap.

The researchers developed a neural network method based on a Convolutional Variational Autoencoder (VAE) for efficiently compressing Ca II (8542 Å) spectral data. This method has achieved compression ratios up to 107, while maintaining data integrity. Errors between the original and decompressed data remain within the inherent error range of the raw data, ensuring the scientific validity of the compressed data.

Solar physicists at the Fuxian Lake Solar Observatory noted: "The stability of the VAE method is particularly impressive. Even at a compression ratio of 107, the error in Doppler velocity remains below 5 km/s, which is crucial for solar physics research."

The improved compression technique enables more efficient NVST data transmission and sharing. It also offers effective solutions for other solar observatories facing similar data transmission challenges, thus facilitating broader data sharing within the scientific community.

This research was supported by  the projects of the National Natural Science Foundation of China (NSFC), the Yunnan Fundamental Research Projects, the Yunnan Province XingDian Talent Support Program and the “Yunnan Revitalization Talent Support Program” Innovation Team Project.

Figure1: left: Doppler images reconstructed from VAE-compressed spectral data (compression ratios: 30, 58, 107, and 183). Right: Difference images comparing the Doppler images (left) and those derived from the raw spectral data. Image by DONG.

Contact:
DONG Yan
Yunnan Observatories, CAS
Email: dongyan@ynao.ac.cn