STAR-24K: A Public Dataset for Space Common Target Detection

Abstract

The target detection algorithm based on supervised learning is the current mainstream algorithm for target detection. A high-quality dataset is the prerequisite for the target detection algorithm to obtain good detection performance. The larger the number and quality of the dataset, the stronger the generalization ability of the model, that is, the dataset determines the upper limit of the model learning. The convolutional neural network optimizes the network parameters in a strong supervision method. The error is calculated by comparing the predicted frame with the manually labeled real frame, and then the error is passed into the network for continuous optimization. Strongly supervised learning mainly relies on a large number of images as models for continuous learning, so the number and quality of images directly affect the results of learning. This paper proposes a dataset STAR-24K (meaning a dataset for Space TArget Recognition with more than 24,000 images) for detecting common targets in space. Since there is currently no publicly available dataset for space target detection, we extracted some pictures from a series of channels such as pictures and videos released by the official websites of NASA (National Aeronautics and Space Administration) and ESA (The European Space Agency) and expanded them to 24,451 pictures. We evaluate popular object detection algorithms to build a benchmark. Our STAR-24K dataset is publicly available at https://github.com/Zzz-zcy/STAR-24K.

1. Introduction

It has been widely discovered that the recent remarkable success of computer vision technologies, especially those based on deep learning, relies heavily on large-scale and well-annotated datasets. The target detection algorithm based on supervised learning is the current mainstream algorithm for target detection. A high-quality dataset is the prerequisite for the target detection algorithm to obtain good detection performance. The larger the number and quality of the dataset, the stronger the generalization ability of the model, that is, the dataset determines the upper limit of the model learning. For example, CIFAR-10/100 [1] and ImageNet [2] are important activators for deep convolutional neural networks [3]. Pascal VOC [4] and MS COCO [5] are used for common object detection and segmentation, and LFW [6] is used for face recognition. Caltech Pedestrians [7] and MOT benchmark [8] are used for human detection and tracking. Among all these tasks, it is because an excellent dataset promotes the development of various detection technologies. However, due to the sensitivity of satellite and other data, there is no publicly available dataset, evaluating the accuracy of space target detection in optical images is also difficult.

In recent years, with the rapid development of satellite and rocket technology, more and more spacecraft have entered space. When spacecraft are working, they often encounter satellites and space junk from other countries. If two satellites collided, thousands of pieces of space debris would be generated in space, as shown in Fig. 1, where (a) the distribution of debris 20 minutes after the collision; (b) Debris distribution 50 minutes after the collision. The collision of these generated debris on aircraft will have serious consequences, as shown in Table 1. These objects exist in space and will accumulate more and more, and in the long run, it will definitely affect the development of human aerospace industry and the operation of the earth, so the detection of satellites or debris in space is a very meaningful work. Although deep convolutional networks are very effective for feature extraction and object detection, but the premise of target detection must have a good dataset, and the process of iterative training of convolutional neural networks is driven by massive amounts of data. At present, for images of space targets, relevant data sets at home and abroad are very scarce. Therefore, this article collected a large number of satellite and space junk images for expansion and annotation, and constructed a data set STAR-24K (meaning Space TArget Recognition with more than 24, 000 images), which can be used to detect space targets. It is the labeled dataset that matters in the era of deep learning.

Fig. 1. Satellite collision diagram

Table 1. Space debris colliding experiments on aerospace aluminum

2. Related Work

With the increasingly frequent space activities, the number of space targets in the earth's orbit has increased year by year, and it is inevitable that more and more space debris will be brought. The number of cataloged space targets in the Earth's orbit has been increasing year by year, as shown in Fig. 2 [9]. Therefore, the development of space target detection technology has no time to delay.

Fig. 2. The increasing situation of the number of space objects on earth orbit with each passing year.

Space target detection is divided into two types: ground-based and space-based. Among them, ground-based detection is mainly composed of ground-based photoelectric detectors, radar detectors, etc. It has the advantages of mature technology, convenient use and maintenance, but due to the limitation of meteorological conditions, geographical location and other factors [10], there are still many limitations in the detection range and timeliness of the target. Space-based space target detection uses imaging measurement devices installed on the space platform to detect and track targets. It has the advantages of being unaffected by weather and flexible in maneuverability [11], which represents a new research direction in space target detection. Space-based space target visible light imaging process as shown in Fig. 3: the sun, the earth reflected light and moonlight space stray light irradiation, the target after target surface reflection to observation platform, and then through the observation of the CCD sensor system transmission platform and focus, formed in the imaging plane background to black, the goal for the dot light spot observation images. The main task of space target detection is to detect, track, and identify space targets, and determine the shape, size, and movement status of the target. However, many countries' research on space-based space target detection system is relatively late, and there is a big gap between it and foreign countries in target detection and recognition. Therefore, in-depth research on space-based space target detection technology and a large-scale and well-annotated dataset will greatly enhance detection capabilities for space targets.

Fig. 3. Schematic diagram of space-based space target imaging.

In many space target detection systems, the use of optical images to detect and track space targets is still a difficult problem. Due to the large distance between the space target and the detector, the target imaging usually only covers a small number of pixels, which means that the target looks like a low-intensity point embedded in strong background clutter [12]. Therefore, although traditional feature-based target detection methods perform well under uniform background conditions, they may not be able to detect space targets under complex backgrounds, such as template matching [13, 14], morphological operations [15, 16, 17], Threshold method [18, 19], high-pass filtering [20], Hessian matrix [21, 22]. In addition, in the field of infrared surveillance [23, 24, 25], a variety of space target detection methods have also been proposed, but the detection results in space-based visible optical images are not ideal. Reed et al., “[26, 27] proposed a recursive space target detection algorithm based on three-dimensional matched filtering, which theoretically shows the best detection performance for space targets with the same moving speed. However, for targets with unknown speeds, the performance of the algorithm decreases. In 2015, Yao et al., “[28] proposed a space target detection and velocity estimation method based on the maximum projection method. These space target detection algorithms mentioned are not based on deep learning, and many of them use artificially designed target features to achieve single target detection. Then use simulation software such as TASAT, SVST and STK EOIR to simulate the space scene and then carry out the detection task. It can be seen that due to the extreme lack of spatial target datasets, the development of deep learning in the detection of space targets is very slow. Therefore, proposing a dataset on space targets is of great significance to the development of this field. After our investigation, there are three verified data sets about satellite targets, BUAA-SID [29], SPEED [30] and SPEED+ [31]. The BUAA-SID dataset was established in 2010, and the satellite images in the dataset were all simulated by computer rather than real optical images, which lacked a certain scientific nature. The amount of data required for deep learning is also relatively small. SPEED and SPEED+ data sets are both used to solve the satellite pose estimation problem, not to perform satellite detection tasks. However, in the field of space target detection, there is still a lack of data set, so the data set we established is relatively leading in both the quantity and quality of images. While challenging and costly, it is very necessary to establish a large-scale benchmark to facilitate research in this field.

3. Establishment of STAR-24K

3.1 Images Collection

Obtaining images is the first step in dataset production. Since there are fewer tasks related to space target detection and have a certain degree of confidentiality. However, target detection requires a large number of samples to train the detector, and it is very difficult to obtain the required number of datasets for target detection tasks, so it is necessary to manually make a dataset for space target detection. Most of the pictures about space junk, satellites, and planets collected directly from search engines are too illusory and not real enough, and some are of low quality so that there are few pictures that really meet the requirements of dataset production. Therefore, the pictures of the data set produced in this article are mainly derived from three aspects: (1) Select from the pictures published by NASA (National Aeronautics and Space Administration) and ESA (The European Space Agency) on the official websites, and take a frame-by-frame screenshot from the video they released about space exploration. (2) Single-frame interception from short space exploration science videos published by a series of internet video sites such as YouTube. (3) Watch "Space Junk", "Horizon" and other documentaries, from which intercept images related to space satellites, junk, and planets.

In the process of video interception, this article specially selected different angles, different lights, different positions, different scenes and different types of pictures to ensure the diversity of the targets in the pictures and helps prevent under-fitting and over-fitting during the training process. The pictures selected in this article mainly include three categories: satellites, planets, and space junk (as shown in Fig. 4). The three types of targets selected are also the most common objects in space. However, because there are not many resources on space target detection on the Internet, the total number of collected images is 600, which cannot reach the number required for network learning. Therefore, this article first mixes the collected pictures as the original picture, and then uses data augmentation methods to amplify, the amplified pictures reach 24, 451. For general network training, the number of this dataset meets the requirements.

Fig. 4. Some common space targets. (a) On the left is an artificial satellite, on the right is a planet, (b) represents space junk, which is generally unregulated.

3.2 Data Augmentation

Data augmentation technology is the most effective method to solve the problem of small number of original data sets. Data augmentation is a general term for methods to expand data. It can increase the number of samples in the data set, effectively alleviate the problem of model overfitting, and can also bring better generalization capabilities to the model. There are two ways of data augmentation. One is offline augmentation. Offline augmentation refers to performing all the augmentations in advance, and then sending them to the network for learning. This method is suitable for the case where the data set is relatively small, and the number of pictures is doubled several times through methods such as rotation, translation, and flipping. Another data augmentation method is online augmentation. Online augmentation is to send the original data directly to the network for small batch conversion. This method is suitable for large data sets.

This article chooses offline data augmentation. Offline data augmentation can be divided into pixel data augmentation and image data augmentation. Pixel data augmentation includes changing brightness and adding noise. Image data augmentation mainly includes cropping, translation, mirroring, rotation, and occlusion. The methods used in the data augmentation in this article include darkening, brightening, blurring, adding Gaussian noise and salt and pepper noise, mirroring, turning the original image by 90 degrees and turning it by 180 degrees. Then flip 90 degrees and 180 degrees on the darkened, brightened, blurred, and increased noise pictures, and then change the brightness and increase the noise on the flipped pictures (as shown in Fig. 5), and finally make the data set on the original basis through these operations increased by 17 times. Google Brain [32] Team proved that deep learning can achieve strong performance on large amounts of data, and data augmentation is required to increase the training data of deep learning. Scale change, translation and rotation are often used to enhance data amount. In addition, it is also effective to randomly add noise and occlusion to the image. At the same time, some dataset papers also use data augmentation when establishing data sets, such as Yanghua Di et al., “'s ship dataset FGSCR-42 dataset [33].

Fig. 5. Data augmentation by darkening, brightening, blurring, adding Gaussian noise, etc.

Then use the enhanced data for image stitching, and combine the four images into one, which greatly enriches the background and target of the detected object, and because of the distance, many targets in space are small targets. Therefore, splicing in this way also provides help for the detection of small targets and increases the robustness of the model. Sending this new picture to the network for learning is equivalent to changing from only learning one picture at a time to four pictures at a time, which greatly improves the learning ability of the model.

Before data augmentation, the original pictures are named in order, and then the pictures are uniformly scaled to 960×960 pixels, as shown in Fig. 6. Then the pictures are augmented, and the augmented pictures are 17 times larger than the original. After the pictures are augmented, the pictures are seamlessly spliced, and four pictures are randomly selected to form one picture. The picture pixels after splicing are 1920×1920 pixels, as shown in Fig. 7. A total of 600 pictures were collected, and the pictures were divided into 480 pictures in the training set and 120 pictures in the test set. Then use these 600 pictures as the original pictures for data augmentation. Separating the original pictures first and then performing data augmentation can ensure that there is no same picture in the training set and the test set. This can effectively prevent data leakage in the process of training and testing from affecting the performance of the trained model. The data set after data augmentation is 24451, among which the training set is 20451 and the test set is 4000. The number of instances of each class are shown in Table 2. In this data set, we mainly focus on detecting satellites, so satellites account for a large proportion of instances.

Fig. 6. The images scaled to 960×960 pixels are named sequentially.

Fig. 7. Randomly select four pictures for stitching.

Table 2. The number of instances of each class

3.3 Image Annotation

Since the data set is the self-selected images and needs to be manually labeled, this article uses LabelImg software for manual labeling. To obtain high-quality space target annotations for each image, we recruit 10 well-trained annotators and ask them to annotate all the space targets they can distinguish. Three rounds of cross-checking and correction are then carried out to improve the annotation quality. Finally, it takes about one month to complete the whole annotation process, where 20451 images are labeled. See Fig. 8 for example images with annotation.

Fig. 8. Example images with annotation.

Considering the ease of sharing and saving to the computer, the annotations are saved in line with the COCO format to facilitate further research in space target detection by reusing common training and evaluation tools developed in the object detection community. The STAR-24K is split into three disjoint subsets, i.e., train, validation, and test sets, at the ratio of 8:1:1 per space target species.

4. Evaluations

Common target detection networks can be used to detect space targets, but the detection results need to be improved. Considering that the space target detection task requires high real-time performance, there are also certain requirements for detection accuracy. For the comprehensiveness of the results, we eventually selected CenterNet [34], YOLOv3 [35], YOLOv4-tiny [36], YOLOv4 [37] and YOLOv5 [38], as our testing algorithms for building a benchmark on STAR-24K. These baseline models cover the popular and the state-of-the-art deep learning target detection networks [39] [40] [41], thus can offer valuable benchmark results for dataset evaluation. Meanwhile, to demonstrate the effectiveness of data augmentation, we add a set of comparative experiments. After data augmentation, the experiment is to split the data set according to 8:2. As a result, the 600 initial images were divided in a ratio of 8:2. Among them, 480 are used for training, and 120 are used for testing. Compared with the training on YOLOv5 algorithm, the validity of data augmentation is proved (as shown in Table 3).

Table 3. Comparison of initial data and data augmentation

For the task of space target detection, the performance is evaluated as the recall and mAP.50 as metrics. Average precision (AP) refers to the area enclosed under the PR curve, and mean average accuracy (mAP) refers to the average accuracy rate under different categories, which is to calculate AP values for multiple verification sets; Recall refers to the proportion of actual positive samples in all prediction samples relative to samples. Their calculation formula is as follows. mAP.50 is the mean average precision at IoU = 0.50.

\(A P=\frac{T P}{T P+F P}\)       (1)

\(\text { Re call }=\frac{T P}{T P+F N}\)       (2)

Where TP indicates that the actual value is a positive sample, and the predicted value is also a positive sample number; TN indicates that the actual value is a negative sample, and the predicted value is also a negative sample number; FP refers to the number of samples with negative actual value and positive predicted value; FN indicates that the actual value is a positive sample and the predicted value is a negative sample.

\(m A P=\frac{\sum_{i=1}^{n} A P(i)}{n}\)        (3)

Where i is the current category, n is the number of categories, and AP (i) is the accuracy of the current category.

This experiment is conducted in the environment of Ubuntu 16.04 operating system, 16GB memory and Titan x graphics card, then the network weight is obtained through training, and the weight is used to test the pictures of the test set. Benchmark results of the selected baseline models are shown in Table 4. It can be seen that the recall is higher than the other index. Since the number of artificial satellites in the data set is more than that of other targets, the recall will be higher, but the mAP.50 is in the normal range. Fig. 9, Fig. 10, Fig. 11 and Fig. 12 are the test results of algorithms CenterNet, YOLOv4-tiny, YOLOv4 and YOLOv5 respectively from an image selected from dataset STAR-24K. The test results also shows that YOLOv5 performs better.

Table 4. The detection result of baseline model

Fig. 9. CenterNet test result.

Fig. 10. YOLOv4-tiny test result.

Fig. 11. YOLOv4 test result.

Fig. 12. YOLOv5 test result.

5. Conclusion

In this paper, we presented STAR-24K, a new public dataset for space target detection, which mainly includes three categories: artificial satellites, stars, and space junk. The dataset contains 24451 pictures, of which 20451 are in the training set and 4000 are in the test set. As the first public dataset specially released for space target detection, we believe that STAR-24K has the potential of introducing space target recognition as a novel domain to the wider computer vision community. It could be used to evaluate space target detection methods and train space target detection methods as well. Considering the application of the dataset, we have also established a benchmark for space target detection. This could be useful to future development of space target detection algorithms in optical images. In the future work, we will further expand the amount of categories and the size of the dataset. If possible, we will invite more experts to make more precise annotations of the dataset.