{'fraction_non_alphanumeric': 0.021293207466818086, 'fraction_numerical': 0.006600894314713606, 'mean_word_length': 5.104852686308492, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Machine learning aided noise filtration and signalc lassification for CREDO experiment

The wealth of smartphone data collected by the Cosmic Ray Extremely Distributed Observatory(CREDO) greatly surpasses the capabilities of manual analysis. So, efficient means of rejectingthe non-cosmic-ray noise and identification of signals attributable to extensive air showers arenecessary. To address these problems we discuss a Convolutional Neural Network-based method ofartefact rejection and complementary method of particle identification based on common statisticalclassifiers as well as their ensemble extensions. These approaches are based on supervised learning,so we need to provide a representative subset of the CREDO dataset for training and validation.According to this approach over 2300 images were chosen and manually labeled by 5 judges.The images were split into spot, track, worm (collectively named signals) and artefact classes.Then the preprocessing consisting of luminance summation of RGB channels (grayscaling) andbackground removal by adaptive thresholding was performed. For purposes of artefact rejectionthe binary CNN-based classifier was proposed which is able to distinguish between artefacts andsignals. The classifier was fed with input data in the form of Daubechies wavelet transformedimages. In the case of cosmic ray signal classification, the well-known feature-based classifierswere considered. As feature descriptors, we used Zernike moments with additional feature relatedto total image luminance. For the problem of artefact rejection, we obtained an accuracy of 99%. For the 4-class signal classification, the best performing classifiers achieved a recognition rate of 88%.


Introduction
The Cosmic Ray Extremely Distributed Observatory (CREDO) is an international project aimed at registering the various kinds of radiation, of both terrestrial and extraterrestrial origin [1]. Even though CREDO is sensitive to all kinds of radiation, its particular interest is focused on the secondary radiation produced by collisions of the Ultra High Energy Cosmic Rays with the atmosphere and called Extensive Air Showers. The CREDO computing and database infrastructure is a hub to aggregate the data collected by various detectors like Cosmic Watches [2] and other low cost scintillator detectors but the flagship of the project is the CREDO Detector application installed on thousands of smartphones worldwide and able to register the cosmic rays while they pass through CMOS sensors of smartphone cameras [3]. There exist a few other initiatives of this type like DECO [4] and CRAYFIS [5]. The common feature of all these projects is conducting the observations on the planetary scale. This way they are complementary to specialized, highly efficient but still limited in space experiments like Auger [6], Telescope Array [7] or IceCube [8]. Currently there are about 10 7 candidate detections in the CREDO database and to obtain the scientifically significant statistics this number is planned to increase by two orders of magnitude. CREDO is a citizen science type project and its performance relies on the commitment of the volunteer participants. To sustain the users' commitment CREDO utilizes various gamification strategies, eg. periodic Particle Hunters Competitions. These competitions temporarily increase the number candidate detections sent to the CREDO database but this increased activity also results in the surge of artefacts. The artefacts are images that cannot be attributed to particle's passage through the sensor but rather result from hardware malfunction or user's cheating. The artefacts can be eliminated either on-line ie. using the algorithm working on the smartphone [9] or off-line ie. through the analysis and tagging the images stored in the database. The software used to filter the artefacts and pass signals is usually called the trigger. Here we discuss the off-line Convolutional Neural Network (CNN) based trigger, even though the baseline trigger we start our discussion from is both reasonably efficient and does not require extensive computational resources, so that can be used also on line with limited resources available in smartphones. Another problem we want to discuss here is the classification of signals from CMOS sensors. There is some recognized terminology in the literature in this respect, namely spots -for point-like images, tracks -for straight lines and worms -for images containing curvylinear signals. Moreover, some of these traces can be even assigned to particular particle species, eg. thin tracks are believed to be produced by muons while worms -by -electrons [10]. Here we refrain from discussing the relation between trace shape and particle assignment and base our class assignment exclusively upon morphological properties of traces. This is because evidence collected so far from the CREDO database suggests that details of tracks are strongly hardware and software dependent. Eg. the thickness of the track or worm depends the underlying CMOS array resolution as well as on the algorithm used to transform the raw signal into an image saved in one of the commonly used formats. We believe that reliable trace to particle species assignment requires both deep understanding of the image processing algorithm and additional data like time correlations with images registered by nearby devices. Such correlations provide the evidence of possible EAS signal dominated by muons. To perform the signal classification we test a rich array of classical statistical classifiers including kNN, SVM, tree based classifiers as well as dense feed forward networks. We also test their ensemble versions. All these classifiers require a carefully chosen features to perform the training. The reason for which we use feature based classifiers instead of CNNs is that most of the analyses performed so far used various geometrical properties of signal to describe and classify them [11]. Here we use more general image features expressed in terms of Zernike moments which proved their usefulness in the recognition of facial expression [12,13] or gender classification based on facial images [14].
To train both the CNN based trigger and the statistical classifiers used for the four class hit identification we use the set of 2354 examples drawn from the CREDO database and labelled by 5 annotators.

Baseline trigger
To begin with, we consider the baseline trigger, ie. a simple classifier which assigns the hit image to either signal or artefact class based on the most general and distinctive properties if the image. The role of the baseline trigger is twofold. First, it provides a lower bound of the classification accuracy for the more elaborate, eg. neural network based classifiers, while being entirely explainable [15]. Second, relying on the simple heuristics they are much less computationally challenging, so can be effortlessly implemented as online triggers working with limited resources of smartphones. One sees from Fig. 1 that there are indeed two characteristic features that distinguish artefacts and signals (spots, tracks and worms combined). These are the integrated brightness and the number of lit (active) pixels. Thus we define the decision boundary for the base trigger in terms of these two variables that we denote and , respectively. and are computed relative to the adaptively computed threshold defined by Eq. 1.
where is defined by with and denoting the average and standard deviation. For the artefact and signal classes we define the limiting values of and , namely the minimum integrated luminosity and minimum number of active pixels for images labeled as artefacts and maximal integrated luminosity and maximal number of active pixels for images labeled as signals. The parameters determining the decision boundary are defined as the average of respective quantities = ( + )/2 and = ( + )/2. With parameters defining the decision boundary we define the decision boundary itself as the quarter ellipse Thus all hits whose integrated luminosity and number of active pixels fall inside the quarter ellipse are classified as signals and those outside of it, as artefacts. The distribution of the signal and artefact labelled examples in the ( , ) plane is shown in Fig. 2. From Fig. 2 one immediately sees that the decision boundary of the form given by Eq. 3 indeed separates the the artefact and signal examples except for the small admixture of artefacts with small luminosities and number of active pixels. The accuracy obtained for the baseline trigger on the evaluation data set is surprisingly good and reaches almost 99% for signals and 95% for artefacts (see [16] for details).

Artefact rejection with the CNN based trigger
Having defined the baseline trigger we can refine the artefact rejection process by employing a more elaborate classifier. To this end we use the Convolutional Neural Network trained on both the preprocessed and unpreprocessed dataset. The preprocessing phase is performed in two versions. In the first version the raw images are just subject to adaptive thresholding as per Eq. 1. In the second version, apart from thresholding the Daubechies wavelet transform is applied [17]. After the preprocessing the images are fed to the CNN for training and evaluation (see [16] for details on the network architecture and evaluation). The objective of the wavelet transform is to enhance the trigger capability to recognize the borders of the traces left by particles. The Daubechies wavelet transforms the original image into 4 images of two times smaller resolution than the original image according to the formula where f is the original image function, subimage a denotes the average signal while h, v and d denote the horizontal, vertical and diagonal fluctuations, respectively [18]. The fluctuation components are usually decisive for detecting the objects' borders (trace border in this case). However, in this study we found that the impact of using either single or combined Daubechies wavelets does not impact the trigger performance. The overall trigger performance is illustrated by confusion matrices in Fig. 3. It confirms that the signal-artefact separation accuracy is identical within one standard deviation, regardless of either applying a wavelet transform or not. This, quite surprising result which we interpret as a consequence of the fact that (as shown in Fig. 1) both the signal and artefact classes are very diverse morphologically and combining examples with very diverse features, even though determined very accurately, leads to strong feature averaging.

Signal classification
Having established a method of separating the images depicting signals and artefacts we went on to subdivide the signal class into three morphologically distinct classes of spots, tracks and worms. Together with artefacts this makes the problem of the four class categorical classification.
In the literature these classes have been associated with particular particles species [10], however, here we focus just on their morphological differences and leave particle association for further study.
Unlike the trigger where we had used the binary CNN classifier, in this exploratory study of the four class classification we employed a series of feature based classical statistical classifiers, like naive Bayesian, decision trees, kNNs, support vector machines, logistic regression, multilayer perceptrons as well as their ensemble versions. We used Zernike moments as feature carriers for all classifiers [12][13][14]. Since the images to be classified were relatively simple compared to eg. human faces, applying the low order moments seemed to be justified. Therefore we used the 8 ℎ order moments with 25 components. As the Zernike moments are known to be noises sensitive before computing them we had used the same noise cut-off procedure as in binary classification, defined by Eq. 1. Then the components of Zernike moments were used train and evaluate the classifiers. We used two complementary methods of classifier evaluation -the 5-fold cross-validation and 30-fold evaluation on the randomly drawn (with stratification and replacement) 1/5 of the total data set. For a detailed discussion of applied classifiers and optimal values of respective hyperparameters see [19]. Here we summarize only the main results obtained with best performing classifiers that are shown in Table 1. The CV column in Table 1 contains the average classifier cross-validation To illustrate the overall performance across various trace classes we show in Fig.4 the confusion matrix obtained for the best performing single (non-ensemble) -SVC classifier. No surprise that the classifier is most confused to in recognizing worms vs. artefacts. It is also worth noting that the artefact recognition in the four-class scheme is worse than in the binary signal vs. artefact classification. This can be understood by inspecting Fig. 2 where we see that some signals (most likely worms) are sitting within a decision boundary so are treated as "false artefacts". To put them back to their true signal class the classifier needs to penetrate the artefact zone in the ( , ) plane but only at the cost of putting some artefacts in the false signal classes.

Summary and outlook
We have discussed the ANN based methods of the artefact rejection and hit classification applied to the data set containing candidate hits collected by the CREDO experiment. We have shown that the 99% accuracy of the CNN trigger allows for entirely automatic handling of the surge of artefacts resulting from a wide use of gamification in this citizen science initiative. Further analysis performed with a wide array of statistical and ANN based classifiers enables the selection of the hits with particular morphologies. The overall accuracy of the four class signal classification on the test data set is about 88% with the best recognition rate, reaching 98% for spots and the worst -of about 70% for worms. The classifiers are most likely to confuse worms and artefacts.