{'fraction_non_alphanumeric': 0.050021884574501345, 'fraction_numerical': 0.015756893640967922, 'mean_word_length': 4.142765273311897, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Finding the Shape of Lacunae of the Wave Equation Using Artificial Neural Networks

We apply a fully connected neural network to determine the shape of the lacunae in the solutions of the wave equation. Lacunae are the regions of quietness behind the trailing fronts of the propagating waves. The network is trained using a computer simulated data set containing a sufficiently large number of samples. The network is then shown to correctly reconstruct the shape of lacunae including the configurations when it is fully enclosed.


Introduction
Consider the inhomogeneous scalar wave (d'Alembert) equation in the threedimensional space (3D): subject to zero initial conditions, and with the source term that is compactly supported on a bounded (3+1)D domain ⊂ R 3 × [0, +∞). The solution to equation (1) is given by the Kirchhoff integral: The integration in (2) is performed in space over the ball of radius centered at x , but as is taken at retarded moments of time, this can be interpreted as integration in the (3+1)D space-time over the surface of a backward characteristic cone of equation (1)  not intersect with the support of the right-hand side . If there is no intersection, then (x , ) = 0, which implies, in particular, that the solution = (x , ) of equation (1) will have a lacuna (secondary lacuna in the sense of Petrowsky [1]): Mathematically, the lacuna Λ is the intersection of all forward characteristic cones (i.e., light cones of the future) of the wave equation (1) once the vertex of the cone sweeps the support of the right-hand side (x , ). From the standpoint of physics, Λ is the part of space-time where the waves generated by a compactly supported source have already passed and the solution has become zero again. The primary lacuna (as opposed to secondary lacuna (3)) is the part of space-time ahead of the propagating fronts where the waves have not reached yet.
The phenomenon of lacunae is inherently three-dimensional (more precisely, it pertains to spaces of odd dimension). The surface of the lacuna includes the trajectory of aft (trailing) fronts of the propagating waves. The existence of sharp aft fronts in odd-dimension spaces is known as the (strong) Huygens' principle, as opposed to the so-called wave diffusion, which takes place in spaces of even dimension [2,3].
The question of identifying the hyperbolic equations and systems that admit the diffusionless propagation of waves has been first formulated by Hadamard [4][5][6]. He, however, did not know any other examples besides the d'Alembert equation (1). The notion of lacunae was introduced and studied by Petrowsky in [1], where conditions for the coefficients of hyperbolic equations that guaranteed the existence of lacunae have been obtained (see also [3,Chapter VI]). Subsequent developments can be found in [7,8]. However, since work [1] no other constructive examples of either scalar equations or systems that satisfy the Huygens' principle have been found except for the wave equation (1) and its equivalents. Specifically, it was shown in [9] that in the standard (3 + 1)D space-time with Minkowski metric, the only scalar hyperbolic equation that has lacunae is the wave equation (1). The first examples of nontrivial diffusionless equations (i.e., irreducible to the wave equation) were constructed in [10][11][12], but the space must be R for odd 5. Examples of nontrivial diffusionless systems (as opposed to scalar equations) in the standard Minkowski (3 + 1)D space-time were presented in [13][14][15], as well as examples of nontrivial scalar Huygens' equations in a (3 + 1)D space-time equipped with a different metric (the plane wave metric that contains off-diagonal terms), see [14][15][16]. It was shown in [17] that the wave equation on the -dimensional sphere, where 3 is odd, satisfies the Huygens' principle; this spherical wave equation can be transformed to the Euclidean wave equation locally, but not globally.
While the examples of nontrivial diffusionless equations/systems built in [10][11][12][13][14][15][16] are primarily of a theoretical interest, the original wave equation (1) accounts for a variety of physically relevant (albeit sometimes simplified) models in acoustics, electromagnetism, elastodynamics, etc. Accordingly, understanding the shape of the lacunae (3) is of interest for the aforementioned application areas as lacunae represent the regions of "quietness" where the corresponding wave field is zero.
The objective of our work is to determine the shape of the lacunae in the solutions of the wave equation using the fully connected artificial neural networks. In the current paper, we adopt a simplified scenario to construct, test, and verify the proposed machine learning approach. Specifically, while the true phenomenon of lacunae is 3D and applies to solutions given by the Kirchhoff integral (2), hereafter we conduct the analysis and simulations in a one-dimensional (1D) setting. The domain of dependence for (x , ) determined by the Kirchhoff integral is the surface of the backward light cone. To mimic that in 1D, we consider the function = ( , ) and define its domain of dependence as the sum of two backward propagating rays: We do not need a full specification of for our subsequent considerations. We only need a sufficient condition for ( , ) to be equal to zero, which we take as where is a given bounded domain in (1+1)D space-time: ⊂ R × [0, +∞). Accordingly, the function is going to have a lacuna: The lacuna Λ 1 ( ) combines both the secondary and primary lacuna as per the discussion in Section 1. A purely secondary lacuna would be given by [cf.
We emphasize that the proposed 1D construct is not accurate on the substance. Its only purpose is to provide an inexpensive testing framework for the neural networks described in the paper. This construct is designed as a direct counterpart of the physical 3D setting and does not represent a true solution of the 1D wave equation: The solution of (7) subject to zero initial conditions is given by the d'Alembert integral [cf. the Kirchhoff integral (2)]: Unlike (4), the domain of dependence for the 1D solution (8) contains not only the two rays, but the entire in-between region as well: Therefore, the solution = ( , ) defined by (8) will not, generally speaking, have secondary lacunae as presented by (6). The paper is organized as follows. In Section 2, we provide a description of the numerical algorithm to build the data set. In Section 3, we introduce the neural networks (NN) and and discuss their training. In Section 4, we present the numerical results. Section 5 contains our conclusions and identifies directions for future work.

Construction of the Data Set
In this section, we describe the construction of the data set needed for the network training. To this end, we introduce a computational domain Ω := [ , ] × [0, ] and discretize it in space and time using a uniform spatial 1 , . . . , and temporal 1 , . . . , mesh so that where Δ = − −1 and Δ = −1 . We assume that the domain that defines the lacuna Λ 1 in (5) lies inside a subdomain ⊂ Ω, which, for the simplicity of implementation, is taken as We can then denote the nodes inside by where 1 and 1 are the smallest and values that are inside the the set and 1 + −1 and 1 + −1 are the largest and values that are inside the the set , see Figure 1.
The shape of the lacuna can then be determined by identifying the set of nodes, for which the characteristic lines L ( , ) emerging from the node ( , ) ∈ Ω, = 1, . . . , , = 1, . . . , (see formula (4)), pass through the domain . We therefore construct training data sets by implementing the following algorithm: The 1D case is special in the sense that while the dimension of the space is odd, the wave equation (7) is not Huygens' when driven by a source term. It may, however, demonstrate a Huygens' behavior with respect to the initial data [2].

Construction of the Neural Network and Training
In this paper, we use a fully connected feed forward neural network Θ : R × → R × to approximate the shape of the lacunae. To train Θ , we search for the parameter set Θ that minimizes the loss function: where · is a given norm. The fully connected neural network with hidden layers is the composition of functions with Θ ( ) = ( ), = + , = 1, 2, ..., + 1, where is the connection between layers −1 and , and is the activation for the layer . Here, = 0 corresponds to the input layer and = + 1 corresponds to the output layer, and we assume that there are nodes on each layer = 0, . . . , +1. For each connection between layers, ∈ R × −1 is the weight matrix and ∈ R is the bias vector. The parameters in the sets Θ are the element values for the weight matrix and bias vector , and Θ = ∪ +1 =1 Θ is the parameter set that we train the neural network to learn by minimizing the loss function (11). We can represent the architecture of a fully connected feed forward neural network as shown in Figure 3. For each = 1, 2, ..., + 1, the activation function is usually a nonlinear function that is applied to each element of its input vector. There are many activation functions that can be used, some of the most popular activation functions used include the rectified linear unit (ReLU), sigmoid functions, softmax, etc [18].
We train the neural network Θ with the following algorithm: a. The training set is used in the optimization algorithm to find the parameter set Θ that minimizes (11). b. The validation set is used to check that the neural network can generalize to data that are not in the training set.
2. Set hyperparameters: loss function, optimizer, learning rate, number of epochs, batch size, number of hidden layers, number of nodes per hidden layer, and activation functions.
a. loss function: The function that is to be minimized given the training data set, see (11). b. optimizer: The algorithm that is used to try and find the global minimum of the loss function. c. learning rate: The initial step size that the optimization algorithm takes. Depending on the optimization algorithm, the step sizes may change between steps. d. number of epochs: The number of times the optimization algorithm goes through the entire training data set. e. batch size: The number of samples from the training data that will propagate through the network for each update of the parameters. f. number of hidden layers: . g. number of nodes per each hidden layer: , = 1, . . . , . h. activation functions: , = 1, . . . , + 1. • Update the neural network parameters using one step of the optimization algorithm. c. With the current parameter set Θ ( ) , evaluate the output of Θ ( ) (see formula (12)) applied to all of the input values Φ ( ) , = 1, 2, ..., , in the validation set, and calculate the loss value (Θ ( ) ) defined in (11). If this loss value is smaller than at the end of every other epoch before it, set Θ = Θ ( ) .

Retrieve parameters:
Return Θ with the parameter set Θ obtained in step 3c.
Remark To apply the neural network to any input matrix Φ, one needs to reshape the matrix into a vector of size . One can then reshape the vector output to be a matrix of the size of Ψ, which is × .

Remark
The training set is usually about 80% of the total data set and the other 20% is the validation set.
Remark Finding good hyperparameters is often done by a running multiple trials of the training algorithm using different hyperparameters and identifying which one results in the best outcome. The specific hyperparameters we have used are presented in Section 4.

Numerical Results
In this section, we conduct a number of numerical experiments to demonstrate the performance of the machine learning approach to detect the lacunae. All of our models are built and trained using the open source library PyTorch [18].
To define the set ( ) in (9) Recall that ( ) is the number of sets on the right hand side of (9). As mentioned in Algorithm 1, this integer is randomly generated, and in our numerical examples it is an integer between 1 and 4. If ( ) > 1, then we may have a disconnected set  (13) we use is = 5. We generated = 10,000 data pairs for our training process, randomly splitting the data such that = 8,000 and = 2,000. For the hyperparameters in Algorithm 2 we use: Equipped with all the parameters and data, we train a neural network Θ such that for a given set represented by the matrix Φ as described in Algorithm 1, the neural network will produce where Ψ is the matrix representing the location of the lacunae Λ 1 ( ) and its compliment (Λ 1 ( )) .

Example 1: Case ( ) = 1
In the first example, we consider a particular case where ( ) = 1 for each = 1, . . . , , and = 10,000. After training the neural network, Θ , we apply it to a test set that is independent of the training and validation sets. For a given input matrix Φ, let Ψ denote the reference solution matrix with elements , ∈ {−1, 1}, = 1, . . . , , = 1, . . . , , determining whether or not the node ( , ) is in the lacuna, see (10). Then for the neural network solution, Ψ Θ (Φ) ∈ (−1, 1), we state that it correctly identifies that the node ( , ) is in the right set, We can then determine how accurate the neural network is by calculating = # of nodes correctly identified total # of nodes .
From 1,000 test cases, the neural network was able to identify which set each node belonged to with approximately 99.12% accuracy. Figure 4 shows an examples taken from the test set. The top left represents the values for the reference solutions Ψ , the top right represents the values of the neural network solution Ψ , the bottom left shows nodes in the set , and the bottom right represents the difference Ψ − Ψ . For the top graphs, the values range from [−1, 1] with −1 indicating nodes in Λ 1 ( ) with blue dots and 1 indicating nodes in Λ 1 ( ) with red dots. For the reference solution, the element values are either -1 or 1 so all the nodes are either dark blue or dark red respectively. The neural network has values between −1 and 1, thus the node colors in its graph might be different shades of red, blue, and white. On the plot for Ψ − Ψ , the white nodes indicates that the two solution are close to each other, the red nodes indicates that the Ψ is greater than Ψ and the blue nodes indicate Ψ is less than Ψ . Note that, the interior of the sets Λ 1 ( ) and Λ 1 ( ) for the neural network solution are clearly defined as very close to -1 or 1, but there is some uncertainty from the neural networks along the boundary. It is expected that the neural network would in general have more difficulty learning the edges of these sets.

Example 2: Case 1 ≤ ( ) ≤ 4
In this case, we generate our data choosing ( ) to be a random integer such that 1 ≤ ( ) ≤ 4 for each = 1, . . . , . Once trained, the neural network was able to predict which set, Λ 1 ( ) or (Λ 1 ( )) , each node belongs to with approximately 98.55% accuracy over 1,000 test cases where the accuracy is calculated as in (14). Figure 5 shows 2 examples (using the same layout as in Figure 4). The first example is taken from the test set, and in the second example we chose such that the lacuna (5) has a "pocket," i.e., a fully enclosed area. It is a part of the secondary lacuna (6).  Figure 4, the top two rows show an example from the test data set and the bottom two rows show a hand crafted example such that the lacunae has a 'pocket'.

Discussion
We have demonstrated that a fully connected neural network can accurately reconstruct the shape of the lacunae in an artificial one-dimensional setting introduced in Section 1. While we have trained our network to find the shape of a combined lacuna (5), we anticipate that having it identify only the secondary lacunae (6) would not present any additional issues. A challenging next step is to extend the proposed machine learning approach to a realistic three-dimensional setting where the secondary lacunae are defined according to (3) and account for the actual physics of the solutions to the wave equation (1).