Deep learning and k-means clustering in heterotic string vacua with line bundles
Hajime Otsuka Kenta Takemoto
hep-th hep-ph




 



0.2
KEK-TH-2200 

WU-HEP-20-4




Deep learning and k-means clustering
in heterotic string vacua with line bundles

    Hajime Otsuka^a[
E-mail address: hotsuka@post.kek.jp
]
 and Kenta Takemoto^b[
E-mail address: phys-tm.3000@asagi.waseda.jp
]

^aKEK Theory Center, Institute of Particle and Nuclear Studies, KEK,

1-1 Oho, Tsukuba, Ibaraki 305-0801, Japan

^bDepartment of Physics, Waseda University, Tokyo 169-8555, Japan
    December 30, 2023
====================================================================================================================================================================================================================================================================================================================









empty
page-1






§ INTRODUCTION


The deep learning is an attractive method not only for 
image recognition, but also for applications to string theory as well 
as particle physics. 
So far, neural networks have been applied to explore the 
vacuum structure of string theory <cit.>, 
for instance, 
a conjecture for the gauge group rank in F-theory compactifications <cit.>, 
identification of fertile islands in the toroidal orbifold landscape <cit.>, 
the prediction of the Hodge numbers of Calabi-Yau (CY)  manifolds <cit.>, exploring Type IIA compactifications with intersecting D6-branes <cit.>, landscape of Type IIB flux vacua <cit.> and E_8× E_8 heterotic line bundle models <cit.>, and finding the numerical 
CY metric <cit.>. 
(For more details, see, Ref. <cit.>.) 
Utilizing machine learning techniques for exploring 
the Standard Model (SM) vacua from string theory will give new 
insights into the string landscape. 

Among the huge number of CY manifolds, there 
exists a restricted class of non-simply connected 
CYs where Wilson lines can be introduced to obtain 
the SM gauge group. 
To search for the Minimal Supersymmetric SM (MSSM) on a vast 
class of CYs, a hypercharge flux breaking scenario 
is proposed in Refs. <cit.>. 
Such a direct flux breaking scenario does not require an 
existence of Wilson lines and is applicable to a vast class 
of CYs. 
However, in the usual random scan in the string landscape, it is difficult to reveal the  origin of the MSSM-like models from a huge number of compactification parameters. To resolve this issue, we apply the machine learning technique for an exhaustive search of 
four-dimensional (4D) string models based on the direct flux breaking scenario.  
It opens up a new possibility of searching for the three-generation SM. 
Especially, we follow the systematic approach proposed in Ref. <cit.> 
on the basis of SO(32) heterotic string theory with line bundles. 
The direct flux breaking scenario in SO(32) heterotic string theory is motivated by 
its S- and T-dual intersecting D6-brane models in Type IIA string theory, 
where several stacks of D-branes directly lead to the MSSM-like 
gauge group. 

The purpose of this paper is to reveal the nature of SM vacua in SO(32) heterotic line bundle models by employing the neural network technique, in particular the origin of three generations of quarks and leptons. 
So far, the machine learning techniques have been utilized to reproduce known physical quantities, like topological 
data and numerical metric of CYs, but our approach attempts to not only reproduce the known physical data, but also find a characteristic feature of the CY compactifications. 

In this paper, we adopt a deep autoencoder <cit.> to find the parameter region leading to the 
SM-like spectra on CY threefolds, in particular, Complete Intersection CY (CICY) <cit.>.[For CICY lists, see, <cit.>.] The autoencoder is useful to reduce the higher-dimensional parameter space to the 2D one by extracting characteristic features of the data, although the neutral network itself does not have the knowledge of SM spectra. To classify the characteristic features from the autoencoder, we deal with the k-means++ clustering <cit.>. 
After performing the autoencoder and the k-means++ clustering to the vacua, we find that three-generation models are clustered in particular islands, in a similar to 
the analysis in the toroidal orbifold models <cit.> and we call the cluster 
with densest three-generation models “3-generation island". 
By introducing Kullback–Leibler (KL) divergence <cit.>, we also find that the 3-generation island is strongly correlated with the topological data of CY, in particular, 
the second Chern number of CY threefolds.
Our approach enables us to capture the nature of not only the known toroidal orbifold landscape but also a large class of CY compactifications. 

This paper is organized as follows. 
In Section <ref>, we briefly review the phenomenological and theoretical 
constraints in SO(32) heterotic line bundle models to implement the deep autoencoder. 
In Section <ref>, we show the algorithms of the autoencoder and k-means++ clustering utilized in 
the dataset of heterotic line bundle models.  As discussed in Section <ref>, we find that n-generation models are clustered in the 2D space derived by the autoencoder. 
Especially, we focus on the 3-generation island 
and extract its phenomenological consequence. 
Section 5 is devoted to the conclusions and discussions. 



§ LINE BUNDLE MODELS


Before applying the deep autoencoder to the 
dataset of heterotic line bundle models, we briefly review the heterotic 
line bundle models developed in Refs. <cit.> with an 
emphasis on the phenomenological and theoretical constraints. 



 §.§ Consistency conditions



We start from the low-energy effective action of SO(32) heterotic string theory on smooth CY manifolds with 
multiple line bundles.[The following discussion 
is applicable to the E_8× E_8 heterotic string theory as well.]
The total internal gauge bundle consists of the 
multiple internal line bundles L_a with structure group U(1), namely

    W =⊕_a L_a,

where U(1) is supposed to descend from U(N) ⊂ SO(2N) ⊂ SO(32) and the concrete embedding into SO(32) is shown later. 
In contrast to the standard embedding scenario, such a 
non-standard embedding has a phenomenological and theoretical rich structure. 
For instance, SO(32) gauge group is broken down to the 4D gauge group G rather than E_6 because of the non-vanishing background field values of the SO(32) gauge field strength. 
The introduction of the internal line bundles each with structure group U(1) corresponds to an existence of  internal U(1)_a gauge fluxes F_a,

    tr (F_a) = 2π∑_i=1^h^1,1 tr(T_a) m_a^i w_i,

where the m_a^i are flux quanta in the basis of Kähler form w_i, i=1,2,⋯, h^1,1 with h^1,1 being the Hodge number of CY M. Here T_a are U(1)_a generators and “tr" represents for the trace in the fundamental representation. 
We remark that the internal gauge fluxes are now turned on only H^1,1( M), otherwise the 
supersymmetry is broken by the F-term in the 4D effective action. Even when the fluxes 
are restricted on H^1,1( M), it is required to check the D-term condition (zero-slope poly-stability condition) for each U(1)_a. Since it depends on the structure of Kähler cone and the D-term conditions give rise to Diophantine equations, we do not take into account the D-term condition in the neural network. It is possible to check the D-term conditions for each model after obtaining the three-generation models.[For the discussion of Diophantine equations in 
Type IIA intersecting D6-brane models, see, Ref. <cit.>. We leave the implementation of such Diophantine equations to the neural network for future work.] 
As a consequence of the above gauge fluxes, we obtain 
not only the SM-like gauge group, but also the chiral fermions. 
Before going to discuss the phenomenological models, we enumerate the theoretical constraints for gauge fluxes. 
We will take into account these constraints when we implement the line bundle models to the deep neural networks. 

    
  * Tadpole cancellation condition
    
    Internal gauge fluxes cause the gauge and gravitational anomalies in the 4D effective action. To cancel these anomalies, we require the cancellation of the tadpole for the Neveu-Schwarz (NS) sector,
    
    ch_2 (W) +c_2(T M) =( 1/2∑_a,i,j trT_a^2m_a^i m_a^j d_ijk +c_2,k)ŵ^k =N_k ŵ^k,

    where we expand the second Chern character of W and second Chern class of the tangent bundle of CY M in the basis of ŵ^k ∈ H^2,2( M) satisfying
    
    ∫_ Mŵ^k∧ w_i =δ^k_i.

    We denote the triple intersection number of CY by d_ijk=∫_ Mw_i∧ w_j ∧ w_k and we introduce the NS5-branes wrapping holomorphic two-cycles. To ensure the stability of our system, we prohibit an existence of anti NS5-branes, namely
    
    N_k =1/2∑_a,i,j trT_a^2m_a^i m_a^j d_ijk +c_2,k≥ 0.

    
  * K-theory condition
    
    Next, we require the existence of spinor bundles on CY manifolds, corresponding to the requirement of the trivial Stiefel-Whitney class,
    
    c_1 (W) ∈ H^1,1( M, 2ℤ) ↔ trT_a m_a^i ≡ 0   ( mod 2)

    for i=1,2,⋯, h^1,1. This condition is related to K-theory condition developed in the S-dual Type I string theory <cit.>.
    In some cases, we replace (<ref>) with
    
    T_a m_a^i = 0

    for simplicity because of the limitation of the computing power.
    
  * U(1)_Y masslessness conditions
    
    Non-trivial gauge background induces the 4D Stückelberg couplings between string axions associated with Kalb-Ramond B-field and U(1) gauge bosons  through Green-Schwarz terms. Indeed, it causes the mass terms for some anomalous U(1)s <cit.>, 
    
    M_ai =
            {[                                                          2π tr(T_a^2)m_a^i,                                                            (i=1,2,⋯, h^1,1); 1/6 tr(T_aT_bT_cT_d)∑_jkld_jklm_b^jm_c^km_d^l +1/24 tr(T_a^2)∑_jm_a^jc_2,j,                                                                       (i=0) ].
            .

    Hence, when U(1)_Y gauge boson is defined as a linear combination of multiple U(1)s,
    
    U(1)_Y =∑_a f_a U(1)_a,

    the coefficient f_a is constrained to prohibit the couplings 
    between the string axions and U(1)_Y gauge boson,
    
    ∑_a M_aif_a=0,

    with i=0,1,⋯, h^1,1, 
    otherwise U(1)_Y gauge boson will become massive. 
     
    
  * Generations of quarks and leptons
    
    Finally, we show the number of chiral zero-modes on the gauge background. Internal gauge fluxes as well as the curvature lead to the decomposition of the adjoint representation of SO(32),
    
    496 →⊕_p (R_p, C_p),

    where R_p and C_p denote the representations of the 4D gauge group G and W, respectively. 
    From the  Hirzebruch-Riemann-Roch theorem, the net number of chiral zero-modes with U(1)_a charge Y_a is counted by the index
    
    χ(L)    = 
            ∫_ M[ ch_3(L) +1/12c_2(T M)∧ c_1(L) ]
            
       = 1/6∑_i,j,k,a,b,cd_ijkm_a^im_b^jm_c^k Y_aY_bY_c +1/12∑_i,ac_2,im_a^iY_a,

    with L=⊗_aL_a^Y_a. When we include the contribution from the freely-acting discrete symmetry group of CY (Γ), the index is divided by its order |Γ|, namely
    
    χ(L) = -n|Γ|,

    where n represents the net number of chiral zero-modes regarded as the generations of the quarks, leptons/Higgs as well as the exotic particles.[For the freely-acting discrete symmetry group of CICY, see, <cit.>.] To prohibit chiral exotics in the low-energy effective action, we impose χ_ exotic=0 for exotic particles. 
    To clarify the difference between 3-generation models and n-generation models in the autoencoder, we search for the n-generation models as discussed in detail in the next section. 
    



 §.§ n-generation models


For definiteness, we focus on the decomposition of SO(32) 
by the existence of multiple line bundles following Ref. <cit.>,

    SO(32)   → SO(16)× SO(16)_H
        
       → SO(6)× SO(4)× SO(2)^3× SO(16)_H
        
       → SU(3)_C× SU(2)_L×Π_a=1^5 U(1)_a× SO(16)_H,

where the internal gauge fluxes are inserted in the following U(1)_a generators,

    T_1    = diag(1,1,1,0,0,0,0,0,0,⋯, 0),
        
    
        T_2    = diag(0,0,0,1,1,0,0,0,0,⋯, 0),
        
    
        T_3    = diag(0,0,0,0,0,1,0,0,0,⋯, 0),
        
    
        T_4    = diag(0,0,0,0,0,0,1,0,0,⋯, 0),
        
    
        T_5    = diag(0,0,0,0,0,0,0,1,0,⋯, 0),

in the basis of Cartan directions of SO(32) H_r with r=1,2,⋯,16 and SO(32) roots are chosen as (± 1, ± 1, 0,⋯, 0) under H_r. Here, the underline represents the possible permutation. 
Note that U(1)_Y is a linear combination of the above five U(1)_a, U(1)_Y=∑_a=1^5 f_a U(1)_a and SO(16)_H is regarded as a hidden gauge group. 
According to the gauge decomposition SO(32)→ SO(16)× SO(16)_H, the adjoint representation of SO(32) decomposes as

    496→ (120,1)⊕ (16, 16_H)⊕ (1, 120_H),

from which there exist the exotic particles in the (16, 16_H)⊕ (1, 120_H),

    (3, 1,16_H)_1,0,0,0,0⊕ (1, 2,16_H)_0,1,0,0,0⊕ (1, 1,16_H)_0,0,1,0,0⊕
       (1, 1,16_H)_0,0,0,1,0⊕ (1, 1,16_H)_0,0,0,0,1⊕ (1, 1,120_H)_0,0, 0,0,0.

The subscript indices correspond to the U(1)_a charges Y_a. 
On the other hand, the adjoint representation of SO(16) includes the candidates of SM particles:

    𝔮 :     (3, 2)_-1,± 1,0,0,0
    ,
    𝔩 :     (1, 2)_0,± 1, ± 1,0,0⊕ (1, 2)_0,± 1, ∓ 1,0,0⊕ (1, 2)_0,± 1, 0, ± 1,0⊕ (1, 2)_0,± 1, 0, ∓ 1,0⊕
        (1, 2)_0,± 1,0, 0, ± 1⊕ (1, 2)_0,± 1,0, 0, ∓ 1,
    𝔲^c :     (3̅, 1)_-2,0, 0,0,0⊕ (3̅, 1)_1,0,± 1,0,0⊕ (3̅, 1)_1,0,0,± 1,0⊕ (3̅, 1)_1,0,0,0,± 1
    ,
    𝔢^c:    (1, 1)_0,± 2,0,0,0⊕ (1, 1)_0,0,± 1, ± 1,0⊕ (1, 1)_0,0,± 1,∓ 1,0⊕ (1, 1)_0,0,± 1, 0, ± 1⊕
        (1, 1)_0,0,± 1,0,∓ 1⊕ (1, 1)_0,0,0,± 1, ± 1⊕ (1, 1)_0,0, 0,± 1,∓ 1
    .


Then, we define the quarks and lepton/Higgs in {𝔮,𝔩,𝔲^c,𝔢^c} 
such that they have a proper hypercharge, 

    Q    = { q_x∈𝔮| ∑_a f_a Y_a(q_x)=1/6},
    
    
    L    = { l_x∈𝔩| ∑_a f_a Y_a(l_x)=-1/2},
    
    
    U^c    = { u_x^c∈𝔲^c | ∑_a f_a Y_a(u_x^c)=-2/3},
    
    
    D^c    = { d_x^c∈𝔲^c | ∑_a f_a Y_a(d_x^c)=1/3},
    
    
    E^c    = { e_x^c∈𝔢^c | ∑_a f_a Y_a(e_x^c)=1},

where it is noted that Y_a(ϕ) stands for the U(1)_a 
charge of the field ϕ. 
When the particles in {𝔮,𝔩,𝔲^c,𝔢^c} 
satisfy these conditions, they are identified with the quarks and leptons/Higgs, otherwise the others do not belong to the spectra in the SM, regarded as the exotic particles. From the view point of the representations of the SM gauge group, we cannot distinguish between Higgsino fields and the charged leptons, but it is distinguishable when we clarify the SO(32) gauge invariant Yukawa couplings among elementary particles, such as QH_uU^c,QH_dD^c,LH_dE^c. 

In the neural network implemented in the next section, 
we impose the following phenomenological constraints in 
addition to the consistency conditions in Section <ref>:[Note that the hypercharge masslessness condition (<ref>) is simplified as tr(T_a^2)m_a^if_a=0 in the gauge decomposition (<ref>).]

    n_Q=n_L=n_U^c=n_D^c=n_E^c=n,     
    n_ϕ=0     (∀ϕ∈ exotics),

with

    n_Q≡∑_𝔮∈ Qn_𝔮,   n_L≡∑_𝔩∈ Ln_𝔩,   n_U^c≡∑_𝔲^c∈ U^cn_𝔲^c,   n_D^c≡∑_𝔲^c∈ D^cn_𝔲^c,   n_E^c≡∑_𝔢^c∈ E^cn_𝔢^c,

where each n_∗ is evaluated by employing Eq. (<ref>).



§ CLASSIFICATION METHODS FOR SO(32) LINE BUNDLE MODELS



In this section, we show the detailed method to apply the machine learning 
technique to the heterotic line bundle models satisfying the several 
conditions discussed in the previous section. Especially, we restrict ourselves to CICY threefolds with Hodge number h^1,1≤ 5. 
There exist 5, 36, 155, 425 and 856 CICY threefolds with h^1,1=1,2,3,4 and 5 respectively. 
Before going to the detailed description, we outline the processing flow given in the following four key steps:

    
  * Make dataset of n-generation models on the large number of CICYs, satisfying the conditions in Section <ref>.
    
  * Reduce the dimension of input parameters to the 2D charts by the autoencoder.
    
  * Classify the model data based on the results of dimension reduction using k-means++ algorithm. Then, calculate the percentage of 3-generation models for each cluster in the 2D space and decide “3-generation island".
    
  * Find the difference between the 3-generation island and other region. This corresponds to the feature of 3-generation model.


In the following subsections, we give a detailed description of each step. 



 §.§ Collect data


The first step of our method is to obtain a dataset of line bundle models 
satisfying constraints discussed in Section <ref>. It is known that solving the constraints is mathematically difficult because they have many integer variables in the equations 
called Diophantine equations. It was proved that there is no general method to solve 
this equation even though all of the constraints are polynomial<cit.>. 
Then we adopt a brute force approach as detailed in alg:brute force search in which we employ the following simplification <cit.>. 
Since there is no summation over i in K-theory condition (<ref>) and the 
hypercharge masslessness condition (<ref>), both are rewritten as

    m_α^i = ∑_A=3,4,5 K_α Am_A^i   (α=1,2),

with

    K_1A(T_a,f_a)    = (T_2) (T_A^2)f_A- (T_A)(T_2^2)f_2/(T_1)(T_2^2)f_2- (T_2) (T_1^2)f_1,
    
        K_2A(T_a,f_a)    = (T_1) (T_A^2)f_A- (T_A)(T_1^2)f_1/(T_2)(T_1^2)f_1- (T_1) (T_2^2)f_2.

When the K-theory condition is given by (<ref>), only the hypercharge masslessness condition boils down to

    m_1^i = ∑_A=2,3,4,5 K'_A(T_a,f_a)m_A^i,

with

    K'_A(T_a,f_a)   =- (T_A^2)f_A/ (T_1^2) f_1.




We carry out the above brute force approach for three times named as search (I), (II) and (III) with different parameter region shown in Table <ref>. The obtained number of n-generation models for each search is summarized in Table. <ref>. When h^1,1=2 in the search (I) and (II), it is difficult to realize the n-generation models. Then, we concentrate on the other 7 patterns in the  search (I)-(III).






 §.§ Autoencoder

After collecting the data of n-generation line bundle models, 
we perform the autoencoder known as a kind of multi-layer perceptrons (MLP). 
The advantage of the autoencoder is to reduce the higher-dimensional parameter 
space of the input data to the compressed data in the 2D charts and at the same time, to extract characteristic features of the data without giving any information how to extract the data. 
The fundamental component of the MLP is called a perceptron, which transforms a N_0-dimensional vector x_0 into a number x_1

    x_1=h(w⃗'·x⃗+b'),

where h is a generally non-linear function typically chosen as a sigmoid function or ReLU function, w⃗' is a N_1-dimensional vector called weight and the number b' represents a bias. A layer consists of multiple perceptrons. When the layer consists of N_1 perceptrons, they transform N_0-dimensional vector into N_1-dimensional vector, and weight and bias become a N_0× N_1 matrix and N_1 dimensional vector respectively.

Let us suppose that the input data is the N_0-dimensional vector and the output is N_M-dimensional vector. Then, the n-th layer in MLP has  N_n-1-dimensional input vector x⃗_n-1=(x_n-1,1,⋯,x_n-1,N_n-1) and N_n-dimensional output vector x⃗_n=(x_n,1,⋯,x_n,N_n). These two are related by

    x_n,i=h_n(w^n_ijx_n-1,j+b^n_i),

where the weight w^n and the bias b^n are described by N_n× N_n-1 matrix 
and N_n components, respectively. 
In this way, MLP is constructed by connecting M layers in series, as drawn in AE. 
In the context of MLP, learning corresponds to tune the weights and biases to minimize an error function E(x⃗_M,y⃗) representing the difference between training data y⃗ and outputs of MLP x⃗_M.

The autoencoder consists of the MLP with N_0>N_1>⋯ > N_b and N_b < N_b+1 < ⋯ < N_M=N_0 (b=(M-1)/2) as shown in AE, in which the first and latter half are called encoder and  decoder, respectively. 
Here, we denote the training data by x⃗_0 and design x⃗_0 such that 
the outputs resemble the inputs as closely as possible.
After learning, x⃗_b has lower dimension than x⃗_0 but it is possible to construct x⃗_M=x⃗_0. It indicates that all information (features) of inputs is compressed into the outputs of b-th layer.



The application of the autoencoder to our system is described as follows. 
We take the input vector as (|Γ|, d_ijk, c_2,i, m_a^i) deciding the line bundle models. When h^1,1≤ 5, dimension of this vector is not more than 161 dimension, hence we take N_0=N_M=161. Note that the meaningless parameters such as d_145 for CICY threefolds with h^1,1=3 are filled by 0 and this procedure does not affect any constraints we consider. 
In addition, we take N_b=2 to compare our results with the heterotic orbifold 
results <cit.> and to 
visualize the result easily. 
After trial and error, we arrive at the expression for 7 layers in the encoder and decoder with the following dimensions,

    N_n=(161,80.40,20.10,5,3,2,3,5,10,20,40,80,161).

The activation functions are chosen as sigmoid functions for h_1,⋯,h_b and 
identical maps for h_0,h_b+1,⋯,h_M, respectively and the error function 
is given by

    E(x⃗_0,x⃗_M)=|x⃗_M-x⃗_0|^2.

The learning method is followed by Adam-Optimizer in TensorFlow<cit.>. To avoid becoming trapped in a local minima of the error function, we first decompose the autoencoder into 
partial autoencoders with three-layer (N_q-1,N_q,N_M+1-q) (q=1,⋯,7) and after that the whole autoencoder is learned to minimize the error function.
The learning is repeated (40000, 20000, 20000, 20000, 18000, 16000, 14000) times for each partial autoencoder and 20000 times for the whole autoencoder. 
Since the two-dimensional scatter plots of the bottleneck layer have a 
cluster structure as demonstrated later, we apply the clustering method to 
the result of autoencoders.




 §.§ K-means++ clustering

To classify the compressed information in the bottleneck layer, 
we adopt the famous k-means++ method which has advantages that the algorithm itself is simple and computational cost is not significant. 
We employ KMeans class in scikit-learn for k-means++ clustering<cit.>. 

K-means++ classifies a given data 𝒟 with distance d(·,·) into N_ cl clusters as explained in alg:k-means++. In our case, 𝒟 
corresponds to the encoded vectors x⃗_b and we take the distance as Euclidean  norm.



To decide an appropriate N_ cl, we employ so called elbow method explained in alg:elbow method from which a critical value of N_ cl (N_ cl^*) is estimated. Then, we tune the number of clusters around N_ cl^* by eye-estimation in order to extract the 3-generation structure significantly.







 §.§ Statistical analysis

To find the factor of differences between the 3-generation island and other region, we introduce the KL divergence KL(ρ_1,ρ_2) which represents the distance between two distributions ρ_1 and ρ_2. It is defined by

    KL(ρ_1,ρ_2)=∑_m ρ_1(m)logρ_1(m)/ρ_2(m),

where ρ(m) denotes a probability of taking m under ρ. 
Note that KL(ρ_1,ρ_2)=0 under ρ_1=ρ_2.

In our case, ρ_1 and ρ_2 stand for the distributions of an input parameter X∈(d_ijk,m_a^i,c_2,i,|Γ|) of the 3-generation island and all region, respectively. 
In the following, we define KL(X)def= KL(ρ_1,ρ_2). 
Note that X with small KL(X) does not contribute to the identification of the 3-generation island, 
whereas X with large KL divergence plays a crucial role in distinguishing between the 3-generation island and other region.




§ RESULTS



In this section, we summarize the results by implementing the autoencoder and k-means++ clustering in SO(32) heterotic line bundle models. 
After discussing the search (I) in detail in Section <ref>, 
we show the other searches in Section <ref>. 
Finally, we count the number of Higgs pairs which are vector-like under 
the SM gauge group, but chiral with respect to other extra U(1)s 
by checking the Yukawa couplings of quarks and leptons. 





 §.§ Search (I) with h^1,1=3 and N_ cl=26



First, we discuss the case of search (I) with h^1,1=3 and N_ cl=26 as a concrete example. 
clustering shows the result of k-means++ clustering of x⃗_b at the bottleneck layer  and black circles correspond to centroids of each colored cluster.
3genisland represents the ratio between n=3 models and n≠0-generation models in each cluster. The density of 3-generation models in the deep blue region is higher than the other region. The cluster located around (0.54,0.47) in 3genisland has the highest ratio (≅ 19.15%) among total 26 clusters and then this cluster is identified with the 3-generation island. 
Recalling that this 3-generation island contains only 2.08% of the whole line bundle models we consider, it is easy to find the 3-generation models by focusing on this fertile island. Such a phenomena is also discussed in the heterotic ℤ_6-II orbifold landscape <cit.>. 
It is remarkable that 19 clusters in all 26 clusters do not have 3-generation model. 
In this respect, we argue that our clustering procedure extracts features of n=3 models.


In Table <ref>, we list the KL(X) for all X where X=d_ijk are shown only i≤ j≤ k because of the permutation symmetry of the indices. As mentioned above, X with large KL(X) captures an 
information of the 3-generation island. We find that 3-generation models have a strong correlation with the topological data of CY rather than flux parameters. Especially,  characteristic values of c_2,i, in particular 36 or 54, are selected as shown in Figs. <ref>, <ref> and <ref>. We conclude that 3-generation models prefer c_2,i∈18ℤ, although there is no bias in  c_2,i we employed. Small KL(d_111) in Table <ref> indicates that almost all of CICYs with h^1,1=3 leading to n-generation models have the triple self-intersection number d_111=0. Other d_ijk are typically 0, 3 or 9. 
Such specific values also represent a characteristic feature in the 3-generation island due to their large KLs, but they take different values for different h^1,1 and/or searches.







 §.§ Other cases



We perform a similar analysis for other searches. In the case of search (III) with h^1,1=2, 
there are two islands with the same percentage of n=3 models and we define both of two as the 3-generation island. 


From Table <ref> summarizing all our searches, 
particular values of the second Chern numbers of CICYs 
are favored in the class of 3-generation models, although there is no bias in the topological data of CICYs we employed. For the cases of c_2,i=24, their KL divergences are relatively smaller than others, indicating that these cases are not important for analysis of 3-generation models. 
For instance, in search (I) with h^1,1=4, KL(c_2,1)=0.026 and KL(c_2,2)=0.231 are suppressed compared 
with KL(c_2,3)=0.558 and KL(c_2,4)=1.363. 
From the above discussion, favored second Chern numbers in the 3-generation island are provided by c_2,i∈18ℤ. 
Then we conclude that 3-generation models have a strong correlation with c_2,i∈ 18ℤ.



Here we comment on whether the 3-generation 

island is realized by the decoder described in (at most) 161-dimensional parameter spaces. 
The region obtained by feeding points in the 3-generation island to the decoder is a two-dimensional plane embedded in the 161-dimensional parameter spaces because our activation functions of decoder are the identical map,
although the relations of parameters in the 161-dimensional spaces are non-linear. 
Then the decoder is not competent for specifying the 3-generation island in the 161-dimensional spaces.

Let us also comment on the geometrical interpretation of this second Chern number of CICY. 
The instanton number of the tangent bundle of CY T M with the curvature two-form R is given by

    N_ ins^T M=-1/8π^2∫_ M R∧ *R=-1/8π^2∫_ M R∧ R∧Ω,

where we employ the self-dual condition of the curvature two-form

    *R=Ω∧ R

with Ω satisfying the condition dΩ =0<cit.>.  
Recall that Kähler form of CY manifolds is a closed form, 
it is possible to take Ω=w_i, namely

    N_ ins,i^T M= -1/8π^2∫_ M R∧ R∧ w_i=∫_ M c_2(T M)∧ w_i = c_2,i


It is interesting to ask why these specific instanton numbers are favored in a class of 
3-generation models. We hope to report on this relationship in the future. 



 §.§ Number of generations of Higgs



In this section, we count the number of generations of Higgs (Higgsino) by implementing the analysis of 
the previous subsection. 
Note that we only take into account the Higgs pairs which are vector-like under the SM gauge group, but chiral with respect to other extra U(1)s. 
For definiteness, we restrict ourselves to n_ϕ≥ 0 (∀ϕ∈𝔮∪𝔩∪𝔲^c∪𝔢^c) cases and 
define the Higgs doublets from 𝔩 by checking the Yukawa couplings of quarks and leptons. 
In the obtained models, the generation number of up-type Higgs n_H_u is same with 
that of down-type Higgs n_H_d. The condition n_ϕ≥ 0 is so tight that only 3 cases (search (I), (II) and (III) with h^1,1=3) are able to be analyzed in our numerical analysis. 
Figs. <ref>, <ref> and <ref> show histograms of 
the number of Higgs pairs n_H in the 3-generation island. 
From these figures, we find that n_H listed in Table <ref> 
(except for n_H=0) is favored in the 3-generation island. 
Although there are not so many models in our limited search, it turns out that 1-pair Higgs models 
is disfavored. 
We expect that an existence of a large number of Higgs pairs is a generic property in heterotic string vacua. 






§ CONCLUSIONS AND DISCUSSIONS



In this paper, we applied the deep autoencoders 
and k-means++ clustering to the string landscape 
by employing the topological data of 
CY threefolds and internal gauge fluxes as input data. 
In particular, we investigated SO(32) heterotic string vacua on smooth CICY threefolds with line bundles, taking into account the phenomenological and theoretical consistency conditions. 
After training the autoencoder on at most 161 input data, 
satisfying the consistency conditions as well as reproducing the n-generations of quarks and leptons without chiral exotics, 
we draw a 2D chart of the landscape of n-generation models by 
utilizing the k-means++ algorithm. 
It turned out that 3-generation models cluster in particular islands in the 2D chart 
and we called the cluster with densest three-generation models “3-generation island". 
Such a structure has also been pointed out in the Mini-Landscape of heterotic ℤ_6-II orbifold models <cit.>.[A decision tree employed in <cit.> is applicable to our analysis which is valid to approximate the classification of models, but it is beyond our purpose.] 
We expect that the presence of 3-generation island will be a universal phenomena in the string landscape including E_8× E_8 heterotic line bundle models as well as intersecting/magnetized D-brane models. 

By estimating the KL divergences of model parameters, we find that the clustered 3-generation island has a strong correlation with the topological 
data of CY threefolds, in particular, second Chern class of CY threefolds, namely c_2(T M) ∈ H^2,2( M, 18ℤ), although there is no bias in the second Chern class of CICYs we employed. 
It indicates that second Chern numbers of CYs provide a guideline to obtain 3-generation MSSM-like models. We leave to reveal the underlying reason for future work. 
It is interesting to apply our analysis to other regions of the string landscape and check the values of second Chern number of CYs for 3-generation models. 

We also counted the number of Higgs pairs which are vector-like under the SM gauge group, but chiral with respect to other extra U(1)s. Our results show that the 3-generation island contains a large number of Higgs pairs. 
It will motivate us to study the phenomenology of multi-Higgs models discussed in the bottom-up approach. 

Finally, we comment on possible applications of our method to other regions of the string landscape. 
It is straightforward to extend our analysis to E_8× E_8 heterotic line bundle models by changing the gauge group decomposition.[See for the recent discussion of E_8× E_8 heterotic line bundle models using deep reinforcement learning, Ref. <cit.>.] 
For the D-brane models, it is enough to add the input data such as the position of D-branes, an amount of magnetic flux (in Type IIB magnetized D-brane models) 
and intersection angles (in Type IIA intersecting D-brane models), taking into account the proper tadpole cancellation conditions. We hope to report on this interesting work in the future. 




 §.§ Acknowledgements


We would like to thank H. Abe, S. H. Lim and A. Otsuka for useful discussions. 
H. O. was supported in part by JSPS KAKENHI Grant Numbers JP19J00664 and JP20K14477.


99
=-2pt

He:2017aed
  Y. H. He,
  
  arXiv:1706.02714 [hep-th].

He:2017set
  Y. H. He,
  
  Phys. Lett. B 774 (2017) 564.

Krefl:2017yox
  D. Krefl and R. K. Seong,
  
  Phys. Rev. D 96 (2017) no.6,  066014
  [arXiv:1706.03346 [hep-th]].

Ruehle:2017mzq
  F. Ruehle,
  
  JHEP 1708 (2017) 038
  [arXiv:1706.07024 [hep-th]].

Carifio:2017bov
  J. Carifio, J. Halverson, D. Krioukov and B. D. Nelson,
  
  JHEP 1709 (2017) 157
  [arXiv:1707.00655 [hep-th]].

Mutter:2018sra
  A. Mütter, E. Parr and P. K. S. Vaudrevange,
  
  Nucl. Phys. B 940 (2019) 113
  [arXiv:1811.05993 [hep-th]].

Bull:2019cij
  K. Bull, Y. H. He, V. Jejjala and C. Mishra,
  
  Phys. Lett. B 795 (2019) 700
  [arXiv:1903.03113 [hep-th]].

Halverson:2019tkf
  J. Halverson, B. Nelson and F. Ruehle,
  
  JHEP 1906 (2019) 003
  [arXiv:1903.11616 [hep-th]].
  
Cole:2019enn
  A. Cole, A. Schachner and G. Shiu,
  
  JHEP 1911 (2019) 045
  [arXiv:1907.10072 [hep-th]].



Larfors:2020ugo
  M. Larfors and R. Schneider,
  
  arXiv:2003.04817 [hep-th].  


Ashmore:2019wzb
  A. Ashmore, Y. H. He and B. A. Ovrut,
  
  arXiv:1910.08605 [hep-th].
  
  

Ruehle:2020jrk
  F. Ruehle,
  
  Phys. Rept.  839 (2020) 1.

Beasley:2008kw
  C. Beasley, J. J. Heckman and C. Vafa,
  
  JHEP 0901 (2009) 059
  [arXiv:0806.0102 [hep-th]].
  
Donagi:2008kj
  R. Donagi and M. Wijnholt,
  
  Adv. Theor. Math. Phys.  15 (2011) no.6,  1523
  [arXiv:0808.2223 [hep-th]].  

Blumenhagen:2005ga
  R. Blumenhagen, G. Honecker and T. Weigand,
  
  JHEP 0506 (2005) 020
  [hep-th/0504232].

Blumenhagen:2005pm
  R. Blumenhagen, G. Honecker and T. Weigand,
  
  JHEP 0508 (2005) 009
  [hep-th/0507041].

Blumenhagen:2006ux
  R. Blumenhagen, S. Moster and T. Weigand,
  
  Nucl. Phys. B 751 (2006) 186
  [hep-th/0603015].

Blumenhagen:2006wj
  R. Blumenhagen, S. Moster, R. Reinbacher and T. Weigand,
  
  JHEP 0705 (2007) 041
  [hep-th/0612039].

Otsuka:2018rki
  H. Otsuka and K. Takemoto,
  
  JHEP 1811 (2018) 034
  [arXiv:1809.00838 [hep-th]].
  
Hinton
G. E. Hinton and R. R. Salakhutdinov, Science 313 (2006), no. 5786, 504.

Candelas:1987kf
  P. Candelas, A. M. Dale, C. A. Lutken and R. Schimmrigk,
  
  Nucl. Phys. B 298 (1988) 493.

Candelas:1987du
  P. Candelas, C. A. Lutken and R. Schimmrigk,
  
  Nucl. Phys. B 306 (1988) 113.

CICY
 A. Lukas et al., http://www-thphys.physics.ox.ac.uk/projects/CalabiYau/cicylist/cicylist.txt

KM++
D. Arthur and S. Vassilvitskii, 

 Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms pp. 1027-1035. 

KL
 S. Kullback and R. A. Leibler, “On Information and Sufficiency”, Ann. Math. Statist., Volume 22, Number 1 (1951), 79-86.

Anderson:2011ns
  L. B. Anderson, J. Gray, A. Lukas and E. Palti,
  
  Phys. Rev. D 84 (2011) 106005
  
  [arXiv:1106.4804 [hep-th]].

Anderson:2012yf
  L. B. Anderson, J. Gray, A. Lukas and E. Palti,
  
  JHEP 1206 (2012) 113
  
  [arXiv:1202.1757 [hep-th]].  

Abe:2015mua
  H. Abe, T. Kobayashi, H. Otsuka and Y. Takano,
  
  JHEP 1509 (2015) 056
  [arXiv:1503.06770 [hep-th]].
  
Otsuka:2018oyf
  H. Otsuka,
  
  JHEP 1805 (2018) 045
  [arXiv:1801.03684 [hep-th]].

Witten:1998cd
  E. Witten,
  
  JHEP 9812 (1998) 019
  [hep-th/9810188].  

Uranga:2000xp
  A. M. Uranga,
  
  Nucl. Phys. B 598 (2001) 225
  [hep-th/0011048].

Braun:2010vc
  V. Braun,
  
  JHEP 1104 (2011) 005
  [arXiv:1003.3235 [hep-th]].
  
Diophantine
  M. Davis,
  
  The American Mathematical Monthly 80, no. 3 (1973) 233-69.

Abadi:2016kic
  M. Abadi et al.,
  
  arXiv:1603.04467 [cs.DC].

sklearn
  F. Pedregosa et al.,
  
  J. Machine Learning Res.  12 (2011) 2825 
  [arXiv:1201.0490 [cs.LG]].
  
Kanno
  H. Kanno,
  
  Prog. Theor. Phys. Suppl.  135 (1999) 18.


  

