Committee machine that votes for similarity between materials
Duong-Nguyen Nguyen Tien-Lam Pham Viet-Cuong Nguyen Tuan-Dung Ho Truyen Tran Keisuke Takahashi Hieu-Chi Dam















^1Japan Advanced Institute of Science and Technology, 1-1 Asahidai, Nomi, Ishikawa 923-1292, Japan

^2ESICMM, National Institute for Materials Science, 1-2-1 Sengen, Tsukuba, Ibaraki 305-0047, Japan

^3HPC Systems Inc., 3-9-15 Kaigan, Minato-ku, Tokyo 108-0022, Japan

^4Applied Artificial Intelligence Institute, Deakin University, Geelong, Australia. 

^5Center for Materials Research by Information Integration, National Institute for Materials Science 1-2-1 Sengen, Tsukuba, Ibaraki 305-0047, Japan

^6JST, PRESTO, 4-1-8 Honcho, Kawaguchi, Saitama, 332-0012, Japan

^*dam@jaist.ac.jp







We developed a method for measuring the similarity between materials, focusing on specific physical properties. The obtained information can be utilized to understand the underlying mechanisms and to support the prediction of the physical properties of materials. The method consists of three steps: variable evaluation based on non-linear regression, regression-based clustering, and similarity measurement with a committee machine constructed from the clustering results. Three datasets of well-characterized crystalline materials represented by critical atomic predicting variables are used as test beds. Herein, we focus on the formation energy, lattice parameter, and Curie temperature of the examined materials. Based on the information obtained on the similarities between the materials, a hierarchical clustering technique is applied to learn the cluster structures of the materials that facilitate interpreting  the mechanism, and  an improvement of regression models is introduced for predicting the physical properties of the materials. Our experiments show that rational and meaningful group structures can be obtained and that the prediction accuracy of the materials’ physical properties can be significantly increased, confirming the rationality of the proposed similarity measure.










Committee machine that votes for similarity between materials
    
Duong-Nguyen Nguyen^1, Tien-Lam Pham^1,2, Viet-Cuong Nguyen^3, Tuan-Dung Ho^1, Truyen Tran^4, Keisuke Takahashi^5, Hieu-Chi Dam^1,5,6, *   


    December 30, 2023
==================================================================================================================================================







§ INTRODUCTION 

Computational materials science encompasses a range of methods that are used to model materials and simulate their responses on different length and time scales <cit.>. The majority of problems addressed by computational materials science are related to methods that focus on two central tasks. The first aims to predict the physical properties of materials, and the second aims to describe and interpret the underlying mechanism <cit.>. In the first task of predicting physical properties, computer-based quantum mechanics techniques <cit.> in the form of well-established first-principles calculations are generally performed with high accuracy and are applicable to any material, but with high computational cost. Recently, the increase in advanced machine learning techniques <cit.> and volume of computational material databases <cit.> has provided new opportunities for researchers to automatically construct prediction models (from a huge amount of precomputed data) that predict specific physical properties with the same level of high accuracy, while dramatically reducing the computational costs <cit.>.

By contrast, the second task, describing and interpreting the mechanisms underlying the physical properties of materials, relies mostly on the experience, insight, and even luck of the experts involved. In fact, comprehension of multivariate data with nonlinear correlations is typically extremely challenging, even for experts. Thus, the utilization of data mining and machine learning techniques to discover hidden structures and latent semantics in multidimensional data <cit.> of materials is promising, but only limited works have been reported so far <cit.>. 

To apply well-established machine-learning methods to solve problems in materials science, the primitive representation of materials must usually be converted into vectors, such that the comparison and calculations using the new representation reflect the nature of the materials and the underlying mechanisms of the chemical and physical phenomena. However, real-world applications, especially for solving the second task, often focus on physical properties of which the mechanism is not fully understood <cit.>. In these cases, it is almost impossible to appropriately represent the materials as vectors of features so that comparisons using well-established mathematical calculations can reflect the similarity/dissimilarity between them. Therefore, a true data-driven approach for solving materials science problems still requires much further fundamental development.

In this study, we focus on establishing a data-driven protocol for solving the second task of computational materials science. Focusing on a specific physical property, we aim to develop a method for measuring the similarity between materials from the viewpoint of the underlying mechanisms that work in these materials. The method for measuring this similarity consists of three steps: (1) variable evaluation based on non-linear regression, (2) regression-based clustering, and (3) similarity measurement with a committee machine <cit.> constructed based on the clustering results. The variable evaluation <cit.> aims to identify and remove unneeded, irrelevant, and redundant variables from the data <cit.>. We carried out this analysis in an exhaustive manner by testing all combinations of predicting variables to find the variables with the potential to yield good prediction accuracy (PA) for the target variable. The regression-based clustering method is developed from the well-known K-means clustering method <cit.> with major modifications for breaking down a large dataset into a set of separated smaller datasets, in each of which the target variables can be predicted by a different linear model. Regression-based clustering models are then constructed for all the selected potential combinations of predicting variables, so as to construct a committee machine that votes for the similarity between the materials.

Three datasets of well-characterized crystalline materials represented by appropriate predicting variables, together with their physical properties as determined through first-principles calculations or measured experimentally, are used as test beds. Our experiments show that the proposed similarity measure can derive rational and meaningful material groupings and  can significantly improve the PA of the physical properties of the examined materials.



§ METHODS

We consider a dataset 𝒟 of p materials. Assume that a material with index i  is described by an m-dimensional predicting variable vector x_i = ( x_i^1, x_i^2, … , x_i^m)∈ℝ^m. The dataset 𝒟 is then represented using a (p × m)  matrix. The target physical property values of the materials are stored as a p-dimensional target vector y = ( y_1, y_2 … y_p ) ∈ℝ^p. The entire data analysis flow is shown in Figure <ref>. 




 §.§ Kernel regression-based variable evaluation

To develop a better understanding of the processes that generated the data, we first utilize an exhaustive search to evaluate all variable combinations <cit.> to identify and remove unneeded, irrelevant, and redundant variables <cit.>. We begin by learning non-linear functions for predicting the values of a specific physical property (target quantity) of the materials. We apply the Gaussian kernel ridge regression (GKR) technique  <cit.>, which has recently been applied successfully to several challenges in materials science  <cit.>. For GKR, the predicted property y=f(x) at a point x is expressed as the weighted sum of Gaussians:


    f(x) = ∑_i=1^p c_iexp( -||x_i - x||_2^2/2 σ^2)


 where p is the number of training data points, σ^2 is a parameter corresponding to the variance of the Gaussian kernel function, and ||x_i - x||_2^2 = ∑_α = 0^m( x_i^α - x^α)^2 is the squared L^2 norm of the difference between the two m-dimensional vectors x_i and x. The coefficients c_i are determined by minimizing


    ∑_i=1^p[ f(x_i) - y_i] ^2 +λ∑_i=1^p||c_i||_2^2

where y_i is the observed data value for data instance i. The regularization parameters λ and σ are selected with the help of cross-validation, by excluding some of the materials as a validation set during the training process and measuring the coefficient of determination R^2, which is defined <cit.> as follows:


    R^2 = 1 - ∑_j=1^p_vld[ f(x_j) - y_j ]^2/∑_j=1^p_vld[ y̅ - y_j]^2

Here, p_vld is the number of data points and y̅ is the average of the validation set used to compare the values predicted for the excluded materials with the known observed values. In this study, we use R^2 as a measure of PA. 

To accurately estimate PA, we cross-validate the GKR <cit.> using the collected data repeatedly. To obtain a set of proper variable combinations that can accurately predict the target variable, we train the GKR models for all possible combinations of numerical predicting variables. It should be noted that, since we do not know yet the effect of each predicting variable on the target quantity, all the numerical predicting variables are normalized in the same manner in this analysis. With each combination, we search for the regularization parameters to maximize PA of the corresponding GKR model. Note that  each of the selected combinations contributes a perspective on the correlation between the target and the predicting variables. Thus, an ensemble averaging <cit.> technique can be applied to combine all the pre-screened regression models to improve the PA. Further, the similarity between materials regarding the mechanism of the chemical and physical phenomena associated with the target quantity can be investigated more comprehensively if we consider all the perspectives. Consequently, we need to construct regression-based clustering models for each obtained potential combination to build the committee machine.



 §.§ Regression-based clustering

In practice, a single linear model is often severely limited for modeling real data, because the data set can be non-linear or the data itself can be heterogeneous and contain multiple subsets, each of which fits best with a different linear model. However, in traditional data analysis, linear models are often preferred because of their highly advantageous interpretability. Within a linear model, one can intuitively understand how the predicting variables contribute to the target variable. Therefore, several efforts have been devoted to developing subspace segmentation techniques to decompose a high-dimensional dataset into a set of separate small datasets, each of which can be approximated well by different linear subspaces by employing principal component analysis <cit.>. 

In this study, our primary interest is the local linearity between the predicting variables and the target variable, which may reflect the nature of the underlying physics around the point of observation. Therefore, we employ a simple strategy, in which the subspace segmentation is an integration of a conventional clustering method and linear regression analysis. It should be noted that the subspaces may have fewer dimensions than the whole space. Hence, we apply the sparse linear regression analysis using L1 regularization <cit.> instead of the original one.

Our proposed regression-based clustering method is based on the well-known K-means clustering method with two modifications. (1) The sparse linear regression model derived from data associated with materials in a particular cluster (group) is considered to be its common characteristic (center). The dissimilarities in the characteristics of each material in a group relative to the shared (common) nature of that group (the distance to the center) are measured according to its deviation from the corresponding linear regression model. (2) The sum of the differences of all materials in a group from the corresponding linear regression model of another group is used to measure the dissimilarity in the characteristics of that group with regard to the other group. The sum of the dissimilarities between one group and another and that determined in the reverse direction are used to assess the divergence between the two groups. 

After performing the variable evaluation, we assume we have selected combinations of predicting variables that yield non-linear regression models of high PA. With one of the selected combinations,  m' numerical variables are selected from the original m numerical variables. A material in the dataset is then described by an m'-dimensional predicting variable vector 
x'_i = ( x_i^1, x_i^2, … , x_i^m' )∈ℝ^m', and the data are represented using a  (p × m')  matrix.

Given the set  𝒟 of p data points represented by m'-dimensional numerical vectors, a natural number k ≤ p represents the number of clusters for a given experiment. We assume that there are k linear regression models and that each data point in 𝒟 follows one of them. The aim is to determine those k linear regression models, accordingly, to divide 𝒟 into k non-empty disjoint clusters. Our algorithm searches for a partition of 𝒟 into k non-empty disjoint clusters (𝒟_1, 𝒟_2,  … , 𝒟_k) that minimize the overall sum of the residuals between the observed and predicted values (using the corresponding models) of the target variable. The problem can be formulated in terms of an optimization problem as follows.

For a given experiment with cluster number k, minimize

    P(W, M) = ∑_i=1^k∑_j=1^p w_ij∥ y_j - y_j^M_i∥

subject to

    ∀ j: ∑_i=1^k w_ij = 1,  w_ij∈{0, 1}
       1 ≤ k ≤ p, 1 ≤ i ≤ k, 1 ≤ j ≤ p

where y_j and  y_j^M_i are the observed value and the value predicted by model M_i (of k models) for the target property of the material with index j; W = [ w_ij]_p × k is a partition matrix (w_ij takes a value of 1 if object x_j belongs to cluster 𝒟_i and 0 otherwise), and M = ( M_1, M_2, … , M_k) is the set of regression models corresponding to clusters (𝒟_1, 𝒟_2,  … , 𝒟_k).

P can be optimized by iteratively solving two smaller problems:



  * fix M = M̂ and solve the reduced problem P(W, M) to find Ŵ (re-assign data points to the cluster of the closest center);

  * fix W = Ŵ and solve the reduced problem P(W, M) to find M̂ (reconstruct the linear model for each cluster).


Our regression-based clustering algorithm comprises three steps and iterates until P(W, M) converges to some local minimum values:



  * The dataset is appropriately partitioned into k subsets, 1≤ k ≤ p. Multiple linear regression analyses are independently performed with the L1 regularization method <cit.> on each subset to learn the set of potential candidates for the sparse linear regression models M^(0) = {  M_1^(0), M_2^(0), … , M_k^(0)}. This represents the initial step t = 0;


  * M^(t) is retained and problem P(W, M^(t)) is solved to obtain W^(t), by assigning data points in 𝒟 to clusters based upon models  M_1^(t), M_2^(t), … , M_k^(t);


  * W^(t) is fixed and M^(t) is generated such that P(W, M^(t+1)) is minimized. That is, new regression models are learned according to the current partition in step 2. If the convergence condition or a given termination condition is fulfilled, the result is output, and the iterations are stopped. Otherwise, t is set to t+1 and the algorithm returns to step 2.


The group number k is chosen considering two criteria: high linearity between the predicting and target variables for all members of the group, and no model representing two different groups. The first criterion has higher priority and can be  quantitatively evaluated by using the Pearson correlation scores between the predicted and observed values for the target variable of the data instances in each group, by applying the corresponding linear model. The second criterion is implemented to avoid the case in which one group with high linearity is further divided into two subgroups that can be represented by the same linear model. The determination of k, therefore, can be formulated in terms of an optimization problem, as follows:


    k = _k ≤ p[log1-min_1 ≤ i ≤ k R^2_i,i/min_1 ≤ i ≤ k R^2_i,i+ max_1 ≤ i≠ j≤ k  R^2_i,j]

where R^2_i,i and R^2_i,j are the Pearson correlation scores between the predicted and observed values for the target variable when we apply the linear model M_i to data instances in clusters i and j, respectively. 

The first term in this optimization function monotonically decreases with respect to the range of min_1 ≤ i ≤ k R^2_i,i varying from 0 to 1. When min_1 ≤ i ≤ k R^2_i,i approaches 1 (the entire cluster exhibits almost perfect linearity between the target and predicting variables), the optimization function drops on a log scale to emphasize the expected region. In contrast, the optimization function exponentially increases when min_1 ≤ i ≤ k R^2_i,i approaches 0 (one of the clusters shows no linearity between the target and predicting variables). The second term in this optimization function is introduced to avoid overestimation of k, in which a group with high linearity further divides into two subgroups that can be represented by the same linear model. It should be noted that the criterion for determining k is also the criterion for evaluating a regression-based clustering model. Further, cluster labels can be assigned for a material  without knowing the value of the target physical property, by using the estimated value obtained from a prediction model, a non-linear regression model.



 §.§ Similarity measure with committee machine

A clustering model, obtained through regression-based clustering for a particular combination of predicting variables, represents a specific partitioning of the dataset into groups in which the linear correlations between the predicting and target variables can be observed. The materials belonging to the same group potentially have the same actuating mechanisms for the target physical property. However, materials that actually have the same actuating mechanisms for a specific physical property should be observed similarly in many circumstances. Therefore, the similarity between materials, focusing on a specific physical property, should be measured in a multilateral manner. For this purpose, for each pre-screening of the sets of predicting variables that yield non-linear regression models of high PA (section <ref>), we construct a regression-based clustering model. A committee machine that votes for the similarity between materials is then constructed from all obtained clustering models. The similarity between two materials can be measured naively using the committee algorithm <cit.>, by counting the number of clustering models that partition these two materials into the same cluster. The affinity matrix A of all pairs of materials in the dataset is then constructed as follows:


    A_a,b = 1/|S_h|∑_∀S∈ S_h∑_i=1^k_S w^S_ia w^S_ib

 where S_h is the set of all pre-screened combinations of  predicting variables that yield non-linear regression models of high PA and k_s is the cluster number. Further, W^S = [ w^S_ij]_p × k_S is the partition matrix of the clustering models obtained through the regression-based clustering analysis using the combination of predicting variables S (w^S_ia takes a value of 1 if material a belongs to cluster i and 0 otherwise). By using this affinity matrix, one can easily implement a hierarchical clustering technique <cit.> to obtain a hierarchical structure of groups of materials that have similar correlations between the predicting and target variables.




§ RESULTS AND DISCUSSION

We applied the methods described above to a sequential analysis for automatic extraction of physicochemical information relating to considered materials from three available datasets. For each dataset, a brute force examination of all combinations of numerical predicting variables was conducted using a non-linear regression technique, to identify combinations of predicting variables that yielded regression models of high PA for the later analysis process. For each of the pre-screened combinations, physically meaningful patterns in the form of material groups, as well as the linear relationships between the selected predicting and target variables, could be detected automatically for the materials in each group utilizing the regression-based clustering technique. The committee machine was then constructed from the obtained clustering models. Subsequently, a hierarchical structure of material groups similar to each other could be extracted using the hierarchical clustering technique. We evaluated the obtained results from both qualitative and quantitative perspectives. The qualitative evaluations were based on the rationality and interpretability of the obtained hierarchy with reference to the domain knowledge; the quantitative evaluations were performed based on the PA of the predictive models constructed with reference to the obtained similarity between materials.




 §.§ Experiment 1: Mining quantum calculated formation energy data of Fm3̅m AB materials


In this experiment, we collected computational data for 239 binary AB materials from the Materials Project database <cit.>. The A atoms were virtually all metallic forms: alkali, alkaline earth, transition, and post-transition metals, as well as lanthanides. The B elements, by contrast, were mostly all metalloids and non-metallic atoms. We set the computed formation energy E_form of each AB material as the physical property of interest. To simplify the demonstration of our method, we limited the collected compounds to those possessing the same cubic structure as the Fm3̅m symmetry group (the NaCl structure).

To represent each material, we used a set of 17 predicting variables divided into three categories, as summarized in Table <ref>. The first and second categories pertained to the predicting variables of the atomic properties of the element A and element B constituents; these included eight numerical predicting variables: (1) atomic number (Z_A, Z_B); (2) atomic radius (r_A, r_B); (3) average ionic radius (r_ionA, r_ionB), (4) ionization potential (IP_A, IP_B); (5) electronegativity (χ_A, χ_B); (6) number of electrons in outer shell (n_eA, n_eB); (7) boiling temperature  (T_bA, T_bB); and (8) melting temperature  (T_mA, T_mB) of the corresponding single substances. The boiling and melting temperatures were as measured under standard conditions (0^∘C, 10^5 Pa). 

Information related to crystal structure is very valuable for understanding the physical properties of materials. Therefore, we designed the third category with structural predicting variables whose values were calculated from the crystal structures of the materials. In this experiment, owing to the similarities in the crystal structures of the collected materials, we utilized only the unit cell volume (V_cell) as the structural predicting variable. The computed E_form of each material was set as the target variable. 





A kernel regression-based variable evaluation was performed for these data with 3-times 10-fold cross-validations. We first examined how E_form can be predicted from the designed predicting variables for all collected materials. We performed a screening for all possible (2^17 - 1 = 131,071) variable combinations. Hence, we found a total of 34,468 variable combinations deriving GKR models with R^2 scores exceeding 0.90 (Fig.<ref>). Among them, there were 139 variable combinations deriving GKR models with R^2 scores exceeding 0.96. These predicting variable combinations were then considered as candidates for the next step of the analysis. The highest prediction accuracy PA in this experiment is 0.967 (mean of absolute error, abbreviated as MAE: 0.122 eV), obtained using the combination {V_cell, χ_A, n_eA, n_eB, IP_A, T_bA, T_mA, r_B}. Moreover, we could obtain superior PA with an R^2 score of 0.972 (MAE: 0.117 eV) by taking ensemble averages <cit.> of GKR models, which were constructed using the 139 selected variable combinations.

We performed regression-based clustering analyses for all 139 selected variable combinations with 1000 initial randomized states. By using evaluation criteria similar to those for determining the number of clusters (formula <ref>), the 200 best clustering results among these trials were selected to construct a committee machine that voted for the similarity between materials. The obtained affinity matrix for all the Fm3̅m  AB materials is shown in Fig.<ref>a. The similarity between each material pair varies from 0 to 1. A cell of the affinity matrix takes a 0 value when the corresponding two materials are never included in the same cluster by a regression-based clustering model. In contrast, a cell of the affinity matrix takes a value of 1 when the corresponding two materials always appear in the same cluster according to every regression-based clustering model. By using this similarity, we could roughly divide all the materials into two groups, as represented by the upper left and bottom right of Fig.<ref>a. 






Figure <ref>b shows an enlarged view of the affinity matrix for two groups of typical materials denoted by G1 and G2. We can clearly see that the affinities between materials within each of the two groups, G1 and G2, exceed 0.7, showing high intra-group similarities. In contrast, the affinities between materials in different groups are smaller than 0.2, showing significant dissimilarity between G1 and G2. Further detailed investigation reveals that the materials in G1 are oxide, nitride, and carbide. The maximum common positive oxidation number of the A elements is greater than or equal to the maximum common negative oxidation number of the B elements for the compounds in this group. On the other hand, the materials in G2 are halides of alkaline metal, oxide, nitride, and carbide, for which the maximum common positive oxidation number of the A elements is less than or equal to the maximum common negative oxidation number of the B elements. Further investigation shows that only seven among 24 compounds in G1 have computed electronic structures with a band gap. In contrast, half of the compounds in G2 have computed electronic structures with a band gap. The obtained results suggest that the bonding nature of compounds in G1 is different from that of compounds in G2. 

The linearities between the target variable and the predicting variables for the two groups are summarized in Fig.<ref>c. The diagonal plots show the correlations between the observed and predicted values for the target variables obtained using linear models of the predicting variables for the materials in the two groups. The off-diagonal plots show the correlations between the observed values and predicted values for the target variables obtained using the linear models of the other groups. We could again confirm the intra-group similarity and the dissimilarity between different groups in terms of the linearity between the target and predicting variables for the compounds in the two groups. 

To quantitatively evaluate the validity of the analysis process, we embedded the similarity measured by the committee machine into the regression of E_form of the Fm3̅m  AB materials. To predict the value of the target variable for a new material, instead of using the entire available dataset, we used only one-third of the available materials having the highest similarity to the new material. It should again be noted that the similarity between the materials in the dataset and the new material can be determined without knowing the value of the target physical property, using the value predicted by ensemble averaging of the non-linear regression models. 




Table <ref> summarizes the PA in predicting E_form values  of the Fm3̅m materials  obtained using several regression models with the designed predicting variables. The non-linear model obtained using ensemble averaging  of the best non-linear regression models, having an R^2 score of 0.972 (MAE: 0.117 eV), could be improved significantly to an R^2 score of 0.982 (MAE: 0.101 eV) regarding the information from the similarity measurement (Fig.<ref>). Therefore, the obtained results provide significant evidence to support our hypothesis that the similarity voted by the committee machine reflects the similarity in the actuating mechanisms of the target material physical property. 

 


 §.§ Experiment 2: Mining quantum calculated lattice parameter for body-centered cubic structure data


In this experiment, a dataset of 1541 binary AB body-centered cubic (BCC) crystals with a 1:1 element ratio was collected from Ref.<cit.>. We focused on the computed lattice constant value L_const of the crystals. The A elements corresponded to almost all transition metals {Ag, Al, As, Au, Co, Cr, Cu, Fe, Ga, Li, Mg, Na, Ni, Os, Pd, Pt, Rh, Ru, Si, Ti, V, W, and Zn} and the B elements corresponded to those with atomic numbers in the ranges of 1–42, 44–57, and 72–83. This dataset included unrealistic materials such as the binary material AgHe, which incorporates He, an element that is known to possess a closed shell structure and is, therefore, unlikely to form a solid. 

To describe each material, we used a combination of 17 variables that related to basic physical properties of the A and B constituent elements, as summarized in Table <ref>. These chosen properties were as follows: the (1) atomic radius (r_A, r_B); (2) mass (m_A, m_B); (3) atomic number (Z_A, Z_B); (4) number of electrons in outermost shell (n_eA, n_eB); (5) atomic orbital (ℓ_A, ℓ_B); and (6) electronegativity (χ_A, χ_B). The atomic orbital values were converted from categorical symbols s, p, d, f to numerical values representing the orbitals, 0, 1, 2, 3, respectively. To embed the structure information, four more properties were included: (7) the density of atoms per unit volume (ρ_A, ρ_B); (8) the unit cell density ρ; (9) the difference in electronegativity  d_χ; and (10) the sum of the atomic orbital B and difference in electronegativity sum_AD (see Ref.<cit.>).  

A kernel regression-based variable selection with 3-times 10-fold cross-validation was performed to examine all combinations of the 17 variables. From the total number of screening variable combinations (2^17 - 1 = 131,071), we found 60,568 variable combinations for deriving regression models with R^2 scores exceeding 0.90 (Fig.<ref>). Among them, there were 57 variable combinations yielding regression models with R^2 scores exceeding 0.9895. The highest PA for this experiment is 0.989 (MAE:  0.014 Å), which was obtained using the combination {ρ, ℓ_A,  r_covB, m_A, m_B, ρ_B, n_eB}. We could obtain a better PA with an R^2 score of 0.991 (MAE: 0.013 Å) by taking ensemble averaging of GKR models which derived from the 57 selected variable combinations. This result is a considerable improvement in comparison with the maximum PA (R^2 score: 0.90) of the support vector regression technique with the feature selection strategy mentioned in <cit.>.





In the regression-based clustering analysis, the 57 selected variable combinations accompanied by 1000 initial randomized states for each combination were used to search for the most probable clustering results to construct the committee machine. The affinity matrix obtained for all materials is shown in Fig.<ref>a, after rearrangement by a hierarchical clustering algorithm <cit.>. By utilizing this similarity, we could roughly divide all materials in the dataset into three groups: G1, G2, and G3. Further investigation revealed that most materials in G1 are constructed from two heavy transition metals. In contrast, the materials in G2 and G3 are constructed from a metal and a non-metal element, oxide and nitride. For a given A element, the L_const of the materials in G1 increases with the atomic number of the B element. On the other hand, the L_const of the materials in G2 remains constant for the materials sharing the same A element. Further, the L_const for the materials in group G3 mainly depends on the electronegativity difference between the constituent elements A and B. Note that the materials in these three groups are visualized in detail in the Supplemental Materials. The linearities between the observed and predicting variables for these groups are shown in Fig.<ref>b.

To predict the L_const of a new material, we applied the same strategy as that explained in the previous experiment. Table <ref> summarizes the PA values obtained in our experiments. The non-linear model obtained using ensemble averaging of the best 57 non-linear regression models and having an R^2 score of 0.991 (MAE: 0.013 Å) could be marginally improved to an R^2 score of 0.992 (MAE: 0.011 Å) by including information from the similarity measurement (Fig.<ref>). 


 




 §.§ Experiment 3: Mining experimentally observed Curie temperature data of rare-earth–transition metal alloys

In this experiment, we collected experimental data related to 101 binary alloys consisting of transition and rare-earth metals from the NIMS AtomWork database <cit.>, which included the crystal structures of the alloys and their observed Curie temperatures T_c. 

To represent the structural and physical properties of each binary alloy, we used a combination of 21 variables divided into three categories, as summarized in Table <ref>. The first and second categories contained predicting variables describing the atomic properties of the transition metal elements and rare-earth elements, respectively. The properties were as follows: (1) atomic number (Z_R, Z_T); (2) covalent radius (r_covR, r_covT); (3) first ionization (IP_R, IP_T); and (4) electronegativity (χ_R, χ_T).  In addition, predicting variables related to the magnetic properties were included: the (5) total spin quantum number (S_3d, S_4f); (6) total orbital angular momentum quantum number (L_3d, L_4f); and (7) total angular momentum (J_3d, J_4f). For R metallic elements, additional variables J_4fg_j and J_4f(1-g_j) were added, because of the strong spin-orbit coupling effect. 

As in the two previous experiments, a third category variable was chosen which contained values calculated from the crystal structures of the alloys reported in the AtomWork database <cit.>. The designed predicting variables included the transition (C_T) and rare-earth (C_R) metal concentrations. Note that, if we use the atomic percentage for the concentration, the two quantities are not independent. Therefore, in this work, we measured the concentrations in units of atoms/Å^3; this unit is more informative than the atomic percentage as it contains information on the constituent atomic size. As a consequence, (C_T) and (C_R) were not completely dependent. Other additional structure variables were also added: the mean radius of the unit cell between two rare-earth  elements r_RR,  between two  transition metal elements r_TT, and between transition and rare-earth elements r_TR. We set the experimentally observed T_c as the target variable.

A kernel regression-based variable selection analysis was performed for these data using leave-one-out cross-validation. Among all the examined variable combinations, (2^21-1 = 2,097,151), we found 84,870 combinations for which the corresponding GKR models exhibited R^2 scores exceeding 0.90 (Fig.<ref>). Among them, there were 59 variable combinations yielding GKR models associated with R^2 scores exceeding 0.95. These predicting variable combinations were selected for the next analysis step. The highest PA in this experiment was 0.968 (MAE: 42.74 K), obtained using the combination {C_R, Z_R, Z_T, χ_T, r_covT, L_3d, J_3d}. We could obtain a better PA with an R^2 score of 0.974 (MAE: 37.87 K), by applying ensemble averaging to the GKR models, which were derived from the selected 59 variable combinations.  We considered these variable combinations as candidates for the next step of the analysis.

In the regression-based clustering analysis, 59 variable combinations with 1000 initial randomized states were used to search for the most probable clustering results to construct the committee machine to vote for the similarity between the alloys. The obtained affinity matrix for all the alloys is shown in Fig.<ref>a. An enlarged view of the three groups of alloys having high similarity (denoted G1, G2, and G3) is shown in Fig.<ref>b. Further investigation revealed that G1 includes Mn- and Co-based alloys with high T_c, Mn_23Pr_6 (448 K), Mn_23Sm_6 (450 K), Co_5Pr (931 K), and Co_5Nd (910 K). Other low-T_c Co-based alloys, Co_2Pr (45 K) and Co_2Nd (108 K), are counted as having higher similarity with Ni-based alloys in G3, Ni_5Nd (7 K) and Ni_2Ho (16 K). In contrast, G2 includes all the Fe-based Fe_17RE_2 alloys, where RE represents different rare-earth metals. To confirm the value of our similarity measure, Fig.<ref>c shows the linearities between the observed and predicting variables for these groups, as well as the dissimilarities among these groups. 

In the next analysis step, we utilized the obtained similarity measure to predict T_c for a new material by using the same strategy used in the two previous experiments.  The non-linear model obtained using ensemble averaging of the best non-linear regression models and having an R^2 score of 0.974 (MAE: 37.87 K) could be improved significantly to attain an R^2 score of 0.991 (MAE: 24.16 K) by utilizing the information from the similarity measurement (Fig.<ref> and Table <ref>). The obtained results provide significant evidence to support our hypothesis that the similarity voted for by the committee machine indicates the similarity in the actuating mechanisms of the T_c of the binary alloys.



§ CONCLUSION

In this work, we proposed a method to measure the similarities between materials, focusing on specific physical properties, to describe and interpret the actual mechanism underlying a physical phenomenon in a given problem. The proposed method consists of three steps: variable evaluation based on non-linear regression, regression-based clustering, and similarity measurement with a committee machine constructed from the clustering result. Three datasets of well-characterized crystalline materials represented by key atomic predicting variables were used as test beds. The formation energy, lattice parameter, and Curie temperature were considered as target physical properties of the examined materials. Our experiments show that rational and meaningful group structures can be obtained with the help of the proposed approach. The similarity measure information helped significantly increase the prediction accuracy for the material physical properties. Through use of ensemble top kernel ridge prediction models, the R^2 score increased from 0.972 to 0.982 for the formation energy prediction problem; 0.991 to 0.992 for the lattice constant prediction problem, and 0.974 to 0.991 for the Curie temperature prediction problem after utilizing the similarity information. Thus, our results indicate that our  proposed data analysis flow can systematically facilitate further understanding of a given phenomenon by identifying similarities among materials in the problem dataset. 


This work was partly supported by PRESTO and by the “Materials Research by Information Integration Initiative” (MI^2I) project of the Support Program for Start-Up Innovation Hub, from the Japan Science and Technology Agency (JST), and by JSPS KAKENHI Grant-in-Aid for Young Scientists (B) (Grant JP17K14803), Japan. 




§ AUTHOR CONTRIBUTIONS STATEMENT

Duong -Nguyen Nguyen and Hieu-Chi Dam conceived the experiments;  Duong-Nguyen Nguyen, Viet-Cuong Nguyen, and Tuan-Dung Ho conducted the experiments; and Duong-Nguyen Nguyen, Tien-Lam Pham, Truyen Tran, Keisuke Takahashi, and Hieu-Chi Dam analyzed the results. Duong-Nguyen Nguyen and Hieu-Chi Dam wrote the paper, and all authors reviewed the manuscript. 

The authors declare no competing interests.




