abstract: We have measured Hall effect, magnetotransport and magnetostriction on the field induced phases of single crystalline UPt$_2$Si$_2$ in magnetic fields up to 60 T at temperatures down to 50 mK, this way firmly establishing the phase diagram for magnetic fields $B \| a$ and $c$ axes. Moreover, for $B \| c$ axis we observe strong changes in the Hall effect at the phase boundaries. From a comparison to band structure calculations utilizing the concept of a dual nature of the uranium 5$f$ electrons, we propose that these represent field induced topological changes of the Fermi surface due to at least one Lifshitz transition. Furthermore, we find a unique history dependence of the magnetotransport and magnetostriction data, indicating that the proposed Lifshitz type transition is of a discontinuous nature, as predicted for interacting electron systems.
address:  $^{1}$Institut für Physik der Kondensierten Materie, TU Braunschweig, D-38106 Braunschweig, Germany
  $^{2}$Institut für Mathematische Physik, TU Braunschweig, D-38106 Braunschweig, Germany
  $^{3}$Laboratoire National des Champs Magn$\acute{\textrm{e}}$tiques Intenses (LNCMI-EMFL), CNRS, UGA, 38042 Grenoble, France
  $^{4}$National High Magnetic Field Laboratory, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, USA
  $^{5}$Kamerlingh Onnes Laboratory and Institute Lorentz, Leiden University, 2300RA Leiden, The Netherlands
author: D. Schulze Grachtrup$^{1}$, N. Steinki$^{1}$, S. Süllow$^{1}$, Z. Cakir$^{2}$, G. Zwicknagl$^{2}$, Y. Krupko$^{3}$, I. Sheikin$^{3}$, M. Jaime$^{4}$, J.A. Mydosh$^{5}$
date: 2024-10-01
title: Magnetic phase diagram and electronic structure of UPt$_2$Si$_2$ at high magnetic fields: a possible field-induced Lifshitz transition

# Introduction

Lifshitz transitions - that is, quantum phase transitions involving topological changes of the Fermi surface, and thus referred to as electronic topological transitions (ETT) - have been proposed to play a major role for the physics of correlated electron systems. Here, a variety of exotic field-induced phases, as well as unconventional pressure induced phases (including unconventional superconducting ones) have been observed and attributed to Lifshitz transitions .

The theory of ETT was developed to account for the ground state properties of certain materials under wide variation of external parameters such as pressure . It considered non-interacting electrons at zero temperature, yielding a continuous transition of 2$\frac{1}{2}$ order, which reflects the exponent in the Ehrenfest expression in three dimensions. Later, based on various experimental observations, the case of interacting electrons was treated in detail . Here, conceptually, a new (low) energy scale is associated with the interacting electron system, which may produce ETT in experimentally accessible magnetic field and pressure ranges of a few 10 T and GPa. As well, it was predicted that for interacting electron systems the transitions inherently become discontinuous .

Regarding the experimental verification of electronic topological transitions, cases of real materials exhibiting Lifshitz transitions are rare. On general grounds, it has been demonstrated that anomalies from ETT should be observable in various transport properties . Yet, ETT exist only for zero temperature, and smear out with finite temperature. It is a formidable experimental task to identify a Lifshitz type transition, requiring experiments down to low temperatures under extreme conditions. As well, for correlated electron systems, calculating the band structure as function of external control parameters is a very challenging task.

A case in point is the intermetallic $5f$ electron system UPt$_2$Si$_2$. The material belongs to the large class of U$T_2M_2$-compounds ($T$ = transition metal, $M$ = Si or Ge) and crystallizes in the tetragonal CaBe$_2$Ge$_2$ structure (space group P4/*nmm*) . In zero magnetic field, it undergoes an antiferromagnetic (AFM) transition at $T_N = 32$ K. The magnetic structure consists of moments $\mu_{ord} \sim 2.5 \mu_B$ ferromagnetically aligned within the $ab$ plane, and antiferromagnetically coupled and pointing along the $c$ axis . The simple magnetic structure with a large magnetic moment, combined with a moderately enhanced electronic contribution to the specific heat $\gamma = 32$ mJ/mole K$^2$, was taken as indicator for UPt$_2$Si$_2$ to be one of the rare examples of an uranium intermetallic local moment magnet. Correspondingly, a crystal electric field (CEF) scheme for the $5f^2$ state of the uranium ion was proposed, that has been used to explain initial high field magnetization measurements and the anisotropy of the susceptibility . Additional fine structure in the magnetization observable in the field range $\sim 20 - 40$ T was not considered to be at odds with the CEF concept.

Based on an extensive reinvestigation of the magnetization we have demonstrated that the agreement between CEF model and experimental data does not hold up to high fields. For magnetic fields $B \parallel a$ axis, aside from the suppression of AFM order, there is a hysteretic high field ($\sim$ 40 T) regime, whose nature is not understood as yet (see Fig. ). Moreover, for fields $B \parallel c$ axis, above 24 T the experimental data strongly deviates from the CEF theoretical predictions. In particular, a complex series of field induced phases is observed above 24 T, and which cannot be attributed only to spin reorientation processes and/or crystal field effects . As an alternative to the CEF model, we have proposed that an itinerant picture of the properties of UPt$_2$Si$_2$ is more appropriate, a view supported by recent band structure calculations . Moreover, these calculations have highlighted the relevance of correlation effects in this system . As well, the general character of the band structure has been revealed to be "quasi-two-dimensional" as result of the comparatively low crystallographic symmetry. This two-dimensional character is reflected for instance in the highly anisotropic resistivity of UPt$_2$Si$_2$ .

In Ref. , we have argued that the observation of the field induced phases in UPt$_2$Si$_2$ is related to Lifshitz type transitions. To test the validity of this concept in UPt$_2$Si$_2$, studies at lowest temperatures and using experimental tools directly testing the Fermi surface (FS) and the order of the phase transitions are required. A test of the FS by means of quantum oscillation measurements cannot be performed for UPt$_2$Si$_2$, as the oscillations are suppressed by intrinsic structural disorder from strained Pt(2)/Si(2) layers in the CaBe$_2$Ge$_2$ lattice . Therefore, more integral – and less disorder–dependent – probes of the FS need to be investigated to check the Lifshitz scenario for UPt$_2$Si$_2$. In addition, to establish the order of the field-induced phase transitions experimental probes sensitive to the structural properties may be used. If combined with band structure calculations, it will allow an assessment of the nature of the field-induced phases in UPt$_2$Si$_2$.

In this situation, we present a study on UPt$_2$Si$_2$ under extreme conditions, that is at temperatures down to 50 mK and in fields up to 60 T, using Hall effect, magnetotransport and magnetostriction. Our experiments clearly demonstrate changes of the FS in high magnetic fields. Our study is complemented by band structure calculations utilizing the concept of a dual nature of the uranium 5$f$ electrons, which simulate the effect of magnetic fields on the topology of the band structure. Resulting from these calculations, it is verified that Lifshitz type transitions may be induced in high magnetic fields in UPt$_2$Si$_2$.

# Experimental details

The experiments presented here were performed on single crystalline UPt$_2$Si$_2$, the samples being as-cast, bar shaped with a cross section $\sim 1 \times 1$ mm$^2$ and length of a few mm. Material from the same single crystal has been characterized in the Refs. \[\].

The electronic transport studies were carried out in the Laboratoire National des Champs Magn$\acute{\textrm{e}}$tiques Intenses in dc fields up to 34 T directed along $a$ and $c$ axes. For the experiments inside the magnet bore a dilution cryostat was installed. Data were taken with a standard lock-in setup, with a reasonable signal-to-noise ratio obtained with a measurement current of 1 mA directed along the $a$ axis. It resulted in an equilibrium temperature of 120 mK, with additional experiments with lower currents carried out down to 50 mK. Accordingly, we now have access to a wide range of the field/temperature plane up to a $B/T$ ratio of almost 700 T/K.

The sample was fitted onto a rotatable sample holder and immersed into the helium mixture. Up to 10 electrical contacts were glued on the sample surfaces with silver paint to allow for simultaneaous measurements of transverse magnetoresisitivity and Hall effect for each magnetic field direction. In a second round of experiments, the same set-up was used, but now to measure the longitudinal magnetoresistivity. Finally, axial magnetostriction experiments were performed at the Los Alamos High Field Laboratory in pulsed fields up to 60 T directed along $a$ and $c$ axes. Here, the base temperature was 1.4 K, with the experiment performed using an optical fiber with a Bragg grating .

# Results

For fields $B \parallel a$ axis up to 34 T and low $T$, the Hall resistivity $\rho_{xy}$ is linear in $B$ (Fig. (a)). Consistent with Ref. \[\], there are no phase transitions in this field range. Next, Hall effect data for $B \parallel c$ axis up to 34 T are included in Fig. (a), the full set of (low temperature) data is summarized in Fig. . For fields up to 23 T, *viz.*, in the AFM phase $I$, $\rho_{xy}$ increases linearily with field . At the phase boundary $I - III$ there is a distinct upward curvature in $\rho_{xy}(B)$, which becomes linear in field again in phase $III$. At the phase $III - V$ boundary there is a - now downwards - curvature in $\rho_{xy}$. Within experimental scatter, no hysteresis is observed between field-sweep-up and -down measurements, and there is no temperature dependence in the range below 2 K. Qualitatively, this behavior is reminiscent to that of Rh-doped URu$_2$Si$_2$ .

For parameterization, the Hall effect is fitted using the expression $\rho_{xy} = R_H B$. For $B \parallel a$ axis the data in phase $I$ yield a Hall coefficient $R_H = 1.7 \times 10^{-9}$ m$^3$/C (solid line in Fig. (a)). Correspondingly, for $B \parallel c$ axis the linear regimes from 0 to 23 T and 25 to 32 T lead to Hall coefficients $R_H = 3.6 \times 10^{-8}$ m$^3$/C and $R_H = 5.7 \times 10^{-8}$ m$^3$/C, respectively. Overall, these values are broadly consistent with the typical behavior of heavy fermion intermetallics. We note that, although for $B \parallel c$ axis at highest fields (34 T) the system resides in phase $V$ (as is proven by the observation of hysteresis in the magnetoresistivity, see below), in the Hall effect we observe non-monotonic behavior in this field range. The reason for the unusual behavior is not clear, and will require experiments to still higher fields to solve.

In magnetic materials, the Hall effect contains two terms. The *normal* contribution $\rho_{xy}^{nor} = R_N B$ measures the carrier density $n$ in units of the electron charge $e$: $R_N = \left( n e \right)^{-1}$. The *anomalous* Hall contribution $\rho_{xy}^{ano}$ reflects terms dependent on the resistivity and/or magnetization (reported in the Refs. \[\]). Therefore, and adding to the data published in Ref. \[\], low temperature magnetoresistivity has been carried out (Figs. (b) and ). At lowest temperatures for $B \parallel a$ axis up to 34 T, for the magnetoresistivity we find to good approximation $\rho_{xx}(B) = \rho_{xx}(B=0) + a B^{\frac{5}{2}}$. In accordance with the Hall effect and Ref. \[\], we find no evidence for phase transitions.

In contrast, for $B \parallel c$ axis the transitions from phase $I$ into $III$ and $III$ into $V$ are reflected by distinct anomalies in the magnetoresistivity. The transition $I \rightarrow III$ is accompanied by a steep increase of the magnetoresistivity, with the midpoint of the upturn close to the transition field determined from magnetization. Conversely, the transition $III \rightarrow V$ shows up as a corresponding drop of the magnetoresistivity.

Surprisingly, the magnetoresistivity $B \parallel c$ axis is accompanied by a curious type of hysteresis (Figs. (b), , ): Measurements of $\rho_{xx}(B)$ by sweeping from zero field into phase $V$ and back produce hysteresis in the magnetoresistivity. In contrast, sweeps from zero field only into phase $III$ and back produce no hysteresis in $\rho_{xx}(B)$ at low $T$. To demonstrate this, we have carried out field history dependent longitudinal magnetoresistivity measurements ($B \parallel c \parallel I$) at low temperatures, which we summarize in Fig. . Here, we have first swept the field from zero into phase $III$ (up to 30 T) and back to zero for various temperatures below 1 K. For this measurement sequence no hysteresis is observed in the magnetoresistivity. Conversely, for field sweeps at the same temperatures up into phase $V$ (final field: 35 T) we detect hysteresis in the magnetoresistivity in field-sweep-down vs. field-sweep-up data. This observation indicates that the phase $III-V$ boundary denotes a first order phase transition. Moreover, this observation is consistent with our previous magnetoresistivity study , where we swept the field up to 28 T, *i.e.*, into phase $III$, and did not observe hysteresis.

In terms of the anomalous Hall contribution $\rho_{xy}^{ano}$, the absence of hysteresis in the Hall effect and its presence in the magnetoresistivity (Fig. ) implies that $\rho_{xy}^{ano}$ is not dependent on $\rho_{xx}(B)$. Then, the upturn in $\rho_{xy}$ at the phase $I-III$ boundary might be attributed to the corresponding upturn in the magnetization $M$ (see Refs. \[\]). Conversely, at the phase $III-V$ boundary the downturn in $\rho_{xy}$ is clearly at odds with the upturn in $M$. It implies that this phase transition is accompanied by a carrier density change, and which may involve a qualitative change of the Fermi surface as in an ETT.

To complement our study on UPt$_2$Si$_2$ with a structural probe, we have carried out axial magnetostriction experiments, which we depict in Fig. . For $B \parallel a$ axis we find a contraction of the sample for all fields. Further, a slight change of slope occurs at elevated fields, becoming hysteretic in the temperature/field range, where magnetization hysteresis is observed. The (hysteretic) features denote the transition from phase $I$ into the paramagnetic state.

For $B \parallel c$ axis the crystal UPt$_2$Si$_2$ expands for fields up to $\sim$ 40 T, and at highest fields the magnetostriction saturates. The field-induced phase transitions are identified as additional structure in the magnetostriction. Similar to the magnetization, at low temperatures there is a twofold structure in the data reflecting the transition from phase $I$ into $III$, and finally into phase $V$. The critical fields of the different phases are identified as points of maximum slope in the field-sweep-up measurements.

Similar to the magnetoresistivity, we find a history dependent hysteresis at low $T$ (Fig. ): For measurements from zero field up into phase $III$ and back no hysteresis is observed. Conversely, when the final field lies within phase $V$, structural hysteresis appears. Thus, the structural probe magnetostriction verifies that the phase transition $III - V$ is of a first order nature.

With the present data set we complete our high field studies on UPt$_2$Si$_2$. By combining the new data with those from Ref. \[\], we present the magnetic phase diagrams of UPt$_2$Si$_2$ for $B \parallel a$ and $c$ axes in Fig. . Altogether, our new set of data fully confirm the essential findings on the magnetic phase diagrams of UPt$_2$Si$_2$ as reported in Ref. \[\]. In particular, for the field $B \parallel a$ axis, the new data points derived from magnetostriction measurements, which define the phase border lines from the AFM phase $I$ into the hysteretic regime $II$ and the paramagnetic regime sit well on top of those previously established. Furthermore, in the intermediate temperature regime $\sim$ 20 – 30 K the new data now define the border lines more accurately as was possible with the data presented in Ref. \[\].

For the field $B \parallel c$ axis at low temperatures the border lines between the phases $I$, $III$ and $V$ are perfectly reproduced with our new data. Furthermore, the existence and nature of the phase border lines has been established down into the mK-range. Finally, with the new data the phase border lines are now more accurately determined in the intermediate temperature regime $\sim$ 15 – 25 K.

As we have noted before , we believe that there must be additional phase border lines in high magnetic fields between the paramagnetic regime and a phase $IV$, and between phase $IV$ and $III/V$. These observations are based on the qualitative change of the field dependent character of the magnetization (see Fig. 1 with field $B \parallel c$ axis in Ref. ). First, the magnetization at 20 K and above evolves monotonically in a Brillouin-function like fashion with field, while for the data taken at 16 K and below there is a hysteretic metamagnetic transition at around 25 T. Therefore, the metamagnetic transition must be into a phase different from the paramagnetic regime, *viz.*, into phase $IV$. As well, the magnetization taken in the temperature range $\sim$ 10 to 16 K exhibits a single metamagnetic transition, while for data at 10 K and below there is a two-step transition (the same has been observed in the magnetostriction, see above). In a similar line of argumentation, it suggests a transition into a new phase $V$ at low temperatures.

From our previous data we could only roughly estimate the position of these phase border lines and the associated tricritical points. With our magnetostriction data we can more accurately define the evolution of the phase border lines, as is done in Fig. . Notably, the new data suggest that for $B \parallel c$ axis the border line of phase $I$ exhibits a shallow maximum around $\sim$ 20 K/25 T. Such behavior would be highly unusual for a common antiferromagnet. Instead, it appears that the competition with the high field phase $IV$ produces this anomalous evolution of the phase border line. Hence, our new data are fully consistent with the phase diagram scenario labelled "A" in Ref. . Within this scenario, we conclude that the upper tricritical point lies at around 25 T and 19 K. Unfortunately, close inspection of our various data sets utilizing different experimental tools does not provide a clear-cut signature unambiguously defining the tricritical point. As well, the precise evolution of the border between phase $IV$ and $V$ is rather awkward, as is the detailed structure of the area around the tricritical phase $III-IV-V$ point. Ultimately, to unambiguously establish and define these phase border lines, and to definitely discard scenario "B" from Ref. , it would still require experiments in high magnetic fields carried out as function of temperature. Unfortunately, given the high field range, such experiments are rarely carried out.

# Band structure calculations

For $B \parallel c$ axis the experimental evidence is consistent with the transition into phase $V$ to be an ETT in a correlated electron system. We have observed a significant change of the Hall coefficient at the phase $III-V$ transition for $B \parallel c$ axis, in line with a Lifshitz type character. As well, the first order nature of this transition is consistent with a Lifshitz type transition for an interacting electron system. Now, as a next step, and to complement our experimental study, we have carried out additional band structure calculations, aiming to identify features in the band structure beyond those established in Ref. \[\].

For an ETT, the topological changes in the iso-energy surfaces result from critical points in the band dispersion, *i.e.*, minima, saddle points, and maxima which give rise to van Hove singularities in the density of states. The changes in the topology of the iso-energy surfaces include the appearance or disappearance of small pockets, the formation of voids and the disruption of necks. Therefore, the focus of the present calculations is on critical points in the quasiparticle dispersion of UPt$_{2}$Si$_{2}$. For magnetic field-induced Lifshitz transitions, the critical points have to be rather close to the Fermi energy. All in all, it is thus the occurrence of these pockets, voids or necks that we are searching for in the band structure.

The present calculations assume that there are itinerant $5f$ electrons which form partially filled coherent bands. We analyse the Fermi surface where the energy bands are calculated under the following assumptions about the nature of the $5f$ electrons: We begin by adopting DFT treating all $5f$ electrons as band states. This approximation scheme cannot fully capture the correlation effects. To simulate the latter we calculate the band structure under the assumption that two of the $5f$ electrons are localized while one may be itinerant and hybridize with the conduction states. For simplicity, we first treat all $5f$ channels as equivalent and account for orbital selection in a second step. For the last step, we single out the $5f$ electron in the $j=5/2$, $j_{z}=\pm1/2$-channel as the hybridized electron.

More specifically, the band structure results reported in the present paper were obtained by the fully relativistic formulation of the linear muffin-tin orbitals (LMTO) method . The spin-orbit interaction is fully taken into account by solving the Dirac equation. The results are compared to the relativistic calculations by Elgazzar et al. . Because the heavy fermion compound UPt$_{2}$Si$_{2}$ crystallizes in the tetragonal CaBe$_{2}$Ge$_{2}$ structure, the crystal structure is relatively open and, consequently, the atomic-sphere approximation (ASA) cannot be expected to give a sufficiently accurate description of the electronic band structure. The combined correction term which contains the leading corrections to the ASA alters the conduction bands in a characteristic way and hence cannot be neglected. Exchange and correlation effects were introduced using the Barth-Hedin potential . The band structure was converged for 405 ${\bf k}$-points in the irreducible wedge, whose volume equals 1/16 of the Brillouin zone. The density of states (DOS) was evaluated by the tetrahedron method with linear interpolation for the energies . For the conduction band the DOS was calculated at 0.25 mRy ($\approx$ 0.0034 eV) intervals.

The effective potential seen by the conduction states is approximately constructed as a superposition of contributions, which have spherical symmetry inside atomic and "empty" spheres surrounding lattice or interstitial sites, respectively. The empty spheres should be viewed as auxiliary constructions that permit an improved description of the electron density as well as the potential within the framework of the ASA. In UPt$_{2}$Si$_{2}$ the dominant contribution to the charge in the interstitial region comes from the Pt-$d$ states.

Our calculations are done at the experimental lattice parameters and do not correspond to the equilibrium geometry of an LDA calculation. The total energy evaluated for the experimental structure will therefore exceed its theoretical minimum value. This difficulty, which is generally encountered in metals with strongly correlated electrons, is a direct consequence of the LDA description of these systems in terms of a single-particle picture. This can be seen by considering two limiting cases. First, treating the $f$ electrons as part of the ion core implies that their contribution to binding is neglected. As a consequence, the equilibrium values of the lattice constants are often overestimated. Secondly, describing the $f$ electrons as band electrons yields a relatively narrow, partially filled $f$ band at the Fermi level. The calculated LDA DOS at the Fermi level is large compared with that of ordinary metals. An effective single-particle description such as the LDA predicts an electronic compressibility which is enhanced over that of ordinary metals by the same factor. The behavior anticipated for independent fermions, however, is in contradiction with experiment which yields compressibility values for heavy fermion metals which are comparable to those of ordinary metals. The large electronic compressibility predicted erroneously by the LDA in $f$-metals leads to overbinding (*i.e.*, the theoretical values of the equilibrium lattice constants are too small). It is a direct consequence of the failure of the independent particle picture.

The calculations were done using two energy panels; *i.e.*, two separate LMTO calculations were performed to determine self-consistently the uranium 6$p$ states and the conduction bands, respectively. Treating the U-6$p$ semi-core states as band states accounts for the small overlap between these core states. The resulting narrow bands far below the Fermi energy only hybridize weakly with the conduction bands. This hybridization is then neglected in our method, but its influence on the shape of the potential is taken into account. The charge contributions of the other core states were taken from atomic calculations and kept frozen during the iterative procedure. For the lower panel we included $s-p-d$ angular momentum components in the basis at the U and Pt sites and $s-p$ components at the Si and interstitial sites. For the upper panel we included $s-p-d-f$ components at the U site and $s-p-d$ on the remaining sites.

The Fermi surface obtained by the three approaches regarding the degree of localization of the 5$f$-electrons are summarized in Fig. . We find four bands crossing the Fermi energy, and denote the corresponding FS sheets as 1, 2, 3 and 4. Globally, the LDA result agrees well with the one obtained by Elgazzar et al. apart from the fact that the "appendices" are absent in our sheet 1 (see Elgazzar band labelled 113).

With respect to the FS topology, we find that the FS sheets 3 and 4 are remarkably insensitive to the treatment of the $5f$ states. The number of itinerant $5f$ electrons affects only the sheets 1 and 2. Correspondingly, we have inspected closely the response of these sheets on magnetic field by analyzing the iso-energy surfaces for shifts away from the Fermi energy by $6$ meV. For a magnetic moment $\sim 2.5$ $\mu_B$ as in UPt$_2$Si$_2$ this value corresponds to a magnetic field of $\sim 30$ T.

In particular, for sheet 2 we find a qualitative change of the shape of the FS for such a small energy shift, thus providing direct band structure evidence for an ETT (see Fig. ). Clearly, a void formation/neck disruption is visible as the iso-energy surface is tuned from -$6$ meV to +$6$ meV around the Fermi energy. For FS sheet 1 in the energy range considered we find no topological change.

Following the identification of Fermi surface sheet 2 as the one being topologically affected by magnetic fields of the order of 30 T, in a final step we have specified the character of the itinerant electron by allowing the $5f$, $j=5/2$, $j_{z}=\pm1/2$ channel to hybridize with the conduction bands. Again, the iso-energy surfaces shifted by $\pm 6$ meV against the Fermi energy clearly reflect an ETT, as demonstrated in Fig. . Thus, in our band structure calculations, and assuming one out of three $5f$ electrons being delocalized we find ETT on the Fermi surfaces of a correlated electron system, *viz.*, UPt$_2$Si$_2$, consistent with our experiments.

# Conclusion

In summary, we provide experimental evidence for the possibility of a field-induced first-order Lifshitz type transition in the correlated electron system UPt$_2$Si$_2$ through a combined study of the electronic and structural properties. Furthermore, for the FS, critical points close to the Fermi energy $E_{F}$ are found in the band dispersion when two of the $5f$ electrons are treated as localized, implying that field-induced Lifshitz transitions are to be expected. In contrast, for the all-itinerant model, the critical points leading to Lifshitz transitions are too far from the Fermi energy as to be relevant in an experimental context. Thus, with our study we demonstrate consistence of our experiments with the predictions made based on the dual model of $5f$ electrons for the case of an uranium intermetallic with strong electronic correlations.

Finally, the question arises about the nature of the other magnetic phases in UPt$_2$Si$_2$ for $B \parallel c$ axis. When associating the first-order transition into phase $V$ with an ETT, oppositely the second-order character of the phase $I - III$ transition would signal a more ordinary type of transition. The character of the transition, as seen in the magnetization (see Ref. ), together with the size of the jump of the magnetization, could be consistent with for instance a spin-flop transition. In turn, this observation raises questions about the character of phase $IV$, as it shows up in the magnetization in a similar fashion as the $I - III$ transition. In terms of the magnetization, the difference between phases $III$ and $IV$ is not obvious.

Conversely, following a different line of arguments, and while thermal smearing might prohibit a definite identification, conceptually, phase $IV$ can have the FS topology of phase $I/III$, $V$ or a different one. If the FS topology is not that of phases $I/III$, consequently, there would be multiple Lifshitz transitions in the phase diagram for $B \parallel c$ axis of UPt$_2$Si$_2$. Taking this observation into a more general context, the interplay of spin reorientation/anisotropy and FS topology may give rise to a complex set of field-induced phases in UPt$_2$Si$_2$, and which might bear relevance to related exotic phenomena such as the complex phase diagram of URu$_2$Si$_2$ .


abstract: According to the World Health Organization, distracted driving is one of the leading cause of motor accidents and deaths in the world. In our study, we tackle the problem of distracted driving by aiming to build a robust multi-class classifier to detect and identify different forms of driver inattention using the State Farm Distracted Driving Dataset. We utilize combinations of pretrained image classification models, classical data augmentation, OpenCV based image preprocessing and skin segmentation augmentation approaches. Our best performing model combines several augmentation techniques, including skin segmentation, facial blurring, and classical augmentation techniques. This model achieves an approximately $15\%$ increase in F1 score over the baseline, thus showing the promise in these techniques in enhancing the power of neural networks for the task of distracted driver detection.
author: Nikka Mofid[^1]
`email@example.com`; Jasmine Bayrooti
`email@example.com`; Shreya Ravi
`email@example.com`
bibliography: bibliography.bib
title: LaTeX Author Guidelines for CVPR Proceedings; Keep Your AI-es on the Road:
Tackling Distracted Driver Detection with Convolutional Neural Networks and Targeted Data Augmentation

# Introduction

According to the World Health Organization, distracted driving is one of the leading causes of motor accidents. Every year, 1.35 million people across the world are killed as a result of road traffic and many more are injured . Driver inattention is a major contributor to highway crashes, with the National Highway Traffic Safety Administration (NHTSA) estimating that approximately 25% of police reported crashes involve some form of driver inattention with distracted driving being one of the leading causes . Furthermore, recent studies have shown that drivers engaged in live or phone conversations were less aware of traffic movements around them. Thus, distracted driving often leads to dangerous and sometimes deadly situations, and so finding measures to identify and reduce distracted driving has become a major focus in research.

According to the Center for Disease Control and Prevention, there are three major kinds of distraction: visual, cognitive, and manual . In order to tackle the widespread problem of distracted driving in society, in this study we aim to identify distracted drivers using the State Farm Distracted Driving Dataset by building a model that can identify visual and manual types of distracted driving and classify the form of distraction using pretrained image classification models, data augmentation, and OpenCV based image preprocessing techniques. As it is difficult to visually identify cognitive distraction from single images of drivers, we focus on the other two kinds of distraction.

The input to our algorithm is a $224$ x $224$ x $3$ image. We then use a Convolutional Neural Network (CNN) to output a predicted class characterizing the nature of the driver and their distractedness. The $10$ output classes that our model seeks to identify are: "safe driving, "texting right", "talking on phone right", "texting left", "talking on phone left", "operating radio", "drinking", "reaching behind", "hair and makeup", and "talking to passenger".

Due to a limited number of drivers included in the dataset, creating a model that can generalize well to new drivers and environments not included in the dataset is one of the major challenges we face. We intend to create a robust and generalizable model through a careful crafting of our experiment with the described methodology.

# Related Works

Recent research on improving CNN models have followed an ablation study approach. For instance, Tong He et al used an ablation study to empirically evaluate various refinements' impacts on the final model accuracy, which enabled them to improve ResNet-50's best validation accuracy from 75.30% to 79.29% on the ImageNet dataset . Other recent work on pruning state-of-the-art deep CNNs focuses on optimizing the network structure by removing filters and changing their sizes in order to find good combinations of units to keep in the structure that improve training speed and accuracy . We aim to follow a similar ablation study approach keeping the model constant and focusing instead on changing preprocessing augmentation techniques to provide interpretability for which kinds of techniques improve the classifier's performance. Our ablation approach to this problem sets our work apart from other attempts to solve similar problems. For instance, one study seeks to classify images of drivers into similar categories as ours using a number of different CNN models but using two different datasets, one for night and another for day driving . Our approach has the advantage of greater transparency in techniques but is likely to perform worse on images of drivers at night due to the nature of our dataset. We also use different metrics, namely F1 score, precision, and recall, to gain a deeper understanding of our model's performance beyond accuracy, which has been typically used.

Currently, there are a number of state-of-the-art deep CNN models which would be promising to use for our base model. We consider several papers, including a study developing a distracted driver detection system, based on images taken from a camera observing the driver. In one such study, the researchers experimented with using pretrained VGG-16, AlexNet, GoogleNet, and ResNet models to identify distracted drivers. They found that the ResNet model converged the fastest of the four models, however had the highest rate of misclassifications . Because distracted driving can endanger lives, it is critical to investigate all possible cases of distracted driving, so the model should have a low false negative rate but can have a higher false positive rate. Therefore, considering the results of this study, we decided to use the ResNet-50 model architecture as our base model for the ablation study. The ResNet-50 model has also independently been shown to have good performance in classifying distracted driver behaviors in similar studies .

As mentioned in the [Introduction](#sec:Introduction), one major challenge we face is the limited number of drivers and constant point of view across all images in the dataset. Thus, we are setting out to not only achieve high performance on the test set, which is equally non-representative of real world images as the training set, but also to find promising methods of identifying distracted drivers in more generalized images with different drivers and from slightly different points of view. To address this, we intend to experiment with techniques for improving the generalization of our model. Recent research supports the use of data augmentation for improving model generalization as it exploits domain knowledge to increase the amount of training data and improve the generalization without reducing the effective capacity or introducing model-dependent parameters, in contrast to regularization, which also improves generalization but blindly reduces the effective capacity of the model . We follow data augmentation procedures laid out by Mikolajczyk et al to compare and analyze multiple methods of augmentation on-the-fly for our task of image classification, including classical image transformations like rotating, cropping, and color jittering .

To further combat the challenge of generalizing to different drivers, we need to encourage the model to consider the driver's posture and hand position rather than attending too much to the driver's facial features or skin. Facial blurring via a Gaussian blur of appropriate noise-determined variance has been shown to be an effective method of anonymizing people for privacy or surveillance reasons . We intend to apply a similar simple procedure to discourage the model from overfitting to the individual drivers in our dataset. Additionally, recent work has shown skin color segmentation to be useful for various applications including posture detection and hand gesture analysis . To address the challenges of varying illumination and background clutter that come with skin segmentation, we will use a combination of the chrominance channels of the HSV, YCbCr and Normalized RGB color spaces to achieve better accuracy for skin segmentation than in just single color spaces .

# Dataset and Features

## Dataset Details

We are using the Distracted Driving dataset provided by State Farm, which contains images of drivers, distracted and non-distracted, taken by constant-placed 2D dashboard cameras of dimension $640$ x $480$ pixels in RGB . The dataset contains $22,424$ labelled unique images of drivers that fall into one of the following $10$ distracted driver classifications (as described in [Introduction](#sec:Introduction)): "safe driving, "texting right", "talking on phone right", "texting left", "talking on phone left", "operating radio", "drinking", "reaching behind", "hair and makeup", and "talking to passenger". The data used to label these images are in CSV form that lists the driver subject number with their distraction "class" and unique image identification number. The class distribution of the images is mostly even, but two of the classes ("safe driving" and "reaching behind") vary about $10\%$ from the median number of images per class and one of the classes ("hair and makeup") varies about $20\%$ from the median, which can affect the class predictions that our model prioritizes.

Our problem is technically challenging due to the fact that there are only $26$ unique drivers in the dataset, so we need to be careful not to overfit to the drivers in our training data. In addition, the images of drivers in different distracted poses were taken from frames of videos collected by State Farm, thus many of the images are similar, which further exacerbates the overfitting problem.

## Data Preprocessing

For our baseline model, we utilized ResNet-50 with pretrained weights. In order to enhance model performance, we used preprocessing steps that resized the image to $256 \times 256$ spatial dimensions, center cropped to $224 \times 224$ spatial dimensions normalized, and normalized with a fixed mean and standard deviation (specifically a mean of $[0.485, 0.456, 0.406]$ and a standard deviation of $[0.229, 0.224, 0.225]$). We let our ResNet-50 baseline model perform the feature extraction.

As part of our experimentation, we performed several other preprocessing and image processing steps on our train and test data in order to prevent the model from overfitting on certain features of the driver (i.e the drivers face) and help the model focus on the important parts of the image (i.e the driver's posture). As described in the following sections, we used OpenCV to identify an eye in the image and blurred out a region relative to the eye that approximately covered the face of the driver in order to combat overfitting to the driver's facial features and also performed skin segmentation, highlighting the driver's skin while blacking out the rest of the image in order to draw the models attention to the driver's posture. Afterwards, we ensembled our methods, including classical data augmentation techniques, to see if we could use them in conjunction build a strong classifier.

## Train and Test Split

In our initial approach, we decided to split our data into $80\%$ train data and $20\%$ test data, resulting in approximately $18,000$ train images and approximately $4,500$ test images. When we ran our pre-trained ResNet-50 model with additional fully connected layers for our 10-class classification problem, we got results that seemed too good, and we realized that this was likely because our train and test set were incredibly similar due to the aforementioned limited number of drivers and limitations of video data being treated as individual images. Because of this, the model was overfitting on both the test and train data (despite not seeing the test data during train time) but would likely not generalize well to any new drivers and environments. Thus, we decided to take a new approach with the the train and test split.

We chose $5$ drivers at random from the $26$ (because this is approximately $20\%$ of the data) and included all of their images in the test data. We used data from the other $21$ drivers for the training set, thus ensuring minimal overlap between the two datasets. Using this approach, our baseline results were more realistic and would likely generalize well to new drivers.

# Methods

## Baseline

ResNet has been a state-of-the-art convolutional model since it won the ImageNet classification challenge in 2015. This model presents residual modules as a solution to the problem of optimizing deeper networks, thereby enabling researchers to take advantage of the greater representation power of deep networks while not trading off on training error. These residual modules learn a residual mapping rather than directly trying to fit the desired mapping underlying the data.

The ResNet model includes stacks of residual blocks, consisting of two $3$ x $3$ convolutional layers to learn the residual mapping, as shown in figure 3. The full architecture is shown in the appendix. As described in the [Related Works](#sec:RelatedWorks) section, we chose to use ResNet-50 for our model as it performed well on similar challenges and converged quickly. Thus, we are using a pretrained ResNet-50 as our baseline model with a fully connected layer of hidden size $512$, a ReLU nonlinearity layer, a dropout layer (with probability $0.2$), and two fully connected layers of output size $10$ (for the $10$ distracted driver classes). For our baseline, we trained these last layers of the model using our driver-separated training set with the ResNet-50 specific preprocessing detailed in [Data Preprocessing](#sec:DataPreprocessing). We trained the model for 25 epochs using an Adam optimizer and achieved decent results, as specified later. For the loss function, we use cross-entropy loss with multi-class softmax where M is the number of classes ($10$ for our problem):

$$Loss = -{(y\log(p) + (1 - y)\log(1 - p))}$$ $$-\sum_{c=1}^My_{o,c}\log(p_{o,c})$$

## Classical Data Augmentation

Data augmentation is a popular technique for improving model generalization and preventing overfitting by allowing the validation error to continue decreasing with the training error. This is because augmented data represents a more comprehensive input space, through which the differences between the training and any possible testing sets are more likely to be minimized . Since overfitting and generalization are the two main challenges that we face with our dataset, we decided to experiment with two on-the-fly data augmentation techniques—specifically, random rotation and brightness adjustment.

Since all the drivers in the dataset were facing to the right, we used random rotation augmentation to produce examples of our driver rotated slightly between $0$ and $45$ degrees in both directions. In order to further increase our dataset size, we decided to add a second augmentation introducing a small, random amount of color jitter (characterized by brightness, saturation, contrast, and hue adjustments) to add a non-spatial based form of noise into our dataset. It is important to note that we do this data augmentation "online" or on the fly, performing the transforms using the torch transforms library each time the train and test data is loaded into our model.

The input to our random rotation and brightness adjustment transforms is the images from the State Farm Distracted Driving Dataset, resized and cropped to fit the ResNet-50 standard size of $224$ x $224$ x $3$ and with the appropriate color normalization, as explained in . We use the Pytorch transforms library in order to perform this random rotation and color jitter. The output of this procedure is two sets of the transformed input images. We then concatenated these transformed image sets with our original input images and shuffled, giving an augmented set of three times the size of our original training dataset. Numerically, using these two augmentation techniques in tandem, we produced a training set of $55,032$ images from the original training set of $18,344$ images.

## OpenCV Facial Blurring

In order to further combat the challenge of overfitting to the drivers and generalizing to different kinds of images, we need to encourage the model to consider the driver's posture so that it does not memorize patterns in the appearance of the 21 drivers' faces. As described by T. Muraki et al, facial blurring via a Gaussian blur is an effective way of anonymizing people for privacy purposes . The same technique, called Random Blur, can be applied to our problem to blur our the driver's face in order to discourage the model from overfitting to the individual drivers in our dataset. We took inspiration from the random blur techniques and experimented with facial blurring as an augmentation technique on both the training and test sets.

In order to perform facial blurring, for each train and test image, we identified the position of the driver's eyes via a pretrained eye detector from haarcodes in OpenCV . From the eye size and location, we then extrapolated the coordinates to find the approximate area corresponding to the driver's face and blurred out the facial region by smoothing over the image in that approximate area using a Gaussian Blur with appropriate variance. We then used these images in combination with the original input images to assemble new Facial Blur augmented training and test sets for a ResNet-50 model with pretrained weights.

The inputs to our facial blurring script were the images from the State Farm distracted driving dataset resized and cropped to fit the Resnet-50 standards as explained in [Data Preprocessing](#sec:DataPreprocessing). The outputs of this blurring script are all the same images from the train and test sets with the drivers' faces blurred. Unlike the classical augmentation method, we decided to perform this technique offline since it is is more complex and time consuming. Using this technique, we were able to double the size of our training set while improving the generalization of our model.

## Skin Segmentation

After performing our first set of experiments and analyzing the saliency maps from our models, we noticed that the model was focusing on the posture, particularly arm and hand position of the driver, for many of the successfully classified images. Thus, we hypothesized that if we could encourage the model to focus more on the driver's posture we would be able to improve the performance of our classifier and help the model generalize better to new images of drivers.

Skin segmentation is a technique that has been widely used in many biometric applications, face recognition, and hand gesture recognition. It is the act of separating skin and non-skin pixels in an arbitrary image and has been shown to be effective for posture detection and hand gesture analysis, as discussed in [Related Works](#sec:RelatedWorks). There are many different methods of performing skin segmentation. In this study, we focus on the method presented by R. Rahmat et al, which combines the lighting channels of the HSV, YCbCr and Normalized RGB color spaces to obtain high accuracy of skin detection . This method enables us to find a simple, effective, and efficient way to identify the skin of a driver and black out the rest of the image to teach our model to focus on a driver posture.

We implemented a version of this combination of "chrominance" technique based on Will Brennan's Skin Detector, which uses masking and thresholding of RGB (red, green, blue values) and HSV (hue, saturation, value) through the OpenCV library in order to leave the colored the portion of the image which contains skin (falling within the range of RGB and HSV defined by the user) intact and blacks out the rest of the image. In order to detect the range of colors which we would consider "skin", we wrote a python script to allow us to toggle the RGB and HSV values of our input images to find a range of RGB and HSV values that would, on most images, black out the background and leave the skin intact. It is important to note that there is some bias in this technique as it does not work as well on persons of color and is dependent on the lighting in the image, but as there is little ethnic diversity in our dataset and the images are lighted homogeneously, we were able to find an RGB and HSV color range that fit many images.

The inputs for our skin segmentation script were the original set of images from the distracted driving dataset and the ranges of RGB and HSV values, which we would use as the ranges for skin segmentation. The outputs are used as input images to the model and show the original image with only the skin of the driver remaining in color and the background blacked out. As with the facial blurring set, we produced these images offline as the generation was complex and time consuming. We then used these images to augment our training set by resizing and cropping them to fit the ResNet-50 standard size of $224$ x $224$ x $3$ and the appropriate color normalization, and then concatenating them with our original input images and shuffling. Through this technique we were able to double the size of our dataset while also teaching our model to focus on the driver's posture and less on any background noise.

## Combining Preprocessing Augmentation Techniques

Finally, through our experimentation, we found that we had developed several different powerful pre-processing augmentation techniques. In order to round out our ablation study, we decided to explore combining these techniques to see if a specific combination of pre-processing augmentation steps could boost our model's performance. As such, we experimented with different combinations of the techniques by concatenating together the appropriate datasets we had generated for facial blurring and/or skin segmentation and generating the classically augmented images on-the-fly. We found that by combining a number of techniques, we were able to achieve high scores on our multi-class classification task, as described further in the [Results and Discussion](#sec:Results) section.

# Results and Discussion

## Hyperparameters

We tuned a few main hyperparameters such as learning rate, mini-batch size, and probability of dropout in the final dropout layer (introduced as part of the final classification layers of the model). We started the learning rate and betas of the Adam optimizer with the default recommended values ($10^{-3}$ for the learning rate and $[0.9, 0.99]$ for the betas), and tuned the learning rate to be $3*10^{-3}$ because it allowed for faster training with slightly improved results. We used a mini-batch size of $64$ images to fit the memory requirements of our CPU and GPU and allow for faster convergence while also being large enough to have minimal noise in the calculated gradients. Finally, we chose the dropout probability to be 20% (meaning there is an 80% chance of retaining a given unit) based on the recommendations of the paper that introduced dropout as a regularization technique .

## Evaluation Metrics

We used F1 score to evaluate our best model as well as accuracy, precision, and recall as metrics to gain a quick understanding of where our model tended to fail, which then informed the further quantitative and qualitative metrics we examined.

F1 score is a measure of model performance that gives a summary of the precision and recall metrics and is calculated as follows:

$$\texttt{F1} = 2 \times \frac{\texttt{precision} \times \texttt{recall}}{\texttt{precision} + \texttt{recall}}$$

Precision gives a measure of how many of the images predicted to be in a given class $c$ actually belong to that class, and is calculated as follows:

$$\frac{\sum_{c \in C} \texttt{true\_positives}_c}{\sum_{c \in C} \texttt{true\_positives}_c + \texttt{false\_positives}_c}$$.

Recall gives a measure of how many of the images that belong to a given class $c$ are actually predicted to be in that class, and is calculated as follows:

$$\frac{\sum_{c \in C} \texttt{true\_positives}_c}{\sum_{c \in C} \texttt{true\_positives}_c + \texttt{false\_negatives}_c}$$.

## Quantitative Results

The results of our experiments are detailed in the table in Figure . We also generated the Confusion Matrices in Figure and the Appendix to provide more insight into our model's behavior. Please refer to the [Methods](#sec:Methods) section for further explanation of our experiments.

## Qualitative Results

The Saliency maps for our different model are shown below in Figure .

## Quantitative Results Discussion and Analysis

### Results Table Analysis

Looking at Figure , one can see a breakdown of our ablation study to build a strong classifier for Distracted Driving. It is clear that adding pre-processing data augmentation techniques had a significant positive impact on performance.

First of all, we found that using data augmentation significantly improved the performance of the ResNet-50 model. This is likely because data augmentation builds up the model's tolerance to noise so it can better generalize to new images in the test set. This is particularly important for addressing our dataset limitations of having very few drivers present in our overall dataset and discouraging the model from memorizing patterns specific to these individuals. We observed a considerable increase in performance (${\sim}10\%$ as a result of augmenting the data using the techniques described in [Classical Data Augmentation](#sec:DataAugmentation). This is likely because we used augmentations that most similarly represented data that the model would likely encounter, such as slight rotations and slight color changes.

For our next experiment, we directly blurred out the driver's face to prevent the model from learning about facial specific features. We believed this could improve our model's performance over the baseline by teaching the model to focus more on the posture of the driver rather than the face, a more salient feature. However, we found that this actually led our performance to decrease, leading us to conclude that the drivers face's contains important information that the model uses to make its predictions, perhaps by tracking eye placement. So instead, we used facial blurring as an augmentation technique so the model would learn to treat facial features as noise. This approach gave an improvement on the baseline and seems promising as an augmentation technique in conjunction with rotation and color jitter. We hypothesize this is because it allows the model to learn the key facial features necessary while also learning to focus on posture as well.

Seeing improvement in our results by augmenting with blurred train data, we decided to explore augmenting with skin segmented data as detailed in [Skin Segmentation](#sec:skinseg). The model performed slightly worse than the baseline when augmented with only skin segmented data, but achieved the best performance when augmented with skin segmented data in tandem with our classically augmented and facial blurring dataset. We think that, with skin segmentation augmented images alone, the model performed slightly worse because it needed some information from the background, particularly the steering wheel, which we fully blacked out and that skin segmentation augmented images alone did not help solve the overfitting problem on the train set.

Thus, we conclude that our final model, which combines all the image pre-processing methods, performs best because together the images help our model optimally combat the overfitting problem while still providing enough information about salient parts of the image to optimize its classifications. This model also likely performs so well because it receives the most data as the input is augmented by classical augmentation techniques of random rotation and brightness as well as blur augmentation and skin segmentation augmentation, leading the input data to be four times as large as the original training set size.

### Confusion Matrix Analysis

A confusion matrix shows the summary of predictions for a classification problem with the x axis showing the model's class predictions and the y axis showing the true class labels. Each element $(a,b)$ in the confusion matrix shows the probability of the true class being $b$ given the model predicted the class to be $a$. In Figure we show the confusion matrices for a few of our different experiment models.

The expectation of the probability for a random model to predict a given class correctly is $10\%$. The confusion matrix for the ResNet-50 baseline in Figure illustrates that our baseline model performed significantly better, predicting any given class correctly with a probability of over $45\%$ for $9$ out of $10$ classes. We did notice however some concerning results: that the probability of correctly classifying a class 8 image, "hair and makeup", was only 26.1%, with these images being misclassified as class 4 and class 2 images with 20.26% and 15.38% probabilities respectively. Although classes 2 and 4 are likely to be confused with each other because they both represent drivers talking on the phone, it is concerning that the model confuses these classes with "hair and makeup". We address this problem using data augmentation because class 8 had the least training data in the original dataset.

We observe that the class probabilities of correct classification improve as a whole when using the full ensemble model, consisting of facial blurring and skin segmentation images in addition to classically augmented images. For instance, the probability of correctly classifying class 9 images, "talking to passenger", increases from 48.34% in the baseline to 67.39% in the ensembled model. Similarly, we see that adding more positive training samples for class 8, which had the least amount in our original dataset, improved its classification accuracy from 26.10% in the baseline to 41.28% in the ensembled model. Interestingly however, the probability for correctly classifying class 3 images drops from 91.75% in the baseline model to 79.88% in the ensembled model. Although this may seem like a drop in performance, the original high accuracy was likely due to the model not predicting images to be class 3 very often, which is why the probabilities in the class 3 row other than the the correct class are high for the baseline model, but are low for the ensembled model. This means that the ensembled model still did better in differentiating between the classes and more often predicted class 3, meaning our ensembling steps have helped increase the robustness and generalization of our model.

We also observed the trend that, across most of our confusion matrices, safe driving has a fairly low false positive rate except for class 1 images in some cases. This means that identifying distracted driving has a low false negative rate but a higher false positive rate, as we had hoped in the [Related Works](#sec:RelatedWorks) section.

## Qualitative Results Discussion and Analysis

We generated the saliency maps shown in Figure in order to see what parts of the image the model attended to, how that may have affected the model's behavior, and how we could address the issues with our different experiments. Considering the saliency map of an image correctly classified as safe driving, we observe that the model was primarily focusing on the arms and posture of the driver.

Seeing that this was a successful signal for the model, we decided to amplify it, at first by blurring the faces of the driver and adding those images as a supplement to the training images in order to teach the model to treat the faces as noise and not overfit to them so that the model would focus more on posture of the driver when identifying distraction. Considering the saliency map for the output of the blur augmented images model, we see a higher concentration of red on the arms of model and key chest and neck points, which shows that the model focused more on the body of the driver as desired. This is likely one the reasons that the model which used the blur augmented images performed even better than the baseline.

Along the same line of reasoning, we decided to augment the input images to the model with skin segmented images, where only the drivers' posture was highlighted and the rest of the image was blacked out as shown in Figure . As shown in the results table, we actually found that augmenting with skin segmented images alone led the model to do slightly worse than the baseline. Looking at the saliency map, this is likely because the model was trying to make sense of the blacked out portions of the images while also focusing more on the arms and posture of the driver. Thus, the model may have overfit to these blacked out regions, which we had intended to be ignored, thereby causing the model to perform worse than the baseline.

Therefore, the model did the best using a combination of all the different pre-processing augmentation techniques since the saliency maps show that the model focused very clearly on the driver's posture while ignoring the background and paying less attention to the face. In turn, the model did not overfit to the train images, which we hypothesize is because the classically augmented images provided the model with more of a variety to train on. Looking at the saliency map, it is clear to see that with a combination of all our data pre-processing augmentation techniques, the model learns to focus on the most salient features of the image and ignore noise, thus leading it to perform very well.

# Conclusion and Future Work

As shown in [Results and Discussion](#sec:Results), we observed significant improvement over our baseline when augmenting the dataset with random rotation and brightness adjustment, facial blurring, and a combination of all augmentation methods in addition to skin segmented images. The final method described achieved the highest scores with an F1 score of $0.662$. This combination of all pre-processing augmentation methods performed the best as the augmented images highlighted different key features which helped combat the overfitting problem and enabled the model learn to generalize well while focusing on the most important features for classifying distracted driving. Additionally, using all these pre-processing augmentation steps quadrupled the original training set size, which helped boost performance by providing more learning examples. Our study shows that through thoughtful image processing techniques and careful analysis of model behavior, one can teach a model to focus on important characteristics salient to the given problem while reducing overfitting, hence leading to a more robust model.

For future work, we are interested in experimenting with changes to our model's architecture such as adding in Spatial Transformers to allow spatial manipulation of data within the network and increase the model's spatial invariance to the input data. Spatial transformer networks have been shown to be effective for mitigating reliance on pre-set data augmentation techniques while improving model generalizability and have been integrated into standard models like PointNet . We hope that inserting the spatial transformer modules into the ResNet-50 architecture will similarly improve our image classifier by enabling the model to learn invariance to scale, rotation, croppings, and non-rigid deformations. We are also interested in exploring unfreezing layers of our model, experimenting with more powerful ResNets and other pretrained models, and ensembling different pretrained models in combination with our preprocessing data augmentation techniques to try to further boost performance.

We are very excited for future possibilities building on the work we have laid out in this study!

# Appendix

[^1]: Equal Contribution, Stanford University


**Molecular dynamics simulation of melting of finite and infinite size silicene**

Tjun Kit Min$^a$, Tiem Leong Yoon$^a$, Thong Leng Lim$^b$
$^a$ Faculty of Engineering and Technology, Multimedia University, Melaka
$^b$ Malaysia School of Physics, University Sains Malaysia, Penang, Malaysia

We report the melting temperature of free-standing silicene by carrying out molecular dynamics (MD) simulation experiments using optimzed Stillinger-Weber (SW) potential by Zhang *et al.* . The melting scenario of a free-standing silicene is well captured visually in our MD simulations. The data are systematically analyzed using a few qualitatively different indicators, including caloric curve, radial distribution function and a numerical indicator known as 'global similarity index'. The optimized SW potential consistently yield a melting temperature of 1500 K for the simulated free-standing, infinite silicene.

: Molecular dynamics, Melting, Silicene, Optimized Stillinger-Weber potential

# Introduction

Silicene, a two-dimensional nanosheet made up of silicon atoms arranged in a honeycomb lattice, has been predicted theoretically by Takeda and Shiraishi in year 1994. Subsequent DFT calculations such as and revive the interest of the prediction by showing that silicene is indeed energetically stable, and a feasible possibility. Silicene, unlike grapehene which prefers sp2 hybridization, is not flat. Rather, due to the preference of the bounding for sp3 hybridization, the silicene sheet has a buckled configuration, where the out-of-plane buckle parameter is predicted to be 0.44 Å according to DFT calculations. Having a close resemblance to graphene, silicene offers many possibilities as a functional material of advanced applications, such as photovoltaic, and optoelectronic devices , thin-film solar cell absorbers beyond bulk Si and hydrogen storage . One advantage of silicene over other 2D materials is that it is in principle easier to get integrated into nanodevices which are mainly silicon-based.

Silicene is a relatively new 2D material, and was first synthesized on supported substrates in a series of discovery since 2007 . Following the successful synthesis of silicene on supported substrate, many theoretical studies and simulations on the structural, mechanical, electronic and thermal properties of silicene on supported substrate have been published .

The structural properties of a free-standing silicene sheet get modified when grown on a substrate. The silicene experimentally synthesized so far are not free-standing but sitting on a substrate. We have yet to see any report of experimentally synthesized free-standing silicene. Having said that, investigation of free-standing silicene serves the purpose to understand the pristine system in the absence of interactions with surfaces. The understanding of the basic properties of silicene without the interference from surface interaction with substrate shall provide useful insight for higher level manipulation of silicene, such as the 'van der Waals' heterostructures envisaged by .

As far as we are aware of, very limited work on the melting behavior and thermal stability of free-standing silicene is reported in the literature. Bocchetti *et al.* simulated the melting behavior of free-standing silicene via Monte Carlo method with original and a modified version of Tersoff potential parameter set (known as ARK) for silicon atom . According to Bocchetti *et al.*, original Tersoff parameters for silicon atom results in a melting of the free-standing silicene at 3600 K, meanwhile the melting temperature obtained using ARK parameter set is $\sim$ 1750 K. Berdiyorov *et al.* simulated the influence of defect on the thermal stability of free-standing silicene via MD using Reactive force-field (ReaxFF) . He found that pristine silicene is stable up to 1500 K. As a general observation, melting properties and thermal stability of free-standing silicene obtained in MD simulations varies from cases to cases depending on the details of the simulation procedure. Most importantly, the simulation results are strongly forcefield-dependent. Apart from simulating thermal stability, MD simulation has been also been applied to investigate or predict thermal conductivity of both free-standing silicene. A wide range of potentials is employed in these simulations. For example, Zhang *et al.* developed a set of Stillinger-Weber potential parameters specifically for a single-layer Si sheet to simulate the thermal conductivity . Others use Tersoff potential with original parameter sets. In this paper we wish to contribute to the existing knowledge about the melting temperature of free-standing silicene sheet by carrying out MD simulations using modified Stillinger-Weber (SW) potentials. Our MD simulations are carried out using the LAMMPS package . To our best knowledge, optimized SW forcefield has never been used for simulating the melting scenario of an infinite silicene sheet.

# Methodology

To obtain an infinitely large free-standing silicene sheet, we first construct a bulk Si structure using a diamond unit cell with a lattice constant of 5.431 Å. This is done by replicating a total of $30 \times 30 \times 5$ diamond unit cells to form a simulation box that is subjected to periodic boundary condition. The structure is then re-oriented such that the (1 1 1) surface is aligned along the $z$-axis. This is done because the Si diamond lattice in the (1 1 1) direction resembles a silicene sheet with a non-zero buckling parameter. After re-orientation, all atom-containing $xy$-planes perpendicular to the $z$-axis are removed from the bulk until only a single sheet is left. A total of 72000 atoms is removed in the process. The removal of the silicon atoms creates a region of vacuum with 41.5 Å thick along the $z$-direction on both sides of the remaining silicon atom plane, which is now resembling a free-standing silicene sheet with surface area of $130.56 \times 150.76$ Å$^2$, consisting of 9600 atoms. The sheet is infinite in extent (in the $x$- and $y$-directions) due to periodic boundary condition imposed in the MD simulation.

The data file of the free-standing silicene sheet is then inserted into the LAMMPS package to determine its melting point by carrying out simulated annealing simulations. SW potential is well known for its wide usage in semiconducting materials. It is first developed in 1985 for various phase of silicon crystal and it is in good agreement in predicting various physical properties regarding silicon crystals. The formalism of SW and its detail parameters is referred to the original Stillinger *et al.* paper . As it turns out, during our trial attempts to perform silicene melting simulation, conventional SW parameters of , as implemented in the LAMMPS codes, do not produce stable free-standing silicene sheet. On the other hand, using modified version of silicon SW potential by Zhang *et al.* (the so-called optimized SW), we are able to obtain a stable 2D honeycomb structured silicene sheet upon energy-minimization at temperatures below its melting point. Moreover, the sheet turns out to have a buckling value of 0.44 Å out of the $xy$-plane. The average bond length of Si-Si after energy minimization is 2.4 Å. Hence, the optimized SW silicon potential by is used exclusively in the present work. The details of the procedure are summarized as follow:

1.  The free-standing silicene is relaxed via conjugate gradient minimization (energy minimization) technique. The resultant structure is depicted in Fig. .

2.  Simulated annealing procedure is proceeded in NVT ensemble using the Nose-Hoover thermostat with a time step $\Delta t=0.5$ fs. The system is then equilibrated at 1 K for a period of $5\times 10^4 \Delta t$. Temperature is then raised to 300 K at a heating rate of $1 \times 10^8$ K/s. After reaching 300 K, the system is equilibrated for a period of $2.5 \times10^4 \Delta t$.

3.  The stage is now set for raising the system's temperature to a large target temperature $T$ (one at which the silicene will surely melt). To raise the temperature from 300 K to $T$, we have to fix a heating rate. We have performed a convergence test by running trial MD melting simulations to obtain the melting temperature of the free-standing silicene at various heating rates by fixing the target temperature $T=2000$ K. From our convergence test, it is found that by choosing a heating rate of 1.7$\times10^{12}$ K/s, resultant melting temperature obtained is insensitive to variation in the heating rate pegged at the chosen value. The system is heated from 300 K for a total $2.0\times 10^6 \Delta t$ (equivalent to $1$ ns) at the mentioned heating rate until it reaches 2000 K.

4.  Evolution of the silicenes's MD trajectory in the simulation is monitored visually as well as quantitatively. Radial distribution function, and caloric curve of the silicene sheet are numerically sampled and measured. In addition, we also measure a numerical descriptor, known as 'global similarity index', $\xi _{i}$, to gauge the melting process. Detailed description of the global similarity index is discussed in the following.

The definition of the global similarity index is based on generic chemical similarity idea for detecting configurational changes along the trajectory during the heating process. The evolution of the global similarity during the simulated annealing process, which carries statistical information of geometrical variations in the melting mechanism of the silicene will be elaborated in Section . The functional form of $\xi_{i}$ is proposed to take the form of $$\label{xi}
\xi_i={1 \over n}\sum_{s=1}^{n}(k_{s,i}+1)^{-1}$$

$$\label{ksi}
k_{s,i}=\lvert \sqrt{d_{s,i}} - \sqrt{d_{s,0}} \rvert$$ where $d_{s,i}$ and $d_{s,0}$ represent the sorted distance of atoms relative to the average positions (center of mass) of all the atoms in the system in the $i$-th (denoted as the subscript $i$) and the 0-th "frame" during a MD simulation, while $n$ corresponds to the number of atoms, which is an integer equals to the number of pairs of $k_{s,i}$. The value of $\xi_{i}=1$ corresponds to total identicalness between the configuration of the system in the $i$-frame as compared to that in the zeroth frame (which in practice is chosen to be a reference frame in which the system appears in a relatively perfect and stable configuration). $\xi_{i} \rightarrow 0$ infers vast difference.

Having defined the similarity index in Eqs. , , one can optimize the sensitivity of the index with respect to degrees of freedom corresponding to few parameters such as translation and rotation of the particles relative to one another. Note that the average positions, a.k.a., mean, that enters the definition of the parameter $d_{s}$ can be non-uniquely defined. Different definitions of mean capture different aspects of configurational information contained in the system. Several definitions of mean are possible, namely, (a) arithmetic mean \[see Eq. \], (b) harmonic mean \[see Eq. \] and (c) quadratic mean \[see Eq. \].

$$\label{am}
\bar x = \frac{x_1 + x_2 + \cdots + x_n}{n}$$

$$\label{hm}
\bar x = \frac{1}{
{1 \over x_1 }+ {1 \over x_2 } + \cdots + {1 \over x_n }
}$$

$$\label{qm}
{\bar x} = \sqrt{\frac{x_1^2 + x_2^2 + \cdots + x_n^2}{n}}$$ However, during a melting process, variations in the configuration of the atoms *a priori* are not known. In principle, variation in a certain mode of motion among the atoms could be more sensitively picked up by a particular definition of mean then the other. To cover all possibilities, the parameter $d_{s}$ that enters the definition of $\xi_{i}$ in Eqs. ,  is calculated by averaging overall of the above three types of mean, i.e., $${\bar \xi_{i}}=\frac{1}{N_{\mathrm{COR}}} \sum_{\mathrm{COR}}^{} \xi _{i}^{\mathrm{COR}},$$ where $\mathrm{COR}$ stands for center of reference, $\mathrm{COR}$=$\{$arithmetic mean, harmonic mean, quadratic mean$\}$, $N_{\mathrm{COR}}=3$, $\xi_{i}^{\mathrm{COR}}$ is global similarity index defined based on a $\mathrm{COR}$ mean. Including all types of $\mathrm{COR}$ can in principle increase the sensitivity as well as accuracy of the index in capturing variations in the geometrical configuration of the system. Similarity index of the silicene frames in the MD simulation during its heating process is captured in successive temporal sequence. If the silicene sheet remains intact, its geometrical configuration should be very close to the one at the initial frame, i.e., ${\bar \xi_{i}} \approx 1$. If the silicene melts, we would expect ${\bar \xi_{i}}$ to deviate from 1 and in the extreme case, becomes 0. In the following we shall drop the bar symbol in the definition of $\xi_{i}$ when the index is referred. As it turns out, chemical similarity is rarely used for very large system In the present case, the system being simulated consists of 9600 Si particles. This is considered a moderately large one. Global similarity index (which is a modified form of chemical similarity index) as defined above has a high sensitivity to detect even minor distortions in the geometrical configuration of silicene, and hence a very convenient tool to pinpoint the location of melting point during the temporal evolution of the MD.

It is commented that caloric curve and the other two quantities, $g(r), \xi_i$, are two qualitatively different indicators for gauging the melting process. The former is a thermodynamical quantity whereas $g(r), \xi_i$ embed only geometrical information of how atoms distribute in 3D space.

# Results and discussion

We first present the results of pair-correlation function, $g(r)$, measured in the simulation. Each pair-correlation function shown in Fig.  is obtained by sampling the data in the MD simulation at that specific temperature. At room temperature, a sharp peak is measured at $\sim$`<!-- -->`{=html}2.5 Å, indicating an average bond length of silicene at this value. As temperature increases, the peak at 2.5 Å is lowered and begins to widen.

The $g(r)$ curves with $T\leq 1400$ K are clearly distinctive from those with $T\geq 1500$ K. The former has their first peaks located at 2.5 Å, while subsequent peaks are also clearly visible. In addition, $g(r)$ is approximately zero between the first and second peaks. This group of curves essentially share a qualitatively similar geometrical distribution. We hence conclude that silicene sheet with temperature $T\leq 1400$ K remains intact and has not shown signs of complete melting.

However, for curves with $T\geq 1500$ K, the first peak shifted to a larger position at $\sim$`<!-- -->`{=html}2.8 Å, and the secondary peaks disappear. In addition, $g(r)$ beyond the first peak levitated to non-zero values, and remains relatively flat and constant beyond $\sim 3.0$ Å. The disparity between these two groups of curves indicates that a qualitative change in the geometrical distribution of the silicene sheet has occurred when temperature crosses the 1400-1500 K benchmark. At 1500 K melting process has taken place. The features of the $g(r)$ curves for $T\geq 1500$ K are indicative evidence that the silicene sheet has melted into a liquid state at $T\geq$`<!-- -->`{=html}1500 K. The liquid state of the melted silicon system beyond 1500 K can be visualised in the last few snaphots in Table . These are liquid blobs made up of silicon atoms sticking together with an average bond length of $\sim$ 2.8 Å.

Figures shown in Tables  and visually depict progressive melting process of the silicene sheet in the simulated annealing simulation obtained when the system is heated from 300 K towards 2000 K. At room temperature, the silicene sheet is thermally stable and maintains its hexagonal shape at 300 K. It is not perfectly flat though, with some puckering distortion along the direction perpendicular to the surface if viewed from the side. The equilibrium bond length of Si-Si is 2.5 Å. At room temperature, the Si particles vibrate at their respective equilibrium positions, maintaining a defect-free silicene sheet. As the temperature increases, the particles begin to vibrate more violently and vigorously around $\sim$`<!-- -->`{=html}1100 K. Visually, it can be observed that vacancies begin to appear at selected areas on the silicene sheet as Si particles are ejected from their existing sites due to the thermal energy gained from the reservoir. At this temperature, however, the structure remains overall intact. For temperature at or above 1500 K, disintegration of the silicene sheet is clearly sighted. For temperatures less than the melting temperature, no sighting of silicene melting is observed. In other words, visual sighting provides another direct indication of melting at and above $T=1500$ K.

Caloric curves sampled for the simulated silicene sheet at various temperatures provide an independent measurement of the melting point. It is seen that in Fig.  an abrupt drop in the potential energy plot occurs at $\sim$`<!-- -->`{=html}1500 K. From here onward, the sheet begins to tear apart and lose molecular cohesion, signaling the onset of full-fledge melting process. Concurrent to the abrupt drop in the caloric curve, disintegration of the sheet at $\sim$`<!-- -->`{=html}1500 K is also visually sighted in MD simulation video. The atoms of the silicene sheet break apart, forming disconnected 3D amorphous blobs, such as that displayed in the last few snapshots in Table .

The melting process is now analyzed in terms of global similarity index, which is yet another independent measurement of the melting temperature. The structure during heating process is frame-by-frame compared with the reference structure at 300 K (the '0-th frame'). The curve of global similarity index as a function of temperature is shown in Fig. . As temperature increases, the index gradually decreases. At 1500 K the global similarity index drastically drops from $\sim 1$ to 0.02, inferring that the configuration of the system at or above 1500 K is drastically different from that at the reference configuration at 300 K. The melting point as indicated by global similarity index coincides with that obtained from other gauging indicators discussed above.

At this point we wish to comment that the emergence of local point vacancies, visually sighted at $\sim1100$ K, when silicon atoms leaving their existing sites, cannot be effectively captured by all melting indicators.

The final structure obtained when the system is quenched to 300 K is as shown in Fig. . It is found that the resultant structure does not revert to its original planar form. Instead, these atoms condensed into some kind of 3-D amorphous "columns".

# Conclusions

We have performed a MD experiment to measure the melting point of a finite but large free-standing silicene sheet using optimized SW potential by Zhang *et al.*. The melting scenario of the silicene sheet has been visually captured. Analysis of the statistical data sampled from the MD simulation, including caloric curve, radial distribution function and global similarity index, reveals that the silicene sheet melts at 1500 K. This result serves the purpose of being an independent measurement of the melting temperature of free-standing silicene. Our value is consistent with that obtained by Berdiyorov *et al* who uses ReaxFF , but is in contrast to which predicts 1750 K using ARK parameter set. The prediction of free-standing silicent sheet will serve as a reference information to be verified by experiments when they become available.

Simulated annealing of silicene by using optimized Stillinger-Weber potential. Initially the silicene has equilibrium bond length of 2.5 Å. An abrupt change can be viewed at 1500 K in which the sheet is tearing apart and the melting process has begun. After the melting point, the Si particles settle down to form four smaller "islands" which has equilibrium bond length of 2.8 Å.

300K

1100K

1200K

1300K

1400K

1500K

 (Continuation from Table 1)

1600K
1700K

1800K

1900K

2000K


abstract: The Computational Visual Media Conference series, of which this is to be the first conference, is intended to provide a major new international forum for exchanging novel research ideas and significant practical results both underpinning and applying Visual Media.
author: First Author
Institution1
Institution1 address
`email@example.com`; Second Author
Institution2
First line of institution2 address
<http://www.author.org/~second>
bibliography: cvmbib.bib
title: A Template for CVM Proceedings

# Introduction

With the rapid progress of Internet technology, large-scale visual data can be found on the Internet, bringing significant opportunities for novel processing of visual information, as well as commercial applications. The Computational Visual Media Conference series, of which this is to be the first conference, is intended to provide a major new international forum for exchanging novel research ideas and significant practical results both underpinning and applying Visual Media. The primary rationale for this new conference series is to target cross disciplinary research which amalgamates aspects of computer graphics, computer vision, machine learning, image processing, video processing, visualization and geometric computing. Original research is sought in areas concerning the classification, composition, retrieval, synthesis, and understanding of visual media.

## Date

Computational Visual Media Conference will be held on xxx to xxx.

## Language

English is the official language of the conference.

# About the paper submission

## Paper length

cvm papers may be between 4 pages and 14 pages. Over length papers will simply not be reviewed.

## Draft and final copy

The LaTeX style defines a printed ruler which should be present in the version submitted for review. The ruler is provided in order that reviewers may comment on particular lines in the paper without circumlocution. The camera ready copy should not contain a ruler. (LaTeX users may uncomment the `\cvmfinalcopy` command in the document preamble.)

## Blind review

Many authors misunderstand the concept of anonymizing for blind review. Blind review does not mean that one must remove citations to one's own work—in fact it is often impossible to review a paper unless the previous citations are known and available. Blind review means that you do not use the words "my" or "our" when citing previous work.

## Miscellaneous

Compare the following:

|                     |                   |
|:--------------------|:------------------|
| `$conf_a$`          | $conf_a$          |
| `$\mathit{conf}_a$` | $\mathit{conf}_a$ |

See The TeXbook, p165.

The space after , meaning "for example", should not be a sentence-ending space. So is correct, *e.g.* is not. The provided `\eg` macro takes care of this.

When citing a multi-author paper, you may save space by using "et alia", shortened to "" (not "*et. al.*" as "*et*" is a complete word.) However, use it only when there are three or more authors. Thus, the following is correct: " Frobnication has been trendy lately. It was introduced by Alpher , and subsequently developed by Alpher and Fotheringham-Smythe , and Alpher  ."

This is incorrect: "... subsequently developed by Alpher   ..." because reference  has just two authors. If you use the `\etal` macro provided, then you need not worry about double periods when used at the end of a sentence as in Alpher .

For this citation style, keep multiple citations in numerical (not chronological) order, so prefer to .

## References

List and number all bibliographical references in 9-point Times, single-spaced, at the end of your paper. When referenced in the text, enclose the citation number in square brackets, for example . Where appropriate, include the name(s) of editors of referenced books.

| Name | Performance |
|:-----|:-----------:|
| A    |     OK      |
| B    |     Bad     |
| Ours |    Great    |

An example for using tables.

## Illustrations, graphs, and photographs

All graphics should be centered. Please ensure that any point you wish to make is resolvable in a printed copy of the paper. Resize fonts in figures to match the font in the body text, and choose line widths which render effectively in print. Many readers (and reviewers), even of an electronic copy, will choose to print your paper in order to read it. You cannot insist that they do otherwise, and therefore must not assume that they can zoom in to see tiny details on a graphic.

When placing figures in LaTeX, it's almost always best to use `\includegraphics`, and to specify the figure width as a multiple of the line width as in the example below

       \usepackage[dvips]{graphicx} ...
       \includegraphics[width=0.8\linewidth]
                       {myfile.eps}