Breakdown of Magnons in a Strongly Spin-Orbital Coupled Magnet
Stephen M. Winter Kira Riedl Pavel A. Maksimov Alexander L. Chernyshev Andreas Honecker Roser Valenti




The description of quantized collective excitations stands as a landmark in the quantum theory of condensed matter. A prominent example occurs in conventional magnets, which support bosonic magnons - quantized harmonic fluctuations of the ordered spins. In striking contrast is the recent discovery that strongly spin-orbital coupled magnets, such as α-RuCl_3, may display a broad excitation continuum inconsistent with conventional magnons.
Due to incomplete knowledge of the underlying interactions unraveling the nature of this continuum remains challenging. 
The most discussed explanation refers to a coherent continuum of fractional excitations analogous to the celebrated Kitaev spin liquid.
Here we present a more general scenario.
We propose that the observed continuum represents incoherent excitations originating from strong magnetic anharmoniticity that naturally occurs in such materials. This scenario fully explains the observed inelastic magnetic response of α-RuCl_3 and reveals the presence of nontrivial excitations in such materials extending well beyond the Kitaev state. 


winter@physik.uni-frankfurt.de
Institut für Theoretische Physik, Goethe-Universität Frankfurt,
Max-von-Laue-Strasse 1, 60438 Frankfurt am Main, Germany

Institut für Theoretische Physik, Goethe-Universität Frankfurt,
Max-von-Laue-Strasse 1, 60438 Frankfurt am Main, Germany

Department of Physics and Astronomy, University of California, Irvine, California 92697, USA

Department of Physics and Astronomy, University of California, Irvine, California 92697, USA

Laboratoire de Physique Théorique et Modélisation, CNRS UMR 8089,
Université de Cergy-Pontoise, 95302 Cergy-Pontoise Cedex, France

Institut für Theoretische Physik, Goethe-Universität Frankfurt,
Max-von-Laue-Strasse 1, 60438 Frankfurt am Main, Germany

Breakdown of Magnons in a Strongly Spin-Orbital Coupled Magnet
    Roser Valentí
    December 30, 2023
==============================================================




From magnons in ordered magnets to  phonons in periodic crystals, the appearance of bosonic collective excitations is ubiquitous in condensed phases of matter <cit.>. For this reason, special attention is given to those states that support more exotic collective modes, for which the conventional paradigm breaks down. In the context of magnetic phases, the breakdown of magnons is commonly thought to require closeness to an unconventional state such as a quantum spin liquid <cit.>. A notable example occurs in Kitaev's exactly solvable honeycomb model <cit.>, for which strongly anisotropic and bond-dependent interactions fractionalize conventional spin excitations into Majorana spinons and fluxes. This Kitaev state has recently risen to prominence due to the suggestion that it may be realized in heavy metal 4d^5 and 5d^5 insulators via a specific interplay between the crystal field and strong spin-orbit coupling <cit.> and, consequently, a variety of candidate materials based on Ir^4+ and Ru^3+ have been intensively explored <cit.>. Encouragingly,  evidence of a continuum of magnetic excitations that is inconsistent with conventional magnons was found in the majority of such materials,
including the two-dimensional (2D) honeycomb Na_2IrO_3 <cit.> and α-RuCl_3 <cit.>, as well as the
3D analogues β-,γ-Li_2IrO_3 <cit.>, despite all of them having magnetically ordered ground states.  


While the observed excitation continua in these systems have been interpreted in terms of signatures of the Kitaev state, the low-symmetry crystalline environment of the real materials also allows various additional interactions beyond Kitaev's model <cit.>, which are thought to be large based on both experimental <cit.> and theoretical <cit.>
considerations. In this sense, understanding the mechanism for the breakdown of magnons and the appearance of a broad
continuum of magnetic excitations remains a key challenge. 




In this work, we study a representative case α-RuCl_3, which forms a layered 2D honeycomb lattice and displays zigzag magnetic order below T_N∼ 7 K <cit.>. We specifically address the recent inelastic
neutron scattering (INS) measurements, which have revealed low-energy magnons <cit.> coexisting with an intense excitation continuum <cit.>. The latter continuum possesses a distinctive six-fold star shape in momentum space, and large intensity at the 2D Γ-point over a wide energy range E = 2 - 15 meV <cit.>. 
To resolve the nature of this continuum, we take two complementary approaches. We first theoretically investigate the neutron spectra over a range of relevant magnetic interactions in order to 
determine the correct spin Hamiltonian for α-RuCl_3, which has been a subject
of intense recent discussion <cit.>.
Second, we identify the conditions that lead to the
breakdown of  conventional magnons in the presence of strongly anisotropic and frustrated interactions, revealing that nontrivial
excitations naturally persist well beyond the Kitaev spin liquid.
 



Results 

The model.
Based on previous ab initio studies <cit.>,
the largest terms in the spin Hamiltonian of α-RuCl_3 are generally expected to include
nearest neighbour Heisenberg J_1, Kitaev K_1, and off-diagonal Γ_1
couplings, supplemented by a 3rd neighbour Heisenberg J_3 term: 
    ℋ =    ∑_⟨ i,j⟩ J_1  𝐒_i ·𝐒_j + K_1 S_i^γ S_j^γ + Γ_1 (S_i^α S_j^β + S_i^β S_j^α) 
        +  ∑_⟨⟨⟨ i,j⟩⟩⟩ J_3 𝐒_i ·𝐒_j

where ⟨ i,j⟩ and ⟨⟨⟨ i,j⟩⟩⟩ refer to summation over first and third neighbour bonds, respectively (see Fig. 1). The bond-dependent variables {α,β,γ} distinguish the three types of first neighbour bonds, with {α,β,γ} = {y,z,x}, {z,x,y}, and {x,y,z} for the X-, Y-, and Z-bonds, respectively. The third neighbour interactions are bond-independent. The phase diagram of this model has been discussed previously <cit.>, and
is further detailed in Supplementary Note 1; here we review the key aspects. 



In the
limit J_1= Γ_1=J_3=0, the ground state is a gapless
Z_2 spin liquid for either positive or negative K_1, as demonstrated in Kitaev's seminal work <cit.>. Small perturbations from
the pure K_1 limit may induce various magnetically ordered states, such as the zigzag antiferromagnetic state observed in α-RuCl_3 and  shown in Fig. 1. The simplest perturbation is the introduction of a finite J_1, which yields the well-studied (J_1,K_1) nearest
neighbour Heisenberg-Kitaev (nnHK) model. This model hosts zigzag order in the region K_1 >
0, J_1 <0, as discussed in Supplementary Note 1. Accordingly, previous analysis of the powder INS experiments within the context of the nnHK model <cit.>, suggested that K_1
∼ +7 meV, and |J_1/K_1| ∼ 0.3-0.7 for α-RuCl_3. On this basis, the excitation continua observed experimentally were initially interpreted in terms of proximity to
the antiferromagnetic (AFM) K_1 > 0 spin liquid <cit.>.  However, the further consideration of finite 
Γ_1 and J_3 interactions in Eq. (<ref>) significantly expands the
experimentally relevant region, as both interactions generally stabilize zigzag
order. Indeed, recent ab initio studies <cit.> have
suggested that the zigzag order in α-RuCl_3 emerges from J_1 ∼ 0, K_1 < 0, Γ_1 > 0, and J_3 >0, with
|Γ_1/K_1| ∼ 0.5 - 1.0, and |J_3/K_1| ∼ 0.1-0.5, as reviewed in Supplementary Note 2. That is, K_1 is ferromagnetic, and supplemented by significant Γ_1 and J_3 interactions. Such interactions would represent large deviations from both Kitaev's original model and the region
identified by initial experimental analysis. Before discussing the origin of the excitation continua, it is therefore crucial to first pinpoint the relevant interaction parameters.

In order to address this issue directly, we have computed the neutron
scattering intensity ℐ(𝐤,ω) for a variety of
interactions within the zigzag ordered phase via both linear spin-wave theory
(LSWT) and exact diagonalization (ED). For the latter case, we combine results
from various periodic 20- and 24-site clusters compatible with the zigzag state
in order to probe a wider range of 𝐤-points (see Methods section).
Full results for the complete range of models are presented in Supplementary Note 5. Here, we highlight the key results for two representative sets of interactions. Within the (J_1,K_1) nnHK model, we focus on  (J_1 = -2.2,K_1=+7.4 meV; |J_1/K_1|=0.3), which lies on the
border of the region initially identified in Ref. <cit.>, close to the spin liquid. 
Beyond the nnHK model, we consider
  (J_1 = -0.5,K_1=-5.0,Γ_1=+2.5,J_3=+0.5 meV) for which
parameters have been  guided by recent ab initio studies <cit.>, and further optimized to improve agreement with the experimental spectra. Results for Models 1 and 2 are first presented in Fig. <ref> and Fig. <ref>, which show detailed ω- and 𝐤-dependence of ℐ(𝐤,ω), along with the evolution of the spectra upon changing parameters towards the K_1 >0 or K_1 < 0 spin liquid regions. 



Nearest neighbour Heisenberg-Kitaev (nnHK) model.
We begin by analyzing the spectra  ℐ(𝐤,ω) within the zigzag phase of the (J_1,K_1) nnHK model, starting with Model 1
(Fig. <ref>).
 Despite proximity to the spin liquid, the ED
calculations on Model 1 (Fig. <ref>b) show sharp dispersive modes appearing over the majority of the
Brillouin zone that are consistent with the conventional magnons of
LSWT (Fig. <ref>a). Indeed, for energies below the main spin-wave branch (ω = 1.3-2.3 meV), intensity is localized around the M- and Y-points, corresponding to the pseudo-Goldstone modes of the zigzag order (Fig. <ref>c). ED calculations show clear spin-wave cones emerging from such points and extending to higher energies. Large deviations from LSWT are observed only for the highest energy excitations, which appear near the 2D Γ-point for energies ω > 5 meV. Here, the ED calculations display a broad continuum-like feature centred at ω∼ K_1 that resembles the response expected for the K_1 > 0 Kitaev spin liquid, as shown in Fig. <ref>d. However, comparison with the experimental ℐ(Γ,ω) shows poor agreement; while the experimental intensity extends from 2-15 meV, the ED results for Model 1 predict intensity only at high energies > 5 meV. Indeed, the evolution of the Γ-point intensity with
|J_1/K_1| is shown in Fig. <ref>e. On approaching the K_1 >0
spin liquid by decreasing |J_1/K_1|, excitations at the Γ-point shift to higher
energy, such that none of the parameters in the vicinity of the spin liquid
reproduce the experimental ω-dependence of ℐ(Γ,ω). Similar conclusions can also be drawn from recent DMRG
studies of the nnHK model <cit.>. We therefore conclude that 
the broad features observed experimentally in ℐ(Γ,ω) at relatively low energies <cit.> are incompatible with the nnHK model with J_1 < 0 and K_1>0.




Extended ab initio guided model.
In order to treat the effect of interactions beyond the nnHK model, we consider
now the ab initio guided Model 2. In
contrast to Model 1, ED calculations on Model 2 (Fig. <ref>b), show large deviations from
standard LSWT (Fig. <ref>a) over a wide range of 𝐤 and ω. This model reproduces many of the experimental spectral features <cit.>.
 In particular, sharp single-magnon-like peaks appear only near the
pseudo-Goldstone modes at the M- and Y-points. Elsewhere in the Brillouin zone, broad continuum-like
features are observed within the ED resolution. As demonstrated in Fig. <ref>c, we find significant intensity at low energies (ω < 2.3 meV), at both the Γ- and
(M,Y)-points. For the intermediate energy region (ω = 5.5 - 8.5 meV), ℐ(𝐤)
resembles the six-fold star shape observed in Ref. <cit.>. At higher
energies (ω > 10.5 meV) scattering intensity is mainly located at the
Γ-point, also in accord with experiment. Furthermore, the ED results for the Γ-point intensity ℐ(Γ, ω) show a broad range of
excitations peaked around 4 and 6 meV, and extending up to ∼ 15 meV (Fig. <ref>d). Therefore, ED
calculations on Model 2 reproduce all of the main experimental spectral 
features, validating the range of interactions indicated by ab initio
calculations. The only aspect that is not quantitatively reproduced within the Model 2 is the magnitude of the gap at the M-point (∼ 0.8 meV at the level of LSWT vs. ∼ 2 meV experimentally <cit.>). This discrepancy may result from deviations from C_3 symmetry, which are allowed within the C2/m space group <cit.>, but not considered here for simplicity (see Supplementary Figure 11). Interestingly, the spectral features at the Γ-point become
dramatically sharper on approaching the K_1<0 spin liquid, as shown in the evolution of ℐ(Γ,ω) with the ratio |Γ_1/K_1| (Fig. <ref>e). This result reveals that the broad continuum may not be directly
associated with a proximity to the Kitaev state.





Magnon stability beyond LSWT.
To gain further insight into the reason for such a drastic contrast between the stability of magnons in Models 1 and 2, it is useful to consider possible magnon decay channels in the zigzag ordered phase. At the level of
LSWT, the spin-wave Hamiltonian is truncated at quadratic order, and can be written ℋ_2 =
∑_𝐤,𝐦ϵ_𝐤,m a_𝐤,m^†
a_𝐤,m in terms of magnon creation (annihilation) operators
a^† (a), where ϵ_𝐤,m denotes the dispersion of
the mth magnon band. In this harmonic approximation, magnons represent
sharp, well-defined excitations. However, when higher order anharmonic terms are included, the total magnon number N_tot=∑_𝐤,m a_𝐤,m^† a_𝐤,m is typically not a conserved quantity, such that the stability of magnons is not guaranteed beyond quadratic order. Quantum fluctuations associated with the higher-order anharmonic decay
terms may mix sharp single-magnon modes with the
multi-magnon continuum <cit.>. Similar considerations also apply to the breakdown of other collective modes, such as phonons in anharmonic crystals <cit.>. From this perspective, a large decay rate is expected for any single magnon mode that is energetically degenerate with the multi-particle continuum, unless there are specific symmetries guaranteeing that the two do not couple. It is therefore useful to consider the prerequisites for magnon breakdown in the presence of the strongly anisotropic interactions of Eq. (<ref>). 

Magnon decay channels for the nnHK model. We first examine the stability of magnons in the nnHK model. For pure J_1 and K_1 interactions, the total spin projections S_tot^γ = ∑_i
S_i^γ are conserved along the cubic axes γ = {x,y,z} modulo two.
Since the ordered moment also lies along one of the cubic axes in the zigzag
phase <cit.>
(see Fig. <ref>c), the
possible magnon decay channels are restricted. In the local picture,
the relevant quantum fluctuations are local singlet S_i^x S_j^x |↑↓⟩ = |↓↑⟩ and triplet S_i^x S_j^x
|↑↑⟩ = |↓↓⟩ fluctuations shown
in Fig. <ref>a, with Δ S_tot^z = 0 and 2, respectively. In
the magnon picture, the Hamiltonian can only contain even order terms (i.e.
ℋ = ℋ_2+ℋ_4+...), analogous to conventional
Heisenberg antiferromagnets with collinear ordered spins <cit.>. For example, the fourth order decay process due to ℋ_4 mixes the one-magnon states with the
three-magnon continuum (Δ N_tot = ± 2), where

    ℋ_4 =   ∑_1-4 V^4_123 a^†_1 a^†_2a^†_3a_4 δ(𝐤_1+𝐤_2+𝐤_3-𝐤_4)+H.c.

Here, the bold index (𝐧≡𝐤_n,m_n) labels both momentum and magnon band. This process is pictured in Fig. <ref>b. As noted above, the effect of such terms depends crucially on the availability of low-energy three-magnon states in which to decay. 


The density of three-magnon states for Model 1 is  shown in
Fig. <ref>d, based on the one-magnon dispersions obtained in LSWT. At each 𝐤-point, the lowest energy
three-magnon state a_𝐪_1^† a_𝐪_2^†
a_𝐪_3^† |0⟩, (with
𝐪_1+𝐪_2+𝐪_3 = 𝐤) is obtained by placing
two particles in the pseudo-Goldstone modes at opposite M-points (𝐪_1
+ 𝐪_2 = 0), and the third particle at 𝐪_3 = 𝐤,
with total energy E_3^min (𝐤) = ϵ_𝐤,1 +
2ϵ_M,1. This implies E_3^min (𝐤)  ≥ϵ_𝐤,1. That is, the three-magnon states lie above the lowest
one-magnon band at every 𝐤-point. As a result, every magnon in the lowest band remains kinetically stable, due to the absence of low-energy three-particle states in which to decay. Precisely this condition ensures the stability of low-energy magnons in conventional isotropic antiferromagnets, and explains the sharp magnon-like peaks observed in Fig. <ref>b for Model 1. 
Strong spectral broadening in the nnHK model can occur only for high-lying
excitations with ϵ_𝐤,m > ϵ_𝐤,1=E_3^min,
where the density of three-magnon states is finite, such as at the 2D Γ-point. On approaching the spin liquid (at J_1/K_1 = 0), this condition is relaxed due to the vanishing dispersion of the lowest magnon band (i.e.
ϵ_𝐤,1→ 0), which corresponds to a vanishing energy cost the singlet fluctuations shown on the left of Fig. <ref>a. The relevant fluctuations in the limit J_1/K_1 → 0 therefore correspond to Δ N_tot = ± 2. For other values of J_1/K_1, the majority of magnons are expected to remain stable due to the absence of low-energy three-magnon states.




Magnon decay channels for the extended model. In Model 2, the character of the quantum fluctuations away from zigzag order is notably different
(Fig. <ref>). The finite Γ_1 interaction reduces the local symmetry and
leads to rotation of the ordered moments away from the cubic axes <cit.> (Fig. <ref>c).
 In the local picture, this allows additional
single-spin fluctuations S_i^xS_j^z|↑↑⟩ = |↓↑⟩ (Fig. <ref>a), which correspond to odd-order
anharmonic terms ℋ_3,ℋ_5,... in the magnon Hamiltonian,
where <cit.>:

    ℋ_3 =    ∑_1-3Λ_12^3 a^†_1
    a^†_2a_3δ(𝐤_1+𝐤_2-𝐤_3) + H.c.

At lowest order, such terms mix the single-magnon states with the two-magnon continuum (Δ N_tot = ± 1), via the scattering process depicted in Fig. <ref>b. The density of two-magnon states is shown in Fig. <ref>d, for the zigzag domain with 𝐐 = Y. In this case, at each 𝐤-point the lowest energy two-magnon state a_𝐪_1^† a_𝐪_2^† |0⟩  is obtained by placing one particle in the pseudo-Goldstone mode at an M-point, and the second particle at 𝐪_2 = 𝐤-M, with total energy E_2^min(𝐤) = ϵ_𝐤-M + ϵ_M≠ E_3^min. 
It should be emphasized that this condition differs from that of a conventional Heisenberg antiferromagnet (for which E_2^min = E_3^min) <cit.>. In the case of Model 2, the difference is directly related to the strong anisotropic K_1 and Γ_1 interactions, which shift the pseudo-Goldstone modes to the M-points, such that only high energy magnons remain at the Γ-point or ordering wavevector 𝐐 <cit.>. This shift therefore leads to an offset of the low-energy even and odd magnon states in 𝐤-space such that
E_2^min(𝐤) < ϵ_𝐤,1 over a wide region of the
Brillouin zone; there are many two-magnon states with equal or lower energy
than the one-magnon states. 
Provided there is a finite Γ_1, the
spontaneous decay of single magnons into the two-particle continuum is
therefore allowed even for the lowest magnon band. The decay rate is expected to be particularly large near the zone center, which represents a minimum in E_2^min. 
Similar kinematic conditions may also occur in other systems <cit.>.
For Model 2, the
pseudo-Goldstone magnons near the M-points remain coherent due to the absence
of low-energy two particle states in which to decay (Fig. <ref>d). This explains the experimental observation of sharp magnon-like modes near the M-points <cit.>. In
contrast, the magnon bands in the remainder of the Brillouin zone directly
overlap with the two-particle continuum. It is therefore natural to anticipate a large decay rate even for the lowest magnon bands. 

To confirm this idea, we have computed the three-magnon interactions and decay rates
for all magnon bands for Model 2 using the self-consistent imaginary Dyson equation (iDE) approach <cit.>. Within this approach, it is assumed that the real part of the magnon self-energy is already captured by the LSWT parameters, while the imaginary part is obtained self-consistently (see Methods and Supplementary Note 3). The iDE approach therefore represents an extension of LSWT, in which the one-magnon excitations are broadened according to the momentum and band-dependent decay rate γ_𝐤,n, while other contributions to the neutron intensity from multi-magnon excitations are also absent <cit.>. As a result, comparison of LSWT, ED, and iDE results (Fig. <ref>) allows for the identification of the origin of different contributions to the spectra. 

The predicted neutron scattering intensity within the iDE approach (Fig. <ref>b) captures many of the most notable features that are observed in the ED and experimental data,
showing a significant improvement over the LSWT results (Fig. <ref>a).
First, there is an almost complete washout of the two high-energy one-magnon modes due to strong decays. This implies that the higher-energy features > 4 meV appearing in ED are primarily multi-magnon in character (including the 6 meV peak at the Γ-point). The appearance of these higher energy features in the inelastic neutron response may arise partly from direct contributions from the broadened two-magnon continuum via the longitudinal component of the structure factor, which is not included in the iDE approach (see Supplementary Note 3). Second, the broadening of the two lower magnon bands in the iDE results and the resultant variation of their intensities are in a close agreement with the exact diagonalization – particularly in a wide region near the Γ point (see also Supplementary Figure 5). These are precisely the features with which the LSWT results were most incompatible. Over much of the Brillouin zone – and especially for the higher magnon bands – the computed γ_𝐤,n is on the same scale as the one-magnon bandwidth, confirming the absence of coherent magnons. 



Discussion.

The general requirements for strong two-magnon decays are less restrictive than a proximity to a spin-liquid state. Indeed, a large decay rate is ensured by the following three conditions: large anisotropic interactions, deviation of the ordered
moments away from the high-symmetry axes, and strong overlap of the one-magnon states with the multi-magnon continuum (see Supplementary Note 3). Of these, the first two conditions ensure that the scattering vertex Λ_12^3 is large –  of the order of the underlying interactions, i.e. Λ_12^3∼𝒪(K_1,Γ_1). For α-RuCl_3, the strong overlap with the multi-magnon continuum is ensured by shifting of the low-energy magnons away from the Γ-point. Since the bottom of the two-magnon continuum must always have an energetic minimum at the Γ-point, the shifting of the pseudo-Goldstone modes to a finite momentum ensures the remaining higher-energy magnons are degenerate with the continuum near the zone center. Experimentally, these conditions are also
likely to be satisfied by the zigzag ordered Na_2IrO_3 <cit.>, and spiral magnets α-, β-, and
γ-Li_2IrO_3 <cit.>. This
picture is also consistent with recent indications that the magnetically
disordered phase observed at high pressure in β-Li_2IrO_3 <cit.> is driven primarily by large Γ_1
interactions <cit.>. 

With this in mind, there are two general scenarios that can explain the
observed continuum excitations in α-RuCl_3 and the iridates
A_2IrO_3. In the first scenario, which has been advanced by several studies, the excitations can be treated as free particles with a small number of flavours. Such excitations are weakly
interacting and have well-defined dispersions, but possess quantum numbers (e.g. Δ S_tot = ± 1/2) or topological properties inconsistent with the experimental neutron scattering selection rules (i.e. Δ S_tot = 0, ± 1). The appearance of the broad continuum in energy therefore results only from the fact that these fractional excitations must be created in multiples. If they could have been created individually, they would have represented long-lived and coherent quasiparticles with sharply peaked energies. 
This scenario indeed describes the Kitaev spin liquid, where the special symmetries of the Hamiltonian allow an
exact description in terms of two flavours of particles: non-interacting Majorana spinons and localized fluxes <cit.>. Such excitations are long-lived, but belong to nontrivial topological sectors, and therefore cannot be created individually by any local operations. For the Kitaev spin liquid, the predicted continuum therefore represents coherent multiparticle excitations. 

In contrast, upon moving away from the pure Kitaev point, the relevant symmetries that protect the spinons and fluxes are lifted both by additional magnetic interactions and by spontaneous symmetry breaking of the magnetic order. This tends to
confine spinons into gauge neutral objects such as magnons <cit.>. Despite this latter 
tendency, we have
argued that coherent magnons are unlikely to appear at large Γ_1 due to
the strong anharmonicity in the magnon Hamiltonian. While this leaves open the possibility that
nearly free Majorana spinons persist into the zigzag ordered phase in some regions of the Brillouin zone, a more
general scenario is that the observed continua represent fully incoherent excitations. In this second scenario, the excitations are not describable in terms of any type of free particles with small decay rates and well defined dispersions. The broad continua therefore reflects the absence of coherent quasiparticles altogether, rather than particular experimental selection rules related to fractionalization. 
At present, it is not clear which of
these scenarios applies to the iridates and α-RuCl_3, although a key
role must be played by both the Kitaev K_1 and off-diagonal
couplings such as Γ_1. In any case, the study of these materials calls into question the stability of magnetic quasiparticles in the presence of strongly anisotropic interactions. 


In summary, we have shown that all main features of the
magnetic excitations in α-RuCl_3 <cit.> are consistent with strongly
anisotropic interactions having signs and relative magnitudes in agreement with
ab initio predictions. The ferromagnetic Kitaev coupling (K_1 <0)
is supplemented by a
significant off-diagonal term (Γ_1 >0) that plays a crucial role in
establishing both the zigzag order and the observed continua. In the presence
of such interactions, the conventional magnon description breaks down even deep
in the ordered phase, due to strong coupling of the one-magnon and two-magnon
states. This effect is expected to persist over a large 
range of the phase diagram suggesting that the
observed continua in α-RuCl_3 and the iridates A_2IrO_3 represent
a rich and general phenomenon extending beyond the Kitaev spin liquid. For this
class of strongly spin-orbital coupled magnets, the presence of complex and
frustrated anisotropic interactions leads naturally to dominant anharmonic
effects in the inelastic magnetic response. Fully describing the dynamics of
these and similar materials therefore represents a formidable challenge
that is likely to reveal aspects not found in conventional isotropic
magnets. 


Methods 

Exact diagonalization.
The neutron scattering intensity was computed via:
 
    ℐ(𝐤,ω) ∝     f^2(𝐤)∫ dt  ∑_μ,ν (δ_μ,ν - k_μ k_ν/k^2)×
        ×∑_i,j⟨ S_i^μ (t) S_j^ν (0)⟩ e^-i𝐤· (𝐫_i-𝐫_j)-iω t

where f(𝐤) is the atomic form factor of Ru^3+ from Ref. <cit.>. ED calculations were performed using the Lanczos algorithm <cit.>, on several 20- and 24-site clusters with periodic boundary conditions. Such periodic clusters are detailed in Supplementary Note 4. Excitations were computed using the continued fraction method <cit.>. Further details and additional results are presented in the supplemental material; these extensive calculations go beyond previous ED studies <cit.>, which focused mainly on the static properties, or a limited portion of the phase diagram. ED results shown for the high-symmetry Γ, M, Y, X, and
Γ^' points were averaged over all clusters. The 
ED 𝐤-dependence of ℐ(𝐤,ω), integrated over the 
energy windows E = 1.3-2.3, 5.5-8.5, and 10.5+ meV (Fig. <ref>c and <ref>c), was obtained from 
a single 24-site cluster respecting all symmetries of the model. 
The discrete ED spectra were Gaussian broadened by 0.5 meV, consistent with the width
of experimental features <cit.>. The intensities were also
averaged over the same range of out-of-plane momentum as in the experiment <cit.>.

Linear spin-wave theory.
LSWT results shown in Fig. 1 and 2 were obtained with the aid of SpinW <cit.>. Following the approach with the ED data, the discrete LSWT spectra were as well Gaussian broadened by 0.5 meV and the intensities were also
averaged over the same range of out-of-plane momentum as with ED and in the experiment <cit.>.


Imaginary self-consistent Dyson equation approach.
In order to calculate magnon decay rates γ_ k,n, we have evaluated three-magnon interaction vertices by performing rotation to local reference frames of spins. The obtained value of the real-space interaction is quite large, about ∼ 3 meV. Next, the
Born approximation calculation of the decay rates results in unphysical divergencies <cit.>, thus the
self-energy Σ_ k,n needs to be regularized. We have used the so-called iDE approach: 
a self-consistent solution on the imaginary part of the Dyson's equation, Σ_ k,n(ϵ_𝐤,n+iγ_ k,n)= -iγ_ k,n, see Ref. <cit.>. We have obtained the 
regularized broadening for the magnon spectrum and have calculated the transverse part of the dynamical structure factor, shown
in Fig. <ref>, by adding the calculated decay rates to experimental resolution of 0.25 meV. 
The spectral function is approximated as a Lorentzian. More technical details can be found in the Supplementary Note 3.

Code availabilty 

Custom computer codes used in this study are
available from the corresponding author upon reasonable request. Documentation of
the codes is not available.

Data availability 

Data is available from the corresponding author upon
reasonable request.

Acknowledgements 

The authors acknowledge useful discussions with J. Chaloupka, A. Banerjee, S. E. Nagler, A. A. Tsirlin, R. Moessner, F. Pollmann, and M. Zhitomirsky. S. M. W. acknowledges support through an NSERC Canada
  Postdoctoral Fellowship. R. V. and K. R. acknowledge support by the
  Deutsche Forschungsgemeinschaft through grant SFB/TR 49. The work of P. A. M. and A. L. C. was supported by the U.S. Department of Energy, Office of Science, Basic Energy Sciences 
under Award No. DE-FG02-04ER46174.
  
Author Contributions

R. V. and S. M. W. conceived the project. K. R., S. M. W. and A. H. performed and analyzed the ED calculations. P. A. M. and A. L. C. performed and analyzed the iDE results. All authors contributed equally to the manuscript.

Competing interests

The authors declare no competing financial interests.


10
url<#>1urlprefixURL 


lifshitz2013statistical
authorLifshitz, E. M. & authorPitaevskii, L. P.
titleStatistical Physics
  (publisherPergamon Press, year1980).

balents2010spin
authorBalents, L.
titleSpin liquids in frustrated magnets.
journalNature volume464,
  pages199–208 (year2010).

PhysRevLett.86.1335
authorColdea, R., authorTennant, D. A.,
  authorTsvelik, A. M. & authorTylczynski, Z.
titleExperimental Realization of a 2D Fractional
  Quantum Spin Liquid.
journalPhys. Rev. Lett.
  volume86, pages1335–1338
  (year2001).

han2012fractionalized
authorHan, T.-H. et al.
titleFractionalized excitations in the spin-liquid state
  of a kagome-lattice antiferromagnet.
journalNature volume492,
  pages406–410 (year2012).

kitaev2006anyons
authorKitaev, A.
titleAnyons in an exactly solved model and beyond.
journalAnn. Phys. volume321,
  pages2–111 (year2006).

PhysRevLett.102.017205
authorJackeli, G. & authorKhaliullin, G.
titleMott Insulators in the Strong Spin-Orbit
  Coupling Limit: From Heisenberg to a Quantum Compass and Kitaev
  Models.
journalPhys. Rev. Lett.
  volume102, pages017205
  (year2009).

rau2016spin
authorRau, J. G., authorLee, E. K.-H. &
  authorKee, H.-Y.
titleSpin-Orbit Physics Giving Rise to Novel
  Phases in Correlated Systems: Iridates and Related Materials.
journalAnnu. Rev. Condens. Matter Phys.
  volume7, pages195–221
  (year2016).

gretarsson2013magnetic
authorGretarsson, H. et al.
titleMagnetic excitation spectrum of Na_2IrO_3
  probed with resonant inelastic x-ray scattering.
journalPhys. Rev. B volume87,
  pages220407 (year2013).

chun2015direct
authorHwan Chun, S. et al.
titleDirect evidence for dominant bond-directional
  interactions in a honeycomb lattice iridate Na_2IrO_3.
journalNat. Phys. volume11,
  pages462–466 (year2015).

sandilands2015scattering
authorSandilands, L. J., authorTian, Y.,
  authorPlumb, K. W., authorKim, Y.-J. &
  authorBurch, K. S.
titleScattering Continuum and Possible
  Fractionalized Excitations in α-RuCl_3.
journalPhys. Rev. Lett.
  volume114, pages147201
  (year2015).

nasu2016fermionic
authorNasu, J., authorKnolle, J.,
  authorKovrizhin, D. L., authorMotome, Y. &
  authorMoessner, R.
titleFermionic response from fractionalization in an
  insulating two-dimensional magnet.
journalNat. Phys. volume12,
  pages912–915 (year2016).

banerjee2016neutron
authorBanerjee, A. et al.
titleNeutron scattering in the proximate quantum spin
  liquid α-RuCl_3.
journalScience volume356,
  pages1055–1059 (year2017).

banerjee2016proximate
authorBanerjee, A. et al.
titleProximate Kitaev quantum spin liquid behaviour in a
  honeycomb magnet.
journalNat. Mater. volume15,
  pages733–740 (year2016).

incarnation
authorDo, S.-H. et al.
titleIncarnation of Majorana Fermions in Kitaev
  Quantum Spin Lattice. Preprint at https://arxiv.org/abs/1703.01081
  (year2017).

glamazda2016raman
authorGlamazda, A., authorLemmens, P.,
  authorDo, S.-H., authorChoi, Y. S. &
  authorChoi, K.-Y.
titleRaman spectroscopic signature of fractionalized
  excitations in the harmonic-honeycomb iridates β- and
  γ-Li_2IrO_3.
journalNat. Commun. volume7,
  pages12286 (year2016).

chaloupka2010kitaev
authorChaloupka, J., authorJackeli, G. &
  authorKhaliullin, G.
titleKitaev-Heisenberg Model on a Honeycomb
  Lattice: Possible Exotic Phases in Iridium Oxides
  A_2IrO_3.
journalPhys. Rev. Lett.
  volume105, pages027204
  (year2010).

rau2014generic
authorRau, J. G., authorLee, E. K.-H. &
  authorKee, H.-Y.
titleGeneric Spin Model for the Honeycomb Iridates
  beyond the Kitaev Limit.
journalPhys. Rev. Lett.
  volume112, pages077204
  (year2014).

PhysRevB.93.214431
authorWinter, S. M., authorLi, Y.,
  authorJeschke, H. O. & authorValentí, R.
titleChallenges in design of Kitaev materials:
  Magnetic interactions from competing energy scales.
journalPhys. Rev. B volume93,
  pages214431 (year2016).

sears2015magnetic
authorSears, J. A. et al.
titleMagnetic order in α-RuCl_3: A
  honeycomb-lattice quantum magnet with strong spin-orbit coupling.
journalPhys. Rev. B volume91,
  pages144420 (year2015).

PhysRevB.94.064435
authorChaloupka, J. & authorKhaliullin, G.
titleMagnetic anisotropy in the Kitaev model systems
  Na_2IrO_3 and RuCl_3.
journalPhys. Rev. B volume94,
  pages064435 (year2016).

PhysRevB.91.245134
authorKimchi, I., authorColdea, R. &
  authorVishwanath, A.
titleUnified theory of spiral magnetism in the
  harmonic-honeycomb iridates ,, and
  Li_2IrO_3.
journalPhys. Rev. B volume91,
  pages245134 (year2015).

0295-5075-112-6-67004
authorKim, H.-S., authorLee, E. K.-H. &
  authorKim, Y.-B.
titlePredominance of the Kitaev interaction in a
  three-dimensional honeycomb iridate: From ab initio to spin model.
journalEPL volume112,
  pages67004 (year2015).

PhysRevB.93.134423
authorCao, H. B. et al.
titleLow-temperature crystal and magnetic structure of
  α-RuCl_3.
journalPhys. Rev. B volume93,
  pages134423 (year2016).

ranneutron
authorRan, K. et al.
titleSpin-Wave Excitations Evidencing the Kitaev
  Interaction in Single Crystalline α-RuCl_3.
journalPhys. Rev. Lett.
  volume118, pages107203
  (year2017).

kim2016crystal
authorKim, H.-S. & authorKee, H.-Y.
titleCrystal structure and magnetism in
  α-RuCl_3: An ab initio study.
journalPhys. Rev. B volume93,
  pages155143 (year2016).

yadav2016spin
authorYadav, R. et al.
titleKitaev exchange and field-induced quantum spin-liquid
  states in honeycomb α-RuCl_3.
journalSci. Rep. volume6,
  pages37925 (year2016).

wanginteractions
authorWang, W., authorDong, Z.-Y., authorYu,
  S.-L. & authorLi, J.-X.
titleTheoretical investigation of the magnetic dynamics
  and superconducting pairing symmetry in α-RuCl_3. Preprint at
  https://arxiv.org/abs/1612.09515  (year2016).

houinteractions
authorHou, Y. S., authorXiang, H. J. &
  authorGong, X. G.
titleUnveiling Magnetic Interactions of Ruthenium
  Trichloride via Constraining Direction of Orbital moments:
  Potential Routes to Realize Quantum Spin Liquid. Preprint at
  https://arxiv.org/abs/1612.00761  (year2016).

katukuri2014kitaev
authorKatukuri, V. M. et al.
titleKitaev interactions between j=1/2 moments in
  honeycomb Na_2IrO_3 are large and ferromagnetic: insights from ab initio quantum chemistry calculations.
journalNew J. Phys. volume16,
  pages013056 (year2014).

PhysRevB.92.115127
authorKnolle, J., authorKovrizhin, D. L.,
  authorChalker, J. T. & authorMoessner, R.
titleDynamics of fractionalization in quantum spin
  liquids.
journalPhys. Rev. B volume92,
  pages115127 (year2015).

PhysRevLett.112.207203
authorKnolle, J., authorKovrizhin, D. L.,
  authorChalker, J. T. & authorMoessner, R.
titleDynamics of a Two-Dimensional Quantum Spin
  Liquid: Signatures of Emergent Majorana Fermions and Fluxes.
journalPhys. Rev. Lett.
  volume112, pages207203
  (year2014).

frank2017
authorGohlke, M., authorVerresen, R.,
  authorMoessner, R. & authorPollmann, F.
titleDynamics of the Kitaev-Heisenberg Model.
  Preprint at https://arxiv.org/abs/1701.04678  (year2017).

PhysRevB.92.235119
authorJohnson, R. D. et al.
titleMonoclinic crystal structure of α-RuCl_3
  and the zigzag antiferromagnetic ground state.
journalPhys. Rev. B volume92,
  pages235119 (year2015).

harris1971dynamics
authorHarris, A. B., authorKumar, D.,
  authorHalperin, B. I. & authorHohenberg, P. C.
titleDynamics of an Antiferromagnet at Low
  Temperatures: Spin-Wave Damping and Hydrodynamics.
journalPhys. Rev. B volume3,
  pages961–1024 (year1971).

chernyshev2006magnon
authorChernyshev, A. L. & authorZhitomirsky, M. E.
titleMagnon Decay in Noncollinear Quantum
  Antiferromagnets.
journalPhys. Rev. Lett.
  volume97, pages207202
  (year2006).

zhitomirsky2013colloquium
authorZhitomirsky, M. E. & authorChernyshev, A. L.
titleColloquium: Spontaneous magnon decays.
journalRev. Mod. Phys.
  volume85, pages219–242
  (year2013).

PhysRevB.2.1172
authorRuvalds, J. & authorZawadowski, A.
titleTwo-Phonon Resonances and Hybridization of the
  Resonance with Single-Phonon States.
journalPhys. Rev. B volume2,
  pages1172–1175 (year1970).

kosevich
authorKosevich, A. M.
titleThe Crystal Lattice: Phonons, Solitons,
  Dislocations, Superlattices (publisherWiley-VCH Verlag GmbH
  & Co. KGaA, year2006).

PhysRevB.94.085109
authorSizyuk, Y., authorWölfle, P. &
  authorPerkins, N. B.
titleSelection of direction of the ordered moments in
  Na_2IrO_3 and α-RuCl_3.
journalPhys. Rev. B volume94,
  pages085109 (year2016).

chaloupka2015hidden
authorChaloupka, J. & authorKhaliullin, G.
titleHidden symmetries of the extended
  Kitaev-Heisenberg model: Implications for the honeycomb-lattice
  iridates A_2IrO_3.
journalPhys. Rev. B volume92,
  pages024413 (year2015).

PhysRevB.93.235130
authorHwang, K. & authorKim, Y. B.
titleTheory of triplon dynamics in the quantum magnet
  BiCu_2PO_6.
journalPhys. Rev. B volume93,
  pages235130 (year2016).

PhysRevB.94.140407
authorMaksimov, P. A., authorZhitomirsky, M. E. &
  authorChernyshev, A. L.
titleField-induced decays in XXZ triangular-lattice
  antiferromagnets.
journalPhys. Rev. B volume94,
  pages140407(R) (year2016).

PhysRevB.88.094407
authorMourigal, M., authorFuhrman, W. T.,
  authorChernyshev, A. L. & authorZhitomirsky, M. E.
titleDynamical structure factor of the triangular-lattice
  antiferromagnet.
journalPhys. Rev. B volume88,
  pages094407 (year2013).

PhysRevLett.113.197201
authorBiffin, A. et al.
titleNoncoplanar and Counterrotating Incommensurate
  Magnetic Order Stabilized by Kitaev Interactions in
  γ-Li_2IrO_3.
journalPhys. Rev. Lett.
  volume113, pages197201
  (year2014).

PhysRevB.90.205116
authorBiffin, A. et al.
titleUnconventional magnetic order on the hyperhoneycomb
  Kitaev lattice in β-Li_2IrO_3: Full solution via magnetic
  resonant x-ray diffraction.
journalPhys. Rev. B volume90,
  pages205116 (year2014).

PhysRevB.93.195158
authorWilliams, S. C. et al.
titleIncommensurate counterrotating magnetic order
  stabilized by Kitaev interactions in the layered honeycomb
  α-Li_2IrO_3.
journalPhys. Rev. B volume93,
  pages195158 (year2016).

PhysRevLett.114.077202
authorTakayama, T. et al.
titleHyperhoneycomb Iridate β-Li_2IrO_3 as
  a Platform for Kitaev Magnetism.
journalPhys. Rev. Lett.
  volume114, pages077202
  (year2015).

PhysRevB.94.245127
authorKim, H.-S., authorKim, Y. B. &
  authorKee, H.-Y.
titleRevealing frustrated local moment model for
  pressurized hyperhoneycomb iridate: Paving the way toward a quantum spin
  liquid.
journalPhys. Rev. B volume94,
  pages245127 (year2016).

PhysRevB.84.155121
authorMandal, S., authorBhattacharjee, S.,
  authorSengupta, K., authorShankar, R. &
  authorBaskaran, G.
titleConfinement-deconfinement transition and spin
  correlations in a generalized Kitaev model.
journalPhys. Rev. B volume84,
  pages155121 (year2011).

PhysRevB.86.224417
authorSchaffer, R., authorBhattacharjee, S. &
  authorKim, Y.-B.
titleQuantum phase transition in Heisenberg-Kitaev
  model.
journalPhys. Rev. B volume86,
  pages224417 (year2012).

Cromer:a04464
authorCromer, D. T. & authorWaber, J. T.
titleScattering Factors Computed from Relativistic
  Dirac-Slater Wave Functions.
journalActa Cryst. volume18,
  pages104–109 (year1965).

lanczos1950iteration
authorLanczos, C.
titleAn Iteration Method for the Solution of the
  Eigenvalue Problem of Linear Differential and Integral
  Operators.
journalJ. Res. Nat. Bur. Stand.
  volume45, pages255–282
  (year1950).

RevModPhys.66.763
authorDagotto, E.
titleCorrelated electrons in high-temperature
  superconductors.
journalRev. Mod. Phys.
  volume66, pages763–840
  (year1994).

toth2015linear
authorToth, S. & authorLake, B.
titleLinear spin-wave theory for single-Q incommensurate
  magnetic structures.
journalJ. Phys. Condens. Matter
  volume27, pages166002
  (year2015).







§ FIGURES











[pages=1]Supplemental_v12.pdf
[pages=2]Supplemental_v12.pdf
[pages=3]Supplemental_v12.pdf
[pages=4]Supplemental_v12.pdf
[pages=5]Supplemental_v12.pdf
[pages=6]Supplemental_v12.pdf
[pages=7]Supplemental_v12.pdf
[pages=8]Supplemental_v12.pdf
[pages=9]Supplemental_v12.pdf
[pages=10]Supplemental_v12.pdf
[pages=11]Supplemental_v12.pdf
[pages=12]Supplemental_v12.pdf
[pages=13]Supplemental_v12.pdf
[pages=14]Supplemental_v12.pdf
[pages=15]Supplemental_v12.pdf
[pages=16]Supplemental_v12.pdf
[pages=17]Supplemental_v12.pdf
[pages=18]Supplemental_v12.pdf
[pages=19]Supplemental_v12.pdf
[pages=20]Supplemental_v12.pdf
[pages=21]Supplemental_v12.pdf
[pages=22]Supplemental_v12.pdf
[pages=23]Supplemental_v12.pdf
[pages=24]Supplemental_v12.pdf
[pages=25]Supplemental_v12.pdf
