\section{Introduction}
Like any complex machine, the construction of a large-scale quantum computer will require a rapid design-and-test cycle of individual components and their assemblage. Early recognition of this need by quantum information scientists led to widespread, (but not universal~\cite{Nielsen05}), adoption of the process fidelity as the gold-standard for the quality of quantum operations \cite{Wineland98}. While the quantity had solid theoretical motivations, experimentalists lacked a simple method for its measurement for two or more qubits. In practice, experimentalists were stuck choosing between comprehensive but resource-intensive process tomography~\cite{OBrien04} and convenient but less descriptive Bell-state tomography~\cite{Ballance16, Gaebler16, Benhelm08, Wright19}. 

The advent of randomized benchmarking (RB) marked a powerful advancement in experimental validation of quantum operations~\cite{Emerson05,Knill08}. The new tool provided algorithmic advantages, including robustness to state-preparation and measurement (SPAM) errors~\cite{Magesan11, Magesan12a},
multiplicative precision in estimates~\cite{Harper19}, and, under certain assumptions, an accurate direct estimate of the fidelity~\cite{Wallman18}. In the case of multi-qubit gate benchmarking, one disadvantage of the protocol is the requirement for individual addressing. While this demand is also needed for universal quantum computation, it can impose a significant engineering requirement on the design-and-test cycle, especially in trapped-ion systems. In this work, we alleviate this requirement and provide a benchmarking protocol that uses only global operations.

Other methods for RB with limited gate sets have also been considered in the context of logical RB~\cite{Brown18, Hashagen2018}, and as a way to simplify standard RB~\cite{Franca18, Helsen19b}, but these methods suffer from added sampling overhead and require a detailed study of the gates' group structure to identify different decay rates. Our method circumvents these drawbacks.

The basic idea behind our procedure is to isolate a subspace of the total system to perform RB, hence we call the method subspace randomized benchmarking (SRB). We then use leakage benchmarking~\cite{Wallman16, Wood18, Andrews19} to identify errors that may drive population outside of this subspace. We show that these different methods can be combined into a single procedure and each effect can be individually measured. This analysis can identify many common errors, and we consider several examples numerically.

In addition to developing the theory behind SRB, we demonstrate the procedure in a trapped ion system. We benchmark the M\o lmer-S\o renson (MS) interaction and collective single-qubit gates (identical single-qubit gates applied to all ions) using two ions in a four-ion crystal. The other ions are a different species and are used for sympathetic cooling to prevent the temperature from changing throughout the gate sequences. With this system, we are able to run gate sequences with as many as 400 MS gates. The SRB procedure returns an estimated entangling gate error of $2(1) \times 10^{-3}$.

The remainder of this paper is organized into the following sections:
\begin{itemize}
	\item[\textbf{II.}] \textbf{Standard RB -} A review of the standard randomized benchmarking protocol and how to interpret the experimental results.
	\item[\textbf{III.}] \textbf{Subspace RB -} Our construction of SRB and how subspace leakage is considered.
	\item[\textbf{IV.}] \textbf{Extracting information from SRB -} Analysis of how different errors are manifested in SRB.
	\item[\textbf{V.}] \textbf{Experimental platform -} A description of the SRB demonstration.
	\item[\textbf{VI.}] \textbf{Results and discussion -} Estimates of the subspace and leakage errors.
	\item[\textbf{VII.}] \textbf{Generalized subspace RB -} Extensions to other systems.
\end{itemize}

\section{Standard Randomized Benchmarking} \label{sec:standard_RB}
We begin by outlining the standard RB procedure~\cite{Magesan12a} with further details given in App.~\ref{app:standard_RB}. RB provides an estimate of the fidelity of a finite set of gates
$\mathsf{C}=\{C_1, \dots, C_N\}$. The standard RB construction requires that 
the gates form a representation of a finite group, which is usually the Clifford group. A Clifford gate is the unitary implementation of a Clifford group element and defined by their action on Pauli gates. A Pauli gate is the unitary implementation of a Pauli group element, which we label  $\mathds{1}, X, Y$, and $Z$ and tensor products
thereof. A Clifford gate maps Pauli gates to Pauli gates
with a sign, e.g. $C X C^{\dagger} = -Y$. The Clifford gates are selected in RB for two reasons: (1) it is
efficient to simulate their evolution by the Gottesman-Knill theorem~\cite{Gottesman98}, and (2) they form a unitary two-design which simplifies the analysis of an RB experiment~\cite{Magesan12a}.

A standard RB experiment has sequences of gates shown in Fig.~\ref{fig:seq}, and proceeds as follows:
\begin{enumerate}
\item Prepare $\ket{0}$
\item Randomly select $\ell$ gates from $\mathsf{C}$ and apply to $\ket{0}$
\item Calculate the inverting gate $C_{\textrm{inv}}= C^{\dagger}_1 \cdots C^{\dagger}_{\ell} \in \mathsf{C}$
\item Measure survival $p({\ell}) =|\braket{0 | C_{\textrm{inv}} C_{\ell} \cdots C_{1} | 0 }|^2$
\item Repeat (1-4) $K$ times with different random gates and estimate the average survival
$\overline{p}({\ell}) = \tfrac{1}{K} \sum_i p_{i}(\ell)$
\item Repeat (1-5) with different $\ell$ 
\end{enumerate}
We typically pick $\ell$ exponentially distributed between 1 and $\ell_{\textrm{max}}\approx1/\epsilon$ where $\epsilon$ is the 
expected error~\cite{Meier06}. We typically pick $K\approx 100$ sequences, but rigorous estimates also exist based on expected fidelity~\cite{Wallman14, Helsen19a}.

Although the net action is chosen to be the identity operation, noise and errors in the state-preparation, gates, and measurement cause $p(\ell) \leq 1$. The power of RB is to relate the
average survival $\overline{p}({\ell})$ to the process fidelity of the Clifford gates. If each Clifford gate has the same error channel then randomization over the Clifford gates depolarizes each error channel due to the unitary two-design property~\cite{Magesan11}. The resulting average survival is
\begin{equation} \label{eq:rb_decay}
\overline{p}({\ell}) =A r^{\ell} +B,
\end{equation}
where $A$ and $B$ relate to SPAM errors, and $r$ is referred to as the standard decay rate. By evaluating $\overline{p}(\ell)$
for different values of $\ell$, we can extract $A$, $B$, and $r$ by least-squares fitting. The process fidelity of the Clifford gate's error channel is then defined as
\begin{equation}
F_{\textrm{Cliff}} = \tfrac{(d^2-1) r + 1}{d^2}.
\end{equation}

In practice, Pauli and Clifford gates are implemented by compiling physically available gates. In trapped-ion systems (and many
other quantum systems) the physical single-qubit gates are implemented as rotations in the $X-Y$ plane of the Bloch sphere with
arbitrary amplitude $U(\theta,\phi) = \exp[-i \tfrac{\theta}{2}(X \cos(\phi) + Y \sin(\phi))]$. The entangling 
operations for many trapped-ion experiments, including ours, is the MS gate, $U_{MS}(\theta, \phi) = \exp[-i
\tfrac{\theta}{2} (X \cos(\phi)+ Y \sin(\phi))^{\otimes 2}]$. As discussed later, we use the phase gate
$U_{ZZ} = \exp[-i \tfrac{\pi}{4} ZZ]$, which is created by applying single-qubit ``wrapper pulses'' before and after the
MS gate~\cite{Lee05}. In the system considered, single-qubit gates are not individually addressable, but we do have access to collective single-qubit gates $U(\theta,\phi) = \exp[-i \tfrac{\theta}{2}(X \cos(\phi) + Y \sin(\phi))]^{\otimes 2}$.

\begin{figure} 
  \centering
    \includegraphics[width=0.5\textwidth]{SSRB_circuits.pdf}
\caption{Circuits used for standard RB and SRB. Blue gates are collective single-qubit gates, gray gates are entangling phase gates, and green gates are individually addressed single-qubit gates.  (a) Standard RB for a transport-based trapped-ion architecture. Individually addressed gates require a transport operation and additional cooling, which involves significant engineering overhead and may increase the error rate per Clifford. Alternative individual addressing methods exist but also have engineering overheads and similarly affect the error rates. (b) SRB only has collective and phase gates that do not require transport and cooling can be implemented during the collective gates. This reduces total run time and other errors.}
\label{fig:seq}
\end{figure}

\section{Subspace randomized benchmarking} \label{sec:ssrb}
Our procedure for RB, which we call subspace randomized benchmarking (SRB), follows the same general prescription
outlined above except we consider a restricted set of physical gates. This restriction could be due to the physical limitations,
i.e. the system cannot implement a universal set, or because some gates have much lower fidelity and must be
avoided. In either case, the remaining set of gates is not universal, rendering standard RB impossible. We now construct a protocol with a restricted set of gates that retains most of RB's favorable attributes and still yields
detailed information about the gate in question.

We introduce the procedure in the context of a trapped-ion system where the two qubits are encoded in two ions in the same potential well.
Without tightly focused laser beams or complex transport operations, the available one- and two-qubit gates will be symmetric 
with respect to qubit exchange. This limited set of physical gates cannot generate the two-qubit Clifford gates needed for 
standard RB. Nevertheless, the goal of the setup is to determine a quantitative measure of the
entangling gate's performance. This is a common setup for many trapped-ion experiments that are trying to demonstrate
new entangling gates or high-fidelity performance~\cite{Ballance16, Gaebler16, Wright19}.

The first step in SRB is to identify a subspace of the total Hilbert space where we can perform RB, which we call the RB
subspace. For the trapped-ion system, the symmetric subspace of the two-qubits is a natural choice. States in this subspace are symmetric under the
SWAP operation, with basis $\{\ket{00},\tfrac{1}{\sqrt{2}}(\ket{01} + \ket{10}), \ket{11} \}$. The remaining space is spanned by the anti-symmetric state $\tfrac{1}{\sqrt{2}}(\ket{01} - \ket{10})$, and we refer to it as
the leakage subspace. It is easy to prepare the $\ket{00}$ state via optical pumping with high fidelity~\cite{Olmschenk07}. The
collective single-qubit gates and phase gate are block-diagonal in this basis, and therefore maintain population in each
subspace. Measurements in the system return the number of qubits in the $\ket{1}$ state (the number of ions fluorescing). If the population is isolated to the symmetric subspace then this measurement
corresponds to a projection measurement in the basis for RB subspace defined above.

The next step is to identify a set of gates in the RB subspace that form a unitary two-design, which was the property that allowed us to derive Eq.~\eqref{eq:rb_decay}. In standard RB, we use Clifford gates, which map Pauli gates to Pauli gates. For prime dimensions, we can generalize the Pauli gates to Weyl gates and the corresponding Clifford gates map Weyl gates to Weyl gates~\cite{Hostens05}. These generalized Cliffords also form a unitary-two design, which will act to depolarize any error channel~\cite{Dankert09}. For $d_1 = 3$ (or qutrit) there are 9 qutrit-Weyl gates and 216 qutrit-Clifford gates. 

To generate each qutrit-Clifford gate we use techniques from optimal control theory. In the RB subspace the collective
gates act as spin-1 $J_x$ and $J_y$ angular momentum operators, and the phase gate acts as a $J_z^2$ operator. Then,
control of the symmetric subspace is analogous to control of a spin-1 particle. We make use of the Cartan decomposition~\cite{Khaneja01,Shauro2015} to
decide the order of collective and phase gates and then perform a numerical search~\cite{Machnes11} for each of the 216 qutrit-Clifford gates to
find amplitudes and phases. We find that only three phase gates are required per qutrit-Clifford arranged as shown in Fig.~\ref{fig:seq}. This does require full simulation of the system, but for two qubits, we find that it takes a reasonable amount of time. Further details are given in App.~\ref{app:control}. 

The protocol then proceeds the same as standard RB but with the qutrit-Clifford gates. After taking data with various sequence lengths, we estimate the RB decay parameter $r$, and calculate the subspace fidelity, which we denote with lower-case $f$ to differentiate from the full-space fidelity $F$,
\begin{equation}
f_{\textrm{Cliff}} = \tfrac{(d_1^2-1) r + 1}{d_1^2},
\end{equation}
where $d_1$ is the dimension of the RB subspace. The subspace fidelity only provides an estimate for errors that are isolated to the RB subspace. We call the procedure given so far SRB-\textit{lite}.

An obvious problem with SRB-\textit{lite} is that there may exist errors that are not isolated to the RB subspace. In a quantum algorithm, the input state to each gate is unlikely to be symmetric and only measuring errors on the symmetric subspace may miss errors that degrade the computation. Moreover, some errors
might couple the RB and leakage subspaces, violating the assumptions we made to derive the RB decay curve.
A similar problem arises in single-qubit RB if there are errors that cause population to leave the 
computational subspace, called leakage errors. Previously, several protocols to measure leakage rates were proposed, which we refer to as leakage benchmarking (LB)~\cite{Wallman16, Wood18, Andrews19}. We now show LB can be performed along with SRB-\textit{lite} to create a more complete picture of the errors in the MS gate. We call this combined procedure SRB.

In Ref.~\cite{Wood18}, leakage errors are quantified by a leakage rate $L$ (rate that which population moves from RB to leakage subspace) and seepage rate $S$ (rate that which population moves from the leakage to the RB subspace),
\begin{align}
L &= \tfrac{1}{\sqrt{d_1 d_2}} \textrm{Tr}(\Lambda[\mathds{1}_1]\mathds{1}_2), \\
S &= \tfrac{1}{\sqrt{d_1 d_2}} \textrm{Tr}(\mathds{1}_1 \Lambda[\mathds{1}_2]),
\end{align}
where $\mathds{1}_1$ and $\mathds{1}_2$ are projectors onto the RB and leakage subspaces respectively. Our definitions differ slightly from
Ref.~\cite{Wood18} by factors $d_1$ (dimension of RB subspace) and $d_2$ (dimension of the leakage subspace) in order to simplify the derivations in App.~\ref{app:ssrb}.
Leakage and seepage errors cause additional terms to enter into Eq.~\eqref{eq:rb_decay}~\cite{Wallman16, Wood18},
\begin{equation} \label{eq:rb_wleakage}
\overline{p}({\ell})=A r^{\ell} + B + C t^{\ell+1},
\end{equation}
where $A$, $B$ and $C$ relate to SPAM errors, $r$ is the standard RB decay, and $t$ relates to leakage and seepage rates. 

We use a different technique that more effectively extracts the decay rates $r$ and $t$ than previous LB proposals.  It requires including an additional random Weyl gate $Q_k$, which can be
compiled into the final gate $C_{\textrm{inv}} = Q_k C_{\ell}^{\dagger} \dots C_1^{\dagger}$. This method was also proposed in Refs.~\cite{Knill08, Meier06} to catch pathological errors in standard RB. A similar method was explored in Ref.~\cite{Harper19} to reduce the fitting parameters in order to derive multiplicative precision in standard RB. In addition to both of these advantages, we find that it also allows us to separate the analysis of the standard decay $r$ from the effects of leakage errors and reduce the assumptions required in the derivation. Further details are given in App.~\ref{app:ssrb}.

We again randomize over all sequences, which includes randomization over the final Weyl gate. Then, we process the
data in two ways:
\begin{itemize}
\item \textit{Standard analysis}: For the standard RB decay, we only use the measurement outcomes $m_k$ such that $|\braket{m_k | Q_k | 0}|^2= 1$ in the absence of errors. Averaging over Weyl gates, with the related outcome $m_k$, fixes $B = 1/d_1$ and $C =0$, leaving a single exponential
decay to fit, 
\begin{equation} \label{eq:standard_decay}
\overline{p}_S({\ell}) =A r^{\ell} + \tfrac{1}{d_1},
\end{equation}
This differs from Ref.~\cite{Wood18} since $B$ is fixed but does not require assumptions on SPAM errors like in Ref.~\cite{Andrews19}. 

\item \textit{Leakage analysis}: To measure the leakage decay, we pick $m=0$ outcomes, corresponding to the survival with the
additional Weyl gate. Averaging over all sequences gives the average leakage decay
\begin{equation} \label{eq:leakage_decay}
\overline{p}_L({\ell}) = C t^{\ell+1} + B.
\end{equation}
\end{itemize}

In Ref.~\cite{Wood18} $L$ and $S$ are used to quantify leakage errors, but we propose that $t$ is more reliable and almost as
valuable. This is because the estimates of $L$ and $S$ rely on the fits of $C$ and $B$, which are limited by finite sampling and SPAM errors. The estimation method for $t$ is almost identical to the estimation of $r$, and so follows the standard RB paradigm. The downside of only studying $t$ is that we cannot differentiate between leakage and seepage errors, but our analysis finds that these rates are usually similar for standard errors in the trapped-ion system.

\begin{figure} 
  \centering
    \includegraphics[width=0.5\textwidth]{SSRB_connections.pdf}
\caption{Connections between measured quantities in SRB (standard decay rate $r$ and leakage decay rate $t$), the
extended-sub fidelity, and the full fidelity. The top row shows the
connections between SRB and qutrit-Clifford gates. The bottom row shows the connection between interleaved SRB and the
phase gate. If we assume the phase gate errors are depolarizing and dominate each Clifford gate then we can connect the top
and bottom rows by the approximation $f_{ZZ}' = 1- \tfrac{1}{3}(1- f_{\textrm{Cliff}}')$.}
\label{fig:connections}
\end{figure}

The results of both analyses then contribute to an estimate of the subspace fidelity of the qutrit-Clifford gates on the RB subspace.
The leakage rate contributes to this estimate since the leakage errors reduce population in the RB subspace. We propose a variant of subspace fidelity that includes the seepage rate 
and is only dependent on $r$, $t$, and the dimensions, which we call the extended-sub fidelity,
\begin{equation} \label{eq:extnd_fid}
f_{\textrm{Cliff}}' = \tfrac{d_1^2}{d_1^2 + 1}f_{\textrm{Cliff}} + \tfrac{1}{d_1^2 + 1} S=\tfrac{8 r + t + 1}{10},
\end{equation}
where the second equality is specific for the trapped-ion system. This quantity has the similar advantages of estimating
$t$ over $L$ and $S$ as well as capturing additional information about seepage errors in the fidelity. 

To extract the extended-sub fidelity of a single phase gate we scale $f_{\textrm{Cliff}}'$ by the number of phase gates per Clifford, which is three,
\begin{equation}
f_{ZZ}' \approx 1-\tfrac{1}{3}(1-f_{\textrm{Cliff}}') = \frac{8(r/3) + t/3 + 7}{10}
\end{equation}
A similar procedure can be applied with $f_{\textrm{Cliff}}$ from SRB-\textit{lite}. This procedure is only exact when phase-gate errors dominate and each phase-gate has a depolarizing error channel. It is very likely that the phase gate errors dominate but unlikely that the errors are purely depolarizing in any real experiment. The quantity still serves as a rough estimate of the contribution of each phase gate to the error in the qutrit-Clifford gates. We report the estimated phase-gate error, which is $1-f_{ZZ}'$.

Another option to estimate phase gate fidelity is to perform interleaved randomized benchmarking (IRB)~\cite{Magesan12b} to bound the phase-gate error. This is shown as the bottom row of Fig.~\ref{fig:connections}. For SRB, such an interleaved procedure is more complicated since the phase gate is not a qutrit-Clifford gate. Instead of randomizing over the qutrit-Clifford gates we randomize over the larger group of arbitrary SU(3) unitary operators. The procedure is then the same as standard IRB; perform SRB and then perform SRB with interleaved phase gates. The standard IRB procedure also assumes that the errors are depolarizing and returns weak bounds on the fidelity if this is not true~\cite{Magesan12b}. While this could be improved by including other experiments (like unitarity benchmarking~\cite{Carignan19}), we opt for the simpler procedure given above for estimating the phase-gate error and stress that the estimates are subject to the depolarizing assumption.


\section{Extracting information from SRB} \label{sec:sims}
To understand how errors affect SRB, we preformed an extensive study of typical errors in trapped-ion experiments. We found that each
error falls into one of two categories: (1) Errors that preserve the subspaces and (2) errors that mix population
between RB and leakage subspaces. The results are summarized in Table~\ref{tab:table1} and further details of each error channel are given in App.~\ref{app:errors}.

Table~\ref{tab:table1} also shows how errors have different characteristics that are measurable by SRB. For example,
intensity errors fall into type-1 since they preserve the subspaces, and therefore we expect an SRB experiment to
measure $r< 1$ and $t \approx 1$. Spontaneous emission causes mixing between the subspaces so that we expect $r< 1$ and $t < 1$. An interesting result from the study is that several errors can be tuned between type-1 and -2 by adjusting ion
spacing and the laser/motional phase. This could also be used to further isolate dominant errors in the phase gate.

While SRB can detect most of the common errors in the phase gate, it will miss one type of error. We refer to these as `worst-case' errors, and they correspond to errors that change the relative phase between the RB and leakage subspaces. For example, if the ions reordered this would look like a logical SWAP error, which generates a $-1$ phase between the two subspaces. This type of error will go unnoticed in an SRB experiment but could cause serious problems in a full computation. In general though, these errors are unlikely as they do not arise from any of the known MS error mechanisms~\cite{Sorensen00} and our idling SWAP rate, (from background gas collisions), is observed to be on the timescale of tens of minutes.
\begin{table}
  \begin{center}
    \begin{tabular}{|l|c|c|c|c|}
      \hline
\textrm{\textbf{Error channel}$\quad\quad\quad\quad\quad$} & $\,\mathbf{m}_+,\mathbf{l}_+\,$ & $\,\mathbf{m}_+,\mathbf{l}_-\,$ & $\, \mathbf{m}_-,\mathbf{l}_+\,$ &
$\,\mathbf{m}_-,\mathbf{l}_-\,$ \\
      \hline
      Intensity error & $\checkmark$ & $\checkmark$ & $\checkmark$ & $\checkmark$ \\
      \hline
      Lamb-Dicke & $\checkmark$ & $\checkmark$ & $\checkmark$ & $\checkmark$\\
      \hline
      Debye-Waller & $\checkmark$ & $\checkmark$ &  $\checkmark$ & $\checkmark$\\
      \hline
Trap frequency error & $\checkmark$ & $\rightleftarrows$ & $\rightleftarrows$ & $\checkmark$\\      \hline
Spectator mode coupling & $\rightleftarrows$ & $\rightleftarrows$ & $\rightleftarrows$ &
$\rightleftarrows$\\
      \hline
      Carrier coupling & $\checkmark$ &$\rightleftarrows$ & $\checkmark$ & $\rightleftarrows$\\
      \hline
      Carrier frequency error & $\checkmark$ & $\checkmark$ & $\checkmark$ & $\checkmark$ \\
      \hline
      Uneven ion illumination & $\rightleftarrows$ &  $\rightleftarrows$ & $\rightleftarrows$ & $\rightleftarrows$\\
      \hline
      Heating & $\checkmark$ & $\checkmark$ & $\checkmark$ & $\checkmark$\\
      \hline
	Spontaneous emission & $\rightleftarrows$ & $\rightleftarrows$ & $\rightleftarrows$ &
$\rightleftarrows$\\	\hline
	Stray optical pumping & $\rightleftarrows$ & $\rightleftarrows$ & $\rightleftarrows$ &
$\rightleftarrows$\\ \hline
	Inhomogeneous fields & $\rightleftarrows$ & $\rightleftarrows$ & $\rightleftarrows$ &
$\rightleftarrows$\\
      \hline
    \end{tabular}
\caption{This table categorizes common errors in the phase gate according to whether or
not they preserve the RB and leakage subspaces. The different columns denote the qubit exchange symmetry
of the motion (m) and that of the laser phase (l).
Errors that preserve the subspaces are marked $\checkmark$ (type-1), whereas the errors that mix the
subspaces are marked $\rightleftarrows$ (type-2).}
\label{tab:table1}
  \end{center}
\end{table}

\section{Experimental platform}
We implement our SRB procedure in a linear RF Paul trap with a chain of two $^{171}$Yb$^{+}$ data qubits and two $^{138}
$Ba$^{+}$ ions for sympathetic cooling. The motion of the ions is described by shared motional modes with
transverse (X,Y) and axial (Z) harmonic confinement at frequencies $\{\omega_x,\omega_y,\omega_z\} = 2\pi \times
\{2.72,3.24,1.15\}$ MHz for a single $^{171}$Yb$^{+}$. The axial modes of motion and the three lowest frequency radial modes of
the four ion chain are cooled to $\bar{n} < 1$ via resolved sideband cooling of both Yb$^+$ and Ba$^+$ before the start
of an SRB sequence~\cite{Monroe95}. The axial in-phase mode is cooled during an SRB sequence using sideband cooling on the Ba
ions to mitigate effects of motional heating that takes place during the SRB sequence.

The data qubits are encoded in the $^2 S_{1/2}$ hyperfine levels of the $^{171}$Yb$^+$ and
initialized and read out using standard techniques~\cite{Olmschenk07}. The qubit levels are coupled via Raman transitions
driven by a pair of $375$ nm phase-locked lasers with a $12.6$ GHz frequency offset to match the
hyperfine frequency $\omega_{0}$ of $^{171}$Yb$^{+}$ in a magnetic field of 5.2 Gauss.

Collective single-qubit operations are preformed with co-propagating, $\sigma^+$-polarized Raman beams. Two qubit operations are
realized via the $\hat \sigma_{\phi}$ gate in the phase-sensitive geometry~\cite{Sorensen00,Lee05}. The ion chain is illuminated
with lin$\perp$lin-polarized Raman beams focused to a spot size $\approx$ 35 $\mu$m and propagating at a relative angle of 90 degrees, such that the
wavevector difference $\Delta \vec{k} \parallel \hat z$. The beatnote frequency of the two beams satisfies $\delta
\omega = \omega_{0} \pm \omega_{z,4} \pm \delta$, where $\omega_{z,4}$ is the highest frequency axial mode and $2\pi/\delta\approx t_{gate} = 30 \mu s$.

The detection is performed by state-dependent-resonance florescence, which returns photon counts proportional to the number of ions in the $\ket{1}$ (bright) state. For a detection duration of 400 $\mu$s, we detect an average of 9 photons from each ion in $\ket 1$ and 0.1
photons from each ion in $\ket 0$. Empirically, we find that the threshold measurement technique~\cite{Olmschenk07} results
in large measurement errors for 1 and 2 excited ion outcomes, which degrades the RB fitting. Instead, we perform detector tomography and process the RB data to reduce the measurement error~\cite{Keith18}. This allows us to reconstruct relative frequencies of the desired projection operators.


\begin{figure*} 
  \centering
    \includegraphics[width=1.0\textwidth]{experimental_fig.pdf}
  \caption{Experimental data from trapped-ion system. (a) SRB-\textit{lite} with estimated phase-gate errors of $1-f_{ZZ} =1.5(1) \times 10^{-3}$. The shaded region shows the one-sigma confidence interval of the decay fit from the semi-parametric bootstrap resampling. (b) Two example SRB experiments with standard analysis in circles with solid lines and leakage analysis in triangles with dashed lines: (blue) high-fidelity SRB with estimated phase gate errors of $1-f_{ZZ}' =2(1) \times 10^{-3}$, and (green) injected intensity error to show the change in the standard decay. The leakage points are offset in the x-axis for clarity. (c) SRB experiments with different magnitude intensity errors injected. The lines are from the analytic model discussed in App.~\ref{app:sims}. Circles with solid line show the standard decay rates for each magnitude error, while triangles with dashed line show the leakage decay rate for each magnitude error. The blue and green points from (b) are highlighted in the respective colors.}
\label{fig:high fid}
\end{figure*}

\section{Results and discussion} \label{sec:discussion}
We first performed SRB-\textit{lite} with sequences of qutrit-Clifford gates $[7,21,35,49,91,147]$ each with 33-36 randomizations (results plotted in Fig.~\ref{fig:high fid}a). Each sequence was repeated 100 times to build statistics. The points were fit to Eq.~\eqref{eq:rb_decay} with the least-squares method to give $r = 0.9949(3)$, which we used to estimate $f_{\textrm{Cliff}} = 0.9954(3)$ and phase-gate error rate of $1-f_{ZZ} = 1.5(1)\times 10^{-3}$.  Uncertainties correspond to one-sigma confidence intervals estimated by a semi-parametric bootstrap method~\cite{Meier06}. In this experiment, the qutrit-Cliffords were found with a slightly different method that resulted in an average of 3.056 phase-gates per qutrit-Clifford gate. The SRB-\textit{lite} method does not require detector tomography to reconstruct the relative frequencies needed. This makes the experiment simpler but does not identify the leakage error or reduce the number of fit parameters by fixing the $B$ parameter.

Next, we implemented SRB with sequences of qutrit-Clifford gates $[3, 17, 40]$ each with 50 random sequences (results plotted in Fig.~\ref{fig:high fid}b, solid lines with open markers). Each sequence was repeated 50 times to build statistics. For the leakage analysis, we used the even-parity outcomes. The results were fit to the decay curves derived in Sec.~\ref{sec:ssrb} with the least-squares method and plotted in Fig.~\ref{fig:high fid}a.
We found a standard decay of $r=0.9934(8)$ and a leakage decay of $t=0.985(23)$. These estimates translate to an extended-sub fidelity for the qutrit-Cliffords of $f_{\textrm{Cliff}}'=0.993(3)$ and a phase-gate error of $1-f_{ZZ}' = 2(1)\times 10^{-3}$. 

To validate that phase-gate errors dominate in SRB and SRB-\textit{lite}, we preformed standard single-qubit RB and measured a Clifford error of $1-F_{\textrm{Cliff}}=5.3(3) \times 10^{-5}$. We also note that the qutrit-Clifford gates have a very similar form to methods for generating arbitrary two-qubit unitaries, as demonstrated in Ref.~\cite{Hanneke10}, which makes the extended-sub fidelity for the qutrit-Cliffords a relevant metric as well. 

We also experimentally explored how SRB behaves when we inject errors into the system. We chose to inject over-rotation errors in the MS gate which translate to over-rotations in the phase gate plotted in Fig.~\ref{fig:high fid}c. For these tests, we used the same sequence lengths, reps, and trials per sequence as in Fig~\ref{fig:high fid}b. The standard decay $r$ decreases quadratically with the rotation magnitude while $t$ remains constant. The measured values match closely with the dashed lines, which are from a theoretical model described further in App.~\ref{app:sims}. The zero over rotation corresponds to the solid lines in Fig.~\ref{fig:high fid}b and the maximum over rotation value corresponds to the dashed line in Fig.~\ref{fig:high fid}b. Due to the close ion spacing we were not able to inject errors that cause leakage to the anti-symmetric subspace but we do study them numerically in App.~\ref{app:sims}. 

In certain cases, the leakage analysis may not be a reliable quantitative measure of leakage errors. This can happen if there are other leakage subspaces or there is not adequate control of the leakage subspace (see App.~\ref{app:ssrb}). These effects violate the assumptions made in the derivation of Eq.~\eqref{eq:leakage_decay} and cause other exponential decay terms to contribute to $\overline{p}_L(\ell)$. We also observe from the semi-parametric bootstrap of the numerical and experimental tests that the $t$ decay rate has much higher variance. This is due to a much shallower decay, which can make fitting difficult, and the large variance in the values of $\overline{p}_L(\ell)$. All of these issues will degrade the estimate of $t$.

However, the standard analysis is robust to all of these effects by including Weyl gates with the final inversion gate. This means the decay rate $r$ is still a valuable quantity and the leakage analysis could be used for qualitative information about leakage. The same analysis could be applied to any LB experiment to still extract information about gate fidelity in the presence of unknown or uncontrollable leakage.


\section{Generalized SRB}
SRB can be generalized to many other systems that are restricted to a non-universal set of gates. In this section we give a brief description of the general SRB procedure and outline the requirements.

In general, the set of available gates in a system is generated by a small set of interactions described by Hamiltonians $\{H_i\}$. We assume that each Hamiltonian can be implemented with arbitrary amplitude i.e. $U_i= \textrm{exp}[-i \theta_i H_i]$. These Hamiltonians form a representation of a Lie Algebra under commutation. If that algebra is equal to $\mathfrak{su}(d)$ for a $d$-dimenstional Hilbert space then the system is said to be controllable~\cite{Khaneja01} and any unitary in SU($d$) can be created with proper choices of $\{\theta_i\}$.

For SRB, we consider a set of Hamiltonians do not generate $\mathfrak{su}(d)$. Instead they generate a representation of another Lie algebra $\mathfrak{g}$, which is a sub algebra of $\mathfrak{su}(d)$. To perform SRB, we identify a subspace of the Hilbert space with dimension $d_1$ where $\mathfrak{g}$ forms $\mathfrak{su}(d_1)$. (If there are multiple such subspaces it is best to pick the largest one to capture the most information about the system.) The system is then said to be subspace controllable, and we can generate any unitary in that subspace. 

Identifying such a subspace may be a non-trivial task. For the trapped-ion example, we were aided by standard results on angular momentum addition in quantum mechanics, i.e. two spin $1/2$ systems add to form spin-1 (triplet) or spin-0 (singlet) states. Such relations could be applied to similar systems with arbitrary dimension subsystems. However, there may be extra leakage subspaces, which complicate the leakage analysis. In these cases, the leakage analysis will only provide identification of leakage rates and not quantitative measurement of their magnitude. Luckily, we can still extract some information on the quality of the gates from the standard decay analysis with the inversion procedure discussed in Sec.~\ref{sec:ssrb}. We consider this case in App.~\ref{sec:ssrb}. 

Once the required subspace is identified, we then generate a set of unitary gates that forms a two-design. For the procedure discussed in Sec.~\ref{sec:ssrb}, $d_1$ must be a prime or a power of a prime. This restriction is due to the construction of the Pauli or Weyl operators that are used to generate the Clifford group. If $d_1$ is not prime or a power of a prime, we could consider other unitary two-designs or randomize over Haar-random unitaries.

The selected gates are then generated by optimal control techniques~\cite{Machnes11}. Again, optimal control requires a numerical search with a full simulation of the unitary. This limits the $d_1$ to be small enough to simulate the corresponding unitary operations classically. While this is a limitation of SRB, standard RB is also limited to small system sizes (the largest published result is on three qubits~\cite{McKay19}). This is for two reasons: (1) the size of the Clifford group grows exponentially with the number of qubits and the optimal decomposition for each Clifford is found by exhaustive search, and (2) the difference in depth between Cliffords blows up with qubit dimension, which violates the standard RB assumption that each Clifford has the same error.

The remainder of the general procedure then follows standard RB and the trapped-ion example considered above. Random gates are applied for different sequence lengths and the total decay rate is measured. We can again determine the standard and leakage decays to derive the extended-sub fidelity.


\section{Conclusions}
We proposed a new method for benchmarking entangling qubit gates in trapped-ion systems, which is experimentally easier to implement and provides more information than standard methods used in such systems. While the method does not capture as much information as standard RB, it can be combined with LB to identify most errors that occur. We also proposed a modified LB procedure that allows for separating the leakage decay from other error rates that should be useful with any LB procedure.

We implemented this technique on a trapped-ion system to measure the quality of entangling gates. We verified a low error phase gate with $1-f_{ZZ}' = 2(1) \times 10^{-3}$. For our system, SRB is faster than both standard RB with two qubits (since it does not require transport) and Bell-state fidelity measurements (since it does not have standard quantum limit scaling with precision). This makes it an excellent method to use in the design-and-test cycle of tuning up high-fidelity entangling gates.


\begin{acknowledgments}
The authors would like to thank Patty Lee for her helpful comments on an earlier version of this manuscript.
\end{acknowledgments}



