Reverse-Engineering Deep ReLU Networks

Abstract: It has been widely assumed that a neural network cannot be recovered from its outputs, as the network depends on its parameters in a highly nonlinear way. Here, we prove that in fact it is often possible to identify the architecture, weights, and biases of an unknown deep ReLU network by observing only its output. Every ReLU network defines a piecewise linear function, where the boundaries between linear regions correspond to inputs for which some neuron in the network switches between inactive and active ReLU states. By dissecting the set of region boundaries into components associated with particular neurons, we show both theoretically and empirically that it is possible to recover the weights of neurons and their arrangement within the network, up to isomorphism.

\section{Introduction}
A deep neural network computes a function from inputs to outputs, where the structure and parameters of the network control the function that is expressed. While each parameter influences the overall function, this influence is highly nonlinear, and the effects of different neurons within the network would appear superficially to be hopelessly entangled. Consequently, it has been widely supposed in the field that it is impossible to recover the structure and parameters of a network merely by observing its output on different inputs.

Were it possible to reverse-engineer a neural network from the function it computes, there could be serious implications for security and privacy. In many deployed deep learning systems, the output is freely available but the network used to generate that output is not. The ability to recover a confidential network could even expose data used to train the network if such data could be reconstructed from the network's weights.

The related question of reverse-engineering biological neural networks is of foundational interest in neuroscience. Experimental neuroscientists can record the output of some neurons (e.g.~complex cells in primary visual cortex) but not others (e.g.~simple cells) and must also infer the synaptic weights between them. Our understanding of the brain would be greatly improved if it were possible to identify the internal components of a neural circuit based on recordings of the output of that circuit.

In this work, we show mathematically and empirically that it is, in fact, often possible to recover the structure and weights of an unknown ReLU network by querying it. Our approach leverages the fact that a ReLU network is piecewise linear and transitions between linear pieces when one of the ReLUs of the network transitions from its inactive to its active state. For neurons in the first layer of the network, such transitions occur along hyperplanes through input space; the equations of these hyperplanes determine the weights and biases of the first layer (up to sign and constant scaling). For neurons in subsequent layers, transitions occur along ``bent hyperplanes'' that bend where they intersect bent hyperplanes associated with earlier layers. Measuring the intersections between bent hyperplanes allows us to recover the weights between the corresponding neurons.

Our principal contributions are:
\begin{itemize}
    \item We prove that the architecture, weights, and biases of a deep ReLU network can be recovered (up to isomorphism) from the arrangement of regions on which the network is linear.
    \item We describe an algorithm to recover the network by approximating the boundaries between these linear regions, with the only information given about the network being its output on specified queries.
    \item We demonstrate the success of our algorithm in reverse-engineering both trained and untrained ReLU networks.
\end{itemize}

Each of these contributions significantly advances the state of the art. No prior work has, to our knowledge, been able to deduce even the first layer of a fully-connected network with 2 hidden layers. By contrast, our theoretical results and algorithm hold for reverse-engineering any layer of a network of any depth. Furthermore, we show empirically that our algorithm is able to reconstruct the first layer of 2-, 3-, and 4-layer networks, as well as the second layer of 2-layer networks.

\begin{figure*}[ht]
\begin{center}
\includegraphics[scale=0.43]{figures/new_combo_figure.png}
\caption{Left: Architecture of a ReLU network $\mathcal{N}({\bf x}):\mathbb{R}^2\to \mathbb{R}$ with two hidden layers of width 5. Center: Graph of the piecewise linear function $\mathcal{N}({\bf x})$ as a function of the two input variables. Right: Boundaries between linear regions of $\mathcal{N}$, essentially a ``flattened'' form of the center image. Boundaries $B_z$ corresponding to neurons $z$ from the first and second layers are shown in blue and red, respectively.}
\label{fig:boundaries}
\vspace{-0.2in}
\end{center}
\end{figure*}

\section{Related work}
Various works within the deep learning literature have considered the problem of inferring a network given its output on inputs drawn (non-adaptively) from a given distribution. It is known that this problem is in general hard \citep{goel2017reliably}, though positive results have been found for certain specific choices of distribution in the case that the network has only one or two layers \citep{ge2019learning, goel2017learning}.  By contrast, we consider the problem of reconstructing a network of arbitrary depth, given the ability to issue queries at specified input points. In this work, we leverage the theory of linear regions within a ReLU network, an area studied e.g.~by \citet{telgarsky2015representation, raghu2017expressive, hanin2019complexity}. Most recently, \citet{hanin2019deep} considered the boundaries between linear regions as arrangements of ``bent hyperplanes''. \citet{milli2019model, jagielski2019high} show the effectiveness of this strategy for networks with one hidden layer. For inference of other properties of unknown networks, see e.g.~\citet{oh2019towards}.

Neuroscientists have long considered similar problems with biological neural networks, albeit armed with prior knowledge about network structure. For example, it is believed that complex cells in the primary visual cortex, which are often seen as translation-invariant edge detectors, obtain their invariance through what is effectively a two-layer neural network \citep{kording2004complex}. A first layer is believed to extract edges, while a second layer essentially implements max-pooling. Many biological neurons appear to be well modeled as the ReLU of a linear combination of their inputs \citep{chance2002gain}. \citet{heggelund1981receptive} perform physical experiments akin to our approach of identifying one ReLU at a time, by applying inputs that move individual neurons above their critical threshold one by one. Being able to solve such problems more generically would be useful for a range of neuroscience applications.

\section{Theoretical framework}
\label{sec:prelim}

We will, in general, consider fully connected, feed-forward neural networks (multi-layer perceptrons) with ReLU activations. Each such network $\mathcal{N}$ defines a function $\mathcal{N}({\bf x})$ from input space $\mathbb{R}^{n_\text{in}}$ to output space $\mathbb{R}^{n_\text{out}}$. We denote the layer widths of the network by $n_\text{in}$ (input layer), $n_1, n_2,\ldots, n_d$, $n_\text{out}$ (output layer). We denote by ${\bf W}^k$ the weight matrix from layer $(k-1)$ to layer $k$, where layer $0$ is the input; and ${\bf b}^k$ denotes the bias vector for layer $k$. Given a neuron $z$ in the network, we use $z({\bf x})$ to denote its preactivation for input ${\bf x} \in \mathbb{R}^{n_\text{in}}$. Thus, for the $j$th neuron in layer $k$, we have $$z_j^k({\bf x}) = \sum_{i=1}^{n_{k-1}} {\bf W}^k_{ij}\ReLU(z_i^{k-1}({\bf x}) + {\bf b}^k_i).$$

\subsection{Isomorphisms of networks}
\label{subsec:isomorphisms}
Before showing how to reverse-engineer the parameters of a neural network, we must consider to what extent these parameters can be inferred unambiguously. For every network, there are a number of other networks that define exactly the same function from input space to output space. We say that two networks $\mathcal{N}$ and $\mathcal{N}'$ (possibly with different parameters or architecture) are \emph{isomorphic} if $\mathcal{N}({\bf x})=\mathcal{N}'({\bf x})$ for all ${\bf x}\in\mathbb{R}^{n_\text{in}}$. For a fully connected network with ReLU activation, there are two obvious network isomorphisms:

\textbf{Permutation.} The order of neurons in each layer of a network $\mathcal{N}$ does not affect the underlying function. Formally, let $p_{k, \sigma}(\mathcal{N})$ be the network obtained from $\mathcal{N}$ by permuting layer $k$ according to $\sigma$ (along with the corresponding weight vectors and biases). Then, $p_{k, \sigma}(\mathcal{N})$ is isomorphic to $\mathcal{N}$ for every layer $k$ and permutation $\sigma$.

\textbf{Scaling.} Due to the ReLU's equivariance under multiplication, it is possible to scale the incoming weights and biases of any neuron, while inversely scaling the outgoing weights, leaving the overall function unchanged. Formally, for $z$ the $i$th neuron in layer $k$ and for any $c>0$, let $s_{z,c}(\mathcal{N})$ be the network obtained from $\mathcal{N}$ by replacing ${\bf W}^k_{\cdot i}$, ${\bf b}^k_i$, and ${\bf W}^{k+1}$ by $c{\bf W}^k_{\cdot i}$, $cb^k_i$, and $(1/c){\bf W}^{k+1}_{i\cdot}$, respectively. It is simple to prove that $s_{z,c}(\mathcal{N})$ is isomorphic to $\mathcal{N}$ (see Appendix \ref{app:lem}).

It is worth noting that permutation and scaling are isomorphisms for \emph{every} ReLU network, but some networks may have additional isomorphisms. It is shown in \citet{phuong2019functional} that there exist ReLU networks of every architecture for which all isomorphisms are given by some combination of permutation and scaling.\footnote{See also \citet{fefferman1994reconstructing} for a proof that an analogous set of isomorphisms is comprehensive for networks with tanh activation.} As we demonstrate in \S\ref{subsec:boundaries}, there exists a positive-measure set of ReLU networks with additional isomorphisms.

In this work, we show that, where additional isomorphisms do not occur, it is in fact often possible both in theory and in practice to reverse-engineer ReLU networks up to permutation and scaling.

\subsection{Linear regions}
\label{subsec:regions}
Consider a network $\mathcal{N}$ and neuron $z\in \mathcal{N}$. We denote by $B_z$ the set of ${\bf x}$ for which $z({\bf x})=0$. We call $B_z$ the \emph{boundary} associated with neuron $z$, and we say that $B = \bigcup B_z$ is the \emph{boundary} of the overall network. We refer to the connected components of $\mathbb{R}^{n_\text{in}}\setminus B$ as \emph{linear regions}. Thus, each linear region corresponds to a pattern of activations across the ReLUs in the network; within a region, each ReLU computes a fixed linear function (see Fig.~\ref{fig:boundaries}).

Throughout this paper, we will make the \emph{Linear Regions Assumption}: Each region represents a maximal connected component of input space on which the piecewise linear function $\mathcal{N}({\bf x})$ is given by a single linear function. While this assumption has tacitly been made in the prior literature, it is noted in \citet{hanin2019deep} that there are cases where it does not hold. For example, if an entire layer of the network is zeroed out for some inputs, then $\mathcal{N}({\bf x})$ may be given by the same linear function across several adjacent regions, despite these regions corresponding to different ReLU activation patterns.

\subsection{Main result}
\citet{hanin2019deep} show that for all but a measure-zero set of networks, $B_z$ is an $(n_\text{in}-1)$-dimensional piecewise linear surface in $\mathbb{R}^{n_\text{in}}$. We say that $B_z$ \emph{bends} at a point if $B_z$ is nonlinear at that point. As observed in \citet{hanin2019deep}, $B_z$ can bend only at points where it intersects boundaries $B_{z'}$ for $z'$ in an earlier layer of the network (see Fig.~\ref{fig:boundaries}, in which input dimension is 2 and the $B_z$ are simply bent lines). The following theorem (proven in Appendix \ref{app:thm_structure}) shows that the converse also holds.

\begin{thm}[Boundaries imply network structure] Let $\mathcal{N}$ be a fully connected ReLU network satisfying the Linear Regions Assumption. Then, the following statement holds except for $\mathcal{N}$ in a measure-zero set of networks: For every neuron $z$, the boundary $B_z$ bends exactly where it intersects a boundary $B_{z'}$ such that $z'$ is in an earlier layer than $z$.
\label{thm:structure}
\end{thm}

From this theorem, it follows that for any two intersecting boundaries $B_z$ and $B_{z'}$, one of the following must hold: $B_z$ bends at their intersection (in which case $z$ occurs in a deeper layer of the network), $B_{z'}$ bends (in which case $z'$ occurs in a deeper layer), or neither bends (in which case $z$ and $z'$ occur in the same layer). It is not possible for both $B_z$ and $B_{z'}$ to bend at their intersection -- unless that intersection is also contained in another boundary, which is vanishingly unlikely in general. Therefore, the architecture of the network can be determined by evaluating the boundaries $B_z$ and where they bend in relation to one another.

In fact, a much stronger statement holds; namely, the weights and biases of the network can also be determined from the boundaries:

\begin{thm}[Boundaries imply network weights] Let $\mathcal{N}$ be a fully connected ReLU network satisfying the Linear Regions Assumption. Suppose that for any two neurons $z$ and $z'$ that are connected in $\mathcal{N}$, the boundaries $B_z$ and $B_{z'}$ intersect. Then, given the set of boundaries between linear regions, it is possible to recover both the complete architecture of $\mathcal{N}$ and the weights and biases of every hidden layer, up to permutation and scaling, except for $\mathcal{N}$ in a measure-zero set of networks.
\label{thm:weights}
\end{thm}

This result follows from Theorem \ref{thm:correct}, which states that not only is it possible to recover the structure and weights of a network from the boundaries between linear regions, but the boundaries themselves can effectively be approximated by the algorithm we describe in \S\ref{sec:alg}, enabling the network to be reverse-engineered merely by querying it. We prove Theorem \ref{thm:correct} in Appendix \ref{app:thm_correct}.

As an intuition for why Theorem \ref{thm:weights} holds, observe first that for neurons $z$ in the first layer of the network, the boundaries $B_z$ are simply hyperplanes (see Fig.~\ref{fig:boundaries}). The equations of these hyperplanes reveal the weights from the input to the first layer (up to permutation, scaling, and sign). For each subsequent layer, the weight between neurons $z$ and $z'$ can be determined by calculating how $B_{z'}$ bends when it crosses $B_z$, as this dictates how much the input to $z$ changes when neuron $z'$ is zeroed out by its ReLU.

\section{Algorithm}
\label{sec:alg}

We now describe an algorithm to reverse-engineer a network $\mathcal{N}$ by approximating the boundaries between linear regions. Our algorithm assumes that the output of the network $\mathcal{N}({\bf x})$ can be queried for different inputs ${\bf x}$, but does not assume any \emph{a priori} knowledge of the linear regions or boundaries.

\label{sec:algorithm}
\begin{figure*}[ht]
\begin{center}
\includegraphics[scale=0.38]{figures/algorithm.png}
\caption{Schematic of our algorithms for identifying the first layer and additional layers.}
\vspace{-0.2in}
\label{fig:algorithm}
\end{center}
\end{figure*}

\subsection{The first layer}
\label{subsec:first_layer}
We begin by identifying the first layer of the network $\mathcal{N}$, for which we must infer the number of neurons, the weight matrix ${\bf W}^1$, and the bias vector ${\bf b}^1$. As noted above, for each $z=z_i^1$ in the first layer, the boundary $B_z$ is a hyperplane with equation ${\bf W}^1_{\cdot i}{\bf x} + {\bf b}^1_i = 0$. For each neuron $z$ in a later layer of the network, the boundary $B_z$ will, in general, bend and not be a (full) hyperplane (Theorem \ref{thm:structure}). We may therefore find the number of neurons in layer 1 by counting the hyperplanes contained in the network's boundary $B$, and we can infer weights and biases by determining the equations of these hyperplanes.
\begin{algorithm}[H]
\caption{The first layer}\label{fig:first_layer}
\begin{algorithmic}
\STATE Initialize $P_1=P_2=S_1=\{\}$
\FOR{$t=1,\ldots,L$}
	\STATE Sample line segment $\ell$
	\STATE $P_1\gets P_1\cup \texttt{PointsOnLine}(\ell)$
\ENDFOR
\FOR{${\bf p} \in P_1$}
	\STATE $H =\texttt{InferHyperplane}({\bf p})$
	\IF{$\texttt{TestHyperplane}(H)$}
		\STATE $S_1\gets S_1 \cup \texttt{GetParams}(H)$
	\ELSE \STATE $P_2\gets P_2 \cup \{{\bf p}\}$
	\ENDIF
\ENDFOR
\STATE $\text{\textbf{return} parameters }S_1,$\\
$\hskip .4 in\text{unused sample points }P_2$
\end{algorithmic}
\end{algorithm}
\vspace{-0.1in}

\textbf{Boundary points along a line.} Our algorithm is based upon the identification of points on the boundary $B$. One of our core algorithmic primitives is a subroutine \texttt{PointsOnLine} that takes as input a line segment $\ell \subset \mathbb{R}^n_\text{in}$ and approximates the set $\ell \cap B$ of boundary points along $\ell$. The algorithm proceeds by leveraging the fact that boundary points subdivide $\ell$ into regions within which $\mathcal{N}({\bf x})$ is linear. We maintain a list of points in order along $\ell$ (initialized to the endpoints and midpoint of $\ell$) and iteratively perform the following operation: For each three consecutive points on our list, ${\bf x}_1,{\bf x}_2,{\bf x}_3$, we determine if the vectors $(\mathcal{N}({\bf x}_2) - \mathcal{N}({\bf x}_1))/||{\bf x}_2 - {\bf x}_1||_2$ and $(\mathcal{N}({\bf x}_3) - \mathcal{N}({\bf x}_2))/||{\bf x}_3 - {\bf x}_2||_2$ are equal (to within computation error) -- if so, we remove the point ${\bf x}_2$ from our list, otherwise we add the points $({\bf x}_1 + 2{\bf x}_2)/3$ and $({\bf x}_3 + 2{\bf x}_2)/3$ to our list.\footnote{These weighted averages speed up the search algorithm by biasing it towards the center of the segment, which is where we expect the most intersections given our choice of segments.} The points in the list converge by binary search to the set of discontinuities of the gradient $\nabla\mathcal{N}({\bf x})$, which are our desired boundary points. Note that \texttt{PointsOnLine} is where we make use of our ability to query the network.

\textbf{Sampling boundary points.} In order to identify the boundaries $B_z$ for $z$ in layer 1, we begin by identifying a set of boundary points with at least one on each $B_z$. A randomly chosen line segment through input space will intersect some of the $B_z$ -- indeed, if it is long enough, it will intersect any fixed hyperplane with probability 1. We sample line segments $\ell$ in $\mathbb{R}^n_\text{in}$ and run \texttt{PointsOnLine} on each. Many sampling distributions are possible, but in our implementation we choose to sample segments of fixed (long) length, tangent at their midpoints to a sphere of fixed (large) radius.\footnote{The algorithm is not sensitive to the exact settings of these hyperparameters, provided they are relatively large.} This ensures that each of our sample lines remains far from the origin, where boundaries are in closer proximity and therefore more easily confused with one another (this will become useful in the next step). Let $P_1$ be the overall set of boundary points identified on our sample line segments.

\textbf{Inferring hyperplanes.} We now fit a hyperplane to each of the boundary points we have identified. For each ${\bf p}\in P_1$, there is a neuron $z$ such that ${\bf p} \in B_z$. The boundary $B_z$ is piecewise linear, with nonlinearities only along other boundaries, and with probability $1$, ${\bf p}$ does not lie on a boundary besides $B_z$. Therefore, within a small enough neighborhood of ${\bf p}$, $B_z$ is given by a hyperplane, which we call the \emph{local hyperplane} at ${\bf p}$. If $z$ is in layer 1, then $B_z$ equals the local hyperplane. The subroutine \texttt{InferHyperplane} takes as input a point ${\bf p}$ on a boundary $B_z$ and approximates the local hyperplane within which ${\bf p}$ lies. This algorithm proceeds by sampling many small line segments around ${\bf p}$, running \texttt{PointsOnLine} to find their points of intersection with $B_z$, and performing linear regression to find the equation of the hyperplane containing these points.

\textbf{Testing hyperplanes.} Not all of the hyperplanes we have identified are actually boundaries for neurons in layer 1, so we need to test which hyperplanes are contained in $B$ in their entirety, and which are the local hyperplanes of boundaries that bend. The subroutine \texttt{TestHyperplane} takes as input a point ${\bf p}$ and a hyperplane $H$ containing that point, and determines whether the entire hyperplane $H$ is contained in the boundary $B$ of the network. This algorithm proceeds by sampling points within $H$ that lie far from ${\bf p}$ and applying \texttt{PointsOnLine} to a short line segment around each such point to check whether these points all lie on $B$. Applying \texttt{TestHyperplane} to those hyperplanes inferred in the preceding step allows us to determine those $B_z$ for which $z$ is in layer 1.

\textbf{From hyperplanes to parameters.} Finally, we identify the first layer of $\mathcal{N}$ from the equations of hyperplanes contained in $B$. The number of neurons in layer 1 is given simply by the number of distinct $B_z$ that are hyperplanes.  As we have observed, for $z=z_i^1$ in layer 1, the hyperplane $B_z$ is given by ${\bf W}^1_{\cdot i}{\bf x} + {\bf b}^1_i = 0$. We can thus determine ${\bf W}^1_{\cdot i}$ and ${\bf b}^1_i$ up to multiplication by a constant. However, we have already observed that scaling ${\bf W}^1_{\cdot i}$ and ${\bf b}^1_i$ by a positive constant (while inversely scaling ${\bf W}^2_{i\cdot}$) is a network isomorphism (\S\ref{subsec:isomorphisms}). Therefore, we need only determine the true sign of the multiplicative constant, corresponding to determining which side of the hyperplane is zeroed out by the ReLU. This determination of sign will be performed in \S\ref{subsec:other_layers}.

\subsection{Additional layers}
\label{subsec:other_layers}
We now assume that the weights ${\bf W}^1, \ldots, {\bf W}^{k-1}$ and biases ${\bf b}^1, \ldots, {\bf b}^{k-1}$ have already been determined within the network $\mathcal{N}$, with the exception of the sign choice for weights and biases at each neuron in layer $k-1$. We now show how it is possible to determine the weights ${\bf W}^k$ and biases ${\bf b}^k$, along with the correct signs for ${\bf W}^{k-1}$ and ${\bf b}^{k-1}$.
\begin{algorithm}[H]
\caption{Additional layers}\label{fig:additional_layers}
\begin{algorithmic}
\STATE Input $P_k$ and $S_1,\ldots,S_{k-1}$
\STATE Initialize $S_k=\{\}$
\FOR{${\bf p}_1 \in P_{k-1}$ on boundary $B_z$}
	\STATE Initialize $A_z = \{{\bf p}_1\}$, $L_z = \mathcal{H}_z=\{\}$
	\WHILE{$L_z \not\supseteq \text{Layer }k-1$}
	\STATE Pick ${\bf p}_i\in A$ and ${\bf v}$
	\STATE ${\bf p}',B_{z'}=\texttt{ClosestBoundary}({\bf p}_i,{\bf v})$
	\IF{${\bf p}'$ on boundary}
	\STATE $A_z\gets A_z\cup \{{\bf p}' + {\bf\epsilon}\}$
	\STATE $L_z\gets L_z\cup \{z'\}$
	\STATE $\mathcal{H}_z\gets \mathcal{H}_z\cup \{\texttt{InferHyperplane}({\bf p}_i)\}$
	\ELSE \STATE $P_k\gets P_k\cup \{{\bf p}_1\}$; \textbf{break}
	\ENDIF
	\ENDWHILE
	\IF{$L_z\supseteq \text{Layer }k-1$}
		\STATE $S_k\gets \texttt{GetParams}(T_z)$
	\ENDIF
\ENDFOR
\STATE $\text{\textbf{return} parameters }S_k,\text{unused sample points }P_{k+1}$
\end{algorithmic}
\end{algorithm}
\vspace{-0.1in}
\textbf{Closest boundary along a line.} In this part of our algorithm, we will need the ability to move along a boundary to its intersection with another boundary. For this purpose, the subroutine \texttt{ClosestBoundary} will be useful. It takes as input a point ${\bf p}$, a vector ${\bf v}$ and the network parameters as determined up to layer $k-1$, and outputs the smallest $c>0$ such that ${\bf q} = {\bf p} + c{\bf v}$ lies on $B_z$ for some $z$ in layer at most $k-1$. In order to compute $c$, we consider the region $R$ within which ${\bf p}$ lies, which is associated with a certain pattern of active and inactive ReLUs. For each boundary $B_z$, we calculate the hyperplane equation which would define $B_z$ were it to intersect $R$, due to the fixed activation pattern within $R$, and we calculate the distance from ${\bf p}$ to this hyperplane. While not every boundary $B_z$ intersects $R$, the closest boundary does, allowing us to find the desired $c$.

\textbf{Unused boundary points.} In order to identify the boundaries $B_z$ for $z$ in layer $k$, we wish to identify a set of boundary points with at least one on each such boundary. However, in previous steps of our algorithm, a set $P_{k-1}$ of boundary points was created, of which some were used in ascertaining the parameters of earlier layers. We now consider the subset $P_k \subset P_{k-1}$ of points that were not found to belong to $B_z$, for $z$ in layers $1$ through $k-1$. These points have already had their local hyperplanes determined.

\textbf{Exploring boundary intersections.} Consider a point ${\bf p}_1\in P_k$ such that ${\bf p}_1 \in B_z$. Note that $B_z$ will, in general, have nonlinearities where it intersects $B_{z'}$ such that $z'$ lies in an earlier layer than $z$. We explore these intersections, and in particular attempt to find a point of $B_z \cap B_{z'}$ for every $z'$ in layer $k-1$. Given the local hyperplane $H$ at ${\bf p}_1$, we pick a direction ${\bf v}$ along $H$ and apply \texttt{ClosestBoundary} to calculate the closest point of intersection ${\bf p}'$ with $B_{z'}$ for all $z'$ already identified in the network. (We discuss below how to pick ${\bf v}$.) Note that if $z$ is in layer $k$, then ${\bf p}'$ must be on $B_z$ as well as $B_{z'}$, while if $z$ is in a later layer of the network, then there may exist unidentified neurons in layers below $z$ and therefore $B_z$ may bend before meeting $B_{z'}$. We check if ${\bf p}'$ lies on $B_z$ by applying \texttt{PointsOnLine}, and if so apply \texttt{InferHyperplane} to calculate the local hyperplane of $B_z$ on the other side of $B_{z'}$ from ${\bf p}_1$. We select a representative point ${\bf p}_2$ on this local hyperplane.  We repeat the process of exploration from the points ${\bf p}_1, {\bf p}_2, \ldots$ until one of the following occurs: (i) a point of $B_z \cap B_{z'}$ has been identified for every $z'$ in layer $k-1$ (this may be impossible; see \S\ref{subsec:boundaries}), (ii) $z$ is determined to be in a layer deeper than $k$ (as a result of ${\bf p}'$ not lying on $B_z$), or (iii) a maximum number of iterations has been reached.

\textbf{How to explore.} An important step in our algorithm is exploring points of $B_z$ that lie on other boundaries. Given a set of points $A_z = \{{\bf p}_1, {\bf p}_2, \ldots, {\bf p}_m\}$ on $B_z$, there are several methods for picking a point ${\bf p}_i$ and direction ${\bf v}$ along the local hyperplane at ${\bf p}_i$ to apply \texttt{ClosestBoundary}. One approach is to pick a random point ${\bf p}_i$ from those already identified and a random direction ${\bf v}$; this has the advantage of simplicity. However, it is somewhat faster to consider for which $z'$ the intersection $B_z\cap B_{z'}$ has not yet been identified and attempt specifically to find points on these intersections. One approach for this is to pick a missing $z'$ and identify for which ${\bf p}_i$ the boundary $B_{z'}$ lies on the boundary of the region containing ${\bf p}_i$ and solve a linear program to find ${\bf v}$. Another approach is to pick a missing $z'$ and a point ${\bf p}_i$, calculate the hyperplane $H$ which would describe $B_{z'}$ under the activation pattern of ${\bf p}_i$, and choose ${\bf v}$ along the local hyperplane to ${\bf p}_i$ such that the distance to $H$ is minimized. This is the approach which we take in our implementation, though more sophisticated approaches may exist and present an interesting avenue for further work.

\textbf{From boundaries to parameters.} We now identify layer $k$ of $\mathcal{N}$, along with the sign of the parameters of layer $k-1$, by measuring the extent to which boundaries bend at their intersection. We are also able to identify the correct signs at layer $k-1$ by solving an overconstrained system of constraints capturing the influence of neurons in layer $k-1$ on different regions of input space. Theorem \ref{thm:correct} formalizes the inductive step that allows us to go from what we know at layer $k-1$ (weights and biases, up to scaling and sign) to the equivalent set of information for layer $k$.

\section{Discussion}
\subsection{Correctness and sample complexity}
\label{subsec:complexity}
The following theorem (proven in Appendix \ref{app:thm_correct}) establishes the validity of our algorithm above.

\begin{thm}[Algorithm proof of correctness] The above algorithm successfully recovers, up to permutation and scaling, the structure and weights of any deep ReLU network for which the assumptions of Theorem \ref{thm:weights} hold. No prior knowledge of the weights, structure, or linear region boundaries is assumed, merely the ability to query the network.

Specifically, the following holds true for fully connected ReLU networks $\mathcal{N}$ satisfying the Linear Region Assumption (\S\ref{subsec:regions}), excluding a set of networks with measure zero: Suppose that the weights and biases of $\mathcal{N}$ are known up through layer $k-1$, with the exception that for each neuron in layer $k-1$, the sign of the incoming weights and the bias is unknown. Suppose also that for each $z$ in layer $k$, there exists an ordered set of points $A_z= \{{\bf p}_1, {\bf p}_2, \ldots, {\bf p}_m\}$ such that: (i) Each point lies on the boundary of $B_z$, and in (the interior of) a distinct region with respect to the earlier-layer boundaries already known; (ii) each point (except for ${\bf p}_1$) has a precursor in an adjacent region; (iii) for each such pair of points, the local hyperplanes of $B_z$ are known, as is the boundary $B_{z'}$ dividing them ($z'$ in an earlier layer); (iv) the set of such $z'$ includes all of layer $k-1$.

Then, it is possible to recover the weights and biases for layer $k$, with the exception that for each neuron, the sign of the incoming weights and the bias is unknown. It is also possible to recover the sign for every neuron in layer $k-1$.
\label{thm:correct}
\end{thm}

Note that even when assumption (iv) of the Theorem is violated, the algorithm recovers the weights corresponding to whichever boundaries are successfully crossed, as we verify empirically in \S\ref{sec:experiments}.

We expect the number of queries necessary to obtain weights and biases (up to sign) for the first layer should grow as $O(n_\text{in} (\sum_i n_i)\log n_1)$, which for constant-width networks is only slightly above the number of parameters being inferred. Namely, if the biases in the network are bounded above, then each sufficiently long line has constant probability of hitting a given hyperplane, suggesting $\log n_1$ lines are required according to a coupon collector-style argument. \citet{hanin2019complexity} prove that under natural assumptions, the number of boundary points intersecting a given line through input space grows linearly in the total number of neurons in the network. Finally, each boundary point on a line requires $O(n_\text{in})$ queries in order to fit a hyperplane.

For deeper layers, the number of queries depends on the approach taken to explore the intersections between boundaries. It is possible, depending on the arrangement of intersections, for the number of queries to grow linearly in the number of parameters, as each weight can be inferred by examining a single intersection between boundaries.

\subsection{Assumptions}
\label{subsec:boundaries}
It is possible that for some neurons $z$ and $z'$ in consecutive layers, there is no point of intersection between the boundaries $B_z$ and $B_{z'}$ (or that this intersection is very small), making it impossible to infer the weight between $z$ and $z'$ by our algorithm. Some such cases represent an ambiguity in the underlying network -- an additional isomorphism to those described in \S\ref{subsec:isomorphisms}. Namely, $B_z\cap B_{z'}$ is empty if one of the following cases holds: (1) whenever $z$ is active, $z'$ is inactive; (2) whenever $z$ is active, $z'$ is active; (3) whenever $z$ is inactive, $z'$ is inactive; or (4) whenever $z$ is inactive, $z'$ is active.  In case 1, observe that a slight perturbation to the weight $w$ between $z$ and $z'$ has no effect upon the network's output; thus $w$ is not uniquely determined. Cases 2-4 present a more complicated picture; depending on the network, there may or may not be additional isomorphisms.

For simplicity in our algorithm, we have not considered the relatively rare cases where boundaries $B_z$ are disconnected or bounded. If $B_z$ is disconnected, then it may not be possible to find a connected path along it that intersects all boundaries arising from the preceding layer. In this case, it is simple to infer that two independently identified pieces of the boundary belong to the same neuron to infer the full weight vector.  Next, if $B_z$ is bounded for some $z$, then it is a closed $(d-1)$-dimensional surface within $d$-dimensional input space\footnote{For 2D input, such $B_z$ must be topological circles, but for higher dimensions, it is conceivable for them to be more complicated surfaces, such as toroidal polyhedra.}. While our algorithm requires no modification in this case, bounded $B_z$ may be more difficult to find by intersection with randomly chosen lines, and a more principled sampling method may be helpful.

\subsection{Other architectures} While we have expressed our algorithm in terms of multi-layer perceptrons with ReLU activation, it also extends to various other architectures of neural network. Other piecewise linear activation functions admit similar algorithms. For a network with convolutional layers, it is possible to use the same approach to infer the weights between neurons, with two caveats: (i) As we have stated it, the algorithm does not account for weight-sharing -- the number of ``neurons'' in each layer is thus dependent on the input size, and is very large for reasonably sized images. (ii) Pooling layers \emph{do} affect the partition into activation regions, and indeed introduce new discontinuities into the gradient; our algorithm therefore does not apply.

For skip connections as in ResNets \citep{he2016deep}, our algorithm should hold with slight modifications, which we outline here. As in a multi-layer perceptron, each boundary component $B_z$ is a bent hyperplane that bends when it intersects $B_{z'}$ for $z'$ at an earlier layer than $z$. However, potential weights must in this case be considered between neurons in any two different layers. Deriving the weights for such skip connections is somewhat more complex than for multi-layer perceptrons, as the ``bend'' is influenced not merely by the skip connection but by the weights along all other paths between the two neurons through the network. Thus, it is necessary to “move backward” through the network -- for a neuron in layer $k$, one must first derive the weights of connections arising from the preceding layer $k-1$, then from $k-2$, and so on.

\section{Experiments}
\label{sec:experiments}
We demonstrate the success of our algorithm on both untrained and trained networks. In keeping with literature on ReLU network initialization \citep{he2015delving,hanin2018start}, networks were initialized using i.i.d.~normal weights with variance $2/\text{fan-in}$ and i.i.d.~normal biases with unit variance. Networks were trained on either the MNIST dataset ($n_\text{in} = 784,n_\text{out} = 10$) or a memorization task of 1000 ``datapoints'' ($n_\text{in} = 10,n_\text{out} = 2$) with coordinates drawn i.i.d.~from a unit Gaussian and given arbitrary binary labels. Training was performed using the Adam optimizer \citep{kingma2014adam} and a cross-entropy loss applied to the softmax of the final layer, over 20 epochs for MNIST and 1000 epochs for the memorization task. The trained networks (when sufficiently large) were able to attain near-perfect accuracy.
\begin{figure*}[ht]
\begin{center}
\includegraphics[scale=0.24]{figures/new_layer1.png}
\vspace{-0.2in}
\caption{Results of our first-layer algorithm, applied to networks with two or more hidden layers as the width of the first layer varies. All other layers have fixed width 10. Untrained networks have input and output dimension 10, those trained on the memorization task have input dimension 10 and output dimension 2, and those trained on MNIST have input dimension 784 and output dimension 10. Left: The number of queries issued by our algorithm per parameter identified in the network $\mathcal{N}$; the algorithm is terminated once the desired number of neurons have been identified. Right: Log normalized error $\log(||\hat{{\bf W}}^1 - {\bf W}^1||_2 / ||\hat{{\bf W}^1}||_2)$ for $\hat{{\bf W}}^1$ the approximated weights. Weight vectors were scaled to unit norm to account for isomorphism (see \S\ref{subsec:isomorphisms}).  Curves are averaged over 5 runs in the case of MNIST and 40 runs otherwise, with standard deviations between runs shown as shaded regions.}
\vspace{-0.1in}
\label{fig:layer1}
\end{center}
\end{figure*}

We observe that both the first-layer algorithm and additional-layer algorithm identified weights and biases to within extremely high accuracy (see Figs.~\ref{fig:layer1} and \ref{fig:layer2}). In order to compare estimated and true parameters, we scaled weights at each neuron to unit norm and accounted for possible permutations of the neurons within a layer. (As described in \S\ref{subsec:isomorphisms}, these transformations do not change the function computed by the network and represent unavoidable isomorphisms in recovering the network.) Figures show the log normalized error $\log(||\hat{{\bf W}}^k - {\bf W}^k||_2 / ||\hat{{\bf W}^k}||_2)$ between true weights ${\bf W}^k$ and approximated weights $\hat{{\bf W}}^k$ and likewise the log normalized error $\log(||\hat{{\bf b}}^k - {\bf b}^k||_2 / ||\hat{{\bf b}^k}||_2)$ between true biases ${\bf b}^k$ and approximated weights $\hat{{\bf b}}^k$.
\begin{figure*}[ht]
\begin{center}
\includegraphics[scale=0.24]{figures/new_layer2.png}
\vspace{-0.2in}
\caption{Results of our algorithm for additional layers, applied to networks with two layers, the first layer of width 10, as the width of the second layer varies. Left: Number of estimated layer-2 neurons. Right: Log normalized error between estimated and corresponding true neurons for approximated layer-2 weights and biases. Curves are averaged over 4 runs, with standard deviations shown as shaded regions.}
\vspace{-0.2in}
\label{fig:layer2}
\end{center}
\end{figure*}

In keeping with our analysis in \S\ref{subsec:complexity}, the number of queries needed to recover the first layer of a network scales linearly with the number of neurons in that layer (Fig.~\ref{fig:layer1}). In the second layer, a small fraction of weights are sometimes not identified (see \S\ref{subsec:boundaries} for a discussion of reasons why such behavior is inevitable); even in such cases, the algorithm is able to correctly predict the remaining parameters (Fig.~\ref{fig:layer2}).

\section{Conclusion}
In this work, we have proven that it is possible to recover the architecture, weights, and biases of deep ReLU networks from the boundaries between linear regions defined by the network. In many cases, we show that it is possible to reconstruct these boundaries and thus the network itself merely by querying the network's output on certain inputs. Networks can, we observe, often be reconstructed unambiguously up to permutation of neurons within each layer and scaling of weights and biases at individual neurons. In some cases, there are additional isomorphisms, in which case our algorithm is able to reconstruct a significant fraction of the parameters of the network.

Our approach works for a wide variety of networks, though not all. It is limited to ReLU or otherwise piecewise linear activation functions, though we believe it possible that a continuous version of this method could potentially be developed in future work for use with sigmoidal activation. If used with convolutional layers, our method does not account for the symmetries of the network and therefore scales with the size of the input as well as the number of features, resulting in high computation. Finally, the method is not robust to defenses such as adding noise to the outputs of the network, and therefore can be thwarted by a network designer that seeks to hide their weights/architecture.

We believe that the methods we have introduced here will lead to considerable advances in reverse-engineering neural networks, both in the context of deep learning and, more speculatively, in neuroscience. While the implementation we have demonstrated here is effective in small instances, we anticipate future work that optimizes these methods for efficient use with different architectures and at scale.

Machine Learning (cs.LG); Machine Learning (stat.ML)

Reverse-Engineering Deep ReLU Networks


Remark on a theorem in Mumford's Red Book of Varieties and Schemes

Abstract: In this paper, we firstly point out, by a counter example, that Proposition 6.4 of Section 6 in Bump's book (Algebraic Geometry) is error, and then give a correct statement with proof. We finally point out a gap in the proof of Theorem 3, in Chapter I Section 8, of Mumford's red book, and indicate a way to complete it.

\section{Introduction}

In order to show \cite[Chapter I, \S~8, Theorem3]{M}, Bump divided his proof into four propositions in \cite{B}. But, there is a mistake in one of these propositions (\cite[Proposition 6.4]{B}). The original proof of Mumford is based on similar ideas, so we find a similar gap in the proof of \cite[Chapter I, \S~6, Theorem 3]{M}. In the following, we first give a counter example of the last statement in \cite[Proposition 6.4]{B}, then present a correct statement and prove it. Finally, we complete the proof of Theorem 3 in Mumford's red book \cite{M}.

{\medskip}{\noindent}{\bf Acknowledgments.} I would like to express my sincere gratitude to Jilong Tong for an interesting discussion.

\section{A counter example}

Let $k$ be an algebraically closed field. According to  ~\cite{B}, an algebraic set is a \emph{variety} if it is irreducible. The following proposition can be found in the book of Bump:

\begin{proposition}[\cite{B} Proposition 6.4]\label{p:1}~Let $\phi:X\rightarrow Y$ be a finite dominant morphism of affine varieties. Then $\phi$ is surjective. The fibers of $\phi$ are all finite. If $Z$ is a closed subset of $X,$
then $\phi(Z)$ is closed, and $\operatorname{dim}(Z)=\operatorname{dim}(\phi(Z)).$ If $W$ is closed subvariety of $Y$, and $Z$ is any irreducible component of $\phi^{-1}(W),$ then $\phi(Z)=W,$ and $\operatorname{dim} (Z)=\operatorname{dim} (W).$
\end{proposition}

The last part of  this proposition is wrong, and here is a counter example. Assume $k=\mathbb{C}$. Let $X=V(x^2+y^2+(z-\frac{1}{2})^2=\frac{1}{4})\subset \mathbb{A}^3$, and $Y$ the image of the morphism below
$$
\phi:X\longrightarrow \mathbb{A}^3, \quad (x,y,z)\longmapsto ((1-2z)x,(1-3z)y,(1-z)z).
$$
Let $W=\phi(X\cap V(z-\frac{1}{4}))$. Note that $X\cap V(z-\frac{1}{4})$ is an irreducible closed subset of $X$.

\noindent {\bf Claim:} $\phi$ is a finite morphism, and
$$
\phi^{-1}(W)=X\cap V\left(z-\frac{1}{4}\right)\bigcup\left\{\left(\frac{\sqrt{3}}{4},0,\frac{3}{4}\right), \left(-\frac{\sqrt{3}}{4},0,\frac{3}{4}\right)\right\}
$$
In particular, $Z:=\{(-\frac{\sqrt{3}}{4},0,\frac{3}{4})\}$ is an irreducible component of $\phi^{-1}(W)$ such that $\phi(W)\neq Z$.
\begin{proof} Set $A=k[x,y,z]/(x^2+y^2+z^2-z)$. By abuse of notation, we shall use the same symbols to denote the images of $x,y,z\in k[x,y,z]$ in the quotient $A$. We first show that $\phi$ is a finite morphism. Consider the following morphism of $k$-algebras induced by $\phi$
$$
\lambda:k[a,b,c]\rightarrow A, \quad (a,b,c)\mapsto ((1-2z)x,(1-3z)y,(1-z)z),
$$
which makes $A$ an algebra over $k[a,b,c]$. We need show that $A$ is integral over $k[a,b,c]$. It is clear that $z$ is integral over $k[a,b,c]$ since $z^2-z+\lambda(c)=0$ by the definition of $\lambda$. Moreover, as $x^2+y^2+z^2-z=0$ in $A$, to see that $A$ is integral over $k[a,b,c]$, it suffices to show that $y$ is integral over the $k$-subalgebra $k[\lambda(a),\lambda(b),\lambda(c),z]\subseteq A$ of $A$ generated by $\lambda(a),\lambda(b),\lambda(c),z\in A$. We shall do this by finding an integral relation for it. First, by the definition of $\lambda$, we have
$$
zx =\frac{x-\lambda(a)}{2},\quad zy  =\frac{y-\lambda(b)}{3},\quad \textrm{and} \quad
x^2+y^2 =\lambda(c),
$$
giving
\begin{eqnarray*}
\lambda(b)^2 & =& [(1-3z)y]^2=y^2-6zy^2+9z^2y^2\\
&= & y^2-6zy^2+9(z^2\lambda(c)-z^2x^2)\\
& = & y^2-6y\frac{y-\lambda(b)}{3}-9(\frac{x-\lambda(a)}{2})^2+9z^2\lambda(c)\\
& = & -y^2+2\lambda(b)y-\frac{9}{4}x^2+\frac{9}{2}\lambda(a)x-\frac{9}{4}\lambda(a)^2+9z^2\lambda(c)\\
&=& -y^2-\frac{9}{4}(\lambda(c)-y^2)+2\lambda(b)y+\frac{9}{2}\lambda(a)x-\frac{9}{4}\lambda(a)^2+9z^2\lambda(c)\\
&=& \frac{5}{4}y^2+2\lambda(b)y+\frac{9}{2}\lambda(a)x-\frac{9}{4}\lambda(a)^2+9z^2\lambda(c)-\frac{9}{4}\lambda(c).
\end{eqnarray*}
Let $w=-\frac{9}{4}\lambda(a)^2+9z^2\lambda(c)-\frac{9}{4}\lambda(c)\in k[\lambda(a),\lambda(b),\lambda(c),z]\subseteq A$. Consequently,
\begin{eqnarray*}
(1-2z)\lambda(b)^2 & = & (1-2z)[\frac{5}{4}y^2+2\lambda(b)y+\frac{9}{2}\lambda(a)x+w]\\
& = & \frac{5}{4}y^2(1-2z)+2\lambda(b)(1-2z)y+\frac{9}{2}\lambda(a)^2+(1-2z)w \\ & = & \frac{5}{4}y^2-\frac{5}{2}zy^2+2\lambda(b)(1-2z)y+\frac{9}{2}\lambda(a)^2+(1-2z)w \\
& =& \frac{5}{4}y^2-\frac{5}{2}y\frac{y-\lambda(b)}{3}+2\lambda(b)(1-2z)y+\frac{9}{2}\lambda(a)^2+(1-2z)w \\
& =& \frac{5}{12}y^2+\frac{5}{6}\lambda(b)y+2\lambda(b)(1-2z)y+\frac{9}{2}\lambda(a)^2+(1-2z)w.
\end{eqnarray*}
In particular, we obtain the following equality in $A$:
$$
\frac{5}{12}y^2+\frac{5}{6}\lambda(b)y+2\lambda(b)(1-2z)y+\frac{9}{2}\lambda(a)^2+(1-2z)w-(1-2z)\lambda(b)^2 =0.
$$
As $w\in k[\lambda(a),\lambda(b),\lambda(c),z]\subseteq A$, we deduce that $y$ is integral over $k[\lambda(a),\lambda(b),\lambda(c),z]$, thus also integral over $k[a,b,c]$. Therefore, $\phi$ is finite, as claimed.

We now determine $\phi^{-1}(W)$ by computing the fibers of $\phi$. For $i=1,2$, let $(x_i,y_i,z_i)\in X\subset \mathbb A^3$ such that $\phi(x_1,y_1,z_1)=\phi(x_2,y_2,z_2)$. So
\begin{eqnarray*}
\begin{cases} (1-2z_1)x_1=(1-2z_2)x_2\\ (1-3z_1)y_1=(1-3z_2)y_2\\ (1-z_1)z_1=(1-z_2)z_2=:-a.
\end{cases}
\end{eqnarray*}
In particular, $z_1,z_2$ are roots of $z^2-z-a=0$, and
 \begin{eqnarray*}
\begin{cases} x_1^2+y_1^2+a=0\\ x_2^2+y_2^2+a=0.
\end{cases}
\end{eqnarray*}
We shall distinguish the following four different cases:
\begin{itemize}
\item Case 1: $z_1=z_2=\frac{1}{3}$. We have
\begin{eqnarray*}
\begin{cases} x_1=x_2\\y_1=\pm y_2\\
z_1=z_2=\frac{1}{3}.
\end{cases}
\end{eqnarray*}

\item Case 2: $z_1=z_2=\frac{1}{2}$. We have
\begin{eqnarray*}
\begin{cases} x_1=\pm x_2\\y_1= y_2\\
z_1=z_2=\frac{1}{2}.
\end{cases}
\end{eqnarray*}

\item Case 3: $z_1=z_2\notin \{\frac{1}{2}, \frac{1}{3}\}$. We have
\begin{eqnarray*}
\begin{cases} x_1= x_2\\y_1= y_2\\
z_1=z_2.
\end{cases}
\end{eqnarray*}

\item Case 4: $z_1\neq z_2$. Then $z_1,z_2$ are the two roots of $z^2-z-a=0$. Consequently,
\begin{eqnarray*}
\begin{cases} z_1+z_2=1\\z_1z_2=-a,
\end{cases}
\end{eqnarray*}
and
\begin{eqnarray*}
\begin{cases} (1-2z_1)x_1=[1-2(1-z_1)]x_2=(2z_1-1)x_2,\\ (1-3z_1)y_1=[1-3(1-z_1)]y_2=(3z_1-2)y_2\\ (1-z_1)z_1=(1-z_2)z_2
\end{cases}\\
\Rightarrow \begin{cases} x_1=-x_2{\quad\quad\quad\quad\quad\quad}\\ y_1^2=y_2^2{\quad\quad\quad\quad\quad\quad}\\
(1-3z_1)y_1=(3z_1-2)y_2{\quad\quad\quad\quad\quad\quad}\\
(1-z_1)z_1=(1-z_2)z_2=-a.{\quad\quad\quad\quad\quad\quad}
\end{cases}
\end{eqnarray*}
In particular, $y_1=\pm y_2$. If $y_1=y_2\neq 0$, we have $1-3z_1=3z_1-2,$ thus $z_1=z_2=\frac{1}{2}$, which is impossible. If $y_1=-y_2\neq 0,$ we have $1-3z_1=2-3z_1$, giving also a
contradiction. So, we must have $y_1=y_2=0$ in this case. Hence
\begin{eqnarray*}
\begin{cases} x_1=-x_2\\y_1=y_2=0\\z_1+z_2=1, \text{~and~} z_1\neq z_2.
\end{cases}
\end{eqnarray*}
\end{itemize}

Based on the above discussion, we deduce
$$\phi^{-1}(W)=X\cap V(z-\frac{1}{4})\bigcup \left\{\left(\frac{\sqrt{3}}{4},0,\frac{3}{4}\right), \left(-\frac{\sqrt{3}}{4},0,\frac{3}{4}\right)\right\},
$$
and the last statement then follows easily. This completes the proof of our claim.
\end{proof}

We now give a corrected form of the last part of Proposition \ref{p:1}.

\begin{proposition}\label{p:2} Let $\phi:X\rightarrow Y$ be a finite dominant morphism of affine varieties. Assume $Y$ is \emph{normal}. Let $W$ be an irreducible closed subvariety of $Y$, and $Z$ an irreducible component of $\phi^{-1}(W)$. Then $\phi(Z)=W$, and $\operatorname{dim} (Z)=\operatorname{dim} (W)$.
\end{proposition}

\begin{proof} Let $A=k[X]$ and $B=k[Y]$. As $X$ and $Y$ are affine varieties, one can identify $B$ as a $k$-subalgebra of $A$ using the dominant morphism $\phi$. Write $ W=V(\mathfrak q)$, with $\mathfrak q\subset B$ a prime ideal. So $\phi^{-1}(W)=V(\mathfrak qA)$. Let $\widetilde{\mathfrak q}_1,\widetilde{\mathfrak q}_2,\cdots,\widetilde{\mathfrak q}_r$ be the minimal prime ideals of $V(\mathfrak qA)$. Then $V(\widetilde{\mathfrak q}_1),V(\widetilde{\mathfrak q}_2),\cdot\cdot\cdot,V(\widetilde{\mathfrak q}_r)$ are the irreducible components of $\phi^{-1}(W)$. Since $A$ is integral over $B$,  thanks to \cite[Section 1, Proposition 4.3]{B}, we have $\mathfrak qA\cap B=\mathfrak q$. We now claim that $\mathfrak q=\widetilde{\mathfrak q}_i\cap B$ for all $i$. Clearly $\widetilde{\mathfrak q}_i\cap B\supseteq \mathfrak qA\cap B= \mathfrak q$. Suppose there exists some $i$ such that $\widetilde{\mathfrak q}_i\cap B\supsetneq \mathfrak q$. Because $Y$ is normal, by going-down theorem, there exists a prime ideal ${\widetilde{\mathfrak q}'_i}$ of $A$ such that $\widetilde{\mathfrak q}'_i\subsetneqq \widetilde{\mathfrak q}_i, $ and ${\widetilde{\mathfrak q}'_i}\cap B=\mathfrak q$. In particular, $\widetilde{q}_i\supsetneq \widetilde{q}'_i\supseteq qA$. But this contradicts to the fact that $\widetilde{\mathfrak q}_i\in V(\mathfrak qA)$ is a minimal ideal, proving our claim. Consequently, $\phi$ maps the generic point of $Z$ to that of $W$. So, by the second part of Proposition \ref{p:1}, we find $\phi(Z)=W$ and $\operatorname{dim}(Z)=\operatorname{dim}(W)$.

\end{proof}

\section{The proof}
We now in the position to complete the proof of Theorem 3 at Section 6 of Chapter 1 of Mumford's red book. First of all, we need a lemma, which is a special case of a more general well-known statement.

\begin{lemma}\label{L:1}  Let $A$ be a $k$-algebra of finite type. Assume $A$ is a domain. Then, there exists some $f\in A\setminus \{0\}$, such that the localisation $A_f$ is normal.
\end{lemma}

\begin{proof} Let $K$ denote the fraction field of $A$, and $A'$ the integral closure of $A$ in $K$. Since $A$ is a finitely generated over a field, $A'$ is finite as an $A$-module by \cite[Chapter 12 Theorem 72]{Mat}. In particular, there exist $x_1,\cdots,x_n\in A'$ with $A'=\sum_{i=1}^n A\cdot x_i\subseteq K$.  As $K$ is the fraction field of $A$, one can find $f\in A\setminus \{0\}$ such that $fx_i\in A$ for all $i$. Therefore, $A[1/f]\subset A'[1/f]=\sum_{i=1}^{r}A[1/f]x_i\subset A[1/f]\subseteq K$. Thus, $A_f=A[1/f]=A'[1/f]=A'_f$ is the localisation of the normal ring $A'$, hence is normal as well.
\end{proof}

\begin{theorem}[\cite{M} Chapter I \S~6 Theorem 3]\label{t:1} Let $f:X\rightarrow Y$ be a dominating morphism of varieties and let $r=\operatorname{dim} X-\operatorname{dim} Y$. Then there exists a nonempty open set $U\subset Y$ such that:
\begin{enumerate}
\item $U\subset f(X)$, and
\item for all irreducible closed subsets $W\subset Y$ such that $W\cap U\neq\emptyset,$ and for all irreducible components $Z$ of $f^{-1}(W)$ such that $Z\cap f^{-1}(U)\neq\emptyset$,
$$\operatorname{dim} Z=\operatorname{dim} W+r$$
or
$$\mathrm{codim}(Z, X)=\mathrm{codim}(W,Y).$$
\end{enumerate}
\end{theorem}

\begin{proof} As in the original proof of Mumford, we reduce to the following case: $X,Y$ are affine, and there exists some non-empty open subset $U\subset Y$, such that the induced map $f^{-1}(U)\rightarrow U$ is decomposed as
\[
f^{-1}(U)\stackrel{\pi}{\longrightarrow}U\times \mathbb A^r \longrightarrow U,
\]
where the first map $\pi$ is finite and dominant, while the second is the natural projection. So $\pi$ is surjective by the first part of Proposition \ref{p:1}, and $U\subset f(X)$. Shrinking $U$ if necessary, we further assume $U$ normal according to Lemma \ref{L:1} above. In particular, the affine variety $U\times \mathbb A^r$ is also normal. To finish the proof, let $W\subset Y$ be an irreducible closed subset that meets $U$, and let $Z\subset X$ be an irreducible component of $f^{-1}(W)$ such that $Z\cap f^{-1}(U)\neq \emptyset$. Let $W_0=W\cap U$, and $Z_0=Z\cap f^{-1}(U)$. Then $\operatorname{dim}(W)=\operatorname{dim}(W_0)$ and $\operatorname{dim}(Z)=\operatorname{dim}(Z_0)$. Since $W_0$ is an irreducible closed subset of $U$, one checks that $W_0\times \mathbb A^r$ is an irreducible closed subset of $U\times \mathbb A^r$. Moreover, $Z_0$ is an irreducible component of $\pi^{-1}(W_0\times \mathbb A^r)=f^{-1}(W_0)$. As $W_0\times \mathbb A^r$ is normal, by Proposition \ref{p:2}, $\pi(Z_0)=W_0\times \mathbb A^r$ and $\operatorname{dim}(Z_0)=\operatorname{dim}(W_0\times \mathbb A^r)=\operatorname{dim}(W_0)+r$. Consequently, $\operatorname{dim}(Z)=\operatorname{dim}(W)+r$, as claimed by (2).

\end{proof}

\bigskip


Maximizing Algebraic Connectivity in Interconnected Networks

Abstract: Algebraic connectivity, the second eigenvalue of the Laplacian matrix, is a measure of node and link connectivity on networks. When studying interconnected networks it is useful to consider a multiplex model, where the component networks operate together with inter-layer links among them. In order to have a well-connected multilayer structure, it is necessary to optimally design these inter-layer links considering realistic constraints. In this work, we solve the problem of finding an optimal weight distribution for one-to-one inter-layer links under budget constraint. We show that for the special multiplex configurations with identical layers, the uniform weight distribution is always optimal. On the other hand, when the two layers are arbitrary, increasing the budget reveals the existence of two different regimes. Up to a certain threshold budget, the second eigenvalue of the supra-Laplacian is simple, the optimal weight distribution is uniform, and the Fiedler vector is constant on each layer. Increasing the budget past the threshold, the optimal weight distribution can be non-uniform. The interesting consequence of this result is that there is no need to solve the optimization problem when the available budget is less than the threshold, which can be easily found analytically.

\section{Introduction}

Real-world networks are often connected together and therefore influence each other \cite{Arenas2014multilayer}. Robust design of interdependent networks is critical to allow uninterrupted flow of information, power, and goods in spite of possible errors and attacks \cite{Buldyrev2010}.
The second eigenvalue of the Laplacian matrix, $\lambda_2(L)$, is a good measure of network robustness \cite{Jamakovic2007}.
Fiedler shows that algebraic connectivity increases by adding links \cite{Fiedler1973}.
Moreover, it is harder to bisect a network with higher algebraic connectivity \cite{Fallat2003}.
The second eigenvalue of the Laplacian matrix is also a measure of the speed of mixing for a Markov process on a network \cite{bremaud2013markov}.
Boyd et al. maximize the mixing rate by assigning optimum link weights in the setting of a single layer (\cite{boyd2004fastestChain} and \cite{Boyd2006FastestMP}).

For multiplex networks (see Fig. \ref{fig:multilayer}), a natural question is the following. Given fixed network layers, how should the weights be assigned to inter-layer links in order to maximize algebraic connectivity?
\begin{figure}[b]
\includegraphics[scale=1]{Multilayer.pdf}
\caption{A schematic of a multiplex network $\mathcal{G}$ with two layers $\mathcal{G}_1$, $\mathcal{G}_2$, connecting through an inter-layer one-to-one structure $\mathcal{G}_3$.}
\label{fig:multilayer}
\end{figure}

The behavior of $\lambda_2$, in the case of identical weights, i.e., with a fixed coupling weight $p$ for every inter-layer link, has been studied recently.
For instance, Gomez et al. observe that $\lambda_2(L)$ grows linearly with $p$ up to a critical $p^*$, and then has a non-linear behavior afterwards \cite{gomez2013diffusion}.
Radicchi and Arenas find bounds for this threshold value $p^*$ \cite{Radicchi2013}. Sahneh et al. compute the exact value analytically \cite{sahneh2014exact}.

Martin-Hernandez et al. analyze the algebraic connectivity and Fiedler vector of multiplex structures, with addition of a number of inter-layer links in two configurations;  diagonal (one-to-one) and random \cite{Gregorio2014algebraic}. They show that for the first case, algebraic connectivity saturates after adding a sufficient number of links.
Li et al. adopt a network flow approach to propose a heuristic that improves robustness of large multiplex networks by choosing from a set of inter-layer links with predefined weights \cite{Xin2015connectivity}.

In this letter we remove the constraint of identical interlinking weights and pose the problem of finding the maximum algebraic connectivity for  a one-to-one interconnected structure between different layers in the presence of limited resources.
We show that up to the threshold budget $p^*N$---where $p^*$ is the same threshold studied in \cite{gomez2013diffusion}, \cite{Radicchi2013}, and \cite{sahneh2014exact}---the uniform distribution of identical weights is actually optimal. For larger budgets, the optimal distribution of weights is generally not uniform.

Let $\mathcal{G}=\left( \mathcal{V},\mathcal{E}\right) $
represents a network and by $\mathcal{V}=\left\{ 1,\ldots,N\right\} $
and $\mathcal{E}\subset\mathcal{V}\times\mathcal{V}$, we denote the set of nodes and links.
For a link $e$ between nodes $u$ and $v$, i.e, $e:\{u,v\}\in\mathcal{E}$, we define a nonnegative value $w_{uv}$ as the weight of the link.
The Laplacian matrix of $\mathcal{G}$ can be defined as:
\begin{equation}\label{eq:LapRepre}
L= \sum_{ij\in \mathcal{E}}w_{ij}B_{ij}
\end{equation}
where $B_{ij}:= (e_i-e_j)(e_i-e_j)^T$ is the incidence matrix for link $ij$, and $e_i$ is a vector with $i$th component one and rest of its elements are zero.

For a multiplex network with two layers $\mathcal{G}_1 = \left\{ \mathcal{V}_1,\mathcal{E}_1\right\}$ and $\mathcal{G}_2=\left\{ \mathcal{V}_2,\mathcal{E}_2\right\}$ and $|\mathcal{V}_1| =| \mathcal{V}_2|$, we consider a bipartite graph $\mathcal{G}_3=\left\{ \mathcal{V},\mathcal{E}_3\right\}$ with $\mathcal{E}_3\subseteq \mathcal{V}_1\times \mathcal{V}_2$. The multiplex network $\mathcal{G}$ is composed from $\mathcal{G}_1$, $\mathcal{G}_2$, and $\mathcal{G}_3$ (Fig. \ref{fig:multilayer}).
We want to design optimal weights for $\mathcal{G}_3$ to improve the algebraic connectivity of $\mathcal{G}$ as much as possible with a limited budget, i.e., $\sum w_{ij} = c$.
Using Eq. \eqref{eq:LapRepre}, the Laplacian matrix of $\mathcal{G}$ (supra-Laplacian matrix), is:
\begin{equation}
\label{eq:Laplacian}
L(w)=
\sum_{ij\in \mathcal{E}_2\cup \mathcal{E}_3}B_{ij}+\sum_{ij\in \mathcal{E}_3}w_{ij}B_{ij},
\end{equation}
where we use the notation $L(w)$ to make explicit the dependence of the Laplacian on the interlayer weights $w$.
From Eq. \eqref{eq:Laplacian}, the Laplacian, $L$, of the combined network takes the form

\begin{equation*}
  L(w) =
  \begin{bmatrix}
    L_1 & 0\\
    0 & L_2
  \end{bmatrix}
  +
  \begin{bmatrix}
    W & -W\\
    -W & W
  \end{bmatrix},
\end{equation*}
where $L_1$ and $L_2$ are the Laplacians of the individual layers and $W=\diag(w)$ with $w\ge 0$ the inter-layer link weights satisfying the budget constraint $w^T\mathbf{1} = c$.  We assume the two layers are connected independently, so that $\lambda_3(L)>0$, for all choices of $c$ and $w$.

The second eigenvalue can be characterized as the solution to to the optimization problem
\begin{equation}
  \label{eq:lambda2}
  \lambda_2(L) = \min_{\substack{v\ne 0\\v^T\mathbf{1}=0}} \frac{v^TLv}{\|v\|^2}.
\end{equation}
The optimal weight problem, then, can be phrased as follows.  Given a budget $c\ge 0$, solve the problem
\begin{equation}
  \label{eq:F}
  F(c) := \max_{\substack{w\ge 0\\w^T\mathbf{1} = c}} \lambda_2(L(w)).
\end{equation}

Since $L$ is an affine function of $w$, and $\lambda_2$ is a concave function of $L$, it follows that~\eqref{eq:F} is a convex optimization problem.  In fact, it can be recast as an SDP (similarly to the construction in~\cite{Boyd2006FastestMP}) and, thus, can be solved efficiently even for large networks using standard numerical methods.

Returning to~\eqref{eq:lambda2}, it is convenient to write $v$ in component form $v=(v_1^T,v_2^T)^T$ so that~\eqref{eq:lambda2} implies
\begin{equation}
  \label{eq:main-inequality}
  \begin{split}
  v_1^TL_1v_1 + & v_2^TL_2v_2 + (v_1-v_2)^TW(v_1-v_2) \\
  & - \lambda_2(L)\left(\|v_1\|^2+\|v_2\|^2\right)
  \ge 0 \qquad\forall \  v_1^T\mathbf{1} = -v_2^T\mathbf{1}.
  \end{split}
\end{equation}
Since $v$ must satisfy $v_1^T\mathbf{1} = -v_2^T\mathbf{1}$, we write $v_1$ and $v_2$ of the form
\begin{equation*}
  v_1 = \alpha\mathbf{1} + u_1,\qquad v_2 = -\alpha\mathbf{1} + u_2,\qquad
  u_1^T\mathbf{1} = u_2^T\mathbf{1} = 0,
\end{equation*}
for some constant $\alpha$.
Rewriting the terms in~\eqref{eq:main-inequality}, we observe that
\begin{equation*}
  \begin{split}
    (v_1-v_2)^T&W(v_1-v_2) \\
    &=(2\alpha\mathbf{1} + u_1-u_2)^TW(2\alpha\mathbf{1}
    + u_1-u_2)\\
    &= 4\alpha^2c + 4\alpha w^T(u_1-u_2) \\
    &~~+ (u_1-u_2)^TW(u_1-u_2)
  \end{split}
\end{equation*}
and that
\begin{equation*}
  \|v_i\|^2 = \|\alpha\mathbf{1}\|^2 + \|u_i\|^2 = \alpha^2N + \|u_i\|^2\qquad
  \text{for }i=1,2.
\end{equation*}
Thus, Eq.~\eqref{eq:main-inequality} implies that
\begin{equation}
  \label{eq:reduced-inequality}
  \begin{split}
    u_1^TL_1u_1 &+ u_2^TL_2u_2 + 4\alpha^2 c + 4\alpha w^T(u_1-u_2)\\
     &+    (u_1-u_2)^TW(u_1-u_2) - \\
     & \lambda_2(L)\left(2\alpha^2N + \|u_1\|^2 + \|u_2\|^2\right) \ge 0\\
     &    \qquad\forall \alpha, u_1^T\mathbf{1}=u_2^T\mathbf{1}=0.
    \end{split}
\end{equation}
Setting $u_1=u_2=0$ in~\eqref{eq:reduced-inequality}, then, gives the inequality
\begin{equation*}
  4\alpha^2c - 2\alpha^2N\lambda_2(L) \ge 0\qquad\forall\alpha
\end{equation*}
which can only be true if
\begin{equation*}
  \lambda_2(L) \le \frac{2c}{N}.
\end{equation*}
Thus, we have proven the following theorem.
\begin{theorem}\label{thm:1}
  \label{thm:uniform-bound}
  For the two-layer problem described above, we have the bound
  \begin{equation}
    \label{eq:uniform-bound}
      F(c) \le \frac{2c}{N}.
  \end{equation}
\end{theorem}

Now we turn our attention to the question of attainability of~\eqref{eq:uniform-bound}.  This question is answered by the following theorem.

\begin{theorem}\label{thm:2}
  \label{thm:achieve-bound}
  The inequality in~\eqref{eq:uniform-bound} can only be satisfied as equality if $w=\frac{c}{N}\mathbf{1}$.
\end{theorem}

\begin{proof}
  Suppose the weights $w$ are chosen such that the Laplacian $L$ satisfies $\lambda_2(L)=2\frac{c}{N}$.  Then~\eqref{eq:reduced-inequality} simplifies to
  \begin{equation*}
    \begin{split}
      u_1^TL_1u_1 &+ u_2^TL_2u_2 + 4\alpha w^T(u_1-u_2)\\
      & + (u_1-u_2)^TW(u_1-u_2) \\
      &- \frac{2c}{N}\left(\|u_1\|^2 + \|u_2\|^2\right) \ge 0
    \ \ \forall \  \alpha, u_1^T\mathbf{1}=u_2^T\mathbf{1}=0.
    \end{split}
  \end{equation*}
  This can only be true if the linear coefficient in $\alpha$, $4w^T(u_1-u_2)$, vanishes for every choice of $u_1,u_2$ satisfying $u_1^T\mathbf{1} = u_2^T\mathbf{1} = 0$.  This implies that $w$ is parallel to $\mathbf{1}$ and, since $w^T\mathbf{1} = c$, the theorem follows.
\end{proof}

The previous theorem shows that when the bound~\eqref{eq:uniform-bound} is attained, it can only be attained by the uniform choice of weights $w=\frac{c}{N}\mathbf{1}$.  The next theorem characterizes exactly the budgets for which the bound is attained.

\begin{theorem}\label{thm:3}
  For a given two-layer network, define the constant
  \begin{equation}
    \label{eq:cstar}
    \begin{split}
    c^* := &N \min_{\substack{u_1^T\mathbf{1}=u_2^T\mathbf{1}=0\\u_1 + u_2\ne 0}}
      \frac{u_1^TL_1u_1 + u_2^TL_2u_2}{\|u_1+u_2\|^2}\\
      \end{split}
  \end{equation}
  Then, for all budgets $c\ge 0$, $F(c)=\frac{2c}{N}$ if and only if $c\le c^*$.
\end{theorem}

\begin{proof}
  In light of Theorem~\ref{thm:achieve-bound}, we restrict our attention to the case of uniform weights $w=\frac{c}{N}\mathbf{1}$ and use the notation $L=L(c)=L(w)$.  From Theorem~\ref{thm:achieve-bound}, we see that $F(c)=\frac{2c}{N}$ if and only if $\lambda_2(L(c))=\frac{2c}{N}$.  It is straightforward to check that $\frac{2c}{N}$ is an eigenvalue of $L(c)$ for any $c\ge 0$, with eigenvector $(\mathbf{1}^T,-\mathbf{1}^T)^T$.  Since $L(c)$ is positive semi-definite and $\lambda_1(L(c))=0$ for all $c$, it follows that $\lambda_2(L(c))\le\frac{2c}{N}$ for all $c$.  Thus, we have $F(c)=\frac{2c}{N}$ if and only if $\lambda_2(L(c))\ge \frac{2c}{N}$.  Now, as before, we write a vector $v$ orthogonal to $\mathbf{1}$ as
  \begin{equation*}
    v =
    \begin{bmatrix}
      v_1\\v_2
    \end{bmatrix}
    =
    \begin{bmatrix}
      \alpha\mathbf{1} + u_1\\
      -\alpha\mathbf{1} + u_2
    \end{bmatrix},\quad u_1^T\mathbf{1} = u_2^T\mathbf{1} = 0.
  \end{equation*}
  Recalling~\eqref{eq:lambda2}, we observe that $\lambda_2(L(c))\ge\frac{2c}{N}$ if and only if the following inequality holds for every such choice of $v$ (or, equivalently, every choice of $\alpha$, $u_1$ and $u_2$).
  \begin{equation*}
    \begin{split}
      0 &\le v^TLv - \frac{2c}{N}\|v\|^2 \\
      & = v_1^TL_1v_1 + v_2^TL_2v_2 + \frac{c}{N}\|v_1-v_2\|^2
      - \frac{2c}{N}\left(\|v_1\|^2 + \|v_2\|^2\right) \\
      &= u_1^TL_1u_1 + u_2^TL_2u_2 - \frac{c}{N}\|u_1+u_2\|^2.
    \end{split}
  \end{equation*}
  This inequality holds for all $u_1^T\mathbf{1}=u_2^T\mathbf{1}=0$ if and only if $c\le c^*$ as defined in~\eqref{eq:cstar}, completing the proof.

\end{proof}

The threshold obtained by Eq. \eqref{eq:cstar} is exactly equivalent to the threshold found in \cite{sahneh2014exact} (see Supplemental Material):
\begin{equation}
\label{eq:exact}
c^* = N\lambda_{2}\left(\left(L_1^\dagger + L_2^\dagger\right)^\dagger\right),
\end{equation}
where $L^\dagger$ represents the Moore-Penrose pseudoinverse of $L$.
At the threshold a rough lower-bound for $\lambda_2(L)$ is
\begin{equation}
    \label{ineq:threshold}
 \lambda_2(L)=\frac{2}{N}c^*\geq \min\{\lambda_2(L_1),\lambda_2(L_2)\}.
\end{equation}
One way to see this is to observe that:
\begin{equation*}
\frac{u_1^TL_1u_1 + u_2^TL_2u_2}{\|u_1+u_2\|^2} \geq    \frac{\|u_1\|^2+\|u_2\|^2}{\|u_1+u_2\|^2}\min\{\lambda_2(L_1),\lambda_2(L_2)\}.
\end{equation*}
Inequality~\eqref{ineq:threshold} then follows from the parallelogram law \cite{cantrell2000modern}.
An upper bound for $\lambda_2(L)$ is given in~\cite{gomez2013diffusion}
\begin{equation}
  \label{eq:gomez-bound}
  \lambda_2(L) \le \frac{1}{2}\lambda_2(L_1+L_2).
\end{equation}
In the special case of identical layers ($L_1=L_2$) with corresponding nodes connected, the bound in \eqref{eq:gomez-bound} is attained with uniform weights at the threshold budget $c^*$~\cite{Radicchi2013}. This can be seen by combining (\ref{ineq:threshold}) and (\ref{eq:gomez-bound}).  Therefore, in this case, uniform weights are optimal for budgets $c\le c^*$, and increasing the budget beyond $c^*$  cannot increase the algebraic connectivity, regardless of the weight allocation.

For general structures, it is possible to substantially improve the algebraic connectivity by increasing the budget beyond $c^*$ using an optimal weight distribution.  Figs.~\ref{fig:CompareUniformOptimal}a and~\ref{fig:CompareUniformOptimal}b compare the optimal value of $\lambda_2(L)$ to the one obtained by the uniform distribution as the budget $c$ varies for two different network structures. In both cases, the optimal distribution gives a higher algebraic connectivity after the threshold.

 \begin{figure*}
\subfloat[]{%
  \includegraphics[clip,width=.8\columnwidth]{ER_ER_30.pdf}%
}
\subfloat[]{%
  \includegraphics[clip,width=.8\columnwidth]{BA_BA30_1reg.pdf}%
}

\subfloat[]{%
  \includegraphics[clip,width=.8\columnwidth]{eigsBABA30_uni_trend.pdf}%
}
\subfloat[]{%
  \includegraphics[clip,width=.8\columnwidth]{eigsBABA30trend.pdf}%
}
\caption{(a) and (b) Plots of $\lambda_2(L)$ with different amount of available budget. The solid red line is for the optimal weights and the dashed black line is for uniform weights. The threshold budget and upper-bound is shown with yellow and blue dashed lines respectively.
The upper-bound is from Eq.~\eqref{eq:gomez-bound} and the threshold is from Eq.~\eqref{eq:exact}.
(a) A structure of two Erd\"{o}s-Renyi networks each with $30$ nodes and (b) a structure of two scale-free networks each with $30$ nodes. (c) First seven eigenvalues of Laplacian matrix of $\mathcal{G}$ considering a uniform distribution of weights for the multiplex in (b).
(d) First seven eigenvalues of Laplacian matrix of $\mathcal{G}$ considering an optimal distribution of weights for the multiplex in (b).}
\label{fig:CompareUniformOptimal}
\end{figure*}

In Fig. \ref{fig:CompareUniformOptimal}c, we plot the first seven eigenvalues of $L$ (omitting the zero eigenvalue) for a multiplex with identical weights on the inter-layer links. Because $\frac{2c}{N}$ is always an eigenvalue and $\lambda_3(L)>\frac{2c}{N}$ for $c\rightarrow 0$, increasing $c$, $\lambda_2(L)$ and $\lambda_3(L)$ cross.
For the same multiplex with optimal distribution of inter-layer weights, we plot the eigenvalues in Fig. \ref{fig:CompareUniformOptimal}d.
When increasing the budget beyond the threshold, the second and third eigenvalues
 coalesce and are strictly less than $\frac{2c}{N}$.
Since \eqref{eq:F} is a convex optimization problem, we know the optimal $w_i$'s vary continously with $c$, and smooothly
away from the finite set of budgets where eigenvalue multiplicities change.

When $c\leq c^*$, the Fiedler vector is $v = \frac{1}{\sqrt{2N}}[\mathbf{1}, -\mathbf{1}]$ and the Fiedler cut distinguishes the layers. For $c>c^*$, due to the multiplicity of $\lambda_2(L)$, there is a corresponding Fiedler eigenspace.  Therefore, the two layers are not as easily recognizable as before.

In Fig. \ref{fig:CompareUniformOptimal}, we also observe that  for $c>c^*$,  $\lambda_2$ increases more slowly. Moreover, as Fig.~\ref{fig:WD} shows, we can have very non-uniform weights in this case.
\begin{figure*}
\subfloat[]{%
  \includegraphics[clip,width=.53\columnwidth]{DisBA100nodes_45.pdf}%
}
\subfloat[]{%
  \includegraphics[clip,width=.53\columnwidth]{DisBA100nodes_60.pdf}%
}
\subfloat[]{%
  \includegraphics[clip,width=.53\columnwidth]{DisBA100nodes_150.pdf}%
}
\subfloat[]{%
  \includegraphics[clip,width=.53\columnwidth]{DisBA100nodes_300.pdf}%
}
\caption{Optimal weight distribution for different amount of budgets. The stucture of a multiplex with two scale free network layers, with $N = 100$ nodes and $\Vert E_1\Vert =  196$ and $\Vert E_2\Vert = 291$. In (a) budget is lower than threshold and uniform distribution is optimal. In this example, the threshold budget $c^*$ is $51.4$.}\label{fig:WD}
\end{figure*}
These optimal weights  represent  the importance  of each link in improving the algebraic connectivity of the whole network.

In summary, we have shown that before a threshold budget that can be analytically computed, the largest possible algebraic connectivity is a linear function of the budget and can  only be attained by the uniform weight distribution. Since the threshold budget is always strictly positive, for low enough budgets it is not necessary to solve \eqref{eq:F}. On the other hand, for larger budgets,  \eqref{eq:F} can be solved with efficient semi-definite programming solvers to find the optimal weights. In particular, heuristic methods based solely on the information of each layer are too
blunt to notice this threshold phenomenon.
\bibliographystyle{apsrev4-1}


A structured proof of Kolmogorov's Superposition Theorem

Abstract: We present a well-structured detailed exposition of a well-known proof of the following celebrated result solving Hilbert's 13th problem on superpositions. For functions of 2 variables the statement is as follows.Kolmogorov Theorem. There are continuous functions $\varphi_1,\ldots,\varphi_5 : [\,0, 1\,]\to [\,0,1\,]$ such that for any continuous function $f: [\,0,1\,]^2\to\mathbb R$ there is a continuous function $h: [\,0,3\,]\to\mathbb R$ such that for any $x,y\in [\,0, 1\,]$ we have $$f(x,y)=\sum\limits_{k=1}^5 h\left(\varphi_k(x)+\sqrt{2}\,\varphi_k(y)\right).$$ The proof is accessible to non-specialists, in particular, to students familiar with only basic properties of continuous functions.

\section{Introduction}

We present a well-structured detailed exposition of a well-known proof of the following celebrated result solving Hilbert's 13th problem. See Remark \ref{r:hist} for motivation and comparison to earlier expositions.

We regard an ordered set of functions $\varphi_1,\varphi_2,\ldots,\varphi_{2n+1} \colon [\,0,1\,]\to [\,0,1\,]$ as a map (=vector-function) $\varphi\colon [\,0,1\,]\to[\,0,1\,]^{2n+1}$.
For numbers $\alpha_1,\ldots,\alpha_n$, a continuous map $\varphi\colon [\,0,1\,]\to [\,0,1\,]^{2n+1}$ and a continuous function $h\colon{\mathbb R}} \def\Z{{\mathbb Z}} \def\C{\Bbb C} \def\Q{\Bbb Q\to{\mathbb R}} \def\Z{{\mathbb Z}} \def\C{\Bbb C} \def\Q{\Bbb Q$ denote
\[
(S_\varphi h)(x_1, \ldots, x_n) := \sum_{k=1}^{2n+1} h\left(\sum_{i=1}^n \alpha_i \varphi_k(x_i)\right).
\]

\begin{theorem}[Kolmogorov]\label{thKolm}
For any integer $n>1$ there are numbers $\alpha_1,\ldots,\alpha_n$
and continuous functions $\varphi_1, \ldots, \varphi_{2n+1}\colon[\,0,1\,]\to[\,0,1\,]$ such that for any continuous function $f\colon[\,0,1\,]^n\to{\mathbb R}} \def\Z{{\mathbb Z}} \def\C{\Bbb C} \def\Q{\Bbb Q$
there is a continuous function $h\colon{\mathbb R}} \def\Z{{\mathbb Z}} \def\C{\Bbb C} \def\Q{\Bbb Q\to{\mathbb R}} \def\Z{{\mathbb Z}} \def\C{\Bbb C} \def\Q{\Bbb Q$
such that $f=S_\varphi h$.
\end{theorem}

\begin{remark}\label{r:hist}\leavevmode
(a) We can take $\alpha_1,\ldots,\alpha_n$ to be square roots of pairwise different prime numbers. Or, for $n=2$, $\alpha_1=1$ and $\alpha_2=\sqrt2$.
Then for the function $f(x, y) = x + \sqrt{2}\,y + 2$ we may take $h(x) = x$, $\varphi_1(x) = x$, $\varphi_2(x) = \frac{2}{1 + \sqrt{2}}$,  and $\varphi_3 = \varphi_4 =\varphi_5 =0$.
    However, we do not know explicit $\varphi,h$ even for as simple functions  as addition and multiplication. And of course we do not know explicit universal $\varphi$.

(b) For classical expositions of a proof, for statement and discussion of Hilbert's 13th problem see e.g. \cite{Ar, He}, \cite[Chapter 17]{LGM}, \cite[\S1-\S4]{St}.

    In order to make our exposition well-structured we e.g. explicitly introduce  the notion of $\lambda$-prekolmogorov maps and explicitly state the Approximative Theorem \ref{thKolmApprox}.
    In order to make our exposition more detailed we e.g. explicitly state  Propositions \ref{propQuarters} and \ref{propRationalSeparatingMap}, and present their simple proof.

(c) This topic is actively studied not only in analysis but also in topology and computer science, see e.g. surveys \cite{St, Vi, Sk, Br} and the references therein.

\end{remark}

We present a proof for $n=2$, taking  $\alpha_1=1$ and $\alpha_2=\sqrt2$.
Proof for the general case is analogous.
For a proof we need some notation and definitions.

We tacitly consider only continuous functions.
Denote $I:=[\,0, 1\,]$.
For a compact subset $M \subset \mathbb R^s$ and a function $f\colon M\to\mathbb R$ denote $\norm{f}:=\sup_{z\in M}|f(z)|$.
A \textbf{$\lambda$-prekolmogorov map} for a non-zero function $f\colon I^2\to\mathbb R$ is a map $\varphi\colon I\to I^5$ for which there exists a function $h\colon [\,0,3\,]\to\mathbb R$ such that
\[
    \norm{f - S_{\varphi}h} < \lambda \norm{f} \quad\text{and}\quad \norm{h} \leqslant \norm{f}.
\]
For a function $f\equiv0$ any map $\varphi\colon I\to I^5$ is $\lambda$-prekolmogorov.

\begin{theorem}[Approximative Kolmogorov theorem]\label{thKolmApprox}

There is a map $\varphi\colon I\to I^5$ which is $(7/8)$-prekolmogorov for any function $f\colon I^2\to\mathbb R$.
\end{theorem}

\begin{proof}[Deduction of Theorem \ref{thKolm} for $n=2$, $\alpha_1=1$ and $\alpha_2=\sqrt2$ from Theorem \ref{thKolmApprox}]
Define functions $h_m\colon[\,0,3\,]\to\mathbb R$ inductively.
Let $h_0\equiv0$.
Let $h_m$ be a function obtained by applying Theorem \ref{thKolmApprox} to $f-\sum\limits_{k=0}^{m-1} S_\varphi h_k$.
Then denoting $\lambda = 7/8$ we have
\begin{equation}
    \norm{f - \sum\limits_{k=0}^m S_\varphi h_k}
    < \lambda\norm{f - \sum\limits_{k=0}^{m-1} S_\varphi h_k} < \ldots < \lambda^m\norm{f}.  \tag{*}\label{leqNorm}
\end{equation}
Since $\norm{h_m} \leqslant \norm{f - \sum\limits_{k=0}^{m-1} S_\varphi h_k} < \lambda^{m-1}\norm{f}$, the functional series $\sum\limits_{m=0}^\infty h_m$ uniformly converges to a function $h\colon[\,0,3\,]\to\mathbb R$.
Then $\sum\limits_{m=0}^\infty S_\varphi h_m = S_\varphi\sum\limits_{m=0}^\infty h_m=S_\varphi h$.
Since $\lambda^m\xrightarrow{m \to \infty} 0$, passing to the limit in \eqref{leqNorm} we obtain $\norm{f - S_{\varphi}h}= 0$, i.e. $S_{\varphi}h=f$.
\end{proof}

For a proof of Theorem \ref{thKolmApprox} we need the following Lemma on stability and approximation.

\begin{lemma}\label{lemStability}
    For any function $f\colon I^2\to{\mathbb R}} \def\Z{{\mathbb Z}} \def\C{\Bbb C} \def\Q{\Bbb Q$ the set $PK(f)\subset C(I, I^5)=C(I)^5$  of all ($6/7$)-prekolmogorov maps for $f$  is open and dense in $C(I)^5$.
\end{lemma}

For a map $\varphi \in C(I)^5$ denote $\norm{\varphi} = \max\limits_{k=1,\ldots,5} \norm{\varphi_k}$.

\begin{proof}[Proof of Theorem \ref{thKolmApprox} modulo Lemma \ref{lemStability}]
It is well-known that the metric space $C(I)^5$ is complete.
Denote by $Q$ the set of all polynomial functions $I^2\to\mathbb R$ with rational coefficients.
By Lemma \ref{lemStability} for any $g \in Q$ the set $PK(g)$ is open and dense in $C(I)^5$.
Then by the Baire category theorem the countable intersection $\bigcap_{g\in Q} PK(g)$ is not empty.
(If you are not familiar with the theorem, then just take this phrase for granted.)
Thus it remains to prove that any map from this intersection is a ($7/8$)-prekolmogorov map for any function $f\colon I^2 \to {\mathbb R}} \def\Z{{\mathbb Z}} \def\C{\Bbb C} \def\Q{\Bbb Q$.

Take $\varphi \in \bigcap_{g\in Q} PK(g)$.
The Weierstrass theorem asserts that $Q$ is dense. Hence there is $g \in Q$ such that $\norm{f - g} < \frac{1}{56} \norm{f}$ and $\norm{g} \leqslant \norm{f}$. By definition of $PK(g)$, there is a function $h\colon[\,0,3\,]\to{\mathbb R}} \def\Z{{\mathbb Z}} \def\C{\Bbb C} \def\Q{\Bbb Q$ such that $\norm{h} \leqslant \norm{g} \leqslant \norm{f}$ and $\norm{g - S_\varphi h} < \frac{6}{7} \norm{g} \leqslant \frac{6}{7}\norm{f}$. Then
\[
    \norm{f - S_\varphi h} \leqslant \norm{f - g} + \norm{g - S_\varphi h} < \frac{1}{56}\norm{f} + \frac{6}{7}\norm{f} = \frac{7}{8}\norm{f}.
\]
Hence $\varphi$ is a ($7/8$)-prekolmogorov map for $f$.
\end{proof}

\section{Proof of the stability and approximation (Lemma \ref{lemStability})}\label{s:prostability}

\begin{proof}[Proof that $PK(f)$ is open]
Suppose $\psi\in PK(f)$ is a map. Then there is a function $h\colon[\,0,3\,]\to{\mathbb R}} \def\Z{{\mathbb Z}} \def\C{\Bbb C} \def\Q{\Bbb Q$ such that $\norm{f - S_\psi h} < \frac{6}{7}\norm{f}$.
Denote
\[
    \varepsilon:= \frac{1}{5}\left(\frac{6}{7}\norm{f} - \norm{f - S_\psi h}\right) > 0.
\]
Since $h$ is uniformly continuous, there is $\delta$ such that $|h(x) - h(y)| < \varepsilon$ for $|x-y| < \delta$.
It suffices to show that any map $\varphi$ which is $(\delta/3)$-close to $\psi$ is contained in $PK(f)$.

For any $x,y\in I$ and $k = 1,\ldots,5$
\begin{align*}
    &\left|\left(\psi_k(x) + \sqrt{2}\,\psi_k(y)\right) -
    \left(\varphi_k(x) + \sqrt{2}\,\varphi_k(y)\right)\right| <
    \left(1 + \sqrt{2}\right)\delta/3 < \delta.\\
\intertext{Consequently,}
    &\left|h\left(\psi_k(x) + \sqrt{2}\,\psi_k(y)\right) -
    h\left(\varphi_k(x) + \sqrt{2}\,\varphi_k(y)\right) \right| < \varepsilon.
\end{align*}
Therefore $\norm{S_\psi h - S_\varphi h} < 5 \varepsilon$. Finally,
\[
    \norm{f - S_\varphi h} \leqslant
    \norm{f - S_\psi h} + \norm{S_\psi h - S_\varphi h} <
    \frac{6}{7}\norm{f} - 5\varepsilon + 5\varepsilon =
    \frac{6}{7}\norm{f}.
\]
Hence $\varphi \in PK(f)$.
\end{proof}

\begin{figure}[ht]
    \centering
    \includegraphics[height=100mm]{squares.png}
    \caption{The green family of squares $I_{i,2}(26) \times I_{j,2}(26)$ parametrized by $i,j \in Z_2(26)$. The red squares are $I_{2,k} \times I_{1,k}$ for $k=1,3,4,5$.}
    \label{fig:squares}
\end{figure}

Proof of the density in Lemma \ref{lemStability} is based on the following construction (Figure \ref{fig:squares}).

Denote $[\,a,b\,] + c := [\,a+c,b+c\,]$ and $[\,a,b\,] \cdot d := [\,ad,bd\,]$.

Take any $k = 1,\ldots,5$. Denote
$$
    Z_k = Z_k(N) := \left[\,-1,\dfrac{N-k}5\,\right] \bigcap \Z.
$$
For every $j \in Z_k(N)$  denote
$$
I_{jk} = I_{jk}(N) := \dfrac{4I+5j + k}{N} \bigcap I\quad\text{and}\quad l_{jk} = l_{jk}(N) := \max\left\{\dfrac{5j + k}{N}, 0\right\}.
$$
Then $l_{jk}$ is the left end of the interval $I_{jk}$.

\begin{proposition}\label{propQuarters}
    Let $N$ be a positive integer. For any $(x, y) \in I^2$
    and for

    at least three $k=1,\ldots,5$

    there are $i, j \in Z_k$
    such that
    \[

        (x, y) \in I_{ik}(N) \times I_{jk}(N).
    \]
\end{proposition}
\begin{proof}[Proof.]
    We have
    \[
        (x, y) \in I_{ik}(N) \times I_{jk}(N) \quad\Leftrightarrow\quad

        (Nx, Ny) \in I_{ik}(1) \times I_{jk}(1).
    \]

    Denote $m = \left[\frac{Nx}{5}\right]$ and $r = [Nx] \bmod 5$.
    Then $Nx \in [\,5m + r, 5m + r + 1)$ and for any $s = 0,1,2,3$
    we have $$
    Nx \in [\,5m + (r - s), 5m + (r - s + 4)\,] = 4I + 5m + (r-s).$$
    Therefore for $k \equiv r - s \pmod 5$ there is $i \in Z_k$ such that
    $Nx \in I_{ik}(1)$.

    Then for

    at least four $k = 1,\ldots,5$
    there is the appropriate $i$. Similarily to this,

    for at least four $k=1,\ldots,5$
    there is $j\in Z_k$ such that

    $Ny \in I_{jk}(1)$.

    Then for at least three $k=1,\ldots,5$ there are
    $i,j \in Z_k$ such that $(Nx, Ny) \in I_{ik}(1) \times I_{jk}(1)$.
\end{proof}

A function $\varphi\colon I\to I$ \emph{rationally separates} a family of pairwise disjoint intervals on the line if $\varphi$ has constant rational pairwise different values on the intervals.

\begin{proposition}\label{propRationalSeparatingMap}
\renewcommand{\theenumi}{\alph{enumi}}
Let $\varepsilon$ be a positive number.

\begin{enumerate}[(a)]

    \item\label{pointRationalSeparating}
    Let $\psi:I\to I$ be a function and $k\in\{1,\ldots,5\}$.

    Then there is an integer $N_0 = N_0(\psi, k) > 0$ such that for any integer $N>N_0$ and for any finite set $G$ of rational numbers
    there is a function $\varphi:I\to I$ such that

    \begin{itemize}
        \item $\norm{\varphi - \psi} < \varepsilon$;

        \item $\varphi$ rationally separates the family of intervals $I_{jk} = I_{jk}(N)$ parametrized by $j\in Z_k(N)$;

        \item $\varphi(I_{jk}) \not\in G$ for any pair $(j,k)$.

    \end{itemize}

    \item\label{pointRationalSeparatingMap}
    Let $\psi\colon I\to I^5$ be a map.

    Then there is an integer $N_0 > 0$ such that for any integer $N>N_0$ there is a map $\varphi\colon I\to I^5$ such that
    \begin{itemize}
        \item $\norm{\varphi - \psi} < \varepsilon$;

        \item for every $k=1,\ldots,5$ the function $\varphi_k$ rationally separates the family of intervals $I_{jk} = I_{jk}(N)$ parametrized by $j\in Z_k(N)$;

        \item numbers $\varphi_k(I_{jk})$ are pairwise different for different pairs $(j,k)$.
    \end{itemize}
\end{enumerate}

\end{proposition}

\begin{proof}[Proof of \eqref{pointRationalSeparating}.]
    Since $\psi$ is uniformly continuous,
    there is $\delta > 0$ such that
    $\abs{\psi(x) - \psi(y)} < \dfrac{\varepsilon}{2}$
    for $\abs{x - y} < \delta$.
    Define $N_0: = \left\lceil\dfrac{5}{\delta}\right\rceil$.
    Take any $N>N_0$.
    Take a piecewise linear function $\varphi\colon I\to I$ such that
    \begin{itemize}
        \item for each interval $I_{jk}$ the value $\varphi(I_{jk})$

        is rational and is $\frac{\varepsilon}{2}$-close to
        $\psi\!\left(l_{jk}\right)$;

        \item all values $\varphi(I_{jk})$ are different and are not in $G$;

        \item $\varphi$ is linear on the gaps between the intervals.
    \end{itemize}

Since $0 \in I_{-1,k}$ and $1 \in I_{\max Z_k, k}$ for any $k=1,\ldots,5$, the definition of $\varphi(0)$ and $\varphi(1)$ is meaningful.

    Now $\varphi$ rationally separates the family of intervals $I_{jk}$ and its values on the intervals are not from $G$. It remains to show that
    \begin{equation}
        \abs{\varphi(x) - \psi(x)} < \varepsilon\quad \text{ for any } x \in I. \tag{**}\label{ineqEpsilonClose}
    \end{equation}
    For points $x \in I_{jk}$, since $\varphi(x) = \varphi(l_{jk})$ and $|x - l_{jk}| \leqslant \frac{4}{N} < \delta$, we have
    \[
        \bigl|\varphi(x) - \psi(x)\bigr| \leqslant
        \bigl|\varphi(l_{jk}) - \psi(l_{jk})\bigr| +
        \bigr|\psi(l_{jk}) - \psi(x)\bigr| <
        \frac{\varepsilon}{2} + \frac{\varepsilon}{2} = \varepsilon.
    \]
    For points $x$ which do not belong to any interval $I_{jk}$,
    choose $j \in Z_k$ such that $x$ is between two intervals $I_{jk}$ and $I_{j+1,k}$, i.e. $ l_{jk}+\frac4N < x < l_{j+1,k}$.

    Ends of intervals are related by the obvious equation $l_{jk} + \frac{5}{N} = l_{j+1,k}$.
    Then there is $\alpha \in (0,1)$ such that $x=l_{j+1,k}-\frac{\alpha}N$.

    Now \eqref{ineqEpsilonClose} follows because
    \[
    \begin{split}
        \bigl|\varphi(x) - \psi(x)\bigr| &\leqslant
        \alpha\abs{\varphi\left(l_{j+1,k}-\tfrac1N\right) - \psi(x)} +
        (1 - \alpha)\bigl|\varphi(l_{j+1,k}) - \psi(x)\bigr|
        < \\ &<
        \alpha \varepsilon + (1 - \alpha) \varepsilon
        = \varepsilon.
    \end{split}
    \]
    Here the first inequality follows because $\varphi$ is linear on the gap $l_{j+1,k}+[\,-\frac1N,0\,]$. The second inequality if proved as follows.
    Since $\varphi(l_{j+1,k}-\frac1N) = \varphi(l_{jk})$ and $|x - l_{jk}| \leqslant \frac{5}{N} < \delta$, we have
    \[
        \abs{\varphi(l_{j+1,k}-\tfrac1N) - \psi(x)} \leqslant

        |\varphi(l_{jk}) - \psi(l_{jk})| + |\psi(l_{jk}) - \psi(x)| <
        \frac{\varepsilon}{2} + \frac{\varepsilon}{2} < \varepsilon.
    \]
    Since

    $|x - l_{j+1,k}| \leqslant \frac{1}{N} < \delta$, we have
    \[
        \abs{\varphi(l_{j+1,k}) - \psi(x)} \leqslant
        \abs{\varphi(l_{j+1,k}) - \psi(l_{j+1,k})} + \abs{\psi(l_{j+1,k}) - \psi(x)} <
        \frac{\varepsilon}{2} + \frac{\varepsilon}{2} < \varepsilon.
    \]

\end{proof}

\begin{proof}[Proof of \eqref{pointRationalSeparatingMap}.]
    Choose $N_0 = \max\limits_{k=1,\ldots,5} N_0(\psi_k, k)$, where $N_0(\psi_k, k)$ come from applications of part (a).

    Take any $N > N_0$.
    Apply part (a) to $\psi_1$, $k=1$ and $G = \varnothing$. We obtain a function $\varphi_1$.

    Then apply part (a) to $\psi_2$, $k=2$ and $G = \bigl\{\varphi_1(I_{j,1})\bigr\}_{j \in Z_1}$.
    We obtain a function $\varphi_2$.

    Analogously, on the $k$th step we apply part (a) to the function $\psi_k$ and the set $G = \bigcup\limits_{s=1}^{k-1} \bigl\{\varphi_s(I_{js})\bigr\}_{j \in Z_s}$.
    We obtain a function $\varphi_k$.

        After five steps we obtain the required map $\varphi:=(\varphi_1,\ldots,\varphi_5)$.
\end{proof}

\begin{proof}[Proof that $PK(f)$ is dense in Lemma \ref{lemStability}]

    If $f \equiv 0$, the statement is trivial.
    So assume that $\norm{f}>0$.

    Since $f$ is uniformly continuous, there is an integer $N_1$ such that $|f(x, y) - f(x', y')| < \frac{1}{6}\norm{f}$ for $|x - x'| < \frac{4}{N_1}$ and $|y - y'| < \frac{4}{N_1}$.
    Apply  Proposition \ref{propRationalSeparatingMap}.\ref{pointRationalSeparatingMap}
    to the map $\psi$ and $\varepsilon$.
    We obtain a number $N_0$.
    Let $N := \max(N_1, N_0) + 1$.

    For each $k$ define a function
    \[
        \tilde \varphi_k\colon I^2\to [\,0,3\,]\quad \text{by}\quad \tilde\varphi_k(x, y) = \varphi_k(x)+\sqrt{2}\,\varphi_k(y).
    \]

In the next paragraph we prove that $\tilde\varphi_k$ has different constant values on different squares that are cartesian products of the intervals $I_{jk}$.

    Indeed, suppose for some $x_1, x_2, y_1, y_2$ belonging to some of the intervals we have
    $\varphi_k(x_1) + \sqrt{2}\,\varphi_k(y_1) = \varphi_k(x_2) + \sqrt{2}\,\varphi_k(y_2)$. By definition of rational separability, the numbers $\varphi_k(x_1), \varphi_k(x_2), \varphi_k(y_1)$ and $\varphi_k(y_2)$ are rational.
    Then $\varphi_k(x_1) = \varphi_k(x_2)$ and $\varphi_k(y_1) = \varphi_k(y_2)$.
    Consequently, the pairs $x_1, x_2$ and $y_1, y_2$ are from the same intervals. Then the points $(x_1, y_1)$ and $(x_2, y_2)$ are from the same square.

In the next paragraph we prove that the numbers $\tilde\varphi_k(I_{ik} \times I_{jk})$ are  pairwise different for different triples $(i,j,k)$.

    Indeed, suppose $\tilde\varphi_k(I_{ik} \times I_{jk}) = \tilde\varphi_n(I_{pn} \times I_{qn})$. Then $\varphi_k(I_{ik}) = \varphi_n(I_{pn})$. So $k=n$ and $i=p$. Analogously $j=q$.

Take a piecewise linear function $h\colon[\,0,3\,]\to{\mathbb R}} \def\Z{{\mathbb Z}} \def\C{\Bbb C} \def\Q{\Bbb Q$ such that
\begin{itemize}
    \item $h\bigl(\tilde\varphi_k(I_{ik} \times I_{jk})\bigr) = \frac{1}{3}f(l_{ik}, l_{jk})$ for any $k=1,\ldots,5$ and $i,j \in Z_k$;

    \item $\norm{h} \leqslant \frac{1}{3}\norm{f}$.
\end{itemize}
Now the lemma follows because for any $z \in I^2$
\begin{equation}
    |S_\varphi h(z) - f(z)| \leqslant \frac{1}{6}\norm{f} + \frac{2}{3}\norm{f} < \frac{6}{7}\norm{f}.
    \tag{***}\label{leqPK}
\end{equation}
Here the second inequality is obvious.
The first inequality is proved as follows.
Since $N > N_1$, we have $|f(z) - f(z')| < \frac{1}{6}\norm{f}$ for any $k=1,\ldots,5$,
any $i,j \in Z_k$
and any $z,z' \in I_{ik} \times I_{jk}$.
By Proposition \ref{propQuarters}, there are at least three
triples $(i_s,j_s,k_s) \in Z_{k_s} \times Z_{k_s} \times \{1,\ldots,5\}$, $s = 1,2,3$, such that $I_{i_sk_s}\times I_{j_sk_s} \ni z$.
Therefore $h\bigl(\tilde\varphi_{k_s}(z)\bigr) = \dfrac{1}{3}f(l_{i_sk_s}, l_{j_sk_s})$. These values differ from $\dfrac{1}{3}f(z)$ by at most  $\dfrac{1}{18}\norm{f}$.
Hence
\[
 \abs{\sum_{s=1}^3 h\bigl(\tilde\varphi_{k_s}(z)\bigr) - f\bigl(z\bigr)} \leqslant \dfrac{1}{6}\norm{f}.
\]
Another two values $h\bigl(\tilde\varphi_k(z)\bigr)$ do not exceed $\dfrac{1}{3}\norm{f}$ by the absolute value.
\end{proof}

Functional Analysis (math.FA); Machine Learning (cs.LG)