SECO: Secure Inference With Model Splitting Across Multi-Server Hierarchy
Shuangyi Chen Ashish Khisti

























































SECO: Secure Inference With Model Splitting 
 Across Multi-Server Hierarchy
    
  Shuangyi Chen1,
                    and Ashish Khisti 1
  1
  Department of Electrical and Computer Engineering,
                    University of Toronto, 

                    {shuangyi.chen@mail.utoronto.ca, akhisti@ece.utoronto.ca}

    April 24, 2024
===================================================================================================================================================================================================================================================




In the context of prediction-as-a-service, concerns about the privacy of the data and the model have been brought up and tackled via secure inference protocols. These protocols are built up by using single or multiple cryptographic tools designed under a variety of different security assumptions.

In this paper, we introduce SECO, a secure inference protocol that enables a user holding an input data vector and multiple server nodes deployed with a split neural network model to collaboratively compute the prediction, without compromising either party's data privacy. We extend prior work on secure inference that requires the entire neural network model to be located on a single server node, to a multi-server hierarchy, where the user communicates to a  gateway server node, which in turn communicates to remote server nodes. The inference task is split across the server nodes and must be performed over an encrypted copy of the data vector.

We adopt multiparty homomorphic encryption and multiparty garbled circuit schemes, making the system secure against dishonest majority of semi-honest servers as well as protecting the partial model structure from the user. We evaluate SECO on multiple models, achieving the reduction of computation and communication cost for the user, making the protocol applicable to user's devices with limited resources.


Distributed machine learning, Security and privacy




















§ INTRODUCTION

Prediction-as-a-service using neural network models enables a company to deploy a neural network on the cloud and allows users to upload their input and obtain the corresponding prediction results. However, this may lead to privacy concerns and legal issues. There are regulatory constraints such as Health Insurance Portability and Accountability Act (HIPAA) and EU General Data Protection Regulation (GDPR), restricting the way data could be used or shared.  A secure inference protocol enables a user, holding its input, and servers deployed with the neural network to collaboratively evaluate the prediction result, preserving the privacy of both the input and the model parameters. Many cryptographic tools, such as Homomorphic Encryption (HE) <cit.> and secure multiparty computation (MPC), have been applied in this area. Homomorphic encryption allows the data holder to encrypt its data with the public key, any other party who obtains the encrypted data can compute on it without decryption, and only the party holding the secret key can decrypt the computed result. MPC is usually a mixed approach that combines secret sharing and Garbled Circuit <cit.>, etc. Both HE and MPC can protect data privacy. Pure HE-based secure inference protocols <cit.><cit.><cit.><cit.>  provides the data and model confidentiality as well as protects the structure of the model. However, it involves heavy computation and does not immediately generalize to non-linear function computations used during predictions.  One popular approach in HE-based protocols is to replace the activation functions in neural networks with approximated polynomials at the cost of reduced accuracy. While MPC protocols are not as computationally heavy as HE-based protocols, they require multiple rounds of communication and can also leak some information about the model to the users. Furthermore, there are hybrid approaches that combine HE and MPC. GAZELLE <cit.> relies on a hybrid approach by employing HE and two-party computation, performing linear function evaluation with HE and non-linear function evaluation with Garbled Circuit. DELPHI <cit.> builds upon techniques from GAZELLE and modifies it to a two-phase protocol. It reduces the use of heavy cryptographic tools in the online phase, thus speeding up online inference. However, these methods come with certain limitations. Firstly, they reveal the neural network's architecture to the user, which is problematic when the model's structure is intended to remain confidential, a common scenario with today's large language models. Secondly, they require user equipment to possess substantial computational power and communication capabilities. Additionally, they operate under the assumption that the model is hosted on a single server, which is impractical for very large models. For instance, processing a model with the size of GPT-4 necessitates a cluster of multiple GPUs. To process such a large model effectively, distributing the model parameters across several servers, each with limited GPU resources, can enhance processing power.




With these considerations, we propose SECO, a hybrid HE-MPC secure inference protocol that involves a hierarchy of servers as shown in Figure <ref>. The user communicates with a gateway server that stores a part (the parameters associated with the first few layers) of the neural network model. The remainder of the neural network model is stored on remote servers, with each server holding a share of the weights of the linear layers. Our setup extends the secure inference protocol proposed in <cit.> to such a multi-server scenario, which has several advantages. First, the user only needs to participate during the evaluation involving part of the network model stored on the gateway server. 
It does not need to participate during evaluations involving the network model stored on the remote servers. This reduces the communication and computational requirements for the user device while increasing the load on the servers, which enables the use of SECO on lightweight user devices.

Furthermore although as in prior works <cit.>, the structures of the layers in the neural network model stored on the gateway server must be revealed to the user, the structures of the layers on the remote server are not revealed to the user. SECO offers the benefit of hiding most of the neural network's architecture, crucial for protecting the proprietary structures of large models like GPT-4, where both the model's parameters and the model's design are valuable assets. We demonstrate that a straightforward application of prior works to the hierarchical server setting can lead to information leakage. Our proposed setting involves storing a secret share of model weights at the remote server. However, for secure inference of the partial model on the remote servers, a straightforward extension of DELPHI's 2PC protocol to three-party setting with distributed weights is not privacy-preserving due to the limitation of the plain HE scheme. To privately perform the execution of secure inference on the remote servers, we design a sub-protocol relying on multiparty HE, secret sharing, and multiparty Garbled Circuit schemes, ensuring the privacy of each individual data while facilitating efficient communication. We achieve this by 1) designing new secret sharing scheme and integrating it with multiparty HE for secure computation of linear functions 2) departing from the conventional approach for the use of Garbled Circuits (GC) in DELPHI by redesigning the assignment of roles for secure computation of non-linear functions. We formally demonstrate that our proposed method prevents information leakage in the presence of dishonest majority of honest-but-curious servers. 
We have implemented SECO and reported the performance. The results indicate that SECO significantly enhances efficiency by reducing the user's online participation time by up to 9.9× and decreasing the user's online communication costs by as much as 18.5× compared to DELPHI. It also provides a versatile tradeoff between the communication and computation loads between the user and server in addition to protecting more information about the neural network than prior works <cit.>.



§ RELATED WORK



 §.§ Privacy-Preserving Inference on Neural Networks

Secure Inference has received considerable attention in recent years. It is developed based on the following techniques: homomorphic encryption <cit.>, secret sharing and garbled circuits <cit.>. Some earlier works utilize a single technique and their models are simple machine learning models such as linear regression <cit.> <cit.>, linear classifiers <cit.> <cit.> <cit.>, etc. 




  
HE-based Protocols Pure HE-based protocols usually protect the complete model information including the architecture but lack support for non-approximated activation functions. CryptoNets <cit.> was one of the earliest works to use homomorphic encryption on neural network inference. CryptoDL <cit.> improves upon CryptoNets by choosing the interval and the degree of approximations based on heuristics on the data distribution. CHET <cit.> also improves upon CryptoNets by modifying the approximated activation function that CryptoNets uses to f(x) = ax^2 +bx with learnable parameters a and b. TAPAS <cit.> utilizes FHE <cit.> to allow arbitrary-layer neural network processing and proposes tricks to accelerate computation on encrypted data. nGraph-HE <cit.> develops HE-aware graph-compiler optimizations on general matrix-matrix multiplication operations and convolution-batch-norm operation. CHIMERA <cit.> uses different HE schemes for ReLU functions and other neural network functions. It needs to switch between two HE schemes to evaluate the model, leading to high computational costs. Cheetah <cit.> presents a set of optimizations to speed up HE-based secure inference, including auto-tuning HE security parameters, the order scheduler for HE operations, and a hardware accelerator architecture. 
Pure HE-based protocols involve heavy computation and their performance are not practical  due to their costly homomorphic computations. 



  
MPC-based Protocols
Pure MPC-based protocols are usually high-throughput but with high communication overhead due to multiplication operations. Furthermore, they usually need to expose the structure of the model to all parties. ABY <cit.> is a 2PC protocol that supports arbitrary neural network functions by combining Arithmetic, Boolean circuits, and Yao’s garbled circuits. ABY3 <cit.> improves ABY by extending it to 3-party settings and optimizing the conversion between Arithmetic, Boolean circuits, and garbled circuits. 3PC protocol SecureNN <cit.> distributes 2-out-of-2 secret sharing of the input among two of three servers. However, it is only demonstrated on small DNNs.  
FALCON, a 3PC protocol proposed by Wagh et al.<cit.> uses replicated secret sharing to reduce the number of interactions. However, the use of replicated secret sharing in the context of prediction-as-a-service necessitates an increase in communication costs for the user.

To sum up, MPC-based protocols rely on assumptions such as non-collusion, or an honest majority among the servers. Such a strong assumption of the system might not always align with the practical environments. Conversely, SECO assumes dishonest majority among servers, meaning the user's input privacy can be protected even when most of the servers are corrupted by the adversary. Thus, SECO can offer a higher degree of trustworthiness to users than pure MPC-based protocols. Furthermore, these 3PC protocols require the user to divide the input into multiple random shares. Each input share is then dispatched to three different servers, thereby increasing the communication costs for the user. 



  
HE-MPC-based Protocols
HE-MPC-based Protocols are designed by using both HE and MPC in a hybrid scheme. GAZELLE <cit.> designs schemes for HE neural network operations. It uses HE to evaluate linear layers and Garbled Circuits to evaluate non-linear layers, achieving efficient secure inference with high accuracy. DELPHI <cit.> modifies GAZELLE by designing two phases with the offline phase using HE, speeding up the online inference by evaluating the neural network in plaintext. It also proposes a novel neural network architecture search planner to determine where to approximate ReLU in the neural network, trading off between performance and accuracy. MP2ML <cit.> is proposed based on a novel combination of nGraph-HE <cit.> and ABY <cit.>, combining additive secret sharing and CKKS <cit.> HE scheme. However, HE-MPC-based Protocols face several limitations. Firstly, they require revealing the neural network's architecture to users, problematic for confidential models like large language models. Secondly, they demand substantial computational power and communication capabilities from user equipment, which can be a barrier for those with limited resources. Thirdly, these methods assume the model is hosted on a single server, impractical for large models like GPT-4 that need multiple GPUs. SECO is derived from these 2PC HE-MPC-based Protocols and is optimized to address the mentioned limitations.



 §.§ Split Neural Network


Split neural network is a specific type of neural network architecture. In this method, the neural network is divided into two or more sections, and each section is processed on different devices or servers. It is often used to reduce load for devices with limited resources. SplitFed <cit.> is a federated learning framework that combines federated learning with split learning for better model privacy. However, it applies differential privacy rather than cryptographic tools. 
Pereteanu et al. introduce Split HE <cit.>, a framework where the server provides the user with the middle portion of the neural network. In this setup, the user conducts plaintext inference on the middle section of the network, while the server manages the inference of other parts under homomorphic encryption. However, this approach harms the privacy of the deployed model.
Khan et al. propose a secure training protocol named Split Ways <cit.> based on U-shaped split learning, which incorporates homomorphic encryption on the client side to protect user input privacy. However, while they deploy only the fully-connected layer on the server, the user is still responsible for training the majority of the neural network. 
Split HE and Split Ways explore the integration of split neural networks with cryptographic tools. However, they do not fully utilize the server's computing resources and continue to depend significantly on the user's device.




§ PRELIMINARIES








































 §.§ Split Neural Network


Split Neural Network <cit.><cit.> involves splitting a neural network model at different intermediate layers. Consider a neural network function 𝐅 composed of a sequence of layers {𝐅_1,...,𝐅_L}. Each layer 𝐅_i can be a linear layer or an activation layer. The prediction result of the model for a given input 𝐱_1 can be computed by 𝐅(𝐱_1) = 𝐅_L(𝐅_L-1(...(𝐅_1(𝐱_1)))). We split the model at the l^th layer to get two models 𝐅_1 = {𝐅_1,...,𝐅_l}, 𝐅_2  = {𝐅_l+1,...,𝐅_L} which could be stored at two different nodes. The output can be expressed as 𝐅(𝐱_1) = 𝐅_2(𝐅_1(𝐱_1)). We adjust l to adjust the processing (training or inference) load for the parties deployed with 𝐅_1 and 𝐅_2.


 §.§ Cryptographic Blocks

We use the following cryptographic blocks to build the protocol.


  §.§.§ The Plain BFV Homomorphic Encryption
 
The Brakerskil-Fan-Vercauteren scheme <cit.> is a Ring-Learning with Errors (RLWE)-based cryptosystem that supports additive and multiplicative homomorphic operations. We define the plaintext space to be R_t = Z_t [X]/(X^n+1) and the ciphertext space to be R_q = Z_q [X]/(X^n+1), where n is a power of 2. We denote Δ = ⌈q/t⌉. BFV Homomorphic Encryption scheme can be expressed as a tuple of functionalities =(, , , ). The scheme is based on two kinds of distributions: the key distribution R_3 = Z_3 [X]/(X^N+1) with coefficients uniformly distributed in {-1,0,1}; the error distribution χ over R_q with coefficients distributed according to a centered discrete Gaussian.

    
  * → (,):It takes a security parameter as input and returns a public key  and a secret key .
    
    
  * (, m)→ ct: It takes as input the public key  and message m, and outputs the ciphertext of m denoted as ct.
     

  * (, ct)→ m: It takes as input the secret key  and the ciphertext ct, and outputs message m.
    

  * (,{ct_i,pt_i},f)→ ct_f: It takes as input the public key , valid ciphertexts {ct_i} encrypting {m_i} or plaintexts {pt_i} encoding {m_i} and operation f, and returns a valid ciphertext ct_f encrypting f({m_i}_i). The class of function f(·) that are supported include element-wise addition , subtraction  and multiplication  and slot rotation , as well as a combination of those basic operations.



  §.§.§ Multiparty BFV Homomorphic Encryption (MPHE)

The Multiparty version of BFV Homomorphic Encryption scheme (MPHE) <cit.> is extended from the plain BFV scheme. It enables multiple distributed parties, each with input in private, to collaboratively compute a function of those inputs without leaking any party's input. In this scheme, a common public key collectively generated by parties is known to all parties, while the corresponding secret key  is distributed among parties. Below we introduce the main functions used in the protocol. Consider a scenario that N parties want to collaboratively compute a function of their private inputs.


    
  * ()→ (_i,_i) :Each party runs () and generates a public key _i and secret key _i. This procedure requires a public polynomial p_1, which is agreed upon by all N parties.
    
  * ({_i}_i=0^N-1)→ :It returns a common public key , for a set of public keys {_i}_i=1^N-1. Its corresponding secret key  is the aggregation of {_i}_i=1^N-1.
    
    
  * (, m)→ ct :It returns a ciphertext ct by running (,m).
    
  * (,{ct_i,pt_i},f)→ ct_f :It returns a ciphertext ct_f by running (,{ct_i,pt_i},f).
    
  * (ct, _i)→ pd_i :Used by each party i. It takes as input a publicly known ciphertext ct, party's secret key _i, outputs a partial decryption pd_i.
    
    
  * ({pd_i}_i=0^N-1,ct)→ m': It takes as input a set of partial decryption {pd_i}_i=0^N-1 and the corresponding publicly known ciphertext ct. By aggregating the partial decryption, it outputs m' which is equal to (ct,), provided that the noise powers are again selected in a suitable manner.
     




  §.§.§ Oblivious Transfer (OT)

Oblivious transfer <cit.> <cit.> is a protocol involving two parties, a sender who has an input two messages m_0,m_1, and a receiver who holds a bit b as input. Via OT, the receiver obtains m_b. The security of the protocol guarantees that the sender does not learn anything about bit b and the receiver does not learn anything about the message m_1-b.


  §.§.§ Garbled Circuits

A Garbled Circuit scheme <cit.> <cit.> is a cryptographic tool involving two parties joint computing a function C(x_1,x_2) where x_1 and x_2 are inputs provided by the garbler and the evaluator. The scheme should keep the inputs fully private. It is consists of a tuple of algorithms =() with the following syntax:

    
  * (, C)→ (C,: Used by the garbler. Assume C is composed of G gates (e.g., XOR, AND, etc.) in total. The garbler garbles the circuit by 1) Assigning two random k-bit labels to each wire in the circuit C corresponding to 0 and 1 where k is the security parameter usually set to 128; 2) For each gate g, g∈ [G] in C, computing a garbled table C_g with 4 rows corresponding to 4 combinations of inputs labels. Each row is the encryption of the output with two corresponding input labels as the encryption key <cit.>. Output C = {C_g}_g=0^G-1 and labels corresponding to input wires .
    
  * (, x_i) → _x_i : This function involves both the garbler and the evaluator. If i=2, for input x_2 held by the garbler, the garbler maps x_2 with the labels generated for input wire corresponding to x_2 to obtain _x_2, then sends _x_2 to the evaluator. If i=1, for input x_1 held by the evaluator, the evaluator cannot simply send x_1 to the garbler to obtain _x_1. This undermines input privacy. The garbler and the evaluator engage in an OT protocol where the garbler provides labels for inputs wires belonging to the evaluator and the garbler provides the bit-expression of x_1. The evaluator will receive the labels corresponding to x_1 without exposing x_1 to the garbler via OT. This step outputs the correct labels _x_i corresponding to actual input x_i.
    
    
  * (C,{_x_i}_ i ∈{1,2})→ y: Used by the evaluator. The evaluator starts from the first gate and uses two input labels with the garbled table C_0 to obtain the output. Then the evaluator performs the evaluation for each gate in order until obtaining the output y of the last gate, where y = C(x_1,x_2).
    

A two-party Garbled Circuit scheme can be easily extended to a multiparty computations scheme of arbitrary complexity with an arbitrary number of participants providing inputs. The input providers obtain the labels for their private inputs via OT from the garbler, then they transmit the labels to the evaluator, which reveals nothing about the inputs since the labels are random.



  §.§.§ Additive Secret Share

Let q be a prime. A 2-of-2 additive secret sharing of x ∈Z_q is a pair (⟨ x ⟩_1,⟨ x ⟩_2)=(x-r,r)∈Z_q^2 for a random r ∈Z_q such that x=⟨ x ⟩_1+⟨ x ⟩_2. Additive secret sharing is perfectly hidden, for example, given a share ⟨ x ⟩_1 or ⟨ x ⟩_2, the value x is perfectly hidden.
 n-of-n additive secret sharing can be easily extended from 2-of-2 additive secret sharing. A n-of-n additive secret sharing of x ∈Z_q can be constructed by generating (r_1,...,r_n-1) ∈Z_q^n-1 uniformly at random and setting r_n = x-∑_i=1^n-1r_i(mod q).
 

 §.§ Prior Work: DELPHI

 Here we describe the protocol we built upon: DELPHI <cit.>. DELPHI is a cryptographic neural network inference protocol. There are two parties in the system of DELPHI: the user and the server. DELPHI uses additive secret sharing pre-built with homomorphic encryption to evaluate linear layers, and garbled circuits pre-built to evaluate activation functions. Let 𝐅 = {𝐅_i}_i=1^L described in Section <ref> be the deployed model on the server, 𝐱_i = 𝐅_i-1(𝐅_i-2(...(𝐅_1(𝐱_1)))) is the the intermediate prediction result of the first i-1 layers where 𝐱_1∈Z_q is the input held by the user.
 
 
 In the preprocessing phase, the client and the server pre-compute the secret share which will be used for the inference phase. For each linear layer i∈[L], with the user providing the ciphertext of 𝐫_i and the server providing the plaintext of 𝐅_i and 𝐬_i, two parties interact to compute 𝐅_i 𝐫_i-𝐬_i where 𝐫_i and 𝐬_i are the random masks for building secret shares, resulting in each of the party holding one of the two secret shares of 𝐅_i 𝐫_i. For ReLUs following linear layers, the server constructs a garbled circuit C_i based on circuit C computing ReLU function. Then the server transmits C_i to the user and input labels of 𝐅_i 𝐫_i-𝐬_i and 𝐫_i+1 via Oblivious Transfer to the user. We note that the algorithm for computing the ciphertext of 𝐅_i 𝐫_i in DELPHI relies on GAZELLE's design <cit.> for HE linear operations (convolutional and fully-connected layer). We do not provide details of this algorithm but represent it as a function . With function , ciphertext ct_𝐫_i encrypting 𝐫_i and plaintext pt_F_i encoding 𝐅_i as input,  will output the ciphertext of 𝐅_i 𝐫_i.
 
 In the inference phase, assuming 𝐱_i is the prediction result of the first i-1 layers, at the beginning of the i^th layer, the server and the user hold the shares of 𝐱_i: 𝐱_i-𝐫_i and 𝐫_i. The server evaluates to get 𝐅_i(𝐱_i-𝐫_i)+𝐬_i, resulting in the server and the user holding the secret shares of 𝐅_i𝐱_i. For evaluating ReLU, the server transmits the input labels corresponding to its secret share to the user. Then the client evaluates C_i to obtain a secret share of the ReLU output.

 

§ SECURE INFERENCE: HIERARCHICAL SERVER CONFIGURATIONS

 
Our proposed setup involves splitting the neural network model across three server nodes. We assume that the user communicates with a gateway server that holds part of the neural network model. The remainder of the model is stored on remote servers that do not directly communicate with the user.  One motivation for considering such a scenario is that the interactions of the user and the server are required in computing each layer in the DELPHI protocol. This increases the computational and communication load on the user. Furthermore, the user gets access to information about the neural network including the shape of each layer and the total number of model layers since the user needs to generate randomness for each linear layer to build up additive secret shares, which is undesirable.  By deploying a subset of the model on remote servers that do not directly interact with the user we can partially address these concerns.


 §.§ Baseline Configuration

We now explain why a straightforward extension of the DELPHI protocol that splits the neural network model across two server nodes as shown in Fig. <ref> will lead to information leakage. In this setting, the user and the gateway server as well as the gateway server and the remote server respectively execute DELPHI to evaluate the two split models in order as Figure <ref> shows. In this case, the user can be offline when two servers evaluate the second split model and the architecture of the second split model can be kept secret from the user. 

Let us consider deploying 𝐅_1,𝐅_2 (described in Section <ref>) on server A and B respectively. When evaluating 𝐅_2 involving server A and B, server B holds the masked intermediate prediction result 𝐱_i-𝐫_i while randomness 𝐫_i is only contributed by server A. Note that if the two servers collude to recover 𝐱_i they can get access to full parameters of the first i-1 layers, then execute the white-box model inversion attack proposed in <cit.> to reconstruct the user's input.



 §.§ Proposed Scheme

We consider a setting with a hierarchy of servers where there are a gateway server and two remote servers as Figure <ref> shows. We deploy 𝐅_1 on the gateway server A and the processing of it follows DELPHI. For the second split model 𝐅_2, the linear layers are randomly divided into 2 shares. We distribute the shares among 2 remote servers. 









We summarize the main advantages of our proposed architecture as follows:  1) hide partial architecture information of the model from the user and reduce the user's computation and communication cost 2) defend the possible white-box model inversion attack resulting from the model leakage and intermediate prediction result recovery caused by server collusion.  We give the detailed description of the protocol in Section <ref>.



§ SYSTEM OVERVIEW







As shown in Figure <ref>, there are four parties in our setting: a user, and hierarchical service providers including server A, B, and C. The hierarchy contains two layers of servers: gateway server A in the first layer, and server B and C in the second layer. Three servers cooperate to provide prediction service of a model 𝐅. The model 𝐅 can be expressed as 𝐅=(𝐅_A:{𝐅_i}_i ∈{1,...,l},𝐅_BC:{𝐅_i}_i∈{l+1,...,L}) where L is the total number of model layers. Server A holds the first l layers 𝐅_A. For i^th layer i ∈{l+1,...,L}, Server B and C hold the random share of linear layer 𝐅_i, which can be expressed as 𝐅_i=∑_j=2^3𝐅_i^j (j=2 for server B, j=3 for server C).


 §.§ Threat Model and Privacy Goals



  
Threat model. We assume a passive-adversary model with corruption of a user or up to two servers (any two). The parties will not deviate from the protocol but the adversary can corrupt the user or up to 2 servers so can know corrupted parties' private data and the observations. We assume the adversary cannot corrupt the user and the server simultaneously. Additionally, we assume server A belongs to the service provider while server B and C are external. Deployment of random shares on external servers won't compromise the proprietary model privacy.





  
Privacy Goals.
For the user, we aim to design a protocol that enables the user to only learn the result of the inference and the architecture of the model on gateway server A. All other information about the model including the model parameters, the total number of layers, and the architecture of the layers on server B, C should be kept secret from the user. 

Assuming up to 2 servers corrupted by the adversary, we have the privacy goals for the servers as the followings: 1) the adversary should not learn any honest party's model parameters or the intermediate prediction result 𝐱_i; 2) the adversary should not infer the user's input.



§ THE PROTOCOL: FORMAL DESCRIPTION

In this section, we introduce our secure inference protocol. The protocol takes as input the user's data 𝐱_1∈Z_q and the split model (𝐅_A={𝐅_i}_i ∈{1,...l},𝐅_BC={𝐅_i}_i ∈{l+1,...L}), and enables four parties to execute secure inference collaboratively. The protocol can be considered as the combination of a 2-PC protocol, executed between the user and the gateway server for the inference of 𝐅_A, and a 3-PC protocol, executed between the gateway server and two remote servers for the inference of 𝐅_BC. The 2-PC protocol is adapted from DELPHI <cit.> and the 3-PC protocol is designed to achieve the secure neural network computations in the presence of dishonest majority of honest-but-curious servers. The protocol consists of three phases as shown in Figure <ref>: a setup phase, a preprocessing phase, and an online inference phase. 





 §.§ Setup Phase

The setup phase is executed before the user appears in the system.


  §.§.§ Key Generation

This step is to generate keys for homomorphic computation. 

    
  * Each server runs  to generate key pair (_j,_j) (j=0 for the user, j=1 for server A, j=2 for server B, j=3 for server C).
    
  * Server A collects the public keys from the three servers, runs  to obtain a common public key , then transmits  to the user, server B and C. 
    

At the end of the step, each server holds a common public key .


  §.§.§ Model Preparation
: This step is the preparation for 𝐅_BC which can be executed before the user appears in the system. Like in DELPHI, the key insight is to pre-compute the additive secret shares of 𝐅_i𝐫_i for each linear layer of 𝐅_BC. Server A, B, and C collaboratively compute the secret shares of 𝐅_i𝐫_i with MPHE scheme, where 𝐅_i is distributed among server B and C and the randomness 𝐫_i is solely contributed by the user if i=l+1 or jointly contributed by three servers if i∈{l+2,...,L}. The preprocessing of the l+1^th linear layer is required for a secure transition from the inference of 𝐅_A to 𝐅_BC. Additionally, three servers construct garbled circuits for each activation layer in 𝐅_BC. 


  
Preprocessing for other linear layers. For the i^th layer i ∈{l+2,...L}, server A, B, and C contribute to the randomness 𝐫_i and collaborate to compute three secret shares of (𝐅^2_l+1+𝐅^3_l+1)𝐫_l+1 . With server B and C providing the ciphertexts of private model parameter 𝐅^j_i, input randomness 𝐫^j_i and output randomness 𝐬^j_i, server A computes its share 𝐄_i^1=(𝐅^2_i+𝐅^3_i)(𝐫^1_i+𝐫^2_i+𝐫^3_i)-𝐬^2_i-𝐬^3_i, resulting in building the additive shares of (𝐅^2_i+𝐅^3_i)(𝐫^1_i+𝐫^2_i+𝐫^3_i) among three servers. See Algorithm <ref> for details. In Algorithm <ref>, we use  (Algorithm <ref>) which is a protocol for the distribute decryption of MPHE scheme involving three servers.


  
Preprocessing for activation layers
Three servers collaborate to build the garbled circuit on server A and prepare the evaluation of the activation layers. 
With server B as the garbler, server C as the evaluator, and server A  providing input as the third party, server B first prepares the garbled circuit C_i and labels by running (,
C_i) where  is the security parameters and C_i is described in Algorithm <ref>. To transmit the labels corresponding to actual input values, server B sends the labels of the input value provided by itself to server C through a public channel. For input values from server A and C, server A and C first obtain the actual labels from server B via OT. Then server C collects the labels obtained by server A through a public channel.


 §.§ Preprocessing Phase

In the preprocessing phase, the user first generates a key pair (_0,_0) and broadcasts _0 to each server in the system through Server A. Server A sends common public key  to the user. For preprocessing of 𝐅_A, the user and server A collaboratively pre-compute the secret shares of weights for linear layers and construct garbled circuits for activation layers. 


  §.§.§ Preprocessing for 𝐅_A



This step follows the approach in DELPHI.





  
Preprocessing for linear layers.
For the i^th linear layer i ∈ [1,...,l], with the user providing the ciphertext of 𝐫_i, server A computes the ciphertext of 𝐅_i 𝐫_i-𝐬_i where 𝐫_i is the randomness of the layer's input size and 𝐬_i is the randomness of the layer's output size, then the user decrypts it using _0, resulting in the user holding 𝐅_i 𝐫_i-𝐬_i and server A holding 𝐬_i which are two additive shares of 𝐅_i 𝐫_i.






















  
Preprocessing for activation layers. 
 After the preprocessing for linear layers, the user and server A collaborate to build the garbled circuits for activation layers in 𝐅_A. C_i is the circuit to compute the activation function. Server A will execute  to generate a set of the garbled tables C_i according to gates in circuit C_i and labels corresponding to all input wires in C_i. Then server A transmits the labels for actual input values to the user. For input values held by server A such as one-time password 𝐝_i+1, server A generates labels based on the bit expression of 𝐝_i+1 and directly sends it to the user. For input values held by the user such as 𝐅_i𝐫_i-𝐬_i,𝐫_i+1, server A and the user engage in OT protocol to let the user obtain the corresponding labels without server A knowing the actual value.
 






















  
Preprocessing for the l+1^th linear layer. 
In this step, the protocol involves 4 parties to prepare for the transition from the inference of 𝐅_A to 𝐅_BC. 
The user, server A, B, and C collaborate to build the additive shares of (𝐅^2_l+1+𝐅^3_l+1)𝐫_l+1 among three servers, that is, with server B and C providing the ciphertexts of their own 𝐅^j_l+1 and the ciphertxts of the output randomness 𝐬^j_i+1, with the user providing the ciphertext of input randomness 𝐫_l+1, server A homomorphically computes the ciphertext of 𝐄_l+1=(𝐅^2_l+1+𝐅^3_l+1)𝐫_l+1-𝐬^2_l+1-𝐬^3_l+1 and then involves server B and C to decrypt it. At the end of the preprocessing of the l+1^th linear layer, three server holds the respective secret share of (𝐅^2_l+1+𝐅^3_l+1)𝐫_l+1. Note all the encryption in this stage are conducted with MPHE public key . 






























 §.§ Online Inference Phase

This section introduces the inference phase in the order of model layers. First, as preamble, the user computes 𝐱_1-𝐫_1 and sends it to server A.



  
Inference of 𝐅_A The inference of 𝐅_A involves the participation of the user and server A. Before evaluating the i^th, i∈ [1,...,l] layer, server A holds the masked input 𝐱_i-𝐫_i and the user holds the input randomness 𝐫_i+1 for next layer. Server A first computes 𝐅_i(𝐱_i-𝐫_i)+𝐬_i to evaluate the linear part and sends the labels of it to the user. Collecting the labels of all the input wires in C_i, the user executes  to obtain a masked ReLU output as the masked input for the next layer.





































  
Transition After the evaluation of 𝐅_A, server A holds 𝐱_l+1-𝐫_l+1. Server A forwards it to server B and C as a transition step from the inference of 𝐅_A to 𝐅_BC.


  
Inference of 𝐅_BC
The inference of 𝐅_BC involves the participation of server A, B, and C. At the beginning of the inference for i^th layer i ∈{l+1,...,L}, server B and C hold 𝐱_i-𝐫_i where 𝐫_i is solely contributed by the user when i=l+1, and is contributed by server A, B and C for linear other layers of 𝐅_BC. Server A, B and C collaborate

to do inference on i^th layer i ∈{l+1,...,L}. First server B and C will compute 𝐄_i^j=𝐅_i^j(𝐱_i-𝐫_i)+𝐬_i^j, resulting in three servers holding three additive shares of the real prediction result for the current layer (𝐅_i^2+𝐅_i^3)𝐱_i. The garbled circuit C_i (Algorithm <ref>) first aggregates shares to recover (𝐅_i^2+𝐅_i^3)𝐱_i, then computes ReLU of it, finally masks the ReLU output with the randomness of the next layer contributed by three servers. The entire process is in the encrypted domain. The garbled circuit for each ReLU layer of 𝐅_BC are located at server C. Thus server C can evaluate the garbled circuit when collecting all labels for input wires of C_i in a secure way. 





 §.§ Output

After the evaluation of 𝐅_BC, server A holds the share (𝐅_L^2+𝐅_L^3)𝐫_L-𝐬_L^2-𝐬_L^3, server B holds 𝐅_L^2(𝐱_L-𝐫_L)+𝐬_L^2 and server C holds 𝐅_L^3(𝐱_L-𝐫_L)+𝐬_L^3.

    
  * Server B and C computes  (_0,𝐅_L^i(𝐱_L-𝐫_L)+𝐬_L^i) and sends this ciphertext to server A.
    
  * Server A homomorphically add the received ciphertext to compute the ciphertexts ct_𝐲̂ of prediction result 𝐲̂ where 𝐲̂=(F_L^2+F_L^3)𝐱_L, then transmits ct_𝐲̂ to the user.
    
  * The user decrypts ct_𝐲̂ with sk_0 to obtain the prediction result 𝐲̂.



 §.§ Discussion


We next highlight key technical contributions and the motivations for our design choices.



  
Secret Sharing for 3PC and Integration with MPHE
To sum up, during the executions, the protocol processes model 𝐅_A and 𝐅_BC. The processing of 𝐅_A follows DELPHI which uses plain HE scheme to build two secret shares of 𝐅_i𝐫_i among two parties in the preprocessing phase, where 𝐫_i is randomly generated by the user; In the inference phase, 𝐫_i will be used as the mask for the intermediate prediction result 𝐱_i and it can be canceled by the secret share built in the preprocessing phase. Our protocol follows this idea to build three secret shares of {𝐅_i𝐫_i}_i=l+1^L in the preprocessing phase of 𝐅_BC, where each of the servers contributes its own part to the randomness 𝐫_i. However, using a plain HE scheme to compute the secret shares is not privacy-preserving in this case because the model parameters and randomness are distributed. To ensure that each of the three servers exclusively accesses its designated information and no extraneous data, we design a new secret sharing scheme as in Table <ref>, so the shares can be computed with MPHE using the supported operations as described in Section <ref>, enabling secure computation of linear functions. 

            




  
Assignments of Servers in GC
For the secure computation of non-linear functions, we deviate from the conventional approach of assigning the layer's input provider as the evaluator in Garbled Circuits (GC), like in DELPHI. Instead, we appoint a remote server as the evaluator and another as the garbler. This approach substantially diminishes the communication complexity during the inference phase, as communication now occurs exclusively between these two remote servers. Additionally, our protocol maintains the security of user's input even in the event of two remote servers being compromised, because of the gateway server's contribution to the masking term during the setup phase. Last but not the least, we design a transition from user-server interaction to server-server interaction to ensure no information leakage in this process.




§ INFORMATION LEAKAGE ANALYSIS

In this section,  we show the protocol described in Section <ref> achieves the privacy goals described in Section <ref> under a passive-adversary model. Based on Composition Theorem for semi-honest model (Theorem 7.3.3 in <cit.>), by proving the security of the protocol in each phase, the security of the entire protocol is proven. 






We prove the security of the preprocessing and inference phase following the simulation paradigm <cit.>. According to the simulation paradigm, a protocol is secure if whatever can be computed by a party involved in the protocol can be computed given only the input and the output of the party. So the security can be modeled by defining a simulator that can generate the view of the party during the procedure given the input and the output. If the view of the party can be simulated based on the input and the output only and is computationally indistinguishable from the real view of the party, it implies the party only learns what can be computed given the input and the output, so the protocol is secure. 



 §.§ Setup phase

The setup phase is independent of the rest of the protocol. For a given user and the servers, it has to be run only once. The protocol used in the setup phase is a composition of (.) and (.). 

In Mouchet et al. <cit.>, it shows that the protocol in the setup phase can securely and correctly generate a valid plain BFV key pair. Provided with valid plain BFV key pair,  and  can output valid BFV ciphertext,

which guarantees the semantic security of HE and MPHE schemes described in Proposition <ref> and Proposition <ref>. The semantic security of HE and MPHE schemes provides the security basis for the following two phases.


    For any two messages m_1, m_2, no adversary has an advantage (better than 1/2 chance) in distinguishing between distributions (, m_1) and (, m_2).
    


    For any subsets of at most colluded N-1 clients, for any two messages m_1 and m_2, no adversary has an advantage (better than 1/2 chance) in distinguishing between distributions (, m_1) and (,m_2).




 §.§ Preprocessing and Inference Phase

In this section, we prove the privacy goals in Section <ref> are achieved in the preprocessing and inference phase following the simulation paradigm <cit.>.




 We can prove the security of the preprocessing and inference phase by 1) providing the real view of the adversary for the case when the user or two servers are corrupted 2) describing simulators for cases where two servers or the user are corrupted 3) comparing each term in the view of the adversary and proving the indistinguishability. We have three privacy goals that are to preserve the privacy of 1) the user's input 2) the intermediate prediction result 3) the complete model parameters. Intuitively, by distributing the model parameters, the honest server's partial model parameters are protected thus the adversary cannot obtain the complete model. The user's input and the intermediate prediction result are protected by the randomness. To prove they are fully secure, we can determine the indistinguishability of each element in the view separately. Note Composition Theorem <cit.> can be applied to the sequential compositions of arbitrary protocols involving multiple parties. Based on Composition Theorem <cit.>, the security of the protocols for two phases is proven when the composing sub-protocols are secure. The security proof of cryptographic sub-protocols (section <ref>) are provided in <cit.><cit.><cit.> assuming the passive adversary model.


We name the protocols of two phases as  and  and define the corresponding ideal functionality as 


    f_pre   (_0,,{C_i}_i=1^L,{𝐅_i}_i=1^l,{𝐅_i^j}_i=l+1,j=2,3^i=L) 
       = {{_i^U}_i=1^l,{_i^A}_i=l+1^L,{C_i}_i=1^L}
    
        f_inf   (𝐱_1,{_i^U}_i=1^l,{_i^A}_i=l+1^L,{C_i}_i=1^L) 
       = {{_i}_i=1^L}

where {_i^U=𝐅_i𝐫_i-𝐬_i}_i=1^l, {_i^A=(𝐅_i^2+𝐅_i^3)(𝐫_i^1+𝐫_i^2+𝐫_i^3)-𝐬_i^2-𝐬_i^3}_i=l+1^L, {_i= 𝐱_i-𝐫_i}_i=1^L. Next we we discuss the cases where server A and B are corrupted and the user is corrupted. 

The proofs for other cases, wherein either servers B and C or servers A and C are corrupted, can be readily extended based on the methodology outlined in the following proof.


 

  
Server A, B corrupted
The real view v_AB of the adversary corrupting server A and B includes:

    
  * Ciphertexts set ct_S_AB ={{ct_r_i}_i=1^l+1,{ct_r_i^2,ct_r_i^3}_i=l+2^L, 
{ct_F_i^2,ct_F_i^3,ct_s_i^2,ct_s_i^2}_i=l+2^L,
    {ct__i^A}_i=l+1^L}
    
  * The masked input  {_i=𝐱_i-𝐫_i}_i=1^l+1,{_i=𝐱_i-𝐫_i^3}_l+2^L+1
     
  * The secret shares set ^A_i=(𝐅_i^2+𝐅_i^3)𝐫_i-𝐬_i^2-𝐬_i^3,i∈{l+1,...,L}
    
  * Garbled circuit {C_i}_i=1^L and labels {_i}_i=l+1^L 

We define a simulator S_AB that simulates the view of an adversary corrupting server A and B. For a given value x, we denote the simulated equivalent by x and the encryption of it by ct_x. S_AB is given _0,,𝐅_A,{𝐅_i^2}_i=l+1^L , (q_i, w_i)_i=1^L and proceeds as follows:

    
  * S_AB chooses a uniform random tape for server A and B.
    
  * In the preprocessing phase:
    
    
  * S_AB, as the user, computes {ct_r_i=(_0, 𝐫_i)}_i=1^l+1 and sends it to server A where 𝐫_i←Z_q^q_i. Then S_AB receives the evaluated ciphertext from server A.
    
  * With S_AB as the evaluator of the garbled circuits for i^th, i∈{1,...,l} non-linear layer, S_AB receives C_i from server A and labels corresponding to random input generated by S_AB.
    
  * Server B computes and sends the ciphertexts 
    
    ct_F_i^2   =(,𝐅_i^2)
    ct_r_i^2   =(, 𝐫_i^2)
    ct_s_i^2   =(, 𝐬_i^2)

    to server A.
    
  * S_AB, as server C, sets 𝐅_i^3=0, then generates 𝐫_i^3←Z_q^q_i, 𝐬_i^3←Z_q^w_i and encrypts them with ; During distribute decryption, server B receives the evaluated ciphertext from server A and server A computes _i^A = (𝐅_i^2+𝐅_i^3)(𝐫_i^1+𝐫_i^2+𝐫_i^3)-𝐬_i^2-𝐬_i^3 
    
    
  * In the inference phase:
    
        
  * Preamble: S_AB, as the user, sends -𝐫_1 to server A to evaluate the first linear layer. Then S_AB receives the corresponding GC label.
        
  * S_AB evaluates C_i to obtain the output of garbled circuits and sends it to server A.
        
  * Transition: S_AB, as server C, receives the output of first l layers
        
  * S_AB obtains labels corresponding to  𝐬_i^3 and 𝐫_i+1^3 from server B, computes the output of garbled circuits and sends it to server B
    

 
The view simulated by S_AB is computaionally indistinguishable from the real view v_AB when the adversary is corrupting server A and B.

 We observe that the  and  protocol can be privately reducible to the , ,  and other protocols. Therefore, the secuirty of  and  protocol follow from the standalone security of each protocol by applying Composition Theorem for semi-honest model <cit.>. The indistinguishability of ciphertext is guaranteed by the semantic security (Proposition <ref> and Proposition <ref>) of protocol  and , and the indistinguishability of garbled circuits with labels is guaranteed by the security property of protocol  <cit.>. We only need to show the indistinguishability of the masked input and the the secret shares, then we can prove that  and  preserve their security. Due to using different random values which share identical distribution hence are statistically equivalent, we get {_i}_i=1^Lc≡{_i}_i=1^L,{^A_i}_i=l+1^L  c≡{^A_i}_i=l+1^L. 









The arguments above yields the following security property. 

    SECO introduced in Section <ref> securely realizes f_pre and f_inf in the presence of semi-honest adversary controlling server A and B.




  §.§.§ The user corrupted

We prove that the simulated view and the real view are computationally indistinguishable, thus information such as the model parameters which is beyond the view cannot be computed if the adversary corrupts the user.

The real view v_U of the adversary corrupting the user includes:


  * The masked input {_i=𝐱_i-𝐫_i-𝐝_i}_i=1^l

  * The additive share {_i^U=𝐅_i𝐫_i-𝐬_i}_i=1^l

  * Garbled circuit {C_i}_i=1^l and labels {_i}_i=1^l 


We define a simulator S_U that simulates the view of an adversary corrupting the user. S_U is given 𝐱_1, _0, and proceeds as follows:

    
  * S_U chooses a uniform random tape for the user.
    
  * In the preprocessing phase:
    
    
  * S_U, as server A, receives {ct_r_i=(_0, 𝐫_i)}_i=1^l+1 with 𝐫_i←Z_q^q_i. With 𝐅_i set to 0, the user receives the evaluated {ct_s_i=(_0, -𝐬_i)}_i=1^l+1 from S_U for a randomly chosen 𝐬_i from Z_q^w_i.
    
  * With S_U acting as the garbler of the garbled circuits for i^th, i∈{1,...,l} non-linear layer, the user receives C_i from S_U and labels corresponding to 𝐫_i and 𝐬_i. S_U sets the output of the circuit to be a random value.
 
    
    
  * In the inference phase:
    
        
  * Preamble: S_U, as server A, receives 𝐱_1-𝐫_1 from the user
        
  * S_U, as server A, sets the  𝐅_i=0 evaluates to obtain 𝐬_i and sends the corresponding simulated labels to the user.
        
  * The user evaluates C_i to obtain the output of garbled circuits which is set randomly, then sends it to S_U.
    

 
The view simulated by S_U is computaionally indistinguishable from the real view v_U when the adversary is corrupting the user.


Similar to the proof of Theorem <ref>, the views are computationally indistinguishable based on the security properties of , ,  and the use of randomness that are statistically equivalent.

The arguments above yields the following security property. 

    SECO introduced in Section <ref> securely realizes f_pre and f_inf in the presence of semi-honest adversary controlling the user.



§ EXPERIMENTS

In this section, we experimentally evaluate the performance of SECO and present our experimental results comparing SECO and other protocols. Currently, SECO supports the deployment of CNN models. 


 §.§ Implementation

Our implementation is based on SEAL library <cit.> and DELPHI's open source code [https://github.com/mc2-project/delphi]. We first implemented the operations for multiparty key-generation and distribute-decryption for BFV scheme <cit.> in SEAL. Then we modified DELPHI's implementation to support the collaborative inference involving multiple parties as described in Section <ref>.  For convolutional functions, we employ modulus-switching to reduce the ciphertext size as in Cryptflow2 <cit.>.

We use the same BFV security parameters as DELPHI uses, with 8192 as the degree of polynomial modulus and 2061584302081 as plaintext modulus.



















 §.§ Setup

We run our benchmarks in the WLAN setting with four instances in Australia or North America.
All the instances have Intel Xeon Processor running at 2.3 GHz with 32GB of RAM. Each experiment is conducted 5 times, and the results presented are the average of these runs.






































 §.§ Performance



  §.§.§ Compare with HE-MPC-based Protocols


We experimentally evaluate SECO in terms of execution time and communication cost for preprocessing and inference phase with the deployed model split at different layers. We present the comparison results of SECO, DELPHI <cit.>, and DELPHI-3. DELPHI-3 is a straightforward extension of DELPHI to 3PC setting. This extension employs MPHE for processing linear functions, while utilizing Garbled Circuits (GC) for non-linear operations. A key distinction between DELPHI-3 and SECO, lies in the allocation of roles within the GC scheme. In SECO, two remote servers are designated as the garbler and the evaluator in the GC process. In DELPHI-3, the gateway server assumes the role of the evaluator, while a remote server functions as the garbler. This will lead to different performance of SECO and DELPHI-3 in only online inference phase. 

We evaluate three schemes on ResNet32 <cit.> (with 62 layers in total). 


  
User's experience
Figure <ref> shows the comparison of the online execution time. As illustrated in the figure, the blue bars indicate the duration for which the user must remain connected and actively process the computation while the light blue bars represent the time required to process the model on remote servers. The total height of the bars in the figure represents the user's waiting time, spanning from the start of the query to the receipt of the prediction result. We can tell that the fewer layers deployed on gateway server A, the less execution time of SECO in the preprocessing phase. When all the layers of the model are on server A, SECO is reduced to DELPHI and only the user and server A are involved in the computation. When the number of layers on server A is reduced to two, SECO offers the most efficient experience in terms of both the shortest waiting and participation time for the user comparing to DELPHI, thereby potentially minimizing the user's computational burden. We note that SECO can operate on a device with just 2 GB of memory as the user device (with less than 10 layers of ResNet32 on the gateway server), whereas DELPHI cannot support such a device, making meaningful comparison impossible. This capability allows SECO to utilize lightweight devices, such as the smart watch, as user devices. Depending on the numbers of layers on server A, SECO's results are between 2.6×-1× faster than DELPHI in terms of user's waiting time. Accordingly, SECO's efficiency ranges from 9.9×-1× faster than DELPHI in terms of user's participation time.

Figure <ref> shows the comparison of the preprocessing execution time, revealing a similar trend as the observation in Figure <ref>. Depending on the numbers of layers on server A, SECO's results are between 11×-1× faster than DELPHI in terms of user's preprocessing time. We note that SECO's preprocessing only involves the preprocessing of 𝐅_A. The three server pre-compute the secret shares for 𝐅_BC before the user appears in the system.

As illustrated in Figure <ref> and Figure <ref>, SECO demonstrates up to 18.5× more efficient compared to DELPHI in terms of the user's online communication cost, additionally, it is up to 16.6× more efficient than DELPHI regarding the user's preprocessing communication cost. Even though SECO's total communication cost in the online inference phase is higher than DELPHI's, SECO reduces the user's cost. Figure <ref> shows that SECO can provide a tradeoff in communication load between user and server nodes, which might be desirable to service provider.



  
Effect of schemes with different assignments of servers in GC Figure <ref> and Figure <ref> depict the online performance of SECO, DELPHI, and DELPHI-3. We can tell from figures, the time and the cost for processing 𝐅_A are the same for SECO and DELPHI-3 as the 2PC protocol in both cases is derived from DELPHI. However, the time and cost for processing 𝐅_BC are significantly reduced in SECO. The inefficiency in DELPHI-3 arises because the evaluator of the garbled circuit obtains labels for server A's share during 𝐅_BC pre-computation, making server A's participation in the inference phase unnecessary. In contrast, SECO designates server C as the evaluator, involving only servers B and C in the inference phase. Our approach streamlines the process, avoiding the need for three-server involvement as in DELPHI-3.






  §.§.§ Compare with MPC-based protocols


In this section, we compare SECO with pure MPC-based protocols. To make a fair comparison, we deploy all the layers of the model on remote servers in SECO. So the user can send the masked input and receive the prediction result without further interaction like in pure MPC-based protocols. Figure <ref> shows the communication cost for the user in protocols including SECO, Falcon <cit.>, and SecureNN <cit.>. Although there is only a one-way transmission from user to servers, due to the use of replicated secret shares, the user in Falcon needs to duplicates each of the three input shares, and the user must transmit two distinct shares to the respective server. This process leads to a communication cost that is 6× greater compared to that in SECO. SecureNN uses a 2-out-of-2 secret sharing scheme between two servers, and the user needs to deliver each of two shares to the respective servers. It differs from the 2-out-of-2 secret sharing used between a user and a server in SECO. This process results in the communication cost 2× greater compared to SECO. 
Furthermore, we assess the execution time starting from when the user transmits the input until the user receives the prediction result. In Table <ref>, we evaluate 3 protocols on MiniONN and LeNet. The outperformance of SECO is attributed to the online inference only involving two (remote) servers while other protocols are executed between three servers. Although SECO is faster than other MPC-based protocols, it incurs a communication cost between servers that is 36 to 48 times higher than that of Falcon. We remind that SECO operates under a different system and threat model. SECO can protect the user's input even when the adversary corrupts two of the three servers while other protocols assume non-collusion between servers. The increase in communication cost can be justified by SECO's emphasis on enhancing the system's robustness and trustworthiness for the user.










§ CONCLUSION

In this paper, we design and implement SECO, a novel hybrid HE-MPC-based secure inference protocol with model splitting in a multi-server hierarchy setting. SECO ensures the confidentiality of the input data withstanding passive adversaries corrupting up to 2 servers, and also protects the model parameters from the user. Furthermore, SECO hides more information about the model architecture than other HE-MPC-based protocols and pure MPC-based protocols. In the experiments, SECO is shown to minimize the user device's obligation in computation and communication, making it applicable to devices with limited resources.

















ieeetr



§ NOTATION

Table <ref> provides the summary of the symbols used throughout this work.



§ SECURITY PROOF
 
We provide the security proof for other cases when server B and C are corrupted or server A and C are corrupted.


  
Server B, C corrupted
The real view v_BC of the adversary corrupting server B and C includes:

    
  * Ciphertexts set ct_S_BC ={{ct_r_i^2,ct_r_i^3}_i=l+2^L,
{ct_F_i^2,ct_F_i^3,ct_s_i^2,ct_s_i^2}_i=l+2^L,{ct__i^A}_i=l+1^L}
    
  * The masked input of  _i=𝐱_i-𝐫_i^1, i∈{l+2,...,L} 
    
  * Garbled circuit {C_i}_i=l+1^L and {_i}_i=l+1^L obtained by Server B and C


We define a real-world simulator S_BC that simulates the view of an adversary corrupting server B and C. S_BC is given {𝐅_i^2, 𝐅_i^3}_i=l+1^L, _0,
,{(q_i, w_i)}_i=l+1^L,_l+1 and proceeds as follows:

    
  * S_BC chooses a uniform random tape for server B and C.
    
  * In the preprocessing phase:
    
    
  * S_BC, as server A, receives ciphertexts 
        (, 𝐅_i^j),
(, 𝐫_i^j), 
    (, 𝐬_i^j)
    from server B and C.
    
  * S_BC generates 𝐫_i^1←Z_Q^q_i and computes _i^A = (𝐅_i^3+𝐅_i^2)(𝐫_i^1+𝐫_i^2+𝐫_i^3)-𝐬_i^3-𝐬_i^2
    
  * Server B sends the garbled circuit C_i and labels for inputs from server B to server C; S_BC obtains the labels from server B and forwards it to server C
    
    
  * In the inference phase:
    
        
  * Transition: S_BC generates a random value as the output of the first l layers _l+1 and sends it to server B and C
        
        
        
    


The view simulated by S_BC is computaionally indistinguishable from the real view v_BC when the adversary corrupting server B and C.


Similar to the proof of Theorem <ref>, the views are computationally indistinguishable based on the security properties of , , , and the use of randomness that are statistically equivalent.

The arguments above yields the following security property. 

    SECO introduced in Section <ref> securely realizes f_pre and f_inf in the presence of semi-honest adversary controlling server B and C.








     






  
Server A, C corrupted

The real view v_AC of the adversary corrupting server A and C includes:

    
  * Ciphertexts set ct_S_AC ={{ct_r_i}_i=1^l+1,{ct_r_i^2,ct_r_i^3}_i=l+2^L,
{ct_F_i^2,ct_F_i^3,ct_s_i^2,ct_s_i^2}_i=l+2^L,{ct__i^A}_i=l+1^L}
    
  * The masked input of the i^th, i∈{l+2,...,L} layer _i=𝐱_i-𝐫_i^2
     
  * The additive secret shares set ^A_i=(𝐅_i^2+𝐅_i^3)𝐫_i-𝐬_i^2-𝐬_i^3,i∈{l+1,...,L}
    
  * Garbled circuit {C_i}_i=1^L and {_i}_i=l+1^L for garbled circuits

We define a real-world simulator S_AC that simulates the view of an adversary corrupting server A and C. S_AC is given {𝐅_i^3, (q_i, w_i)}_i=l+1^L,
𝐅_A,_0, and proceeds as follows:

    
  * S_AC chooses a uniform random tape for server A and C.
    
  * In the preprocessing phase:
    
    
  * S_AC, as the user, computes (_0, 𝐫_i), i∈{1,...,l+1} to server A where 𝐫_i is randomly chosen from Z_Q^q_i. Then S_AC receives the evaluated ciphertext from server A.
    
  * With S_AC as the evaluator of the garbled circuits for i^th, i∈{1,...,l} non-linear layer, S_AC receives C_i, i∈{1,...,l} from server A and labels corresponding to random input generated by S_AC.
    
  * S_AC, as server B, sets 𝐅_i^2=0, then generates 𝐫_i^2←Z_Q^q_i, 𝐬_i^2←Z_Q^w_i and encrypts them respectively with ; For processing distribute decryption, server C receives the evaluated ciphertext from server A and _i^A = (𝐅_i^3+𝐅_i^2)(𝐫_i^3+𝐫_i^2+𝐫_i^1)-𝐬_i^3-𝐬_i^2 from S_AB
    
  * S_AC, as server B, runs the simulator for garbled circuits and generates C_i, i∈{l+1,...,L}, then sends labels corresponding to 𝐫_i^2 to server C
    
    
  * In the inference phase:
    
        
  * Preamble: S_AC sends -𝐫_1 to server A to evaluate the first linear layer.
        
  * S_AC, as the user, obtains labels from server A
        
  * S_AC, as the user, sends a random value as the output of garbled circuits back to server A.
        
  * Transition: S_AC, as server B, receives the output of the first l layers
        
  * S_AC, as server B, sends labels corresponding to 𝐬_i^2 to server C
        
  * S_AC receives the output of garbled circuits from server C
    


The view simulated by S_AC is computaionally indistinguishable from the real view v_AC when the adversary corrupting server A and C.


Similar to the proof of Theorem <ref>, the views are computationally indistinguishable based on the security properties of , , , and the use of randomness that are statistically equivalent.

The arguments above yields the following security property. 

    SECO introduced in Section <ref> securely realizes f_pre and f_inf in the presence of semi-honest adversary controlling server A and C.




§ DELPHI-3
 
DELPHI-3 is a 3PC protocol directly expanded from DELPHI. DELPHI-3 sets server B as the garbler and server A as the evaluator for GCs. 


  
Preprocessing for 𝐅_BC The preprocessing of linear layers is the same with SECO as described in Algorithm <ref>. The preprocessing of ReLU is as follows: with server B as the garbler, server A as the evaluator, and server C providing input as the third party, server B first prepares the garbled circuit C_i and labels by running (,C_i) where  is the security parameters and C_i is described in Algorithm <ref> in Appendix <ref>. To transmit the labels corresponding to actual input values, server B sends the labels of the input value provided by itself to server A through a public channel. For input values from server A and C, server A and C first obtain the actual labels from server B via OT. Then server A collects the labels obtained by server C through a public channel. 


  
Online-inference for 𝐅_BC The inference of 𝐅_BC involves the participation of server A, B, and C. At the beginning of the inference for i^th layer i ∈{l+1,...,L}, server B and C hold 𝐱_i-𝐫_i where 𝐫_i is solely contributed by the user when i=l+1, and is contributed by server A, B and C for linear other layers of 𝐅_BC. Server A, B and C collaborate

to do inference on i^th layer i ∈{l+1,...,L}. First server B and C will compute 𝐅_i^j(𝐱_i-𝐫_i)+𝐬_i^j, resulting in three servers holding three additive shares of the real prediction result for the current layer (𝐅_i^2+𝐅_i^3)𝐱_i. The garbled circuit C_i on server A (Algorithm <ref> in Appendix <ref>) first aggregates shares to recover (𝐅_i^2+𝐅_i^3)𝐱_i, then computes ReLU of it, finally masks the ReLU output with the randomness of the next layer contributed by three servers. The entire process is in the encrypted domain. Server A can evaluate the garbled circuit when collecting all labels for input wires of C_i. Note the inefficiency of this method is that despite three servers holding the respective additive shares of the real prediction result, the evaluator of the garbled circuit already obtains the labels for server A's share during the pre-computation for 𝐅_BC. Thus server A does not need to involve in the inference phase of 𝐅_BC. We observe this inefficiency and set server C as the evaluator in SECO instead.



§ SETUP PHASE IN SECO

The setup phase in SECO includes the MPHE key generation and the preparation for 𝐅_BC. This phase involves three servers. In Table <ref> and Table <ref>, we provide the execution time and communication cost for the setup phase.







§ STRUCTURES OF NEURAL NETWORKS



  
MiniONN This is a 4 layer network with 2 convolutional and 2 fully-connected layers selected from prior work MiniONN <cit.>. The structure is provided in Table <ref>. It has around 10,500 parameters in total.


  
LeNet This network, proposed by LeCun et al. <cit.> contains 2 convolutional layers and 2 fully connected layers with about 431K parameters. Table <ref> shows the structure.









§ PSEUDOCODE






















































 























