With the proliferation of training data, distributed machine learning (DML) is becoming more competent for large-scale learning tasks. However, privacy concerns have to be given priority in DML, since training data may contain sensitive information of users. In this paper, we propose a privacy-preserving ADMM-based DML framework with two novel features: First, we remove the assumption commonly made in the literature that the users trust the server collecting their data. Second, the framework provides heterogeneous privacy for users depending on data's sensitive levels and servers' trust degrees. The challenging issue is to keep the accumulation of privacy losses over ADMM iterations minimal. In the proposed framework, a local randomization approach, which is differentially private, is adopted to provide users with self-controlled privacy guarantee for the most sensitive information. Further, the ADMM algorithm is perturbed through a combined noise-adding method, which simultaneously preserves privacy for users' less sensitive information and strengthens the privacy protection of the most sensitive information. We provide detailed analyses on the performance of the trained model according to its generalization error. Finally, we conduct extensive experiments using real-world datasets to validate the theoretical results and evaluate the classification performance of the proposed framework.
@cite_30: 
@cite_48: We present a fully homomorphic encryption scheme that is based solely on the(standard) learning with errors (LWE) assumption. Applying known results on LWE, the security of our scheme is based on the worst-case hardness of short vector problems'' on arbitrary lattices. Our construction improves on previous works in two aspects: We show that somewhat homomorphic'' encryption can be based on LWE, using a new re-linearization technique. In contrast, all previous schemes relied on complexity assumptions related to ideals in various rings. We deviate from the "squashing paradigm'' used in all previous works. We introduce a new dimension-modulus reduction technique, which shortens the cipher texts and reduces the decryption complexity of our scheme, without introducing additional assumptions . Our scheme has very short cipher texts and we therefore use it to construct an asymptotically efficient LWE-based single-server private information retrieval (PIR) protocol. The communication complexity of our protocol (in the public-key model) is @math bits per single-bit query (here, @math is a security parameter).
@cite_21: Collaborative computing uses multiple data servers to jointly complete data analysis, e.g., statistical analysis and inference. One major obstruction for it lies in privacy concern, which is directly associated with nodes’ participation and the fidelity of received data. Existing privacy-preserving paradigms for cloud computing and distributed data aggregation only provide nodes with homogeneous privacy protection without consideration of nodes’ diverse trust degrees to different data servers. We propose a two-phase framework that computes the average value while preserving heterogeneous privacy for nodes’ private data. The new challenge is that in the premise of meeting privacy requirements, we should guarantee the proposed framework has the same computation accuracy with existing privacy-aware solutions. In this paper, nodes obtain heterogeneous privacy protection in the face of different data servers through one-shot noise perturbation. Based on the definition of KL privacy, we derive the analytical expressions of the privacy preserving degrees (PPDs) and quantify the relation between different PPDs. Then, we obtain the closed-form expression of computation accuracy. Furthermore, an efficient incentive mechanism is proposed to achieve optimized computation accuracy when data servers have fixed budgets. Finally, extensive simulations are conducted to verify the obtained theoretical results.
@cite_1: Over the past five years a new approach to privacy-preserving data analysis has born fruit [13, 18, 7, 19, 5, 37, 35, 8, 32]. This approach differs from much (but not all!) of the related literature in the statistics, databases, theory, and cryptography communities, in that a formal and ad omnia privacy guarantee is defined, and the data analysis techniques presented are rigorously proved to satisfy the guarantee. The key privacy guarantee that has emerged is differential privacy. Roughly speaking, this ensures that (almost, and quantifiably) no risk is incurred by joining a statistical database. In this survey, we recall the definition of differential privacy and two basic techniques for achieving it. We then show some interesting applications of these techniques, presenting algorithms for three specific tasks and three general results on differentially private learning.
@cite_6: DES, the Data Encryption Standard, is one of several cryptographic standards. The authors of this text detail their cryptanalytic "attack" upon DES and several other systems, using creative and novel tactics to demonstrate how they broke DES up into 16 rounds of coding. The methodology used offers valuable insights to cryptographers and cryptanalysts alike in creating new encryption standards, strengthening current ones, and exploring new ways to test important data protection schemes. This book introduces a new cryptographic method, called differential cryptanalysis, which can be applied to analyze cryptosystems. It describes the cryptanalysis of DES, deals with the influence of its building blocks on security, and analyzes modified variants. The differential cryptanalysis of "Feal" and several other cryptosystems is also described. This method can also be used to cryptanalyze hash functions, as is exemplified by the cryptanalysis of "Snefru".
@cite_0: With the wide deployment of public cloud computing infrastructures, using clouds to host data query services has become an appealing solution for the advantages on scalability and cost-saving. However, some data might be sensitive that the data owner does not want to move to the cloud unless the data confidentiality and query privacy are guaranteed. On the other hand, a secured query service should still provide efficient query processing and significantly reduce the in-house workload to fully realize the benefits of cloud computing. We propose the random space perturbation (RASP) data perturbation method to provide secure and efficient range query and kNN query services for protected data in the cloud. The RASP data perturbation method combines order preserving encryption, dimensionality expansion, random noise injection, and random projection, to provide strong resilience to attacks on the perturbed data and queries. It also preserves multidimensional ranges, which allows existing indexing techniques to be applied to speedup range query processing. The kNN-R algorithm is designed to work with the RASP range query algorithm to process the kNN queries. We have carefully analyzed the attacks on data and queries under a precisely defined threat model and realistic security assumptions. Extensive experiments have been conducted to show the advantages of this approach on efficiency and security.
@cite_19: We consider a problem where multiple agents participate in solving a quadratic optimization problem subject to linear inequality constraints in a privacy-preserving manner. Several variables of the objective function as well as the constraints are privacy-sensitive and are known to different agents. We propose a privacy-preserving protocol based on partially homomorphic encryption where each agent encrypts its own information before sending it to an untrusted cloud computing infrastructure. To find the optimal solution the cloud applies a gradient descent algorithm on the encrypted data without the ability to decrypt it. The privacy of the proposed protocol against coalitions of colluding agents is analyzed using the cryptography notion of zero knowledge proofs.
@cite_45: 
@cite_2: Randomized Aggregatable Privacy-Preserving Ordinal Response, or RAPPOR, is a technology for crowdsourcing statistics from end-user client software, anonymously, with strong privacy guarantees. In short, RAPPORs allow the forest of client data to be studied, without permitting the possibility of looking at individual trees. By applying randomized response in a novel manner, RAPPOR provides the mechanisms for such collection as well as for efficient, high-utility analysis of the collected data. In particular, RAPPOR permits statistics to be collected on the population of client-side strings with strong privacy guarantees for each client, and without linkability of their reports. This paper describes and motivates RAPPOR, details its differential-privacy and utility guarantees, discusses its practical deployment and properties in the face of different attack models, and, finally, gives results of its application to both synthetic and real-world data.
@cite_17: 
For privacy-preserving problems, cryptographic techniques @cite_6 @cite_30 @cite_17 are often used to protect the encrypted information not to be inferred when the key is unknown. In particular, homomorphic encryption methods @cite_48 , @cite_19 allow untrustworthy servers to calculate with encrypted data, and this approach has been applied in an ADMM scheme @cite_45 . Nevertheless, such schemes unavoidably bring extra computation and communication overheads, which may not be suitable for large-scale deployment. Another commonly used approach to preserve privacy is random value perturbation @cite_1 , @cite_2 , @cite_0 . DP has been increasingly acknowledged as the de facto criterion for non-encryption-based data privacy. This approach requires less costs but still provides strong privacy guarantee, though there exists a tradeoff between privacy and performance @cite_21 .