An Efficient Differentially-Private Weighted Support Vector Machine Algorithm with Noisy Gradient Descent

The proliferation of digital data has led to significant opportunities for analysis and strategic initiatives, where sensitive personal details is being utilized. Unfortunately, this has led to growing privacy concerns over time. Differential privacy (DP) is the most widely used framework to resolve the aforementioned privacy issues. The DP guarantee can be achieved by incorporating randomness into the algorithm pipeline. However, generalized differentially private empirical risk minimization (DP-ERM) algorithms typically employ the output perturbation or objective perturbation approaches, which do not perform well when the ambient dimension of data is large. Additionally, weighted support vector machine (wSVM), which has unique applications in individualized treatment rules and outcome-weighted learning (OWL) suffers from low utility if output perturbation or objective perturbation is employed. To address these challenges, a differentially-private wSVM algorithm has been developed in this work that incorporates gradient perturbation into the optimization process of wSVM. The performance and the privacy-utility tradeoff of the proposed approach are evaluated and compared with that of the existing DP-wSVM algorithm on three publicly available real datasets. The proposed algorithm is shown to ensure an accurate and efficient classification performance in pharmacogenetics and OWL applications with a more balanced tradeoff between privacy and utility, irrespective of ambient data dimension. Additionally, a theoretical analysis of the privacy guarantee of the proposed algorithm has been performed, which paves the way of a tight accounting of the overall privacy budget using Rényi Differential Privacy. The proposed algorithm achieves a close approximation of the performance of the non-private algorithm, while ensuring strict privacy. Last but not the least, the proposed algorithm achieves better results compared to the existing differentially private wSVM approach.