Vol-3194/paper71
Wolfgang Fahl
Paper
| Paper | |
|---|---|
 edit
| |
| description | |
| id | Vol-3194/paper71 |
| wikidataid | Q117344932→Q117344932 |
| title | Risk Analysis for Unsupervised Privacy-Preserving Tools |
| pdfUrl | https://ceur-ws.org/Vol-3194/paper71.pdf |
| dblpUrl | https://dblp.org/rec/conf/sebd/CuzzocreaCH22 |
| volume | Vol-3194→Vol-3194 |
| session | → |
Paper[edit]
| Paper | |
|---|---|
| edit
| |
| description | |
| id | Vol-3194/paper71 |
| wikidataid | Q117344932→Q117344932 |
| title | Risk Analysis for Unsupervised Privacy-Preserving Tools |
| pdfUrl | https://ceur-ws.org/Vol-3194/paper71.pdf |
| dblpUrl | https://dblp.org/rec/conf/sebd/CuzzocreaCH22 |
| volume | Vol-3194→Vol-3194 |
| session | → |
Risk Analysis for Unsupervised Privacy-Preserving Tools[edit]
Risk Analysis for Unsupervised Privacy-Preserving
Tools⋆
Alfredo Cuzzocrea1,2,* , Christophe Cerisara2 and Mojtaba Hajian1
1
iDEA Lab, University of Calabria, Rende, Italy
2
LORIA, University of Lorraine, Nancy, France
Abstract
Current state-of-the-art methods dealing with robustness to inference attacks on privacy of deep neural
networks, such as the ones based on differential privacy and training loss regularization, mainly propose
approaches that try to improve the compromise between privacy guarantees and decrease in model
accuracy. We propose a new research direction that challenges this view, and that is based on novel
approximations of the training objective of deep learning models. The resulting loss offers several
important advantages with respect to both privacy and model accuracy: it may exploit unlabeled corpora,
it both regularizes the model and improves its generalization properties, and it encodes corpora into a
latent low-dimensional parametric representation that complies with Federated Learning architectures.
Arguments are detailed in the paper to support the proposed approach and its potential beneficial impact
with regard to preserving both privacy and quality of deep learning.
Keywords
Differential Privacy, Privacy Regularization, Unsupervised Risk
1. Introduction
Data is ubiquitous but, assuming that there is enough computational power, time and human
efforts, two other major issues severely limit its exploitation in machine learning:
• Data is rarely free and is always costly either to produce (e.g., writing a tweet) or capture
(e.g., compressing sensor data streams in the industry – e.g., [1] ). So stakeholders who
have invested time or money in this process own rights over the data.
• Information in data may be harmful to specific people or groups of people, and should
not be openly accessible.
These two issues, namely copyrights and privacy, despite originating from fundamentally
divergent motivations and contexts, may be considered jointly from the technical point of view,
because they both can be addressed from similar computational approaches that restrict access
to the data. This because, as widely recognized, managing big data sources is still an annoying
SEBD’22: 30th Symposium on Advanced Database Systems, June 19–22, 2022, Tirrenia (Pisa), Italy
⋆
This research has been made in the context of the Excellence Chair in Computer Engineering at LORIA, University
of Lorraine, Nancy, France.
*
Corresponding author.
$ alfredo.cuzzocrea@unical.it (A. Cuzzocrea); cerisara@loria.fr (C. Cerisara); klenac@riteh.hr (M. Hajian)
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
�problem (e.g., [2, 3]). Still, training large models on shared data pools is desirable to maximize
performances.
One widely known approach to achieve this is differential privacy (DP). However, DP suffers
from several major issues, which we review next, and we propose in the following an alternative
approach that partly addresses some of these challenges.
2. Limits of Differential Privacy
The first limitation of DP is due to the fact that noise is injected in the training process: this
noise inevitably impacts the classification or regression performances of the model. Therefore, a
compromise between quality of the model and the level of protection of private information has
to be found. Several studies report that, in practical applications, in order to reach acceptable
level of privacy, the quality of the model has to be severely degraded, which makes the model
nearly useless for the target task [4].
Another major drawback of DP is a direct consequence of the core principle of DP that aims
at preventing the model from memorizing individual samples from the training corpus. This
principle comes in contradiction with recent works [5], which prove that memorization of
singleton labels that typically occur in the long-tail distribution of labels (e.g., the long tail
of the Zipf law distribution of words frequencies in natural language), is required so that the
model may be able to generalize to infrequent sample sub-populations. This result shows
that alternative approaches to DP shall be considered to protect privacy if we want to train
high-quality models with good generalization properties.
3. Regularization for Privacy
We argue next that DP can be advantageously replaced in deep neural networks by a combination
of data protection approach, and non-destructive regularization techniques during training.
First, privacy can only be guaranteed when the data itself is not accessible to other practition-
ers than the data producers themselves. Federated Learning is currently one of the privileged
approach to protect data, as the data itself does not leave the data producer’s premises. Every
computation that requires access to this particular data, such as training a deep neural network,
is realized locally on such premises.
Second, the model itself, after or during training, shall not disclose private information.
Instead of degrading the model to achieve this goal, as DP does, we argue that the models shall
rather be modified to prevent membership inference attacks. This is of course a less strong
guarantee than the one obtained by DP, because making the model robust to a selected set
of membership inference attacks does not guarantee that, later, someone will design a novel
privacy attack to which our model may not be robust. But compared to the loss in quality
incurred by DP models, we believe that this potential threat is more acceptable, and may be
dealt with later on if it ever happens.
�3.1. Privacy Attacks and Mitigations
We focus next on blackbox membership inference attacks, which are one of the most general
and common types of privacy attacks against deep learning models.
The first family of such attacks rely on training a shadow model to mimick the behavior of
the model under attack [6]. However, training such shadow models is becoming more and more
difficult, if not impossible, given the size and cost of recent deep neural networks, especially in the
Natural Language Processing domain, such as GPT3 or GShard and its 600 billion parameters [7].
Furthermore, other studies [8] have shown that as good and sometimes even better attacks
may be achieved by simple metrics computed on the output logits of the target model. When
considering these families of attacks, a straightforward objective to mitigate them is to prevent
the outputs of the model to be different between in-training and out-of-training samples. This
can be achieved by adding regularization terms to the loss during training of the model. Such
regularization may be the standard L2-norm, or dedicated adversarial terms [9]. However,
similarly to differential privacy, such regularization terms alter the parameters search space
landscape during training and moves away the regularized optimum from the task objective,
which is classification accuracy. Consequently, this may also result in a decrease in performances
of the trained model.
3.2. On Regularization
Our claim that, conversely to differential privacy, regularization approaches shall not inevitably
lead to a decrease in the accuracy of the trained model, and so regularization constitutes a better
option to investigate than DP to maximize both privacy and accuracy.
The loss function that is optimized during training is composed of two terms: the main error
loss, which usually minimizes the empirical risk, and the regularization term, which commonly
minimizes the model’s parameters complexity. Minimizing the empirical risk with the main
error loss makes the model overfits to the training dataset, which negatively impacts both its
generalization capabilities and its robustness to membership inference attacks. Therefore, a
regularization term, such as the L2-norm, is used to counterbalance such negative consequences.
By smoothing the parameters search space, this regularization term reduces overfitting, which
improves generalization as well as robustness to membership inference attacks. But regulariza-
tion may also have a negative impact on the model accuracy, because it commonly only depends
on the values of the model’s parameters, and not on the task-specific evidence. Therefore, a
compromise has classically to be found between the respective weights of both terms in the
total loss.
Our proposal in this paper rather aims at designing a better regularization term that would
both prevent overfitting and optimize the classification risk. We believe an interesting research
direction towards this goal might be to give up the standard empirical risk approximation, as it
is done in [10]. We briefly describe the underlying principle next and how it could be applied to
mitigate membership inference attacks without impacting the model accuracy.
�3.3. Unsupervised Risk Approximation
Let us consider without loss of generality a binary classifier that is trained with the hinge loss;
our objective is to minimize the error that the classifier makes on unknown test data: this
objective is formalized with the classification risk 𝑅(𝜃):
[︁(︀ )︀ ]︁
𝑅(𝜃) = 𝐸𝑝(𝑥,𝑦) 1 − 𝑓 (𝑥) · (2𝑦 − 1) + (1)
∫︁
= 𝑃 (𝑦 = 0) 𝑝(𝑓 (𝑥) = 𝛼|𝑦 = 0)(1 + 𝛼)+ 𝑑𝛼 +
∫︁
𝑃 (𝑦 = 1) 𝑝(𝑓 (𝑥) = 𝛼|𝑦 = 1)(1 − 𝛼)+ 𝑑𝛼
where 𝑥 are the observations, 𝑦 the true class (𝑦 is unknown, because we consider here unsuper-
vised training) and 𝑓 (𝑥) is the scalar output score for observation 𝑥 of a deep neural network
parameterized by 𝜃. Class 0 (resp. class 1) is chosen when 𝑓 (𝑥) is negative (resp. positive). In
the first equation, the expected value of the hinge loss is computed over the full continuous
data distribution 𝑝(𝑥, 𝑦), including any unknown test corpus that will be created in the future.
Usually, this unknown distribution 𝑝(𝑥, 𝑦) is approximated by a finite labeled corpus, which
leads to the classical supervised training algorithm with empirical risk minimization. We do not
consider such an approximation here, because it requires to know the gold labels 𝑦, and because
it is the root cause of overfitting. We rather follow two assumptions proposed in [11], which
state that the prior 𝑃 (𝑦) is known and that the class-conditional distribution of the output score
𝑝(𝑓 (𝑥)|𝑦) is Gaussian. We will discuss next some conditions proposed in [10] to fulfill these
assumptions. But for now, these assumptions allow us to derive Equation-1 into the following
closed-form equation of the risk:
(︃ (︂ )︂)︃
𝑃 (𝑦 = 0) −1 − 𝜇0
𝑅(𝜇, 𝜎) = (1 + 𝜇0 ) 1 − √ +
2 𝜎0 2
𝑃 (𝑦 = 0)𝜎02 𝑁 (−1; 𝜇0 , 𝜎0 ) + (2)
(︃ (︂ )︂)︃
𝑃 (𝑦 = 1) 1 − 𝜇1
(1 − 𝜇1 ) 1 + √ +
2 𝜎1 2
𝑃 (𝑦 = 1)𝜎12 𝑁 (1; 𝜇1 , 𝜎1 )
where (𝜇0 , 𝜎0 ) and (𝜇1 , 𝜎1 ) are the parameters of the Gaussians respectively associated with
class 0 and class 1.
This equation has several important properties with regard to our privacy objective:
• The Gaussian parameters 𝜇 = (𝜇0 , 𝜇1 ) and 𝜎 = (𝜎0 , 𝜎1 ) can be estimated from an
unlabeled corpus with standard Gaussian mixture estimation algorithms; the mixture
coefficient being the known prior 𝑃 (𝑦).
• (𝜇, 𝜎) depend deterministically on the model parameters 𝜃; this enables to train 𝜃 with
gradient descent and with the chain rule:
𝜕𝑅(𝜃) 𝜕𝑅(𝜃) 𝜕(𝜇, 𝜎)
= ×
𝜕𝜃 𝜕(𝜇, 𝜎) 𝜕𝜃
� The Gaussians thus act as a proxy that decouples the model parameters from the corpus:
once the gradients with respect to each Gaussian have been computed, the deep model
can be trained without any information from the corpus. This is important in the context
of distributed privacy-protecting architectures.
• Such a training process uses the unlabeled corpus of observations only to estimate 4
parameters: the 2-dimensional vectors (𝜇, 𝜎); then, the large number of parameters 𝜃 of
the deep neural network may be trained only from (𝜇, 𝜎), without any data. This makes
optimizing the risk extremely robust to overfitting.
However, this training process provably converges towards the optimum classification risk
min𝜃 𝑅(𝜃) only when both assumptions are fulfilled. The first assumption about the known
prior is not a major issue, as 𝑃 (𝑦) can often be estimated from prior knowledge in many
applications, such as the prevalence of a disease in healthcare diagnostics, and preliminary
experiments suggest that unsupervised optimization is relatively robust to small estimation
errors of 𝑃 (𝑦).
About the second assumption, it is shown in [10] that the bi-Gaussianity assumption is valid
in a neighborhood of the minimum of the empirical risk. Therefore, we suggest to not use
Equation-2 as the first risk to optimize, but rather as a regularizer that should be applied after
standard supervised training. The advantages of our regularizer, compared to the other ones, is
that it both reduces overfitting, improves generalization and optimizes the test accuracy of the
model.
3.4. Optimization Process
The proposed approach may thus be decomposed into the following stages:
• In the first stage, the deep neural network is trained classically with the supervised
empirical risk objective, which gives an initial set of parameters 𝜃. At this stage, the
accuracy of the model is good but it is sensitive to membership inference attacks.
• In the second stage, we collect an additional unsupervised corpus of data from the
application. This second corpus does not need to be labeled, which greatly reduces the
cost of the collection process, as raw unlabeled data is often readily available in many
application domains. If this is not an option, then the initial training corpus that has
been used in the first stage may also be used in stage 2, although better generalization
properties may be obtained with a larger unlabeled corpus.
• In the third stage, the model parameters are optimized without supervision by iterating
the following steps:
– Make a forward pass over the unlabeled corpus to obtain the distribution 𝑝(𝑓 (𝑥)).
– Compute the bi-Gaussian parameters (𝜇, 𝜎) from this distribution with, e.g., the
Linde-Buzo-Gray algorithm or any other related method.
– Apply one step of gradient descent to optimize 𝑅(𝜃) given (𝜇, 𝜎).
During the third step, the model parameters 𝜃 will slowly deviate from the initial minimum of
the empirical risk, which is prone to overfitting, and rather converge towards our approximation
�of the optimal true classifier risk 𝑅(𝜃), which does not depend on the finite training corpus and
is thus immune to overfitting.
Of course, the quality of the approximation of 𝑅(𝜃) by Equation-2 depends on the represen-
tativity of the second corpus collected in stage 2; but this corpus does not need to be labeled,
and can thus be much larger than the training corpus used in stage 1. Furthermore, only 4
parameters are trained on this corpus, which makes overfitting of these Gaussian parameters
nearly impossible.
In other words, the rationale of this approach is to exploit large-capacity deep neural networks
to project the observed features into a simple latent space where the class-conditional Gaus-
sianity assumption is valid. Note that quite a similar relationship between a simple Gaussian
and a complex feature space is also built in related works, such as the well-known variational
auto-encoder [12], which confirms that such a projection is achievable through neural networks
with enough capacity. Then, in this simple latent space, the corpus distribution is discarded and
replaced by the low-dimensional Gaussian mixture; this is this “replacement” step that actually
performs regularization, as all the specific details and outliers observed in the corpus are deleted.
The Gaussian mixture approximation also generalizes beyond the training corpus, and implicitly
covers domain samples that have not been seen in the training corpus. Optimizing Equation-2
with a few gradient steps then attemps to reduce the overlapping between both Gaussians,
which provably converges towards the true classifier risk. Only a few gradient steps must be
performed before the Gaussian parameters shall be re-estimated from the data in order to avoid
the Gaussian mixture to diverge from the observations.
The main challenge and most important aspect in this paradigm is to start from an initial
Gaussian mixture representation that clusters the data into the target classes of interest. This
is why we propose to first completely train the model in a supervised way, and then only
regularize it a-posteriori, instead of mixing regularization with the supervised training loss, as
it is usually done. Our preliminary experiments confirm that this is a viable strategy to fulfill
the Gaussianity assumption. Furthermore, our regularization objective does not deviate from
the classification risk optimum as other regularizers do, and so it does not need to be “guided”
by the main supervised loss and can be applied independently.
3.5. Towards Improved Privacy
Beyond improved generalization, we expect this paradigm to increase the robustness of the
model against membership inference attacks for the following reasons:
• By reducing overfitting: it has indeed been previously shown that the degree of overfit-
ting is correlated to the success of membership inference attacks and that regularizing
the model improves robustness against them.
• By reducing the dependence to the training corpus: we have seen in the previous
section that the proposed approach decouples the training process from the actual corpus
through the Gaussian mixture distribution. The model is thus actually trained without
seeing any specific training sample: it only has access to the generic Gaussian mixture
distribution. Consequently, its dependence to specific training samples shall be more and
more reduced during this process, and thus the possibility to exploit the model’s logits to
know whether a sample is in the training corpus or not also disappears.
� • By combining it with other adversarial privacy terms: the proposed approach
replaces the supervised loss by another unsupervised loss, and so we expect that it
should be compatible with other terms that may be added to the loss to improve the
model’s privacy, especially adversarial terms that prevent the model from being able
to discriminate between samples that belong to the training corpus and the others. We
believe such adversarial terms also constitute an interesting track of research towards
improved privacy.
• By combining it with Federated Learning: the proposed loss is particularly well
suited to a distributed computation framework such as Federated Learning, because of the
Gaussian mixture proxy that it uses to represent the whole training corpus, which boils
down to computing only 4 scalar parameters that are too small to encode any sensitive
private information. It should thus be possible to compute globally these four global
statistics with simple secure multiparty homomorphic operations at a reasonable cost,
while specialized deep neural networks are updated locally, but this option is still to be
investigated.
4. Conclusions
In this position paper, we have briefly analyzed the impact of regularization from the perspective
of robustness of deep neural networks to membership inference attacks, and compared it
with other standard approaches, especially differential privacy. Then, we have proposed a
novel regularization process that relies on a non-standard approximation of the classifier risk,
which gives an unsupervised loss with interesting properties with regard to generalization
and, potentially, privacy. The benefits of this loss for privacy are only conjectured so far, and
they still need to be validated experimentally. However, we have also given several arguments
that support this claim, as well as extensions of the proposed approach to combine it with
other promising research directions. Novel paradigms are required to initiate new tracks of
research and progress towards improved privacy, and this proposal departs from the main lines
of research in the domain, but is also complementary with some of them, such as adversarial
regularization. We believe it opens interesting research directions, but exploring them and
studying experimentally their properties will require time, and this is why we have opted for now
to submit the current state of our work as a position paper. The next steps will be, after having
extensively evaluated the robustness of the approach against membership attacks, to study its
combination with other adversarial regularization terms, as well as its robustness to other types
of privacy attacks, especially white-box attacks that should become more frequent as the number
of large pre-trained deep neural networks that are freely disseminated increase. Another, more
technical advantage of the proposed approach is its relatively moderated computational cost,
which results from the fact that the unsupervised loss can be fully differentiated in closed form
and that good piecewise-linear approximations may be exploited as suggested in [10]. These
questions shall also be experimentally validated in a future work. Finally, extensions of this
approach to multi-class will be required to make the approach applicable in practical cases.
However, despite such extensions being straightforward theoretically, we expect that difficult
challenges will have to be solved in practice, for instance to estimate the 𝑁 Gaussian mixtures
�that shall precisely match the target class-conditional distributions.
Another line of future work deals with the issue of extending the presented methodologies
as to deal with artificial intelligence models applied to data analytics (e.g., [13, 14, 15]).
Acknowledgments
This research has been partially supported by the French PIA project “Lorraine Université
d’Excellence", reference ANR-15-IDEX-04-LUE.
References
[1] A. Cuzzocrea, F. Furfaro, E. Masciari, D. Saccà, C. Sirangelo, Approximate query answering
on sensor network data streams, GeoSensor Networks 49 (2004).
[2] L. Bellatreche, A. Cuzzocrea, S. Benkrid, F&A: A methodology for effectively and efficiently
designing parallel relational data warehouses on heterogenous database clusters, in: Data
Warehousing and Knowledge Discovery, 12th International Conference, DAWAK 2010,
Bilbao, Spain, August/September 2010. Proceedings, volume 6263 of Lecture Notes in
Computer Science, Springer, 2010, pp. 89–104.
[3] M. Ceci, A. Cuzzocrea, D. Malerba, Effectively and efficiently supporting roll-up and
drill-down OLAP operations over continuous dimensions via hierarchical clustering, J.
Intell. Inf. Syst. 44 (2015) 309–333.
[4] B. Jayaraman, D. Evans, Evaluating differentially private machine learning in practice,
in: 28th USENIX Security Symposium (USENIX Security 19), USENIX Association, Santa
Clara, CA, 2019, pp. 1895–1912.
[5] V. Feldman, Does learning require memorization? a short tale about a long tail, in:
Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing,
STOC 2020, Association for Computing Machinery, New York, NY, USA, 2020, p. 954–959.
URL: https://doi.org/10.1145/3357713.3384290. doi:10.1145/3357713.3384290.
[6] R. Shokri, M. Stronati, C. Song, V. Shmatikov, Membership inference attacks against
machine learning models, in: 2017 IEEE Symposium on Security and Privacy, SP 2017,
San Jose, CA, USA, May 22-26, 2017, IEEE Computer Society, 2017, pp. 3–18. URL: https:
//doi.org/10.1109/SP.2017.41. doi:10.1109/SP.2017.41.
[7] D. Lepikhin, H. Lee, Y. Xu, D. Chen, O. Firat, Y. Huang, M. Krikun, N. M. Shazeer, Z. Chen,
Gshard: Scaling giant models with conditional computation and automatic sharding, ArXiv
2006.16668 (2021).
[8] A. Salem, Y. Zhang, M. Humbert, P. Berrang, M. Fritz, M. Backes, Ml-leaks: Model and
data independent membership inference attacks and defenses on machine learning models,
in: Annual Network and Distributed System Security Symposium (NDSS 2019), 2019.
[9] H. Hu, Z. Salcic, G. Dobbie, X. Zhang, Membership inference attacks on machine
learning: A survey, CoRR abs/2103.07853 (2021). URL: https://arxiv.org/abs/2103.07853.
arXiv:2103.07853.
[10] C. Cerisara, P. Caillon, G. Le Berre, Unsupervised post-tuning of deep neural networks, in:
� IJCNN, Proc. of the International Joint Conference on Neural Networks (IJCNN), United
States, 2021.
[11] K. Balasubramanian, P. Donmez, G. Lebanon, Unsupervised supervised learning II: Margin-
based classification without labels, Journal of Machine Learning Research 12 (2011)
3119–3145.
[12] D. P. Kingma, M. Welling, Auto-Encoding Variational Bayes, in: 2nd International Confer-
ence on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Confer-
ence Track Proceedings, 2014. arXiv:http://arxiv.org/abs/1312.6114v10.
[13] S. Ahn, S. V. Couture, A. Cuzzocrea, K. Dam, G. M. Grasso, C. K. Leung, K. L. McCormick,
B. H. Wodi, A fuzzy logic based machine learning tool for supporting big data business
analytics in complex artificial intelligence environments, in: 2019 IEEE International
Conference on Fuzzy Systems, FUZZ-IEEE 2019, New Orleans, LA, USA, June 23-26, 2019,
IEEE, 2019, pp. 1–6.
[14] K. J. Morris, S. D. Egan, J. L. Linsangan, C. K. Leung, A. Cuzzocrea, C. S. H. Hoi, Token-
based adaptive time-series prediction by ensembling linear and non-linear estimators:
A machine learning approach for predictive analytics on big stock data, in: 17th IEEE
International Conference on Machine Learning and Applications, ICMLA 2018, Orlando,
FL, USA, December 17-20, 2018, IEEE, 2018, pp. 1486–1491.
[15] A. A. Audu, A. Cuzzocrea, C. K. Leung, K. A. MacLeod, N. I. Ohin, N. C. Pulgar-Vidal, An
intelligent predictive analytics system for transportation analytics on open data towards
the development of a smart city, in: Complex, Intelligent, and Software Intensive Systems
- Proceedings of the 13th International Conference on Complex, Intelligent, and Software
Intensive Systems, CISIS 2019, Sydney, NSW, Australia, 3-5 July 2019, volume 993 of
Advances in Intelligent Systems and Computing, Springer, 2019, pp. 224–236.
�
Risk Analysis for Unsupervised Privacy-Preserving Tools[edit]
Risk Analysis for Unsupervised Privacy-Preserving
Tools⋆
Alfredo Cuzzocrea1,2,* , Christophe Cerisara2 and Mojtaba Hajian1
1
iDEA Lab, University of Calabria, Rende, Italy
2
LORIA, University of Lorraine, Nancy, France
Abstract
Current state-of-the-art methods dealing with robustness to inference attacks on privacy of deep neural
networks, such as the ones based on differential privacy and training loss regularization, mainly propose
approaches that try to improve the compromise between privacy guarantees and decrease in model
accuracy. We propose a new research direction that challenges this view, and that is based on novel
approximations of the training objective of deep learning models. The resulting loss offers several
important advantages with respect to both privacy and model accuracy: it may exploit unlabeled corpora,
it both regularizes the model and improves its generalization properties, and it encodes corpora into a
latent low-dimensional parametric representation that complies with Federated Learning architectures.
Arguments are detailed in the paper to support the proposed approach and its potential beneficial impact
with regard to preserving both privacy and quality of deep learning.
Keywords
Differential Privacy, Privacy Regularization, Unsupervised Risk
1. Introduction
Data is ubiquitous but, assuming that there is enough computational power, time and human
efforts, two other major issues severely limit its exploitation in machine learning:
• Data is rarely free and is always costly either to produce (e.g., writing a tweet) or capture
(e.g., compressing sensor data streams in the industry – e.g., [1] ). So stakeholders who
have invested time or money in this process own rights over the data.
• Information in data may be harmful to specific people or groups of people, and should
not be openly accessible.
These two issues, namely copyrights and privacy, despite originating from fundamentally
divergent motivations and contexts, may be considered jointly from the technical point of view,
because they both can be addressed from similar computational approaches that restrict access
to the data. This because, as widely recognized, managing big data sources is still an annoying
SEBD’22: 30th Symposium on Advanced Database Systems, June 19–22, 2022, Tirrenia (Pisa), Italy
⋆
This research has been made in the context of the Excellence Chair in Computer Engineering at LORIA, University
of Lorraine, Nancy, France.
*
Corresponding author.
$ alfredo.cuzzocrea@unical.it (A. Cuzzocrea); cerisara@loria.fr (C. Cerisara); klenac@riteh.hr (M. Hajian)
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
�problem (e.g., [2, 3]). Still, training large models on shared data pools is desirable to maximize
performances.
One widely known approach to achieve this is differential privacy (DP). However, DP suffers
from several major issues, which we review next, and we propose in the following an alternative
approach that partly addresses some of these challenges.
2. Limits of Differential Privacy
The first limitation of DP is due to the fact that noise is injected in the training process: this
noise inevitably impacts the classification or regression performances of the model. Therefore, a
compromise between quality of the model and the level of protection of private information has
to be found. Several studies report that, in practical applications, in order to reach acceptable
level of privacy, the quality of the model has to be severely degraded, which makes the model
nearly useless for the target task [4].
Another major drawback of DP is a direct consequence of the core principle of DP that aims
at preventing the model from memorizing individual samples from the training corpus. This
principle comes in contradiction with recent works [5], which prove that memorization of
singleton labels that typically occur in the long-tail distribution of labels (e.g., the long tail
of the Zipf law distribution of words frequencies in natural language), is required so that the
model may be able to generalize to infrequent sample sub-populations. This result shows
that alternative approaches to DP shall be considered to protect privacy if we want to train
high-quality models with good generalization properties.
3. Regularization for Privacy
We argue next that DP can be advantageously replaced in deep neural networks by a combination
of data protection approach, and non-destructive regularization techniques during training.
First, privacy can only be guaranteed when the data itself is not accessible to other practition-
ers than the data producers themselves. Federated Learning is currently one of the privileged
approach to protect data, as the data itself does not leave the data producer’s premises. Every
computation that requires access to this particular data, such as training a deep neural network,
is realized locally on such premises.
Second, the model itself, after or during training, shall not disclose private information.
Instead of degrading the model to achieve this goal, as DP does, we argue that the models shall
rather be modified to prevent membership inference attacks. This is of course a less strong
guarantee than the one obtained by DP, because making the model robust to a selected set
of membership inference attacks does not guarantee that, later, someone will design a novel
privacy attack to which our model may not be robust. But compared to the loss in quality
incurred by DP models, we believe that this potential threat is more acceptable, and may be
dealt with later on if it ever happens.
�3.1. Privacy Attacks and Mitigations
We focus next on blackbox membership inference attacks, which are one of the most general
and common types of privacy attacks against deep learning models.
The first family of such attacks rely on training a shadow model to mimick the behavior of
the model under attack [6]. However, training such shadow models is becoming more and more
difficult, if not impossible, given the size and cost of recent deep neural networks, especially in the
Natural Language Processing domain, such as GPT3 or GShard and its 600 billion parameters [7].
Furthermore, other studies [8] have shown that as good and sometimes even better attacks
may be achieved by simple metrics computed on the output logits of the target model. When
considering these families of attacks, a straightforward objective to mitigate them is to prevent
the outputs of the model to be different between in-training and out-of-training samples. This
can be achieved by adding regularization terms to the loss during training of the model. Such
regularization may be the standard L2-norm, or dedicated adversarial terms [9]. However,
similarly to differential privacy, such regularization terms alter the parameters search space
landscape during training and moves away the regularized optimum from the task objective,
which is classification accuracy. Consequently, this may also result in a decrease in performances
of the trained model.
3.2. On Regularization
Our claim that, conversely to differential privacy, regularization approaches shall not inevitably
lead to a decrease in the accuracy of the trained model, and so regularization constitutes a better
option to investigate than DP to maximize both privacy and accuracy.
The loss function that is optimized during training is composed of two terms: the main error
loss, which usually minimizes the empirical risk, and the regularization term, which commonly
minimizes the model’s parameters complexity. Minimizing the empirical risk with the main
error loss makes the model overfits to the training dataset, which negatively impacts both its
generalization capabilities and its robustness to membership inference attacks. Therefore, a
regularization term, such as the L2-norm, is used to counterbalance such negative consequences.
By smoothing the parameters search space, this regularization term reduces overfitting, which
improves generalization as well as robustness to membership inference attacks. But regulariza-
tion may also have a negative impact on the model accuracy, because it commonly only depends
on the values of the model’s parameters, and not on the task-specific evidence. Therefore, a
compromise has classically to be found between the respective weights of both terms in the
total loss.
Our proposal in this paper rather aims at designing a better regularization term that would
both prevent overfitting and optimize the classification risk. We believe an interesting research
direction towards this goal might be to give up the standard empirical risk approximation, as it
is done in [10]. We briefly describe the underlying principle next and how it could be applied to
mitigate membership inference attacks without impacting the model accuracy.
�3.3. Unsupervised Risk Approximation
Let us consider without loss of generality a binary classifier that is trained with the hinge loss;
our objective is to minimize the error that the classifier makes on unknown test data: this
objective is formalized with the classification risk 𝑅(𝜃):
[︁(︀ )︀ ]︁
𝑅(𝜃) = 𝐸𝑝(𝑥,𝑦) 1 − 𝑓 (𝑥) · (2𝑦 − 1) + (1)
∫︁
= 𝑃 (𝑦 = 0) 𝑝(𝑓 (𝑥) = 𝛼|𝑦 = 0)(1 + 𝛼)+ 𝑑𝛼 +
∫︁
𝑃 (𝑦 = 1) 𝑝(𝑓 (𝑥) = 𝛼|𝑦 = 1)(1 − 𝛼)+ 𝑑𝛼
where 𝑥 are the observations, 𝑦 the true class (𝑦 is unknown, because we consider here unsuper-
vised training) and 𝑓 (𝑥) is the scalar output score for observation 𝑥 of a deep neural network
parameterized by 𝜃. Class 0 (resp. class 1) is chosen when 𝑓 (𝑥) is negative (resp. positive). In
the first equation, the expected value of the hinge loss is computed over the full continuous
data distribution 𝑝(𝑥, 𝑦), including any unknown test corpus that will be created in the future.
Usually, this unknown distribution 𝑝(𝑥, 𝑦) is approximated by a finite labeled corpus, which
leads to the classical supervised training algorithm with empirical risk minimization. We do not
consider such an approximation here, because it requires to know the gold labels 𝑦, and because
it is the root cause of overfitting. We rather follow two assumptions proposed in [11], which
state that the prior 𝑃 (𝑦) is known and that the class-conditional distribution of the output score
𝑝(𝑓 (𝑥)|𝑦) is Gaussian. We will discuss next some conditions proposed in [10] to fulfill these
assumptions. But for now, these assumptions allow us to derive Equation-1 into the following
closed-form equation of the risk:
(︃ (︂ )︂)︃
𝑃 (𝑦 = 0) −1 − 𝜇0
𝑅(𝜇, 𝜎) = (1 + 𝜇0 ) 1 − √ +
2 𝜎0 2
𝑃 (𝑦 = 0)𝜎02 𝑁 (−1; 𝜇0 , 𝜎0 ) + (2)
(︃ (︂ )︂)︃
𝑃 (𝑦 = 1) 1 − 𝜇1
(1 − 𝜇1 ) 1 + √ +
2 𝜎1 2
𝑃 (𝑦 = 1)𝜎12 𝑁 (1; 𝜇1 , 𝜎1 )
where (𝜇0 , 𝜎0 ) and (𝜇1 , 𝜎1 ) are the parameters of the Gaussians respectively associated with
class 0 and class 1.
This equation has several important properties with regard to our privacy objective:
• The Gaussian parameters 𝜇 = (𝜇0 , 𝜇1 ) and 𝜎 = (𝜎0 , 𝜎1 ) can be estimated from an
unlabeled corpus with standard Gaussian mixture estimation algorithms; the mixture
coefficient being the known prior 𝑃 (𝑦).
• (𝜇, 𝜎) depend deterministically on the model parameters 𝜃; this enables to train 𝜃 with
gradient descent and with the chain rule:
𝜕𝑅(𝜃) 𝜕𝑅(𝜃) 𝜕(𝜇, 𝜎)
= ×
𝜕𝜃 𝜕(𝜇, 𝜎) 𝜕𝜃
� The Gaussians thus act as a proxy that decouples the model parameters from the corpus:
once the gradients with respect to each Gaussian have been computed, the deep model
can be trained without any information from the corpus. This is important in the context
of distributed privacy-protecting architectures.
• Such a training process uses the unlabeled corpus of observations only to estimate 4
parameters: the 2-dimensional vectors (𝜇, 𝜎); then, the large number of parameters 𝜃 of
the deep neural network may be trained only from (𝜇, 𝜎), without any data. This makes
optimizing the risk extremely robust to overfitting.
However, this training process provably converges towards the optimum classification risk
min𝜃 𝑅(𝜃) only when both assumptions are fulfilled. The first assumption about the known
prior is not a major issue, as 𝑃 (𝑦) can often be estimated from prior knowledge in many
applications, such as the prevalence of a disease in healthcare diagnostics, and preliminary
experiments suggest that unsupervised optimization is relatively robust to small estimation
errors of 𝑃 (𝑦).
About the second assumption, it is shown in [10] that the bi-Gaussianity assumption is valid
in a neighborhood of the minimum of the empirical risk. Therefore, we suggest to not use
Equation-2 as the first risk to optimize, but rather as a regularizer that should be applied after
standard supervised training. The advantages of our regularizer, compared to the other ones, is
that it both reduces overfitting, improves generalization and optimizes the test accuracy of the
model.
3.4. Optimization Process
The proposed approach may thus be decomposed into the following stages:
• In the first stage, the deep neural network is trained classically with the supervised
empirical risk objective, which gives an initial set of parameters 𝜃. At this stage, the
accuracy of the model is good but it is sensitive to membership inference attacks.
• In the second stage, we collect an additional unsupervised corpus of data from the
application. This second corpus does not need to be labeled, which greatly reduces the
cost of the collection process, as raw unlabeled data is often readily available in many
application domains. If this is not an option, then the initial training corpus that has
been used in the first stage may also be used in stage 2, although better generalization
properties may be obtained with a larger unlabeled corpus.
• In the third stage, the model parameters are optimized without supervision by iterating
the following steps:
– Make a forward pass over the unlabeled corpus to obtain the distribution 𝑝(𝑓 (𝑥)).
– Compute the bi-Gaussian parameters (𝜇, 𝜎) from this distribution with, e.g., the
Linde-Buzo-Gray algorithm or any other related method.
– Apply one step of gradient descent to optimize 𝑅(𝜃) given (𝜇, 𝜎).
During the third step, the model parameters 𝜃 will slowly deviate from the initial minimum of
the empirical risk, which is prone to overfitting, and rather converge towards our approximation
�of the optimal true classifier risk 𝑅(𝜃), which does not depend on the finite training corpus and
is thus immune to overfitting.
Of course, the quality of the approximation of 𝑅(𝜃) by Equation-2 depends on the represen-
tativity of the second corpus collected in stage 2; but this corpus does not need to be labeled,
and can thus be much larger than the training corpus used in stage 1. Furthermore, only 4
parameters are trained on this corpus, which makes overfitting of these Gaussian parameters
nearly impossible.
In other words, the rationale of this approach is to exploit large-capacity deep neural networks
to project the observed features into a simple latent space where the class-conditional Gaus-
sianity assumption is valid. Note that quite a similar relationship between a simple Gaussian
and a complex feature space is also built in related works, such as the well-known variational
auto-encoder [12], which confirms that such a projection is achievable through neural networks
with enough capacity. Then, in this simple latent space, the corpus distribution is discarded and
replaced by the low-dimensional Gaussian mixture; this is this “replacement” step that actually
performs regularization, as all the specific details and outliers observed in the corpus are deleted.
The Gaussian mixture approximation also generalizes beyond the training corpus, and implicitly
covers domain samples that have not been seen in the training corpus. Optimizing Equation-2
with a few gradient steps then attemps to reduce the overlapping between both Gaussians,
which provably converges towards the true classifier risk. Only a few gradient steps must be
performed before the Gaussian parameters shall be re-estimated from the data in order to avoid
the Gaussian mixture to diverge from the observations.
The main challenge and most important aspect in this paradigm is to start from an initial
Gaussian mixture representation that clusters the data into the target classes of interest. This
is why we propose to first completely train the model in a supervised way, and then only
regularize it a-posteriori, instead of mixing regularization with the supervised training loss, as
it is usually done. Our preliminary experiments confirm that this is a viable strategy to fulfill
the Gaussianity assumption. Furthermore, our regularization objective does not deviate from
the classification risk optimum as other regularizers do, and so it does not need to be “guided”
by the main supervised loss and can be applied independently.
3.5. Towards Improved Privacy
Beyond improved generalization, we expect this paradigm to increase the robustness of the
model against membership inference attacks for the following reasons:
• By reducing overfitting: it has indeed been previously shown that the degree of overfit-
ting is correlated to the success of membership inference attacks and that regularizing
the model improves robustness against them.
• By reducing the dependence to the training corpus: we have seen in the previous
section that the proposed approach decouples the training process from the actual corpus
through the Gaussian mixture distribution. The model is thus actually trained without
seeing any specific training sample: it only has access to the generic Gaussian mixture
distribution. Consequently, its dependence to specific training samples shall be more and
more reduced during this process, and thus the possibility to exploit the model’s logits to
know whether a sample is in the training corpus or not also disappears.
� • By combining it with other adversarial privacy terms: the proposed approach
replaces the supervised loss by another unsupervised loss, and so we expect that it
should be compatible with other terms that may be added to the loss to improve the
model’s privacy, especially adversarial terms that prevent the model from being able
to discriminate between samples that belong to the training corpus and the others. We
believe such adversarial terms also constitute an interesting track of research towards
improved privacy.
• By combining it with Federated Learning: the proposed loss is particularly well
suited to a distributed computation framework such as Federated Learning, because of the
Gaussian mixture proxy that it uses to represent the whole training corpus, which boils
down to computing only 4 scalar parameters that are too small to encode any sensitive
private information. It should thus be possible to compute globally these four global
statistics with simple secure multiparty homomorphic operations at a reasonable cost,
while specialized deep neural networks are updated locally, but this option is still to be
investigated.
4. Conclusions
In this position paper, we have briefly analyzed the impact of regularization from the perspective
of robustness of deep neural networks to membership inference attacks, and compared it
with other standard approaches, especially differential privacy. Then, we have proposed a
novel regularization process that relies on a non-standard approximation of the classifier risk,
which gives an unsupervised loss with interesting properties with regard to generalization
and, potentially, privacy. The benefits of this loss for privacy are only conjectured so far, and
they still need to be validated experimentally. However, we have also given several arguments
that support this claim, as well as extensions of the proposed approach to combine it with
other promising research directions. Novel paradigms are required to initiate new tracks of
research and progress towards improved privacy, and this proposal departs from the main lines
of research in the domain, but is also complementary with some of them, such as adversarial
regularization. We believe it opens interesting research directions, but exploring them and
studying experimentally their properties will require time, and this is why we have opted for now
to submit the current state of our work as a position paper. The next steps will be, after having
extensively evaluated the robustness of the approach against membership attacks, to study its
combination with other adversarial regularization terms, as well as its robustness to other types
of privacy attacks, especially white-box attacks that should become more frequent as the number
of large pre-trained deep neural networks that are freely disseminated increase. Another, more
technical advantage of the proposed approach is its relatively moderated computational cost,
which results from the fact that the unsupervised loss can be fully differentiated in closed form
and that good piecewise-linear approximations may be exploited as suggested in [10]. These
questions shall also be experimentally validated in a future work. Finally, extensions of this
approach to multi-class will be required to make the approach applicable in practical cases.
However, despite such extensions being straightforward theoretically, we expect that difficult
challenges will have to be solved in practice, for instance to estimate the 𝑁 Gaussian mixtures
�that shall precisely match the target class-conditional distributions.
Another line of future work deals with the issue of extending the presented methodologies
as to deal with artificial intelligence models applied to data analytics (e.g., [13, 14, 15]).
Acknowledgments
This research has been partially supported by the French PIA project “Lorraine Université
d’Excellence", reference ANR-15-IDEX-04-LUE.
References
[1] A. Cuzzocrea, F. Furfaro, E. Masciari, D. Saccà, C. Sirangelo, Approximate query answering
on sensor network data streams, GeoSensor Networks 49 (2004).
[2] L. Bellatreche, A. Cuzzocrea, S. Benkrid, F&A: A methodology for effectively and efficiently
designing parallel relational data warehouses on heterogenous database clusters, in: Data
Warehousing and Knowledge Discovery, 12th International Conference, DAWAK 2010,
Bilbao, Spain, August/September 2010. Proceedings, volume 6263 of Lecture Notes in
Computer Science, Springer, 2010, pp. 89–104.
[3] M. Ceci, A. Cuzzocrea, D. Malerba, Effectively and efficiently supporting roll-up and
drill-down OLAP operations over continuous dimensions via hierarchical clustering, J.
Intell. Inf. Syst. 44 (2015) 309–333.
[4] B. Jayaraman, D. Evans, Evaluating differentially private machine learning in practice,
in: 28th USENIX Security Symposium (USENIX Security 19), USENIX Association, Santa
Clara, CA, 2019, pp. 1895–1912.
[5] V. Feldman, Does learning require memorization? a short tale about a long tail, in:
Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing,
STOC 2020, Association for Computing Machinery, New York, NY, USA, 2020, p. 954–959.
URL: https://doi.org/10.1145/3357713.3384290. doi:10.1145/3357713.3384290.
[6] R. Shokri, M. Stronati, C. Song, V. Shmatikov, Membership inference attacks against
machine learning models, in: 2017 IEEE Symposium on Security and Privacy, SP 2017,
San Jose, CA, USA, May 22-26, 2017, IEEE Computer Society, 2017, pp. 3–18. URL: https:
//doi.org/10.1109/SP.2017.41. doi:10.1109/SP.2017.41.
[7] D. Lepikhin, H. Lee, Y. Xu, D. Chen, O. Firat, Y. Huang, M. Krikun, N. M. Shazeer, Z. Chen,
Gshard: Scaling giant models with conditional computation and automatic sharding, ArXiv
2006.16668 (2021).
[8] A. Salem, Y. Zhang, M. Humbert, P. Berrang, M. Fritz, M. Backes, Ml-leaks: Model and
data independent membership inference attacks and defenses on machine learning models,
in: Annual Network and Distributed System Security Symposium (NDSS 2019), 2019.
[9] H. Hu, Z. Salcic, G. Dobbie, X. Zhang, Membership inference attacks on machine
learning: A survey, CoRR abs/2103.07853 (2021). URL: https://arxiv.org/abs/2103.07853.
arXiv:2103.07853.
[10] C. Cerisara, P. Caillon, G. Le Berre, Unsupervised post-tuning of deep neural networks, in:
� IJCNN, Proc. of the International Joint Conference on Neural Networks (IJCNN), United
States, 2021.
[11] K. Balasubramanian, P. Donmez, G. Lebanon, Unsupervised supervised learning II: Margin-
based classification without labels, Journal of Machine Learning Research 12 (2011)
3119–3145.
[12] D. P. Kingma, M. Welling, Auto-Encoding Variational Bayes, in: 2nd International Confer-
ence on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Confer-
ence Track Proceedings, 2014. arXiv:http://arxiv.org/abs/1312.6114v10.
[13] S. Ahn, S. V. Couture, A. Cuzzocrea, K. Dam, G. M. Grasso, C. K. Leung, K. L. McCormick,
B. H. Wodi, A fuzzy logic based machine learning tool for supporting big data business
analytics in complex artificial intelligence environments, in: 2019 IEEE International
Conference on Fuzzy Systems, FUZZ-IEEE 2019, New Orleans, LA, USA, June 23-26, 2019,
IEEE, 2019, pp. 1–6.
[14] K. J. Morris, S. D. Egan, J. L. Linsangan, C. K. Leung, A. Cuzzocrea, C. S. H. Hoi, Token-
based adaptive time-series prediction by ensembling linear and non-linear estimators:
A machine learning approach for predictive analytics on big stock data, in: 17th IEEE
International Conference on Machine Learning and Applications, ICMLA 2018, Orlando,
FL, USA, December 17-20, 2018, IEEE, 2018, pp. 1486–1491.
[15] A. A. Audu, A. Cuzzocrea, C. K. Leung, K. A. MacLeod, N. I. Ohin, N. C. Pulgar-Vidal, An
intelligent predictive analytics system for transportation analytics on open data towards
the development of a smart city, in: Complex, Intelligent, and Software Intensive Systems
- Proceedings of the 13th International Conference on Complex, Intelligent, and Software
Intensive Systems, CISIS 2019, Sydney, NSW, Australia, 3-5 July 2019, volume 993 of
Advances in Intelligent Systems and Computing, Springer, 2019, pp. 224–236.
�