Publications HAL

2024

Preprints, Working Papers, ...

titre
Privately Learning Smooth Distributions on the Hypercube by Projections
auteur
Clément Lalanne, Sébastien Gadat
article
2024
resume
Fueled by the ever-increasing need for statistics that guarantee the privacy of their training sets, this article studies the centrally-private estimation of Sobolev-smooth densities of probability over the hypercube in dimension d. The contributions of this article are two-fold : Firstly, it generalizes the one dimensional results of (Lalanne et al., 2023) to non-integer levels of smoothness and to a high-dimensional setting, which is important for two reasons : it is more suited for modern learning tasks, and it allows understanding the relations between privacy, dimensionality and smoothness, which is a central question with differential privacy. Secondly, this article presents a private strategy of estimation that is data-driven (usually referred to as adaptive in Statistics) in order to privately choose an estimator that achieves a good bias-variance trade-off among a finite family of private projection estimators without prior knowledge of the ground-truth smoothness β. This is achieved by adapting the Lepskii method for private selection, by adding a new penalization term that makes the estimation privacy-aware.
typdoc
Preprints, Working Papers, ...
Accès au texte intégral et bibtex
https://hal.science/hal-04549279/file/wp_tse_1505.pdf BibTex

2023

Journal articles

titre
About the Cost of Central Privacy in Density Estimation
auteur
Clément Lalanne, Aurélien Garivier, Rémi Gribonval
article
Transactions on Machine Learning Research Journal, 2023
resume
We study non-parametric density estimation for densities in Lipschitz and Sobolev spaces, and under central privacy. In particular, we investigate regimes where the privacy budget is not supposed to be constant. We consider the classical definition of central differential privacy, but also the more recent notion of central concentrated differential privacy. We recover the result of Barber & Duchi (2014) stating that histogram estimators are optimal against Lipschitz distributions for the L2 risk, and under regular differential privacy, and we extend it to other norms and notions of privacy. Then, we investigate higher degrees of smoothness, drawing two conclusions: First, and contrary to what happens with constant privacy budget (Wasserman & Zhou, 2010), there are regimes where imposing privacy degrades the regular minimax risk of estimation on Sobolev densities. Second, so-called projection estimators are near-optimal against the same classes of densities in this new setup with pure differential privacy, but contrary to the constant privacy budget case, it comes at the cost of relaxation. With zero concentrated differential privacy, there is no need for relaxation, and we prove that the estimation is optimal.
typdoc
Journal articles
Accès au texte intégral et bibtex
https://hal.science/hal-04133655/file/preprint.pdf BibTex
titre
On the Statistical Complexity of Estimation and Testing under Privacy Constraints
auteur
Clément Lalanne, Aurélien Garivier, Rémi Gribonval
article
Transactions on Machine Learning Research Journal, 2023
resume
The challenge of producing accurate statistics while respecting the privacy of the individuals in a sample is an important area of research. We study minimax lower bounds for classes of differentially private estimators. In particular, we show how to characterize the power of a statistical test under differential privacy in a plug-and-play fashion by solving an appropriate transport problem. With specific coupling constructions, this observation allows us to derive Le Cam-type and Fano-type inequalities not only for regular definitions of differential privacy but also for those based on Renyi divergence. We then proceed to illustrate our results on three simple, fully worked out examples. In particular, we show that the problem class has a huge importance on the provable degradation of utility due to privacy. In certain scenarios, we show that maintaining privacy results in a noticeable reduction in performance only when the level of privacy protection is very high. Conversely, for other problems, even a modest level of privacy protection can lead to a significant decrease in performance. Finally, we demonstrate that the DP-SGLD algorithm, a private convex solver, can be employed for maximum likelihood estimation with a high degree of confidence, as it provides near-optimal results with respect to both the size of the sample and the level of privacy protection. This algorithm is applicable to a broad range of parametric estimation procedures, including exponential families.
typdoc
Journal articles
Accès au texte intégral et bibtex
https://hal.science/hal-03794374/file/tmlr_final.pdf BibTex
titre
Private Quantiles Estimation in the Presence of Atoms
auteur
Clément Lalanne, Clément Gastaud, Nicolas Grislain, Aurélien Garivier, Rémi Gribonval
article
Information and Inference, 2023, ⟨10.1093/imaiai/iaad030⟩
resume
We consider the differentially private estimation of multiple quantiles (MQ) of a distribution from a dataset, a key building block in modern data analysis. We apply the recent non-smoothed Inverse Sensitivity (IS) mechanism to this specific problem. We establish that the resulting method is closely related to the recently published ad hoc algorithm JointExp. In particular, they share the same computational complexity and a similar efficiency. We prove the statistical consistency of these two algorithms for continuous distributions. Furthermore, we demonstrate both theoretically and empirically that this method suffers from an important lack of performance in the case of peaked distributions, which can degrade up to a potentially catastrophic impact in the presence of atoms. Its smoothed version (i.e. by applying a max kernel to its output density) would solve this problem, but remains an open challenge to implement. As a proxy, we propose a simple and numerically efficient method called Heuristically Smoothed JointExp (HSJointExp), which is endowed with performance guarantees for a broad class of distributions and achieves results that are orders of magnitude better on problematic datasets.
typdoc
Journal articles
DOI
DOI : 10.1093/imaiai/iaad030
Accès au texte intégral et bibtex
https://hal.science/hal-03572701/file/preprint.pdf BibTex

Conference papers

titre
Can sparsity improve the privacy of neural networks?
auteur
Antoine Gonon, Léon Zheng, Clément Lalanne, Quoc-Tung Le, Guillaume Lauga, Can Pouliquen
article
GRETSI 2023 - XXIXème Colloque Francophone de Traitement du Signal et des Images, Aug 2023, Grenoble, France
resume
Sparse neural networks are mainly motivated by ressource efficiency since they use fewer parameters than their dense counterparts but still reach comparable accuracies. This article empirically investigates whether sparsity could also improve the privacy of the data used to train the networks. The experiments show positive correlations between the sparsity of the model, its privacy, and its classification error. Simply comparing the privacy of two models with different sparsity levels can yield misleading conclusions on the role of sparsity, because of the additional correlation with the classification error. From this perspective, some caveats are raised about previous works that investigate sparsity and privacy.
typdoc
Conference papers
Accès au texte intégral et bibtex
https://hal.science/hal-04062317/file/HAL_gretsi.pdf BibTex
titre
Private Statistical Estimation of Many Quantiles
auteur
Clément Lalanne, Aurélien Garivier, Rémi Gribonval
article
ICML 2023 - 40th International Conference on Machine Learning, Jul 2023, Honolulu, United States
resume
This work studies the estimation of many statistical quantiles under differential privacy. More precisely, given a distribution and access to i.i.d. samples from it, we study the estimation of the inverse of its cumulative distribution function (the quantile function) at specific points. For instance, this task is of key importance in private data generation. We present two different approaches. The first one consists in privately estimating the empirical quantiles of the samples and using this result as an estimator of the quantiles of the distribution. In particular, we study the statistical properties of the recently published algorithm introduced by Kaplan et al. 2022 that privately estimates the quantiles recursively. The second approach is to use techniques of density estimation in order to uniformly estimate the quantile function on an interval. In particular, we show that there is a tradeoff between the two methods. When we want to estimate many quantiles, it is better to estimate the density rather than estimating the quantile function at specific points.
typdoc
Conference papers
Accès au texte intégral et bibtex
https://hal.science/hal-03986170/file/camera.pdf BibTex

Theses

titre
On the tradeoffs of statistical learning with privacy
auteur
Clément Lalanne
article
Machine Learning [cs.LG]. ENS Lyon, 2023. English. ⟨NNT : 2023ENSL0068⟩
resume
This thesis studies the tradeoffs between statistical learning and privacy. On the one hand, learning, which is defined as estimating meaningful quantities or trends at the scale of a population with only access to sampled observations of that population, will be easier if granted unrestricted access to its training data. On the other hand, the training data can be sensitive, and its unrestricted use can lead to privacy issues. Since the spectrum of security and privacy issues can be very large, it is necessary to specify the range of this thesis. In the considered setup, the data is aggregated by a single actor that uses it to train a statistical model (estimation procedure, neural network, . . . ). This model is then shared to the entire world. The problem that is considered is the inversion one : is it possible to break the privacy of the individual samples of the training data by the sole observation of the trained model ? This thesis studies multiple statistical learning problems when the estimator is constrained to satisfy differential privacy.
typdoc
Theses
Accès au texte intégral et bibtex
https://hal.science/tel-04431813/file/PhD_Thesis___On_the_tradeoffs_of_statistical_learning_with_privacy-10.pdf BibTex

2020

Conference papers

titre
Extraction of Nystagmus Patterns from Eye-Tracker Data with Convolutional Sparse Coding
auteur
Clément Lalanne, Maxence Rateaux, Laurent Oudre, Matthieu Robert, Thomas Moreau
article
EMBC 2020 - 42nd Annual International Conferences of the IEEE Engineering in Medicine and Biology Society in conjunction with the 43rd Annual Conference of the Canadian Medical and Biological Engineering Society, Jul 2020, Montreal / Virtuel, Canada. pp.928-931, ⟨10.1109/EMBC44109.2020.9175621⟩
resume
The analysis of the Nystagmus waveforms from eye-tracking records is crucial for the clinicial interpretation of this pathological movement. A major issue to automatize this analysis is the presence of natural eye movements and eye blink artefacts that are mixed with the signal of interest. We propose a method based on Convolutional Dictionary Learning that is able to automaticcaly highlight the Nystagmus waveforms, separating the natural motion from the pathological movements. We show on simulated signals that our method can indeed improve the pattern recovery rate and provide clinical examples to illustrate how this algorithm performs.
typdoc
Conference papers
DOI
DOI : 10.1109/EMBC44109.2020.9175621
Accès au texte intégral et bibtex
https://hal.science/hal-03022547/file/root.pdf BibTex