A generative model for dimensionality reduction with millions of features and few samples
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Motivation
In this paper, we demonstrate that it is feasible to train a deep generative model for dimensionality reduction with millions of features using few samples, which makes this type of generative model a more versatile alternative to standard methods for dimensionality reduction. Specifically, we hypothesize that for a decoder-only model, the number of training samples required is almost independent of the feature dimensionality in most network architectures.
Results
Through an extensive set of experiments on synthetic non-linear data, we validate this hypothesis. We also train the model on a downsampled version of the 1000 Genomes Project (1KGP) dataset to further assess its behavior under controlled reductions in sample size. Furthermore, we train a deep generative decoder (DGD) on a curated dataset from the International Cancer Genome Consortium (ICGC), which contains 4.4 million features. It is trained on approximately 4,000 samples and tested on 1,000 samples. The resulting latent representation exhibits clear clustering, and when methods are reduced to the same number of dimensions, it outperforms PCA and VAE for tumor type classification. Additionally, the DGD is computationally efficient and can be trained on a 16GB GPU.
Availability and implementation
Code is available at https://github.com/cpancott/ReceptiveDGD .
Contact
corrado.pancotti@helmholtz-munich.de ; akrogh@di.ku.dk
Supplementary information
Supplementary data are available with this preprint.
