In short
PCA keeps whatever varies most in the data, whether or not it helps tell malware apart. A Random Forest is trained to tell malware apart, so its output keeps what separates the two classes. Feeding that to the VQC instead of PCA’s output is the whole idea of this research.
PCA: keep the most variance
PCA is an unsupervised linear transformation algorithm that produces new features called principal components, by determining the maximum variance of the data (Anowar et al., 2021). PCA is used because it keeps the directions of most variance, but it does not take into account which features matter for classification.
A Random Forest: learn what matters first
A Random Forest is a classical model made of many decision trees, each voting on whether a sample is malware.
Classical machine learning models use feature selection to improve performance and training times, and have demonstrated a strong capability in feature selection and feature importance estimation. However, there was no existing work investigating the use of classical machine learning models as a feature selection tool for the purpose of reducing the dimensionality of a dataset prior to quantum data encoding for malware classification.
This research used the Random Forest’s output, and not a selection of its input features, as the VQC’s input, in two ways:
- Grouped votes: the 100 trees are split into one group per qubit; each group’s mean probability of malware becomes one value per qubit.
- Leaf embedding: the leaf that a sample lands in, per tree, is one-hot encoded and then reduced down to one value per qubit with TruncatedSVD.
The result
Both Random Forest routes beat PCA at every qubit count tested, on every one of the 50 test splits. See the results, or the pipeline for exactly how each route works.