In short
Each sample’s API calls are hashed into 262,144 features. A Random Forest is trained on the training fold and its output is turned into a few values per sample, either as grouped votes or as a leaf embedding.
The preprocessing stage involves hashing each sample’s API calls as single calls and pairs of calls (unigrams and bigrams), which gives features. Each sample is L2-normalised. To keep the matrix sparse, the features are standardised without centring. Finally a Random Forest with 100 trees is trained on the training fold only.
For feature reduction three different methods were evaluated:
- Top-k: selects the top features with the highest Gini importance. Top-k was dropped early in the research due to it underperforming.
- Leaf embedding: this method outputs the leaf that a sample lands in, per tree, one-hot encoded. It is then reduced down to values using TruncatedSVD.
- Grouped votes: the 100 trees are split into groups; each group’s mean becomes one value per qubit.
The grouped votes method performed the best. For training rows each group averages only the trees that did not see that row in the bootstrap sample, to prevent the training inputs from being over-confident compared to the test inputs.
The final output of this phase is values per sample for the votes method or a one-hot leaf matrix for the leaf embedding method.
Phase one of the pipeline is arguably the most important, as it is a direct implementation of this research’s hypothesis. The filter model is refitted for every cross-validation split and then applied to the split’s test fold unchanged.
Previous: pipeline · Next: Phase 2 – VQC training