Geometric Mixture Classifier A Discriminative Per-Class Mixture of Hyperplanes for Fast, Transparent Classification

Many real-world categories are multimodal, with a single class occupying several disjoint regions of feature space. Classical linear models such as logistic regression or linear SVMs impose a single global hyperplane and therefore fail on such data, while kernel SVMs and deep networks can capture multimodality but often trade off interpretability, require extensive tuning, or incur high computational costs. We introduce the Geometric Mixture Classifier (GMC), a discriminative model that represents each class as a mixture of hyperplanes. Within a class, GMC combines plane scores using a temperature-controlled soft-OR (log-sum-exp), smoothly approximating the maximum; across classes, it applies a standard softmax to yield probabilistic posteriors. An optional Random Fourier Features (RFF) mapping equips GMC with nonlinear capacity while preserving linear inference in the number of planes and lifted dimensions. To make GMC practical, we develop a training recipe including geometry-aware ini tialization via k-means, automatic plane budgeting with silhouette score, alpha-annealing, usage-aware L2 regularization, label smoothing, and early stopping. Experiments on syn thetic multimodal benchmarks (moons, circles, anisotropic blobs, two-spirals) and real tabular/image datasets (iris, wine, WDBC breast cancer, digits) show that GMC consis tently outperforms linear baselines and k-NN, matches or exceeds RBF-SVM, Random Forests, and compact MLPs, and enables transparent geometric introspection via plane and class-level responsibility visualizations. Because inference scales linearly in the number of planes and features, GMC is CPU efficient, requiring only microseconds per example—comparable to or faster than RBF-SVM and compact MLPs. With post-hoc temperature scaling, calibration improves (ECE reduced from 0.06 to 0.02). GMC thus offers a favorable trade-off between accuracy, interpretability, and efficiency: more expressive than linear models, and lighter and more transparent than kernel or deep models.