Support Vector Machine Series Part 2: Benchmark and Error Forensics on UCI HAR

Introduction

Ninety-six percent accuracy. Impressive headline; useless diagnostic. Aggregate metrics conceal exactly the failure modes that matter in deployment: which classes bleed into each other, where the decision boundary wobbles under minor perturbation, and whether your geometry is genuinely separating activities or merely memorising sensor artefacts.

This article strips the UCI Human Activity Recognition dataset [10] down to class-level forensics. Precision, recall, F1, confusion corridors, tuning stability, PCA geometry signals: all compared across an RBF SVM pipeline and a Random Forest baseline. The question is deliberately narrow. What do these numbers actually reveal once you stop celebrating the aggregate?

This article is not legal advice.

If you have not read the conceptual foundation, start with Part 1: Margins, Kernels, and Core Algorithms. For system-level reliability analogies in resource contention, see deadlock and resource contention lessons.

Terminology

Benchmark evaluation

A controlled experimental comparison of model performance on a standardised dataset, using agreed metrics and reproducible splits to enable fair cross-method assessment.

Confusion matrix

A table that cross-tabulates predicted class labels against true class labels, revealing per-class error patterns such as false positives and false negatives.

Error analysis

The systematic inspection of misclassified instances to identify structured failure patterns, boundary weaknesses, and class-pair confusion corridors.

Human activity recognition (HAR)

A classification task that infers physical activities such as walking, sitting, or standing from sensor data, commonly benchmarked using the UCI HAR smartphone accelerometer dataset.

Cross-validation

A resampling procedure that partitions data into complementary training and validation subsets across multiple folds, providing a more robust estimate of model generalisation than a single train-test split.

Benchmark Setup and Reproducibility Boundaries

The benchmark design used:

1. Subject-compatible train/test split consistent with HAR conventions.
2. Feature scaling fit on training data and applied unchanged to test data.
3. RBF SVM with one-vs-all decomposition and grid search over $C$ and $\gamma$.
4. Random Forest baseline tuned under comparable CV protocol.

R and Python implementations were aligned by hyperparameter grid and fold logic to check language-level consistency ([7]; [8]; [9]; [13]).

Scope note: This is a single-dataset, single-task benchmark. It supports careful operational inference for sensor-based activity classification, not universal model ranking.

Aggregate Metrics: Useful but Insufficient

Top-level outcomes from this run:

RF looks stronger. Every aggregate metric favours it in this run: accuracy, macro F1, kappa. But the conclusion is incomplete, because operational risk does not live in averages; it concentrates in specific class transitions that aggregates actively obscure.

Class-Level Diagnostics: Where SVM Holds and Slips

Selected class profile:

• LAYING: near-ceiling SVM behavior.
• SITTING and STANDING: dominant static-class confusion corridor.
• WALKING-related subclasses: overlap-driven dynamic-class spillovers.

SVM class asymmetry in this run tells a specific story. SITTING recall is high while precision drops: the classifier over-assigns this label, pulling in STANDING instances that share nearly identical accelerometer profiles. STANDING shows the inverse; precision holds but recall erodes, pointing to conservative boundary placement that sacrifices coverage for purity. Meanwhile, WALKING_UPSTAIRS and WALKING_DOWNSTAIRS bleed into each other bidirectionally. Not random scatter. Structured spillover.

This pattern is coherent with kernel locality behaviour and neighbourhood overlap in transformed space ([11]; [12]; [9]).

Error Corridors, Not Random Noise

Largest SVM confusion transitions in this run:

1. STANDING -> SITTING.
2. WALKING -> WALKING_DOWNSTAIRS.
3. WALKING_UPSTAIRS -> WALKING.

RF reduces the same corridors. That is telling: the ambiguity zones are shared, but tree-based partitioning suppresses them more aggressively in this particular feature geometry.

So what do you actually track? Pairwise transition counts. In any production dashboard worth its screen real estate, corridor-specific drift metrics will catch degradation long before a global F1 tick reveals the problem.

Error Topology and Boundary Geometry

The confusion pattern is geometric, not stochastic. LAYING? Trivially separable. The errors cluster where they should: posture-adjacent pairs (SITTING/STANDING) and stair-adjacent pairs (UPSTAIRS/DOWNSTAIRS), exactly the regions where inertial signal overlap is densest. Margin methods perform best when local neighbourhoods retain separability under the selected kernel width [1] [2].

Class-pair topology deserves first-class status in any evaluation artefact. Static posture pairs (SITTING vs STANDING) almost certainly need feature redesign: orientation histograms, postural transition indicators, or gravity-axis decomposition rather than raw accelerometer statistics. Gait subclasses respond better to windowing or frequency-domain features that attenuate the local overlap. And aggregate metrics? Treat them as summaries. Never as deployment decisions.

Why RF Wins Here Without Invalidating SVM Theory

Nothing here contradicts large-margin theory. Tree ensembles absorb overlap-heavy feature neighbourhoods more readily in this dataset; that is an empirical observation about data geometry, not a theoretical refutation. SVM remains coherent and operationally useful. Just not the strongest fit for these corridors in this run.

Model selection is therefore conditional, which is the boring-but-correct conclusion that benchmarking should produce. Reach for SVM when you need explicit boundary control and disciplined regularisation. Reach for RF when local overlap persists despite careful kernel tuning. Keep both in the candidate pool until class-level risk criteria are satisfied.

Calibration and Decision-Threshold Implications

Margins and probabilities are different beasts. Where outputs feed thresholded actions (fall detection alerts, activity-gated medication reminders), both must be audited separately. Platt-style calibration and pairwise coupling give workable pathways, but corridor classes can remain stubbornly overconfident even when macro calibration looks adequate [3] [4].

Robustness Checks Before Promotion

Before promoting this benchmark pattern into production policy, add four checks:

1. Grouped validation by subject/entity to reduce split optimism.
2. Stability analysis around near-optimal hyperparameter neighborhoods.
3. Class-specific calibration diagnostics.
4. Comparator retention over successive retraining cycles, especially when class frequencies drift.

Hyperparameter Stability Signal

SVM reached its best CV region around $\gamma=0.001$ and $C=50$, then softened at higher $C$. The performance surface has a ridge: useful but narrow. RF’s response across tested mtry values was flatter; a plateau rather than a ridge.

What this means in practice: SVM can match RF’s territory with careful tuning, but the stable region is smaller. RF offers broader tolerance. For teams retraining weekly against shifting sensor populations, that plateau is worth something tangible. Narrow ridges amplify drift risk every time hyperparameters are re-selected.

PCA Geometry Reading

PCA tells its own story here. The first few components gobble variance quickly (the easy global structure), but the tail compresses slowly; you need many dimensions to capture the residual discriminative signal. Static versus dynamic regimes separate partially at the macro level. Walking subclasses? They overlap persistently, even after projection.

That geometry explains the SVM behaviour precisely. Global separations are clean; local confusions repeat because the kernel cannot resolve what the feature space itself does not separate.

R and Python Parity: Why It Matters

R and Python produced closely aligned results under matched preprocessing and split assumptions. Good. That means the performance patterns are driven by model-data interaction, not by language artefacts or hidden numerical divergences in LAPACK bindings.

Why belabour this point? Because toolchain tribalism still wastes engineering hours in practice. When your R pipeline and your Python pipeline agree to the third decimal on class-level F1, the conversation shifts to where it belongs: data quality, feature design, and evaluation protocol. Not which language is “better.”

For a broader model-governance continuity perspective, connect this with data provenance and traceability methods.

Practical Conclusions from the Benchmark

1. SVM remains strong and defensible in this task class, but not best-in-run here.
2. RF is empirically superior for this dataset’s overlap-heavy boundary regions.
3. Class-level diagnostics change model selection decisions more than headline metrics.
4. Tuning stability should be treated as a first-class operational criterion.

Continue the Series

For implementation playbooks and deployment controls, continue with Part 3: Tuning, Monitoring, and Deployment Governance Playbook.

Common Questions

Why is macro F1 alone insufficient for SVM evaluation in HAR benchmarking for support vector machine benchmark?

Macro F1 averages class behavior and can hide concentrated failure corridors that dominate real-world risk, especially in near-neighbor class pairs.

Does higher random-forest accuracy mean SVM is the wrong model family for support vector machine benchmark?

No. It means RF is the better fit for this specific dataset geometry and operating objective; SVM can still be strong in other structured regimes.

How should confusion corridors be operationalized in production monitoring for support vector machine benchmark?

Track the highest-frequency class transitions as explicit alert channels and tie retraining or threshold updates to corridor-specific drift.

Why does R-versus-Python parity still matter with similar libsvm tooling for support vector machine benchmark?

Parity checks still matter because they expose silent preprocessing or CV mismatches and improve reproducibility across mixed-language teams.

Conclusion

SVM is stable. SVM is informative. RF is stronger for this overlap profile in this run. The real outcome of the benchmark is not the ranking; it is the diagnostic clarity about where each model family wins, why it wins there, and what that implies for production monitoring. Part 3 converts these diagnostics into a deployment governance playbook: tuning schedules, corridor-specific monitoring, and escalation triggers that connect class-level behaviour to operational decisions.

Technical Appendix