STRATA
Skip to study
XLVII

IV.08 / AI

Kernel methods

When does a similarity function let a classifier recover a boundary missed in the original coordinates?

A source-led introduction1 source references0 related experiments

Start with the question.

Kernel methods allow a learning procedure to use similarities corresponding to a richer feature space without always constructing that space explicitly. Cortes and Vapnik's support-vector work gives a concrete historical setting for studying margins, errors, and generalization. The useful comparison holds the data split fixed while varying the representation and the control on model flexibility.

Make a nonlinear boundary visible.

Place four points at (−1,−1), (−1,1), (1,−1), and (1,1) and give diagonally opposite corners the same class. A straight line in the original plane cannot separate the two classes. Add the feature x·y; its sign separates the pattern. The example shows how a representation can change a classification problem before any claim about a particular kernel's performance.

What would you inspect?

  • Feature scaling, kernel choice, regularization, and support-vector count.
  • Performance on an untouched test set and sensitivity to outliers.

Take the next step.

Compare a linear model and one kernel model on a small documented dataset, using the same preprocessing and validation split.

Read the supporting material.

  1. Support-vector networks

    Corinna Cortes and Vladimir Vapnik · Support-vector classification and benchmark comparison.