Feature Scaling and Pipelines

Some models are sensitive to the scale of their input features in ways that have nothing to do with which features actually matter. This lesson covers fixing that with StandardScaler, and introduces Pipeline, the tool that chains preprocessing and modeling together safely.

Why scale matters

Imagine two features: "age in years" (roughly 0-100) and "income in dollars" (roughly 0-200,000). A distance-based model like k-NN computes distances using raw numbers, income's much larger range would completely dominate the distance calculation, making age nearly irrelevant, regardless of which one is actually more predictive.

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import Pipeline
from sklearn.metrics import accuracy_score

iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
    iris.data, iris.target, test_size=0.2, random_state=42
)

pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('knn', KNeighborsClassifier(n_neighbors=5)),
])

pipeline.fit(X_train, y_train)
preds = pipeline.predict(X_test)
print('Accuracy:', accuracy_score(y_test, preds))

StandardScaler transforms every feature to have mean 0 and standard deviation 1, putting every feature on the same footing before a distance-based (or regularized linear) model ever sees them.

Why Pipeline, not just calling StandardScaler manually

You could call scaler.fit_transform(X_train) and scaler.transform(X_test) by hand, but Pipeline bundles preprocessing and modeling into one object with the exact same fit/predict interface as any single model, which prevents a subtle but serious bug: data leakage.

# What Pipeline does correctly under the hood:
# 1. Fit the scaler ONLY on training data (learn its mean/std from X_train)
# 2. Transform X_train using those training-only statistics
# 3. Transform X_test using the SAME training-only statistics (never refit on test data)

If you accidentally fit a scaler on the combined train+test data, information about the test set (its mean, its spread) leaks into preprocessing before you've ever evaluated anything, quietly inflating your test score. Pipeline.fit(X_train, y_train) guarantees every preprocessing step only ever learns from training data, exactly like the model itself.