The same hundred and thirty observations the principal component page uses, where the outcome depends on a low-variance contrast between six correlated predictors. Principal components rank the common factor first — 92.90% of the predictor variance, correlation −0.0012 with the outcome — because it is built without looking at the outcome. Partial least squares ranks a direction carrying 1.43% of the variance first, correlating 0.7155, because it is built by looking. One supervised component reaches a cross-validated error of 0.6718; the best unsupervised fit needs five to reach 0.6883.
130 rows · 6 predictors · 6 components available
3 supervised components reach a cross-validated error of 0.6625
The unsupervised decomposition needs 5 to reach 0.6883, and 1 supervised component already match it. The first supervised direction carries 1.43% of the predictor variance and correlates 0.7155 with the outcome; the first unsupervised one carries 92.90% and correlates -0.0012. The only difference between the methods is whether the outcome was consulted.
Components needed
3
error 0.6625
Unsupervised needs
5
error 0.6883
First direction holds
1.43%
of the predictor variance
Its link to the outcome
0.7155
unsupervised: -0.0012
The two orderings side by side
Variance, outcome correlation and cross-validated error for each component under both methods
Component
Supervised variance
Supervised correlation
Supervised error
Unsupervised error
1
1.43%
0.7155
0.67182
1.35282
2
90.37%
1.1396e-17
0.67214
1.33844
3
4.02%
-1.5716e-16
0.66245
1.15447
4
1.56%
2.2131e-15
0.66610
1.05984
5
1.36%
1.1022e-14
0.68447
0.68834
6
1.26%
1.0540e-13
0.69092
0.69092
Compare the last two columns going down. The supervised error drops almost all the way at the first row; the unsupervised error works its way there over several. Both end at the same place, because at full rank the two methods span the same space and both become ordinary least squares — the difference is entirely in what order they get there.
The correlations of the later supervised components with the outcome come back at essentially zero. That is the algorithm working as designed: each direction is extracted from what remains after the previous one is removed, so by construction the outcome signal is loaded into the earliest components and taken out of the rest.
Using the outcome to build the directions is exactly what makes overfitting possible, so the component count must be chosen by cross-validation and the error at that count is optimistic if the same data chose it. On small samples the unsupervised version is the more conservative choice for that reason.
Directions built on the outcome Unsupervised beside it Error at every count Both converge at full rank Overfitting risk stated
What this tool shows
One supervised component reaches a cross-validated error of 0.6718. The best unsupervised fit needs five to reach 0.6883. The two methods differ by one word in their construction: partial least squares builds each direction against the outcome, principal components ignore it. On the shared preset the first supervised direction carries 1.43% of the predictor variance and correlates 0.7155 with the outcome; the first unsupervised one carries 92.90% and correlates −0.0012. Both methods end at the same place at full rank — the difference is entirely the order they get there in.
Components built to maximise covariance with the outcome, by the NIPALS algorithm
The unsupervised decomposition computed alongside on every run
Cross-validated error at every component count, under both methods
How many supervised components it takes to match the best unsupervised fit
Why the later components show no correlation with the outcome
The overfitting risk that using the outcome creates
Directions built on the outcome Unsupervised beside it Error at every count Both converge at full rank
Using the outcome to choose directions is the advantage and the risk.
Updated 13 September 2026 · Works in any browser, no installation
Partial least squares reduces correlated predictors to a few directions like principal components does, but builds each direction to maximise covariance with the outcome rather than variance in the predictors. That single change means the first component is usually the most useful one, so far fewer are needed. It also means the outcome has been used to construct the model, which creates a route to overfitting that the unsupervised method does not have.
At a glance
Formula shown
Each direction is w ∝ Xᵀy on the current residuals, normalised. The scores t = Xw are computed, X is deflated by removing their contribution, y is deflated too, and the process repeats. That deflation is why later components show near-zero correlation with the outcome: the signal is loaded into the earliest directions and removed from what remains. Using all p components reproduces ordinary least squares exactly.
Scenario support
Spectroscopy and chemometrics, where the method originated and predictors number in the hundreds; sensor arrays; process monitoring; any regression with many collinear predictors where a compact model is wanted and the sample is large enough to support using the outcome.
Educational estimate
Planning support from the values you enter — not professional advice.
One against five
The preset is the same data the principal component page uses: six predictors sharing a common factor, and an outcome depending on a contrast between them.
The unsupervised decomposition ranks the common factor first, because it is far and away the largest source of variance in the predictors. It carries 92.90% of it, and correlates −0.0012 with the outcome — nothing. The contrast that actually matters is ranked fifth, carrying 1.23%.
Partial least squares ranks the useful direction first. That direction carries just 1.43% of the predictor variance and correlates 0.7155 with the outcome. It is close to the last direction the unsupervised method would reach and the first the supervised method does, and the only difference in how they were built is whether the outcome was in the calculation.
The consequence is the error column. One supervised component gives 0.6718. The unsupervised fit reaches 1.3528 with one, 1.3384 with two, 1.1545 with three, 1.0598 with four, and 0.6883 with five. Its whole ordering has to be worked through to arrive where a single supervised direction starts.
Both end at the same place. At full rank the two methods span the same space and both become ordinary least squares exactly, which the suite confirms to 0.0e+0. Dimension reduction is only interesting before that point, and the ordering is what decides how good the reduced models are.
Why the later components look empty
The correlation column shows the first component at 0.7155 and every later one at essentially zero. That is not a sign the data has nothing left in it; it is the algorithm working as designed.
After each direction is extracted, both the predictors and the outcome are deflated — the part explained by that direction is subtracted, and the next direction is built from what remains. So the outcome signal is deliberately concentrated in the earliest components and removed from the rest, which is precisely why so few are needed.
It also means the correlation column reads differently here than on the unsupervised page. There, each component’s correlation is an independent fact about a fixed direction, and comparing them is meaningful. Here the first one is informative and the rest are near-zero by construction, so the column to read down is the error, not the correlation.
The cost of using the outcome
Every advantage above comes from consulting the outcome when building the directions, and that is also the method’s exposure. The unsupervised decomposition cannot overfit the outcome because it never sees it; partial least squares can.
The practical consequence is that the component count must be chosen by cross-validation rather than by any rule of thumb, and that the error at the chosen count is optimistic if the same data chose it. With a large sample the optimism is small. With a small one it can be substantial, and the comparison against the unsupervised fit is worth running for exactly that reason — it is computed here on every input.
The general rule is that supervised reduction pays off when there are many collinear predictors and enough rows to support looking at the outcome, which is why the method came out of chemometrics, where predictors number in the hundreds and samples are well controlled. On a handful of predictors with a small sample, the more conservative choice is usually right.
Nested cross-validation — an outer loop for the error estimate and an inner one for the component count — is the honest way to report accuracy when the count was tuned. The single-loop figure reported here is the ordinary one and is optimistic in the way the cross-validation page quantifies.
Against the other ways to handle collinearity
Ridge shrinks every direction in proportion to how poorly determined it is, rather than keeping some entirely and discarding others. It is the smoothest of these methods, it keeps all the predictors, and it is usually the safest default when prediction is the goal and the number of predictors is moderate.
Principal components reduce dimensions without touching the outcome. That makes them immune to the overfitting risk above and vulnerable to the ordering problem this page demonstrates. Where the useful direction happens to carry a lot of variance the two methods agree closely.
Lasso answers a different question again: it keeps original predictors rather than combinations of them, so its output is interpretable in a way no component method is. A component is a weighted mixture of every predictor and cannot be described as the effect of any one of them.
Reporting it
Report the number of components and how it was chosen. That single number is the model’s complexity and the thing most likely to have been tuned on the same data used to report the error.
Report the unsupervised comparison where you have it. If the supervised method needs many fewer components, that is a real finding about the data — the useful direction is not a high-variance one — and it is worth a sentence. If both need about the same, the supervised machinery bought nothing and the simpler method is preferable.
Do not describe components as factors with meanings. They are weighted mixtures of every predictor, their loadings shift when predictors are added or removed, and naming them invites readers to treat a coordinate direction as a measured quantity. Coefficients on the original predictor scale are reported for that reason and are the interpretable output.
Sources and methodology
References for the algorithm and its comparison with unsupervised reduction.
Method. Directions are built by the NIPALS scheme: the weight vector is proportional to the covariance between the current predictors and the current outcome, scores are extracted, and both are deflated before the next direction is taken. The component count is chosen by k-fold cross-validation over every possible number, and the unsupervised decomposition is fitted on the same folds so the two error columns are comparable. The suite confirms that using every component reproduces ordinary least squares to 0.0e+0, that the two methods therefore agree exactly at full rank, that the variance shares still sum to one, and that on data whose useful direction carries little variance the supervised method matches the unsupervised best with no more components in at least 70% of generated datasets. That engine is verified on every change against 395 assertions. The count and the per-case breakdown are published on the formula verification page.
Related calculators
Where this goes next:
Principal Component RegressionRegress on principal components with each component's variance share printed beside its correlation with the outcome, and the error at every component count.
Ridge RegressionFit ridge regression across a penalty path chosen by cross-validation, with least squares beside it and the shrinkage applied to every coefficient shown.
Cross-ValidationRun k-fold cross-validation over a penalty path and repeat it across twelve fold assignments, so the variability of the chosen tuning parameter is visible.
Lasso RegressionFit a lasso across a penalty path with cross-validation, and see how often each selected predictor survives when the rows are resampled.
Multiple RegressionFits several predictors with a VIF on every term, and names the configuration people misread: a model significant at p = 0.0103 where neither predictor reaches 0.05, at a VIF of only 7.11.
EigenvalueEigenvalues and the characteristic polynomial up to 6×6 — the polynomial exact and the roots decimal, because roots of a polynomial are generally irrational.
An educational tool. Because the outcome is used to construct the components, this method can overfit in a way principal component regression cannot, and the cross-validated error at a tuned component count is optimistic — nested cross-validation is required for an unbiased estimate. Components are weighted mixtures of every predictor and are not interpretable as effects of individual variables.