Linear regression rests on a few assumptions about the data and the residuals. The relationship between the features and the target is linear. Observations are independent of each other. Residuals have constant variance across the fitted range. Residuals are roughly normal. And no feature is a perfect linear combination of the others.

They matter in two different ways. Linearity and independence affect the coefficients themselves, so breaking them makes the estimates wrong. Constant variance and normal residuals mostly affect the uncertainty around those estimates. The fitted line can look fine while every p-value and confidence interval quietly lies.

The cheapest check is a residual plot against the fitted values. A funnel shape means the variance grows with the prediction. A curve means you are forcing a straight line through a bend. Time-ordered or repeated-measure data usually breaks independence. That makes the model look far more confident than it deserves.

Each coefficient is the expected change in the target when that feature rises by one unit and the others stay fixed. The intercept is the prediction when every feature equals zero. That is often a meaningless point, because zero may sit far outside the observed data.

Units drive everything. A coefficient of 3200 on bedrooms means 3200 more dollars per extra bedroom. You cannot compare raw coefficients across features, because a variable measured in millimetres earns a tiny number. Standardize the features first if you want to rank them by influence.

The phrase "holding the others fixed" is where readings go wrong. When two features move together in reality, nobody can hold one still, so the number is not a causal effect. Signs can even flip when you add or remove a related feature. Always report a coefficient alongside the exact feature set it was fitted with.

Logistic regression predicts the probability of a class rather than a numeric quantity. It computes the same weighted sum of features that linear regression does. The difference is that the sum is passed through a squashing function, so the output stays between zero and one.

Three things change. The target is a category, not a continuous value. Fitting uses maximum likelihood with an iterative solver, not a closed-form least squares formula. The output is a probability, and you turn it into a label by comparing it against a threshold.

Running plain linear regression on zero-one labels is the mistake this fixes. That model happily predicts 1.4 or a negative probability, and neither can be acted on. Its errors also grow at the edges, so points far from the boundary drag the line around. Logistic regression keeps the readable linear score while making the output usable.

The sigmoid is the S-shaped function that maps any real number into the range zero to one. Large negative inputs land near zero, large positive inputs near one, and zero maps to exactly 0.5.

const sigmoid = z => 1 / (1 + Math.exp(-z));
sigmoid(0);   // 0.5
sigmoid(2.2); // 0.900

Logistic regression needs it because a weighted sum of features is unbounded, while a probability is not. The sigmoid also preserves order, so ranking by raw score matches ranking by probability. It is smooth and differentiable everywhere, which is what lets gradient descent fit the weights.

The cost is saturation. Once a score sits far from zero, the curve is almost flat and the gradient nearly vanishes. A confidently wrong prediction then learns very slowly. Unscaled features push scores into that flat region, which is one reason training stalls.

A random forest is a collection of many decision trees whose predictions get combined into one. Each tree sees a different random sample of rows and a different random subset of features at each split. For classification the trees vote; for regression their outputs are averaged.

The difference from one tree is stability. A single deep tree fits the noise in its training data, and swapping a few rows can change its whole structure. Averaging many trees that each err in different directions cancels most of that noise. The forest usually beats the single tree on unseen data by a wide margin.

You pay for that in two ways. Hundreds of trees cost more memory and more prediction time than one. You also lose the readable if-then path and get ranked feature importances instead. Training parallelises easily, since the trees never depend on each other.

K-nearest neighbours (KNN) stores the training rows and predicts by finding the k closest ones to a new point. Classification takes the majority label among those neighbours.

It is lazy because there is no real training step. Fitting just keeps the data in memory, and all the work happens at prediction time. A naive scan costs about O(nd) distance computations per query, with n rows and d features.

It is non-parametric because no fixed set of parameters is ever learned. The shape of the decision boundary comes from the data itself, so it can bend any way the classes require. That is the appeal: you assume nothing about linearity or about the distribution of the features. The cost is that the whole dataset must be kept and searched for every single prediction.

Euclidean distance is the default choice, but the right metric depends on what the features mean.

• Euclidean, the straight-line distance. Fine for continuous features that share comparable scales.
• Manhattan, the sum of absolute differences along each axis. It holds up better in high dimensions and with grid-like or count data.
• Minkowski, the general form with a parameter p. Setting p to 1 gives Manhattan and setting it to 2 gives Euclidean.
• Cosine, the angle between two vectors. It ignores magnitude, which suits text and sparse counts where document length varies.
• Hamming, the count of positions where two rows differ. Use it for binary or categorical features.

The metric is a real hyperparameter, not a detail. Change it and the neighbour set changes, so the prediction changes with it. Mixed numeric and categorical data usually needs a combined measure such as Gower distance, since no single formula fits both types.

Yes, and the change is small: instead of voting on labels, k-nearest neighbours (KNN) averages the target values of the k nearest neighbours. Predicting a house price means taking the mean price of the k most similar houses.

A common refinement weights each neighbour by the inverse of its distance. Closer points then pull the prediction harder, which smooths out the jumps you see when k is large.

Two costs are worth knowing. The prediction surface is a staircase rather than a smooth curve, because a whole region shares the same neighbour set. And the model cannot extrapolate at all. Feed it an input beyond the training range and it returns the average of the same edge neighbours, flat forever. That makes it a poor fit for trending data such as prices over time.