How machine learning works

I know internet is full of videos and articles that explains how machine learning (ML) works. I thought I could write a short essay about it from a mathematical point of view, only for reinforcing my knowledge about it :). Obviously, most of this post will be write by AI, but I will try to give it my personal touch with some personal remarks and some code and interactive views.

Maths behind the ML

I assume you are familiar with the basics of calculus, linear algebra and probability. If not, I recommend start with the basic concepts of them.

ML is not AI. ML is a corelation between two random variables and . We can model that corelation as a function . So we are trying to find a function that minimizes the error between and .

The most basic ML model is the linear regression. We try to find a function that minimizes the error between and .

Here, represents the weight (or the slope of our line), which dictates how much influence the input has on our prediction. The variable is the bias (or the y-intercept), which shifts our line up or down to better fit the data. In a real-world scenario with multiple features (like predicting house prices based on square footage, number of rooms, and age), and become vectors, transforming our simple equation into a dot product: .

Measuring the Mess: What Error Should We Use?

To find the “best” line, we need a mathematical way to define how wrong our current line is. We do this using a Loss Function (or Cost Function).

For linear regression, the undisputed champion of loss functions is the Mean Squared Error (MSE).

If we have data points, the MSE is calculated as:

Personal Remark: Why do we square the error instead of just taking the absolute value? Two reasons! First, squaring ensures that negative and positive errors don’t cancel each other out. Second, and more importantly for the math, a squared function creates a beautiful, smooth, convex curve (think of a bowl shape). This guarantees that when we use calculus to find the minimum point (the bottom of the bowl), there is only one global minimum to find. It makes the optimization math much easier.

When Linear Regression is a Terrible Idea

Linear regression is fantastic because it is interpretable and fast, but it is entirely useless in a few key scenarios:

• Non-Linear Relationships: If your data follows a complex curve, a wave, or a circle, trying to fit a straight line through it is like trying to fit a square peg in a round hole. The model will “underfit” and your predictions will be garbage.
• Classification Problems: If you are trying to predict a category rather than a continuous number (e.g., “Is this email spam or not?” or “Is this a picture of a cat or a dog?”), linear regression fails. A straight line can stretch to infinity, but probabilities must live strictly between 0 and 1. In these cases, we swap to Logistic Regression (which uses a Sigmoid function to squash the output) or jump into neural networks.
• Highly Complex Data: For things like image recognition, audio processing, or natural language processing, the relationships between pixels or words are far too complex for a single linear equation to capture.

Backpropagation: The Calculus Workhorse

When linear models aren’t enough, we stack multiple functions together to create a Neural Network. Instead of a simple , we end up with something like .

This brings up a massive problem: if the network makes a bad prediction, how do we know which specific weight () deep inside this nested mess of functions is responsible for the error?

Enter Backpropagation (Backward Propagation of Errors).

Backpropagation is essentially the Chain Rule from calculus, applied systematically from the output of the network all the way back to the input.

Let’s look at the math. Suppose our Loss depends on an activation output , which depends on a linear equation , which finally depends on our weight . According to the chain rule, the derivative of the Loss with respect to our weight is:

Here is how the backpropagation algorithm actually works in practice:

1. Forward Pass: We push our data through the network to get a prediction. We compare this prediction to the actual and calculate the total Loss .
2. Backward Pass: We calculate the gradient (the partial derivatives) of the loss function with respect to every single weight and bias in the network using the chain rule formula above.
3. Gradient Descent: We update every weight to move slightly in the opposite direction of the gradient. If the derivative is positive (meaning increasing the weight increases the error), we decrease the weight.

The update rule looks like this:

Here, is the learning rate—a tiny number (like 0.01) that ensures we don’t take steps that are too large and completely overshoot the bottom of our mathematical bowl. By repeating this process thousands of times, the network slowly self-corrects, adjusting its weights until the error is minimized.