The way humans (and maybe all animals) have interacted with their environment is by finding patterns and inferring the future. We have become so obsessed with pattern finding that we built machines and taught them how to find them. AI became mainstream because of generative models (chat and image creation apps), but they are just a flavor of something that has been around for more than two decades.

Machine learning algorithms have been used since the 1960s in the military, in medical image processing, and in science and manufacturing. They just weren’t very visible to the average person, even when we started “using them” all the time—search engines like Google, spam filters in email, recommendations in e-commerce like Mercado Libre, facial recognition in phones, and even the camera app use ML algorithms to improve performance. So isn’t it about time we demystify it?

As I have said (and will say again) in other publications: these are just statistical methods applied to a large amount of data to tune the parameters (dials) of a mathematical equation there’s no consciousness involved.

Nowadays, the algorithm alone is not enough. It’s still crucial, but we need to embed it into a system, just as mitochondria are embedded in the cell, and the cell is part of something bigger. This doesn’t mean it should always be used; in fact, if the pattern is too simple or there’s an easier solution that works well, it shouldn’t be used. These are magnifiers of our human capabilities and our biases so their application needs ethical scrutiny; Imagine a loan wordiness classification system that was fed with raze or gender as an input data, What about the algorithm in social platforms that encloses it’s users in a opinion bubble?

The applications of machine learning are quite diverse, but they can be grouped under two points of view:

  1. Regression problems: Basically, finding the line that best fits a set of known points to predict what comes next (for example, predicting the next word in a sentence if words are encoded as numbers).
  2. Classification problems: Any task that involves assigning items to categories, whether two, many, or multi-label—for example, classifying a movie as a good recommendation or not for a specific Netflix user.

Finally, we need a function that helps us measure success in our objective. This is called a loss function ( like a Progress tracker), and a system, unlike a single model, can have more than one loss function, sometimes one per functional requirement (objective). A system can therefore integrate many ML algorithms.

What does a data scientist do?

They must understand the business metrics that define success, grasp the problems to be solved, and build a semantic bridge between real-world success metrics and those that measure an ML model’s success, and more times than others finding middle ground between espectations that are contrary to each other.

A very important part of this process—something we’ll discuss later—is knowing what data is needed or available, exploring it, and understanding it to build models.

The mathematics and code for most of these models are already developed and documented in libraries like scikit-learn or PyTorch. So, as you might predict, the heart of a data scientist’s work is communication and learning quickly to propose and develop possible solutions.

The data part of the system

Models are nothing without data—and there’s no ML without models. This is why ML system development isn’t like traditional software development: it’s not just the code that needs version control. The data and the data artifacts used to transform it also need version control, so the system requires proper data infrastructure.

The data engineer and data architect must make many decisions about how models will consume data, how the data will arrive, and how to make it clean and usable. In real life, data is usually messy and often lacks labels. Without labels, how can you use a loss function to tell your model whether it’s doing well or not? It’s like shooting arrows without a target. Again, data scientists and data engineers need to communicate a lot.

Deploy and monitor

Let’s say your team has already found a model that best solves your problem. Now it’s time to deploy it. Nowadays, deploying it in the cloud is the most popular option. Your The model or models will be placed inside containers (like fishbowls that have everything your fish needs to live) and connected to the rest of your system or apps through an API, which will have to be built—a set of rules that standardizes communication.

The system’s performance must be monitored to detect when it’s time to revisit the data, perhaps there have been statistical changes that explain a decline in performance. These models stop working as soon as the underlying patterns change, and if those patterns are linked to human behavior, that happens more often than you might think.

The people responsible for this are called ML engineers or MLOps professionals. They help build infrastructure not only for deployment but also for monitoring and tracking models and data, with the goal of identifying and mitigating performance decay due to pattern changes. Once again, communication is key.

save an repeat

The data your system makes predictions on can be use to retrain your models, specially when the pattern changes, so the loop closes, again the data infra structure has to be tune, also the models , it’s an alive process, where people is the key ingridient.

Now that you understand better how ML is and will keep shaping our lives in the shadows, and how, even when it seems radically advanced, is just the product of well-educated communicators who happen to know a lot of statistics, linear algebra, and software, how are you going to use this knowledge?.