☆ Save 1. Traditional Machine Learning in Practice: Learning Paradigms, Algorithm Families, and Evaluation Perspectives

10/26/2025

Traditional Machine Learning can be understood as a way of learning rules and patterns that already exist in data so that a model can perform prediction or classification on new inputs. Instead of hand-coding rules based on human intuition or experience, we express recurring relationships in the data as statistical and mathematical models, with the goal of generalizing beyond the training set.

Table of Contents

• Concept of Machine Learning
• Learning Paradigms in Machine Learning
• Families of Machine Learning Algorithms
• Evaluation Metrics and Perspectives on Interpretation

Concept of Machine Learning

Machine Learning is an empirical approach that repeatedly observes data and learns the relationship between inputs and outputs. Instead of a human defining an explicit formula or rule that directly computes the answer, the model estimates and represents the data distribution and underlying trends on its own. Once trained, the model is designed to make predictions for previously unseen inputs by applying generalized rules learned from past data.

Learning Paradigms in Machine Learning

The way a Machine Learning model receives learning signals depends on how the problem is formulated. Whether labels are given or not, and whether the agent must discover structure or a reward-maximizing behavior on its own, leads to different learning paradigms: Supervised Learning, Unsupervised Learning, and Reinforcement Learning.

• Supervised Learning
  A learning paradigm where each input is paired with a ground-truth label, and the model is trained so that its predictions become close to those labels. The model is iteratively improved in the direction of reducing the discrepancy between predicted and true values. Supervised Learning is used for both Classification and Regression problems.
• Unsupervised Learning
  A learning paradigm in which the data do not come with labels. The goal is to discover similarity, group structure, and latent patterns among samples. Unsupervised Learning is used to understand how data are distributed and structured, and is central to tasks such as Clustering and Dimensionality Reduction.
• Reinforcement Learning
  A learning paradigm in which an agent interacts with an environment and learns a behavior policy that maximizes cumulative reward. Through trial and error, the agent updates its strategy to choose better sequences of actions. Reinforcement Learning is widely applied to sequential decision-making problems.

Families of Machine Learning Algorithms

Traditional Machine Learning algorithms are more than just a list of separate techniques. They can be grouped into different families that reflect how they interpret data and construct predictions. The families of clustering·classification·regression·and ensemble methods provide representative views on how to formulate and solve a problem.

• Clustering Algorithms
  Unsupervised methods that discover group structure so that similar data points end up in the same cluster without using ground-truth labels. Clustering algorithms analyze distances and similarities between samples to reveal the underlying grouping of the data.
  Representative algorithms: K-means, K-medoids, Hierarchical Clustering
  Evaluation metrics: Silhouette Coefficient, Homogeneity, Completeness, V-measure
• Classification Algorithms
  Supervised methods that predict which class a given sample belongs to. These algorithms learn decision boundaries and conditions that separate classes, and then assign new data points to one of the known categories.
  Representative algorithms: Logistic Regression, Decision Tree, Random Forest, KNN (k-Nearest Neighbors), SVM (Support Vector Machines)
  Evaluation metrics: Accuracy, Precision, Recall, F1-score, ROC-AUC
• Regression Algorithms
  Supervised methods that predict continuous numerical values by estimating a functional relationship between input and output variables. Regression is used not only for prediction but also for interpreting how input variables influence the output.
  Representative algorithms: Linear Regression, Polynomial Regression, Lasso, Ridge
  Evaluation metrics: MSE, RMSE, MAE, R²
• Ensemble Learning
  An approach that combines multiple base learners instead of relying on a single model, with the aim of improving predictive performance and robustness. By compensating for the bias and variance of individual models, ensemble methods enhance generalization ability.
  Representative approaches: Bagging, Boosting

Evaluation Metrics and Perspectives on Interpretation

Evaluating a Machine Learning model is not just about comparing raw accuracy numbers. The evaluation must be interpreted in light of the problem characteristics and the real objective. Even for the same model, the meaning of its performance can change significantly depending on which metrics we choose to focus on.

• Metrics for Classification
  Accuracy·Precision·Recall·F1-score·ROC-AUC — these metrics jointly capture different types of errors and the effect of class imbalance, helping us interpret how the classifier makes decisions and where it fails.
• Metrics for Regression
  MSE·RMSE·MAE·R² — these metrics quantify prediction error from different angles: the average squared error, the error in the original scale, robustness to outliers, and the explanatory power of the model.
• Metrics for Clustering
  Silhouette Coefficient·Homogeneity·Completeness·V-measure — these metrics evaluate the structural quality of clusters in terms of within-cluster cohesion and between-cluster separation, as well as consistency with ground-truth labels when available.

Recommended prerequisite reading (3/5)

+2

• 8.2 Generative Modeling — Learning Data Distributions and Generating New Samples
• 2.7 Linear Models
• DBSCAN Explained: Density-Based Clustering with Automatic Noise Detection

• Classifier — From Input to Class Label: Understanding Softmax and the Loss Function
• Decision Stump — Why a Single Split Becomes the Building Block of Boosting

Recommended next reading (5/16)

+5

• Traditional Machine Learning vs. Deep Learning — Features are hand-engineered vs. learned from data
• Classification Algorithm — A supervised learning method that learns class boundaries from input features and predicts the category of new data
• Data-driven Approach — Learning Rules from Data for Prediction and Decision-Making
• Learning from Data — Turning Patterns into Predictive Rules Through Generalization
• Pretrained Model — A strategy that reuses representations learned from large-scale data to speed up learning on new tasks

• Learning Systems (Networks) — Improving Performance by Updating Connection Weights Through Experience
• Softmax Regression — A Basic Multiclass Model that Turns Linear Scores into Probabilities
• Voting Ensemble — An ensemble approach that combines multiple models to produce more stable predictions
• Discriminative Model — How AI Learns Direct Input-to-Output Predictions
• 6.8 After Transformers and Modern AI — Sequential Modeling Expanded into Foundation Models and Generative AI

• Polynomial Regression — A Regression Model That Approximates Nonlinear Relationships with Polynomials
• Domain Fine-Tuning — A Fine-Tuning Strategy That Adapts a Pretrained Model to a Specific Data Distribution
• Named Entity Recognition (NER) — Identifying and Classifying People·Places·and Organizations in Text
• Resampling — Oversampling vs Undersampling and Why It Matters for Imbalanced Data
• Generative Model — Learning Data Distributions to Generate New Samples
• End-to-End Learning — Training a system from input to output in a single optimization

Posts on the same topic (0/0)

No other posts in this section yet.

Related concepts (0/0)

No related concept posts yet.

📍 Where this concept fits in the AI learning map

See where this concept sits within the full AI Universe.

📍 Current position in AI Universe

☰

Reset Show completed · Login required Loading…

🌌 AI Universe

‹

›

⭐ Concept

Select a star.

View the full AI Universe

« 8.9 Diffusion Models — G…|4. Multilayer Perceptron… »

🔖 Tags: Classification Algorithms · Ensemble Learning · Evaluation Metrics · Machine Learning · Model Evaluation · Regression · Supervised Learning · Traditional Machine Learning · Unsupervised Learning