2.3 Three Waves in the Evolution of Deep Learning
The evolution of Deep Learning did not happen in a single step. It emerged through several historical phases, in which ideas were proposed·challenged·abandoned·and later revived. From early neuron models through connectionist neural networks to today’s large-scale deep architectures, each wave extended the previous one and addressed its limitations.
10/29/2025
Artificial Neuron — The Smallest Neural-Network Computing Unit That Turns Inputs into a Decision Signal via Weighted Sums and Nonlinearities
Artificial Neuron is the most basic computational unit in a neural network: it combines multiple input signals into a weighted sum and then maps that sum through an activation function to produce a single output. Modern deep learning models are, at their core, large compositions of these tiny units—stacked and connected to learn complex patterns from data.
11/09/2025
Brain and Computer — How Human and Artificial Intelligence Differ Through Information Processing
Brain and Computer is a research theme that compares the human brain and artificial computers from an information-processing perspective to understand how intelligence can be realized. Rather than copying the brain neuron-by-neuron, it contrasts how each system performs computation·memory·and learning—and what that implies for the design of AI systems.
10/28/2025
Connectionist AI (2nd Generation) — An AI Paradigm Where Knowledge Lives in Connection Weights, Not Individual Neurons
Connectionist AI (often framed as “second-generation AI”) is an approach that treats intelligence and knowledge as patterns stored in connection weights between neurons, rather than inside any single neuron. A neuron by itself carries little meaning; what matters is which neurons are connected, and how strongly. In this view, learned knowledge is literally the network’s weight configuration.
01/26/2026
Delta Rule — A Basic Learning Rule That Adjusts Weights in Proportion to the Error
The delta rule is a learning rule that reduces the gap between prediction and target by updating weights in proportion to the error.
11/14/2025
Linear Classification — The basic way to separate data with a linear decision boundary
Linear classification is the most basic classification approach: it uses a linear combination of input features to split data with a straight line·plane·or hyperplane. It’s often the first tool you reach for to understand the structure of a classification problem before moving to more complex models.
10/28/2025
Neural Networks — Perceptron, Activation Functions, and Backpropagation Explained
This hub works as a fast navigation map that connects the full flow in one line: overall neural network structure → perceptron → multi-layer perceptron (MLP) → forward propagation and backpropagation → activation functions → weights, biases, and parameters → deep learning expansion.
You can choose one main starting point, the fastest entry path for your current level, and the next article to read right away all in one place.
04/17/2026
Perceptrons — The simplest artificial neural network that switches on above a threshold and off otherwise
A perceptron is the simplest form of an artificial neural network: it takes inputs, computes a single score, and outputs 1 if that score exceeds a threshold (or meets a criterion), otherwise 0. In practice, it forms a weighted sum of the inputs and then makes a binary decision.
10/29/2025
Step Function
The step function is the simplest activation function: once the input crosses a threshold, the output jumps from 0 to 1.
11/09/2025