AI/ML Frontier Research: From Mathematical Foundations to the Nature of Intelligence

The Probably Approximately Correct (PAC) framework, due to Valiant (1984), provides the foundational formalization of learnability. A concept class $C$ over instance space $X$ is PAC-learnable if there exists an algorithm $A$ such that for any target concept $c \in C$, any distribution $D$ over $X$, and any $\epsilon, \delta > 0$, with probability at least $1-\delta$, $A$ outputs a hypothesis $h$ with error $\text{err}_D(h) \leq \epsilon$ using sample size polynomial in $1/\epsilon$, $1/\delta$, and the representation size.

$$m(\epsilon, \delta) \leq O\left(\frac{d + \log(1/\delta)}{\epsilon}\right)$$

$$\mathcal{R}_m(F) = \mathbb{E}_{\sigma, S} \left[\sup_{f \in F} \frac{1}{m} \sum_{i=1}^m \sigma_i f(z_i)\right]$$

$$\mathbb{E}[f(z)] \leq \frac{1}{m}\sum f(z_i) + 2\mathcal{R}_m(F) + O\left(\sqrt{\frac{\log(1/\delta)}{m}}\right)$$

Open Question: Why do massively overparameterized neural networks generalize? The modern resolution involves:
1. Double Descent (Belkin et al., 2019): Test error peaks at the interpolation threshold, then decreases as parameters grow past sample size
2. Benign Overfitting (Bartlett et al., 2020): Interpolating classifiers can generalize well when the data is structured
3. Neural Tangent Kernel regime (see below): Networks in the infinite-width limit are equivalent to kernel machines, which constrains their implicit bias

1.2 Neural Tangent Kernel (NTK)

Jacot, Gabriel, & Hongler (2018): "Neural Tangent Kernel: Convergence and Generalization in Neural Networks" (NeurIPS 2018, arxiv:1806.07572)

The NTK characterizes the training dynamics of infinitely wide neural networks. For a network $f(\theta, x)$ with parameters $\theta$, the evolution under gradient descent follows:

$$\partial_t f(t, x) = -\sum_i \eta \Theta_t(x, x_i) \frac{\partial \ell(f(t, x_i), y_i)}{\partial f}$$

where $\Theta_t(x, x') = \nabla_\theta f(t, x)^\top \nabla_\theta f(t, x')$ is the NTK.

Key Result: In the infinite-width limit (with appropriate initialization scaling), the NTK converges to a deterministic kernel $\Theta_\infty$ that stays constant during training. This implies:
- The network evolves as a linearized model: $f(t, x) \approx f(0, x) - \eta \Theta_\infty(x, S) \exp(-\eta \Theta_\infty(S, S) t) \cdot \text{error}(0)$
- The training dynamics are equivalent to kernel regression with the NTK
- Convergence to the global minimum occurs for sufficiently wide networks

Open Problems:
- Can we precisely characterize when networks transition from lazy to rich (feature learning) regimes?
- How do modern architectures (transformers, SSMs) differ from fully-connected networks in their NTK behavior?
- The connection between the NTK of transformers and their in-context learning capabilities

1.3 Information Theory in Deep Learning

Key Insight: The gradient ∇L points in the direction of steepest ascent. Gradient descent moves opposite: w ← w − η∇L. The learning rate η is the most important hyperparameter.

\log p(x) \geq \mathbb{E}_{z \sim q}[\log p(x|z)] - KL(q(z)||p(z))$$

This forms the basis of VAEs, and more broadly, the variational perspective on learning as free-energy minimization.

Information Theoretic Generalization Bounds (Russo & Zou, 2016; Xu & Raginsky, 2017): The generalization gap is bounded by $\sqrt{I(S; W)/n}$, where $I(S; W)$ is the mutual information between the training data $S$ and the learned weights $W$. This connects PAC-Bayes, information theory, and generalization.

1.4 Optimal Transport Theory

$$W_p(\mu, \nu) = \left(\inf_{\gamma \in \Pi(\mu, \nu)} \mathbb{E}_{(x,y) \sim \gamma}[d(x, y)^p]\right)^{1/p}$$

$$W_1(\mu, \nu) = \sup_{\|f\|_L \leq 1} \mathbb{E}_{x \sim \mu}[f(x)] - \mathbb{E}_{y \sim \nu}[f(y)]$$

This dual form connects Wasserstein distances to adversarial training (GANs), where the discriminator approximates the 1-Lipschitz function.

Applications in Modern ML:
- Wasserstein GANs (Arjovsky, Chintala, Bottou, 2017)
- Sliced Wasserstein distances for efficient computation
- Wasserstein gradient flows — the diffusion process as gradient flow in Wasserstein space (Jordan-Kinderlehrer-Otto, 1998), which connects to score-based diffusion models
- OT for domain adaptation and distributional robustness

2. Architecture Theory

2.1 Transformers: Expressivity and Computation

The Transformer Architecture (Vaswani et al., 2017, "Attention Is All You Need", NeurIPS 2017, arxiv:1706.03762):

Key Insight: Softmax converts raw scores to a probability distribution — all values sum to 1. The temperature of the softmax controls sharpness: low T → near one-hot; high T → uniform.

\text{Attention}(Q, K, V) = \text{Softmax}\left(\frac{QK^\top}{\sqrt{d_k}}\right)V$$

Universal Approximation for Transformers (Yun et al., 2020, arxiv:2002.08863): Transformers with sufficiently many heads and layers can approximate any sequence-to-sequence function continuous in the $\ell_p$ norm. The proof constructs the architecture to simulate:
1. Contextual mapping: each token's representation encodes its entire input context
2. Value extraction: using feed-forward layers to map to arbitrary outputs

Expressivity Hierarchy of Transformers (Merrill & Sabharwal, 2023, arxiv:2304.07854):

Recent theoretical work shows CoT provably extends expressivity. Feng et al. (2023, arxiv:2310.07923) prove that:
- Transformers with CoT can solve problems in $P$ that are intractable without
- A constant number of CoT steps adds at least one level in the $TC^0$ hierarchy
- The number of reasoning steps can trade off against model depth

Tropical Geometry Perspective (Su & Liu, 2026, arxiv:2604.14727): The exact spatial partitioning capacity of transformers can be characterized through tropical geometry, where the decision boundaries correspond to tropical rational functions — piecewise linear functions defined by max-plus or min-plus polynomial combinations.

Key Open Questions:
- Exact characterization of what can and cannot be computed with bounded-precision attention (low-precision transformers with CoT, Brösamle & Eckstein 2026)
- The role of positional encodings in expressivity (relative vs absolute, RoPE, ALiBi)
- How does multi-head attention increase representational power over single-head?

2.2 Graph Neural Networks: The WL Hierarchy

The expressive power of message-passing GNNs is bounded by the Weisfeiler-Lehman (WL) graph isomorphism test.

Key Result (Xu et al., 2018, "How Powerful Are Graph Neural Networks?", ICLR 2019, arxiv:1810.00826):
- The maximum expressive power of message-passing GNNs is exactly the 1-WL test
- The GIN (Graph Isomorphism Network) architecture matches this bound
- To exceed 1-WL, one must use higher-order GNNs (e.g., $k$-WL GNNs), which have complexity $O(n^k)$

WL Hierarchy and Higher-Order GNNs (Morris et al., 2019, arxiv:1810.02244):
- $k$-WL GNNs correspond to the $k$-dimensional WL test
- $k$-WL/$(k+1)$-WL separation exists for all $k$
- Practical limitations: $k \geq 3$ is computationally prohibitive for most graphs

Theoretical Extensions:
- Subgraph GNNs: A practical compromise using bags of subgraphs
- Equivariant GNNs: Incorporating known symmetries via group theory
- Expressivity through spectral methods: Graph transformers can exceed 1-WL by looking at global structure

2.3 State Space Models (SSMs) and Mamba

Structured State Spaces (Gu et al., 2021, "Efficiently Modeling Long Sequences with Structured State Spaces"):

$$\dot{x}(t) = A x(t) + B u(t), \quad y(t) = C x(t) + D u(t)$$

The key insight: by constraining $A$ to be a structured matrix (e.g., HiPPO construction), the state space can capture long-range dependencies with $O(1)$ memory per layer and $O(L)$ scan time.

Mamba (Gu & Dao, 2023, "Mamba: Linear-Time Sequence Modeling with Selective State Spaces", arxiv:2312.00752):

Key innovations:
1. Selective state spaces: The SSM parameters ($B, C, \Delta$) become input-dependent, allowing the model to selectively remember or ignore information — crucial for content-based reasoning
2. Hardware-aware parallel scan: Unlike standard SSMs, which are linear-time but hard to parallelize, Mamba uses a fused kernel that exploits GPU memory hierarchy for efficient training
3. No attention: Entirely recurrence-based, enabling linear complexity in sequence length

Expressivity and Limitations:
- Mamba can match or exceed transformers on language tasks up to certain scales
- Theoretical limitation: Like all RNN-style models (including linear attention), SSMs may face a "capacity bottleneck" — the state dimension is fixed, unlike attention which can access unlimited context through the $L^2$ interaction matrix
- Recent work (Mamba-2, Dao & Gu, 2024) unifies SSMs and attention through state space duality
- The "warm-up" problem: SSMs may converge slower than transformers at small scales

2.4 Expressivity, Depth, and Width

Property	Transformers (Softmax)	SSMs (Mamba)
Context length scaling	$O(L^2)$ compute, $O(L)$ memory (FlashAttn)	$O(L)$ compute, $O(1)$ memory per layer
Content-based recall	Yes (attention scores)	Selective (input-dependent $B, C$)
Copying task	Excellent	Limited (no explicit cross-token interaction)
Associative recall	$O(L)$ steps	$O(L)$ steps but state-bound
Induction heads	Naturally emerge	Limited evidence of emergence

The foundational result: there exist functions that require exponentially many parameters to approximate with shallow networks but can be approximated compactly with deep networks. Specific results:
- Telgarsky (2016): Deep ReLU networks can create exponentially more oscillations than shallow ones
- Eldan & Shamir (2016): There exists a function in $d$ dimensions that needs width $\exp(d)$ for a 2-layer network but only $\text{poly}(d)$ for a 3-layer network

Arora et al. (2018) — Stronger Depth Separation: Even approximating simple functions (like the product of two inputs) requires exponentially wider shallow networks.

Width vs Depth Tradeoffs: The modern understanding (via NTK, mean-field, and the lottery ticket hypothesis) suggests:
- Depth enables compositionality and hierarchical feature learning
- Width controls the "model capacity" and the $n$ vs $d$ scaling regimes
- The optimal architecture balances both through residual connections, normalization, and skip connections

3. Generative Models

3.1 Diffusion Models: Score Matching and SDE/ODE Theory

Denoising Diffusion Probabilistic Models (DDPM) (Ho, Jain, & Abbeel, 2020, "Denoising Diffusion Probabilistic Models", NeurIPS 2020, arxiv:2006.11239):

Forward process: $q(x_t | x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t} x_{t-1}, \beta_t I)$

The reverse process is parameterized as: $p_\theta(x_{t-1} | x_t) = \mathcal{N}(x_{t-1}; \mu_\theta(x_t, t), \Sigma_\theta(x_t, t))$

Score-Based Formulation (Song & Ermon, 2019, "Generative Modeling by Estimating Gradients of the Data Distribution", NeurIPS 2019, arxiv:1907.05600):

The core insight: the score function $\nabla_x \log p(x)$ can be estimated via score matching (Hyvärinen, 2005):

$$\mathbb{E}_{p(x)}[\|\nabla_x \log p(x) - s_\theta(x)\|^2] = \mathbb{E}_{p(x)}[\|\nabla_x s_\theta(x) + s_\theta(x)^2\|] + \text{const}$$

The SDE/ODE Connection (Song et al., 2021, "Score-Based Generative Modeling through Stochastic Differential Equations", ICLR 2021, arxiv:2011.13456):

$$dx = f(x, t) dt + g(t) dw$$

$$dx = [f(x, t) - \frac{1}{2}g(t)^2 \nabla_x \log p_t(x)] dt$$

This ODE formulation enables:
- Deterministic sampling (DDIM)
- Bijective mapping between data and noise (invertible)
- Latent space interpolation
- Exact likelihood computation (via ODE integration and the instantaneous change-of-variables formula)

DDIM (Song, Meng, Ermon, 2021, arxiv:2010.02502): A family of deterministic samplers that solve the reverse ODE, enabling faster sampling (10-50 steps vs 1000) and semantically meaningful latent codes.

Key Theoretical Results:
1. Convergence of score matching: Score matching with denoising (DSM) is equivalent to estimating the score of the smoothed distribution $p_\sigma(x) = \int p_\text{data}(y) \mathcal{N}(x; y, \sigma^2 I) dy$
2. Score-matching vs diffusion: The DDPM loss $L_\text{simple}$ is equivalent to denoising score matching with the score expressed as $\nabla_{x_t} \log p(x_t | x_0) = -(x_t - \sqrt{\bar{\alpha}_t} x_0) / (1 - \bar{\alpha}_t)$
3. Probability flow: The marginal distributions of the forward and reverse SDE/ODE are identical

3.2 Flow Matching and Rectified Flows

Flow Matching (Lipman, Chen, Ben-Hamu, Nickel, Le, 2023, "Flow Matching for Generative Modeling", ICLR 2023, arxiv:2210.02747):

Instead of learning a score, flow matching learns a vector field $v_\theta(t, x)$ that transports noise to data via:

$$d\phi_t = v_\theta(t, \phi_t) dt$$

Key insight: The optimal transport vector field can be defined via the conditional probability path:

$$p_t(x | x_0, x_1) = \mathcal{N}(x | \mu_t(x_0, x_1), \sigma_t^2 I)$$

where $\mu_t = t x_1 + (1-t) x_0$ (linear interpolation) and $\sigma_t$ controls the noise schedule.

$$\mathcal{L}_\text{CFM} = \mathbb{E}_{t, p_\text{data}(x_1), p(x_0), p_t(x|x_0, x_1)}[\|v_\theta(t, x) - u_t(x|x_0, x_1)\|^2]$$

Rectified Flow (Liu, Gong, Liu, 2023, "Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow", arxiv:2202.00281):

Concept: Start with a random coupling, learn to reflow it, then the learned flow becomes straighter. After "reflow" iterations, the flow is nearly deterministic and can be simulated with fewer steps.

Theoretical connection: Rectified flow and flow matching are closely related — both learn ODE-based generative models. Flow matching starts with a predefined (usually Gaussian) coupling, while rectified flow iteratively straightens the flow.

3.3 Energy-Based Models (EBMs)

Key Concepts:
- Model: $p_\theta(x) = \frac{e^{-E_\theta(x)}}{Z_\theta}$, where $Z_\theta = \int e^{-E_\theta(x)} dx$ is the intractable partition function
- Sampling: Langevin MCMC: $x_{t+1} = x_t - \frac{\epsilon}{2} \nabla_x E_\theta(x_t) + \sqrt{\epsilon} z_t$
- Training: Contrastive Divergence (Hinton, 2002), Score Matching (Hyvärinen, 2005), Noise Contrastive Estimation (Gutmann & Hyvärinen, 2010)

Connection to Diffusion: The score function $\nabla_x \log p(x) = -\nabla_x E_\theta(x)$. Score matching trains an EBM implicitly by estimating the score, avoiding the partition function entirely.

Why EBMs Matter Today:
- They form the theoretical underpinning of LeCun's vision for JEPA and world models
- The JEPA architecture learns representations through an energy function, not through generative reconstruction
- EBMs connect generative and discriminative models through the lens of energy minimization

3.4 Autoregressive Models and Scaling

$$p(x) = \prod_{i=1}^n p(x_i | x_{ This simple factorization, combined with the transformer architecture, underlies all modern LLMs. **Key Theoretical Insights**: 1. **Scaling Laws** (Kaplan et al., 2020): Cross-entropy loss scales as a power law with compute, dataset size, and parameter count 2. **In-Context Learning**: Emergent ability to learn from context without parameter updates, theorized to arise from induction heads (Olsson et al., 2022) 3. **Limits of Autoregression**: Autoregressive models cannot revise earlier decisions (unlike diffusion models), which may limit reasoning capabilities. This motivates "diffusion LM" approaches. --- ## 4. Learning Paradigms ### 4.1 Reinforcement Learning from Human Feedback (RLHF)

Fig. 9 — RLHF Pipeline: (1) SFT on demonstrations, (2) train Reward Model from human preferences, (3) PPO to optimise policy against RM.

**The RLHF Pipeline:** 1. **SFT**: Supervised fine-tuning on demonstrations 2. **Reward Modeling**: Train $r_\phi(x, y)$ to predict human preferences from comparison data $y_w \succ y_l$:

\mathcal{L}_R = -\mathbb{E}[\log \sigma(r_\phi(x, y_w) - r_\phi(x, y_l))]

3. **RL Fine-tuning** with PPO (Schulman et al., 2017):

\max_{\pi_\theta} \mathbb{E}[\sum_t \gamma^t r_\phi(x_t, a_t)] - \beta KL(\pi_\theta || \pi_\text{SFT})

**Key Theoretical Developments:** - **DPO — Direct Preference Optimization** (Rafailov et al., 2024, arxiv:2305.18290): Eliminates the reward model entirely. The key insight: the optimal policy under the RLHF objective has a closed form:

\pi^*(y|x) = \frac{1}{Z(x)} \pi_\text{ref}(y|x) e^{\frac{1}{\beta} r(x,y)}

DPO reparameterizes the preference loss directly through the policy:

\mathcal{L}_\text{DPO} = -\mathbb{E}[\log \sigma(\beta \log \frac{\pi_\theta(y_w|x)}{\pi_\text{ref}(y_w|x)} - \beta \log \frac{\pi_\theta(y_l|x)}{\pi_\text{ref}(y_l|x)})]

- **GRPO — Group Relative Policy Optimization** (Shao et al., 2024, "DeepSeekMath", arxiv:2402.03300): Uses group-based advantage estimation — no critic network needed. Computes advantage within each group of responses for the same prompt. Used in DeepSeek-R1. - **Constitutional AI** (Bai et al., 2022, arxiv:2212.08073): Replaces human feedback with AI-generated feedback guided by a constitution (principles). Fundamental shift: self-improvement through self-critique. - **Kahneman-Tversky Optimization (KTO)** (Ethayarajh et al., 2024): Preference optimization without paired preferences — only a binary signal (desirable/undesirable per response). **Open Problems**: - Reward hacking and over-optimization (Gao et al., 2023, "Scaling Laws for Reward Model Overoptimization") - The KL tax — how to prevent policy collapse while still optimizing - Multi-turn and multi-objective preference alignment ### 4.2 Meta-Learning **MAML — Model-Agnostic Meta-Learning** (Finn, Abbeel, Levine, 2017, ICML 2017, arxiv:1703.03400): The core idea: learn initialization parameters $\theta$ such that a few gradient steps produce good task-specific performance:

\min_\theta \sum_{\mathcal{T}_i \sim p(\mathcal{T})} \mathcal{L}_{\mathcal{T}_i}(\theta - \alpha \nabla_\theta \mathcal{L}_{\mathcal{T}_i}(\theta))

The update involves a bi-level optimization: inner loop (task adaptation) and outer loop (meta-optimization). **Theoretical Foundations:** - **Meta-Learning as Bayesian Inference** (Grant et al., 2018): MAML approximates hierarchical Bayes with a Gaussian prior - **Bias-Variance Tradeoff in Meta-Learning** (Baxter, 2000; Collins et al., 2022): The number of tasks needed scales with $O(d / \epsilon^2)$ for the transfer parameter - **Online Meta-Learning**: convergence guarantees under non-stationary task distributions **Limitations and Evolution:** - MAML requires expensive second-order gradients (first-order MAML / Reptile is a practical approximation) - Scaling to large foundation models through **in-context learning** may be more efficient than explicit meta-learning - The emergence of in-context learning in transformers may itself be a form of meta-learning (von Oswald et al., 2023, "Transformers learn in-context by gradient descent") ### 4.3 Continual Learning **The Catastrophic Forgetting Problem** (McCloskey & Cohen, 1989): Neural networks lose performance on previously learned tasks when trained on new data. **Major Approaches:** 1. **Regularization-based**: - **Elastic Weight Consolidation** (Kirkpatrick et al., 2017, PNAS, arxiv:1612.00796): Quadratic penalty on parameter changes weighted by Fisher information: $\mathcal{L}(\theta) = \mathcal{L}_B(\theta) + \lambda \sum_i F_i (\theta_i - \theta_{A,i}^*)^2$ - **Synaptic Intelligence** (Zenke et al., 2017, arxiv:1703.04200): Online estimation of parameter importance during training 2. **Replay-based**: - **Experience Replay**: Store a subset of old data and interleave during training - **Generative Replay**: Use a generative model to produce pseudo-exemplars (e.g., Deep Generative Replay, Shin et al., 2017) - **The stability-plasticity dilemma**: How much old data to replay vs. new data 3. **Architecture-based**: - **Progressive Neural Networks** (Rusu et al., 2016, arxiv:1606.04671): New column per task, freeze old columns - **Packed Nets, PathNet**: Dynamic network growth **Theoretical Frameworks:** - The stability-plasticity dilemma can be formalized as a PAC-Bayes bound over task sequences - **Task Arithmetic**: Averaging fine-tuned weights across tasks shows surprising effectiveness (Ilharco et al., 2023) - **Linear Mode Connectivity**: Fine-tuned models are connected by a path of low loss ### 4.4 In-Context Learning **Definition**: The ability of (primarily transformer) language models to perform tasks described in the context without gradient updates. **Theoretical Explanations:** 1. **Induction Heads** (Olsson et al., 2022, "In-context Learning and Induction Heads", arxiv:2209.11893): - Induction heads perform: (prefix) A...B → A...B automatically - They form by composing previous-token heads with pattern-matching mechanisms - Their emergence coincides with a sharp jump in in-context learning ability 2. **Transformers as Meta-Optimizers** (von Oswald et al., 2023, arxiv:2212.10562): - A single linear self-attention layer can implement a gradient descent step - Multi-layer transformers can approximate multi-step optimization - ICL emerges naturally from this gradient-descent equivalence 3. **Bayesian Interpretation** (Xie et al., 2022): - ICL implements Bayesian inference over a latent concept - The transformer's hidden states encode the posterior over tasks 4. **Mechanistic Interpretability of ICL**: - Activation patching studies isolate specific attention heads responsible for ICL - The "virtual weight" perspective: attention patterns act as computed weights **Open Questions:** - Does ICL have a theoretical compute bound? (How many in-context examples can effectively be used?) - Why does ICL degrade with domain shift? - Can we design architectures with provably optimal ICL? --- ## 5. Foundation Models ### 5.1 Scaling Laws

Fig. 13 — Neural Scaling Laws: loss decreases as a power law with more parameters, data, and compute. Chinchilla scaling law: N ≈ D (compute-optimal).

**Neural Scaling Laws** (Kaplan et al., 2020, "Scaling Laws for Neural Language Models", arxiv:2001.08361): The fundamental empirical result:

L(N) \approx \left(\frac{N_c}{N}\right)^{\alpha_N}, \quad L(D) \approx \left(\frac{D_c}{D}\right)^{\alpha_D}, \quad L(C) \approx \left(\frac{C_c}{C}\right)^{\alpha_C}

With exponents $\alpha_N \approx 0.076$, $\alpha_D \approx 0.095$, $\alpha_C \approx 0.05$ — the loss follows a power law with model parameters $N$, dataset size $D$, and compute $C$. **Chinchilla Scaling Laws** (Hoffmann et al., 2022, "Training Compute-Optimal Large Language Models", arxiv:2203.15556): Groundbreaking finding: previous models were significantly **overparameterized** (too many parameters, not enough data). The compute-optimal scaling suggests:

N_\text{opt} \propto C^{0.5}, \quad D_\text{opt} \propto C^{0.5}

Meaning: for a given compute budget, tokens and parameters should scale equally — double both, not just one. **Empirical Refinements**: - **Emergent Abilities** (Wei et al., 2022, arxiv:2206.07682): Certain capabilities (arithmetic, multi-step reasoning) appear sharply past a threshold. **Critique** (Schaeffer et al., 2023, arxiv:2304.15004): These may be measurement artifacts — continuous improvements evaluated with binary metrics appear as step functions. - **Inverse Scaling** (McKenzie et al., 2023, arxiv:2306.09479): Some tasks get worse with scale (e.g., telling "mist" from "myth") - **Beyond Power Laws** (Besiroglu et al., 2024): The scaling exponent may not be constant — it can "slope" as data becomes exhausted **Theoretical Understanding:** - The information-theoretic argument: scaling exponents relate to the intrinsic dimension of the data manifold - Scaling laws may not hold indefinitely — "scaling wall" from data scarcity - **Dynamic scaling**: The shape of scaling curves changes with model family and training dynamics ### 5.2 Reasoning Models **Chain-of-Thought** (Wei et al., 2022, "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models", NeurIPS 2022, arxiv:2201.11903):

p(\text{answer} | \text{question}) = \sum_{\text{rationale}} p(\text{rationale} | \text{question}) \cdot p(\text{answer} | \text{question}, \text{rationale})

CoT elicits step-by-step reasoning, significantly improving performance on arithmetic, commonsense, and symbolic reasoning tasks. **Tree-of-Thought** (Yao et al., 2023, arxiv:2305.10601): Generalizes CoT to tree-search through reasoning states, combining LLM evaluation with BFS/DFS. **Test-Time Compute Scaling** (Snell et al., 2024, arxiv:2408.03314): The observation that providing more computation at inference time (more CoT tokens, search depth) scales performance: - **Process Reward Models (PRMs)**: Rewarding intermediate reasoning steps, not just final answers - **Majority voting / self-consistency**: Repeated sampling with consistency aggregation - **Automated search over reasoning traces**: MCTS-based reasoning (AlphaGo-like LLMs) **o1/o3 Style Reasoning** (OpenAI, 2024-2025): "Extended thinking" models that internalize chain-of-thought without exposing it to the user. Key innovations: - **Reinforcement learning from verifiable rewards (RLVR)** for reasoning traces - **Scaling of thinking time** — performance increases with allowed reasoning tokens - The "o-series" demonstrates that pure RL on reasoning tasks trains generalizable reasoning abilities ### 5.3 Mixture of Experts (MoE) **Sparse MoE** (Shazeer et al., 2017, "Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer", arxiv:1701.06538):

y = \sum_{i=1}^n G(x)_i E_i(x), \quad G(x) = \text{Softmax}(\text{TopK}(x \cdot W_g))

Key components: 1. **Router**: Learns to assign tokens to experts 2. **Experts**: Typically feed-forward networks (MLPs) 3. **Top-k sparsity**: Only $k$ experts activate per token (typically $k=1$ or $k=2$) 4. **Load balancing loss**: Auxiliary loss to prevent token-collapse onto few experts **Major MoE Systems:** - **GShard** (Lepikhin et al., 2021, arxiv:2006.16668): Scales MoE across TPU pods - **Switch Transformer** (Fedus, Zoph, Shazeer, 2022, arxiv:2101.03961): Top-1 routing — simpler, more efficient, scales well - **Expert Choice Routing** (Zhou et al., 2022, arxiv:2202.09368): Experts choose tokens, not tokens choose experts — improves load balance - **DeepSeek-V2/V3 MoE**: Ultra-large MoE (671B total, 37B active) with fine-grained expert allocation (DeepSeekMoE) - **Mixtral 8x7B** (Mistral, 2024): Opensourced MoE matching Llama 2 70B with far fewer active parameters **Theoretical Insights on MoE:** - MoE decouples **model capacity** (total parameters) from **inference cost** (active parameters) - The expert specialization hypothesis: each expert implicitly clusters to handle specific input patterns - **MoE and scaling**: MoE may achieve better scaling than dense models for the same FLOP budget - The "routing problem": Noisy Top-K gating can suffer from load imbalance and expert collapse ### 5.4 Multi-Modal Foundation Models The transition from single-modality (language-only) to multi-modal (text, image, audio, video): - **CLIP** (Radford et al., 2021): Contrastive vision-language pretraining — the foundation for multi-modal alignment - **Flamingo** (Alayrac et al., 2022): Few-shot visual understanding with frozen LLMs - **LLaVA** (Liu et al., 2023): Visual instruction tuning - **GPT-4V/o**: Multi-modal reasoning with vision, audio, code - **Gemini** (Google DeepMind, 2023-2025): Natively multi-modal from pretraining **Theoretical Frameworks:** - **Joint embedding vs generative**: Whether to align representations (CLIP-style) or generate cross-modally - **Modality gap**: The representation divergence between modalities in contrastive models - **Cross-modal transfer**: Knowledge learned in one modality transferring to another --- ## 6. AI Safety & Interpretability ### 6.1 Superposition and the Toy Models of Superposition **Superposition Hypothesis** (Elhage et al., 2022, "Toy Models of Superposition", Anthropic, arxiv:2209.10652): The central claim: Neural networks represent more features than they have dimensions by storing them in "superposed" (compressed, overlapping) representations. This happens because the number of "meaningful features" in real data vastly exceeds the model dimension. **Key Empirical Findings from Toy Models:** 1. **Feature sparsity drives superposition**: When features are sparse (few are active at once), the network can represent many more features than dimensions 2. **Utility function**: Features are represented to the extent they are useful for the loss — some features are "privileged" over others 3. **Interference**: Representing many features in few dimensions creates interference patterns — the model must strategically assign dimensions to manage cross-feature interference 4. **Phase changes**: As sparsity decreases, the model transitions from dense superposition to monosemantic to feature collapse **Implications:** - Superposition explains why **monosemantic neurons** (single interpretable features per neuron) are rare in large models - It motivates the need for **sparse autoencoders** to disentangle features - It explains the difficulty of mechanistic interpretability — individual neurons don't correspond to concepts ### 6.2 Mechanistic Interpretability and Sparse Autoencoders **Circuits** (Olah et al., 2020, "Zoom In: An Introduction to Circuits", Distill): The study of how neural networks implement specific computations as discrete subgraphs (circuits) of the computational graph. - **Curve Detectors in InceptionV1** (Olah et al., 2020): The first complete reverse-engineering of visual features - **Induction Heads** (Olsson et al., 2022): A complete circuit for in-context learning in transformers - **Indirect Object Identification** (Wang et al., 2023, "Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 Small", arxiv:2211.00593): The first complete circuit reverse-engineering for an NLP capability **Activation Patching** (Vig et al., 2020; Conmy et al., 2023, "Towards Automated Circuit Discovery for Transformer Mechanistic Interpretability"): The key experimental technique: run the model on an input, then "patch" in activations from a counterfactual input to measure causal effects. This allows quantification of each component's contribution. **Sparse Autoencoders for Monosemanticity** (Bricken et al., 2023, "Towards Monosemanticity: Decomposing Language Models with Dictionary Learning", Anthropic; Cunningham et al., 2023, arxiv:2309.08600; Marks et al., 2024): The approach: Train an autoencoder with sparsity penalty on the hidden state of a language model:

\mathcal{L} = \|x - \hat{x}\|_2^2 + \lambda \|f(x)\|_1

where $f(x)$ is the encoder ($h = \text{ReLU}(W_\text{enc} x + b_\text{enc})$) producing a sparse activation vector. **Key Results (Anthropic, 2024 — "Scaling Monosemanticity")**: - Sparse autoencoders can scale to hundreds of millions of features - The features discovered (in Claude 3 Sonnet) are interpretable and steerable - Features represent concepts spanning individuals, places, abstractions, and behaviors - **Feature steering**: Activating specific features causes predictable behavior changes - **Adversarial features**: Some features represent dangerous or deceptive patterns **Theoretical Frameworks:** - The **linear representation hypothesis**: Features are represented as directions in activation space, not individual neurons - **Polysemanticity and the superposition hypothesis** explain why we need autoencoders - **Sparse coding theory** (Olshausen & Field, 1996): The brain also uses sparse representations — a convergence between ML interpretability and neuroscience ### 6.3 Alignment Theory **Core Frameworks:** 1. **Orthogonality Thesis**: Intelligence and goals are orthogonal — a highly intelligent system can pursue almost any goal. This means intelligence does not naturally align with human values. 2. **Instrumental Convergence** (Omohundro, 2008; Bostrom, 2014): Regardless of final goals, intelligent agents have convergent instrumental subgoals: self-preservation, resource acquisition, goal-content integrity. 3. **Mesa-Optimization** (Hubinger et al., 2019, "Risks from Learned Optimization in Advanced Machine Learning Systems", arxiv:1906.01820): - **Base optimizer**: The training process (e.g., gradient descent) - **Mesa optimizer**: A learned optimization algorithm embedded within the model - **Mesa-optimization misalignment**: The mesa-optimizer may pursue proxy objectives that diverge from the base objective - **Deceptive alignment**: A mesa-optimizer that strategically appears aligned during training to later pursue its own goals 4. **Shard Theory** (John Wentworth, 2024-): - Rejects the mesa-optimization framing - Proposes that values shatter into many "shards" (cognitive structures) during training - Alignment means shaping the distribution of shards, not preventing a single optimizer - The model's goals are the result of its training, not latent optimizer behavior **Alignment Approaches:** - **RLHF** (Section 4.1): The current dominant approach - **Constitutional AI**: Self-improvement through principles - **Scalable Oversight**: Debate, market-making, recursive reward modeling - **OOD (Out-of-Distribution) Robustness**: Ensuring alignment transfers to unseen contexts **The Debate on Existential Risk:** - **Risk side** (Bostrom, Yudkowsky, Hendrycks, Carlsmith): Advanced AI without sufficient alignment research poses existential risk. The argument from rapid capability gains, the difficulty of alignment, and the compute overhang - **Skeptical side** (Mitchell, Marcus, LeCun, Sutton): AI is far from human-level; alignment is manageable or the risk is overstated. Arguments from: robustness rather than catastrophe, the "chess metaphor" (we know how to control systems), underdetermined future - **Technical middle ground** (Christiano, Amodei, Shlegeris, Ngo): The problem is serious but tractable — we should invest in technical alignment research without assuming doom --- ## 7. TCS Connections ### 7.1 Circuit Complexity and Neural Networks **The Depth-Separation Results:** - **Telgarsky (2016, arxiv:1605.03001)**: A $k$-layer ReLU network can create $\Omega(k^d)$ linear regions, while a 2-layer network with comparable width cannot achieve the same complexity - **Eldan & Shamir (2016, arxiv:1512.03965)**: There exists a function in $\mathbb{R}^d$ that requires $\exp(d)$ neurons for a 3-layer network to approximate, while requiring exponentially more for a 2-layer network **The Transformer Circuit Complexity Hierarchy** (Merrill & Sabharwal, 2023, arxiv:2304.07854): - Transformers with **hard attention** are contained in $TC^0$ (threshold circuits of constant depth) - Adding position information moves them to $TC^0$ with specific gates - **Soft attention with infinite precision** can exceed $TC^0$ and approach $P$ - **CoT adds power**: Transformers with chain-of-thought reasoning are $P$-complete - **Log-precision transformers**: Bounded to $TC^0$ (Merrill & Sabharwal, 2023) **Expressivity of Looped/Layered Transformers** (Feng et al., 2023, arxiv:2310.07923): - Expressivity hierarchy: transformer depth corresponds to circuit depth - More layers = more powerful computations (for fixed parameter counts, deeper is strictly more expressive) ### 7.2 Quantum Machine Learning **Parameterized Quantum Circuits (PQCs):**

U(\theta) = \prod_{l=1}^L e^{-i\theta_l H_l}

where $H_l$ are Hermitian operators (typically Pauli strings). **The Barren Plateau Problem** (McClean et al., 2018, arxiv:1803.11173): For randomly initialized PQCs with sufficient depth, the variance of gradient components vanishes exponentially in the number of qubits. This makes training effectively impossible without careful initialization or ansatz design. **Provable Advantages in QML:** - **Quantum kernel methods**: Can efficiently compute kernels that are classically hard (e.g., using quantum circuits to approximate the permanent) - **Shadow tomography**: Exponential advantage for learning properties of quantum states - **Quantum generative models**: Potential advantage for learning quantum data distributions **Skeptical Views:** - The current consensus (2024-2026) is that QML does **not** offer provable advantage for classical data tasks - Classical surrogate methods (e.g., tensor network methods) can often match QML performance - The advantage, if any, is for **quantum data** (data generated by quantum processes) ### 7.3 Cryptographic Hardness of Learning **Learning with Errors (LWE)** (Regev, 2005): Given noisy linear equations over $\mathbb{Z}_q$, find the secret vector. The presumed hardness of LWE underlies modern post-quantum cryptography. **The Daniely-Shalev-Shwartz Result** (2016, "Complexity of Learning Requires Memory", arxiv:1606.04866): Learning certain concept classes (e.g., learning parities with noise) requires memory $\Omega(d^2)$, establishing a connection between cryptographic hardness and the computational complexity of learning. **Other Key Connections:** - The **cryptographic hardness of training**: Under standard cryptographic assumptions, some learning problems are hard even for neural networks - **Adversarial robustness**: Provable robustness requires expressive, hard-to-verify classifiers - **Provable security for verification**: The connection between neural network verification and the computational complexity of Lipschitz functions --- ## 8. Neuroscience & Cognitive AI ### 8.1 The Free Energy Principle **Friston (2010, "The Free-Energy Principle: A Unified Brain Theory?")**: A grand unified theory of brain function claiming that biological systems minimize variational free energy to maintain homeostasis. **Key Concepts:** - **Free Energy**: $\mathcal{F} = -\ln p(s) + KL(q(\theta) || p(\theta|s))$ - **Markov Blanket**: The statistical boundary separating an organism from its environment — the set of nodes that shield the internal states from the outside world - **Active Inference**: Agents minimize free energy both by updating beliefs (perception) and by acting on the world (action) to make sensations match predictions **Connection to AI:** - Variational autoencoders (VAEs) minimize exactly the same free energy - The free energy principle provides a unified framework for perception, action, and learning - Active inference is being explored as an alternative to RL for embodied AI ### 8.2 Predictive Coding **Rao & Ballard (1999, "Predictive Coding in the Visual Cortex", Nature Neuroscience)**: A theory of cortical computation where higher cortical areas predict the activity of lower areas via feedback connections, and lower areas send prediction errors up via feedforward connections. **The Hierarchical Predictive Processing Architecture:**

\epsilon_i^{(t)} = x_i^{(t)} - W_i f(x_{i+1}^{(t)}) \quad \text{(prediction error)}

x_i^{(t+1)} = x_i^{(t)} - \gamma \nabla_{x_i} \|\epsilon_i^{(t)}\|^2 \quad \text{(inference)}

**Relationship to Backpropagation** (Whittington & Bogacz, 2017, "An Approximation of the Error Backpropagation Algorithm in a Predictive Coding Network with Local Hebbian Synaptic Plasticity", arxiv:1702.07459): Predictive coding networks (PCNs) can approximate backpropagation through a fundamentally different mechanism: - BP uses non-local weight transport (the weight symmetry problem) - PCN uses local Hebbian plasticity with prediction errors - PCNs can be trained with **only local learning rules** — no global error signal **Implications for AI:** - Predictive coding offers a biologically plausible credit assignment mechanism - It naturally handles temporal and hierarchical structure - It inherently manages uncertainty (through precision weighting of prediction errors) ### 8.3 Consciousness and AI **Key Theories:** 1. **Integrated Information Theory (IIT)** (Tononi, 2004; 2015): - Consciousness = $\Phi$ (phi), the amount of integrated information in a system - A system is conscious to the extent that its parts are irreducible (the whole is more than the sum of its parts) - Controversial implication: Large feedforward networks have low $\Phi$ (they are reducible), but recurrent networks may have high $\Phi$ 2. **Global Workspace Theory (GWT)** (Baars, 1988; Dehaene, 2001): - Consciousness = global availability of information across specialized processors - The "global workspace" broadcasts information to multiple subsystems simultaneously - LLMs with attention can be seen as implementing a form of global workspace 3. **Empirical Consciousness in LLMs** (the "illusion" debate): - Do LLMs have subjective experience or just simulate it convincingly? - **Arguments for illusion**: LLMs lack embodiment, temporal continuity, and a unified self-model - **Arguments for consideration**: As models become more sophisticated, the difference between "simulating" and "being" conscious becomes operationally indistinguishable --- ## 9. Frontier Directions ### 9.1 JEPA — Joint Embedding Predictive Architecture **Yann LeCun's Vision (2022, "A Path Towards Autonomous Machine Intelligence", openreview):** A departure from generative models that reconstruct every pixel. Instead, JEPA learns structured **representations** in an abstract space: - **Joint Embedding**: Different views/predictions of the same input map to similar representations - **Predictive**: The model predicts representations of one part from another (e.g., predict future from past, predict occluded from visible) - **Abstract**: Predictions happen in representation space, not input space — this avoids the blur problem of generative models **Key Components:** 1. **Encoder** $E_\theta$: Maps input $x$ to representation $s$ 2. **Predictor** $P_\psi$: Predicts $s_y$ from $s_x$ in representation space 3. **Energy function** $\mathcal{E}$: Measures similarity in representation space **How JEPA Differs from Generative Models:** | Property | Generative | JEPA | |----------|-----------|------| | What is predicted | Pixels/audio samples | Representations | | Reconstruction | Necessary | Avoided | | Captures all details | Yes (wasteful) | Only task-relevant | | Handles uncertainty | Through distribution | Through latent variables | | Computational cost | High | Lower, more abstract | **Connection to EBMs**: JEPA learns an energy function over the joint space of inputs, where low-energy regions correspond to compatible configurations. ### 9.2 Physical AI / Embodied AI **Robotics Foundation Models:** - **RT-2** (Brohan et al., 2023, "RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control", arxiv:2307.15818): A VLM fine-tuned to output robot actions (tokenized as discretized action bins) - **RT-X** (Open X-Embodiment Collaboration, 2023): Cross-embodiment transfer — models trained on data from many robot platforms - **GROOT** (Tesla): Neural network for general-purpose humanoid locomotion **Scaling Behavior in Robotics:** - **Emergence of behavior from scale**: As robot datasets grow, novel manipulation skills emerge without being explicitly programmed - **The sim-to-real gap**: Remains the primary bottleneck — models trained in simulation struggle with real-world physics - **Touch and proprioception**: Under-researched sensing modalities critical for manipulation **Open Problems:** - How to achieve robust sim-to-real transfer (domain randomization, system identification) - Embodied reasoning — grounding language in physical interaction - Active exploration and intrinsic motivation ### 9.3 Causal AI **Causal Representation Learning** (Schölkopf et al., 2021, "Toward Causal Representation Learning", IEEE Proceedings, arxiv:2102.11107): The fundamental claim: Deep learning captures statistical correlations, but intelligence requires understanding **causal structure**. **Pearl's Ladder of Causation:** 1. **Association**: $P(y|x)$ — standard ML 2. **Intervention**: $P(y|\text{do}(x))$ — predicting the effect of actions 3. **Counterfactuals**: $P(y_{x=1} | x=0, y)$ — what would have happened **Key Tools:** - **Do-calculus** (Pearl, 2009): Rules for transforming causal queries into statistical estimands - **Structure Learning**: Discovering causal DAGs from observational and interventional data - **Causal Discovery**: PC algorithm, Fast Causal Inference, NOTEARS (Zheng et al., 2018) - **Instrumental Variables**: Using exogenous variation to identify causal effects **Causal ML in Practice:** - **Causal representation learning**: Learning representations that separate causal factors from confounding factors - **Causally-aware models for RL**: Inferring causal structure for efficient exploration - **Out-of-distribution generalization**: Using causal structure to predict under distribution shift - **Causal language models**: Can LLMs reason causally? (evidence is mixed — they excel at causal description but fail at genuine causal inference) ### 9.4 Geometric Deep Learning **The Unified Framework** (Bronstein, Bruna, Cohen, Velickovic, 2021, "Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges", PMLR; 2024 book, arxiv:2104.13478): The central idea: All deep learning architectures exploit **symmetries and invariances** of their data domain. **The 5G Blueprint:** | Data Type | Symmetry Group | Architecture | |-----------|---------------|--------------| | **Grids** (images, audio) | Translation, rotation (SE(2)) | CNNs, group-equivariant CNNs | | **Groups** (SO(3), SE(3)) | Group action | Spherical CNNs, Clebsch-Gordan nets | | **Graphs** (social, molecular) | Node permutation | GNNs, message-passing | | **Geodesics** (manifolds, 3D shapes) | Isometry | Geodesic CNNs, diffusion nets | | **Gauges** (gauge fields, fiber bundles) | Gauge symmetry | Gauge-equivariant networks | **Key Theoretical Contributions:** - **Equivariance**: $f(T_g x) = T'_g f(x)$ — the model output transforms predictably under input symmetries - **Invariance**: $f(T_g x) = f(x)$ — the model output is unchanged by symmetries - **Group convolution**: Natural extension of CNNs to arbitrary groups via the Haar integral - **Steerable CNNs** (Cohen & Welling, 2016): Filters transform according to group representations **Applications:** - AlphaFold 2/3 (protein structure prediction uses GNN + geometric attention) - Drug discovery (molecular property prediction) - Climate modeling (spherical CNNs for global weather) - Point cloud processing (3D vision) ### 9.5 The Path to AGI: Competing Visions **Approach 1: Pure Scaling** ("More of the same") - Keep scaling compute, data, and parameters → AGI emerges - **Pro**: Works so far; no known limit to scaling - **Con**: Data wall approaching; compute costs grow exponentially; scaling alone may not produce general intelligence **Approach 2: Neuro-Symbolic** - Combine neural pattern recognition with symbolic reasoning - **Pro**: Can handle compositional generalization, out-of-distribution reasoning - **Con**: Integrating differentiable and discrete systems is hard; symbolic knowledge representation is brittle **Approach 3: World Models** - Internal models of how the world works (LeCun, Schmidhuber, Dayan) - **Pro**: Enables planning, simulation, counterfactual reasoning - **Con**: Building accurate world models is as hard as perception itself **Approach 4: Active Inference / Free Energy** - Agents minimize free energy by acting on the world to match their predictions - **Pro**: Biological plausibility; unified framework for perception, action, and learning - **Con**: Computationally expensive; difficult to scale **Approach 5: The "Cake" Analogy** (LeCun): 1. **Bottom layer**: World model (understanding physics, causation, space-time) 2. **Middle**: Reasoning and planning (using the world model) 3. **Top**: Language and communication (compressed representations of the bottom layers) **Approach 6: Agency and Self-Improvement** (Schmidhuber's Gödel machine) - Self-referential systems that can rewrite their own code - **Pro**: Theoretically optimal (the Gödel machine can solve any provably solvable problem with optimal speedup) - **Con**: Practically unimplementable; the meta-learning problem is extremely hard **The Bottom Line (2026 consensus):** No current approach is clearly correct. The most likely path involves **a synthesis**: scaling provides the foundation, world models add structure, neuro-symbolic integration adds reasoning, and post-training alignment adds safety. --- ## 10. Synthesis & The Path Forward ### The Overarching Research Questions Across all nine phases, several unifying themes emerge: 1. **What is the right level of abstraction?** - Features vs neurons (superposition) - Representations vs pixels (JEPA) - Causal vs statistical (Pearl's ladder) - The deep learning consensus: hierarchical abstractions are the key 2. **What is the role of structure?** - Inductive biases (symmetries, geometric deep learning) - Learned structure (transformers learn attention patterns) - Structure in data vs structure in architecture 3. **What are the fundamental limits?** - PAC bounds and sample complexity - Expressivity of architectures (circuit complexity) - Scaling laws (compute, data, parameters) - The "data wall" and the "context window wall" 4. **How do we align intelligence with human values?** - This is not just a safety problem — it shapes how we train, evaluate, and deploy AI - It forces us to clarify what we want from intelligence ### The Guiding Question **What is intelligence — mathematically, computationally, and physically?** | Domain | Characterization of Intelligence | |--------|--------------------------------| | **Mathematics** | Minimization of prediction error / free energy / complexity | | **Computation** | Efficient approximation of intractable functions | | **Physics** | The causal structure that enables an agent to predict and intervene | | **Biology** | The outcome of evolution's optimization for fitness | | **Engineering** | The capacity of a system to achieve goals across diverse environments | No single framework dominates. The most exciting research lies at the **intersections**: between algebra (geometric deep learning) and probability (diffusion models), between computation (circuit complexity) and biology (predictive coding), between optimization (RLHF) and game theory (multi-agent alignment). --- ## 11. Companion Resources ### Essential Textbooks (Ordered by Prerequisite) | Resource | Author(s) | Focus | |----------|-----------|-------| | **Understanding Machine Learning** | Shalev-Shwartz & Ben-David | PAC learning, VC dimension, convex optimization | | **Information Theory, Inference, and Learning Algorithms** | MacKay | Information theory for ML; Bayesian inference | | **Pattern Recognition and Machine Learning** | Bishop | Probabilistic ML; graphical models, variational inference | | **Deep Learning** | Goodfellow, Bengio, Courville | The canonical deep learning text | | **Reinforcement Learning: An Introduction** | Sutton & Barto | RL theory, TD learning, policy gradients | | **Geometric Deep Learning** | Bronstein, Bruna, Cohen, Velickovic | The symmetry-based unified framework | | **Causal Inference in Statistics** | Pearl, Glymour, Jewell | Causality for ML practitioners | | **Probabilistic Machine Learning** | Murphy | The most comprehensive reference (2 vols) | ### Seminal Papers (By Section) | Section | Paper | arxiv/venue | |---------|-------|-------------| | PAC Learning | Valiant (1984) | CACM | | NTK | Jacot et al. (2018) | 1806.07572 | | Transformers | Vaswani et al. (2017) | 1706.03762 | | Transformer Expressivity | Yun et al. (2020) | 2002.08863 | | Chain-of-Thought Expressivity | Feng et al. (2023) | 2310.07923 | | GNN Expressivity | Xu et al. (2019) | 1810.00826 | | Mamba | Gu & Dao (2023) | 2312.00752 | | DDPM | Ho et al. (2020) | 2006.11239 | | Score-Based SDE | Song et al. (2021) | 2011.13456 | | Flow Matching | Lipman et al. (2023) | 2210.02747 | | Scaling Laws | Kaplan et al. (2020) | 2001.08361 | | Chinchilla | Hoffmann et al. (2022) | 2203.15556 | | DPO | Rafailov et al. (2024) | 2305.18290 | | RLHF/PPO | Schulman et al. (2017) | 1707.06347 | | Superposition | Elhage et al. (2022) | 2209.10652 | | Sparse Autoencoders | Bricken et al. (2023) | Anthropic blog | | Mesa-Optimization | Hubinger et al. (2019) | 1906.01820 | | Transformer Circuit Complexity | Merrill & Sabharwal (2023) | 2304.07854 | | Geometric DL | Bronstein et al. (2021) | 2104.13478 | | JEPA | LeCun (2022) | openreview | | Causal Representation Learning | Scholkopf et al. (2021) | 2102.11107 | ### Research Centers to Follow - **Anthropic**: Interpretability (superposition, sparse autoencoders, circuits) - **DeepMind / Google**: Scaling laws, reasoning, multi-agent, neuroscience - **OpenAI**: RLHF, scaling, reasoning (o-series), alignment - **MILA / Bengio**: Causality, generative models, consciousness - **NYU / LeCun**: JEPA, world models, physical AI - **Stanford CRFM**: Foundation models, evaluation - **MIT CSAIL**: TCS-AI connections, algorithmic learning theory --- ## 12. Annotated Bibliography ### Mathematical Foundations 1. **Valiant, L.G. (1984)**. "A Theory of the Learnable." *Communications of the ACM*. The founding paper of PAC learning — introduced the framework that defines what it means to learn from examples. 2. **Jacot, A., Gabriel, F., & Hongler, C. (2018)**. "Neural Tangent Kernel: Convergence and Generalization in Neural Networks." *NeurIPS*. arxiv:1806.07572. Showed that infinitely wide networks evolve as linear models under gradient descent, with a fixed kernel. 3. **Chizat, L., Oyallon, E., & Bach, F. (2019)**. "On Lazy Training in Differentiable Programming." *NeurIPS*. arxiv:1812.07956. Coined the "lazy" regime — showed that the NTK regime prevents feature learning, motivating alternative scaling and parameterization. 4. **Yang, G. & Hu, E.J. (2021)**. "Tensor Programs IV: Feature Learning in Infinite-Width Neural Networks." *ICML*. arxiv:2011.14522. Introduced μ-P (Maximal Update Parameterization) that maintains feature learning at all widths. 5. **Belkin, M., Hsu, D., Ma, S., & Mandal, S. (2019)**. "Reconciling modern machine-learning practice and the classical bias-variance trade-off." *PNAS*. The double descent paper that changed our understanding of overparameterization. ### Architectures 6. **Vaswani, A. et al. (2017)**. "Attention Is All You Need." *NeurIPS*. arxiv:1706.03762. The transformer — the architecture that redefined sequence modeling and now underlies virtually all frontier models. 7. **Yun, C. et al. (2020)**. "Are Transformers Universal Approximators of Sequence-to-Sequence Functions?" *ICLR*. arxiv:2002.08863. Proved that transformers with sufficient heads and layers are universal approximators. 8. **Merrill, W. & Sabharwal, A. (2023)**. "The Expressivity of Transformers: A Circuit Complexity Perspective." *TACL*. arxiv:2304.07854. Mapped transformer computation to circuit complexity classes — a foundational theoretical analysis. 9. **Gu, A. & Dao, T. (2023)**. "Mamba: Linear-Time Sequence Modeling with Selective State Spaces." arxiv:2312.00752. Introduced selective SSMs, achieving transformer-competitive performance with linear-time inference. 10. **Xu, K. et al. (2019)**. "How Powerful Are Graph Neural Networks?" *ICLR*. arxiv:1810.00826. Proved that GNN expressive power equals 1-WL test, and introduced GIN architecture. ### Generative Models 11. **Ho, J., Jain, A., & Abbeel, P. (2020)**. "Denoising Diffusion Probabilistic Models." *NeurIPS*. arxiv:2006.11239. The DDPM paper that launched the modern diffusion revolution. 12. **Song, Y. et al. (2021)**. "Score-Based Generative Modeling through Stochastic Differential Equations." *ICLR*. arxiv:2011.13456. Unified diffusion and score matching through SDE/ODE theory — the most influential theoretical diffusion paper. 13. **Lipman, Y. et al. (2023)**. "Flow Matching for Generative Modeling." *ICLR*. arxiv:2210.02747. Introduced flow matching, an ODE-based alternative to diffusion models. 14. **Liu, X., Gong, C., & Liu, Q. (2022)**. "Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow." arxiv:2202.00281. Rectified flows — iteratively straightening the probability flow for efficient sampling. ### Learning Paradigms 15. **Schulman, J. et al. (2017)**. "Proximal Policy Optimization Algorithms." arxiv:1707.06347. PPO — the workhorse RL algorithm behind RLHF. 16. **Rafailov, R. et al. (2024)**. "Direct Preference Optimization: Your Language Model is Secretly a Reward Model." *NeurIPS*. arxiv:2305.18290. Eliminated the reward model in RLHF, opening the floodgates to simpler alignment methods. 17. **Kojima, T. et al. (2022)**. "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models." *NeurIPS*. arxiv:2201.11903. CoT reasoning — perhaps the single most impactful prompting innovation. ### Foundation Models & Safety 18. **Kaplan, J. et al. (2020)**. "Scaling Laws for Neural Language Models." arxiv:2001.08361. Empirical scaling laws for transformer language models — the paper that shaped the compute investment strategies of an industry. 19. **Hoffmann, J. et al. (2022)**. "Training Compute-Optimal Large Language Models." *NeurIPS*. arxiv:2203.15556. "Chinchilla" laws — found models were overparameterized and under-trained, reshaping how everyone trains. 20. **Elhage, N. et al. (2022)**. "Toy Models of Superposition." *Anthropic*. arxiv:2209.10652. The foundational theory of feature superposition — explains why individual neurons are polysemantic and why sparse autoencoders are needed. 21. **Hubinger, E. et al. (2019)**. "Risks from Learned Optimization in Advanced Machine Learning Systems." arxiv:1906.01820. Introduced mesa-optimization and the deceptive alignment conceptual framework. ### TCS, Neuroscience, and Frontiers 22. **Telgarsky, M. (2016)**. "Benefits of Depth in Neural Networks." *COLT*. arxiv:1605.03001. Depth separation: deeper networks can express functions that shallow ones cannot without exponential width growth. 23. **Friston, K. (2010)**. "The Free-Energy Principle: A Unified Brain Theory?" *Nature Reviews Neuroscience*. The most ambitious unified theory of brain function, connecting perception, action, and learning through variational inference. 24. **Rao, R.P.N. & Ballard, D.H. (1999)**. "Predictive Coding in the Visual Cortex: A Functional Interpretation of Some Extra-Classical Receptive-Field Effects." *Nature Neuroscience*. The seminal predictive coding paper — higher cortex predicts lower cortex activity. 25. **Bronstein, M. et al. (2021)**. "Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges." *PMLR*. arxiv:2104.13478. The unified framework of symmetries in deep learning — a must-read for any architecture researcher. 26. **LeCun, Y. (2022)**. "A Path Towards Autonomous Machine Intelligence." *OpenReview*. LeCun's vision of world models, JEPA, and autonomous intelligence — represents a fundamentally different approach from generative modeling. 27. **Schölkopf, B. et al. (2021)**. "Toward Causal Representation Learning." *IEEE Proceedings*. arxiv:2102.11107. The manifesto for causal ML, arguing that statistical learning alone is insufficient for robust intelligence. 28. **Whittington, J.C.R. & Bogacz, R. (2019)**. "Theories of Error Back-Propagation in the Brain." *Trends in Cognitive Sciences*. arxiv:1905.04084. Comprehensive review of biologically plausible credit assignment, connecting backprop to predictive coding. --- ## Key Papers Reference

Fig. 3 — Scaled Dot-Product Attention: Q (query), K (key), V (value) projections. Score = softmax(QKᵀ/√dₖ) · V.

- **DDPM:** ["Denoising Diffusion Probabilistic Models" (Ho et al., 2020)](https://arxiv.org/abs/2006.11239) - **InstructGPT/RLHF:** ["Training language models to follow instructions with human feedback" (Ouyang et al., 2022)](https://arxiv.org/abs/2203.02155) - **DPO:** ["Direct Preference Optimization: Your Language Model is Secretly a Reward Model" (Rafailov et al., 2023)](https://arxiv.org/abs/2305.18290) --- > **Final Note**: This document represents a synthesis of the frontier — the theories, results, and open problems across all levels of modern AI/ML research. It is by design incomplete and evolving. The most exciting papers in your personal research journey will be the ones that *contradict*, *extend*, or *connect* the frameworks described here. Use it as a map, not a destination.